node.js http.get hangs after 5 requests to remote site

by

Here’s the reason of the “exactly 5”: https://nodejs.org/docs/v0.10.36/api/http.html#http_agent_maxsockets

Internally, the http module uses an agent class to manage HTTP requests. That agent will, by default, allow for a maximum of 5 open connections to the same HTTP server.

In your code, you don’t consume the actual response sent by Google. So the agent assumes that you’re not done with the request, and will keep the connection open. And so after 5 requests, the agent won’t allow you to create a new connection anymore and will start waiting for any of the existing connections to complete.

The obvious solution would be to just consume the data:

http.get("http://www.google.com", function(r){
  r.on('data', function() { /* do nothing */ });
  ...
});

If you run into the problem that your /api/internetcheck route is called a lot, so you need to allow for more than 5 concurrent connections, you can either up the connection pool size, or just disable the agent completely (although you would still need to consume the data in both cases);

// increase pool size
http.globalAgent.maxSockets = 100;

// disable agent
http.get({ hostname : 'www.google.com', path : "https://stackoverflow.com/", agent : false }, ...)

Or perhaps use a HEAD request instead of GET.

(PS: in case http.get generates an error, you should still end the HTTP response by using res.end() or something like that).

NOTE: in Node.js versions >= 0.11, maxSockets is set to Infinity.

More Related Contents:

  1. Basic HTTP authentication with Node and Express 4
  2. Error: No default engine was specified and no extension was provided
  3. Nodejs HTTP and HTTPS over same port
  4. Why POST redirects to GET and PUT redirects to PUT?
  5. File uploading with Express 4.0: req.files undefined
  6. Express.js: how to get remote client address
  7. How to get the full URL in Express?
  8. Express-js can’t GET my static files, why?
  9. Node.js / Express.js – How does app.router work?
  10. How to enable cross-origin resource sharing (CORS) in the express.js framework on node.js
  11. Setting the port for node.js server on Heroku
  12. How do I shutdown a Node.js http(s) server immediately?
  13. Cannot POST / error using express
  14. Cannot enqueue Handshake after invoking quit
  15. ExpressJS : How to redirect a POST request with parameters
  16. Node.js (with express & bodyParser): unable to obtain form-data from post request
  17. Print raw html strings on EJS
  18. MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017]
  19. Why can I execute code after “res.send”?
  20. Uploading images using Node.js, Express, and Mongoose
  21. POST request not allowed – 405 Not Allowed – nginx, even with headers included
  22. Use a variable in a Jade include
  23. How to send a pdf file from Node/Express app to the browser
  24. Error: Failed to lookup view in Express
  25. How to set the HTTP Keep-Alive timeout in a nodejs server
  26. Virtual hosting with standalone node.js server
  27. Is there any way to use multiple view engines with Express + Node.js
  28. Displaying an image with EJS in node.js/express
  29. Get route definition in middleware
  30. node.js, Error: Cannot find module ‘express’

Leave a Comment