What is the best way to limit concurrency when using ES6’s Promise.all()?

by

P-Limit

I have compared promise concurrency limitation with a custom script, bluebird, es6-promise-pool, and p-limit. I believe that p-limit has the most simple, stripped down implementation for this need. See their documentation.

Requirements

To be compatible with async in example

My Example

In this example, we need to run a function for every URL in the array (like, maybe an API request). Here this is called fetchData(). If we had an array of thousands of items to process, concurrency would definitely be useful to save on CPU and memory resources. 

const pLimit = require('p-limit');

// Example Concurrency of 3 promise at once
const limit = pLimit(3);

let urls = [
    "http://www.exampleone.com/",
    "http://www.exampletwo.com/",
    "http://www.examplethree.com/",
    "http://www.examplefour.com/",
]

// Create an array of our promises using map (fetchData() returns a promise)
let promises = urls.map(url => {

    // wrap the function we are calling in the limit function we defined above
    return limit(() => fetchData(url));
});

(async () => {
    // Only three promises are run at once (as defined above)
    const result = await Promise.all(promises);
    console.log(result);
})();

The console log result is an array of your resolved promises response data.

More Related Contents:

  1. Is Node.js native Promise.all processing in parallel or sequentially?
  2. Should I refrain from handling Promise rejection asynchronously?
  3. Handling errors in express async middleware
  4. Node.js: When to use Promises vs Callbacks
  5. using await on global scope without async keyword
  6. Weird behavior with Promise throwing “Unhandled promise rejection” error
  7. Promise.resolve().then vs setImmediate vs nextTick
  8. JavaScript binding of global result is undefined [closed]
  9. map function for objects (instead of arrays)
  10. Node JS Promise.all and forEach
  11. How to use preload.js properly in Electron
  12. Why is let slower than var in a for loop in nodejs?
  13. Understanding async/await on NodeJS
  14. How do I handle errors with promises?
  15. Appending .js extension on relative import statements during Typescript compilation (ES6 modules)
  16. node.js displays “undefined” on the console
  17. Why doesn’t instanceof work on instances of Error subclasses under babel-node?
  18. Angular 2 – How to pass URL parameters?
  19. then() callback firing before promise is resolved in node.js [duplicate]
  20. Scrape web pages in real time with Node.js
  21. Javascript console.log(object) vs. concatenating string
  22. Webpack Express Cannot Resolve Module ‘fs’, Request Dependency is Expression
  23. Encrypt string in PHP and decrypt in Node.js
  24. Mongoose: Find, modify, save
  25. Iterate over object keys in node.js
  26. configure node express to serve static bower_components?
  27. Accessing line number in V8 JavaScript (Chrome & Node.js)
  28. Nodejs: how to clone an object
  29. node.js, express, how to get data from body form-data in post request
  30. How to use npm modules in browser? is possible to use them even in local (PC)?

Leave a Comment