How to send 1000 XHTTP Requests with maximum number of parallel requests

by

You could try to use RxJS bufferCount and concatMap operators along with forkJoin().

From bufferCount docs:

Collect emitted values until provided number is fulfilled, emit as
array.

So it collects n number of notifications and emit it as an array. We could then pass the array through forkJoin() for n number of parallel requests.

Try the following

I assume this.urls is a collection of HTTP requests similar to

urls = [
  this.http.get('url1'),
  this.http.get('url2'),
  this.http.get('url3'),
  ...
];

Then the requests triggering code would look like

bufferedRequests() {
  from(this.urls).pipe(
    bufferCount(6),      // <-- adjust number of parallel requests here
    concatMap(buffer => forkJoin(buffer))
  ).subscribe(
    res => console.log(res),
    err => console.log(err),
    () => console.log('complete')
  );
}

According to this comment by a Chromium engineer, actual solution to the max. connections to host/domain limit would be to use WebSockets or Domain sharding. But since you mention it isn’t possible in your environment, you could use the buffered request workaround.

However I wouldn’t try to buffer to the max limit. If you were to send more requests to the same domain than the max allowed, you could see that the additional requests would actually throttle behind till the above requests are finished. So say if were to buffer to the max allowed limit, and your application sends an additional request to the same domain from somewhere else on which the app workflow depends on, the entire app could throttle.

So it’s better to either use WebSockets or Domain sharding. And if neither is possible it’s better to buffer the requests to a number of requests less than* the max allowed limit.

* Obviously, if you’re 100% percent sure no other requests will be triggered to the same domain during the buffering procedure, then you could buffer to max. allowed limit.

More Related Contents:

  1. What is the correct way to share the result of an Angular Http network call in RxJs 5?
  2. What is the difference between Promises and Observables?
  3. What is the difference between Subject and BehaviorSubject?
  4. Angular HTTP GET with TypeScript error http.get(…).map is not a function in [null]
  5. Convert Promise to Observable
  6. Observable type error: cannot read property of undefined
  7. RxJS: takeUntil() Angular component’s ngOnDestroy()
  8. Angular Subscribe within Subscribe
  9. Passing Data with Subjects and Proxies
  10. How to create an RXjs RetryWhen with delay and limit on tries
  11. Handling refresh tokens using rxjs
  12. Best way to “flatten” an array inside an RxJS Observable
  13. How to cancel/unsubscribe all pending HTTP requests in Angular 4+
  14. How to make an http call every 2 minutes with RXJS?
  15. Angular 7 – nesting Observables
  16. Angular subscribe elegant
  17. Making sure observables in for loop are all finished before executing other code
  18. Best way to import Observable from rxjs
  19. Behaviour subject initial value null?
  20. Observable.of is not a function
  21. rxjs/Subject.d.ts error : Class ‘Subject’ incorrectly extends base class ‘Observable’
  22. Angular 2 Observable with multiple subscribers
  23. How to make nested Observable calls in Angular2
  24. How to correctly import operators from the `rxjs` package
  25. What is pipe for in RxJS?
  26. How to throw error from RxJS map operator (angular)
  27. How to return value inside subscribe Angular 4
  28. Angular 5 to 6 Upgrade: Property ‘map’ does not exist on type Observable
  29. Do I need to `complete()` takeUntil Subject inside ngOnDestroy?
  30. Angular2 watch for 302 redirect when fetching resource

Leave a Comment