How to limit download rate of HTTP requests in requests python library?

by

There are several approaches to rate limiting; one of them is token bucket, for which you can find a recipe here and another one here.

Usually you would want to do throttling or rate limiting on socket.send() and socket.recv(). You could play with socket-throttle and see if it does what you need.

This is not to be confused with x-ratelimit rate limiting response headers, which are related to a number of requests rather than a download / transfer rate.

More Related Contents:

  1. Asynchronous Requests with Python requests
  2. How to use Python requests to fake a browser visit a.k.a and generate User Agent?
  3. Python requests – print entire http request (raw)?
  4. Python Requests library redirect new url
  5. Python Requests requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
  6. Python Request Post with param data
  7. Python-Requests close http connection
  8. Python ‘requests’ library – define specific DNS?
  9. Putting a `Cookie` in a `CookieJar`
  10. Python Requests package: Handling xml response
  11. Only download a part of the document using python requests
  12. How to get response SSL certificate from requests in python?
  13. python how to decode http response
  14. How to implement retry mechanism into python requests library?
  15. Python requests speed up using keep-alive
  16. Checking if a website is up via Python
  17. How to get the raw content of a response in requests with Python?
  18. Ideal method for sending multiple HTTP requests over Python? [duplicate]
  19. How to “log in” to a website using Python’s Requests module?
  20. MaxRetryError: HTTPConnectionPool: Max retries exceeded (Caused by ProtocolError(‘Connection aborted.’, error(111, ‘Connection refused’)))
  21. How to use cookies in Python Requests
  22. using requests with TLS doesn’t give SNI support
  23. Unable to get local issuer certificate when using requests in python
  24. How to prevent python requests from percent encoding my URLs?
  25. Python urllib2.urlopen() is slow, need a better way to read several urls
  26. Python Requests getting SSLerror
  27. How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?
  28. How to share variables between HTTP requests in FastAPI?
  29. python requests get cookies
  30. How to send JSON as part of multipart POST-request

Leave a Comment