Python requests speed up using keep-alive

by

Yes, there is. Use requests.Session and it will do keep-alive by default.

I guess I should include a quick example:

import logging
import requests

logging.basicConfig(level=logging.DEBUG)
s = requests.Session()
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
s.get('http://httpbin.org/cookies/set/anothercookie/123456789')
r = s.get("http://httpbin.org/cookies")
print(r.text)

You will note that these log message occur

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies/set/sessioncookie/123456789 HTTP/1.1" 302 223
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 55
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies/set/anothercookie/123456789 HTTP/1.1" 302 223
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90

If you wait a little while, and repeat the last get call

INFO:requests.packages.urllib3.connectionpool:Resetting dropped connection: httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90

Note that it resets the dropped connection, i.e. reestablishing the connection to the server to make the new request.

More Related Contents:

  1. Python requests – print entire http request (raw)?
  2. Python Requests library redirect new url
  3. Python-Requests close http connection
  4. Python ‘requests’ library – define specific DNS?
  5. Only download a part of the document using python requests
  6. How to get response SSL certificate from requests in python?
  7. python how to decode http response
  8. How to implement retry mechanism into python requests library?
  9. How to get the raw content of a response in requests with Python?
  10. How to limit download rate of HTTP requests in requests python library?
  11. Download large file in python with requests
  12. How to download image using requests
  13. How to use Python requests to fake a browser visit a.k.a and generate User Agent?
  14. Using headers with the Python requests library’s get method
  15. Web scraping program cannot find element which I can see in the browser
  16. How to load all entries in an infinite scroll at once to parse the HTML in python
  17. How do I get the IP address from a http request using the requests library?
  18. How to encode UTF8 filename for HTTP headers? (Python, Django)
  19. Python urllib2 Progress Hook
  20. In Python, how do I decode GZIP encoding?
  21. How do I send a POST request as a JSON?
  22. Python Requests vs PyCurl Performance
  23. Using Flask, how do I modify the Cache-Control header for ALL output?
  24. Python: How do I get key/value pairs from the BaseHTTPRequestHandler HTTP POST handler?
  25. Python requests with multithreading
  26. Python: FastAPI error 422 with POST request when sending JSON data
  27. Passing cookies while logging in
  28. How to get around python requests SSL and proxy error?
  29. Http Redirection code 3XX in python requests
  30. Python 3: using requests does not get the full content of a web page

Leave a Comment