Python urllib over TOR? [duplicate]

by

The problem is that httplib.HTTPConnection uses the socket module’s create_connection helper function which does the DNS request via the usual getaddrinfo method before connecting the socket.

The solution is to make your own create_connection function and monkey-patch it into the socket module before importing urllib2, just like we do with the socket class.

import socks
import socket
def create_connection(address, timeout=None, source_address=None):
    sock = socks.socksocket()
    sock.connect(address)
    return sock

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)

# patch the socket module
socket.socket = socks.socksocket
socket.create_connection = create_connection

import urllib2

# Now you can go ahead and scrape those shady darknet .onion sites

More Related Contents:

  1. How can I use a SOCKS 4/5 proxy with urllib2?
  2. Using urllib2 with SOCKS proxy
  3. Proxy with urllib2
  4. Using an HTTP PROXY – Python [duplicate]
  5. Python: download files from google drive using url
  6. Python handling socket.error: [Errno 104] Connection reset by peer
  7. Using MultipartPostHandler to POST form-data with Python
  8. How to make urllib2 requests through Tor in Python?
  9. Python-Requests close http connection
  10. python ignore certificate validation urllib2
  11. How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? [duplicate]
  12. Extracting text from script tag using BeautifulSoup in Python
  13. Fetch a Wikipedia article with Python
  14. Making a POST call instead of GET using urllib2
  15. How to change Tor identity in Python?
  16. How can I speed up fetching pages with urllib2 in python?
  17. How to download any(!) webpage with correct charset in python?
  18. Unescape Python Strings From HTTP
  19. Source interface with Python and urllib2
  20. Python urllib2: Receive JSON response from url
  21. Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str
  22. open url from pythonanywhere
  23. Python POST binary data
  24. Submitting to a web form using python
  25. urllib2 HTTP Error 400: Bad Request
  26. Download and decompress gzipped file in memory?
  27. How to save “complete webpage” not just basic html using Python
  28. how to follow meta refreshes in Python
  29. How to resolve URLError:
  30. urllib2 and json

Leave a Comment