Looks like name resolution is ultimately handled by socket.create_connection. -> urllib2.urlopen -> httplib.HTTPConnection -> socket.create_connection Though once the “Host:” header has been set, you can resolve the host and pass on the IP address through down to the opener. I’d suggest that you subclass httplib.HTTPConnection, and wrap the connect method to modify self.host before passing … Read more
EDIT: I’m expanding the answer to include a more polished example. I have found a lot hostility and misinformation in this post regarding threading v.s. async I/O. Therefore I also adding more argument to refute certain invalid claim. I hope this will help people to choose the right tool for the right job. This is … Read more
You can use HEAD request instead of GET. It will only download the header, but not the content. Then you can check the response status from the headers. For python 2.7.x, you can use httplib: import httplib c = httplib.HTTPConnection(‘www.example.com’) c.request(“HEAD”, ”) if c.getresponse().status == 200: print(‘web site exists’) or urllib2: import urllib2 try: urllib2.urlopen(‘http://www.example.com/some_page’) … Read more
Python’s wonderful standard library to the rescue… import codecs reader = codecs.getreader(“utf-8”) obj = json.load(reader(response)) Works with both py2 and py3. Docs: Python 2, Python3
Strictly speaking URIs can’t contain non-ASCII characters; what you have there is an IRI. To convert an IRI to a plain ASCII URI: non-ASCII characters in the hostname part of the address have to be encoded using the Punycode-based IDNA algorithm; non-ASCII characters in the path, and most of the other parts of the address … Read more