This exception is raised when you reuse httplib.HTTP object for new request while you havn’t called its getresponse() method for previous request. Probably there was some other error before this one that left connection in broken state. The simplest reliable way to fix the problem is creating new connection for each request, not reusing it. … Read more
For the client side, as a built-in option you’d use urllib.request module. For an even higher-level client, try requests. It is quite intuitive and easy to use/install. For the server side, I’ll recommend you to use a small web framework like Flask, Bottle or Tornado. These ones are quite easy to use, and lightweight. For … Read more
At the end of the day, all of the other modules (feedparser, mechanize, and urllib2) call httplib which is where the exception is being thrown. Now, first things first, I also downloaded this with wget and the resulting file was 1854 bytes. Next, I tried with urllib2: >>> import urllib2 >>> url=”http://hattiesburg.legistar.com/Feed.ashx?M=Calendar&ID=543375&GUID=83d4a09c-6b40-4300-a04b-f88884048d49&Mode=2013&Title=City+of+Hattiesburg%2c+MS+-+Calendar+(2013)” >>> f = … Read more
The “can’t start new thread” error almost certainly due to the fact that you have already have too many threads running within your python process, and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you’re creating; the … Read more
httplib and urllib2 are not thread-safe. urllib2 does not provide serialized access to a global (shared) OpenerDirector object, which is used by urllib2.urlopen(). Similarly, httplib does not provide serialized access to HTTPConnection objects (i.e. by using a thread-safe connection pool), so sharing HTTPConnection objects between threads is not safe. I suggest using httplib2 or urllib3 … Read more
Python 2.x: docs.python.org/2/library/httplib.html: Note: HTTPS support is only available if the socket module was compiled with SSL support. Python 3.x: docs.python.org/3/library/http.client.html: Note HTTPS support is only available if Python was compiled with SSL support (through the ssl module). #!/usr/bin/env python import httplib c = httplib.HTTPSConnection(“ccc.de”) c.request(“GET”, “https://stackoverflow.com/”) response = c.getresponse() print response.status, response.reason data = … Read more
It turns out that it’s rather involved. Here’s a monkey-patch while using requests version 1.2.3: Wrapping the _make_request method on HTTPConnectionPool to store the response from socket.getpeername() on the HTTPResponse instance. For me on python 2.7.3, this instance was available on response.raw._original_response. from requests.packages.urllib3.connectionpool import HTTPConnectionPool def _make_request(self,conn,method,url,**kwargs): response = self._old_make_request(conn,method,url,**kwargs) sock = getattr(conn,’sock’,False) if … Read more
I think a more reliable way of closing a connection is to tell the sever explicitly to close it in a way compliant with HTTP specification: HTTP/1.1 defines the “close” connection option for the sender to signal that the connection will be closed after completion of the response. For example, Connection: close in either the … Read more
If you really want to handle with HTTP using Python, I highly recommend Requests: HTTP for Humans. The POST quickstart adapted to your question is: >>> import requests >>> r = requests.post(“http://bugs.python.org”, data={‘number’: 12524, ‘type’: ‘issue’, ‘action’: ‘show’}) >>> print(r.status_code, r.reason) 200 OK >>> print(r.text[:300] + ‘…’) <!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Transitional//EN” “http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”> … Read more