multiprocessing.pool.MaybeEncodingError: ‘TypeError(“cannot serialize ‘_io.BufferedReader’ object”,)’

The http.client.HTTPResponse-object you get back from urlopen() has a _io.BufferedReader-object attached, and this object cannot be pickled. pickle.dumps(urllib.request.urlopen(‘http://www.python.org’).fp) Traceback (most recent call last): … pickle.dumps(urllib.request.urlopen(‘http://www.python.org’).fp) TypeError: cannot serialize ‘_io.BufferedReader’ object multiprocessing.Pool will need to pickle (serialize) the results to send it back to the parent process and this fails here. Since dummy uses threads instead … Read more

TypeError in Threading. function takes x positional argument but y were given

The args kwarg of threading.Thread expects an iterable, and each element in that iterable is being passed to the target function. Since you are providing a string for args: t = threading.Thread(target=startSuggestworker, args=(start_keyword)) each character is being passed as a separate argument to startSuggestworker. Instead, you should provide args a tuple: t = threading.Thread(target=startSuggestworker, args=(start_keyword,)) … Read more

Cancellable threading.Timer in Python

You would call the cancel method after you start the timer: import time import threading def hello(): print “hello, world” time.sleep(2) t = threading.Timer(3.0, hello) t.start() var=”something” if var == ‘something’: t.cancel() You might consider using a while-loop on a Thread, instead of using a Timer. Here is an example appropriated from Nikolaus Gradwohl’s answer … Read more

Python – appending to same file from multiple threads

The solution is to write to the file in one thread only. import Queue # or queue in Python 3 import threading class PrintThread(threading.Thread): def __init__(self, queue): threading.Thread.__init__(self) self.queue = queue def printfiles(self, p): for path, dirs, files in os.walk(p): for f in files: print(f, file=output) def run(self): while True: result = self.queue.get() self.printfiles(result) self.queue.task_done() … Read more

How to Multi-thread an Operation Within a Loop in Python

First, in Python, if your code is CPU-bound, multithreading won’t help, because only one thread can hold the Global Interpreter Lock, and therefore run Python code, at a time. So, you need to use processes, not threads. This is not true if your operation “takes forever to return” because it’s IO-bound—that is, waiting on the … Read more