How to get a faster speed when using multi-threading in python

by

The biggest thing you are doing wrong, that is hurting your throughput the most, is the way you are calling thread.start() and thread.join():

for i in range(0, 10):
   thread = threading.Thread(target = current_post.post)
   thread.start()
   thread.join()

Each time through the loop, you create a thread, start it, and then wait for it to finish Before moving on to the next thread. You aren’t doing anything concurrently at all!

What you should probably be doing instead is:

threads = []

# start all of the threads
for i in range(0, 10):
   thread = threading.Thread(target = current_post.post)
   thread.start()
   threads.append(thread)

# now wait for them all to finish
for thread in threads:
   thread.join()

More Related Contents:

  1. How to use asyncio with threading in Python is Method threading.Timer
  2. Multiprocessing vs Threading Python [duplicate]
  3. How to get POSTed JSON in Flask?
  4. Are lists thread-safe?
  5. Catch a thread’s exception in the caller thread?
  6. How are POST and GET variables handled in Python?
  7. thread starts running before calling Thread.start
  8. How to use multiprocessing queue in Python?
  9. How to make built-in containers (sets, dicts, lists) thread safe?
  10. Using win32com with multithreading
  11. Threads and tkinter
  12. Multithreaded web server in python
  13. Program freezing during the execution of a function in Tkinter
  14. How to run multiple jobs in one Sparkcontext from separate threads in PySpark?
  15. Kill a running subprocess call
  16. Python threading. How do I lock a thread?
  17. Python multithreaded print statements delayed until all threads complete execution
  18. Python Threading inside a class
  19. Meaning of daemon property on Python Threads
  20. How to use threading in Python?
  21. Running a Tkinter form in a separate thread
  22. Python creating a shared variable between threads
  23. How to start threads at the same time in Python [duplicate]
  24. Is a variable swap guaranteed to be atomic in python?
  25. Restarting a thread in Python
  26. What is “backlog” in TCP connections?
  27. Python Global Interpreter Lock (GIL) workaround on multi-core systems using taskset on Linux?
  28. waiting for user input in separate thread
  29. How to close a thread from within?
  30. Embedding python in multithreaded C application

Leave a Comment