Multi-threaded use of SQLAlchemy

by

Session objects are not thread-safe, but are thread-local. From the docs:

“The Session object is entirely designed to be used in a non-concurrent fashion, which in terms of multithreading means “only in one thread at a time” .. some process needs to be in place such that mutltiple calls across many threads don’t actually get a handle to the same session. We call this notion thread local storage.”

If you don’t want to do the work of managing threads and sessions yourself, SQLAlchemy has the ScopedSession object to take care of this for you:

The ScopedSession object by default uses threading.local() as storage, so that a single Session is maintained for all who call upon the ScopedSession registry, but only within the scope of a single thread. Callers who call upon the registry in a different thread get a Session instance that is local to that other thread.

Using this technique, the ScopedSession provides a quick and relatively simple way of providing a single, global object in an application that is safe to be called upon from multiple threads.

See the examples in Contextual/Thread-local Sessions for setting up your own thread-safe sessions:

# set up a scoped_session
from sqlalchemy.orm import scoped_session
from sqlalchemy.orm import sessionmaker

session_factory = sessionmaker(bind=some_engine)
Session = scoped_session(session_factory)

# now all calls to Session() will create a thread-local session
some_session = Session()

# you can now use some_session to run multiple queries, etc.
# remember to close it when you're finished!
Session.remove()

More Related Contents:

  1. How to run functions in parallel?
  2. threading.Timer – repeat function every ‘n’ seconds
  3. Freezing/Hanging tkinter GUI in waiting for the thread to complete
  4. Threading in a PyQt application: Use Qt threads or Python threads?
  5. What is the use of join() in Python threading?
  6. SQLAlchemy IN clause
  7. Flask SQLAlchemy query, specify column names
  8. How can I bind a list to a parameter in a custom query in SQLAlchemy?
  9. How to start and stop a thread
  10. A very simple multithreading parallel URL fetching (without queue)
  11. How to disable SQLAlchemy caching?
  12. How to create a new database using SQLAlchemy?
  13. threading ignores KeyboardInterrupt exception
  14. Overriding python threading.Thread.run()
  15. What kind of problems (if any) would there be combining asyncio with multiprocessing?
  16. Python Threading String Arguments
  17. Terminate multiple threads when any thread completes a task
  18. Change the connection pool size for Python’s “requests” module when in Threading
  19. IntegrityError when inserting data in an association table using SQLAlchemy
  20. AttributeError: ‘int’ object has no attribute ‘_sa_instance_state’
  21. Dynamic Class Creation in SQLAlchemy
  22. SQLAlchemy equivalent to SQL “LIKE” statement
  23. How do I connect to SQL Server via sqlalchemy using Windows Authentication?
  24. Best way to do enum in Sqlalchemy?
  25. How do I INSERT INTO t1 (SELECT * FROM t2) in SQLAlchemy?
  26. Creating partial unique index with sqlalchemy on Postgres
  27. SQLAlchemy: get Model from table name. This may imply appending some function to a metaclass constructor as far as I can see
  28. Python Global Interpreter Lock (GIL) workaround on multi-core systems using taskset on Linux?
  29. How to close a thread from within?
  30. Embedding python in multithreaded C application

Leave a Comment