How do I deal with this race condition in django?

by

This could be an off-shoot of the same problem as here:

Why doesn’t this loop display an updated object count every five seconds?

Basically get_or_create can fail – if you take a look at its source, there you’ll see that it’s: get, if-problem: save+some_trickery, if-still-problem: get again, if-still-problem: surrender and raise.

This means that if there are two simultaneous threads (or processes) running create_or_update_myobj, both trying to get_or_create the same object, then:

  • first thread tries to get it – but it doesn’t yet exist,
  • so, the thread tries to create it, but before the object is created…
  • …second thread tries to get it – and this obviously fails
  • now, because of the default AUTOCOMMIT=OFF for MySQLdb database connection, and REPEATABLE READ serializable level, both threads have frozen their views of MyObj table.
  • subsequently, first thread creates its object and returns it gracefully, but…
  • …second thread cannot create anything as it would violate unique constraint
  • what’s funny, subsequent get on the second thread doesn’t see the object created in the first thread, due to the frozen view of MyObj table

So, if you want to safely get_or_create anything, try something like this:

 @transaction.commit_on_success
 def my_get_or_create(...):
     try:
         obj = MyObj.objects.create(...)
     except IntegrityError:
         transaction.commit()
         obj = MyObj.objects.get(...)
     return obj

Edited on 27/05/2010

There is also a second solution to the problem – using READ COMMITED isolation level, instead of REPEATABLE READ. But it’s less tested (at least in MySQL), so there might be more bugs/problems with it – but at least it allows tying views to transactions, without committing in the middle.

Edited on 22/01/2012

Here are some good blog posts (not mine) about MySQL and Django, related to this question:

http://www.no-ack.org/2010/07/mysql-transactions-and-django.html

http://www.no-ack.org/2011/05/broken-transaction-management-in-mysql.html

More Related Contents:

  1. When saving, how can you check if a field has changed?
  2. RuntimeWarning: DateTimeField received a naive datetime
  3. Rendering a template variable as HTML
  4. How to express a One-To-Many relationship in Django?
  5. How to get the current URL within a Django template?
  6. Only accept a certain file type in FileField, server-side
  7. Is it possible to pass query parameters via Django’s {% url %} template tag?
  8. Django: Set foreign key using integer?
  9. How can I have two foreign keys to the same model in Django?
  10. Putting a django login form on every page
  11. _set in Django for a queryset
  12. Django: How to get current user in admin forms?
  13. Docker app server ip address 127.0.0.1 difference of 0.0.0.0 ip
  14. Accessing “Media” files in Django
  15. How to add data into ManyToMany field?
  16. Raw SQL queries in Django views
  17. ImportError: No module named django.core.wsgi Apache + VirtualEnv + AWS + WSGI
  18. django-orm case-insensitive order by
  19. Django forms request.user
  20. Fighting client-side caching in Django
  21. How do you filter a nested serializer in Django Rest Framework?
  22. Django The ‘image’ attribute has no file associated with it
  23. Django query filter with variable column
  24. Migrating existing auth.User data to new Django 1.5 custom user model?
  25. git push Heroku main – error: src refspec main does not match any
  26. Setting default value for Foreign Key attribute
  27. Django REST Framework image upload
  28. Django: dynamic database file
  29. User groups and permissions
  30. Get type of Django form widget from within template

Leave a Comment