How do I force Django to ignore any caches and reload data?

by

Having had this problem and found two definitive solutions for it I thought it worth posting another answer.

This is a problem with MySQL’s default transaction mode. Django opens a transaction at the start, which means that by default you won’t see changes made in the database.

Demonstrate like this

Run a django shell in terminal 1

>>> MyModel.objects.get(id=1).my_field
u'old'

And another in terminal 2

>>> MyModel.objects.get(id=1).my_field
u'old'
>>> a = MyModel.objects.get(id=1)
>>> a.my_field = "NEW"
>>> a.save()
>>> MyModel.objects.get(id=1).my_field
u'NEW'
>>>

Back to terminal 1 to demonstrate the problem – we still read the old value from the database.

>>> MyModel.objects.get(id=1).my_field
u'old'

Now in terminal 1 demonstrate the solution

>>> from django.db import transaction
>>> 
>>> @transaction.commit_manually
... def flush_transaction():
...     transaction.commit()
... 
>>> MyModel.objects.get(id=1).my_field
u'old'
>>> flush_transaction()
>>> MyModel.objects.get(id=1).my_field
u'NEW'
>>>

The new data is now read

Here is that code in an easy to paste block with docstring

from django.db import transaction

@transaction.commit_manually
def flush_transaction():
    """
    Flush the current transaction so we don't read stale data

    Use in long running processes to make sure fresh data is read from
    the database.  This is a problem with MySQL and the default
    transaction mode.  You can fix it by setting
    "transaction-isolation = READ-COMMITTED" in my.cnf or by calling
    this function at the appropriate moment
    """
    transaction.commit()

The alternative solution is to change my.cnf for MySQL to change the default transaction mode

transaction-isolation = READ-COMMITTED

Note that that is a relatively new feature for Mysql and has some consequences for binary logging / slaving. You could also put this in the django connection preamble if you wanted.

Update 3 years later

Now that Django 1.6 has turned on autocommit in MySQL this is no longer a problem. The example above now works fine without the flush_transaction() code whether your MySQL is in REPEATABLE-READ (the default) or READ-COMMITTED transaction isolation mode.

What was happening in previous versions of Django which ran in non autocommit mode was that the first select statement opened a transaction. Since MySQL’s default mode is REPEATABLE-READ this means that no updates to the database will be read by subsequent select statements – hence the need for the flush_transaction() code above which stops the transaction and starts a new one.

There are still reasons why you might want to use READ-COMMITTED transaction isolation though. If you were to put terminal 1 in a transaction and you wanted to see the writes from the terminal 2 you would need READ-COMMITTED.

The flush_transaction() code now produces a deprecation warning in Django 1.6 so I recommend you remove it.

More Related Contents:

  1. Expire a view-cache in Django?
  2. Pillow installed, but getting “no module named pillow”
  3. How to query as GROUP BY in django?
  4. How to override and extend basic Django admin templates?
  5. Django FileField with upload_to determined at runtime
  6. How can I build multiple submit buttons django form?
  7. Does Django scale? [closed]
  8. In Django, how does one filter a QuerySet with dynamic field lookups?
  9. Django datetime issues (default=datetime.now())
  10. How to compare dates in Django templates
  11. Django [Errno 13] Permission denied: ‘/var/www/media/animals/user_uploads’
  12. Running “unique” tasks with celery
  13. URL-parameters and logic in Django class-based views (TemplateView)
  14. Creating a list of dictionaries with same keys? [duplicate]
  15. ValueError at /image/ Tensor Tensor(“activation_5/Softmax:0”, shape=(?, 4), dtype=float32) is not an element of this graph
  16. How do I perform query filtering in django templates
  17. Django Multiple Choice Field / Checkbox Select Multiple
  18. Django REST: Uploading and serializing multiple images
  19. Django gives Bad Request (400) when DEBUG = False
  20. Django Rest Framework – Authentication credentials were not provided
  21. Get request body as string in Django
  22. Django: Calculate the Sum of the column values through query
  23. How to disable Django’s CSRF validation?
  24. “no such table” exception
  25. How to update an existing Conda environment with a .yml file
  26. What is an efficient way of inserting thousands of records into an SQLite table using Django?
  27. Proper way to consume data from RESTFUL API in django
  28. How do I convert kilometres to degrees in Geodjango/GEOS?
  29. dynamically loading django apps at runtime
  30. Collectstatic error while deploying Django app to Heroku

Leave a Comment