How to replace Django’s primary key with a different integer that is unique for that table

by

The Idea

I would recommend to you the same approach that is used by Instagram. Their requirements seems to closely follow yours.

Generated IDs should be sortable by time (so a list of photo IDs, for
example, could be sorted without fetching more information about the
photos) IDs should ideally be 64 bits (for smaller indexes, and better
storage in systems like Redis) The system should introduce as few new
‘moving parts’ as possible—a large part of how we’ve been able to
scale Instagram with very few engineers is by choosing simple,
easy-to-understand solutions that we trust.

They came up with a system that has 41 bits based on the timestamp, 13 o the database shard and 10 for an auto increment portion. Sincce you don’t appear to be using shards. You can just have 41 bits for a time based copmonent and 23 bits chosen at random. That does produce an extremely unlikely 1 in 8.3 million chance of getting a conflict if you insert records at the same time. But in practice you are never likely to hit this. Right so how about some code:

Generating IDs

START_TIME = a constant that represents a unix timestamp

def make_id():
    '''
    inspired by http://instagram-engineering.tumblr.com/post/10853187575/sharding-ids-at-instagram
        '''
    
    t = int(time.time()*1000) - START_TIME
    u = random.SystemRandom().getrandbits(23)
    id = (t << 23 ) | u
    
    return id


def reverse_id(id):
    t  = id >> 23
    return t + START_TIME

Note, START_TIME in the above code is some arbitary starting time. You can use time.time()*1000 , get the value and set that as START_TIME

Notice that the reverse_id method I have posted allows you to find out at which time the record was created. If you need to keep track of that information you can do so without having to add another field for it! So your primary key is actually saving your storage rather than increasing it!

The Model

Now this is what your model would look like.

class MyClass(models.Model):
   id = models.BigIntegerField(default = fields.make_id, primary_key=True)

If you make changes to your database outside django you would need to create the equivalent of make_id as an sql function

As a foot note. This is somewhat like the approach used by Mongodb to generate it’s _ID for each object.

More Related Contents:

  1. Mac OS X – EnvironmentError: mysql_config not found
  2. Setting Django up to use MySQL
  3. MySQL “incorrect string value” error when save unicode string in Django
  4. Install mysql-python (Windows)
  5. How to use MySQLdb with Python and Django in OSX 10.6?
  6. Django : Table doesn’t exist
  7. Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock
  8. Error loading MySQLdb Module ‘Did you install mysqlclient or MySQL-python?’
  9. How to run Django’s test database only in memory?
  10. How to filter (or replace) unicode characters that would take more than 3 bytes in UTF-8?
  11. MySQL Improperly Configured Reason: unsafe use of relative path
  12. Python Pandas to_sql, how to create a table with a primary key?
  13. Using Django database layer outside of Django?
  14. How to move a model between two Django apps (Django 1.7)
  15. Can one use the Django database layer outside of Django?
  16. InterfaceError (0, ”)
  17. How do i cast char to integer while querying in django ORM?
  18. How to make an Inner Join in django?
  19. Simple Subquery with OuterRef
  20. Using .aggregate() on a value introduced using .extra(select={…}) in a Django Query?
  21. django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
  22. How can I set two primary key fields for my models in Django?
  23. Install mysqlclient for Django Python on Mac OS X Sierra
  24. Django + MySQL on Mac OS 10.6.2 Snow Leopard
  25. Single Django model, multiple tables?
  26. What is PyMySQL and how does it differ from MySQLdb? Can it affect Django deployment?
  27. Django Unique Together (with foreign keys)
  28. What is the best approach to change primary keys in an existing Django app?
  29. How to expire Django session in 5minutes?
  30. How to solve “Page not found (404)” error in Django?

Leave a Comment