Possibility of duplicate Mongo ObjectId’s being generated in two different collections?

by

Short Answer

Just to add a direct response to your initial question: YES, if you use BSON Object ID generation, then for most drivers the IDs are almost certainly going to be unique across collections. See below for what “almost certainly” means.

Long Answer

The BSON Object ID’s generated by Mongo DB drivers are highly likely to be unique across collections. This is mainly because of the last 3 bytes of the ID, which for most drivers is generated via a static incrementing counter. That counter is collection-independent; it’s global. The Java driver, for example, uses a randomly initialized, static AtomicInteger.

So why, in the Mongo docs, do they say that the IDs are “highly likely” to be unique, instead of outright saying that they WILL be unique? Three possibilities can occur where you won’t get a unique ID (please let me know if there are more):

Before this discussion, recall that the BSON Object ID consists of:

[4 bytes seconds since epoch, 3 bytes machine hash, 2 bytes process ID, 3 bytes counter]

Here are the three possibilities, so you judge for yourself how likely it is to get a dupe:

1) Counter overflow: there are 3 bytes in the counter. If you happen to insert over 16,777,216 (2^24) documents in a single second, on the same machine, in the same process, then you may overflow the incrementing counter bytes and end up with two Object IDs that share the same time, machine, process, and counter values.

2) Counter non-incrementing: some Mongo drivers use random numbers instead of incrementing numbers for the counter bytes. In these cases, there is a 1/16,777,216 chance of generating a non-unique ID, but only if those two IDs are generated in the same second (i.e. before the time section of the ID updates to the next second), on the same machine, in the same process.

3) Machine and process hash to the same values. The machine ID and process ID values may, in some highly unlikely scenario, map to the same values for two different machines. If this occurs, and at the same time the two counters on the two different machines, during the same second, generate the same value, then you’ll end up with a duplicate ID.

These are the three scenarios to watch out for. Scenario 1 and 3 seem highly unlikely, and scenario 2 is totally avoidable if you’re using the right driver. You’ll have to check the source of the driver to know for sure.

More Related Contents:

  1. Where does mongodb stand in the CAP theorem?
  2. How do you query for “is not null” in Mongo?
  3. How to stop insertion of Duplicate documents in a mongodb collection
  4. Is mongodb running?
  5. Does MongoDB support floating point types?
  6. MongoDB normalization, foreign key and joining
  7. Get a document in MongoDB without specifying collection
  8. Update field in exact element array in MongoDB
  9. MongoDB: How to update multiple documents with a single command?
  10. How to remove a field completely from a MongoDB document?
  11. Store images in a MongoDB database
  12. Add new field to every document in a MongoDB collection
  13. Querying internal array size in MongoDB
  14. How can I run MongoDB as a Windows service?
  15. Find duplicate records in MongoDB
  16. Best practice to maintain a mgo session
  17. How do you rename a MongoDB database?
  18. How do I do a “NOT IN” query in Mongo?
  19. How to enforce foreign keys in NoSql databases (MongoDB)?
  20. NoSQL (MongoDB) vs Lucene (or Solr) as your database [closed]
  21. MongoDB update with condition
  22. MongoDB lookup when foreign field is an array of objects
  23. Mongodb: Failed to connect to 127.0.0.1:27017, reason: errno:10061
  24. MongoDB nested array query
  25. Filtering embedded documents in MongoDB
  26. Mongodb aggregation $group, restrict length of array
  27. MongoDB/NoSQL: Keeping Document Change History
  28. MongoDB Single Document size limit is 16MB
  29. Unique IDs with mongodb
  30. How to return only value of a field in mongodb

Leave a Comment