MongoDB Duplicate Documents even after adding unique key

Congratulations, you appear to have found a bug. This only happens with MongoDB 3.0.0 in my testing, or at least is not present at MongoDB 2.6.6. Bug now recorded at SERVER-17599

NOTE:
Not actually an “issue” but confirmed “by design”. Dropped the option for version 3.0.0. Still listed in the documentation though.

The problem is that the index is not being created and errors when you attempt to create this on a collection with existing duplicates on the “compound key” fields. On the above, the index creation should yield this in the shell:

{
    "createdCollectionAutomatically" : false,
    "numIndexesBefore" : 1,
    "errmsg" : "exception: E11000 duplicate key error dup key: { : 15.0, : 1.0 }",
    "code" : 11000,
    "ok" : 0
}

When there are no duplicates present you can create the index as you are currently trying and it will be created.

So to work around this, first remove the duplicates with a procedure like this:

db.events.aggregate([
    { "$group": {
        "_id": { "uid": "$uid", "sid": "$sid" },
        "dups": { "$push": "$_id" },
        "count": { "$sum": 1 }
    }},
    { "$match": { "count": { "$gt": 1 } }}
]).forEach(function(doc) {
    doc.dups.shift();
    db.events.remove({ "_id": {"$in": doc.dups }});
});

db.events.createIndex({"uid":1 , "sid": 1},{unique:true})

Then further inserts containing duplicate data will not be inserted and the appropriate error will be recorded.

The final note here is that “dropDups” is/was not a very elegant solution for removing duplicate data. You really want something with more control as demonstrated above.

For the second part, rather than use .insert() use the .update() method. It has an “upsert” option

$collection->update(
    array( "uid" => 1, "sid" => 1 ),
    array( '$set' => $someData ),
    array( 'upsert' => true )
);

So the “found” documents are “modified” and the documents not found are “inserted”. Also see $setOnInsert for a way to only create certain data when the document is actually inserted and not when modified.


For your specific attempt, the correct syntax of .update() is three arguments. “query”, “update” and “options”:

$collection->update(
    array( "uid" => 1, "sid" => 1 ),
    array(
        '$set' => array( "field" => "this" ),
        '$inc' => array( "counter" => 1 ),
        '$setOnInsert' => array( "newField" => "another" )
   ),
   array( "upsert" => true )
);

None of the update operations are allowed to “access the same path” as used in another update operation in that “update” document section.

Leave a Comment