MongoDB aggregation comparison: group(), $group and MapReduce

by

It is somewhat confusing since the names are similar, but the group() command is a different feature and implementation from the $group pipeline operator in the Aggregation Framework.

The group() command, Aggregation Framework, and MapReduce are collectively aggregation features of MongoDB. There is some overlap in features, but I’ll attempt to explain the differences and limitations of each as at MongoDB 2.2.0.

Note: inline result sets mentioned below refer to queries that are processed in memory with results returned at the end of the function call. Alternative output options (currently only available with MapReduce) could include saving results to a new or existing collection.

group() Command

  • Simple syntax and functionality for grouping .. analogous to GROUP BY in SQL.

  • Returns result set inline (as an array of grouped items).

  • Implemented using the JavaScript engine; custom reduce() functions can be written in JavaScript.

  • Current Limitations

    • Will not group into a result set with more than 20,000 keys.

    • Results must fit within the limitations of a BSON document (currently 16MB).

    • Takes a read lock and does not allow any other threads to execute JavaScript while it is running.

    • Does not work with sharded collections.

  • See also: group() command examples.

MapReduce

  • Implements the MapReduce model for processing large data sets.

  • Can choose from one of several output options (inline, new collection, merge, replace, reduce)

  • MapReduce functions are written in JavaScript.

  • Supports non-sharded and sharded input collections.

  • Can be used for incremental aggregation over large collections.

  • MongoDB 2.2 implements much better support for sharded map reduce output.

  • Current Limitations

    • A single emit can only hold half of MongoDB’s maximum BSON document size (16MB).

    • There is a JavaScript lock so a mongod server can only execute one JavaScript function at a point in time .. however, most steps of the MapReduce are very short so locks can be yielded frequently.

    • MapReduce functions can be difficult to debug. You can use print() and printjson() to include diagnostic output in the mongod log.

    • MapReduce is generally not intuitive for programmers trying to translate relational query aggregation experience.

  • See also: Map/Reduce examples.

Aggregation Framework

  • New feature in the MongoDB 2.2.0 production release (August, 2012).

  • Designed with specific goals of improving performance and usability.

  • Returns result set inline.

  • Supports non-sharded and sharded input collections.

  • Uses a “pipeline” approach where objects are transformed as they pass through a series of pipeline operators such as matching, projecting, sorting, and grouping.

  • Pipeline operators need not produce one output document for every input document: operators may also generate new documents or filter out documents.

  • Using projections you can add computed fields, create new virtual sub-objects, and extract sub-fields into the top-level of results.

  • Pipeline operators can be repeated as needed (for example, multiple $project or $group steps.

  • Current Limitations

    • Results are returned inline, so are limited to the maximum document size supported by the server (16MB)

    • Doesn’t support as many output options as MapReduce

    • Limited to operators and expressions supported by the Aggregation Framework (i.e. can’t write custom functions)

    • Newest server feature for aggregation, so has more room to mature in terms of documentation, feature set, and usage.

  • See also: Aggregation Framework examples.

Can someone present an illustration or guide me to a link where these three concepts are explained together, taking the same sample data, so I can compare them easily?

You generally won’t find examples where it would be useful to compare all three approaches, but here are previous StackOverflow questions which show variations:

More Related Contents:

  1. Does MongoDB’s $in clause guarantee order
  2. Check if every element in array matches condition
  3. Remove Duplicates from MongoDB
  4. Retrieve only the queried element in an object array in MongoDB collection
  5. Update MongoDB field using value of another field
  6. MongoDB: Combine data from multiple collections into one..how?
  7. Multiple join conditions using the $lookup operator
  8. Conditional $sum in MongoDB
  9. Return only matched sub-document elements within a nested array
  10. Converting string to date in mongodb
  11. MongoDB Full and Partial Text Search
  12. Mongodb Join on _id field from String to ObjectId
  13. exclude fields in $lookup aggregation
  14. Add some kind of row number to a mongodb aggregate command / pipeline
  15. Multiple Counts with single query in mongodb
  16. Mongo sort on a calculated condition
  17. Find all duplicate documents in a MongoDB collection by a key field
  18. MongoDB sort documents by array elements
  19. How to return array of string with mongodb aggregation
  20. MongoDB: Calculate dwell time between every status value change
  21. MongoDB: how to parse date in 3.6 mongoDb version?
  22. Using stored JavaScript functions in the Aggregation pipeline, MapReduce or runCommand
  23. MongoDB – The argument to $size must be an Array, but was of type: EOO / missing
  24. convert iso date to timestamp in mongo query
  25. Using $slice operator to get last element of array
  26. How to access overall document count during arithmetic aggregation expression
  27. Mongodb aggregation pipeline how to limit a group push
  28. Mongodb, aggregate query with $lookup
  29. mongodb aggregate query isn’t returning proper sum on using $sum
  30. Return the last “true” value for each group

Leave a Comment