Slow pagination over tons of records in mongodb

by

One approach to this problem, if you have large quantities of documents and you are displaying them in sorted order (I’m not sure how useful skip is if you’re not) would be to use the key you’re sorting on to select the next page of results.

So if you start with 

db.myCollection.find().limit(100).sort({created_date:true});

and then extract the created date of the last document returned by the cursor into a variable max_created_date_from_last_result, you can get the next page with the far more efficient (presuming you have an index on created_date) query

db.myCollection.find({created_date : { $gt : max_created_date_from_last_result } }).limit(100).sort({created_date:true});

More Related Contents:

  1. Find document with array that contains a specific value
  2. Update field in exact element array in MongoDB
  3. How to change the type of a field?
  4. Mongodb sort inner array
  5. Return only matched sub-document elements within a nested array
  6. Auto increment in MongoDB to store sequence of Unique User ID
  7. Querying an array of arrays in MongoDB
  8. Is there any option to limit mongodb memory usage?
  9. Unhandled promise rejection: Error: URL malformed, cannot be parsed
  10. mongo group query how to keep fields
  11. Auto compact the deleted space in mongodb?
  12. $unwind empty array
  13. How to return Mongoose results from the find method?
  14. How do I start Mongo DB from Windows?
  15. Find all duplicate documents in a MongoDB collection by a key field
  16. MongoDB $gt/$lt operators with prices stored as strings
  17. Is GridFS fast and reliable enough for production?
  18. Batch insert/update using Mongoid?
  19. Using stored JavaScript functions in the Aggregation pipeline, MapReduce or runCommand
  20. Build a reactive publication with additional fields in each document
  21. Mongodb query specific month|year not date
  22. How to flatten a subdocument into root level in MongoDB?
  23. Remove Duplicates from MongoDB
  24. Push to two separate arrays in one update call in mongodb
  25. How to toggle a boolean field in one document with atomic operation?
  26. How to delete documents by query efficiently in mongo?
  27. Sum in nested document MongoDB
  28. Update embedded object inside array inside array in MongoDB
  29. MongoDB Single Document size limit is 16MB
  30. mongodb aggregate query isn’t returning proper sum on using $sum

Leave a Comment