MongoDB ranged pagination

by

Good question!

“How many is too many?” – that, of course, depends on your data size and performance requirements. I, personally, feel uncomfortable when I skip more than 500-1000 records.

The actual answer depends on your requirements. Here’s what modern sites do (or, at least, some of them).

First, navbar looks like this:

1 2 3 ... 457

They get final page number from total record count and page size. Let’s jump to page 3. That will involve some skipping from the first record. When results arrive, you know id of first record on page 3.

1 2 3 4 5 ... 457

Let’s skip some more and go to page 5.

1 ... 3 4 5 6 7 ... 457

You get the idea. At each point you see first, last and current pages, and also two pages forward and backward from the current page.

Queries

var current_id; // id of first record on current page.

// go to page current+N
db.collection.find({_id: {$gte: current_id}}).
              skip(N * page_size).
              limit(page_size).
              sort({_id: 1});

// go to page current-N
// note that due to the nature of skipping back,
// this query will get you records in reverse order 
// (last records on the page being first in the resultset)
// You should reverse them in the app.
db.collection.find({_id: {$lt: current_id}}).
              skip((N-1)*page_size).
              limit(page_size).
              sort({_id: -1});

More Related Contents:

  1. Is Mongodb Aggregation framework faster than map/reduce?
  2. MongoDB ‘count()’ is very slow. How do we refine/work around with it?
  3. MongoDB’s performance on aggregation queries
  4. database access VS lan access [closed]
  5. INC instruction vs ADD 1: Does it matter?
  6. DataFrame / Dataset groupBy behaviour/optimization
  7. Excel VBA Performance – 1 million rows – Delete rows containing a value, in less than 1 min
  8. What’s the most efficient way to test if two ranges overlap?
  9. What happens after a L2 TLB miss?
  10. Are there any modern CPUs where a cached byte store is actually slower than a word store?
  11. Execute Hive Query with IN clause parameters in parallel
  12. Non-temporal loads and the hardware prefetcher, do they work together?
  13. What is the overhead of Rust’s Option type?
  14. JMeter : How to record HTTPS traffic?
  15. Which loop is faster, while or for?
  16. bsxfun implementation in matrix multiplication
  17. Why is the final reduce step extremely slow in this MapReduce? (HiveQL, HDFS MapReduce)
  18. Unexpectedly poor and weirdly bimodal performance for store loop on Intel Skylake
  19. Why can’t my ultraportable laptop CPU maintain peak performance in HPC
  20. How slow (how many cycles) is calculating a square root?
  21. Performance optimisations of x86-64 assembly – Alignment and branch prediction
  22. How are cache memories shared in multicore Intel CPUs?
  23. Ways to improve git status performance
  24. Modern x86 cost model
  25. dbms_output.put_line
  26. Why do these goroutines not scale their performance from more concurrent executions?
  27. Cycles/cost for L1 Cache hit vs. Register on x86?
  28. Redis 10x more memory usage than data
  29. Best Practices for Multiple OnEdit Functions
  30. Efficient paging in MongoDB using mgo

Leave a Comment