Best Compression algorithm for a sequence of integers

by

We have written recent research papers that survey the best schemes for this problem. Please see:

Daniel Lemire and Leonid Boytsov, Decoding billions of integers per second through vectorization,Software: Practice & Experience 45 (1), 2015.
http://arxiv.org/abs/1209.2137

Daniel Lemire, Nathan Kurz, Leonid Boytsov, SIMD Compression and the Intersection of Sorted Integers, Software: Practice and Experience (to appear) http://arxiv.org/abs/1401.6399

They include an extensive experimental evaluation.

You can find a complete implementation of all techniques in C++11 online:
https://github.com/lemire/FastPFor and https://github.com/lemire/SIMDCompressionAndIntersection

There are also C libraries: https://github.com/lemire/simdcomp and https://github.com/lemire/MaskedVByte

If you prefer Java, please see https://github.com/lemire/JavaFastPFOR

More Related Contents:

  1. An efficient compression algorithm for short text strings [closed]
  2. How do I create a URL shortener? [closed]
  3. Segmented Sieve of Eratosthenes?
  4. How do I calculate a point on a circle’s circumference?
  5. Algorithm for Additive Color Mixing for RGB Values
  6. Algorithm to generate all possible permutations of a list?
  7. Fast permutation -> number -> permutation mapping algorithms
  8. Algorithm to detect intersection of two rectangles?
  9. Generating the partitions of a number
  10. How to detect the BPM of a song in php [closed]
  11. Find the shortest path in a graph which visits certain nodes
  12. Why does this simple shuffle algorithm produce biased results?
  13. Algorithm to separate items of the same type
  14. how to calculate binary search complexity
  15. What’s a good algorithm to generate a maze? [closed]
  16. Array remove duplicate elements
  17. Combined area of overlapping circles
  18. Unsupervised clustering with unknown number of clusters
  19. What is the fastest way to find the closest point to a given point?
  20. Quick Sort Vs Merge Sort [duplicate]
  21. insert, delete, max in O(1)
  22. How Can I Best Guess the Encoding when the BOM (Byte Order Mark) is Missing?
  23. How does Dijkstra’s Algorithm and A-Star compare?
  24. Permute all unique enumerations of a vector in R
  25. Easiest algorithm of Voronoi diagram to implement? [closed]
  26. “Rounding” colour values to the nearest of a small set of colours
  27. Color gradient algorithm
  28. Compute the area of intersection between a circle and a triangle?
  29. Unbiased random number generator using a biased one
  30. CUDA stream compaction algorithm

Leave a Comment