How to understand Locality Sensitive Hashing? [closed]

by

The best tutorial I have seen for LSH is in the book: Mining of Massive Datasets.
Check Chapter 3 – Finding Similar Items
http://infolab.stanford.edu/~ullman/mmds/ch3a.pdf

Also I recommend the below slide:
http://www.cs.jhu.edu/%7Evandurme/papers/VanDurmeLallACL10-slides.pdf .
The example in the slide helps me a lot in understanding the hashing for cosine similarity.

I borrow two slides from Benjamin Van Durme & Ashwin Lall, ACL2010 and try to explain the intuitions of LSH Families for Cosine Distance a bit.
enter image description here

  • In the figure, there are two circles w/ red and yellow colored, representing two two-dimensional data points. We are trying to find their cosine similarity using LSH.
  • The gray lines are some uniformly randomly picked planes.
  • Depending on whether the data point locates above or below a gray line, we mark this relation as 0/1.
  • On the upper-left corner, there are two rows of white/black squares, representing the signature of the two data points respectively. Each square is corresponding to a bit 0(white) or 1(black).
  • So once you have a pool of planes, you can encode the data points with their location respective to the planes. Imagine that when we have more planes in the pool, the angular difference encoded in the signature is closer to the actual difference. Because only planes that resides between the two points will give the two data different bit value.

enter image description here

  • Now we look at the signature of the two data points. As in the example, we use only 6 bits(squares) to represent each data. This is the LSH hash for the original data we have.
  • The hamming distance between the two hashed value is 1, because their signatures only differ by 1 bit.
  • Considering the length of the signature, we can calculate their angular similarity as shown in the graph.

I have some sample code (just 50 lines) in python here which is using cosine similarity.
https://gist.github.com/94a3d425009be0f94751

More Related Contents:

  1. Perceptron learning algorithm not converging to 0
  2. Implementing a HashMap in C [closed]
  3. How to count the number of times a while loop is executed
  4. please help me to find the error in this C code [closed]
  5. Using the pow() function with integers in C
  6. C: Reading strings from binary file
  7. Why is scanf() causing infinite loop in this code?
  8. Difference between passing array and array pointer into function in C
  9. Override a function call in C
  10. How to create a high resolution timer in Linux to measure program performance?
  11. The ternary (conditional) operator in C
  12. Opaque C structs: various ways to declare them
  13. How do I lowercase a string in C?
  14. What’s an object file in C?
  15. C: for loop int initial declaration
  16. Padding in structures in C
  17. How to convert string to float?
  18. Variable definition inside switch statement
  19. How can I create a Makefile for C projects with SRC, OBJ, and BIN subdirectories?
  20. Timeout Function
  21. warning: return type defaults to ‘int’ [-Wreturn-type]
  22. Spaces inserted by the C preprocessor
  23. How do you do exponentiation in C?
  24. signed and unsigned arithmetic implementation on x86
  25. What is the difference between a Linux platform driver and normal device driver?
  26. Checking if a dir. entry returned by readdir is a directory, link or file. dent->d_type isn’t showing the type
  27. C string append
  28. Why can’t a modifiable lvalue have an array type?
  29. functions returning char pointer
  30. How to implement Bitcount using only Bitwise operators?

Leave a Comment