An out-of-memory error in clustering

by

  1. toCharArray will copy the String into a char[] every time, entirely unnecessary.

  2. Don’t use String to store bits or numbers. If you have less than 32 bits, use an int. If you have less than 64 bits, use a long. If more, use a long[].

  3. Try some optimizations based on bit operations. For example, you can compute the Hamming distance with a simple bit count and xor operation. You can also get a cheap lower bound based on the number of set bits – if one has 6 bits, the other 2, at least 4 bits have to be different.

  4. Avoid ArrayList<Integer> and ArrayList<Vertex>. These need roughly 20 bytes per integer rather than 4. That is 400% overhead. Use int[]+size, double array if full (ArrayList does the same, but uses boxed integers).

Use a profiler such as visualvm to see where you wastw memory.

My guess is that String [][] dist2 = new String[200000][276]; is to blame. 200000*276*50 is probably enough to eat all your memory. Get rid of useless strings!

More Related Contents:

  1. What is a StackOverflowError?
  2. How to deal with “java.lang.OutOfMemoryError: Java heap space” error?
  3. Error java.lang.OutOfMemoryError: GC overhead limit exceeded
  4. Catching java.lang.OutOfMemoryError?
  5. What is an OutOfMemoryError and how do I debug and fix it
  6. java.lang.OutOfMemoryError: Java heap space
  7. “java.lang.OutOfMemoryError : unable to create new native Thread”
  8. bitmap size exceeds Vm budget error android
  9. Is it possible to catch out of memory exception in java? [duplicate]
  10. Can the JVM recover from an OutOfMemoryError without a restart
  11. Encryption of video files?
  12. How to load a large xlsx file with Apache POI?
  13. java.lang.OutOfMemoryError: Java heap space in Maven
  14. Processing large xlsx file
  15. Unable to execute dex: GC overhead limit exceeded in Eclipse
  16. Java: Unable to create new native thread
  17. Horrendous performance & large heap footprint of Java 8 constructor reference?
  18. PySpark: java.lang.OutofMemoryError: Java heap space
  19. Java – shutting down on Out of Memory Error
  20. Java OutOfMemoryError strange behaviour
  21. Error While Reading Large Excel Files (xlsx) Via Apache POI
  22. How do I give Jenkins more heap space when it’s running as a daemon on Ubuntu?
  23. Old JaxB and JDK8 Metaspace OutOfMemory Issue
  24. Java OutOfMemoryError in reading a large text file
  25. Is the garbage collector guaranteed to run before Out of Memory Error?
  26. java.lang.OutOfMemoryError: Java heap space with NetBeans
  27. java.lang.OutOfMemoryError: unable to create new native thread error using ChromeDriver and Chrome through Selenium in Spring boot
  28. Java heap space in netbeans.. but I’ve increased the heap size already!
  29. How to avoid OOM (Out of memory) error when retrieving all records from huge table?
  30. OutputStream OutOfMemoryError when sending HTTP

Leave a Comment