How can I find memory leaks in long-running Perl program?

by

It may be relevant that Perl never gives memory back to the system by itself: It’s all up to malloc() and all the rules associated with that.

Knowing how malloc() allocates memory is important to answering the greater question, and it varies from system to system, but in general most malloc() implementations are optimized for programs allocating and deallocating in stack-like orders. Perl uses reference-counting for tracking memory which means that deallocations which means (unlike a GC-based language which uses malloc() underneath) it is actually not all that difficult to tell where deallocation is going to occur, and in what order.

It may be that you can reorganize your program to take advantage of this fact- by calling undef($old_object) explicitly – and in the right order, in a manner similar to the way C-programmers say free(old_object);

For long-running programs (days, months, etc), where I have loads of load/copy/dump cycles, I garbage-collect using exit() and exec(), and where it’s otherwide unfeasible, I simply pack up my data structures (using Storable) and file descriptors (using $^F) and exec($0) – usually with an environment variable set like $ENV{EXEC_GC_MODE}, and you may need something similar even if you don’t have any leaks of your own simply because Perl is leaking small chunks that your system’s malloc() can’t figure out how to give back.

Of course, if you do have leaks in your code, then the rest of my advice is somewhat more relevant. It was originally posted to another question on this subject, but it didn’t explicitly cover long-running programs.

All perl program memory leaks will either be an XS holding onto a reference, or a circular data structure. Devel::Cycle is a great tool for finding circular references, if you know what structures are likely to contain the loops. Devel::Peek can be used to find objects with a higher-than-expected reference count.

If you don’t know where else to look, Devel::LeakTrace::Fast could be a good first place, but you’ll need a perl built for debugging.

If you suspect the leak is inside XS-space, it’s much harder, and Valgrind will probably be your best bet. Test::Valgrind may help you lower the amount of code you need to search, but this won’t work on Windows, so you’d have to port (at least the leaky portion) to Linux in order to do this.

More Related Contents:

  1. Perl memory usage profiling and leak detection?
  2. How can I convert my shell script to Perl? [closed]
  3. search using perl one liner within a file [closed]
  4. How to re-direct the contents to a file instead of printing it to terminal
  5. How do I remove duplicate items from an array in Perl?
  6. What is the difference between library files and modules?
  7. How do I read in the contents of a directory in Perl?
  8. How can I pass command-line arguments to a Perl program?
  9. Circular references in Javascript / Garbage collector
  10. How do I efficiently parse a CSV file in Perl?
  11. How can I generate all permutations of an array in Perl?
  12. How can I install CPAN modules locally without root access (DynaLoader.pm line 229 error)?
  13. OpenSSL DH Key Too Small Error
  14. Manual installation of a Perl Module
  15. @ARGV is empty using ActivePerl in Windows 7
  16. ImageMagick Command-Line Option Order (and Categories of Command-Line Parameters)
  17. Perl Dereferencing Syntax
  18. How to extrace pg_backend_pid from postgresql in shell script and pass it to another process?
  19. What exactly does Perl’s “bless” do?
  20. What is the difference between `$this`, `@that`, and `%those` in Perl?
  21. Why are use warnings; use strict; not default in Perl?
  22. How to use shell variables in perl command call in a bash shell script?
  23. How do I use symbolic references in Perl?
  24. What is the meaning of @_ in Perl?
  25. How do I list available methods on a given object or package in Perl?
  26. Why do Perl variables need to start with $, %,@ (sigils)?
  27. Print lines from one file that are not contained in another file
  28. Perl DBD::Oracle Module installation
  29. Explain the deviousness of the Perl “preamble”
  30. Why doesn’t Perl’s each() iterate through the entire hash the second time?

Leave a Comment