Is the UNIX `time` command accurate enough for benchmarks? [closed]

by

time produces good enough times for benchmarks that run over one second otherwise the time it took exec()ing a process may be large compared to its run-time.

However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:

sudo chrt -f 99 /usr/bin/time --verbose <benchmark>

Or

sudo chrt -f 99 perf stat -ddd <benchmark>

sudo chrt -f 99 runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your /etc/security/limits.conf so that it doesn’t require a privileged process to use real-time priorities).

It also makes time report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.

perf stat -ddd is even more informative than /usr/bin/time and displays such information as instructions-per-cycle, branch and cache misses, etc.

And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.

More Related Contents:

  1. What do you think about the new Debian release cycle?
  2. How do I recursively grep all directories and subdirectories?
  3. Where can I set environment variables that crontab will use?
  4. How do I write standard error to a file while using “tee” with a pipe?
  5. Is it possible to use “/” in a filename?
  6. What is the meaning of “POSIX”?
  7. What does $@ mean in a shell script?
  8. Better way to rename files based on multiple patterns
  9. How to convert DOS/Windows newline (CRLF) to Unix newline (LF)
  10. Extract file basename without path and extension in bash [duplicate]
  11. How do you find the original user through multiple sudo and su commands?
  12. How can I remove the last character of a file in unix?
  13. Managing log files created by cron jobs
  14. How to run a shell script on a Unix console or Mac terminal?
  15. How do I escape spaces in path for scp copy in Linux?
  16. What is a .pid file and what does it contain?
  17. How to check if a file is empty in Bash?
  18. Should PATH contain directories or full paths to binaries?
  19. Execute a command without keeping it in history [closed]
  20. How do I set up cron to run a file just once at a specific time? [closed]
  21. How can I pipe initial input into process which will then be interactive?
  22. Restarting cron after changing crontab file? [closed]
  23. How can I copy the output of a command directly into my clipboard?
  24. How do I get the path of a process in Unix / Linux?
  25. How to convert \uXXXX unicode to UTF-8 using console tools in *nix
  26. Running shell script in parallel
  27. Difference between shell and environment variables
  28. Why crontab uses OR when both day of month and day of week specified?
  29. How can I use xargs to copy files that have spaces and quotes in their names?
  30. Find all files with name containing string [closed]

Leave a Comment