Benchmarking – How to count number of instructions sent to CPU to find consumed MIPS

perf stat --all-user ./my_program on Linux will use CPU performance counters to record how many user-space instructions it ran, and how many core clock cycles it took. And how much CPU time it used, and will calculate average instructions per core clock cycle for you, e.g.

3,496,129,612      instructions:u            #    2.61  insn per cycle

It calculates IPC for you; this is usually more interesting than instructions per second. uops per clock is usually even more interesting in terms of how close you are to maxing out the front-end, though. You can manually calculate MIPS from instructions and task-clock. For most other events perf prints a comment with a per-second rate.

(If you don’t use --all-user, you can use perf stat -e task-clock:u,instructions:u , … to have those specific events count in user-space only, while other events can count always, including inside interrupt handlers and system calls.)

But see How to calculate MIPS using perf stat for more detail on instructions / task-clock vs. instructions / elapsed_time if you do actually want total or average MIPS across cores, and counting sleep or not.


For an example output from using it on a tiny microbenchmark loop in a static executable, see Can x86’s MOV really be “free”? Why can’t I reproduce this at all?

How can I get real-time information at run-time

Do you mean from within the program, to profile only part of it? There’s a perf API where you can do perf_event_open or something. Or use a different library for direct access to the HW perf counters.

perf stat is great for microbenchmarking a loop that you’ve isolated into a stand-alone program that just runs the hot loop for a second or so.

Or maybe you mean something else. perf stat -I 1000 ... ./a.out will print counter values every 1000 ms (1 second), to see how program behaviour changes in real time with whatever time window you want (down to 10ms intervals).

sudo perf top is system-wide, slightly like Unix top

There’s also perf record --timestamp to record a timestamp with each event sample. perf report -D might be useful along with this. See http://www.brendangregg.com/perf.html, he mentions something about -T (--timestamp). I haven’t really used this; I mostly isolate single loops I’m tuning into a static executable I can run under perf stat.


And is it possible to find the type of instruction set (add, compare, in, jump, etc)?

Intel x86 CPUs at least have a counter for branch instructions, but other types aren’t differentiated, other than FP instructions. This is probably common to most architectures that have perf counters at all.

For Intel CPUs, there’s ocperf.py, a wrapper for perf with symbolic names for more microarchitectural events. (Update: plain perf now knows the names of most uarch-specific counters so you don’t need ocperf.py anymore.)

perf stat -e task_clock,cycles,instructions,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.scalar_double,uops_executed.x87 ./my_program

It’s not designed to tell you what instructions are running, you can already tell that from tracing execution. Most instructions are fully pipelined, so the interesting thing is which ports have the most pressure. The exception is the divide/sqrt unit: there’s a counter for arith.divider_active: “Cycles when divide unit is busy executing divide or square root operations. Accounts for integer and floating-point operations“. The divider isn’t fully pipelined, so a new divps or sqrtps can’t always start even if no older uops are ready to execute on port 0. (http://agner.org/optimize/)

Related: linux perf: how to interpret and find hotspots for using perf to identify hotspots. Especially using top-down profiling you have perf sample the call-stack to see which functions make a lot of expensive child calls. (I mention this in case that’s what you really wanted to know, rather than instruction mix.)

Related:


For exact dynamic instruction counts, you might use an instrumentation tool like Intel PIN, if you’re on x86. https://software.intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool.

perf stat counts for the instructions:u hardware even should also be more or less exact, and is in practice very repeatable across runs of the same program doing the same work.

On recent Intel CPUs, there’s HW support for recording which way conditional / indirect branches went, so you can reconstruct exactly which instructions ran in which order, assuming no self-modifying code and that you can still read any JIT buffers. Intel PT.


Sorry I don’t know what the equivalents are on AMD CPUs.

Leave a Comment