Measuring memory bandwidth from the dot product of two arrays

by

There’s a few things going on here, that come down to:

  • You have to work fairly hard to get every last bit of performance out of the memory subsystem; and
  • Different benchmarks measure different things.

The first helps explain why you need multiple threads to saturate the available memory bandwidth. There is a lot of concurrency in the memory system, and it taking advantage of that will often require some concurrency in your CPU code. One big reason that multiple threads of execution help is latency hiding – while one thread is stalled waiting for data to arrive, another thread may be able to take advantage of some other data that has just become available.

The hardware helps you a lot on a single thread in this case – because the memory access is so predictable, the hardware can prefetch the data ahead of when you need it, giving you some of the advantage of latency hiding even with one thread; but there are limits to what prefetch can do. The prefetcher won’t take it upon itself to cross page boundaries, for instance. The canonical reference for much of this is What Every Programmer Should Know About Memory by Ulrich Drepper, which is now old enough that some gaps are starting to show (Intel’s Hot Chips overview of your Sandy Bridge processor is here – note in particular the tighter integration of the memory management hardware with the CPU).

As to the question about comparing with memset, mbw or STREAM, comparing across benchmarks will always cause headaches, even benchmarks that claim to be measuring the same thing. In particular, “memory bandwidth” isn’t a single number – performance varies quite a bit depending on the operations. Both mbw and Stream do some version of a copy operation, with STREAMs operations being spelled out here (taken straight from the web page, all operands are double-precision floating points):

------------------------------------------------------------------
name        kernel                  bytes/iter      FLOPS/iter
------------------------------------------------------------------
COPY:       a(i) = b(i)                 16              0
SCALE:      a(i) = q*b(i)               16              1
SUM:        a(i) = b(i) + c(i)          24              1
TRIAD:      a(i) = b(i) + q*c(i)        24              2
------------------------------------------------------------------

so roughly 1/2-1/3 of the memory operations in these cases are writes (and everything’s a write in the case of memset). While individual writes can be a little slower than reads, the bigger issue is that it’s much harder to saturate the memory subsystem with writes because of course you can’t do the equivalent of prefetching a write. Interleaving the reads and writes helps, but your dot-product example which is essentially all reads is going to be about the best-possible case for pegging the needle on memory bandwidth.

In addition, the STREAM benchmark is (intentionally) written completely portably, with only some compiler pragmas to suggest vectorization, so beating the STREAM benchmark isn’t necessarily a warning sign, especially when what you’re doing is two streaming reads.

More Related Contents:

  1. How to make threads write their numbers in reverse order using OpenMP?
  2. How to determine CPU and memory consumption from inside a process
  3. What’s the difference between assignment operator and copy constructor?
  4. Why aren’t pointers initialized with NULL by default?
  5. Why is address zero used for the null pointer?
  6. C++ OpenMP Parallel For Loop – Alternatives to std::vector [closed]
  7. How to efficiently perform double/int64 conversions with SSE/AVX?
  8. When to use bit-fields in C?
  9. memory layout C++ objects [closed]
  10. How to read/write arbitrary bits in C/C++
  11. Do class/struct members always get created in memory in the order they were declared?
  12. Mixing C++11 atomics and OpenMP
  13. Loading 8 chars from memory into an __m256 variable as packed single precision floats
  14. Allocating more memory than there exists using malloc
  15. Declare large array on Stack
  16. Loop unrolling to achieve maximum throughput with Ivy Bridge and Haswell
  17. How to include omp.h in OS X?
  18. What is the behavior of “delete” with stack objects? [duplicate]
  19. Can the “Application Error” dialog box be disabled?
  20. g++ variable size array no warning?
  21. How to solve Memory Fragmentation
  22. What is “bit padding” or “padding bits” exactly?
  23. How do I call the original “operator new” if I have overloaded it?
  24. How-to write a password-safe class?
  25. C++: Timing in Linux (using clock()) is out of sync (due to OpenMP?)
  26. Is Python faster and lighter than C++? [closed]
  27. How is its lifetime of a return value extended to the scope of the calling function when it is bound to a const reference in the calling function?
  28. C++11: Replace all non-owning raw pointers with std::shared_ptr()?
  29. Class members and explicit stack/heap allocation
  30. OpenMP and CPU affinity

Leave a Comment