What’s the relative speed of floating point add vs. floating point multiply

by

It also depends on instruction mix. Your processor will have several computation units standing by at any time, and you’ll get maximum throughput if all of them are filled all the time. So, executing a loop of mul’s is just as fast as executing a loop or adds – but the same doesn’t hold if the expression becomes more complex.

For example, take this loop:

for(int j=0;j<NUMITER;j++) {
  for(int i=1;i<NUMEL;i++) {
    bla += 2.1 + arr1[i] + arr2[i] + arr3[i] + arr4[i] ;
  }
}

for NUMITER=10^7, NUMEL=10^2, both arrays initialized to small positive numbers (NaN is much slower), this takes 6.0 seconds using doubles on a 64-bit proc. If I replace the loop with

bla += 2.1 * arr1[i] + arr2[i] + arr3[i] * arr4[i] ;

It only takes 1.7 seconds… so since we “overdid” the additions, the muls were essentially free; and the reduction in additions helped. It get’s more confusing:

bla += 2.1 + arr1[i] * arr2[i] + arr3[i] * arr4[i] ;

— same mul/add distribution, but now the constant is added in rather than multiplied in — takes 3.7 seconds. Your processor is likely optimized to perform typical numerical computations more efficiently; so dot-product like sums of muls and scaled sums are about as good as it gets; adding constants isn’t nearly as common, so that’s slower…

bla += someval + arr1[i] * arr2[i] + arr3[i] * arr4[i] ; /*someval == 2.1*/

again takes 1.7 seconds.

bla += someval + arr1[i] + arr2[i] + arr3[i] + arr4[i] ; /*someval == 2.1*/

(same as initial loop, but without expensive constant addition: 2.1 seconds)

bla += someval * arr1[i] * arr2[i] * arr3[i] * arr4[i] ; /*someval == 2.1*/

(mostly muls, but one addition:1.9 seconds)

So, basically; it’s hard to say which is faster, but if you wish to avoid bottlenecks, more important is to have a sane mix, avoid NaN or INF, avoid adding constants. Whatever you do, make sure you test, and test various compiler settings, since often small changes can just make the difference.

Some more cases:

bla *= someval; // someval very near 1.0; takes 2.1 seconds
bla *= arr1[i] ;// arr1[i] all very near 1.0; takes 66(!) seconds
bla += someval + arr1[i] * arr2[i] + arr3[i] * arr4[i] ; // 1.6 seconds
bla += someval + arr1[i] * arr2[i] + arr3[i] * arr4[i] ; //32-bit mode, 2.2 seconds
bla += someval + arr1[i] * arr2[i] + arr3[i] * arr4[i] ; //32-bit mode, floats 2.2 seconds
bla += someval * arr1[i]* arr2[i];// 0.9 in x64, 1.6 in x86
bla += someval * arr1[i];// 0.55 in x64, 0.8 in x86
bla += arr1[i] * arr2[i];// 0.8 in x64, 0.8 in x86, 0.95 in CLR+x64, 0.8 in CLR+x86

More Related Contents:

  1. Can I use the AVX FMA units to do bit-exact 52 bit integer multiplications?
  2. Why are floating point numbers inaccurate?
  3. Why not use Double or Float to represent currency?
  4. Floating point inaccuracy examples
  5. What is the rationale for all comparisons returning false for IEEE754 NaN values?
  6. Why do I see a double variable initialized to some value like 21.4 as 21.399999618530273?
  7. What range of numbers can be represented in a 16-, 32- and 64-bit IEEE-754 systems?
  8. How is floating point stored? When does it matter?
  9. Why is NaN not equal to NaN? [duplicate]
  10. Floating point vs integer calculations on modern hardware
  11. What is the instruction that gives branchless FP min and max on x86?
  12. Why is SSE scalar sqrt(x) slower than rsqrt(x) * x?
  13. Change floating point rounding mode
  14. Does float have a negative zero? (-0f)
  15. Floating point error in representation?
  16. Extended (80-bit) double floating point in x87, not SSE2 – we don’t miss it?
  17. What is the fastest way to convert float to int on x86
  18. Why does IEEE 754 reserve so many NaN values?
  19. Why can’t I compare reals in Standard ML?
  20. How do I use floating point number literals when using generic types?
  21. How do I get the minimum or maximum value of an iterator containing floating point numbers?
  22. Modern x86 cost model
  23. Optimize for fast multiplication but slow addition: FMA and doubledouble
  24. What is the size of float and double in C and C++? [duplicate]
  25. How can I use a HashMap with f64 as key in Rust?
  26. How to perform round to even with floating point numbers
  27. What uses do floating point NaN payloads have?
  28. Having parameter (constant) variable with NaN value in Fortran
  29. Using software floating point on x86 linux
  30. Maximum number of decimal digits that can affect a double

Leave a Comment