How to use Fused Multiply-Add (FMA) instructions with SSE/AVX

by

The compiler is allowed to fuse a separated add and multiply, even though this changes the final result (by making it more accurate).

An FMA has only one rounding (it effectively keeps infinite precision for the internal temporary multiply result), while an ADD + MUL has two.

The IEEE and C standards allow this when #pragma STDC FP_CONTRACT ON is in effect, and compilers are allowed to have it ON by default (but not all do). Gcc contracts into FMA by default (with the default -std=gnu*, but not -std=c*, e.g. -std=c++14). For Clang, it’s only enabled with -ffp-contract=fast. (With just the #pragma enabled, only within a single expression like a+b*c, not across separate C++ statements.).

This is different from strict vs. relaxed floating point (or in gcc terms, -ffast-math vs. -fno-fast-math) that would allow other kinds of optimizations that could increase the rounding error depending on the input values. This one is special because of the infinite precision of the FMA internal temporary; if there was any rounding at all in the internal temporary, this wouldn’t be allowed in strict FP.

Even if you enable relaxed floating-point, the compiler might still choose not to fuse since it might expect you to know what you’re doing if you’re already using intrinsics.

So the best way to make sure you actually get the FMA instructions you want is you actually use the provided intrinsics for them:

FMA3 Intrinsics: (AVX2 – Intel Haswell)

  • _mm_fmadd_pd(), _mm256_fmadd_pd()
  • _mm_fmadd_ps(), _mm256_fmadd_ps()
  • and about a gazillion other variations…

FMA4 Intrinsics: (XOP – AMD Bulldozer)

  • _mm_macc_pd(), _mm256_macc_pd()
  • _mm_macc_ps(), _mm256_macc_ps()
  • and about a gazillion other variations…

More Related Contents:

  1. AVX/SSE version of xorshift128+
  2. Can x86’s MOV really be “free”? Why can’t I reproduce this at all?
  3. Why does mulss take only 3 cycles on Haswell, different from Agner’s instruction tables? (Unrolling FP loops with multiple accumulators)
  4. How to solve the 32-byte-alignment issue for AVX load/store operations?
  5. print a __m128i variable
  6. How to efficiently perform double/int64 conversions with SSE/AVX?
  7. Maximum memory which malloc can allocate
  8. SIMD signed with unsigned multiplication for 64-bit * 64-bit to 128-bit
  9. How to perform the inverse of _mm256_movemask_epi8 (VPMOVMSKB)?
  10. Obtaining peak bandwidth on Haswell in the L1 cache: only getting 62%
  11. What’s missing/sub-optimal in this memcpy implementation?
  12. Loading 8 chars from memory into an __m256 variable as packed single precision floats
  13. Simd matmul program gives different numerical results
  14. Using AVX CPU instructions: Poor performance without “/arch:AVX”
  15. What’s the difference between logical SSE intrinsics?
  16. Loop unrolling to achieve maximum throughput with Ivy Bridge and Haswell
  17. Fastest Implementation of the Natural Exponential Function Using SSE
  18. CPU and Data alignment
  19. Fast counting the number of set bits in __m128i register
  20. In GNU C inline asm, what are the size-override modifiers for xmm/ymm/zmm for a single operand?
  21. Fused multiply add and default rounding modes
  22. Do I get a performance penalty when mixing SSE integer/float SIMD instructions
  23. cpu dispatcher for visual studio for AVX and SSE
  24. Getting max value in a __m128i vector with SSE?
  25. How to determine if memory is aligned?
  26. Where do the values of uninitialized variables come from, in practice on real CPUs?
  27. Getting started with Intel x86 SSE SIMD instructions
  28. Compare 16 byte strings with SSE
  29. Preventing GCC from automatically using AVX and FMA instructions when compiled with -mavx and -mfma
  30. Efficient 4×4 matrix multiplication (C vs assembly)

Leave a Comment