What’s the difference between logical SSE intrinsics?

by
  1. Is there any difference between using one or another intrinsic (with appropriate type casting). Won’t there be any hidden costs like longer execution in some specific situation?

Yes, there can be performance reasons to choose one vs. the other.

1: Sometimes there is an extra cycle or two of latency (forwarding delay) if the output of an integer execution unit needs to be routed to the input of an FP execution unit, or vice versa. It takes a LOT of wires to move 128b of data to any of many possible destinations, so CPU designers have to make tradeoffs, like only having a direct path from every FP output to every FP input, not to ALL possible inputs.

See this answer, or Agner Fog’s microarchitecture doc for bypass-delays. Search for “Data bypass delays on Nehalem” in Agner’s doc; it has some good practical examples and discussion. He has a section on it for every microarch he has analysed.

However, the delays for passing data between the
different domains or different types of registers are smaller on the
Sandy Bridge and Ivy Bridge than on the Nehalem, and often zero. —
Agner Fog’s micro arch doc

Remember that latency doesn’t matter if it isn’t on the critical path of your code (except sometimes on Haswell/Skylake where it infects later use of the produced value, long after actual bypass :/). Using pshufd instead of movaps + shufps can be a win if uop throughput is your bottleneck, rather than latency of your critical path.

2: The ...ps version takes 1 fewer byte of code than the other two for legacy-SSE encoding. (Not AVX). This will align the following instructions differently, which can matter for the decoders and/or uop cache lines. Generally smaller is better for better code density in I-cache and fetching code from RAM, and packing into the uop cache.

3: Recent Intel CPUs can only run the FP versions on port5.

  • Merom (Core2) and Penryn: orps can run on p0/p1/p5, but integer-domain only. Presumably all 3 versions decoded into the exact same uop. So the cross-domain forwarding delay happens. (AMD CPUs do this too: FP bitwise instructions run in the ivec domain.)

  • Nehalem / Sandybridge / IvB / Haswell / Broadwell: por can run on p0/p1/p5, but orps can run only on port5. p5 is also needed by shuffles, but the FMA, FP add, and FP mul units are on ports 0/1.

  • Skylake: por and orps both have 3-per-cycle throughput. Intel’s optimization manual has some info about bypass forwarding delays: to/from FP instructions it depends on which port the uop ran on. (Usually still port 5 because the FP add/mul/fma units are on ports 0 and 1.) See also Haswell AVX/FMA latencies tested 1 cycle slower than Intel’s guide says – “bypass” latency can affect every use of the register until it’s overwritten.

Note that on SnB/IvB (AVX but not AVX2), only p5 needs to handle 256b logical ops, as vpor ymm, ymm requires AVX2. This was probably not the reason for the change, since Nehalem did this.

How to choose wisely:

Keep in mind that compilers can use por for _mm_or_pd if they want, so some of this applies mostly to hand-written asm. But some compilers are somewhat faithful to the intrinsics you choose.

If logical op throughput on port5 could be a bottleneck, then use the integer versions, even on FP data. This is especially true if you want to use integer shuffles or other data-movement instructions.

AMD CPUs always use the integer domain for logicals, so if you have multiple integer-domain things to do, do them all at once to minimize round-trips between domains. Shorter latencies will get things cleared out of the reorder buffer faster, even if a dep chain isn’t the bottleneck for your code.

If you just want to set/clear/flip a bit in FP vectors between FP add and mul instructions, use the ...ps logicals, even on double-precision data, because single and double FP are the same domain on every CPU in existence, and the ...ps versions are one byte shorter (without AVX).

There are practical / human-factor reasons for using the ...pd versions, though, with intrinsics. Readability of your code by other humans is a factor: They’ll wonder why you’re treating your data as singles when it’s actually doubles. For C/C++ intrinsics, littering your code with casts between __m128 and __m128d is not worth it. (And hopefully a compiler will use orps for _mm_or_pd anyway, if compiling without AVX where it will actually save a byte.)

If tuning on the level of insn alignment matters, write in asm directly, not intrinsics! (Having the instruction one byte longer might align things better for uop cache line density and/or decoders, but with prefixes and addressing modes you can extend instructions in general)

For integer data, use the integer versions. Saving one instruction byte isn’t worth the bypass-delay between paddd or whatever, and integer code often keeps port5 fully occupied with shuffles. For Haswell, many shuffle / insert / extract / pack / unpack instructions became p5 only, instead of p1/p5 for SnB/IvB. (Ice Lake finally added a shuffle unit on another port for some more common shuffles.)

  1. These intrinsics maps to three different x86 instructions (por, orps,
    orpd). Does anyone have any ideas why Intel is wasting precious opcode
    space for several instructions which do the same thing?

If you look at the history of these instruction sets, you can kind of see how we got here.

por  (MMX):     0F EB /r
orps (SSE):     0F 56 /r
orpd (SSE2): 66 0F 56 /r
por  (SSE2): 66 0F EB /r

MMX existed before SSE, so it looks like opcodes for SSE (...ps) instructions were chosen out of the same 0F xx space. Then for SSE2, the ...pd version added a 66 operand-size prefix to the ...ps opcode, and the integer version added a 66 prefix to the MMX version.

They could have left out orpd and/or por, but they didn’t. Perhaps they thought that future CPU designs might have longer forwarding paths between different domains, and so using the matching instruction for your data would be a bigger deal. Even though there are separate opcodes, AMD and early Intel treated them all the same, as int-vector.

Related / near duplicate:

More Related Contents:

  1. print a __m128i variable
  2. Fast counting the number of set bits in __m128i register
  3. Do I get a performance penalty when mixing SSE integer/float SIMD instructions
  4. Fastest Implementation of the Natural Exponential Function Using SSE
  5. Emulating shifts on 32 bytes with AVX
  6. C++ error: ‘_mm_sin_ps’ was not declared in this scope
  7. How to determine if memory is aligned?
  8. Getting started with Intel x86 SSE SIMD instructions
  9. Get member of __m128 by index?
  10. Compare 16 byte strings with SSE
  11. Most efficient way to check if all __m128i components are 0 [using
  12. How to convert 32-bit float to 8-bit signed char? (4:1 packing of int32 to int8 __m256i)
  13. inlining failed in call to always_inline ‘_mm_mullo_epi32’: target specific option mismatch
  14. Intel SSE: Why does `_mm_extract_ps` return `int` instead of `float`?
  15. How to implement atoi using SIMD?
  16. clflush to invalidate cache line via C function
  17. parallel prefix (cumulative) sum with SSE
  18. How to merge a scalar into a vector without the compiler wasting an instruction zeroing upper elements? Design limitation in Intel’s intrinsics?
  19. How to perform the inverse of _mm256_movemask_epi8 (VPMOVMSKB)?
  20. Efficient 4×4 matrix vector multiplication with SSE: horizontal add and dot product – what’s the point?
  21. Header files for x86 SIMD intrinsics
  22. Sum reduction of unsigned bytes without overflow, using SSE2 on Intel
  23. AVX/SSE version of xorshift128+
  24. Fastest method to calculate sum of all packed 32-bit integers using AVX512 or AVX2
  25. How to use Fused Multiply-Add (FMA) instructions with SSE/AVX
  26. How to use MSVC intrinsics to get the equivalent of this GCC code?
  27. SSE multiplication of 4 32-bit integers
  28. Produce loops without cmp instruction in GCC
  29. SSE reduction of float vector
  30. Why vectorizing the loop over 64-bit elements does not have performance improvement over large buffers?

Leave a Comment