Fastest way to unpack 32 bits to a 32 byte SIMD vector

by

To “broadcast” the 32 bits of a 32-bit integer x to 32 bytes of a 256-bit YMM register z or 16 bytes of a two 128-bit XMM registers z_low and z_high you can do the following.

With AVX2:

__m256i y = _mm256_set1_epi32(x);
__m256i z = _mm256_shuffle_epi8(y,mask1);
z = _mm256_and_si256(z,mask2);

Without AVX2 it’s best to do this with SSE: 

__m128i y = _mm_set1_epi32(x);      
__m128i z_low  = _mm_shuffle_epi8(y,mask_low);
__m128i z_high = _mm_shuffle_epi8(y,mask_high); 
z_low  = _mm_and_si128(z_low ,mask2);
z_high = _mm_and_si128(z_high,mask2);

The masks and a working example are shown below. If you plan to do this several times you should probably
define the masks outside of the main loop.

#include <immintrin.h>
#include <stdio.h>

int main() {
    int x = 0x87654321;

    static const char mask1a[32] = {
        0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00,
        0x01, 0x01, 0x01, 0x01,
        0x01, 0x01, 0x01, 0x01,
        0x02, 0x02, 0x02, 0x02,
        0x02, 0x02, 0x02, 0x02,
        0x03, 0x03, 0x03, 0x03,
        0x03, 0x03, 0x03, 0x03
    };

    static const char mask2a[32] = {
        0x01, 0x02, 0x04, 0x08,
        0x10, 0x20, 0x40, 0x80,
        0x01, 0x02, 0x04, 0x08,
        0x10, 0x20, 0x40, 0x80,
        0x01, 0x02, 0x04, 0x08,
        0x10, 0x20, 0x40, 0x80,
        0x01, 0x02, 0x04, 0x08,
        0x10, 0x20, 0x40, 0x80,
    };

char out[32];

#if defined ( __AVX2__ )
    __m256i mask2 = _mm256_loadu_si256((__m256i*)mask2a);
    __m256i mask1  = _mm256_loadu_si256((__m256i*)mask1a);

    __m256i y =    _mm256_set1_epi32(x);
    __m256i z =    _mm256_shuffle_epi8(y,mask1);
    z = _mm256_and_si256(z,mask2);

    _mm256_storeu_si256((__m256i*)out,z);

#else
    __m128i mask2 = _mm_loadu_si128((__m128i*)mask2a);
    __m128i mask_low  = _mm_loadu_si128((__m128i*)&mask1a[ 0]);
    __m128i mask_high = _mm_loadu_si128((__m128i*)&mask1a[16]);    

    __m128i y = _mm_set1_epi32(x); 
    __m128i z_low  = _mm_shuffle_epi8(y,mask_low);
    __m128i z_high = _mm_shuffle_epi8(y,mask_high);
    z_low  = _mm_and_si128(z_low,mask2);
    z_high = _mm_and_si128(z_high,mask2);

    _mm_storeu_si128((__m128i*)&out[ 0],z_low);
    _mm_storeu_si128((__m128i*)&out[16],z_high);
#endif
    for(int i=0; i<8; i++) {
        for(int j=0; j<4; j++) {        
            printf("%x ", out[4*i+j]);
        }printf("\n");
    } printf("\n");
}

To get 0 or -1 in each vector element:

It takes one extra step _mm256_cmpeq_epi8 against all-zeros. Any non-zero turns into 0, and zero turns into -1. If we don’t want this inversion, use andnot instead of and. It inverts its first operand.

__m256i expand_bits_to_bytes(uint32_t x)
{
    __m256i xbcast = _mm256_set1_epi32(x);    // we only use the low 32bits of each lane, but this is fine with AVX2

    // Each byte gets the source byte containing the corresponding bit
    __m256i shufmask = _mm256_set_epi64x(
        0x0303030303030303, 0x0202020202020202,
        0x0101010101010101, 0x0000000000000000);
    __m256i shuf  = _mm256_shuffle_epi8(xbcast, shufmask);

    __m256i andmask  = _mm256_set1_epi64x(0x8040201008040201);  // every 8 bits -> 8 bytes, pattern repeats.
    __m256i isolated_inverted = _mm256_andnot_si256(shuf, andmask);

    // this is the extra step: compare each byte == 0 to produce 0 or -1
    return _mm256_cmpeq_epi8(isolated_inverted, _mm256_setzero_si256());
     // alternative: compare against the AND mask to get 0 or -1,
     // avoiding the need for a vector zero constant.
}

See it on the Godbolt Compiler Explorer.

Also see is there an inverse instruction to the movemask instruction in intel avx2? for other element sizes.

More Related Contents:

  1. Fastest Implementation of Exponential Function Using AVX
  2. Find the first instance of a character using simd
  3. is there an inverse instruction to the movemask instruction in intel avx2?
  4. How to perform the inverse of _mm256_movemask_epi8 (VPMOVMSKB)?
  5. How to efficiently convert an 8-bit bitmap to array of 0/1 integers with x86 SIMD [duplicate]
  6. Convention for displaying vector registers
  7. Fastest way to do horizontal vector sum with AVX instructions [duplicate]
  8. Load address calculation when using AVX2 gather instructions
  9. What are the best instruction sequences to generate vector constants on the fly?
  10. What’s missing/sub-optimal in this memcpy implementation?
  11. Loading 8 chars from memory into an __m256 variable as packed single precision floats
  12. Fastest way to compute absolute value using SSE
  13. Header files for x86 SIMD intrinsics
  14. Sum reduction of unsigned bytes without overflow, using SSE2 on Intel
  15. Per-element atomicity of vector load/store and gather/scatter?
  16. Transpose an 8×8 float using AVX/AVX2
  17. SSE multiplication of 4 32-bit integers
  18. Can I use the AVX FMA units to do bit-exact 52 bit integer multiplications?
  19. How to convert 32-bit float to 8-bit signed char? (4:1 packing of int32 to int8 __m256i)
  20. Half-precision floating-point arithmetic on Intel chips
  21. practical BigNum AVX/SSE possible?
  22. Simd matmul program gives different numerical results
  23. inlining failed in call to always_inline ‘__m256d _mm256_broadcast_sd(const double*)’
  24. Difference between x86, x32, and x64 architectures?
  25. Emulating shifts on 32 bytes with AVX
  26. SIMD math libraries for SSE and AVX
  27. Count each bit-position separately over many 64-bit bitmasks, with AVX but not AVX2
  28. Are load ops deallocated from the RS when they dispatch, complete or some other time?
  29. Efficient sse shuffle mask generation for left-packing byte elements
  30. What is the maximum possible IPC can be achieved by Intel Nehalem Microarchitecture?

Leave a Comment