Very fast memcpy for image processing?

by

Courtesy of William Chan and Google. 30-70% faster than memcpy in Microsoft Visual Studio 2005.

void X_aligned_memcpy_sse2(void* dest, const void* src, const unsigned long size)
{

  __asm
  {
    mov esi, src;    //src pointer
    mov edi, dest;   //dest pointer

    mov ebx, size;   //ebx is our counter 
    shr ebx, 7;      //divide by 128 (8 * 128bit registers)


    loop_copy:
      prefetchnta 128[ESI]; //SSE2 prefetch
      prefetchnta 160[ESI];
      prefetchnta 192[ESI];
      prefetchnta 224[ESI];

      movdqa xmm0, 0[ESI]; //move data from src to registers
      movdqa xmm1, 16[ESI];
      movdqa xmm2, 32[ESI];
      movdqa xmm3, 48[ESI];
      movdqa xmm4, 64[ESI];
      movdqa xmm5, 80[ESI];
      movdqa xmm6, 96[ESI];
      movdqa xmm7, 112[ESI];

      movntdq 0[EDI], xmm0; //move data from registers to dest
      movntdq 16[EDI], xmm1;
      movntdq 32[EDI], xmm2;
      movntdq 48[EDI], xmm3;
      movntdq 64[EDI], xmm4;
      movntdq 80[EDI], xmm5;
      movntdq 96[EDI], xmm6;
      movntdq 112[EDI], xmm7;

      add esi, 128;
      add edi, 128;
      dec ebx;

      jnz loop_copy; //loop please
    loop_copy_end:
  }
}

You may be able to optimize it further depending on your exact situation and any assumptions you are able to make.

You may also want to check out the memcpy source (memcpy.asm) and strip out its special case handling. It may be possible to optimise further!

More Related Contents:

  1. Is it safe to read past the end of a buffer within the same page on x86 and x64?
  2. What is the instruction that gives branchless FP min and max on x86?
  3. What is the fastest way to convert float to int on x86
  4. Bit popcount for large buffer, with Core 2 CPU (SSSE3)
  5. Why does C++ code for testing the Collatz conjecture run faster than hand-written assembly?
  6. Why does mulss take only 3 cycles on Haswell, different from Agner’s instruction tables? (Unrolling FP loops with multiple accumulators)
  7. Using LEA on values that aren’t addresses / pointers?
  8. Can I use Intel syntax of x86 assembly with GCC?
  9. Stack allocation, padding, and alignment
  10. What is exactly the base pointer and stack pointer? To what do they point?
  11. Syscall implementation of exit()
  12. Loop with function call faster than an empty loop
  13. What’s missing/sub-optimal in this memcpy implementation?
  14. Efficient integer compare function
  15. What parts of this HelloWorld assembly code are essential if I were to write the program in assembly?
  16. x86_64 ASM – maximum bytes for an instruction?
  17. How to power down the computer from a freestanding environment?
  18. Fastest way to calculate a 128-bit integer modulo a 64-bit integer
  19. multi-word addition using the carry flag
  20. Getting max value in a __m128i vector with SSE?
  21. Why GCC compiled C program needs .eh_frame section?
  22. How does a mutex lock and unlock functions prevents CPU reordering?
  23. Calling C functions from x86 assembly language
  24. Count each bit-position separately over many 64-bit bitmasks, with AVX but not AVX2
  25. calling assembly function from c
  26. Is it possible to call a non-exported function that resides in an exe?
  27. Writing a Linux int 80h system-call wrapper in GNU C inline assembly [duplicate]
  28. Produce loops without cmp instruction in GCC
  29. What is the effect of second argument in _builtin_prefetch()?
  30. Efficient 4×4 matrix multiplication (C vs assembly)

Leave a Comment