Inline assembly that clobbers the red zone

by

From your original question I did not realize gcc limited red-zone use to leaf functions. I don’t think that’s required by the x86_64 ABI, but it is a reasonable simplifying assumption for a compiler. In that case you only need to make the function calling your assembly routine a non-leaf for purposes of compilation:

int global;

was_leaf()
{
    if (global) other();
}

GCC can’t tell if global will be true, so it can’t optimize away the call to other() so was_leaf() is not a leaf function anymore. I compiled this (with more code that triggered stack usage) and observed that as a leaf it did not move %rsp and with the modification shown it did.

I also tried simply allocating more than 128 bytes (just char buf[150]) in a leaf but I was shocked to see it only did a partial subtraction:

    pushq   %rbp
    movq    %rsp, %rbp
    subq    $40, %rsp
    movb    $7, -155(%rbp)

If I put the leaf-defeating code back in that becomes subq $160, %rsp

More Related Contents:

  1. Assembly code fsqrt and fmul instructions
  2. Using base pointer register in C++ inline asm
  3. Can I use Intel syntax of x86 assembly with GCC?
  4. How can I indicate that the memory *pointed* to by an inline ASM argument may be used?
  5. What is the difference between ‘asm’, ‘__asm’ and ‘__asm__’?
  6. Order of local variable allocation on the stack
  7. Stack allocation, padding, and alignment
  8. Why does integer division by -1 (negative one) result in FPE?
  9. When to use earlyclobber constraint in extended GCC inline assembly?
  10. Is it possible to tell the branch predictor how likely it is to follow the branch?
  11. What does it mean to align the stack?
  12. Syscall implementation of exit()
  13. Looping over arrays with inline assembly
  14. Drawing a character in VGA memory with GNU C inline assembly
  15. Efficient integer compare function
  16. How to access C variable for inline assembly manipulation?
  17. Reading a register value into a C variable [duplicate]
  18. Why am I able to perform floating point operations inside a Linux kernel module?
  19. In GNU C inline asm, what are the size-override modifiers for xmm/ymm/zmm for a single operand?
  20. multi-word addition using the carry flag
  21. Why we need Clobbered registers list in Inline Assembly?
  22. calling assembly function from c
  23. The difference between asm, asm volatile and clobbering memory
  24. Getting started with Intel x86 SSE SIMD instructions
  25. Does __asm{}; return the value of eax?
  26. Compare 16 byte strings with SSE
  27. Why is gcc allowed to speculatively load from a struct?
  28. How to tell GCC to generate 16-bit code for real mode
  29. Using condition flags as GNU C inline asm outputs
  30. What is the effect of second argument in _builtin_prefetch()?

Leave a Comment