Why is there no "sub rsp" instruction in this function prologue and why are function parameters stored at negative rbp offsets?

Solution 1:

The System V ABI for x86-64 specifies a red zone of 128 bytes below %rsp. These 128 bytes belong to the function as long as it doesn't call any other function (it is a leaf function).

Signal handlers (and functions called by a debugger) need to respect the red zone, since they are effectively involuntary function calls.
All of the local variables of your test_function, which is a leaf function, fit into the red zone, thus no adjustment of %rsp is needed. (Also, the function has no visible side-effects and would be optimized out on any reasonable optimization setting).

You can compile with -mno-red-zone to stop the compiler from using space below the stack pointer. Kernel code has to do this because hardware interrupts don't implement a red-zone.

Solution 2:

But my code seems to store them in a negative offset, just like local variables

The first x86_64 arguments are passed on registers, not on the stack. So when rbp is set to rsp, they are not on the stack, and cannot be on a positive offset.

They are being pushed only to:

  • save register state for a second function call.

    In this case, this is not required since it is a leaf function.

  • make register allocation easier.

    But an optimized allocator could do a better job without memory spill here.

The situation would be different if you had:

  • x86_64 function with lots of arguments. Those that don't fit on registers go on the stack.
  • IA-32, where every argument goes on the stack.

the lack of a "sub rsp $n_bytes" for "saving some memory for local variables".

The missing sub rsp on red zone of leaf function part of the question had already been asked at: Why does the x86-64 GCC function prologue allocate less stack than the local variables?