StuBS
Loading...
Searching...
No Matches
x86_64 System V ABI: Register and stack layout

*Reminder: We assume here that the NASM assembler is used. It expects commands in the format: OP target, source.

For thread switching in software, the state of the processor must be saved in order to restore it later and continue the execution of the program at this point. What belongs to the state of the processor is highly dependent on the architecture and may need to be researched in detail in the ABI documentation. In this course we use the x86_64 architecture and only a small part of it, i.e. we will only concentrate on the essential registers.

An Application Binary Interface (ABI) in general is a calling convention that is defined by the architecture as well as by the compiler and programming language. It regulates how parameters are passed to functions and how structures are stored in memory. The convention serves as an interface both between separately compiled code (possibly different compilers) and between manually written assembler functions and generated code. An ABI is particularly important when using shared libraries, as these are pre-compiled and can therefore only be used with a specific ABI. The object files used in our case also require an ABI, as the linker also works with precompiled objects (unlike shared libraries, which are only loaded into the address space at load time). The C ABI has become widely accepted (mainly for historical reasons) and can be accessed from many other compiled languages. It has thus become the standard interface between different programming languages. We also want to use it in the following to mediate between assembler and C++ code.

Registers

The x86_64 architecture has 16 general purpose registers, a flags register (rflags) and the instruction pointer (rip). The general purpose registers are described below. They originally had a fixed purpose and therefore have corresponding names. However, most registers can now be used for virtually all instructions:

  • rax (accumulator for some arithmetic instructions)
  • rbx (formerly base)
  • rcx (counter for loops)
  • rdx (data register)
  • rsi (source index for string operations)
  • rdi (destination index for string operations)
  • rbp (base pointer)
  • rsp (stack pointer)
  • r8 through r15 (additional registers available in 64-bit mode)

The x86 architecture began as a 16-bit microprocessor family and evolved through 32-bit to the current 64-bit architecture. In 64-bit mode, the registers were extended to 64 bits, with names prefixed with 'r' instead of 'e'. The 64-bit registers can still be accessed in smaller parts:

  • The lower 32 bits (e.g., eax part of rax)
  • The lower 16 bits (e.g., ax part of rax)
  • The lowest 8 bits (e.g., al part of rax)
  • The second-lowest 8 bits (e.g., ah part of rax, only for the original eight registers)

Additionally, the new registers r8 through r15 can be accessed as:

  • 64-bit registers: r8 through r15
  • 32-bit portions: r8d through r15d
  • 16-bit portions: r8w through r15w
  • 8-bit portions: r8b through r15b

rsi and rdi have a special meaning for string operations. rsp is important for the stack because the stack pointer points to the last entry of the stack.

Volatile vs. non-volatile registers

In programs, calls to sub-functions are a common mechanism. In a call, a distinction is made between the calling function (caller) and the called function (callee). Both caller and callee need registers to do their work. The caller, however, often expects the same value in its registers before and after the call. This means that the call should be transparent for the caller with regard to the registers (with exceptions for the registers that return values). Ideally, we want two completely separate sets of registers for each function call.

Unfortunately, registers are limited on the hardware side, so the usual way is to save registers to the stack before the call and restore them after the call. This saving can happen on the side of the caller, i.e. before the call instruction (caller-saved, volatile in relation to the caller) or on the side of the callee (callee-saved, non-volatile in relation to the caller). The callee would first save the register values to the stack and restore the registers at the end of the function before the ret instruction.

In the x86_64 System V ABI, the registers are divided into two categories:

  • Volatile (caller-saved) registers: rax, rcx, rdx, rsi, rdi, r8, r9, r10, r11, and rflags
  • Non-volatile (callee-saved) registers: rbx, rbp, rsp, r12, r13, r14, and r15

This means if the caller wants to preserve values in volatile registers across a function call, it must save them itself. For non-volatile registers, the called function is responsible for preserving their values if it needs to use them.

Passing parameters via registers

Unlike the 32-bit x86 ABI, the x86_64 System V ABI passes the first six integer or pointer arguments in registers rather than on the stack. The registers are used in the following order:

  1. rdi - First argument
  2. rsi - Second argument
  3. rdx - Third argument
  4. rcx - Fourth argument
  5. r8 - Fifth argument
  6. r9 - Sixth argument

Any additional arguments are passed on the stack.

Floating-point arguments are passed in the XMM registers (xmm0 through xmm7), but we will focus on integer arguments in this course.

Let's take the following function as an example, which expects two parameters:

int foobar(int a, int b) {
// placeholder
return a+b;
}

A calling function would pass the parameters in registers:

caller:
// ...
// Assumption: a is stored in some register (rax)
// b is stored in some register (rbx)
mov rdi, rax ; First parameter (a) goes in rdi
mov rsi, rbx ; Second parameter (b) goes in rsi
call foobar
// ...

With the first instruction of foobar, the parameters are already in the respective registers. The stack only contains the return address:

0xff
...
return address<- rsp
...
0x00

The function can directly use these registers to access its parameters:

foobar:
// Parameters are already in rdi (a) and rsi (b)
mov rax, rdi ; Move first parameter to rax
add rax, rsi ; Add second parameter to it
// ...

An important requirement of the System V ABI is that the stack pointer (rsp) must be aligned to a 16-byte boundary before making a call. This is primarily to support SSE instructions which may require aligned memory access.

Return values

In the x86_64 System V ABI, integer return values up to 64 bits are passed via the rax register. For larger return values or when returning structures, additional registers may be used (rdx for the upper 64 bits of a 128-bit value).

For example, the return inside the foobar function could look like this:

foobar:
// ...
mov rax, rdi ; Move a to rax
add rax, rsi ; Add b to rax
ret ; Return value is now in rax

The instructions call and ret have an influence on the control flow. In addition to the actual jump to the function, call places the return address on the stack. ret expects the return address as the top value on the stack, removes it from the stack and jumps to the specified position. The instruction following the call is executed next.