Stack Overflows

Control Flow Lives on the Stack

Every call pushes a return address onto the stack. Every ret pops it back into RIP. That address sits a few bytes away from any local buffer in the same frame — so if you can write past the buffer, you can write the address the CPU jumps to next. That’s the entire premise of a stack overflow exploit.

Anatomy of a Frame

The memory layout lesson covers the stack in general; here’s the frame that matters for exploitation, addresses increasing downward on the page but the stack itself growing toward lower addresses:

higher addresses
+-------------------+
| return address     |  <- overwritten last, most valuable target
+-------------------+
| saved RBP          |  <- overwritten first once buffer overflows
+-------------------+
| local buffer       |  <- attacker-controlled write starts here
| ...                |
+-------------------+
lower addresses          <- RSP points near here

A write into buffer that isn’t bounds-checked walks upward through this layout: fill the buffer, then the saved RBP slot, then the return address itself.

A Vulnerable Function, Disassembled

This is the shape lab-19 hands you — a function that copies caller-controlled input into a fixed-size stack buffer with no length check:

push rbp
mov  rbp, rsp
sub  rsp, 0x30          ; reserve 48 bytes for locals
lea  rax, [rbp-0x30]    ; rax = &buffer (start of the 48-byte region)
mov  rdi, rax           ; 1st arg (dest) = buffer
mov  rsi, [rbp+0x10]    ; 2nd arg (src)  = input pointer
call strcpy             ; strcpy(buffer, input) — no bounds check
leave
ret

strcpy copies bytes until it hits a null terminator — it has no concept of how big buffer actually is. If input is longer than 48 bytes, the copy keeps going straight through the saved RBP and into the return address.

Computing the Offset

sub rsp, 0x30 tells you the compiler reserved 48 decimal bytes for the frame’s locals, and lea rax, [rbp-0x30] confirms the buffer starts at the very bottom of that reservation — right at rbp-0x30. To reach the return address from the start of the buffer you cross two things:

1. The 48-byte buffer itself (0x30)
2. The 8-byte saved RBP slot sitting between the buffer and the return address

48 + 8 = 56. Byte 57 of your input is the first byte of the return address. This is the calculation you always do first: find the frame size from sub rsp, N, find the buffer’s offset from lea, add 8 for saved RBP.

“Smashing” the Stack

The term comes from overwriting the return address with a value you chose — often the address of shellcode you also placed in the input, or later, the address of existing code (see Return-Oriented Programming). When leave; ret runs, RIP loads your value and execution jumps wherever you pointed it. The function never “returns” in the normal sense — it returns into a fiction you constructed.

Red flags to scan for in disassembly: strcpy, gets, sprintf, strcat, or any raw memcpy/read into a stack buffer where the length argument is attacker-controlled or missing entirely. Bounded variants (strncpy, snprintf) aren’t automatically safe either — a wrong size argument reintroduces the same bug.

Interactive: Stack Visualizer

Step through the frame construction and the strcpy call. Watch the buffer fill and cross into the saved RBP and return-address slots, then confirm the 56-byte offset yourself before touching the lab.

Takeaway

A stack overflow isn’t magic — it’s a copy with no brakes, walking uphill until it hits the one value the CPU trusts unconditionally.

Lab 19: Stack Smash Analysis → Stack Smash Analysis