Stack exploitation

The 30-second version. Module 01 chapter 04 left the frame to two trusting instructions: leave pops the saved frame pointer, ret pops the return address into the instruction pointer — and nothing checks where a write lands. A copy starting in a 64-byte local buffer can walk straight up the frame and replace the return address: at return, the attacker's bytes are the program counter. This chapter builds that attack from the frame upward, then the defense — the stack canary, glibc's NUL-terminated secret at fs:0x28 — and its three roads around: leaking it, overwriting its home in TLS, inheriting it unchanged through fork().

One missing bound check: the frame read as an attack surface

Start from the machinery module 01 chapter 04 derived: allocation is sub rsp, N, deallocation is leave; ret, and the design has zero bookkeeping — no headers, no bounds — because the compiler proves every access stays in the frame; the proof lives only in the compiler's head. At runtime the frame is bytes: data (locals) and control data (saved RBP, return address) interleaved in one region, separated by nothing but distance. The attacker needs one copy whose length comes from anywhere but the destination.

The stack grows downward, so a local buffer sits at the low end of the frame with the control data above it. A linear upward copy — exactly what strcpy and friends do — crosses, in order: the remaining locals, the canary (if built with one), the saved frame pointer, the return address. The epilogue then pops those bytes into rbp and rip. That is the whole mechanism of the classic stack smash (Aleph One, "Smashing the Stack for Fun and Profit," Phrack 49, 1996) — the first diagram every defender must draw:

      higher addresses (an upward copy climbs TOWARD these)
+----------------------------------------+  caller's frame / arg area
|  RETURN ADDRESS (saved RIP)            |  rbp+0x08  <- ret pops THIS into rip
+----------------------------------------+  rbp       <- leave pops this into rbp
|  saved RBP (caller's frame pointer)    |             <- corrupt = poisoned backtrace
+----------------------------------------+  rbp-0x08
|  STACK CANARY (8 B, low byte = 00)     |             <- must be reproduced exactly
+----------------------------------------+  rbp-0x10
|  alignment slack (0-8 B, builds vary)  |             <- absorbs stray bytes
+----------------------------------------+
|  char buf[64]   <- "AAAAAAAAAA..."     |             <- the copy starts HERE
+----------------------------------------+  rbp-0x50   bottom of frame proper
      lower addresses (rsp, then deeper frames)

The minimal vulnerable program:

/* smash.c — deliberately vulnerable training target. Never ship this shape. */
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

void win(void) { puts("*** control flow hijacked: win() reached ***"); exit(0); }

void greet(void) {
    char buf[64];        /* destination: 64 bytes */
    read(0, buf, 256);   /* bug: 256 is a source-side constant, not buf's size */
    write(1, buf, 64);   /* echo only the legal part */
}

int main(void) { puts("feed me:"); greet(); puts("returned normally"); }

Input comes from read(), not strcpy(argv[1]), because argv is NUL-terminated strings: no 0x00 survives execve, and real 64-bit addresses are full of NULs (0x0000000000401156). Payloads travel by stdin, sockets, or files — the channels production bugs read from.

The arithmetic is pure chapter 04: the defenseless build puts buf at [rbp-0x40], so the saved return address sits at 64 + 8 = 72 bytes. Send 72 filler bytes plus the 8-byte little-endian address of win, and ret delivers you there — verified on this chapter's platform (gcc 14.2.0, glibc 2.41, Debian 13, x86-64):

$ python3 -c 'import sys; sys.stdout.buffer.write(
    b"A"*72 + (0x401156).to_bytes(8,"little"))' | ./smash
*** control flow hijacked: win() reached ***

What gcc actually emits now — and why the classic lab must opt out

The 1996 target no longer exists by accident; two defaults stand in the way.

The canary. With -fstack-protector-strong, gcc brackets the function with a prologue that stashes the secret and an epilogue that verifies it (gcc 14.2.0, -O0, trimmed):

push   rbp
mov    rbp,rsp
sub    rsp,0x50                 ; frame grew: 64 buf + 8 canary + 8 alignment
mov    rax,QWORD PTR fs:0x28    ; load the canary master from TLS
mov    QWORD PTR [rbp-0x8],rax  ; frame's copy, just below saved RBP
...
mov    rax,QWORD PTR [rbp-0x8]  ; epilogue: reload the copy
sub    rax,QWORD PTR fs:0x28    ; compare against the master
je     <ok>                     ; equal -> return normally
call   __stack_chk_fail@plt     ; not equal -> abort, loudly
<ok>:  leave
       ret

gcc 14 here emits sub + je; clang and older gcc emit the textbook xor rax, QWORD PTR fs:0x28 / jne __stack_chk_fail — same predicate, different spelling; recognize both.

PIE. The defenseless build has greet at a fixed 0x401173; the default build is position-independent and ASLR moves it every run, killing the "hardcode win" step — the attacker needs a leak first (chapter 06 quantifies the entropy). So the classic lab opts out with -fno-stack-protector -no-pie — and one surprise remains: readelf -lW shows GNU_STACK RW, not RWX. NX has been default for two decades; the lab recovers the canary-free, fixed-address layout, not 1996's executable stack — chapter 03 is why attackers stopped caring.

One distro trap, exposed by this chapter's own platform: defaults are per-distribution, not per-gcc. Debian 13's bare gcc 14 enables PIE but not the protector (Debian injects -fstack-protector-strong via dpkg-buildflags for packages, not ad-hoc builds); Ubuntu's gcc: strong since 16.10, PIE since 17.10. Ask, don't assume: gcc -Q --help=common | grep stack-protector.

Deriving the canary: a secret in the path, a NUL at the front

Build the defense from the attack's shape. The overflow is linear and upward, so it must physically cross every byte between the buffer and the return address. Place an 8-byte secret in that path and refuse to return if the frame's copy no longer matches the master. Three requirements fall out; glibc answers each:

1. Unpredictable. A guessable secret just gets replayed in the overflow. The value is random per exec: the kernel hands each new process 16 random bytes via the ELF auxiliary vector (AT_RANDOM); glibc derives the canary's 8 bytes from them (glibc 2.41, x86-64).
2. Master off the stack. A secret in the frame is a copy the attacker reads on the way past. The master lives in TLS — fs:0x28, the stack_guard field of glibc's tcbhead_t — behind a segment register with a per-thread base.
3. The low byte is 0x00 — the terminator canary. It earns its keep twice: - It stops string-shaped leaks. The bug pair's other half is a read running past the buffer — printf("%s", buf) on unterminated data, strlen, anything that walks until a zero. Climbing into the canary, such a read stops dead at its first byte: zero of the seven secret bytes leak. Without the terminator, the same bug prints all seven and hands over the canary. - It absorbs the classic off-by-one. strcpy writes one terminating NUL past the copied bytes. A stray NUL landing on the canary's low byte changes 0x00 into 0x00 — a no-op. The most common byte of stack corruption in C is swallowed by design.

What the canary does not do matters as much: it is a detector, not a corrector — when __stack_chk_fail fires, every local is already overwritten; the process dies to stop control-flow capture, not to repair data. It guards the return path only — the arg area above the return address is outside its promise — and does nothing about leaks of stack contents (module 01 chapter 04's dead-frame bytes).

The canary's home: TLS, and the three roads around it

A mitigation is a claim about what an attacker cannot do; the canary's claim has three roads around it, in rising order of effort.

Road 1: leak it. Any read primitive reaching the frame's canary slot yields the seven secret bytes, replayed in the overflow. The terminator stops string-shaped reads, not reads in general — format-string bugs, length-confused echoes, partial-overwrite-then-read pairs all work (module 02 chapter 01's primitive ladder: leak first, corrupt second). One good leak and the canary is a toll already paid.

Road 2: overwrite the master. The epilogue compares the frame's copy against fs:0x28. For a pthread_create'd thread, glibc maps the TCB and its TLS block at the top of that thread's stack region — the guard page is at the bottom (growth end), and nothing patrols the top border. So a bug that accepts very long input — a decompressor filling a stack buffer in a loop, a path-builder appending without bounds — can write up through its own frame, the caller frames, out the top of the stack, and into tcbhead_t, replacing stack_guard with attacker bytes. Corrupted copy equals corrupted master; the check passes. The distance is on the order of the remaining stack — megabytes against glibc's 8 MB default (glibc 2.41). The main thread's TCB is an ld.so mapping elsewhere — hence a threaded-daemon road; check with p $fs_base.

Road 3: never learn it, never change it. fork() copies the parent's address space, TLS included: the child's canary is byte-for-byte the parent's. A daemon forking a child per connection makes the arithmetic cruel: a failing child dies of SIGABRT — one dropped connection — while the canary never changes; only exec consults AT_RANDOM again, and the parent never execs. The attacker brute-forces byte by byte: the low byte is known (0x00), seven remain, at most 255 wrong values each — worst case 7 × 255 = 1,785 crashes (expected ~893) against a daemon that will fork 100,000 connections. This is the "crash resets the canary" trap: the belief that a crashing child re-randomized the secret. It does not — fork inherits, exec re-rolls. The historical defense is the inetd shape: fork plus exec per connection, buying a fresh AT_RANDOM each time.

The pattern across all three roads: the canary defends against linear, secret-blind overflows, and its guarantee ends where some other mechanism — leak discipline, mapping layout, process architecture — must take over.

The off-by-one: one NUL byte against one frame pointer

The classic smash needs enough input to reach the return address. A subtler weapon needs exactly one byte — and turns the frame pointer itself into a delayed detonator.

On 32-bit x86, pack a buffer against the top of the frame and let a copy run one byte long — for (i = 0; i <= 256; i++) buf[i] = src[i]; with char buf[256], or a strcpy one byte too long. The 257th byte (the terminating NUL, for strcpy) lands on the low byte of saved EBP:

        32-bit frame, no canary (the 1990s layout)
+-----------------------+  ebp+0x04   return address  (UNTOUCHED)
+-----------------------+  ebp         saved EBP: 0xffffc8f0
| low byte was f0       |              one NUL writes 0x00 -> 0xffffc800
+-----------------------+  ebp-0x100   buf[256] ... attacker-planted data below

The function's return address is untouched, so it returns normally — the crash is not here. The poison detonates in the caller's epilogue, one frame later: leave does mov esp, ebp; pop ebp, so the caller's ebp becomes the poisoned 0xffffc800, shifted down by up to 252 bytes (i386 stacks are 4-byte aligned) — into the attacker's dead-but-intact buffer bytes. Every [ebp-...] access in the caller now reads attacker data, and when the caller runs leave; ret, its ret pops from the fake frame: rip control, one frame later, from a single NUL.

The technique entered the literature around 1999 (klog's Phrack write-up on overwriting EBP) as "poison NULL byte" — a name that survives mostly in heap lore, where the same one-NUL idea attacks glibc chunk metadata (module 02 chapter 04).

Why did x86-64 kill it? Derive, don't memorize:

• The canary stands in the byte path. Between the buffer's end and saved RBP lie 8 canary bytes. The stray NUL lands on the canary's low byte — 0x00 → 0x00, absorbed silently. Reaching saved RBP's low byte at all means crossing all eight canary bytes, and the process aborts.
• 16-byte alignment. The System V AMD64 ABI forces rsp 16-byte aligned at call boundaries, so saved RBP's low nibble is already zero and compilers insert slack — the poison's ≤ 240-byte shift rarely lands on controlled bytes.
• Frame pointers became optional. This chapter's own -O2 build ends with add rsp,0x50; pop rbx; ret — no leave, no saved RBP to poison. (Policy, not law: Ubuntu 24.04 re-enabled frame pointers for profiling, so the surface breathes with compiler flags.)

For a defender the lesson is sharper than "the bug is dead": on a modern build the off-by-one NUL is absorbed and invisible — no crash, no canary trip. Silence is not safety.

What insiders know

• The terminator earns its low byte twice: it stops string-primitive leaks (%s, strlen) at the canary's first byte, and makes the one-NUL off-by-one a no-op (0x00 over 0x00). glibc 2.41 derives the 8 bytes from the kernel's 16-byte AT_RANDOM blob, low byte forced to 0x00.
• The master lives at fs:0x28, in tcbhead_t.stack_guard — the first 0x30 bytes of glibc's struct pthread. A pthread's TCB is mapped at the top of that thread's stack region (guard page at the growth end only), so the "long overflow into your own canary" road is a threaded-daemon road with megabytes of runway. The main thread's TCB is an ld.so mapping elsewhere — check with p $fs_base.
• fork() inherits the canary verbatim; only exec re-rolls it. The "crash resets the canary" assumption is the trap: brute-forcing a fork-per-connection daemon costs ≤ 255 crashes per byte × 7 unknown bytes = ≤ 1,785 dead children. The inetd-style fork+exec per connection is the historical answer.
• The gcc flag genealogy: -fstack-protector (gcc 4.1, 2006; only frames with arrays > 8 bytes or alloca), -fstack-protector-all (every frame), -fstack-protector-strong (gcc 4.9, 2014; any array or address-taken local — catches char buf[64]), -fstack-protector-explicit (gcc 6, attribute-marked only). Ubuntu gcc = strong since 16.10 + PIE since 17.10; Debian's bare gcc = PIE only, protector injected via dpkg-buildflags.
• Off-by-one lore: the saved-EBP overwrite dates to klog's 1999 Phrack write-up — its window was a buffer's top byte abutting saved EBP's low byte on i386. The name "poison NULL byte" now lives mostly in heap lore (glibc size-field off-by-ones, module 02 chapter 04); on x86-64 the canary occupies the stack variant's byte path.
• Triage signature: canary failure is SIGABRT (signal 6; shell exit 134) with *** stack smashing detected ***: terminated on stderr (glibc 2.41 wording; older glibc also printed the program path). A SIGSEGV with rip full of 0x41s means no canary was in that frame — a different rung missing.
• Modern builds find this bug at compile/CI time: gcc 14 flags the lab's read(0, buf, 256) with -Wstringop-overflow even at -O0; -D_FORTIFY_SOURCE=2 (needs -O1+) swaps in __memcpy_chk-style calls checked against compile-time object sizes; ASan's redzones catch the first byte past the buffer, including the off-by-one the canary absorbs.

See it yourself

The whole chapter on one laptop (gcc 14.2.0, glibc 2.41, Debian 13, x86-64; addresses vary by compiler and distro — the instruction sequences are the durable part):

$ gcc -g -O0 -fno-stack-protector -no-pie smash.c -o smash        # the 1996 target
$ gcc -g -O0 -fstack-protector-strong -pie smash.c -o smash_prot  # the 2024 defense
smash.c:14:5: warning: 'read' writing 256 bytes into a region of size 64
   overflows the destination [-Wstringop-overflow=]

First: the compiler already told you — treat -Wstringop-overflow as the review comment it is. (On Ubuntu those are the defaults.)

Part 1 — the classic, no defenses:

$ gdb -q ./smash
(gdb) disas greet
   sub    rsp,0x40            ; frame = 0x40: buf is the only local
   lea    rax,[rbp-0x40]      ; &buf -- the attack starts here
   ...
   leave                      ; mov rsp,rbp ; pop rbp
   ret                        ; pops saved RIP into rip
(gdb) p win
$1 = {void (void)} 0x401156 <win>
(gdb) run < <(python3 -c 'import sys;
      sys.stdout.buffer.write(b"A"*72 + (0x401156).to_bytes(8,"little"))')
feed me:
*** control flow hijacked: win() reached ***

72 filler bytes reach the saved RIP exactly (64 buffer + 8 saved RBP); 80 overwrite it with 0x4141414141414141 — Program received signal SIGSEGV, exit 139. The distance between "echoed my input" and "executed my input": 8 bytes.

Part 2 — the defense, same function:

$ gdb -q ./smash_prot
(gdb) disas greet
   sub    rsp,0x50                 ; frame grew: buf + canary + alignment
   mov    rax,QWORD PTR fs:0x28    ; <-- load canary master from TLS
   mov    QWORD PTR [rbp-0x8],rax  ; <-- frame's copy, below saved RBP
   ...
   mov    rax,QWORD PTR [rbp-0x8]  ; <-- epilogue check
   sub    rax,QWORD PTR fs:0x28    ; (clang / older gcc: xor + jne)
   je     0x11ee <greet+0x58>
   call   0x1050 <__stack_chk_fail@plt>
(gdb) run < <(python3 -c 'import sys; sys.stdout.buffer.write(b"A"*80)')
Program received signal SIGABRT, Aborted.

Outside gdb the same run prints *** stack smashing detected ***: terminated to stderr, exit 134. Eighty bytes cross the canary at offset 72 — detector tripped, control flow not taken.

Part 3 — look at the secret itself:

(gdb) p $fs_base
$1 = 0x7ffff7c8e740
(gdb) x/8bx $fs_base+0x28
0x7ffff7c8e768: 0x00  0x3a  0x9c  0x12  0x4f  0x6b  0xd2  0x77

Your value differs every run (AT_RANDOM-derived), but the first byte is always 0x00 — the terminator, in plain view. Run again and the canary changes; catch fork in gdb, re-read in the child, and the copy is identical.

How to use this knowledge

• Crash triage and CVE reading. SIGABRT + "stack smashing detected" means a canary did its job; SIGSEGV with garbage rip means no canary was in the path — enough to name the missing rung in any write-up.
• Code review shapes. Any copy whose bound is a source-side constant (the lab's read(0, buf, 256)), any <= in a fill loop, any stack buffer fed by unbounded string calls — block those diffs.
• Build audits. Never assume distro or embedded-toolchain defaults: check gcc -Q --help=common; for shipped binaries check readelf for Type: DYN (PIE), GNU_STACK RW, and __stack_chk_fail imports.
• Process architecture. A fork-per-connection daemon ships a brute-forceable canary (≤ 1,785 crashes), and per-thread stacks put the master above the stack it guards — design facts for any long-lived service's threat model.
• CI that finds what the canary absorbs. Test builds with -fsanitize=address, release builds with -D_FORTIFY_SOURCE=2: ASan flags the first byte past the buffer — including the off-by-one production builds swallow.

Connections

• Module 01 chapter 04 — the frame machinery under attack: leave; ret, and the zero-bookkeeping design that leaves buffers unfenced.
• Module 01 chapter 07 — the assembly fluency this chapter assumes: reading a prologue fast enough to find &buf and the frame size in seconds.
• Module 02 chapter 03 — where saved-RIP control goes under NX: the ret2libc/ROP answer to "you control rip, but the stack won't execute."
• Module 02 chapter 06 — the canary as rung one of the mitigation ladder, with ASLR entropy, RELRO, and the shadow stacks that would retire canaries.
• Module 03 chapter 02 — the overflow and off-by-one as bug-class shapes in real diffs.

Quiz

1. Concept. Derive the canary's position: why must the secret sit between the locals and the saved RBP rather than above the return address? What property of the classic overflow makes that the only spot that pays?
2. Concept. glibc's canary has its low byte forced to 0x00. Name the two attack families that single byte blunts, and the mechanism of each.
3. Predict the number. A no-protector build has char buf[64] at [rbp-0x40]; a protected build has it at [rbp-0x50], canary at [rbp-0x8]. For each build: bytes from buf's start to (a) the first canary byte, (b) the return address.
4. Predict the behavior. A fork-per-connection daemon has a triggerable overflow; the attacker crashes children while brute-forcing the canary. Does the 1,000th child see the same canary as the 1st, and what is the worst-case crash count?
5. Predict the output. In gdb, x/8bx $fs_base+0x28 prints 00 5f 3a c1 77 2b 90 e4. What is the canary's 64-bit value, and how many of its bytes can a printf("%s", buf) running past the buffer leak?
6. Bug diagnosis. A pen-test report: "attacker triggered stack-smashing aborts in three children, then gave up; the canary appears unbreakable." What is wrong with that conclusion, and which process change closes the hole?
7. Bug diagnosis. A 32-bit program has for (i = 0; i <= 256; i++) buf[i] = src[i]; with char buf[256]. It crashes not in this function but in its caller's epilogue, one frame later — explain. Why does the same source on x86-64 with -fstack-protector-strong abort inside the function if the copy is 256+8 bytes, yet do nothing at all at exactly 257?
8. Cross-chapter. You control the saved RIP of the lab's -no-pie smash. Name the two module 01 chapter 04 instructions that made this possible. Via chapter 03: why does pointing RIP at stack bytes you wrote fail on default builds, and what do attackers do instead?