LAB-10Medium

Anti-Debug Bypass

Identify an anti-debugging check and determine how to bypass it.

15 min5 prerequisites5 tools

Disassembly

00:  mov rax, 1
01:  test rax, rax
02:  jne detected
03:  mov rcx, 0
04:  jmp end
detected:
05:  mov rcx, 1
end:
06:  ret 

Hints

Hint 1

test rax, rax sets ZF=1 only when RAX is zero.

Hint 2

A nonzero RAX simulates a debugger being detected.

Hint 3

Bypass = make the check believe no debugger is present.

Submit Findings

Prerequisites

  • d4-anti-debugging
  • a3-control-flow
  • x64dbg / GDB basics
  • Common anti-debug APIs (IsDebuggerPresent, CheckRemoteDebuggerPresent, NtQueryInformationProcess)
  • Patch techniques (NOP, conditional jump inversion, register spoofing)

Methodology

Investigation Methodology

1. Recognize the Pattern

  • mov rax, 1 / mov rax, 0 → setting a boolean result
  • test rax, rax / cmp rax, 0 → checking the result
  • jne / je / jz / jnz → conditional branch based on check

This is the canonical anti-debug check pattern: call API → test result → branch.

2. Identify the Simulated API

  • IsDebuggerPresent returns TRUE (nonzero) if debugger attached
  • CheckRemoteDebuggerPresent returns TRUE via output parameter
  • NtQueryInformationProcess with ProcessDebugPort / ProcessDebugFlags
  • PEB BeingDebugged flag (offset 0x2 in PEB)

In this lab: RAX=1 hardcoded → simulates IsDebuggerPresent returning TRUE.

3. Trace Execution Paths

Debugger Present (RAX=1):

  • test rax, rax → ZF=0
  • jne detected → taken → RCX=1 → return 1 (detected)

No Debugger (RAX=0):

  • test rax, rax → ZF=1
  • jne detected → not taken → fall through
  • mov rcx, 0 → RCX=0
  • jmp end → skip detected branch
  • Return 0 (not detected)

4. Bypass Strategies

Method How Pros Cons
Register Spoof Set RAX=0 before test Clean, no binary modification Requires debugger attach
NOP the Jump Patch jne → nop (2 bytes: 0F 85 → 90 90) Permanent, no debugger needed Modifies binary
Invert Jump Patch jne → je (0F 85 → 0F 84) Permanent, flips logic Modifies binary
Patch API Call If real API: hook/redirect return value Works on real binaries More complex
PEB Patch Set BeingDebugged = 0 in PEB Affects all IsDebuggerPresent calls Process-wide, detectable

5. Verify Bypass

  • Re-run: confirm hidden path executes (RCX=0 returned)
  • Check for secondary checks (multiple anti-debug layers)
  • Ensure functionality preserved (some checks are integrity checks)

Common Pitfalls

  • Patching the wrong instruction — verify address in debugger
  • Forgetting that `test rax, rax` is 2 bytes (`48 85 C0`), `jne rel32` is 6 bytes (`0F 85 xx xx xx xx`)
  • Missing secondary/tertiary anti-debug checks
  • Breaking functionality — some 'checks' are actually used for logic
  • Not handling timing checks (RDTSC, GetTickCount) — requires different bypass

Tools

  • x64dbg / x32dbg (Windows)
  • GDB / GEF / pwndbg (Linux)
  • Ghidra (decompiler shows high-level logic)
  • Binary Ninja (control flow graph)
  • ScyllaHide / TitanHide (anti-anti-debug plugins)

References

  • Lesson D4: Anti-Debugging
  • Lesson A3: Control Flow — conditional jumps
  • Peter Ferrie: 'Anti-Unpacker Tricks' (reference)
  • x64dbg docs: Patching, Breakpoints
  • PEB Structure: BeingDebugged flag

Extensions & Challenges

  • Implement a ScyllaHide plugin for a custom anti-debug technique
  • Analyze a packed sample (UPX, Themida) — identify all anti-debug checks
  • Research: Hardware breakpoint detection (DR0-DR7)
  • Research: Timing-based detection (RDTSC, QueryPerformanceCounter)
  • Build a script to auto-patch common anti-debug patterns