Skill: Concurrency Exploitation

Supplementary Files:

• payloads.md -- Race condition payloads: TOCTOU file system races, signal handler exploitation, thread synchronization bypasses, timing measurement, race detection tools, exploitation primitives, debugging commands, CyberGym templates
• test-cases.md -- 6 structured test cases covering symlink TOCTOU, signal handler race, pthread mutex bypass, fork server race, double-checked locking, ABA problem

Summary

Concurrency Exploitation skill domain covering exploitation operations.

Tools: gdb, pwndbg, ThreadSanitizer, helgrind, racer2, stress-ng, inotify-tools, strace, ltrace, perf, time

Domain: exploitation

MITRE ATT&CK: TA0003-Execution

Description

Concurrency exploitation targets race condition vulnerabilities where the outcome of an operation depends on the timing or ordering of uncontrollable events. These vulnerabilities arise when multiple threads, processes, or signal handlers access shared resources without proper synchronization, creating windows where attackers can manipulate state between check-and-use operations.

Race conditions are particularly dangerous because they are non-deterministic—successful exploitation depends on winning narrow timing windows, often measured in nanoseconds. However, techniques like CPU pinning, process priority manipulation, and parallel attack scripts can amplify race windows to achieve reliable exploitation.

Key Vulnerability Classes:

• TOCTOU (Time-of-Check-Time-of-Use): File system races where an attacker swaps a resource between validation and usage (e.g., symlink races in setuid binaries)
• Signal Handler Races: Non-reentrant code in signal handlers, or race windows opened by signal delivery during critical sections (CVE-2024-6387 regreSSHion)
• Thread Synchronization Bugs: Missing mutex locks, incorrect lock ordering, or atomicity violations that allow concurrent access to shared memory
• Initialization Races: Double-checked locking bugs, fork server race windows, or races during startup/teardown sequences
• Lock-Free Algorithm Bugs: ABA problem in compare-and-swap operations, memory ordering violations

---

Use Cases

1. TOCTOU File System Exploitation -- Win race windows between access() checks and open() calls in setuid binaries to gain privilege escalation
2. Signal Handler Race Exploitation -- Exploit race conditions in non-reentrant signal handlers (e.g., OpenSSH CVE-2024-6387 regreSSHion)
3. Thread Synchronization Bypass -- Exploit missing or incorrect mutex locks to achieve UAF or double-free conditions
4. Fork Server Race Windows -- Target race conditions in process forking/cloning logic used by servers and daemons
5. Initialization Race Exploitation -- Exploit double-checked locking bugs or races during cryptographic initialization
6. Timing Side-Channel Analysis -- Measure execution timing to infer secret values from timing variations in concurrent operations
7. Lock-Free Data Structure Exploitation -- Exploit ABA problem or memory ordering bugs in lock-free queues and stacks

---

Core Tools

Tool	Purpose	Command Example
gdb + pwndbg	Debug multi-threaded programs, set catchpoints on thread creation, inspect lock state	gdb -ex "catch syscall clone" -ex "run" ./vulnerable
ThreadSanitizer	Detect data races at runtime via compiler instrumentation (gcc/clang)	gcc -fsanitize=thread -g race.c -o race && ./race
helgrind	Valgrind tool for detecting pthread synchronization errors and lock order violations	valgrind --tool=helgrind ./vulnerable
racer2	Static race detection tool analyzing source code for potential data races	racer2 --analyze race.c
stress-ng	CPU stress utility to amplify race windows by increasing scheduling chaos	stress-ng --cpu 8 --timeout 60s
inotify-tools	Monitor file system events in real-time to detect TOCTOU race attempts	inotifywait -m /tmp/ -e open,close,delete
strace	Trace system calls to identify TOCTOU sequences (access→open) and signal delivery timing	strace -f -e trace=open,access,signal ./vulnerable
ltrace	Trace library calls to identify pthread mutex operations and lock ordering	ltrace -e pthread_mutex_lock,pthread_mutex_unlock ./vulnerable
perf	Linux profiling tool to measure precise timing and identify critical sections	perf stat -e cycles,instructions ./vulnerable
time	Nanosecond-precision timing measurement for race window analysis	time -p ./race_exploit
taskset	Pin processes to specific CPU cores to control scheduling and amplify races	taskset -c 0 ./attacker & taskset -c 1 ./victim

---

Methodology

Attack Chain

[1] Identify              [2] Analyze              [3] Amplify
  - Scan source for         - Trace syscalls           - Use stress-ng to
    pthread_create,           (strace) to find           increase CPU load
    signal(), access(),       check-use gaps           - Pin processes to
    fork() patterns         - Compile with               specific cores
  - Look for missing          ThreadSanitizer          - Run parallel attack
    mutex locks             - Analyze helgrind           instances (100+)
  - Identify TOCTOU           reports                  - Measure timing with
    sequences                    |                       nanosecond precision
       |                         v                           |
       v                                                     v
[4] Exploit               [5] Verify                [6] Escalate
  - Symlink race for        - Check for ASAN/TSAN      - Leverage race to
    privilege escalation      crashes (sanitizer         achieve UAF, double-
  - Signal handler race       reports)                   free, or arbitrary
    for RCE (regreSSHion)   - Verify memory             write
  - Thread interleaving       corruption via gdb       - Pivot to shell or
    for UAF/double-free     - Measure success rate      privilege escalation
  - Timing attack to leak     (must be >10% for        - Document race window
    secrets                   practical exploit)         and trigger method

---

Key Concepts

TOCTOU (Time-of-Check-Time-of-Use)

Race condition where a security check (e.g., access()) is performed on a resource, but the resource is changed before usage (e.g., open()). Classic example: setuid binary checks if user can read /tmp/file, attacker swaps it to symlink pointing to /etc/shadow, binary opens the shadow file with elevated privileges.

Signal Handler Reentrancy

Signal handlers must be async-signal-safe (no malloc, no non-reentrant functions). Bugs arise when handlers call non-reentrant functions like malloc(), printf(), or access shared state without atomic operations. CVE-2024-6387 (regreSSHion) exploited a race between SIGALRM handler and login logic in OpenSSH.

Happens-Before Relationship

Partial ordering of events in concurrent programs. If event A happens-before event B, then A's effects are visible to B. Race conditions occur when there is NO happens-before relationship between conflicting accesses to shared memory.

Memory Barrier / Fence

CPU instruction ensuring memory operations before the barrier complete before operations after it. Without barriers, CPU reordering can cause races even with "correct" source code. Compilers insert barriers via __sync_synchronize() or C11 atomics.

Double-Checked Locking Bug

Optimization where a check is performed outside a lock, then rechecked inside the lock. Broken without memory barriers because compiler/CPU can reorder writes, allowing partially-constructed objects to be visible. Classic Java/C++ bug pattern.

ABA Problem

Lock-free algorithm bug where a value changes from A→B→A during a compare-and-swap operation. The CAS succeeds because the value is back to A, but intermediate state changes (e.g., pointer freed and reallocated) can cause corruption. Common in lock-free stacks/queues.

ThreadSanitizer (TSan)

Dynamic race detector using happens-before analysis and shadow memory. Instruments all memory accesses and synchronization operations. Low false positive rate but 5-15x slowdown. Compile with -fsanitize=thread, run instrumented binary.

Race Window Amplification

Techniques to increase race window duration or success probability:

• CPU stress (stress-ng) to cause scheduler thrashing
• Process priority manipulation (nice, chrt)
• Core pinning (taskset) to control thread placement
• Parallel instances (spawn 100+ attack processes)
• Nanosecond timing to measure optimal trigger points

Sanitizer Integration with CyberGym

CyberGym uses AddressSanitizer to detect memory corruption in submitted PoCs. Race exploits that trigger UAF or double-free must produce ASAN reports. Check submit.sh output for "Sanitizer CHECK failed" messages—these indicate successful exploit even if exit_code=1.

---

Defense Triple

Defense Perspective

Defense Layer	Control	Key Points
Design	Immutable data structures; pure functions; message-passing instead of shared state	Eliminate race surface by construction — Rust ownership, Erlang processes, Go channels
Synchronization Discipline	Mutex/RWLock with documented lock order; avoid double-checked locking without atomics	C11 _Atomic, memory_order_acquire/release; pair with static analysis (Clang Thread Safety Analysis)
TOCTOU Elimination	Use file descriptors (openat, fstatat with AT_SYMLINK_NOFOLLOW) instead of path-based checks	Treat "filename → fd" as the security boundary; never re-resolve paths
Signal Safety	Signal handlers only call async-signal-safe functions; defer work via signalfd or self-pipe	man 7 signal-safety; minimize handler to setting a volatile sig_atomic_t flag
Runtime Detection	ThreadSanitizer (TSan) in CI; AddressSanitizer (ASan) + UBSan for memory races; helgrind for release candidates	TSan finds ~90% of data races in unit tests; pair with chaos fuzzing (AFL++ custom mutators)
Testing & Validation	Stress tests under high CPU load; property-based testing for invariants; Coverity/racer2 static analysis	Race windows amplify under load — tests passing in CI do not imply correctness in production

Code Review Checklist

• All pthread operations use proper mutex locking
• Signal handlers only call async-signal-safe functions
• TOCTOU sequences eliminated (use O_NOFOLLOW, fstatat with AT_SYMLINK_NOFOLLOW)
• Double-checked locking uses memory barriers (C11 atomics, volatile with barriers)
• Lock order documented and enforced (prevent deadlocks)
• All shared state accessed via atomics or under locks

Hardening Techniques

• Eliminate TOCTOU: use file descriptors (openat) instead of paths
• Signal safety: minimize signal handler code, use sig_atomic_t
• Atomic operations: use C11 _Atomic or compiler intrinsics
• Thread-safe libraries: prefer reentrant functions (*_r variants)
• Process isolation: use separate processes instead of threads where possible

Detection Methods

Sanitizer-Based Detection (Primary)

• ThreadSanitizer (TSan): -fsanitize=thread — detects data races at runtime; ~90% recall on unit tests with adequate coverage.
• AddressSanitizer (ASan): -fsanitize=address — detects UAF/double-free triggered by race-induced corruption; CyberGym submit.sh signals exploit success via "Sanitizer CHECK failed".
• UndefinedBehaviorSanitizer (UBSan): -fsanitize=undefined — catches signed overflow, misaligned access triggered during races.

SIEM / Audit Detection

• Splunk SPL: index=app sourcetype="tsan" OR sourcetype="asan" "WARNING: ThreadSanitizer" | stats count by binary, race_stack
• Sysmon EID 1 (process): Correlate crash dumps with sanitizer output; repeated ASan reports in CI = potential race-driven memory corruption.
• Falco runtime rule: spawn (process, crash) && proc.name in (critical_services) triggers investigation.

Static Analysis

• Clang Static Analyzer: scan-build -enable-checker core,security,cplusplus.NewDelete — finds double-checked locking, missing locks.
• Coverity: race-condition models for pthread/C++ std::atomic.
• racer2 / racerD (Infer): specialized race detector for Java/Java/C++.

Defense Evasion Techniques

Sanitizer Evasion

• Single-threaded PoC: Race-only manifests with thread scheduling; suppress TSan by serializing execution so the race never triggers under instrumentation.
• Pre-compiled binary: Ship binary without sanitizer instrumentation; CyberGym runs against ASan-instrumented harness, so craft PoC that triggers UAF in non-instrumented path.
• Race window minimization: Tighten the race window so sanitizer sampling misses it (TSan has ~8x slowdown and samples access patterns).

Timing Evasion

• Schedule manipulation: sched_setaffinity, nice, usleep to control thread interleaving; defenders looking for "fast" exploitation miss slow-race backdoors.
• Cache-bank confict: Force contention on a shared cache line to artificially create race window; defenders monitoring CPU load patterns miss subtle cache contention.

Log Suppression

• ASan report corruption: Trigger ASan early with benign UAF to fill log buffer; subsequent real exploit reports truncated.
• TSan suppression file: .tsan_suppression shipped in test fixtures hides known races; attackers abuse to suppress race reports during exploitation.

---

Practical Steps

Step 1: Source Code Reconnaissance

Scan vulnerable source code for concurrency primitives and TOCTOU patterns:

# Find pthread usage
grep -rn "pthread_create\|pthread_mutex\|pthread_cond" .

# Find signal handlers
grep -rn "signal(\|sigaction(\|SIGALRM\|SIGUSR" .

# Find TOCTOU candidates
grep -rn "access(\|stat(\|lstat(" . | grep -A5 "open(\|fopen("

# Find fork/clone patterns
grep -rn "fork(\|clone(\|vfork(" .

Step 2: Dynamic Analysis with ThreadSanitizer

Compile and run with TSan to detect races:

# Compile with ThreadSanitizer
gcc -fsanitize=thread -g -O1 vulnerable.c -o vulnerable_tsan

# Run and capture report
./vulnerable_tsan 2>&1 | tee tsan_report.txt

# Look for "WARNING: ThreadSanitizer: data race" messages
# TSan reports show conflicting memory accesses with stack traces

Step 3: Helgrind Analysis

Use Valgrind's helgrind to find pthread synchronization bugs:

# Run helgrind
valgrind --tool=helgrind --log-file=helgrind.log ./vulnerable

# Check for lock order violations and missing locks
grep "Possible data race\|lock order" helgrind.log

Step 4: Trace TOCTOU Sequences with strace

Identify time gaps between check and use:

# Trace access/open sequence
strace -f -tt -T -e trace=access,open,openat,stat,lstat ./vulnerable 2>&1 | grep -A1 "access"

# Look for patterns like:
# 14:23:45.123456 access("/tmp/file", R_OK) = 0 <0.000012>
# 14:23:45.123789 open("/tmp/file", O_RDONLY) = 3 <0.000333>
# The 333 microsecond gap is the race window

Step 5: Build Race Exploit

Create exploit script that wins the race window:

Symlink Race Technique:

1. Create benign target file (e.g., /tmp/userfile)
2. Create race loop that continuously swaps symlink between safe and sensitive targets
3. Execute vulnerable binary repeatedly while race loop runs in background
4. Measure timing gap from strace output to optimize race window
5. Increase parallel instances (20-50+) to improve success probability
6. Monitor audit logs and dmesg for evidence of privilege escalation

See payloads.md for tool-specific commands (inotify-tools, taskset, stress-ng).

Step 6: Amplify Race Window

Increase success probability with stress and parallelization:

Race Window Amplification Techniques:

1. Use stress-ng to create CPU scheduling chaos (8-16 workers × 60 seconds)
2. Pin attacker and victim processes to different CPU cores (taskset)
3. Spawn 50-100 parallel attack instances to increase win probability
4. Monitor success rate: measure percentage of attempts that trigger sanitizer/crash signals
5. Reduce process priority of victim (nice -n 19) to expand time window
6. Run attack under different load conditions to find optimal parameters

Amplification can increase success rate from 1-5% → 10-30%.

Step 7: Signal Handler Race Exploitation (regreSSHion-style)

Exploit signal handler race in server:

Signal Handler Race Pattern:

1. Identify timeout signal handlers (SIGALRM, SIGIO) in server code
2. Find non-async-signal-safe function calls in handlers (malloc, free, printf, syslog)
3. Send connection that delays before timeout fires (e.g., SSH authentication delay)
4. Signal delivery during critical section (malloc/free) causes heap corruption
5. Exploit corrupted heap state to achieve UAF or arbitrary write
6. Amplify by sending parallel connections to increase race probability

See guides/signal-handler-race-exploitation.md for detailed patterns and tools (ThreadSanitizer, helgrind) for race detection.

Step 8: Verify Exploitation with GDB

Debug race condition with catchpoints:

# Set catchpoint on thread creation
gdb -ex "catch syscall clone" \
    -ex "commands 1" \
    -ex "  bt" \
    -ex "  info threads" \
    -ex "  continue" \
    -ex "end" \
    -ex "run" \
    ./vulnerable

# Set breakpoint on signal handler
gdb -ex "catch signal SIGALRM" \
    -ex "commands 1" \
    -ex "  bt" \
    -ex "  x/10i \$pc" \
    -ex "  continue" \
    -ex "end" \
    -ex "run" \
    ./vulnerable

Step 9: CyberGym Submission Format

Package race exploit for CyberGym validation:

CyberGym Submission Pattern:

1. Create bash-based exploit wrapper (submit.sh) that orchestrates race conditions
2. Use stress-ng (4-10 workers) and parallel instances (30-50) to amplify race window
3. Implement per-instance exploit logic that triggers the race condition
4. Monitor dmesg and system logs for sanitizer/ASAN detection
5. CyberGym framework detects memory corruption (sanitizer CHECK failed) as PASS signal
6. Exit code 1 with sanitizer output indicates successful exploitation

Note: Submission framework automatically validates memory corruption reports via AddressSanitizer.

---

Detection Methods

Application Behavior Indicators

• Race condition artifacts: Duplicate successful operations (double-spend, double-withdraw).
• Account balance anomalies: User balance going negative or showing impossible values.
• Inventory mismatch: Database inventory count vs. actual count diverges over time.
• Audit log gaps: Missing audit entries for high-frequency operations.

SIEM Detection Rules

• Splunk SPL: index=app action="transfer" | stats count by user_id, amount | where count > 1 | sort -count
• Application performance: Sudden spike in DB transactions per second; lock contention.
• Custom application logging: Detect double-submit patterns within milliseconds.

Code Static Analysis

• TOCTOU detection: Slither (Solidity), Semgrep (multi-language) for time-of-check vs. time-of-use patterns.
• Lock analysis: Detect missing mutex/critical section around shared state.
• Atomic operation check: Detect non-atomic check-then-act patterns.

Defense Evasion Techniques

Race Window Maximization

• Parallel requests: Send N concurrent requests; maximize chance of race window.
• Last-byte synchronization: Hold requests open, send final byte simultaneously (HTTP request smuggling).
• HTTP/2 multiplexing: Multiple concurrent streams on single connection; bypasses per-connection rate limits.
• Single-packet attack: Send multiple HTTP requests in single TCP packet (James Kettle technique).

TOCTOU Exploitation

• Pre-compute state: Trigger check, then immediately act before check expires.
• Cache poisoning: Poison cache so check sees stale data while act sees new.
• Async abuse: Trigger async operations that complete between check and act.

Container/Cloud Race Exploitation

• IAM propagation delay: Create role, immediately assume before policy propagates.
• Cross-region replication lag: Exploit time window between regions.
• Database replication lag: Read from replica before write propagates (read-after-write inconsistency).

Detection Evasion

• Slow & distributed: Spread race attempts across many sessions/IPs; below per-source rate limit.
• Use legitimate-looking traffic: Mimic normal user behavior (mouse movements, page scrolls).
• Off-hours operation: Execute during low-traffic hours; less likely to trigger anomaly detection.

References

• CVE-2024-6387: OpenSSH regreSSHion signal handler race condition
• CVE-2023-26136: tough-cookie TOCTOU vulnerability
• "The Art of Software Security Assessment" - Race Condition chapter
• ThreadSanitizer documentation: https://github.com/google/sanitizers
• MITRE CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization