Skill: Detection Engineering

Supplementary Files:

• payloads.md — Sigma rule anatomy & authoring patterns, SigmaCLI usage, YARA rule anatomy & patterns, yarGen auto-generation, Loki scanning, Splunk SPL / Kusto KQL / Elastic EQL detection libraries, hayabusa & zircollo Sigma-to-EVTX pipelines, MITRE ATT&CK mapping, GitHub Actions / GitLab CI detection CI/CD, false-positive tuning methodology, and the detection-as-code lifecycle checklist
• test-cases.md — 12 structured test cases (Sigma authoring, YARA authoring, SigmaCLI translation, yarGen generation, Loki scan, hayabusa scan, zircollo offline, ATT&CK mapping, CI pipeline, FP tuning, KQL/SPL/EQL backends, lifecycle) with severity ratings and summary tables
• guides/detection-engineering-playbook.md — End-to-end detection engineering playbook (lifecycle, rule anatomy deep-dives, CI/CD pipeline recipes, FP-tuning matrix, ATT&CK coverage measurement, integration with threat-hunting and logging-monitoring)
• guides/soc-playbook-mapping-to-nist-csf-2-0.md — SOC playbook mapping detections to NIST CSF 2.0 and MITRE ATT&CK (Function mapping for Govern/Identify/Protect/Detect/Respond/Recover, Sigma → ATT&CK → CSF traceability matrix, detection-as-code pipeline, use-case prioritization matrix, detection maturity model DMM, purple-team validation with Atomic Red Team + RTA, and SOC KPIs)

Summary

Detection Engineering skill domain covering the craft of writing, testing, deploying, and retiring detection rules as code.

Tools: SigmaHQ + SigmaCLI, YARA + Yara-Rules, yarGen, Loki, hayabusa, zircollo, Splunk SPL, Microsoft Sentinel KQL, Elastic EQL/Lucene, MITRE ATT&CK, EVTX-ATTACK-SAMPLES, pySigma, sigma-scanner

Domain: defense

MITRE ATT&CK: TA0040-Detection (detection engineering discipline); rules in this skill provide coverage across every ATT&CK technique via the Sigma taxonomy.

Description

Detection engineering is the discipline of treating detection content — Sigma rules, YARA signatures, SIEM queries, EDR detections — as first-class software artifacts with version control, peer review, automated tests, staged rollouts, and retirement criteria. The detection engineer reads threat intelligence and red-team tradecraft, drafts a Sigma rule (or YARA signature, or SPL query) that captures the TTP, writes unit tests that assert the rule fires on known-malicious samples from EVTX-ATTACK-SAMPLES and stays silent on known-benign baseline data, ships it through a GitHub Actions / GitLab CI pipeline that validates YAML schema and ATT&CK tag coverage, deploys to staging for a false-positive soak, and only then promotes to production. This is detection-as-code and it is what separates a SOC that pages on noise from a SOC that pages on signal.

Difference from threat-hunting: Threat hunting is the analytic practice of forming a hypothesis and walking the telemetry to test whether an adversary is inside. Detection engineering is the craft practice of authoring, testing, and shipping the rules that hunting and the SOC rely on. Hunting consumes telemetry; detection engineering manufactures the reusable detectors that consume telemetry. A hunt that finds something becomes a detection-engineering work item; a detection that fires becomes a hunt hypothesis for refinement. They are paired: hunting supplies the empirical truth that detection engineers encode, and detection engineers supply the detectors that hunters lean on for breadth.

Difference from logging-monitoring: Logging & monitoring is infrastructure — the sensors, shippers, indexes, retention, and tamper-proof storage that produce the telemetry in the first place. Detection engineering is content — the queries and rules that run against that infrastructure. You cannot engineer a detection for an event that is not logged; you can log forever and have zero detections. logging-monitoring builds the grid; detection-engineering writes what runs on it.

Difference from deception-honeypot: Deception manufactures high-fidelity telemetry from fake assets — every hit is signal because no legitimate user touches a decoy. Detection engineering trades in noisy telemetry — production logs where legitimate admin activity blends with adversary tradecraft — and the craft is in writing rules that suppress the former while catching the latter. The two complement: a honeypot produces a near-zero-FP detection (covered in deception-honeypot); detection engineering produces the bulk of SOC detections where FP tuning is the central craft problem.

Detection engineering lifecycle: threat intel / TTP → rule draft → unit test against positive and negative corpora → CI validation (schema, ATT&CK tags, syntax) → staging soak → FP tuning → production ship → coverage layer update → quarterly review → retirement. This skill teaches every phase.

Use Cases

• Author a Sigma rule for a new TTP: A threat-intel report describes a Cobalt Strike beacon loading msxml3.dll via rundll32.exe (T1059.001 / T1129). The detection engineer drafts a Sigma rule with logsource: process_creation, selection on Image|endswith: \rundll32.exe and CommandLine|contains: msxml3.dll, attaches ATT&CK tags attack.t1059.001 and attack.t1129, and ships through CI to staging.
• Author a YARA rule for a malware family: A new loader family is observed with a consistent PE rich-header pattern and a unique XOR-decryption loop. The detection engineer drafts a YARA rule with condition: uint16(0) == 0x5A4D and $rich and $xor_loop, test-runs it against a folder of 1000 benign PE files to verify zero FP, and submits to the internal YARA repo.
• Translate a Sigma rule to three SIEM backends: One Sigma YAML → Splunk SPL (via the splunk backend), Microsoft Sentinel KQL (via the azure-sentinel or microsoft-365-defender backend), and Elastic EQL (via the elastalert or eql backend). The Sigma file is the source of truth; the platform queries are generated artifacts.
• Generate a YARA rule automatically from malware samples: Run yarGen.py against a folder of 30 unique loader samples. yarGen identifies strings that appear in the samples but not in a benign corpus, and emits a draft YARA rule. The engineer reviews, tightens the condition, and ships.
• Scan a host's EVTX with Sigma offline: Copy Security.evtx, System.evtx, and Microsoft-Windows-Sysmon%4Operational.evtx from a suspect host, then run zircollo or hayabusa against the EVTX with the SigmaHQ rule repo. Within minutes, every ATT&CK technique observed in the host's history is enumerated.
• Scan a directory of files for YARA matches: Run loki.py --rules rules/ -p /opt/samples to apply every YARA rule in rules/ plus Loki's built-in IOC database (Suricata ET, ThreatFox) against every file. Useful at the IR evidence-collection stage to triage which binaries match known malware families.
• Build a detection CI/CD pipeline: A GitHub Actions workflow runs on every pull request to sigma/rules/. It validates YAML schema, asserts every rule has id, status, tags, level; runs sigma-cli convert to each SIEM backend to catch translation errors; runs the rule against EVTX-ATTACK-SAMPLES to confirm it fires on the positive corpus; runs against a benign EVTX corpus to confirm it stays silent on the negative corpus.
• Tune a noisy detection's false-positive rate: A detection that fires 200×/day on legitimate SCCM activity. The engineer pulls the hits, identifies the common parent process and signer, adds a filter (filter: ParentImage|endswith: \sccm.exe), re-runs against 30 days of historical data, and confirms the new FP rate is <1/day before re-shipping.
• Measure ATT&CK coverage and identify gaps: Generate a MITRE ATT&CK Navigator layer from the union of tags: across every Sigma rule in the repo. Red cells = no coverage; yellow = partial; green = full. The quarterly review with the red team prioritizes new detections for the red cells.
• Retire a deprecated detection: A Sigma rule for a 5-year-old malware family that has not fired in 24 months. The engineer moves it to rules/deprecated/, sets status: deprecated, files a retirement ticket, and removes it from production SIEM import. Coverage layer is updated.

Core Tools

Sigma Ecosystem

Tool	Purpose	Command / Usage
Sigma	Vendor-neutral YAML signature format for log detection — the "YARA for logs." Author once, translate to any SIEM.	Author .yml; the YAML is the source of truth
SigmaHQ/sigma	The reference rule repository — 3000+ community Sigma rules mapped to ATT&CK, organized by tactic.	git clone https://github.com/SigmaHQ/sigma; consume rules or submit PRs
SigmaCLI (sigma-cli)	Command-line interface to the pySigma translation engine. Converts Sigma YAML → Splunk, Sentinel, Elastic, QRadar, ArcSight, Logpoint, etc.	sigma-cli convert -t splunk -p splunk_windows rule.yml
pySigma	The Python library underlying SigmaCLI; supports custom backends, pipelines, and validators.	from sigma.collection import SigmaCollection; from sigma.backends.splunk import SplunkBackend

YARA Ecosystem

Tool	Purpose	Command / Usage
YARA	Pattern-matching engine for files and memory — the malware-family classification standard.	yara -r rules.yar /opt/samples/
Yara-Rules/rules	The community YARA rule repository — signature packs for malware families, exploits, and tools.	git clone https://github.com/Yara-Rules/rules.git
yarGen	Auto-generates YARA rules from a set of malware samples by extracting strings that do NOT appear in a benign corpus.	python3 yarGen.py -m /opt/malware/ -o /tmp/rule.yar --top 20
Loki (Neo23x0)	IOC + YARA scanner — Sweeps a host or directory against a curated YARA + IOC database (Suricata ET, ThreatFox, custom).	python3 loki.py -p /opt/samples --rules /opt/signatures/

EVTX Scanners (Sigma-to-EVTX)

Tool	Purpose	Command / Usage
hayabusa (Yamato Security)	Fast Sigma-to-EVTX scanner written in Rust — applies the full SigmaHQ rule set to a folder of .evtx files in minutes.	./hayabusa csv-timeline -d /opt/evtx/ -o timeline.csv -r rules/
zircollo	Lightweight Sigma-to-EVTX scanner — converts Sigma rules to SQLite SQL, then queries EVTX-to-SQLite exports.	python3 zircollo.py -e evtx/ -r rules.yml -o matches.json
EVTX-ATTACK-SAMPLES (sbousseaden)	Reference corpus of attack-generated EVTX files mapped to ATT&CK — the positive test corpus for detection testing.	git clone https://github.com/sbousseaden/EVTX-ATTACK-SAMPLES.git

SIEM Query Languages (Detection Backends)

Platform	Language	Example Detection
Splunk	SPL (Search Processing Language)	index=win sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational EventCode=1 Image="*\\rundll32.exe" CommandLine="*MiniDump*"
Microsoft Sentinel / Defender	KQL (Kusto Query Language)	DeviceProcessEvents | where FileName =~ "rundll32.exe" | where ProcessCommandLine has "MiniDump"
Elastic Stack	Lucene + EQL (Event Query Language) + ES|QL	process where process.name == "rundll32.exe" and process.command_line == "*MiniDump*"

Frameworks & References

Framework	Purpose
MITRE ATT&CK Enterprise	Canonical TTP taxonomy — every Sigma rule carries attack.tXXXX[.XXX] tags; coverage is measured by tag union.
MITRE ATT&CK Navigator	Visualizes coverage as a heatmap layer — generated from Sigma rule tags.
OSSEM	Common data model — used in Sigma field mapping for cross-vendor portability.
SigmaHQ pySigma-pipeline-sysmon	Field-name mapping pipeline that translates generic Sigma fields to Sysmon-specific names.

Methodology

Detection Engineering Six-Phase Lifecycle

Phase 1            Phase 2            Phase 3            Phase 4            Phase 5            Phase 6
Threat Intel   →   Rule Draft      →  Unit Test      →  CI Validation   →  Staging Soak   →  Production &
& TTP Intake       Authoring          (pos + neg corpus)  (schema + tags +    (FP measurement)    Retirement
   │                  │                  │                  backends)            │                  │
   ▼                  ▼                  ▼                  ▼                    ▼                  ▼
Read TI report,  Draft Sigma YAML  Test rule against   GitHub Actions:     Deploy to staging  Ship to prod,
red-team report, or YARA signature EVTX-ATTACK-       sigma-cli convert   SIEM, observe for  update ATT&CK
or ATT&CK         matching the      SAMPLES (must      to every backend;   7-30 days; FP     Navigator layer;
technique →       technique         fire) + benign     schema validate;   tune below        quarterly review
detection gap                       EVTX (must stay    lint ATT&CK tags   threshold; ship   → retire if silent
                                    silent)                                                  > 24 months

Phase 1: Threat Intel & TTP Intake

Every detection begins with a TTP (technique, tactic, procedure) drawn from threat intel, red-team findings, or an ATT&CK coverage gap.

Intake sources:
- Threat intel reports (vendor blogs, CISA advisories, MISP events)
- Red-team debrief ("we emulated T1003.001 via comsvcs.dll; did you catch it?")
- ATT&CK Navigator coverage layer (red cells = uncovered techniques)
- Incident retrospectives ("we missed X; build a detection for it")
- Community Sigma rule PRs to SigmaHQ/sigma

Output: a one-paragraph "detection design" capturing:
  - ATT&CK ID(s) covered
  - Log source(s) required (Sysmon EID 1, WinEvent 4688, Zeek dns.log, etc.)
  - Hypothesis: "if TTP X happened, we would see Y in log source Z"
  - Expected FP rate (TTP-level hunts have lower FP than IOC-level)
  - Pyramid-of-Pain elevation (TTP-level > behavior-level > IOC-level)

Phase 2: Rule Draft Authoring

Draft the Sigma YAML (or YARA signature, or SIEM-native query). The Sigma file is the source of truth.

# Sigma rule draft — Mimikatz credential dumping (T1003.001)
title: Mimikatz Command Line Patterns
id: c6e3dea0-e1a7-4d2e-9f17-c1f3f0d4a7b2
status: experimental
description: >-
  Detects Mimikatz command-line invocations matching the canonical
  sekurlsa::logonpasswords / lsadump::sam / kerberos::ptt patterns.
  TTP: T1003.001 LSASS Memory, T1003.002 Security Account Manager.
references:
  - https://attack.mitre.org/techniques/T1003/001/
  - https://github.com/gentilkiwi/mimikatz
author: kali-claw detection-engineering skill
date: 2026/06/17
tags:
  - attack.credential_access
  - attack.t1003.001
  - attack.t1003.002
logsource:
  product: windows
  category: process_creation
detection:
  selection_mimikatz_image:
    Image|endswith:
      - '\mimikatz.exe'
  selection_mimikatz_commandline:
    CommandLine|contains:
      - 'sekurlsa::logonpasswords'
      - 'sekurlsa::minidump'
      - 'lsadump::sam'
      - 'lsadump::dcsync'
      - 'kerberos::ptt'
      - 'kerberos::golden'
      - 'crypto::capi'
      - 'privilege::debug'
  filter_legitimate_admin:
    ParentImage|endswith:
      - '\sccm.exe'         # SCCM admin tasks
      - '\wsca.exe'         # in-house IT tooling
  condition: selection_mimikatz_image or
             (CommandLine and selection_mimikatz_commandline and not filter_legitimate_admin)
fields:
  - Computer
  - User
  - Image
  - CommandLine
  - ParentImage
falsepositives:
  - Authorized mimikatz use by the internal red team (filter by ParentUser)
  - Security training exercises (filter by host naming convention)
level: high

Phase 3: Unit Testing (Positive + Negative Corpora)

A detection without tests is a hypothesis, not a detector. Test on:

Positive corpus (rule MUST fire):
  - EVTX-ATTACK-SAMPLES/Windows/.../mimikatz-logonpasswords.evtx
  - Red-team emulation logs (atomic-red-team or CALDERA output)
  - Lab-generated: run mimikatz, capture Sysmon log, ship to corpus

Negative corpus (rule MUST stay silent):
  - 30 days of benign EVTX from production workstations
  - SCCM admin activity
  - Antivirus scans
  - Software-deployment activity

Test command (hayabusa):
  ./hayabusa csv-timeline -d evtx-positive/ -r rule.yml
  # Expect: 1+ match in the positive folder

  ./hayabusa csv-timeline -d evtx-negative/ -r rule.yml
  # Expect: 0 matches in the negative folder

Phase 4: CI Validation

Every PR to the detections repo runs through a CI pipeline. Failures block merge.

# .github/workflows/detection-ci.yml
name: Detection CI
on: [pull_request, push]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.11' }

      - name: Install SigmaCLI
        run: pip install sigma-cli pySigma-backend-splunk pySigma-backend-elasticsearch

      - name: Schema validate every Sigma rule
        run: |
          for f in $(find rules/ -name '*.yml'); do
            sigma-cli parse "$f" || exit 1
          done

      - name: ATT&CK tag coverage check
        run: |
          python3 scripts/check-tags.py rules/
          # Asserts every rule has >= 1 attack.tXXXX tag

      - name: Translate to every backend (catch syntax errors)
        run: |
          for f in $(find rules/ -name '*.yml'); do
            sigma-cli convert -t splunk "$f" > /dev/null || exit 1
            sigma-cli convert -t lucene "$f" > /dev/null || exit 1
            sigma-cli convert -t kibana-ndjson "$f" > /dev/null || exit 1
          done

      - name: Positive corpus test
        run: |
          docker run --rm -v $PWD:/work yamatosecurity/hayabusa:latest \
            csv-timeline -d /work/test-data/positive/ -r /work/rules/ \
            -o /tmp/pos.csv
          test -s /tmp/pos.csv  # must produce non-empty output

      - name: Negative corpus test
        run: |
          docker run --rm -v $PWD:/work yamatosecurity/hayabusa:latest \
            csv-timeline -d /work/test-data/negative/ -r /work/rules/ \
            -o /tmp/neg.csv
          # Expect zero matches for any rule tagged level>=medium

Phase 5: Staging Soak & FP Tuning

Deploy to the staging SIEM, observe for 7-30 days, measure FP rate. Tune until FP rate < target threshold (typically < 1 hit/day).

FP tuning workflow:
1. Pull every hit from staging over the soak period
2. Group by (ParentImage, ParentUser, Computer, CommandLine signature)
3. For each cluster > 5 hits/day: identify if it is benign
4. If benign: add a filter clause, re-test on the historical soak data
5. Re-deploy, observe for another soak period
6. Repeat until FP rate < target OR escalate to "rule is too noisy, redesign"

Common FP-tuning patterns:
- Filter by parent process (SCCM, GPO, in-house tools)
- Filter by signer (Microsoft, the org's code-signing cert)
- Filter by host naming convention (jump hosts, build servers)
- Constrain CommandLine pattern (anchor to start, require exact flag)
- Add a timeframe aggregation (only fire if N hits in T minutes)

Phase 6: Production & Retirement

Ship to production SIEM. Update the ATT&CK Navigator coverage layer. Set a quarterly review reminder. A rule that goes silent for 24 months is retired.

# Ship: commit the rule, the PR merges after CI passes
git add rules/credential_access/mimikatz-cmdline.yml
git commit -m "feat(detection): T1003.001 mimikatz cmdline patterns (FP <0.5/day over 30d soak)"

# Update coverage layer
python3 scripts/build-navigator-layer.py rules/ -o coverage.json
# Upload to ATT&CK Navigator

# Quarterly review
python3 scripts/rule-liveness.py --since 90d
# Reports each rule's hit count; rules with 0 hits across all SIEMs
# for 24 months are candidates for retirement

Quick Selection Guide

Scenario	Primary Tool	Alternative
Author a vendor-neutral log detection	Sigma YAML + SigmaCLI	YARA (if file-based)
Translate Sigma to Splunk SPL	sigma-cli convert -t splunk	pySigma SplunkBackend
Translate Sigma to Sentinel KQL	sigma-cli convert -t microsoft-365-defender	Hand-translate using ATT&CK workbooks
Translate Sigma to Elastic EQL	sigma-cli convert -t eql	pySigma ElasticsearchBackend
Auto-generate YARA from malware samples	yarGen.py -m /opt/samples	Hand-author after string analysis
Scan a host's EVTX with Sigma rules	hayabusa csv-timeline -d evtx/ -r rules/	zircollo.py -e evtx/ -r rules.yml
Scan files for malware family IOCs	loki.py -p /opt/samples --rules signatures/	yara -r rules.yar /opt/samples/
Test a rule against known-malicious EVTX	EVTX-ATTACK-SAMPLES + hayabusa	Atomic Red Team + lab EVTX
CI-validate a Sigma rule on every PR	GitHub Actions + sigma-cli	GitLab CI + sigma-cli
Build a coverage heatmap	ATT&CK Navigator + Sigma tag union	MITRE CAR coverage mapping

Defense Perspective

This skill IS the defense perspective. Detection engineering is the practice of building the reusable detectors that the SOC, threat hunters, and IR all rely on. Quality gates are central:

Defense Output	Description
Detection-as-code repository	A Git repo (Sigma rules, YARA signatures, SPL/KQL/EQL queries) where every rule has: an author, a date, an ATT&CK tag, a status (experimental/stable/deprecated), a level, and a documented FP rate. CI runs on every PR.
Unit-tested detections	Every rule ships with a positive-corpus test (fires on known-malicious EVTX) and a negative-corpus test (silent on known-benign EVTX). Untested rules do not ship.
CI/CD pipeline	A GitHub Actions / GitLab CI pipeline that validates YAML schema, ATT&CK tag coverage, backend translation, and positive/negative corpus behavior. Pipeline failures block the merge.
FP-tuning discipline	Every production rule has a documented FP rate. Hits above threshold trigger a tuning sprint. Rules that cannot be tuned below threshold are deprecated.
Coverage measurement	An ATT&CK Navigator layer generated from the union of rule tags. Quarterly review with the red team prioritizes new detections for uncovered techniques.
Lifecycle discipline	Rules move experimental → stable → deprecated. Quarterly reviews identify silent rules (24 months without a hit) for retirement. The detections repo does not grow unbounded.
Detection backends	Sigma is the source of truth; SIEM queries are generated. A rule update propagates to every SIEM via CI; no manual duplication.

Practical Steps

Detailed payloads in payloads.md, complete test checklist in test-cases.md.

Exercise 1: Author a Sigma Rule for Mimikatz (T1003.001)

Goal: draft, test, and ship a Sigma rule for Mimikatz command-line invocations.

mkdir -p detections/t1003-001-mimikatz
cd detections/t1003-001-mimikatz

# Draft the Sigma rule (see Phase 2 above for the full YAML)
$EDITOR mimikatz-cmdline.yml

# Validate the schema
sigma-cli parse mimikatz-cmdline.yml

# Translate to Splunk SPL
sigma-cli convert -t splunk -p sysmon mimikatz-cmdline.yml

# Translate to Elastic Lucene
sigma-cli convert -t lucene mimikatz-cmdline.yml

# Translate to Kibana NDJSON (importable rule)
sigma-cli convert -t kibana-ndjson mimikatz-cmdline.yml > kibana-import.ndjson

# Test against positive corpus (EVTX-ATTACK-SAMPLES)
git clone https://github.com/sbousseaden/EVTX-ATTACK-SAMPLES.git /tmp/evtx-attack
hayabusa csv-timeline -d /tmp/evtx-attack/ -r mimikatz-cmdline.yml -o hits.csv
# Expect: matches in Triggers/ credential-access/ subfolder