What Is Detection Engineering? A Beginner's Roadmap
In short
- Detection engineering is the practice of designing, testing and maintaining the rules that turn logs into useful alerts, treated with the same discipline as software.
- The core loop is simple: pick a threat, check you have the right logs, write the rule, test it against real activity, tune it, document it, and review it regularly.
- Free, open resources cover most of what a beginner needs: MITRE ATT&CK, Sigma rules, Atomic Red Team tests and the Alerting and Detection Strategy (ADS) template.
This roadmap is for security analysts who already triage alerts and want to start building them, and for developers or IT staff curious about the defensive side of security. You'll get a plain definition of detection engineering, the lifecycle most teams follow, the frameworks worth learning first, a small example rule, and a step-by-step learning plan you can start this month.
What detection engineering is
Every security tool that alerts you, whether a SIEM, an EDR or a cloud service, runs on detection logic: a query, rule or model that says "this pattern of events is worth a human's attention." Detection engineering is the work of creating that logic on purpose instead of accepting whatever the vendor ships.
A detection engineer asks questions like:
- Which attacker behaviors matter most to our environment?
- Do we actually collect the logs that would show those behaviors?
- Will this rule fire when the attack happens, and stay quiet the rest of the time?
- When it fires, does the analyst know what to do next?
The difference from "writing SIEM searches" is discipline. Detections are versioned, reviewed, tested and retired like code. A rule nobody has tested is a guess; a rule that fires a hundred times a day on normal activity trains people to ignore it.
The detection lifecycle
Teams describe it in different words, but most follow a loop like this:
- Prioritize. Choose a behavior to detect, based on threat intelligence, incidents you've seen, or gaps in coverage. Map it to a MITRE ATT&CK technique so you can track coverage.
- Check the data. Confirm the needed logs exist, are parsed, and arrive reliably. Many detection projects stall here, so it's worth doing before writing anything.
- Research the behavior. Understand how the technique works, what it looks like in your logs, and what normal activity looks similar.
- Write the rule. Build the query or rule, ideally in a format you can store in version control.
- Test. Generate the behavior safely in a lab and confirm the rule fires. Run it against historical data to estimate false positives.
- Document. Record the goal, blind spots, known false positives and response steps.
- Deploy and tune. Ship it, watch how it performs, and adjust.
- Review and retire. Re-test periodically, because log formats, software and attacker tools all change. Remove rules that no longer earn their keep.
Frameworks and tools worth learning first
MITRE ATT&CK
ATT&CK is a public knowledge base of adversary tactics and techniques based on real-world observations. It gives everyone shared names for behaviors, so "T1033 System Owner/User Discovery" means the same thing across tools and teams. As of November 2026 the current release is ATT&CK v20 (October 2026). Its Enterprise matrix lists 227 techniques and 479 sub-techniques. Since v18 (October 2025), ATT&CK also publishes Detection Strategies (high-level approaches to detecting a technique) and Analytics (platform-specific detection logic), which are useful starting points when you don't know where to begin with a technique.
The Pyramid of Pain
David Bianco's Pyramid of Pain (2013) ranks indicators by how much it hurts an attacker when you detect them. From bottom to top: hash values, IP addresses, domain names, network and host artifacts, tools, and tactics, techniques and procedures (TTPs). Hashes and IPs are trivial for attackers to change; behaviors are not. The lesson for beginners: indicator matching has its place, but aim to detect behavior.
Summiting the Pyramid
MITRE's Center for Threat-Informed Defense built on that idea with Summiting the Pyramid, a method for scoring how robust a detection is, meaning how hard it is for an attacker to evade. Rules based on values the attacker controls, like file names or command-line arguments, are less robust than rules tied to system activity the attacker can't easily avoid. The 2026 update (September 2026) adds scoring for precision and implementation coverage, and a calculator that reads Sigma rules. You don't need it on day one, but it's a good way to judge your rules once you've written a few.
Sigma
Sigma is an open, generic rule format for log events. As its maintainers put it, it is to log files what Snort is to network traffic and YARA is to files. You write a rule once in YAML and convert it to the query language of your SIEM with the official sigma-cli tool or the pySigma library. The public SigmaHQ repository holds more than 3,000 rules, which makes it an excellent library to read and learn from.
Atomic Red Team
Atomic Red Team, from Red Canary, is a library of small tests mapped to ATT&CK. You run a test in a lab, then check whether your rule fired. It's the simplest way to move from "I think this works" to "I saw it work." Run these tests only on systems you own and are allowed to test, ideally an isolated lab.
A first rule, end to end
Here's a small Sigma rule for one behavior: a web server process starting whoami.exe. Attackers who gain code execution through a web application often check which account they are running as; a web server has little normal reason to do that.
# Sigma rule: whoami started by an IIS web server worker process
title: Whoami Executed by IIS Worker Process
id: 00000000-0000-0000-0000-000000000000 # generate a real UUID for your rule
status: experimental # new rules start as experimental
description: Detects whoami.exe launched by w3wp.exe, which can indicate a web shell.
references:
- https://attack.mitre.org/techniques/T1033/
author: Your Name
date: 2026-11-10
tags:
- attack.discovery
- attack.t1033
logsource:
category: process_creation # e.g. Sysmon Event ID 1 or Windows 4688
product: windows
detection:
selection:
ParentImage|endswith: '\w3wp.exe'
Image|endswith: '\whoami.exe'
condition: selection
falsepositives:
- Administrators debugging an application pool (rare; confirm with the owner)
level: highTo turn it into a Splunk search, you could use sigma-cli with the Splunk backend and the Sysmon field mapping:
# Install the CLI and the Splunk backend, then convert the rule
pip install sigma-cli
sigma plugin install splunk
sigma convert -t splunk -p sysmon rules/whoami_iis.ymlThen test it: in a lab Windows server running IIS and Sysmon, trigger the behavior with the matching Atomic Red Team test or a harmless test page you control, confirm the alert fires, and search 30 days of production data to see how noisy it would be. Note the limits too. An attacker can rename the binary or use a different discovery method, so this rule is a useful starting point, not complete coverage of T1033.
Document every rule with ADS
Palantir's open Alerting and Detection Strategy (ADS) framework is a simple template that forces you to think a rule through before it goes live. Its sections are:
| Section | What it answers |
|---|---|
| Goal | What behavior should this detect? |
| Categorization | Which ATT&CK tactic and technique? |
| Strategy abstract | How does it work, which data does it use, how is noise reduced? |
| Technical context | What does a responder need to know to understand the alert? |
| Blind spots and assumptions | What could stop it firing, or let an attacker evade it? |
| False positives | Which known benign activity triggers it? |
| Validation | How do you generate a true positive to prove it works? |
| Priority | How urgent is an alert, and why? |
| Response | What should the analyst do when it fires? |
Writing the "blind spots" and "validation" sections is often where you discover a rule isn't ready yet. That's the point.
A beginner's roadmap
A realistic plan for someone with basic security and IT knowledge, studying part-time:
| Stage | Focus | Done when you can… |
|---|---|---|
| 1. Foundations | Windows and Linux logging (Sysmon, Windows Security events, auditd), cloud audit logs, basic networking | Explain what a process creation event contains and where your logs come from |
| 2. Query language | One SIEM language well (for example SPL or KQL), plus basic Python | Answer "which hosts ran this process last week?" without looking up syntax |
| 3. Threat knowledge | ATT&CK tactics and techniques; read SigmaHQ rules for techniques you care about | Explain how three common techniques show up in logs |
| 4. Build a lab | A small Windows and Linux lab with Sysmon and a free or trial SIEM; Atomic Red Team | Run a test and find its traces in your SIEM |
| 5. Write and test | Write Sigma rules, convert them, validate them, document them with ADS | Show five tested, documented rules in a Git repository |
| 6. Operate | Tuning, coverage tracking, peer review, robustness scoring | Explain why each rule exists, how noisy it is and what it misses |
The Git repository from stage 5 doubles as a portfolio. Five well-documented rules with tests say more about your skills than a list of tools on a CV.
Common beginner mistakes
- Writing rules for logs you don't have. Check data first.
- Chasing coverage numbers. A colored-in ATT&CK matrix doesn't mean the rules work. Tested rules count; untested ones don't.
- Only matching indicators. Hash and IP lists expire quickly. Pair them with behavior-based rules.
- No response guidance. An alert without next steps slows down the analyst who receives it, often at 3 a.m.
- Never retiring anything. Noisy, outdated rules cost attention. Review them on a schedule.
A note for Indonesian teams
Under Indonesia's Personal Data Protection Law (UU PDP, Law No. 27 of 2022), Article 46 requires a data controller to notify affected data subjects and the supervising institution in writing within 3 x 24 hours if personal data protection fails. You can't notify about what you haven't noticed, so tested detections on the systems that hold personal data are a practical part of meeting that obligation.
Next steps
- Pick three ATT&CK techniques relevant to your environment and check whether you collect the logs to see them.
- Read the SigmaHQ rules for those techniques and convert one to your SIEM's language.
- Set up a small lab, run one Atomic Red Team test, and write your first ADS document.
Later articles in this series will cover detection as code, Splunk and Microsoft Sentinel query basics side by side, and Python scripts for security automation. If your logs come from AWS, our AWS security checklist for startups covers turning on CloudTrail and GuardDuty first. See also our security services.
Want help deciding what to detect first, or reviewing rules you've written? Email Binjesoft and we can look at your log sources and priorities together. See our services for how we work.
Sources: MITRE ATT&CK: Updates (v20, October 2026) · SecurityWeek: ATT&CK v18 adds Detection Strategies and Analytics · David Bianco: The Pyramid of Pain · MITRE CTID: Summiting the Pyramid 2026 · SigmaHQ: Sigma rules repository · SigmaHQ: Rule conventions · Red Canary: Atomic Red Team · Palantir: Alerting and Detection Strategy framework · UU No. 27 Tahun 2022, Pasal 46