SIEM tuning is the SOC work nobody talks about because it’s unglamorous. There’s no clean exploit chain to demonstrate, no CVE to drop. It’s the slow work of making your detection layer actually useful instead of a source of alert fatigue that trains analysts to ignore everything.
These are notes from production — from real log sources, real false positive storms, and real threats that needed catching.
The False Positive Problem Link to heading
A poorly tuned SIEM is worse than no SIEM. When analysts are triaging 500 alerts a day and 490 of them are junk, two things happen:
- Real incidents get missed in the noise
- Analysts stop trusting the system and start ignoring alert categories entirely
The goal of tuning isn’t to reduce alerts arbitrarily. It’s to make every alert worth looking at — which means removing noise without creating blind spots.
Understand Your Log Sources First Link to heading
Before you touch a rule, understand what each source actually tells you.
Firewall logs → network-layer allows/denies, NAT, policy hits
IDS/IPS logs → signature-based detections, protocol anomalies
Windows DC logs → auth events, group changes, GPO modifications
Endpoint AV/EDR → file detections, process trees, behavioral alerts
Email security → phishing, malicious attachments, spoofing
The common mistake: treating all logs as equivalent inputs and writing rules that join them without accounting for their different cadences, field schemas, and noise floors.
A Windows 4625 (failed logon) is interesting. A thousand 4625s in five minutes from a single source IP is a brute-force attempt. But a thousand 4625s from a hundred different users across eight hours is probably just people mistyping passwords after a password reset policy change.
Context determines meaning.
The Tuning Loop Link to heading
My process for every noisy rule:
1. Identify the rule generating volume
2. Sample the last 100 alerts — what % are true positives?
3. Find the common attributes of the false positives (source, user, time, condition)
4. Write an exception that targets exactly those attributes
5. Monitor for two weeks — did you blind yourself?
6. Document why you made the exception
The documentation step is the one people skip. Six months later, nobody remembers why 10.0.1.50 is whitelisted from brute-force detection, and that host gets compromised.
Common False Positive Patterns Link to heading
Authentication noise Link to heading
Problem: Service accounts generating thousands of “failed logon” alerts because their stored credentials are stale.
# Wazuh rule exception for known service accounts
<rule id="100001" level="0">
<if_sid>60106</if_sid> <!-- Failed Windows logon -->
<user>svc_backup|svc_monitoring|svc_antivirus</user>
<description>Known service account auth failures - suppressed</description>
</rule>
Don’t suppress blindly — suppress with a scope. If svc_backup starts failing auth from a new source IP, you still want to know.
Vulnerability scanner noise Link to heading
Your own scanner hitting your own endpoints will trip a hundred IDS signatures. Tag scanner traffic at the source and exclude it from detection rules — but log it separately so you have a record.
# Splunk: exclude internal scanner from brute-force detection
index=network_ids src_ip!=10.0.0.0/24
| stats count by src_ip, dest_ip, signature
| where count > 50
| table src_ip, dest_ip, signature, count
Windows event log noise from normal operations Link to heading
Events that look suspicious but are routine in your environment:
- 4672 (special privileges assigned) — fires on every admin logon
- 7045 (new service installed) — fires on every software update
- 4698 (scheduled task created) — fires on every Windows update
These need baseline context. How often does this event fire for this user/host combination? Alerting on count > threshold is almost always better than alerting on event exists.
What Actually Catches Threats Link to heading
The most effective detections I’ve run in production don’t detect the tool — they detect the technique.
Lateral movement:
Successful auth from host A to host B to host C within 10 minutes,
where host A has no prior auth history to B or C.
No specific tool signature. Catches psexec, wmiexec, evil-winrm, and anything else that moves laterally.
Credential dumping indicators:
Process access to lsass.exe from a non-system process,
where the accessing process is not in an approved list.
Catches Mimikatz, procdump targeting lsass, and custom tooling — without needing signatures for any of them.
Beaconing detection:
External connections from a single internal host at regular intervals
(variance < 5 seconds) over a 2-hour window.
C2 beacons are designed to blend in with normal traffic. The periodicity is what gives them away.
The False Negative Problem Link to heading
Reducing false positives is only half the job. You also need to ensure you’re not missing real threats by over-suppressing.
Tests I run periodically:
Simulate the attack, check the alert:
# On a test host — simulate brute-force attempt, confirm SIEM catches it
for i in $(seq 1 20); do
ssh invalid_user@10.0.0.5 -o ConnectTimeout=2 2>/dev/null
done
# Verify: did a brute-force alert fire within 2 minutes?
Check exception scope: After adding an exception for a noisy rule, verify the exception is scoped correctly:
Rule: Brute force detected
Exception: src_ip = 10.0.0.100 (vulnerability scanner)
Test: Run a simulated brute force from 10.0.0.101 (not the scanner)
Expected: Alert fires
If it doesn't: Your exception was too broad
Purple team exercises: Run a real attack technique against a test host and verify detection. MITRE ATT&CK is a useful framework here — map each detection rule to a technique, and periodically test that the detection still fires for that technique.
SIEM Hygiene Link to heading
A few operational things that save headaches:
- Version control your rules. Treat SIEM rules like code — PRs, review, rollback capability. A bad rule pushed to production at 2am that floods your email with 10,000 alerts is avoidable.
- Set severity levels carefully. Not everything needs to be Critical. Alert fatigue is proportional to how many Critical alerts analysts see per shift.
- Log retention policy. Know how long you keep raw logs vs. indexed events. An incident that happened 90 days ago and requires logs you deleted 60 days ago is a painful lesson.
- Time sync everything. Correlating events across sources with clock drift is miserable. NTP on every log source, always.
SIEM tuning is never done. Your environment changes, attackers change their techniques, and new log sources come online. The goal is a detection layer that your analysts trust — one where a Critical alert at 3am actually warrants waking someone up.
That trust is built slowly, through the unglamorous work of making the signal-to-noise ratio worth a human’s attention.