Security monitoring and SIEM for a small team: the log pipeline and the first ten detections

During an incident review last year, the team found that the attacker's first login from an unfamiliar country had been recorded in the identity provider's logs eleven days before anyone noticed anything. The log existed; nobody was looking at it, and nothing was configured to look at it for them.

That is the job of security monitoring: turning the logs your systems already produce into a small number of alerts that a person acts on. A security information and event management (SIEM) system is the usual tool, but the tool matters less than the pipeline around it. This post walks through building that pipeline for a team with limited security staff.

Key concepts

• Log: a record of an activity produced by a system, application, device or user action.
• Event: a single occurrence in a log, such as a login or a file download.
• Security event: an event relevant to security, such as a failed login, a permission change or a firewall block.
• Alert: a notification that a rule or model considers an event, or a pattern of events, suspicious enough for a human to look at.
• Incident: a confirmed security problem that needs response.

Most events are not security events, most security events do not need alerts, and most alerts, if designed well, are not incidents. The pipeline is a funnel.

The monitoring flow

Collect -> Normalise -> Store -> Correlate and detect -> Alert -> Triage -> Respond

Collect

Start with the sources that see identity and privileged actions, because most attacks show up there first:

Do not try to collect everything on day one. Volume costs money and attention. Add sources when you have a detection that uses them.

Normalise

Each source names fields differently: user, userPrincipalName, actor.login. Normalisation maps them to a common schema, so one rule can cover several sources. Many SIEMs ship parsers for popular sources; for your own application logs, emit structured JSON with consistent field names from the start.

{
  "timestamp": "2026-05-11T08:42:17Z",
  "event": "auth.login.success",
  "user.id": "u_48121",
  "user.email": "a.person@example.com",
  "source.ip": "203.0.113.77",
  "source.geo.country": "PT",
  "auth.mfa": "passkey",
  "service": "admin-portal"
}

Store

Decide retention per source based on investigation needs and regulation. A common pattern is 90 days searchable in the SIEM and one year or more in cheaper archive storage, with the archive protected from deletion. Make sure the account that produces logs cannot delete them; attackers routinely try.

Correlate and detect

Detections are rules or queries that find suspicious patterns: a single event (logging disabled), a threshold (fifty failed logins), a sequence (new MFA method, then a password change, then a large download) or an anomaly (first login from a new country for this user).

Alert, triage, respond

Every alert needs an owner and a runbook: what it means, how to check whether it is real, and what to do if it is. An alert without a runbook trains people to ignore it.

Ten detections worth writing first

These cover common attack paths, generate few false positives in most environments, and use the priority 1 and 2 sources above.

1. Cloud audit logging disabled or modified, for example CloudTrail StopLogging or DeleteTrail. Almost never legitimate outside planned changes.
2. Root or break-glass account used. Should be rare enough that every use is reviewed.
3. New access key created for a human IAM user, if your policy is to use SSO only.
4. Admin role granted in the identity provider, cloud account or source control organisation.
5. MFA method added or removed, especially followed by a login from a new device.
6. Impossible travel or new country login for privileged users, tuned with known VPN egress addresses.
7. Mailbox forwarding rule to an external domain created.
8. OAuth application granted broad mail or file permissions by a user.
9. Branch protection or required reviews disabled on a production repository.
10. EDR high-severity alert not acknowledged within 15 minutes, escalated to on-call.

A CloudTrail-based example, written as a query you can adapt to most SIEM languages:

SELECT eventTime, userIdentity.arn, sourceIPAddress, eventName, requestParameters
FROM cloudtrail_events
WHERE eventSource = 'cloudtrail.amazonaws.com'
  AND eventName IN ('StopLogging', 'DeleteTrail', 'UpdateTrail', 'PutEventSelectors')
  AND eventTime > now() - interval '15' minute;

Avoiding alert fatigue

Alert fatigue is the main reason monitoring programmes fail. People stop reading alerts that are usually false, and then they miss the real one.

• Start small. Ten good alerts beat two hundred noisy ones.
• Measure precision. For each rule, track how many alerts were true positives. Rules below an agreed threshold are tuned or removed.
• Allow-list carefully. Exclude known automation accounts and change windows, but review the allow-list itself; attackers love excluded accounts.
• Route by severity. Only high-confidence, high-impact alerts page someone. The rest go to a queue reviewed daily.
• Review monthly. Which alerts fired, which were useful, which incidents were found by something other than an alert, and what detection would have caught them earlier.

A triage runbook template

Each detection gets a short runbook in the same format, stored next to the rule:

• What it means: one sentence on the behaviour and why it matters.
• How to check: the two or three queries that confirm or dismiss it, such as "did the user register this MFA method from a known device?".
• Likely benign causes: planned changes, known automation, travel.
• If confirmed: the first containment steps and who to call, linking to the incident response runbook.
• Tuning notes: exclusions added, with date and reason.

Writing the runbook often reveals that a rule cannot be triaged with the logs you have, which is useful to learn before an attacker teaches you.

Build, buy or outsource

For a small company, the realistic options are:

• Cloud-native tools from your main provider, which cover the control plane with little effort.
• A SaaS SIEM that ingests identity, cloud, email and EDR sources with ready-made parsers.
• A managed detection and response (MDR) provider that runs detection and triage around the clock.

Whatever you choose, keep ownership of two things: the list of log sources with their retention, and the runbooks for your own systems. A provider does not know that your finance team never logs in at night.

Key takeaways

• Monitoring turns existing logs into a few actionable alerts; the pipeline matters more than the tool.
• Start with identity, cloud control plane and email logs; add sources when a detection needs them.
• Normalise fields and protect logs from deletion by the systems that produce them.
• Write a small set of high-confidence detections, each with an owner and runbook.
• Fight alert fatigue by measuring precision and reviewing rules monthly.

References

• NIST SP 800-92, Guide to Computer Security Log Management: https://csrc.nist.gov/pubs/sp/800/92/final
• MITRE ATT&CK: https://attack.mitre.org/
• AWS CloudTrail User Guide: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html
• CISA, Best Practices for Event Logging and Threat Detection: https://www.cisa.gov/resources-tools/resources/best-practices-event-logging-and-threat-detection