During an investigation we needed to know who changed a security group in one of 14 AWS accounts three months earlier, and the logs were gone. How we set up one organization trail into a protected log archive account, an Athena table with partition projection, and the queries we use most.
A SIEM is only as good as the logs you feed it and the alerts someone answers. How to build a security monitoring pipeline step by step, which log sources come first, ten detections worth writing on day one, and how to keep alert fatigue away.
A new release started dropping logs in Loki with a 429 and stream limit errors. A logging change had promoted user_id and trace_id to labels, creating hundreds of thousands of streams. How Loki's streams work, how we found the offending labels, and how structured metadata and query-time parsing replaced them.
Prompt and response logs are the fastest way to build an accidental PII database. This guide shows where personal data enters an LLM feature, how to redact it before logging and before the provider, and how to keep retention and access under control.
Most of our worst backend incidents did not start with a big design flaw. They started with a swallowed exception, a missing index or an endpoint that returned everything. Here are ten habits we now block in code review, and what we do instead.
Our single-host production server stopped accepting writes at 03:12 with no space left on device. Docker had quietly filled the disk with a 31 GB container log, old images and build cache. Here is how we found each piece and stopped it from coming back.
Our CloudWatch line went from 2,100 to 6,900 USD in a month without new services. Debug logging enabled during an incident was never turned off, and every load balancer health check produced a log line. Finding the top log groups and fixing levels cut ingestion by 81 percent.