We moved private course videos behind CloudFront signed cookies. The app set the three cookies, yet the browser got 403 Missing Key-Pair-Id for every segment: cookies were scoped to the app's host, not the CDN's, and the behaviour trusted a different key group. How we debugged it with curl.
Our GitHub Actions pipeline for a serverless API failed every time a merge did not change the stack, such as documentation-only commits: Error: No changes to deploy. How CloudFormation empty changesets work, the flag that makes sam deploy exit cleanly, and the samconfig and OIDC setup we use per environment.
A few percent of logins failed with a generic error after we added a Cognito post-authentication trigger that recorded login events in our database. The Lambda ran in a VPC, opened a database connection on cold start, and sometimes exceeded Cognito's 5-second trigger timeout. How we moved the work off the login path.
Order events from our platform account were supposed to reach an SQS queue owned by the analytics account. The rule showed matches, the queue showed nothing, and no error appeared anywhere obvious. Causes: a queue policy without the rule ARN, a KMS key policy that denied EventBridge, and no DLQ to tell us.
During a flash sale our checkout API on ECS Fargate slowed to multi-second responses, yet the service stayed at six tasks. Target tracking on average CPU at 70% never fired: tasks were blocked on I/O, not CPU, and the max capacity was already reached on one service. How we chose better scaling signals.
Our ECS Fargate workers scaled on SQS queue depth. After each burst, scale-in stopped tasks in the middle of 10-minute PDF jobs; messages reappeared and the jobs ran again, sending customers duplicate documents. How we fixed it with task scale-in protection, SIGTERM handling and a visibility timeout that fits the work.
A Lambda function that converts uploaded videos to thumbnails worked for the first few invocations, then failed with No space left on device. Files written to /tmp survived across warm invocations of the same execution environment and filled the 512 MB default. How we found it and fixed cleanup, sizing and streaming.
Our order fulfilment state machine ran for months, then failed for large orders with States.DataLimitExceeded. A Map state collected per-item results into one array that crossed the 256 KiB payload limit. How we found the state that grew, trimmed outputs with ResultSelector and moved bulk data to S3.
We put SNS in front of an SQS queue to fan out order events, and the consumer Lambda started failing with KeyError: 'orderId'. Each SQS message body was now an SNS envelope with our event as an escaped JSON string in Message. How raw message delivery fixes it, and what changes for filtering and attributes.