Step Functions failed with States.DataLimitExceeded: keeping state payloads under 256 KiB

Our order fulfilment workflow is a Step Functions state machine: validate the order, reserve stock for each line item in parallel, create a shipment, charge the payment, send notifications. It had processed over a million orders without trouble. Then, on the first day of a wholesale promotion, every order with more than about 400 line items failed. The execution history showed:

Execution failed
Error: States.DataLimitExceeded
Cause: The state/task 'ReserveStock' returned a result with a size exceeding the maximum number of
bytes service limit.

Retail orders, with a few items each, kept working.

Which state grew?

The state machine, simplified:

{
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": { "FunctionName": "validate-order", "Payload.$": "$" },
      "OutputPath": "$.Payload", "Next": "ReserveStock" },
    "ReserveStock": {
      "Type": "Map",
      "ItemsPath": "$.order.lines",
      "ResultPath": "$.reservations",
      "ItemProcessor": {
        "ProcessorConfig": { "Mode": "INLINE" },
        "StartAt": "ReserveLine",
        "States": {
          "ReserveLine": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke",
            "Parameters": { "FunctionName": "reserve-line", "Payload.$": "$" },
            "End": true }
        }
      },
      "Next": "CreateShipment"
    }
  }
}

Two details mattered. First, the inner task had no OutputPath, so each iteration returned the whole Lambda invoke response, including ExecutedVersion, StatusCode, response headers in SdkHttpMetadata and the function payload. Second, ResultPath: $.reservations appended the array of all those results to the original input, which already contained the full order.

We fetched one execution's history and measured the input to ReserveStock and the result size per iteration:

aws stepfunctions get-execution-history --execution-arn "$ARN" --max-items 2000 \
  --query 'events[?type==`TaskSucceeded`].taskSucceededEventDetails.output' --output text \
  | head -1 | wc -c
# 612

About 600 bytes per iteration, of which the useful data (a reservation id and a warehouse code) was about 70 bytes. With 430 line items, the Map result was roughly 260 KB, plus the order itself: over the limit.

What we ruled out

• A Lambda response limit. Lambda synchronous responses can be up to 6 MB; the Lambda functions succeeded.
• History size limits. The per-execution history limit is 25,000 events; we were far below it.
• A change in our code. The workflow had not changed. Order size had.

Root cause

Step Functions limits the input and output of every state, and the data passed between states, to 256 KiB. The Map state's output combined the full order input with an array of verbose, untrimmed per-item results. For large orders, that combined payload exceeded the limit.

Where payload size comes from

Each state receives a JSON document and produces one. The size grows when:

• task results include service metadata (Lambda invoke responses through the optimised integration wrap the payload with status and headers);
• ResultPath merges results into the existing input instead of replacing it;
• Map states collect one result per item into an array;
• large business objects, such as full orders or documents, travel through every state even when only a few fields are used.

The tools to keep it small

With JSONata as the query language in newer state machines, Arguments and Output play the same role.

Environment

Step 1: return only what the next state needs

Inside the Map, ResultSelector keeps only the two fields we need from the Lambda response:

"ReserveLine": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke",
  "Parameters": {
    "FunctionName": "reserve-line",
    "Payload": { "sku.$": "$.sku", "qty.$": "$.qty", "orderId.$": "$.orderId" }
  },
  "ResultSelector": {
    "reservationId.$": "$.Payload.reservationId",
    "warehouse.$": "$.Payload.warehouse"
  },
  "End": true
}

Each item result dropped from about 600 to about 70 bytes. Parameters also stopped sending the whole order to each iteration, which reduced Lambda invocation payloads.

For a 430-line order, the Map output became about 30 KB. That was enough to fix the incident. But wholesale orders with several thousand lines were planned, so we went further.

Step 2: keep large data out of the state

The full order lives in S3 and DynamoDB already. The state machine now carries an order id and S3 key, and each Lambda reads what it needs:

{
  "orderId": "o_88123",
  "orderKey": "orders/2025/03/o_88123.json",
  "lineCount": 430
}

Step 3: Distributed Map for large item sets

For orders above a few hundred lines, we switched the Map to Distributed mode, which reads items from S3 and writes results to S3, with each child workflow execution having its own payload limits:

"ReserveStock": {
  "Type": "Map",
  "ItemReader": {
    "Resource": "arn:aws:states:::s3:getObject",
    "ReaderConfig": { "InputType": "JSON" },
    "Parameters": { "Bucket": "acme-orders", "Key.$": "$.linesKey" }
  },
  "ItemProcessor": {
    "ProcessorConfig": { "Mode": "DISTRIBUTED", "ExecutionType": "EXPRESS" },
    "StartAt": "ReserveLine",
    "States": { "...": "..." }
  },
  "MaxConcurrency": 50,
  "ResultWriter": {
    "Resource": "arn:aws:states:::s3:putObject",
    "Parameters": { "Bucket": "acme-orders-results", "Prefix.$": "States.Format('reservations/{}', $.orderId)" }
  },
  "ResultPath": "$.reservationResults",
  "Next": "CreateShipment"
}

With ResultWriter, the Map state's output is a small manifest pointing to the results in S3, not the results themselves. CreateShipment reads the reservations from S3.

Step 4: discard results you do not need

Notification tasks at the end of the workflow returned large responses that nothing used. ResultPath: null keeps the input unchanged and drops the result:

"SendConfirmation": { "Type": "Task", "Resource": "arn:aws:states:::sns:publish",
  "Parameters": { "TopicArn": "arn:aws:sns:eu-west-1:111122223333:order-events", "Message.$": "$.orderId" },
  "ResultPath": null, "End": true }

Alternatives and trade-offs

• Inline Map with ResultSelector only: simplest, enough up to a few hundred items with small results.
• Distributed Map: scales to very large item sets and has higher concurrency, at the cost of S3 reads and writes and child executions to monitor.
• Express workflows have the same 256 KiB limit, so switching workflow type does not help.

How to test the fix

We added a test that runs the state machine in a test account with generated orders of 10, 500 and 5,000 lines and asserts success:

for n in 10 500 5000; do
  key=$(./scripts/make-test-order.sh "$n")
  aws stepfunctions start-execution --state-machine-arn "$SM_ARN" \
    --input "{\"orderId\":\"test-$n\",\"orderKey\":\"$key\",\"linesKey\":\"$key.lines\"}"
done

Step Functions also has a TestState API to test a single state definition with given input, which we use in unit tests for ResultSelector expressions.

Confirming in production

The wholesale backlog was resubmitted and completed, including orders with 3,800 lines through the Distributed Map path. The largest state payload we observed afterwards was about 12 KB.

Regression guard

• A CloudWatch alarm on ExecutionsFailed with a metric filter for States.DataLimitExceeded.
• A lint step on state machine definitions flags Task states using lambda:invoke without ResultSelector or OutputPath.
• Load tests with large orders run before every promotion.

Security implications

Passing smaller payloads also means less customer data in execution history, which is retained and visible to anyone with read access to the state machine. Moving order details to S3 with its own access controls reduces exposure. The S3 buckets for items and results are encrypted and have lifecycle rules to delete results after 30 days.

Prevention checklist

• Trim every task result with ResultSelector to the fields the next state needs.
• Pass identifiers and S3 keys, not full business objects, through state machines.
• Use ResultPath: null for results nobody reads.
• Switch to Distributed Map with ItemReader and ResultWriter for large item sets.
• Test workflows with realistic worst-case input sizes.

References

• AWS Step Functions service quotas: https://docs.aws.amazon.com/step-functions/latest/dg/service-quotas.html
• Processing input and output in Step Functions: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-input-output-filtering.html
• Using Map state in Distributed mode: https://docs.aws.amazon.com/step-functions/latest/dg/state-map-distributed.html
• Using Amazon S3 ARNs instead of passing large payloads: https://docs.aws.amazon.com/step-functions/latest/dg/sfn-best-practices.html