AWS Step Functions

The problem

Acme Shop's order fulfilment touches five systems: a Lambda function validates the order, DynamoDB holds the stock, a payment provider charges the card, SNS sends out the confirmation and a container task books the courier. Large orders also need a person to approve them. The obvious way to connect these is glue code: each Lambda function calls the next, or drops a message on the next queue. Then the process exists only as a chain of calls spread over several code bases. Each function needs its own retries and timeouts, a wait for an approval has to be kept somewhere for days, nobody can see where order o-982 stopped, and a failure halfway leaves stock reserved for an order that will never be paid.

AWS Step Functions is AWS's managed workflow service. You write the process down once as a state machine in the Amazon States Language (ASL), a JSON format, and Step Functions runs every execution of it: it calls the services, passes data from one step to the next, retries and catches errors as the definition says, waits for as long as needed and records every step. The functions go back to doing one job each.

How it works

States and executions

• A state machine is a JSON document. StartAt names the first state and States holds the states by name; each one has a Type and usually a Next. The definition can be up to 1 MB, and the console's Workflow Studio draws and edits it as a graph.
• Task does the work: it calls an AWS service, an HTTPS API or an activity worker. Choice branches on the data, Parallel runs a fixed set of branches at the same time and continues when all of them have finished, Map runs the same steps for every item in a list, Wait pauses for a number of seconds or until a timestamp, Pass passes or reshapes data without doing any work, and Succeed and Fail end the execution.
• Map has two modes. Inline runs up to 40 iterations at once inside the parent execution, and their events go into its history. Distributed runs each item, or batch of items, as a child execution with its own history, up to 10,000 in parallel, and can read its items straight from Amazon S3 (CSV, JSON Lines, Parquet and Athena manifests, among others).
• An execution is one run with one JSON input. Acme names each execution after its order (o-981). For a Standard state machine, StartExecution with the same name and input as a running execution returns the original response instead of starting a second one, a different input or a closed execution gets ExecutionAlreadyExists, and the name can be used again 90 days after the execution closes.
• Every state machine is either Standard or Express (below), and the type cannot be changed later.

Calling services

A Task state names what it calls in its Resource:

• Optimized integrations (about twenty services, among them Lambda, DynamoDB, ECS and Fargate, SNS, SQS, EventBridge, AWS Batch, AWS Glue, Athena, SageMaker AI and Amazon Bedrock) have resources such as arn:aws:states:::dynamodb:updateItem, with extra handling: Lambda's JSON result is parsed, and with .sync an ECS RunTask that reports failures fails the task. The DynamoDB integration covers GetItem, PutItem, UpdateItem and DeleteItem, which is how ReserveStock and ReleaseStock change the Stock table with no function in between.
• AWS SDK integrations (arn:aws:states:::aws-sdk:service:action) call almost any API of over 200 AWS services with the SDK's own parameters. In March 2026 AWS added 28 more services, among them Amazon Bedrock AgentCore and S3 Vectors.
• An HTTP Task (arn:aws:states:::http:invoke) calls an HTTPS API outside AWS. It takes its credentials from an EventBridge connection and must get its answer within 60 seconds. Acme keeps the payment call in the charge-card function instead, where the provider's SDK and its error codes live.
• Activities are your own workers that poll Step Functions with GetActivityTask and report back.

Each call follows one of three integration patterns:

For .sync calls in the same account, Step Functions follows the job with EventBridge events and polling. If such a task is abandoned (the execution is stopped, or another branch of its Parallel state fails), Step Functions tries to stop the job, but only on a best-effort basis.

Here are ReserveStock and ChargeCard as PlaceOrder defines them, in JSONata (see the next section):

{
  "ReserveStock": {
    "Type": "Task",
    "Resource": "arn:aws:states:::dynamodb:updateItem",
    "Arguments": {
      "TableName": "Stock",
      "Key": { "sku": { "S": "{% $states.input.sku %}" } },
      "UpdateExpression": "SET available = available - :qty",
      "ConditionExpression": "available >= :qty",
      "ExpressionAttributeValues": { ":qty": { "N": "{% $string($states.input.qty) %}" } }
    },
    "Output": "{% $states.input %}",
    "Next": "ChargeCard"
  },
  "ChargeCard": {
    "Type": "Task",
    "Resource": "arn:aws:states:::lambda:invoke",
    "Arguments": { "FunctionName": "charge-card", "Payload": "{% $states.input %}" },
    "Assign": { "chargeId": "{% $states.result.Payload.chargeId %}" },
    "Output": "{% $states.input %}",
    "TimeoutSeconds": 30,
    "Retry": [ {
      "ErrorEquals": [ "Payment.Timeout" ],
      "IntervalSeconds": 2, "BackoffRate": 2, "MaxAttempts": 2
    } ],
    "Catch": [ {
      "ErrorEquals": [ "States.ALL" ],
      "Output": "{% $merge([$states.input, { 'error': $states.errorOutput }]) %}",
      "Next": "ReleaseStock"
    } ],
    "Next": "NeedsApproval"
  }
}

When the stock is short, the condition makes UpdateItem fail with DynamoDB.ConditionalCheckFailedException, which a Catch on ReserveStock can send to an out-of-stock path. The Choice state that follows ChargeCard is one rule: "Condition": "{% $states.input.total > 50000 %}" goes to WaitForApproval, and everything else to its Default, Fulfil.

Data between states

• Each state's output becomes the next state's input, and every input or output is limited to 256 KiB (as of October 2026). Larger data goes to S3 and travels as a key, the Claim Check pattern; an oversized result fails the state with States.DataLimitExceeded.
• The QueryLanguage field picks how a state selects and shapes data, for the whole state machine or state by state. With JSONata a state has two fields, Arguments (what to send) and Output (what to pass on), whose values can hold expressions in {% %} over $states.input, $states.result, $states.errorOutput (in a Catch) and $states.context (the execution's name, start time and task token). JSONPath uses five fields instead (InputPath, Parameters, ResultSelector, ResultPath, OutputPath). JSONPath is still the default when the field is left out, but since JSONata arrived in November 2024 AWS recommends it for new workflows; PlaceOrder uses it throughout.
• Variables (also November 2024) keep a value for any later state, not just the next one. ChargeCard's Assign stores chargeId from the Lambda result, and SendConfirmation can read $chargeId three states later while the order itself flows on unchanged. Variables live in the scope of their state machine: Parallel branches and Map iterations can read outer variables but keep their own, and a Distributed Map cannot read outer ones. One variable, and one Assign, can hold up to 256 KiB, and all variables of an execution up to 10 MiB.

Errors, retries and timeouts

• Errors have names. A Lambda function's exception type becomes the error name (charge-card throws Payment.Timeout), service errors carry a prefix such as Lambda.ServiceException or DynamoDB.ConditionalCheckFailedException, and Step Functions has its own: States.Timeout, States.HeartbeatTimeout, States.TaskFailed, States.DataLimitExceeded and the wildcard States.ALL.
• Retry (on Task, Parallel and Map states) matches errors by name and tries again: IntervalSeconds before the first retry (1 by default), multiplied by BackoffRate (2 by default) each time, for MaxAttempts retries (3 by default; 0 means never). MaxDelaySeconds caps the wait and JitterStrategy FULL randomizes it, which spreads out retries from many executions; this is Retry with Backoff without writing it. ChargeCard's MaxAttempts 2 means three attempts in all, with waits of 2 s and 4 s, as in step 3.
• Catch takes over when the retries are used up: it sends the execution to a fallback state together with the error output, an object with Error and Cause. In PlaceOrder that state is ReleaseStock, the compensating transaction for ReserveStock, followed by the Fail state OrderFailed, whose Error and Cause become the execution's failure. States.ALL does not catch States.DataLimitExceeded or States.Runtime; name them explicitly where they matter.
• Timeouts. A Task waits up to 99,999,999 seconds by default, so set TimeoutSeconds on every Task, or a lost callback leaves the execution waiting for a year. Workers that run long can call SendTaskHeartbeat, and HeartbeatSeconds fails the task with States.Timeout when the heartbeats stop.
• In a Parallel state, a branch that fails without being caught fails the whole state and stops the other branches; Lambda functions already running carry on regardless.

Waiting for people and other systems

WaitForApproval sends the order and a task token to the queue approvals and pauses. Acme's back-office app reads the queue, shows the order to an approver and, when they decide, calls SendTaskSuccess (or SendTaskFailure) with the token. A Standard execution can wait like this for up to a year, and since Standard workflows are billed per state transition rather than by time, the wait itself adds nothing to the bill. The task carries a 3-day TimeoutSeconds that a Catch could turn into an automatic rejection. The token only works when it is returned by a principal in the same AWS account. Task tokens cannot be used in Express workflows.

{
  "WaitForApproval": {
    "Type": "Task",
    "Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
    "Arguments": {
      "QueueUrl": "https://sqs.us-east-1.amazonaws.com/111122223333/approvals",
      "MessageBody": {
        "orderId": "{% $states.input.orderId %}",
        "total": "{% $states.input.total %}",
        "taskToken": "{% $states.context.Task.Token %}"
      }
    },
    "Output": "{% $states.input %}",
    "TimeoutSeconds": 259200,
    "Next": "Fulfil"
  }
}

# the Orders API starts an execution named after the order
aws stepfunctions start-execution \
  --state-machine-arn arn:aws:states:us-east-1:111122223333:stateMachine:PlaceOrder \
  --name o-983 \
  --input '{"orderId":"o-983","sku":"LT-900","qty":1,"total":68400}'

# later, the back-office app reports the approver's decision with the token from the message
aws stepfunctions send-task-success \
  --task-token "$TASK_TOKEN" \
  --task-output '{"approved":true}'

Standard and Express

All rates above are per account and Region, and new accounts start with lower state transition quotas. Standard suits the order itself: long, auditable and full of steps that must not run twice, such as charging a card. Express suits high-volume, short work such as processing events or IoT data, where each step is idempotent. The two combine: AWS recommends keeping the non-idempotent steps in a Standard parent and nesting an Express workflow for a burst of idempotent ones.

Redrive, versions and aliases

• Redrive (RedriveExecution, November 2023) restarts a failed, aborted or timed-out Standard execution within 14 days of its end. It resumes at the step that did not succeed, with the same input and the same definition, and does not repeat the steps that succeeded; in a Parallel state, only the failed branches run again, and retry counters start over. Had ArrangeShipping failed without a Catch, a redrive after the fix would book the courier without charging the card a second time. An execution that already compensated and ended in a Fail state, like o-982, is better started again once the provider is back.
• Versions are numbered, immutable snapshots of a state machine (up to 1,000), and an alias such as PROD points to one version, or splits new executions between two by weight (up to 100 aliases). Starting executions through the alias lets a new version take 10% of orders first, a canary release, and every execution, redrives included, stays on the version it started with.

Testing

The TestState API runs one state, given on its own or picked out of a whole definition with stateName, without deploying anything, and reports its output, its next state and, at the DEBUG level, each step of the data processing. Since November 2025 it accepts a mock of the service's result or error, so a test needs neither the real service nor an IAM role, and it can also test Map, Parallel, .sync and callback states, and a chosen retry attempt. Step Functions Local, the downloadable emulator, is no longer supported.

# with both retries used up, a timeout from the provider should go to the Catch
aws stepfunctions test-state \
  --definition file://place-order.asl.json --state-name ChargeCard \
  --input '{"orderId":"o-982","sku":"MS-310","qty":1,"total":420}' \
  --mock '{"errorOutput": {"error": "Payment.Timeout", "cause": "no answer from the provider"}}' \
  --state-configuration '{"retrierRetryCount": 2}'
# expected: "status": "CAUGHT_ERROR", "nextState": "ReleaseStock"

Observing executions

• The execution history of a Standard execution lists every event: each state entered and exited with its input and output and the variables it assigned, each task scheduled, started, succeeded or failed. The console draws it on the graph, so support staff can open o-982 and see where it failed and why.
• CloudWatch Logs receive the history at a chosen level (ALL, ERROR, FATAL or OFF), with or without the data (includeExecutionData); Express workflows have no other record. CloudWatch metrics count executions started, succeeded, failed, timed out and throttled, and open executions against the account's limit, and since October 2025 the console shows them, with billing figures, on a metrics dashboard.
• AWS X-Ray traces an execution together with the Lambda functions and services it calls, and Standard workflows send EventBridge events on every status change, which a team can route to an alarm or to the waiting client.

Pricing

As of October 2026 in US East (N. Virginia):

• Standard: $0.025 per 1,000 state transitions after a free tier of 4,000 a month, which does not expire. Each state that runs counts, and so does each retry and each redrive. A workflow that takes ten transitions per order costs about $25 for 100,000 orders a month.
• Express: $1.00 per million executions, plus duration at $0.00001667 per GB-second for the first 1,000 GB-hours a month, rounded up to 100 ms and billed in 64 MB steps of memory, which depends on the size of the definition and the data. A million 30-second executions at 64 MB come to about $31 of duration.
• On top come the services the workflow calls (Lambda, DynamoDB, ECS) and CloudWatch Logs.

Where it fits

• Solutions: order and payment workflows like PlaceOrder; approval and other human-in-the-loop processes; data and ETL pipelines that run a Distributed Map over millions of S3 objects; machine learning and media pipelines (SageMaker AI, MediaConvert); IT and security automation; and AI workflows that call Amazon Bedrock models, or, since June 2026, a Bedrock AgentCore agent as a step (the managed harness behind it is in preview).
• Patterns it implements or supports: Saga (Orchestration), with Step Functions as the orchestrator and Catch states running each Compensating Transaction (steps 1 and 3); Retry with Backoff and Timeout and Fallback as Retry, TimeoutSeconds and Catch; Asynchronous Request-Reply, since StartExecution returns at once and the client polls DescribeExecution or listens for the status-change event; Pipes and Filters as a chain of Task states that each transform the data; and Claim Check for anything over 256 KiB. Steps of an Express workflow, and any step a Retry repeats, should be idempotent.
• Usual neighbours: API Gateway or a Lambda function that starts executions; EventBridge rules and EventBridge Scheduler, which starts executions on a cron or rate schedule; the services it calls, such as Lambda, DynamoDB, SQS, SNS, ECS and Fargate, AWS Batch, AWS Glue and S3; CloudWatch and X-Ray; and an IAM execution role that holds the state machine's permissions.
• Managed offerings: Step Functions is itself the managed service. Its closest relatives elsewhere are Amazon MWAA, which runs Apache Airflow (versions up to 3.3.1 as of September 2026), Temporal Cloud for Temporal, and, inside Lambda, Lambda durable functions. Other clouds offer Azure Durable Functions and Google Cloud Workflows.

When to use it

Use Step Functions when a business process spans several services and steps, has to survive failures halfway, may wait minutes to months, and someone needs to see where each run stands. It fits AWS-heavy systems best, since most AWS APIs are one Task state away. Look elsewhere when the workflow logic is complex enough to want a real programming language, when the system must also run outside AWS, for scheduled batch data pipelines that a data team maintains in Python, and for tight loops of tiny steps, where every transition costs money and history.

Trade-offs

• The process is visible and durable. Every Standard execution has a graph and a complete history, and the platform keeps the state, the retries and the waits, so the functions stay small. In return the flow lives in a JSON definition rather than in code, where reviews, refactoring and type checks are weaker than for a program; the CDK and similar tools can generate the definition from code.
• Limits shape the design. 256 KiB per input or output, 25,000 events per Standard history (an Inline Map over thousands of items reaches it), 5 minutes per Express execution and a 1 MB definition. Large data moves by reference, and long loops move to Distributed Map or to a fresh execution.
• Cost follows the number of steps. Standard charges per transition, retries included, which is little for a ten-step order and a lot for a workflow that loops over many small steps; Express is cheaper there but gives up the history, .sync, task tokens and exactly-once semantics.
• Throughput has quotas. Start and transition rates are throttled per account and Region with soft limits, and Standard allows a million open executions per Region; very large bursts need Express, batching or a quota increase.
• Exactly once has edges. It covers Step Functions' own transitions. A Retry repeats a call on purpose, a Lambda function or ECS task that was aborted may still finish, and an asynchronous Express execution may run twice, so the steps that change data need to tolerate a repeat.
• It is AWS-only. The service runs only in AWS and its integrations call AWS APIs. The ASL specification is public, but leaving AWS still means rewriting or re-hosting the workflows, while Temporal and Airflow run anywhere.

Implementation notes

• Set timeouts everywhere: TimeoutSeconds on each Task (the default is 99,999,999 seconds), a timeout on the whole state machine where it makes sense, and HeartbeatSeconds for long workers.
• Retry the right errors: add the Lambda service errors AWS recommends retrying (Lambda.ServiceException, Lambda.AWSLambdaException, Lambda.SdkClientException, Lambda.ClientExecutionTimeoutException), use backoff with MaxDelaySeconds and JitterStrategy, and keep States.ALL for the last catcher.
• Compensate explicitly: give each step that changes something an undo state, route Catch to it, and end in a Fail state with an Error and Cause that support staff can act on; keep secrets and internal details out of Cause.
• Name executions after a business key such as the order ID, so a retried StartExecution cannot start the same order twice (in a Standard workflow, until 90 days after the execution closes).
• Pass references, not documents: S3 keys for anything that might grow past 256 KiB, and variables instead of carrying every result in the payload.
• Watch the history size: use Distributed Map for large lists, or start a new execution to continue a long-running loop before it reaches 25,000 events.
• Deploy with versions and aliases, define state machines as code (CloudFormation, SAM, CDK or Terraform; the console exports CloudFormation and SAM templates), and test each state with TestState and mocks before deploying.
• Grant the execution role only what it calls, encrypt with a customer managed KMS key where policy requires it, and log at ERROR level, or without execution data, when the payloads hold personal data.