Moving Terraform state locking from DynamoDB to S3 native locking with use_lockfile

For years, the standard Terraform setup on AWS was an S3 bucket for state and a DynamoDB table for locking. The table existed only to stop two runs from writing the same state at once, and it was one more resource to create, secure, pay for and explain to new team members.

Terraform now supports locking with a lock file stored next to the state in S3, using S3's conditional writes, and the DynamoDB arguments of the S3 backend are deprecated. We moved 140 state files across nine AWS accounts to native locking. This post explains how it works, what to configure, and the migration order that let us do it without any window where two runs could collide.

How S3 native locking works

When use_lockfile = true is set on the S3 backend, Terraform acquires the lock by creating an object next to the state file, named after the state key with a .tflock suffix:

s3://acme-tf-state/network/prod/terraform.tfstate
s3://acme-tf-state/network/prod/terraform.tfstate.tflock

The create uses an S3 conditional write (If-None-Match: *), which S3 supports since late 2024: the request succeeds only if no object with that key exists. If two runs race, exactly one succeeds; the other gets a precondition failure and Terraform reports the state as locked, showing the lock info stored in the object. When the run ends, Terraform deletes the lock object.

This is the same guarantee DynamoDB gave, with no extra service.

Prerequisites

• Terraform 1.10 or newer. Native locking arrived as experimental in 1.10 and became generally available in 1.11, where the DynamoDB arguments were deprecated. We standardised on 1.12 for the migration.
• An S3 bucket for state with versioning enabled, which you should have anyway for recovery.
• IAM permissions on the lock object for every principal that runs Terraform.

The backend configuration

Before:

terraform {
  backend "s3" {
    bucket         = "acme-tf-state"
    key            = "network/prod/terraform.tfstate"
    region         = "eu-west-1"
    encrypt        = true
    dynamodb_table = "acme-tf-locks"
  }
}

After:

terraform {
  backend "s3" {
    bucket       = "acme-tf-state"
    key          = "network/prod/terraform.tfstate"
    region       = "eu-west-1"
    encrypt      = true
    use_lockfile = true
  }
}

IAM permissions

The lock file needs read, write and delete on the .tflock object, in addition to what the state already required:

data "aws_iam_policy_document" "tf_state" {
  statement {
    actions   = ["s3:ListBucket"]
    resources = ["arn:aws:s3:::acme-tf-state"]
  }
  statement {
    actions = ["s3:GetObject", "s3:PutObject"]
    resources = [
      "arn:aws:s3:::acme-tf-state/network/prod/terraform.tfstate",
      "arn:aws:s3:::acme-tf-state/network/prod/terraform.tfstate.tflock",
    ]
  }
  statement {
    actions   = ["s3:DeleteObject"]
    resources = ["arn:aws:s3:::acme-tf-state/network/prod/terraform.tfstate.tflock"]
  }
}

The missing s3:DeleteObject on the lock file was the most common error during our rollout. Terraform acquires the lock, runs, then fails to release it, and the next run reports the state as locked.

If the bucket uses SSE-KMS with a customer managed key, the same principals also need kms:Encrypt, kms:Decrypt and kms:GenerateDataKey for the lock object writes.

The migration: both locks, then one

The risk during a migration is a window where some runs use DynamoDB and others use the lock file, so they do not see each other's locks. Terraform supports setting both at once: it acquires the S3 lock and the DynamoDB lock, and a run holding either blocks the other. We used that as a bridge.

Phase 1: both mechanisms

backend "s3" {
  bucket         = "acme-tf-state"
  key            = "network/prod/terraform.tfstate"
  region         = "eu-west-1"
  encrypt        = true
  dynamodb_table = "acme-tf-locks"
  use_lockfile   = true
}

Changing backend configuration requires re-initialisation:

terraform init -reconfigure
terraform plan    # should show no infrastructure changes

Use -reconfigure, not -migrate-state: the bucket and key did not change, so there is no state to move. We merged this change to every repository and waited until all pipelines and every engineer's machine had run init with the new configuration. A week was enough.

Phase 2: lock file only

Once nothing used the old configuration, we removed dynamodb_table, re-initialised again, and after two quiet weeks deleted the DynamoDB table.

Testing the failure modes

We tested three cases on a sandbox state before touching production.

Concurrent runs. Two terminals running terraform apply at the same moment. The second failed immediately:

Error: Error acquiring the state lock

Error message: operation error S3: PutObject, https response error StatusCode: 412,
... PreconditionFailed
Lock Info:
  ID:        3d9f1c2a-...
  Path:      acme-tf-state/sandbox/terraform.tfstate
  Operation: OperationTypeApply
  Who:       isabel@build-07
  Version:   1.12.2
  Created:   2025-09-18 09:14:02 UTC

Crashed run. We killed an apply with kill -9. The .tflock object stayed behind, as expected, and the next run was blocked. Recovery is the same as before:

terraform force-unlock 3d9f1c2a-...

Only do this after confirming no other run is active, for example by checking the CI system. The Who field helps.

Missing delete permission. With s3:DeleteObject removed, the run succeeded and then printed a lock release error; the next run was blocked. This confirmed the importance of the IAM change.

Rolling out across 140 states

Changing one backend block is trivial; changing 140 across many repositories without missing one is not. What worked for us:

• Inventory first. A script listed every backend "s3" block in every repository, with the bucket, key and lock table, so we had a checklist.
• Shared backend files. Repositories that used partial configuration (-backend-config=prod.hcl) needed only a change in the shared file, which covered most states at once.
• IAM before code. The lock object permissions were deployed to every CI role and engineer permission set before any backend change merged, so phase 1 never failed on permissions.
• A tracking table with three columns per state: phase 1 merged, all clients re-initialised, phase 2 merged. The DynamoDB table was deleted only when the last row was complete.

The whole migration took five weeks of elapsed time and about four days of actual work.

Things that did not change

• State versioning and recovery still rely on bucket versioning.
• Workspaces get their own lock files next to their own state keys (env:/staging/network/terraform.tfstate.tflock with the default prefix).
• Partial backend configuration with -backend-config files works the same way; we moved use_lockfile into those shared files.

Common mistakes

Key takeaways

• S3 native locking stores a .tflock object created with a conditional write, giving the same mutual exclusion as DynamoDB without the table.
• It is generally available from Terraform 1.11, where DynamoDB locking is deprecated.
• Grant read, write and delete on the lock object, plus KMS permissions if the bucket uses SSE-KMS.
• Migrate with both mechanisms enabled until every client has re-initialised, then remove DynamoDB.
• Test concurrent runs, crashed runs and missing permissions before rolling out.

References

• Terraform S3 backend documentation: https://developer.hashicorp.com/terraform/language/backend/s3
• Terraform 1.11 changelog: https://github.com/hashicorp/terraform/blob/v1.11/CHANGELOG.md
• Amazon S3 conditional writes: https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-writes.html
• terraform force-unlock: https://developer.hashicorp.com/terraform/cli/commands/force-unlock