Terraform AccessDenied in a new AWS region: Control Tower Region deny SCP explained

We were expanding a payments API into Indonesia and needed a VPC, an EKS cluster and an RDS instance in ap-southeast-3 (Jakarta). The workload account already ran happily in eu-west-1. The deployment role had AdministratorAccess. The very first resource in the plan failed:

Error: creating EC2 VPC: operation error EC2: CreateVpc, https response error StatusCode: 403, RequestID: 6f1c2a4e-0b7d-4c55-9e2a-1d3e5f7a9b0c, api error UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::111122223333:assumed-role/platform-deploy/terraform is not authorized to perform: ec2:CreateVpc on resource: arn:aws:ec2:ap-southeast-3:111122223333:vpc/* with an explicit deny in a service control policy. Encoded authorization failure message: 3fJx...

The same pipeline, same role, same module applied cleanly in eu-west-1 five minutes earlier. Our first reaction was to check the IAM role, which is exactly the wrong place: the error says where the deny comes from, at the end of a very long line.

Reproducing it

Any regional call from any principal in that account fails in the new region and succeeds in the old one:

aws ec2 describe-vpcs --region eu-west-1 --query 'length(Vpcs)'
# 2
aws ec2 describe-vpcs --region ap-southeast-3
# An error occurred (UnauthorizedOperation) when calling the DescribeVpcs operation:
# You are not authorized to perform this operation. User: ... with an explicit deny
# in a service control policy.

Global services kept working, which is a useful clue: aws iam list-roles and aws sts get-caller-identity succeeded even with --region ap-southeast-3.

What we ruled out

• Region not enabled. Jakarta is an opt-in region. If it were not enabled for the account, calls fail with an authentication error about the security token, not an explicit deny. We confirmed it was enabled:

aws account get-region-opt-status --region-name ap-southeast-3
# { "RegionName": "ap-southeast-3", "RegionOptStatus": "ENABLED" }

• Permissions boundary or session policy. Denials from those are reported as such in the message ("with an explicit deny in a permissions boundary"). The role had neither.
• Resource-based policies. CreateVpc has no resource policy involved.

Root cause

Decoding the authorization message (the caller needs sts:DecodeAuthorizationMessage) showed the matching statement:

aws sts decode-authorization-message --encoded-message "$MSG" \
  --query DecodedMessage --output text | jq '.context, .matchedStatements'

The matched statement belonged to a Control Tower managed SCP with a condition on aws:RequestedRegion. Our landing zone had the Region deny setting enabled with governed regions eu-west-1 and us-east-1 only. Every regional API call to any other region is denied for every principal in enrolled accounts, regardless of IAM permissions, except for a list of global services and actions that the control exempts.

What is happening under the hood

Service control policies set the maximum permissions for accounts in an AWS Organizations OU. They grant nothing; an action is allowed only if IAM allows it and no SCP denies it. An explicit deny in an SCP wins over any allow, including AdministratorAccess in the account and the account root user.

A region restriction SCP has this shape. This is our own simplified version for illustration, not the exact Control Tower document:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyOutsideApprovedRegions",
      "Effect": "Deny",
      "NotAction": [
        "iam:*",
        "organizations:*",
        "sts:*",
        "route53:*",
        "cloudfront:*",
        "support:*",
        "budgets:*"
      ],
      "Resource": "*",
      "Condition": {
        "StringNotEquals": {
          "aws:RequestedRegion": ["eu-west-1", "us-east-1"]
        },
        "ArnNotLike": {
          "aws:PrincipalARN": ["arn:aws:iam::*:role/AWSControlTowerExecution"]
        }
      }
    }
  ]
}

Three parts matter:

1. NotAction lists the global services that must keep working. Their API calls are signed for us-east-1 regardless of where you are, which is why IAM and STS calls succeeded in our reproduction.
2. aws:RequestedRegion is the region the request is sent to. Terraform's provider region sets it.
3. Principal exemptions keep Control Tower's own automation working.

Control Tower can apply region restrictions in two places, and both are evaluated:

• the landing zone Region deny setting, driven by the governed regions list in the landing zone settings, which applies to all registered OUs;
• the OU-level configurable control CT.MULTISERVICE.PV.1, which accepts parameters such as AllowedRegions, ExemptedPrincipalArns and ExemptedActions per OU.

If either one denies the region, the call fails. Adding a region in one place and not the other is a common way to get this error a second time.

Environment

Minimal reproduction

You can reproduce the behaviour without Control Tower in a sandbox organization. Create an SCP from the JSON above, attach it to a test OU, move a test account into it, and run:

aws organizations create-policy --name region-allowlist-test \
  --type SERVICE_CONTROL_POLICY --description "repro" \
  --content file://region-allowlist.json
aws organizations attach-policy --policy-id p-examplepolicy \
  --target-id ou-exam-ple12345

# from the test account
aws ec2 describe-vpcs --region ap-southeast-1   # denied
aws ec2 describe-vpcs --region eu-west-1        # allowed
aws iam list-users --region ap-southeast-1      # allowed, global service

Who is affected

Any account enrolled in a Control Tower landing zone with Region deny enabled, any organization with a hand-written region allowlist SCP, and anyone running tools that call a region outside the list as a side effect. Common side effects: ACM certificates for CloudFront must live in us-east-1, WAF with CLOUDFRONT scope is managed in us-east-1, and some Terraform modules create resources in a second provider alias you may not have noticed.