iOS CI build hung at codesign, then failed with errSecInternalComponent: the keychain was locked

To cut build times, we moved iOS release builds from a cloud CI service to a Mac mini in the office, registered as a self-hosted GitHub Actions runner. The same fastlane lane that worked on the cloud runner, with match downloading certificates, now behaved badly:

• Sometimes the job sat at the signing step until the one-hour job timeout, with no output.
• Sometimes it failed quickly:

/usr/bin/codesign --force --sign 9A7F0C... --entitlements ... /Users/runner/Library/Developer/Xcode/
DerivedData/Shop-.../Build/Intermediates.noindex/ArchiveIntermediates/Shop/InstallationBuildProductsLocation/
Applications/Shop.app
Warning: unable to build chain to self-signed root for signer "Apple Distribution: Example Ltd (A1B2C3D4E5)"
/Users/runner/.../Shop.app: errSecInternalComponent
Command CodeSign failed with a nonzero exit code

When a developer logged in to the Mac mini with screen sharing during a hung job, a dialog was waiting on screen: "codesign wants to sign using key 'Apple Distribution: Example Ltd' in your keychain", with fields for the keychain password.

Where match put the certificate

security list-keychains -d user
#    "/Users/runner/Library/Keychains/login.keychain-db"
security find-identity -v -p codesigning
#    1) 9A7F0C... "Apple Distribution: Example Ltd (A1B2C3D4E5)"

match had imported the certificate and private key into the runner user's login keychain. On a cloud CI image, the login keychain is unlocked and configured for non-interactive use. On our Mac, the runner ran as a service without a GUI session, and the login keychain locked itself after a timeout.

The chain warning

"unable to build chain to self-signed root" appeared because the Apple WWDR intermediate certificate on the new Mac was the old one. Codesigning still needs a valid chain for distribution builds.

What we ruled out

• Wrong certificate or profile. The same match repository worked on the cloud runner.
• Xcode version mismatch. Both used Xcode 16.2.
• Disk or permission problems in DerivedData. Local manual archives under the same user worked when a developer was logged in, because the login keychain was unlocked by their session.

Root cause

codesign needs access to the private key in a keychain. On the self-hosted runner, the key was in the login keychain, which was locked when no user session was active, and the key's access control required user confirmation for codesign. With no GUI to answer the prompt, codesign either waited forever or failed with errSecInternalComponent. A missing current WWDR intermediate added the chain warning.

Keychain access for codesign

Three things must be true for non-interactive signing:

1. The keychain containing the identity is in the search list for the user running the build.
2. The keychain is unlocked.
3. The private key's access control allows codesign (and Apple's signing tools) without prompting. On recent macOS versions, that is the key's partition list, set with security set-key-partition-list -S apple-tool:,apple:,codesign:.

Cloud CI images are prepared for this. A Mac you set up yourself is not.

What setup_ci does

fastlane's setup_ci action, when it detects CI, creates a temporary keychain named fastlane_tmp_keychain, unlocks it, sets it as default and adds it to the search list, so match imports certificates there. match then sets the partition list for imported keys. The keychain is created fresh for each run.

Who is affected

Teams running iOS builds on self-hosted Macs (Mac minis, Mac Studios, MacStadium hosts) as CI runners, build agents started as launch daemons without a login session, and any setup where certificates are imported into the login keychain. Cloud macOS runners from CI vendors are usually configured correctly.

Environment

Use setup_ci in the lane

lane :beta do
  setup_ci(force: true)            # force: also on self-hosted runners where CI detection may differ
  match(type: "appstore", readonly: true,
        app_identifier: ["com.example.shop", "com.example.shop.NotificationService"])
  build_app(workspace: "Shop.xcworkspace", scheme: "Shop", export_method: "app-store")
  upload_to_testflight(skip_waiting_for_build_processing: true)
end

force: true makes it create the temporary keychain even when fastlane does not detect the environment as CI. On GitHub Actions the CI variable is set, but we wanted the behaviour to be explicit.

Or manage the keychain yourself

For lanes that do not use setup_ci, the equivalent shell steps:

KEYCHAIN=build.keychain-db
security create-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN"
security set-keychain-settings -lut 21600 "$KEYCHAIN"          # unlock for 6 hours, lock on sleep
security unlock-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN"
security list-keychains -d user -s "$KEYCHAIN" $(security list-keychains -d user | tr -d '"')
security import dist.p12 -k "$KEYCHAIN" -P "$P12_PASSWORD" -T /usr/bin/codesign
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASSWORD" "$KEYCHAIN"

And clean up at the end of the job:

security delete-keychain build.keychain-db

Install the current WWDR intermediate

curl -fsSLO https://www.apple.com/certificateauthority/AppleWWDRCAG3.cer
security import AppleWWDRCAG3.cer -k /Library/Keychains/System.keychain

This was a one-time setup step for the Mac, now part of its provisioning script.

Remove certificates from the login keychain

The earlier runs had left the distribution identity in the login keychain. We deleted it there, so there is exactly one place where signing identities live during a build, and nothing persists between builds.

Diagnosing a hang quickly next time

When signing hangs on CI, two commands on the runner show the state in seconds:

security show-keychain-info ~/Library/Keychains/fastlane_tmp_keychain-db   # "locked" or lock settings
security find-identity -v -p codesigning fastlane_tmp_keychain-db         # is the identity in this keychain

If the identity is listed but signing still prompts, the partition list was not set for the key; re-running security set-key-partition-list for that keychain fixes it.

Alternatives and trade-offs

• Run the runner in a logged-in GUI session with auto-login. It makes the login keychain available, but leaves an unlocked Mac logged in permanently, which is a security trade-off we did not want.
• Ephemeral macOS VMs per job (for example with a virtualisation tool on Apple silicon) give a clean machine every time, at the cost of more setup.

How to test the fix

bundle exec fastlane beta 2>&1 | tee build.log
grep -E "errSecInternalComponent|unable to build chain" build.log || echo "signing clean"
security list-keychains -d user      # after the job: no fastlane_tmp_keychain left behind

Confirming in production

The next twenty release builds on the Mac mini signed without hanging. Build time dropped from about 38 minutes on the cloud runner to 17 minutes locally, which was the reason for the move. The keychain dialog never appeared again.

Regression guard

• The lane fails fast if the signing step exceeds ten minutes, instead of waiting for the one-hour job timeout.
• The Mac's provisioning script installs the current WWDR certificates and verifies them.
• A weekly job runs a signing smoke test on the runner.

Security implications

Distribution private keys on a shared Mac are high-value secrets. A temporary keychain per job, deleted afterwards, means keys exist on disk only during builds. The keychain password and match password are CI secrets, the Mac has FileVault enabled, and only the runner user and two administrators can log in.

Prevention checklist

• Use setup_ci or a dedicated temporary keychain on every CI Mac.
• Unlock the keychain and set the key partition list for non-interactive codesign.
• Install current Apple WWDR intermediate certificates on self-managed Macs.
• Delete temporary keychains at the end of each job.
• Add a short timeout to signing steps so hangs fail quickly.

References

• fastlane, setup_ci: https://docs.fastlane.tools/actions/setup_ci/
• fastlane, match: https://docs.fastlane.tools/actions/match/
• Apple, Certificate Authority (WWDR intermediates): https://www.apple.com/certificateauthority/
• Apple, TN3161 Inside Code Signing: Certificates: https://developer.apple.com/documentation/technotes/tn3161-inside-code-signing-certificates