🤖 feat: verify a TLS trust in the shared Coder transport and move every operator client onto it by ThomasK33 · Pull Request #434 · coder/coder-k8s

ThomasK33 · 2026-10-31T11:07:24Z

Summary

Part 1 of 3 for #275. The shared Coder transport (coder.NewSDKClient) can now verify HTTPS with a TLS trust (a CA bundle and a server name), keyed per trust. The control-plane controller's license client and the CoderProvisioner key client move onto that transport, so every client that sends the operator token uses it. No caller passes a trust yet and the API does not change: part 2 adds spec.tls.operatorAccess together with the trust at every caller. The design below covers all three parts.

Background

With built-in TLS (spec.tls.secretNames), every operator component sends the site-owner operator token to Coder over plain HTTP on the control plane Service. The operator image trusts only public roots, and public CAs do not sign *.svc.cluster.local, so the operator cannot verify status.url (https://<cp>.<ns>.svc.cluster.local:443). #275 tracks verified in-cluster HTTPS.

Design: verified in-cluster HTTPS for operator calls (#275)

API

spec:
  tls:
    secretNames: [coder-tls]
    operatorAccess:            # new, optional
      caBundleRef:             # optional: ConfigMap key with PEM CA certificates
        name: coder-ca
        key: ca.crt
      serverName: coder.example.com   # optional: name the certificate must hold

operatorAccess set: every operator call with the operator token goes to status.url over HTTPS. The operator verifies the Coder certificate with the CA bundle, or with the public roots of its image when caBundleRef is absent. serverName defaults to the host of status.url.
operatorAccess unset: no change. The operator keeps the plain HTTP Service URL.
CEL: operatorAccess needs secretNames, and it needs caBundleRef or serverName (public roots never sign the Service name).

Decisions

Opt-in, fail closed. The CA reference is the switch. With operatorAccess set, the operator never sends the token over HTTP. A missing ConfigMap, a missing key, a bundle without a valid certificate, or a status.url that is not HTTPS yet stop the call. A certificate error is an error, never a reason to retry on HTTP. Redirects that leave the HTTPS origin stay refused (sameorigin).
One parsed trust value. controlplaneurl.NewTLS(pem, serverName) parses the bundle once into a TLS value (cert pool, server name, and a key with the SHA-256 of the bundle). Every PEM block must be a parseable CERTIFICATE, so a private key pasted into the ConfigMap is an error, not a silent skip. controlplaneurl.OperatorTLS(ctx, reader, cp) reads the ConfigMap and returns that value. Every caller gets its URL from CoderAPIURL and its trust from OperatorTLS.
One shared transport. All operator-token clients go through coder.NewSDKClient. Its transport registry keys each pool by origin plus trust (server name and CA hash). Two control planes with different CAs never share a pool, and a new CA gets a new pool. net/http verifies an https:// proxy with the same TLS configuration, so a transport with a trust refuses https:// proxies with a clear error. http:// proxies keep working: the TLS connection to Coder goes through CONNECT, verified end to end.
CA rotation without restarts. Each client creation reads the ConfigMap through the reader it already has (the manager cache already watches ConfigMaps cluster-wide). A changed bundle has a new hash, so the next call uses a new pool with the new roots. The old pool closes after 10 idle minutes (existing eviction).
No new Secret access. The CA comes from a ConfigMap in the control plane namespace. The operator already has cluster-wide ConfigMap get/list/watch, so RBAC does not change. CA certificates are not secret, and trust-manager and kubectl create configmap --from-file both produce this shape.
Status. Dependent CRs report the existing ControlPlaneNotReady wait with the CA error, CoderProvisioner reports OperatorAccessReady=False (OperatorTLSUnavailable), and the aggregated API answers 503. The control plane gets an OperatorCABundleResolved condition (True/False with ConfigMapNotFound, KeyNotFound, InvalidCABundle) and re-reconciles when the ConfigMap changes. A CA that parses but does not sign the Coder certificate shows as the x509 error of the failed calls (license condition, dependents' CoderUnavailable).
CoderProvisioner. Key rotation keys on status.url, which the opt-in does not change, so opting in rotates no key. The applied key's API URL becomes status.url. A status.pendingKeys record from before the opt-in holds the plain HTTP Service URL. For such a record, the operator calls status.url instead when the record's URL is the control plane's own internal HTTP URL (same Service, and tokenBoundTo already binds the record to the control plane UID). It refuses any other plain HTTP URL for a control plane with operatorAccess.

Delivery

The security property (no operator token over HTTP once operatorAccess is set) lands whole: the API field arrives in the same PR that wires every caller.

PR 1 (this PR): the TLS trust value and the trust-keyed shared transport. The control-plane controller's license client and the CoderProvisioner key client move onto the shared transport (coder.NewSDKClient), so every operator-token client uses it. No API change.
PR 2: spec.tls.operatorAccess, OperatorTLS, CoderAPIURL returning HTTPS status.url, the trust at every caller (controlPlaneCoderClient for all CRD controllers, the aggregated API, the license and entitlements calls, CoderProvisioner key calls including records from before the opt-in), and docs.
PR 3: the OperatorCABundleResolved condition with a ConfigMap watch, and a Kind E2E phase with built-in TLS on: the script makes a self-signed CA and a serving certificate with openssl, checks that the operator reaches Coder over HTTPS, and checks that a wrong CA stops the calls.

Implementation

internal/controlplaneurl/tls.go: TLS, a parsed trust (cert pool, server name, key), built by NewTLS. The bundle must hold only PEM certificates: a private key block, a broken certificate, or a cut-off certificate after valid ones is an error. ClientConfig returns a copy of the pool.
internal/aggregated/coder/config.go: Config.TLS. The transport registry adds the trust key to the pool key, sets TLSClientConfig on a new transport, and refuses https:// proxies for it. The client stack around the transport is unchanged (markMaybeSent, the sameorigin round tripper and redirect policy, connection wait).
codercontrolplane_controller.go (newSDKLicenseClient) and coderbootstrap (newAuthenticatedClient) build their clients with coder.NewSDKClient instead of a cloned default transport per call. They keep their timeouts, and the provisioner client keeps the rate-limit bypass header. This also adds the sameorigin round tripper to both.
internal/tlstest: a test CA that signs HTTPS test servers, for this PR and the next two.

Validation

config_tls_test.go drives NewSDKClient against HTTPS servers signed by a test CA. The signing CA passes. Another CA, the system roots, and a wrong server name fail with a certificate error, and the operator token never reaches a server that failed verification.
The same file shows that clients with the same trust reuse one pooled connection, that another server name gets its own connection, and that a client with another CA gets no connection verified with the old CA. With the trust left out of the pool key, the test fails.
A CONNECT proxy test shows that an http:// proxy carries the verified connection (and a wrong CA still fails through it), and that an https:// proxy gets no request.
tls_test.go covers the bundle checks, including the cut-off certificate (the check fails without the trailing-data rule).
Local gates: make verify-vendor test build lint test-integration, make codegen manifests docs-reference build-installer with no diff, make docs-check.
An independent reviewer (gpt-6.1-sol) found no blocker. Its notes led to the pooled-connection test and the pool copy. Its main finding is for part 2 (CoderProvisioner records with the old HTTP URL): see decision 7.

Risks

The license and provisioner clients now share the transport registry: per-deployment connection limits (128) and idle reuse now apply to them, and their transport errors carry the markMaybeSent marks. Their tests pass unchanged.
During a CA rotation, the old and the new trust each keep a pool for up to 10 idle minutes, so one deployment can briefly hold twice the per-deployment connection limit.

Refs #275

Generated with mux • Model: anthropic:claude-opus-5-5 • Thinking: xhigh • Cost: $8.28

… Coder transport (#275) Change-Id: Ia6d57ba9bc939d05a095c607979b817ef9337365 Signed-off-by: Thomas Kosiewski <tk@coder.com>

Change-Id: I6c4c9b145e8680b0fec3d04cfb8d395d32263da9 Signed-off-by: Thomas Kosiewski <tk@coder.com>

chatgpt-codex-connector · 2026-10-31T11:07:37Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-31T12:02:01.182274Z	`c980706`	Manual request
🔒 Security Review	✅ Completed 2026-10-31T12:06:28.743114Z	`c980706`	Manual request

Security findings

Advisory findings (1)

🟡 Wire operator TLS before exposing the opt-in · Medium

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

ThomasK33 · 2026-10-31T11:08:45Z

@codex security review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7eb80da739

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

🛡️ Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: 7eb80da739

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

… every caller Change-Id: I7d0399f56324e15e20fcdc5a9a4fb27dfd8675f0 Signed-off-by: Thomas Kosiewski <tk@coder.com>

ThomasK33 · 2026-10-31T11:26:57Z

@codex review

ThomasK33 · 2026-10-31T11:26:59Z

@codex security review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fee7c6d37c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-31T11:31:07Z

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: fee7c6d37c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Change-Id: I650b1c4f37095eefb2ae1d47911c83488d913d17 Signed-off-by: Thomas Kosiewski <tk@coder.com>

ThomasK33 · 2026-10-31T11:47:13Z

@codex review

ThomasK33 · 2026-10-31T11:47:15Z

@codex security review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae581e4066

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-31T11:50:20Z

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: ae581e4066

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Change-Id: I69584ace20ca19ce1856b47cd651671209999729 Signed-off-by: Thomas Kosiewski <tk@coder.com>

ThomasK33 · 2026-10-31T11:59:57Z

@codex review

ThomasK33 · 2026-10-31T11:59:58Z

@codex security review

chatgpt-codex-connector · 2026-10-31T12:01:59Z

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: c980706f41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-31T12:06:27Z

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: c980706f41

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

ThomasK33 added 2 commits October 31, 2026 11:03

feat: add CoderControlPlane spec.tls.operatorAccess and a trust-keyed…

8a8a73b

… Coder transport (#275) Change-Id: Ia6d57ba9bc939d05a095c607979b817ef9337365 Signed-off-by: Thomas Kosiewski <tk@coder.com>

test: prove pooled connections stay with their TLS trust

7eb80da

Change-Id: I6c4c9b145e8680b0fec3d04cfb8d395d32263da9 Signed-off-by: Thomas Kosiewski <tk@coder.com>

chatgpt-codex-connector Bot reviewed Oct 31, 2026

View reviewed changes

Comment thread internal/controlplaneurl/tls.go Resolved

Comment thread internal/aggregated/coder/config.go Resolved

chatgpt-codex-connector Bot reviewed Oct 31, 2026

View reviewed changes

Comment thread api/v1alpha1/types_shared.go Resolved

refactor: keep the transport trust, move the API to the PR that wires…

fee7c6d

… every caller Change-Id: I7d0399f56324e15e20fcdc5a9a4fb27dfd8675f0 Signed-off-by: Thomas Kosiewski <tk@coder.com>

ThomasK33 changed the title ~~🤖 feat: add CoderControlPlane spec.tls.operatorAccess and a trust-keyed Coder transport~~ 🤖 feat: verify a TLS trust in the shared Coder transport and move every operator client onto it Oct 31, 2026

chatgpt-codex-connector Bot reviewed Oct 31, 2026

View reviewed changes

Comment thread internal/controlplaneurl/tls.go Resolved

fix: reject CA bundles with text before or between PEM blocks

ae581e4

Change-Id: I650b1c4f37095eefb2ae1d47911c83488d913d17 Signed-off-by: Thomas Kosiewski <tk@coder.com>

chatgpt-codex-connector Bot reviewed Oct 31, 2026

View reviewed changes

Comment thread internal/controlplaneurl/tls.go Resolved

fix: reject CA bundles in which pem.Decode skips a block

c980706

Change-Id: I69584ace20ca19ce1856b47cd651671209999729 Signed-off-by: Thomas Kosiewski <tk@coder.com>

ThomasK33 added this pull request to the merge queue Oct 31, 2026

Merged via the queue into main with commit 46d9d1d Oct 31, 2026
9 of 10 checks passed

ThomasK33 deleted the operator-tls-trust branch October 31, 2026 12:39

This was referenced Oct 31, 2026

🤖 feat: add CoderControlPlane spec.tls.operatorAccess and verify every operator call with it #437

Merged

🤖 feat: report OperatorTLSReady on CoderControlPlane and test verified HTTPS in the Kind E2E #439

Merged

Conversation

ThomasK33 commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background

Design: verified in-cluster HTTPS for operator calls (#275)

API

Decisions

Delivery

Implementation

Validation

Risks

Uh oh!

chatgpt-codex-connector Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Security findings

Advisory findings (1)

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

🛡️ Codex Security Review

Uh oh!

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 31, 2026

🛡️ Codex Security Review

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 31, 2026

🛡️ Codex Security Review

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

ThomasK33 commented Oct 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 31, 2026

🛡️ Codex Security Review

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ThomasK33 commented Oct 31, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 31, 2026 •

edited

Loading