Skip to content

feat: allow kosli snapshot azure target other Azure clouds and endpoints - #1331

Merged
mbevc1 merged 7 commits into
mainfrom
20261104_feat_azure_endpoints
Nov 5, 2026
Merged

mbevc1 merged 7 commits into
mainfrom
20261104_feat_azure_endpoints

Conversation

@mbevc1

@mbevc1 mbevc1 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Let kosli snapshot azure run against Azure China, Azure Government, a private cloud or a local stand-in. Its login, Resource Manager and Kudu endpoints were fixed to the public cloud, so sovereign-cloud tenants could not use it, and Cloud Capture's end-to-end job could not run the real CLI against its Azure stand-in.

Closes #1327

New flags

  • --azure-cloud (public, china, usgov; default public)
  • --azure-authority-host and --azure-resource-manager-url, also settable as KOSLI_AZURE_AUTHORITY_HOST and KOSLI_AZURE_RESOURCE_MANAGER_URL

Behaviour changes for existing users

With the default public cloud and no overrides, the endpoints, token audiences and authority are the same as on main. What changes:

  • Kudu token: still requested for https://management.azure.com/, but through the SDK credential (Entra v2 endpoint, cached and retried) instead of a hand-written v1 request that panicked on a missing access_token. The real-Azure integration tests download from Kudu with it and pass.
  • Kudu host: comes from the site's Repository entry in hostNameSslStates, the same lookup the Azure CLI uses. The token is sent only if that host is the app's own SCM host: <app>.scm.azurewebsites.net, or <app>-<hash>.scm.<region>.azurewebsites.net for an app with a unique default hostname. Such apps did not work before. A site that lists no such host fails, naming the app, before any token is requested. App Service Environment hosts stay unsupported, as before.
  • AZURE_AUTHORITY_HOST: still honoured, and now also for the Kudu token, which always went to login.microsoftonline.com before.
  • ACR: one credential and token cache serve every image, instead of a new credential per image.

Design choices

  • The overrides replace endpoints only. Tokens keep the chosen cloud's audiences, so a private cloud with its own Resource Manager audience would need a further flag.
  • In Azure China and Azure Government the Kudu token is for management.core.chinacloudapi.cn/ or management.core.usgovcloudapi.net/, the resources the Azure CLI sends to Kudu there. Neither cloud has been tested against real Azure.
  • Instance discovery is turned off when the authority is overridden, because it asks the public cloud about the new authority.
  • Both overrides must be https URLs: the SDK refuses an http authority and will not send a bearer token over http.
  • --azure-resource-manager-url also lifts the Kudu host check, because the site list then comes from the operator's own endpoint.

Cloud Capture

Set the two environment variables in the Snapshotter to the stand-in's https URL. The stand-in has to serve:

  • /<tenant>/v2.0/.well-known/openid-configuration and the token endpoint it names
  • the site list, each site with a Repository host name, which may be the stand-in's own host
  • the Kudu zip at /api/zip/site/wwwroot/

Checklist

  • Docs are autogenerated from CLI help — make sure the help text reflects your changes
  • Helm chart (charts/k8s-reporter/) updated, if needed. Note: these changes live in a separate PR (not needed)
  • Terraform provider and related changes updated, if needed (not needed)

@github-actions github-actions Bot added go Pull requests that update go code feat labels Nov 4, 2026
@claude

claude Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @mbevc1's task in 3m 2s —— View job


Follow-up review complete

  • Fetch previous review comments
  • Diff BEFORE..AFTER to scope review (BEFORE unreachable — full PR diff reviewed, scoped to the current commit set da32bc9d..27253192)
  • Assess status of each prior finding
  • Check new commits for new bugs/security issues/regressions
  • Post summary comment

All three previously-raised findings are resolved and their review threads are already marked resolved on GitHub:

  1. Kudu token now only goes to the app's own SCM host unless --azure-resource-manager-url overrides the endpoint (checkKuduHost/trustedKuduHost).
  2. Public-cloud Kudu token resource reverted to https://management.azure.com/, matching pre-PR behaviour.
  3. ACR lookups now reuse the client's shared Azure credential instead of minting one per image.

No new bugs, security issues, or regressions found in the reviewed commits. Full details posted as a summary comment.

Comment thread internal/azure/azure_apps.go Resolved
Comment thread internal/azure/azure_apps.go Resolved
Comment thread internal/azure/azure_apps.go Resolved
@mbevc1 mbevc1 changed the title feat: let kosli snapshot azure target other Azure clouds and endpoints feat: allow kosli snapshot azure target other Azure clouds and endpoints Nov 4, 2026
@mbevc1

mbevc1 commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

The Kudu token no longer changes on the default path.

Default path (public cloud, no overrides) compared with main

  • Authority: the credential is given no cloud, as before, so azidentity still uses AZURE_AUTHORITY_HOST when it is set.
  • Resource Manager and ACR: same endpoint and audiences. With no cloud set, the SDK falls back to cloud.AzurePublic and containerregistry.azure.net, which are the values now passed explicitly.
  • Kudu token: requested for https://management.azure.com/, the same resource as before. It now comes through the SDK (cached, retried) instead of a hand-written v1 request that panicked on a missing access_token.
  • Kudu host: taken from the site's Repository entry in hostNameSslStates, the same lookup the Azure CLI uses (_get_scm_url). The token is sent only if that host is the app's own SCM host, either <app>.scm.azurewebsites.net or <app>-<hash>.scm.<region>.azurewebsites.net (unique default hostname). The old code built only the first form, so apps with a unique default hostname now work. App Service Environment hosts stay unsupported, as before.
  • ACR: the client's credential is shared across images, so one token cache serves every image.

Other clouds

  • The Kudu token resource is management.core.chinacloudapi.cn/ or management.core.usgovcloudapi.net/, which the Azure CLI sends to Kudu in those clouds.
  • The Kudu host must end in chinacloudsites.cn or azurewebsites.us.
  • None of this has run against Azure China or Azure Government.

Overrides

  • Overrides replace endpoints only. Tokens keep the base cloud's audiences.
  • Instance discovery is turned off when the authority is overridden.
  • --azure-resource-manager-url also lifts the Kudu host check, because the site list then comes from the operator's own Resource Manager.

Cloud Capture

Set the two environment variables in the Snapshotter to the stand-in's https URL. The stand-in has to serve:

  • instance discovery
  • /<tenant>/v2.0/.well-known/openid-configuration
  • the token endpoint that configuration names
  • the site list, with a Repository host name for each site
  • the Kudu zip at /api/zip/site/wwwroot/

@mbevc1
mbevc1 force-pushed the 20261104_feat_azure_endpoints branch from 490c992 to 9946211 Compare November 4, 2026 18:26
@mbevc1
mbevc1 force-pushed the 20261104_feat_azure_endpoints branch from 528d917 to 9cf9f8b Compare November 5, 2026 10:50
snapshot azure could only authenticate against the Azure public cloud,
so a tenant in Azure China or Azure Government failed before reaching
its registry, even though the ACR login-server check already accepts
.azurecr.cn and .azurecr.us.

--azure-cloud (public, china, usgov; default public) selects the SDK's
cloud configuration and passes it to the client secret credential, the
App Service client factory and the ACR client, so all three use the same
authority, Resource Manager and registry audience.
The zip-deployed app path fetched its Kudu bearer token with a
hand-written v1 client-credentials request to login.microsoftonline.com,
so it ignored the cloud the rest of the command now uses and kept a
second copy of the authority.

The token now comes from GetToken on the credential the App Service
client factory already holds, with the Resource Manager audience of the
chosen cloud, formed as the ARM SDK forms its own scope. The token
request also gains the SDK's retries and caching.
The package of a zip-deployed app was downloaded from
<app>.scm.azurewebsites.net, a suffix that only exists in the public
cloud, so the download failed in Azure China and Azure Government.

The host is now the site's own Repository entry in hostNameSslStates,
which Resource Manager returns for every cloud. A site without one fails
with a message naming the app rather than guessing a host. The download
also goes through the same transport as the SDK clients.
--azure-cloud covers the three sovereign clouds, but a private cloud or
a local stand-in has endpoints of its own, and an end-to-end test in
another process cannot reach the CLI's Go test seams.

--azure-authority-host and --azure-resource-manager-url (or
KOSLI_AZURE_AUTHORITY_HOST and KOSLI_AZURE_RESOURCE_MANAGER_URL) replace
the chosen cloud's endpoints in a copy of its configuration, keeping its
token audiences. Both must be https URLs, which the SDK requires for an
authority and for any endpoint it sends a bearer token to. Instance
discovery is turned off when the authority is overridden, because it
asks the public cloud about the new authority.

A test drives the whole snapshot against a TLS stand-in named only by
the overrides: token, site list and Kudu download.
Two parts of the default public-cloud path had changed with the new
cloud selection:

- The credential was always given a cloud, so azidentity stopped
  honouring AZURE_AUTHORITY_HOST, which it reads only when no cloud is
  set. The public cloud with no authority override now leaves the cloud
  unset for the credential, as before.
- The Kudu token was requested for https://management.core.windows.net/
  instead of https://management.azure.com/, the resource the real-Azure
  integration tests have always used. The public cloud asks for that
  resource again. Azure China and Azure Government ask for their
  management.core resources, which the Azure CLI sends to Kudu there.

The test stand-in now answers instance discovery for the authority named
in the request, as Entra does, so a non-default authority can be tested.
Taking the Kudu host from the site's host names meant the Resource
Manager token went to whatever Repository host the site listed. Before,
it only ever went to <app>.scm.azurewebsites.net.

Unless --azure-resource-manager-url replaces the site list, the host must
now be the app's own SCM host in the chosen cloud's App Service domain:
<app>.scm.<domain>, or <app>-<hash>.scm.<region>.<domain> for an app with
a unique default hostname. Anything else fails before a token is
requested. App Service Environment hosts stay unsupported, as before.
acrImageFingerprint built a new client secret credential for every
image, and each credential has its own token cache, so every ACR app
cost a fresh Entra token request. A client made by NewAzureClient now
lends its credential to the ACR arm; a client built without one, as in
the ACR tests, still gets a new one.

The real-Azure zip test's "unknown" case now asserts the failure it
reaches, a site with no Kudu host name, rather than any error.
@mbevc1
mbevc1 force-pushed the 20261104_feat_azure_endpoints branch from 9cf9f8b to 2725319 Compare November 5, 2026 10:51
@claude

claude Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Follow-up review (commits through 2725319)

Checked the three previously-raised findings against the current code (all three GitHub threads are already marked resolved):

  1. Security — Kudu token sent to arbitrary Repository host (internal/azure/azure_apps.go): Resolved. trustedKuduHost/checkKuduHost (internal/azure/cloud.go) now require the host to match the app's own SCM host in the chosen cloud's App Service domain (<app>.scm.<domain> or <app>-<hash>.scm.<region>.<domain>) unless --azure-resource-manager-url is set. Covered by TestTrustedKuduHostRefusesAnyOtherHost / TestFingerprintZipServiceRefusesAnUntrustedKuduHostBeforeAskingForAToken.
  2. Kudu token resource/audience changed from management.azure.com : Resolved. kuduResources["public"] is back to https://management.azure.com/, matching the pre-PR behaviour (scope https://management.azure.com//.default), verified by TestKuduTokenIsForTheChosenCloudsResourceManager.
  3. Perf — new ClientSecretCredential per ACR image: Resolved. acrImageFingerprint now calls acrCredential, which reuses azureClient.credential when the client was built via NewAzureClient, falling back to a fresh credential only for test-constructed AzureClient{} literals. Verified by TestACRReusesTheClientsCredential.

No new bugs, security issues, or regressions found in the lines touched by these commits — 0 new inline comments.

View job run

@mbevc1
mbevc1 merged commit 79f1a7e into main Nov 5, 2026
23 checks passed
@mbevc1
mbevc1 deleted the 20261104_feat_azure_endpoints branch November 5, 2026 11:09
dangrondahl pushed a commit to kosli-dev/docs that referenced this pull request Nov 6, 2026
Marks Azure apps as supporting include/exclude by name and regex in the
snapshotting scopes table (GA: released in CLI v2.54.0).

Based on [feat: add include and exclude name filters to kosli snapshot
azure](kosli-dev/cli#1328) and [feat: allow
kosli snapshot azure target other Azure clouds and
endpoints](kosli-dev/cli#1331).

Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: let kosli snapshot azure be pointed at other Azure endpoints

2 participants