Repository navigation
Add certificate-based authentication support to Microsoft Graph files… - #69564
bujjibabukatta wants to merge 3 commits into
Conversation
| DEFAULT_SCOPE = "https://graph.microsoft.com/.default" | ||
|
|
||
|
|
||
| def _get_certificate_token( |
There was a problem hiding this comment.
NIT: Maybe we should create a module level factory method in msgraph hook module which returns a CertificateCredential and have the get_credentials method in KiotaRequestAdapterHook and this module call it, that way we avoid code duplication.
For the rest looking good to me, just need to address static checks errors.
There was a problem hiding this comment.
Good catch, thanks! Pulled the cert-building logic into a shared
build_certificate_credential() factory in hooks/msgraph.py — both
get_credentials and fs/msgraph.py's _get_certificate_token call it now,
just passing in the sync vs async CertificateCredential class. No more
duplicated construction logic.
Ran ruff/mypy on both files locally and re-verified the fs module delegates
correctly. Existing tests didn't need changes since they mock at the
_get_certificate_token() boundary.
Pushed!
There was a problem hiding this comment.
Sorry, this was a bad idea of mine, cause I see you have to pass the class as well, as msgraph hook uses the async aio version of it while the fs one uses the sync one, so this refactor doesn't make much sense. I would revert that change as it doesn't offer much honestly and I don't like the fact that we need to pass the class.
So I would revert that last change, if static checks are green, we can merge it.
There was a problem hiding this comment.
Can you please check now?
There was a problem hiding this comment.
Change doesn't seem to be reverted
There was a problem hiding this comment.
I would remove the build_certificate_credential method, as it doesn't offer much benefit as hook and fs use different classes anyway.
…ion between hook and filesystem
|
@bujjibabukatta This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack. Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
|
This PR can be closed as it has been done in this one. |
Support certificate-based auth for Microsoft Graph Filesystem
Closes #69335
What
The Microsoft Graph filesystem (
msgraph://,sharepoint://,onedrive://,msgd://schemes, backed by
airflow.providers.microsoft.azure.fs.msgraph.get_fs) only supportedclient-secret authentication.
KiotaRequestAdapterHook(used by the Microsoft Graphoperator/sensor) already supports certificate-based auth via
certificate_path/certificate_dataconnection extras — this PR brings the filesystem integration toparity, so the same
msgraphconnection can be reused across the hook, operator,sensor, and filesystem regardless of which auth method it uses.
Why this needed a workaround, not a one-line fix
The filesystem is built on the third-party
msgraphfspackage, whoseAbstractMSGraphFSis implemented on top of
authlib'sAsyncOAuth2Client. That client only understandsOAuth2 client-secret grants — it has no concept of
azure-identitycredential objectslike
CertificateCredential. I traced this by walkingmsgraphfs's andauthlib'ssource directly:
authlib's defaultClientAuth.DEFAULT_AUTH_METHODSonly registersclient_secret_basic,client_secret_post, andnone— notprivate_key_jwt, andmsgraphfsnever callsregister_client_auth_method()to add it. So passingtoken_endpoint_auth_method="private_key_jwt"throughoauth2_client_paramswouldfail at request time.
x5t(certificate thumbprint) JWT header that generic RFC 7523private_key_jwtsigning doesn't add automatically — omitting it produces
AADSTS700027: certificate ... not registered on application.acsone/msgraphfs#13about it.Given that, this PR fetches a single access token up front via
azure.identity.CertificateCredential.get_token()(the same credential class alreadyused in
KiotaRequestAdapterHook.get_credentials) and hands it to authlib as a staticbearer
tokeninoauth2_client_params, instead of aclient_secret.Known limitation (documented in the docs and code comment): because authlib isn't
driving token acquisition in this path, it can't transparently refresh the token via the
certificate once it expires (~60–90 min, standard Azure AD access token lifetime). Tasks
that run longer than that need to re-resolve the
ObjectStoragePathto get a freshfilesystem/token. This is called out explicitly in the docs and in the
AirflowExceptionusers would otherwise silently hit.
How
certificate_pathorcertificate_datais set on the connection, it now takesprecedence over
client_secret.certificate_passwordreuses the same connection field asclient_secret(
conn.password), matchingKiotaRequestAdapterHook.get_credentials's existingsemantics — no new connection fields needed, and the existing
msgraphconnectionform (which already has "Certificate path" / "Certificate data" widgets from the hook)
works unchanged.
AirflowExceptioninstead ofsurfacing a raw
azure-identity/msgraphfstraceback.Files changed
providers/microsoft/azure/src/airflow/providers/microsoft/azure/fs/msgraph.py— new_get_certificate_token()helper + branching logic inget_fs()providers/microsoft/azure/tests/unit/microsoft/azure/fs/test_msgraph.py— 5 newtests: cert-path token flow, cert-data + password flow, precedence over
client_secret, failure handling, existing client-secret tests unaffected
providers/microsoft/azure/docs/filesystems/msgraph.rst— new "Certificate-basedAuthentication" section, including the token-refresh limitation
providers/microsoft/azure/newsfragments/69335.feature.rst— new feature noteTesting done
ruff check/ruff format --diff— cleanmypy(isolated, no repo plugin) — cleanover client_secret, credential-failure error handling, and confirmed the existing
client-secret-only tests are unaffected
get_fs()end-to-end (mockedCertificateCredential/MSGDriveFS)against 4 scenarios to confirm the exact
oauth2_client_paramsbuilt in each caseNot in scope / follow-ups
require either patching
msgraphfsupstream or subclassingMSGDriveFS's internalclient construction. Flagging for maintainers in case there's a preferred direction
here; happy to follow up in a separate PR once
acsone/msgraphfs#13is resolvedupstream.
Important
🛠️ Maintainer triage note for @bujjibabukatta · by
@potiuk· 2026-07-08 15:38 UTCHelpful heads-up from the maintainers — please address before this PR can be reviewed:
CI image checks / Static checks). See the contributor guide.Full criteria: Pull Request quality criteria.
The ball is in your court — you've been assigned to this PR. Fix the above, then mark it Ready for review.
Automated triage — may be imperfect; a maintainer takes the next look.