Repository navigation
Devfile Library should have the ability to get resources from a private repo #984
Description
Activity
context
Currently, when a devfile is parsed from a uri
schemaVersion: 2.2.0 metadata: name: my-project-dev parent: uri: https://raw.githubusercontent.com/devfile/registry/main/stacks/nodejs/devfile.yamlthat is in a repo from a supported git provider, the resources from the repo are downloaded.
problem
If we want to download from private repos, we need to authenticate with a token. How should this git token be passed into the library?
- Using a Kubernetes secret
type ParserArgs struct { ... // K8sClient is the Kubernetes client instance used for interacting with a cluster K8sClient client.Client ... }-
Passing in the token itself; could be from an env variables, etc
-
Other?
01/23 - Devfile community call
- potential duplication of work; align with SPI
- consider explicitly mapping repo url with credentials instead of inferring from url
removing from hold; will continue with implementation with the potential of swapping out with SPI in the future
token flow: will scope to single scenario where only the private parent resources are downloaded for now. will need to consider private child and private parent scenario in the futureexperimenting with mocking a happy path test for cloning a repo
edit 03/08
two cases when the git token is used:- main devfile comes from a private repo (requires using a token in
util.DownloadInMemory()) - downloading parent devfile resources from a private repo (requires using a token in the parser)
consumers will pass the token through the
ParserArgsstruct:type ParserArgs struct { ... + // Token is a GitHub, GitLab, or Bitbucket personal access token used with a private git repo uri + Token string ... }the single token passed from client means that:
- main devfile is public and token works only for private parent repo
- main devfile and parent devfile are in the same private repo
future: will need to consider the case where the main and parent devfiles are in different private repos
edit 03/21
wasn't able to mock the happy path for downloading a repo or file;SetTokenvalidates token with a network call and is wrapped by the file and repo downloadingfound edge case where
DownloadInMemoryonly works if Github url is in the raw format, will need to addressedit 03/28
added mocks for testing andGitRawFileAPIto get the raw file endpoints for the different providersedit 03/29
addressing review comments; fixed private main devfile support- main devfile comes from a private repo (requires using a token in
ongoing review progress; updating token location and url parsing
moving back to in progress; unit tests are completed, but functional testing needs to be added / at the minimum explored properly before the PR can be merged. if functional tests can't be added, manual testing needs to be documented as a final option
Found a way to add functional tests and completed the mocking setup; moved parent resource downloading into a separate method that can then be mocked. Currently creating the test cases with the mocks
04/14 - added functional test cases, which required:
- adding the git interface to the devfile context struct
- storing the git interface in the resolverTools for parent cases (context is lost after parseFromKubeCRD)
- token duplication in the context and the git interface
Metadata
Metadata
Assignees
Type
Projects
- Status
/kind user-story
Which area this user story is related to?
/area library
User Story
We would like if the devfile/library had the ability to get resources from a private repository. This would be required for reading devfile.yaml or an outerloop kubernetes deploy.yaml defined in the devfile for example.
The library should have the ability to read the private resources for most of the leading repositories like github, gitlab, bitbucket, etc.
In AppStudio, this is done using SPI but it would require installing their CRs and Controllers. But we could also look at the OpenShift Developer Console(ODC) as to how they do it for their codebase but their implementation would be in TS.
Acceptance Criteria
Target Date: 2023-04-28