Upon release, publish Docker image to ghcr.io

[robertlagrant]

Fixes #3026

Tasks

• Reviewed contribution guidelines
• PR is descriptively titled 📑 and links the original issue above 🔗
• Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
  Run tests locally to check for errors.
• Commits are in uniquely-named feature branch and has no merge conflicts 📁

Thanks!

[pombredanne]

Thanks... not sure why the DCO bot complains about your signoff.. I will override this as this looks fine otherwise.

[pombredanne]

@robertlagrant I have been pondering whether merging this or not silently for a long while!
The hold up is that if we start redistributing built docker images, there is a truckload of obligations wrt. attribution and redistribution that come with it and I would not want ScanCode to be the proverbial cobbler's son. It is easy to forget all the license, origin and vulnerabilities we can carry around with an image! We can scan the images alright with ScanCode.io but that's a piece of work on its own.

It would be great to automate this end-to-end though as a blueprint on how to use ScanCode instead. Do you feel like chipping in on this larger piece of work.

(Also as an aside, I might prefer using a Debian base than a Ubuntu base as there could be additional license restrictions attached to Ubuntu base images)

[robertlagrant]

Hi @pombredanne - FYI this is just to use Ubuntu as the image to run the automation on. All it does is build the existing Dockerfile in the root of the repo, which I think is a Python build.

As for the attribution - I know a good tool that can scan a repo for licences etc :D

[pombredanne]

@robertlagrant Thanks.

As for the attribution - I know a good tool that can scan a repo for licences etc :D

We will run a built image through ScanCode.io for sure as it provide pretty decent support for container images ;)

[elrayle]

@pombredanne I am interested in helping to move this work forward. What do you see as the next step to unblock this PR?

[elrayle]

@pombredanne @robertlagrant Any thoughts on how this PR can be unblocked? I'm happy to help.

For context on the need for this work, there are 28 images for scancode-toolkit in the Docker hub that individuals have created in an attempt to fill this need.

Advantages to generating an image and storing in GitHub Packages:

• easier adoption - use the image instead of trying to build
• discoverability - link to the image in right sidebar of the scancode-toolkit repo
• code reliability - automation can help reveal bugs in the release process early

The hold up seems to be described in the following statement...

The hold up is that if we start redistributing built docker images, there is a truckload of obligations wrt. attribution and redistribution that come with it and I would not want ScanCode to be the proverbial cobbler's son. It is easy to forget all the license, origin and vulnerabilities we can carry around with an image! We can scan the images alright with ScanCode.io but that's a piece of work on its own.

@pombredanne Can you say more about what you think is the solution?

I'm happy to meet to discuss this.

[pombredanne]

@elrayle sorry for the late reply!

I am sold on the benefits of publishing an image alright.
As said above:

• I would prefer this to be based on Debian rather than Ubuntu if possible.

• We should/will need to scan this base docker image with ScanCode.io https://scancodeio.readthedocs.io/en/latest/tutorial_web_ui_analyze_docker_image.html

  • then generate a proper attribution (in ScanCode.io)
  • then collect corresponding source code for the whole image and store these somehow for redistribution as a second image based on the first one that would contain all the sources by building a second image and using apt-get source on the installed packages and fetching all the source tarball of ScanCode such that this has all the source code of all packages available side-by-side with the binaries for anyone to download if needed.
  • ideally all these steps should be automated end-to-end so we rescan, regen attribution and collect source code on each rebuild of the image. ScanCode.io is also containerized and can run from the CLI.

Few folks care to make this for published container images alright, but we should lead by example and there would be some irony to publish an image that's not been scanned given the domain and context of ScanCode ;)

• We also need:
  • a minimal smoke tests to ensure the image works
  • a periodic job that publishes an updated image using an updated base Debian image that has updated its packages, likely running daily (or based on yet another job that would query VulnerableCode to decide if a rebuild is needed because of reported vulnerabilities)

This is not a huge amount of work, but this is real work. I really welcome the help there and can guide you as needed if you are willing to tackle some of it.

The community hangs out on https://matrix.to/#/#aboutcode-org_discuss:gitter.im and we can talk or chat here or elsewhere anytime!

[pombredanne]

@elrayle It is likely simpler and better to start a new PR BTW

[elrayle]

@pombredanne Thanks for the detailed list of expectations. They all make sense. I'm a big fan of leading by example, so agree with your views on making sure the image is scanned and has attributions. I'm planning to put this on my work list starting as early as next week or possibly in a few weeks depending on how well other work clears up. I'll definitely start a new PR.

Is there a Slack channel or other interactive discussion platform for asking questions in real time if clarifications are needed?

[pombredanne]

@elrayle We use matrix (with formerly gitter.im service acquired by element) at https://matrix.to/#/#aboutcode-org_discuss:gitter.im (and we are also on IRC libera.chat at #aboutcode but this is seldom used)

[pombredanne]

You can use your favorite matrix client or the web one from Element at https://app.element.io/#/room/#aboutcode-org_discuss:gitter.im