Repository navigation
Slim down product images #816
Description
We might have some potential to slim down product images. This can reduce build time, image size and attack surface. For example, the Hive Dockerfile has a comment about Hadoop:
Line 102 in 1965d50
Now that we build from source, it might be worth digging into the build processes to:
a) Limit which components we build. It doesn't make sense to build stuff that's never copied to the final image.
b) Revalidate if all the components that are copied into the final image are really needed in production. With Hive, for example, we switched the build to only build the metastore, which significantly reduced the attack surface. Some products consist of multiple components and plugins, which might not all be needed to run the platform.
c) While we're at it, try to generate an SBOM for each component that is copied into the final image (next to the component itself). For most components that should already be the case, see #814
We want to focus on products that are mostly affected by vulnerabilities right now:
- Trino
- Hive
- HBase
Acceptance criteria:
- Document what could be removed and the impacts of the removal
- Document what can't be removed and why it can't be removed
Activity
lfrancke commentedon Aug 27, 2024
I'm fairly certain that there is another similar issue already and there is at least one draft PR for Hadoop on this already.
xeniape commentedon Aug 29, 2024
Hive analysis regarding the Hadoop question in the issue description:
The components copied into Hive from the Hadoop-Image are already slimmed down by some degree since we are only building part of the Hadoop components https://github.com/stackabletech/docker-images/blob/main/hadoop/Dockerfile#L66
Hive therefore only contains a subset of Hadoop (contents shown with dive):
Someone with experience and product knowledge might slim it further down by removing jars that are not needed from the shown folders (and under each
libfolder in there).With Hive 4.0.0 the component we are using from Hive (
standalone-metastore) is split into multiple separate parts:metastore-common,metastore-server, andmetastore-tools, where - depending on the outcome of tests etc. - we might also only include themetastore-servercomponent, further slimming down the attack surface https://github.com/stackabletech/docker-images/pull/818/files#diff-71bbe6452013d0b0b73eca04f83193cb3ece7f3c58310666cbc66c7a954e115aR40In case it's needed for later:
Complete jar list of Hadoop in Hive
razvan commentedon Aug 30, 2024
HBase
The first attempt to remove unused components focused on Phoenix. After realising that > 50% of CVEs in the HBase image come from the jackson-databind 2.4.0 the focus was shifted to removing this dependency.
This PR #820 removes it from the phoenix-server component and the number of CVEs is reduced from 502 to 229.
CI for PR #820 https://testing.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hbase-operator-it-custom/
razvan commentedon Aug 30, 2024
HBase - replace htrace with the noop version
Looked into replacing the htrace dependency with it's no-op version as done in the Omid image by @soenkeliebau . Htrace 3.5.0 brings in the offending
jackson-databindas a transitive dependency into Phoenix.Unfortunately it is not possible to replace htrace 3.5.0 with htrace-noop 3.5.0 . The noop version is only a drop in replacement for
htrace-core4:(On the upside, Phoenix will hopefully replace htrace with opentelemetry soon: apache/phoenix#1282