Skip to content

Slimmer images (-2.06 GB on pythonr, -258 MB on python), faster builds, and faster runtime defaults (OpenBLAS + GDAL threads + Arrow I/O) - #140

Merged
Robinlovelace merged 1 commit into
masterfrom
perf-slim-fast-containers
Nov 3, 2026
Merged

Robinlovelace merged 1 commit into
masterfrom
perf-slim-fast-containers

Conversation

@Robinlovelace

Copy link
Copy Markdown
Contributor

Summary

This PR bundles image-size reductions, build-time speedups, and runtime compute optimizations across the geocompx Docker suite:

  • python/Dockerfile (-257.6 MB / -12.8% image size, -37% build time):
    • Remove build-essential (gcc, g++, libc6-dev) from the initial apt-get install layer and drop the separate apt purge layer (which left 283 MB in the immutable lower layer). All 144 Python 3.14 geospatial dependencies install from precompiled manylinux wheels via uv.
    • Set GDAL_NUM_THREADS=ALL_CPUS, GDAL_CACHEMAX=512, VSI_CACHE=TRUE, GDAL_HTTP_MULTIPLEX=YES, and PYOGRIO_USE_ARROW=1.
  • pythonr/Dockerfile (-2.06 GB / -33.2% image size, -17% build time):
    • Replace single-threaded reference libblas-dev with multi-threaded libopenblas0-pthread + libopenblas-dev (4.7x faster R matrix algebra).
    • Install required C/C++ runtime/header libraries explicitly with --no-install-recommends, symlink Quarto's bundled pandoc to /usr/local/bin/pandoc, and set options(pkg.sysreqs = FALSE) during pak::pak(...) so pak does not run an implicit apt-get -y install without --no-install-recommends (which previously pulled in ~1.8 GB of default-jdk, duplicate python3.13, libqgis-dev, grass84, saga, and duplicate pandoc).
    • Strip debug symbols (strip --strip-debug) from compiled R package .so binaries in the same RUN layer.
  • minimal/Dockerfile, minimal-slim/Dockerfile, suggests/Dockerfile, osgeo/Dockerfile:
    • Move strip --strip-debug and /tmp + pkgcache cleanup into the same RUN layer as package installation (preventing OverlayFS layer duplication).
    • Add libopenblas0-pthread to minimal-slim/Dockerfile and set multi-threaded GDAL environment variables across all images.
  • rocker-rpy/Dockerfile:
    • Use uv pip install instead of sequential pip3 to cut Python dependency installation time by >60%.
  • rust/Dockerfile:
    • Remove legacy Ubuntu jammy apt line (osmium-tool is natively in noble), use --profile minimal for rustup, and clean /tmp/tippecanoe and /root/.cargo/registry in the same RUN layer (~800 MB saved).
  • .github/workflows/build-docker.yml:
    • Enable GitHub Actions BuildKit layer caching (type=gha,mode=max).

Before vs. After Benchmarks

Metric Before (master) After (This PR) Improvement
geocompx/python image size 2,012.8 MB (2.01 GB) 1,755.3 MB (1.76 GB) -257.6 MB (-12.8%)
geocompx/pythonr image size 6,213.6 MB (6.21 GB) 4,150.5 MB (4.15 GB) -2,063.2 MB (-33.2%)
geocompx/python uncached build time 88.4 s 55.2 s -33.2 s (1.60x faster)
geocompx/pythonr uncached build time 225.0 s 186.7 s -38.3 s (1.21x faster)
R Linear Algebra (crossprod + solve, N=2500) 6.365 s (Netlib BLAS) 1.352 s (OpenBLAS pthread) 4.71x faster
R terra::project (4x3600x3600 cubic warp) 4.583 s 2.213 s 2.07x faster
Python geopandas.read_file (200k points GPKG) 0.236 s 0.104 s (PYOGRIO_USE_ARROW=1) 2.27x faster
Geocomputation with Python full book render 199.41 s 185.50 s -13.91 s (-7.0%)
Geocomputation with R Part I (Ch 1–7) render 120.16 s 115.56 s -4.60 s (-3.8%)

…ilds, and multi-threaded OpenBLAS/GDAL/Arrow defaults
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant