Repository navigation
Add support for kokkos / kokkos-kernels and cuda to unified-dev env - #2088
Merged
Merged
Conversation
climbfuji
approved these changes
Aug 17, 2026
climbfuji
left a comment
Collaborator
There was a problem hiding this comment.
Nice work!
The CI tests don't cover the newly added packages, but from the conversation in the issue it is clear that the spack installs were successful and that downstream users successfully tested the installs.
AlexanderRichert-NOAA
approved these changes
Aug 18, 2026
Collaborator
|
Thanks for the work on the package and the pr review. I'll merge once the CI tests are done. |
AlexanderHrabski-NOAA
approved these changes
Aug 18, 2026
7 of 11 tasks
6 of 9 tasks
climbfuji
pushed a commit
that referenced
this pull request
Sep 8, 2026
…and MKL (#2118) This PR adds: - arborx to the UFS WM env package recipe ufs_weather_model_env/package.py - an entry for the oneAPI compiler module to the oneAPI MPI and MKL package specs modules section Package arborx is now added as a variant for ufs_weather_model_env, following on with the addition of kokkos and kokkos-kernels in PR #2088. Per discussion with Ben Koziol re: CICE/CECE/CATChem components, envs were built for oneAPI and GCC with arborx, and with and without CUDA support. Testing leads to a preference for adding arborx but excluding CUDA support at this time. Like kokkos / kokkos-kernels,arborx is added as a variant to ufs_weather_model_env/package.py (default=False) and is enabled only for ufs_weather_model_env. arborx / kokkos / kokkos-kernels are spec'd to build with %cxx=gcc because for H100 GPUs, kokkos relies on nvcc_wrapper for CMake CUDA support, and GCC is the primary target host compiler suggested by NVIDIA, as it is known to provide broad stability. The test envs bear this out. As well, for the backends chosen to be built for arborx and kokkos, GCC is preferred. Comments are provided in configs/sites/tier1/ursa/packages.yaml to turn on CUDA support if desired in the future. Lastly, for oneAPI package yaml files, an entry for the compiler is added to both the MPI and MKL specs modules section, so as to have MPI and MKL module successfully located and loaded at build time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add support for
kokkos / kokkos-kernelsandcuda(where appropriate) to theunified-devenv.kokkos / kokkos-kernelseach specify, per variants, two backends to be built:openmpandserial. Further, for host ursa, GPUs (H100) are available, thus+cuda cuda_arch=90(-DKokkos_ARCH_HOPPER90=ON) is specified.Version
5.1.1for each was spec'd per what users have been testing with; see issue #2081. This can likely change and still result in concretization / build success.Build Notes
kokkos / kokkos-kernelsbuild with bothGCCandIntel oneAPIcompilers whencudasupport is not requested. Whencudasupport is requested, they must be built withGCC, per this error message:That message comes from NVIDIA
nvcc,not from Kokkos.cudadoes not supporticx/icpxas a host compiler, and configuration perhost_config.h.Note also that #2066 is in play here; successful builds of test environments required not using
module:stanzas for host-specificcompiler+MPIand other packages (e.g.MKL) inyamlconfig files, in favor of pathing only.Dependencies
Depends on JCSDA/spack-packages#80
Issues addressed
Resolves #2081
Applications affected
UFS WM
Systems affected
gaea-c6: no cuda support (no GPUs available)ursa: with cuda support for H100sTesting
Checklist