Skip to content

BUG: All NOAA NOMADS Dependent Atmosphere Models Broken #933

Closed
Bug
Closed
BUG: All NOAA NOMADS Dependent Atmosphere Models Broken#933
Bug
Assignees
MateusStano
Labels
BugSomething isn't workingEnvironmentEnviroment Class related features

Description

@MateusStano
MateusStano
opened on Mar 8, 2026

OpenDAP support for NOMADS environmental data fetching no longer works

Image

You can find more information here PDF

The change is big and is definitely not easy to fix. From what I understand there are two ways we can go about fixing this

  • Use the Siphon library

    • We can use a tool called Siphon. It connects to a different type of server (THREDDS) that still works.
    • The Good: It is very similar to our old method. It keeps the data in a format that RocketPy already understands (netCDF4).
    • The Bad: We have to add siphon as a new requirement for RocketPy.

  • Use "GRIB Filter" with basic web requests

    • We can write code to download small data files directly from NOAA using simple web links.
    • More on this here: https://nomads.ncep.noaa.gov/info.php?page=gribfilter
    • The Good: We do not need to add any new libraries
    • The Bad: The files come in a format called "GRIB2." RocketPy usually uses "NetCDF." Reading GRIB2 files seems difficult and ive read that it can be cumbersome to do it on windows

It seems that using Siphon is the way to go here, but I might be missing an easier way to fix the issue

Activity

  1. MateusStano
    added
    BugSomething isn't working
    on Mar 8, 2026
  2. MateusStano
    added the
    Bug
    issue type on Mar 8, 2026
  3. Gui-FernandesBR
    removed the
    Bug
    issue type on Mar 8, 2026
  4. Gui-FernandesBR
    added the
    Bug
    issue type on Mar 8, 2026
  5. Gui-FernandesBR
    added
    EnvironmentEnviroment Class related features
    on Mar 8, 2026

  6. Gui-FernandesBR commented on Mar 8, 2026

    @Gui-FernandesBR
    Gui-FernandesBR
    on Mar 8, 2026
    Member

    This is my claude's review:

    Context

    What happened: NOAA terminated OpenDAP access on nomads.ncep.noaa.gov effective Feb 24, 2026 (SCN25-81). All RocketPy fetchers in fetchers.py call netCDF4.Dataset(opendap_url) which no longer works.

    Affected models: GFS, NAM, RAP, HIRESW, GEFS (all 5 active NOMADS-based models)

    NOAA's recommended migration paths:

    1. GRIB Filter: https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl?... (returns GRIB2 subsets)
    2. HTTPS bulk download: https://nomads.ncep.noaa.gov/pub/data/nccf/com/ (full GRIB2 files)

    Current RocketPy deps: numpy, scipy, matplotlib, netCDF4, requests, pytz, simplekml, dill

    Current Architecture (what breaks)

    fetchers.py:
  fetch_gfs_file_return_dataset()
    → netCDF4.Dataset("https://nomads.ncep.noaa.gov/dods/gfs_0p25/...")  ← DEAD
    → returns netCDF4.Dataset object

environment.py:
  set_atmospheric_model(type="Forecast", file="GFS")
    → calls fetcher → gets netCDF4.Dataset
    → process_forecast_reanalysis(dataset, dictionary)
      → data.variables["tmpprs"][time, :, lat, lon]  ← expects netCDF4 interface
      → bilinear_interpolation → Function objects

    Key files:

    • rocketpy/environment/fetchers.py (lines 95-406) — 6 broken fetchers
    • rocketpy/environment/environment.py (line 1675) — process_forecast_reanalysis() consumes netCDF4.Dataset
    • rocketpy/environment/weather_model_mapping.py — variable name mappings (tmpprs, ugrdprs, etc.)
    • rocketpy/environment/tools.py — helpers (find_time_index, bilinear_interpolation, etc.)

    Option Analysis

    Option 1: Siphon + Unidata THREDDS

    How: Use siphon library to query Unidata's THREDDS server (thredds.ucar.edu), which mirrors NOAA data and returns netCDF4 via NCSS (NetCDF Subset Service).

    python
    from siphon.catalog import TDSCatalog
cat = TDSCatalog('https://thredds.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_0p25deg/catalog.xml')
ncss = list(cat.datasets.values())[0].subset()
query = ncss.query()
query.lonlat_point(lon, lat)  # or lonlat_box for 2x2 grid
query.time(datetime)
query.accept('netcdf4')
query.variables('Temperature_isobaric', 'u-component_of_wind_isobaric', ...)
data = ncss.get_data(query)  # returns netCDF4.Dataset

    Verified available on THREDDS:

    • GFS 0.25° ✅ (grib/NCEP/GFS/Global_0p25deg)
    • NAM CONUS 12km ✅ (grib/NCEP/NAM/CONUS_12km)
    • RAP CONUS 13km ✅ (grib/NCEP/RAP/CONUS_13km)
    • GEFS — needs verification (likely available)
    • HIRESW — needs verification

    New dependency: siphon (pure Python: requests, protobuf, beautifulsoup4)

    Criteria | Rating -- | -- Refactor scope | Small — returns netCDF4, same interface as before New dependencies | Light — siphon is pure Python, pip-installable everywhere Windows support | Excellent — no C libraries needed Server reliability | Medium — single server (thredds.ucar.edu), run by Unidata/NSF Model coverage | Good — GFS, NAM, RAP confirmed; GEFS/HIRESW likely Data subsetting | Yes — NCSS supports point/box queries, variable/level selection Maintenance | Good — maintained by Unidata (same team behind netCDF4)

    Recommendation

    For RocketPy's use case (single-point atmospheric profiles for rocket simulation):

    Primary recommendation: Option 1 (Siphon + THREDDS) as an optional dependency rocketpy[forecast]

    Rationale:

    1. Minimal disruption — returns netCDF4, so process_forecast_reanalysis() needs only variable name mapping updates
    2. Pure Python — installs via pip install siphon on all platforms, no C library issues
    3. Lightweight — doesn't bloat RocketPy with weather-specific heavy deps
    4. Good enough reliability — Unidata THREDDS has been running reliably for years; used by pvlib, MetPy, and many other projects
    5. Clean API — catalog-based discovery instead of URL guessing

    If reliability is a top concern, consider Option 5 (Hybrid) with GRIB filter fallback, accepting the cfgrib dependency cost.

    If NOAA-official matters most, go with Option 2 (GRIB Filter + cfgrib), accepting the larger refactor and C library dependency.

    Implementation Sketch (for chosen approach)

    If Siphon is chosen:

    1. Add optional dependency group in pyproject.toml:
    toml
       [project.optional-dependencies]
   forecast = ["siphon>=0.9"]
    1. Rewrite fetcher functions in fetchers.py:
      • Replace netCDF4.Dataset(opendap_url) with Siphon NCSS queries
      • Return netCDF4.Dataset-compatible object
      • THREDDS catalog URLs for each model
    2. Update weather_model_mapping.py:
      • THREDDS variable names differ (e.g., Temperature_isobaric vs tmpprs)
      • Add THREDDS-specific mappings
    3. Minimal changes to environment.py:
      • Pass lat/lon to fetchers (currently not passed; fetchers return full dataset)
      • OR keep returning full dataset and let process_forecast_reanalysis handle extraction
    4. Update tests

    If GRIB Filter is chosen:

    1. Add optional dependency: cfgrib, eccodes, xarray
    2. Rewrite fetchers to download GRIB2 via HTTP + parse with cfgrib
    3. Rewrite data extraction in process_forecast_reanalysis (or create adapter)
    4. New variable name mappings for GRIB2 conventions
    5. Handle GRIB2-specific data structures (flat messages vs 4D arrays)
    6. Update tests

    Open Questions for Decision

    1. Which option do you want to proceed with?
    2. For Siphon: should we verify GEFS and HIRESW availability on THREDDS before committing?
    3. Should there be a graceful error message when the optional dep is not installed?
    4. Should we deprecate/remove any models in this fix, or maintain all 5?
  7. MateusStano
    self-assigned this
    on Mar 16, 2026
  8. MateusStano
    added 2 commits that reference this issue on Mar 18, 2026

    BUG: All NOAA NOMADS Dependent Atmosphere Models Broken

    c612e43

    ENH: get changes from BUG: All NOAA NOMADS Dependent Atmosphere Model…

    ab6dc3d
  9. MateusStano
    added a commit that references this issue on Apr 4, 2026

    ENH: get changes from BUG: All NOAA NOMADS Dependent Atmosphere Model…

    f8811c9
  10. MateusStano
    closed this as completedin cb15a39on Apr 6, 2026
  11. github-actions
    added a commit that references this issue on Apr 6, 2026

    HOTFIX: Migrate Forecasts to UCAR THREDDS (RocketPy-Team#943)

    cb15a39

  12. Atam3r commented on May 3, 2026

    @Atam3r
    Atam3r
    on May 3, 2026

    I have been looking into using the Monte Carlo simulations functionality in RocketPy recently, and was wondering what the current "expected" way of creating stochastic weather models is?

    The example code listed here returns a ValueError that "NOMADS OPeNDAP is deactivated" as expected due to the deactivation.
    It also states: "Please use another ensemble source or provide a compatible dataset path/URL explicitly."
    Is there any other valid and working ensemble source currently? Or should I look into downloading files locally and then using these?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugSomething isn't workingEnvironmentEnviroment Class related features

Type

Bug

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions