Skip to content

916 — HRRR Forecast to HEC-RAS: Executed Rain-on-Grid Qualification

This notebook performs a deterministic, end-to-end replay of a real NOAA HRRR forecast through a native HEC-DSS gridded-precipitation boundary and HEC-RAS 7.0. It uses the archived 2024-08-08 15Z forecast cycle, selected because it supplies positive but bounded rainfall over the Bald Eagle Creek example model. The replay is deterministic; set RAS_COMMANDER_HRRR_CYCLE to another YYYY-MM-DD HH:00 cycle to repeat the same workflow with a different archive cycle.

The qualification gates are deliberately layered:

  1. download the HRRR APCP messages from the NOAA archive: the running accumulations, and the one-hour accumulation of each forecast hour;
  2. import the 18 one-hour messages with RasDssGrid (the RAS Commander wrapper of the FAST-DSS importer) to 18 hourly PER-CUM SHG grids covering the buffered model footprint;
  3. validate every hourly grid against an independent full-source GDAL warp, and fail if any forcing cell lies outside the source coverage;
  4. create zero-rain and HRRR plans through public ras-commander APIs;
  5. inspect the temporary plan HDF created by HEC-RAS preprocessing;
  6. compute both plans in HEC-RAS 7.0;
  7. compare source, DSS, temporary-HDF, final-HDF, and hydraulic results; and
  8. scan runtime/BCO diagnostics and report mass-balance metrics.

Earlier versions of this notebook drove the forecast plan with grids from the qpkit route (crop with GDAL, difference, reproject and write). On this cycle that route leaves cells missing in every hourly grid and its values do not match a full-source warp of the same HRRR messages, so it fails spatial qualification. It is kept below only as a disclosed comparison; it no longer drives any run.

The model is instructional, not an operational forecast. Its supplied inflow, gate opening, terrain, infiltration, and diffusion-wave settings are retained. Always review the project, precipitation maps, runtime messages, and hydraulic results manually before using an automated workflow for engineering decisions.

Dependencies and execution policy

The forcing that drives the hydraulic runs is written by RasDssGrid, which needs only ras-commander-fast-dss (installed by ras-commander[dss]): no qpkit, GDAL or Java. herbie-data provides deterministic access to archived NOAA HRRR records, and Rasterio (GDAL) provides the independent warp used to validate the forcing. The disclosed comparison section also runs qpkit==0.1.0, a third-party package by WEST Consultants, Inc. (Apache-2.0), whose release depends on the pydsstools wheel. Install the notebook dependencies with:

Text Only
pip install "numpy<2" "scipy<1.17" "ras-commander[notebooks,dss]" "qpkit==0.1.0" "herbie-data>=2024.8.0"

The notebook runs the two hydraulic plans serially. It does not launch concurrent HEC-RAS computations.

Python
# Development mode: import the repository source when this notebook is run
# from the cloned repository. Set False to exercise an installed package.
USE_LOCAL_SOURCE = True

import sys
from pathlib import Path

repo_root = Path.cwd().parent if Path.cwd().name.lower() == "examples" else Path.cwd()
if USE_LOCAL_SOURCE and str(repo_root) not in sys.path:
    sys.path.insert(0, str(repo_root))

import ras_commander

print(f"ras-commander {ras_commander.__version__} ({'repository source' if USE_LOCAL_SOURCE else 'installed package'})")
Text Only
ras-commander 0.122.0 (repository source)
Python
from contextlib import redirect_stderr, redirect_stdout
from datetime import datetime, timedelta, timezone
from importlib.metadata import version as package_version
from io import StringIO
import logging
import os
from pathlib import Path
import re
import shutil
import warnings

logging.getLogger("pydsstools.heclib.logging").setLevel(logging.WARNING)

warnings.filterwarnings(
    "ignore",
    message="A NumPy version >=2.0.0 and <2.8.0 is required for this version of SciPy.*",
    category=UserWarning,
)

import geopandas as gpd
import h5py
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import rasterio
from rasterio.windows import Window, from_bounds
from rasterio.transform import from_origin
from rasterio.warp import Resampling, reproject, transform_bounds
from shapely.geometry import box

# Herbie prints a Unicode configuration message on first use. Redirecting it
# also keeps the executed notebook concise on Windows consoles.
with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
    from herbie import Herbie
    from qpkit import HRRRGridOptions, QPKit
from ras_commander import (
    HdfBndry,
    HdfInfiltration,
    HdfMesh,
    HdfProject,
    HdfResultsMesh,
    HdfResultsPlan,
    RasDssGrid,
    HdfStruc,
    RasCmdr,
    RasDss,
    RasExamples,
    RasPlan,
    RasPreprocess,
    RasUnsteady,
    get_ras_exe,
    init_ras_project,
)
from ras_commander.precip import PrecipHrrr

ipython = get_ipython()
if ipython is not None:
    ipython.run_line_magic("matplotlib", "inline")

for logger_name in (
    "httpx", "pydsstools", "pydsstools.heclib.logging", "qpkit",
    "qpkit.services.dss.hrrr", "rasterio", "ras_commander",
):
    logging.getLogger(logger_name).setLevel(logging.WARNING)

if package_version("qpkit") != "0.1.0":
    raise RuntimeError(f"This qualified workflow requires qpkit 0.1.0; found {package_version('qpkit')}")

run_root = Path(os.environ.get(
    "RAS_COMMANDER_EXAMPLE_RUN_ROOT",
    repo_root / "working" / "example_916_hrrr_qualification",
)).resolve()
run_root.mkdir(parents=True, exist_ok=True)

def rel(path: Path) -> str:
    path = Path(path).resolve()
    try:
        return path.relative_to(repo_root.resolve()).as_posix()
    except ValueError:
        return path.name

RAS_VERSION = "7.0"
FORECAST_HOURS = 18
PRECIPITATION_BUFFER_PERCENT = 50.0
COMPUTATION_INTERVAL = "5SEC"
MAX_WSEL_ERROR_FT = 1.00
MAX_CELLS_WSEL_ERROR_GT_0_1_FT = 5
MESH_NAME = "BaldEagleCr"
TEMPLATE_PLAN = "06"
TEMPLATE_UNSTEADY = "03"
TEMPLATE_GEOMETRY = "09"
ARCHIVE_CYCLE = os.environ.get("RAS_COMMANDER_HRRR_CYCLE", "2024-08-08 15:00")
selected_cycle = pd.Timestamp(ARCHIVE_CYCLE, tz="UTC").to_pydatetime()
simulation_start = selected_cycle.replace(tzinfo=None)
simulation_end = simulation_start + timedelta(hours=FORECAST_HOURS)
ras_exe = get_ras_exe(RAS_VERSION)
source_info = PrecipHrrr.get_info()

display(pd.DataFrame([{
    "data source": source_info["provider"],
    "HRRR cycle (UTC)": selected_cycle,
    "simulation end (UTC)": simulation_end,
    "forecast hours": FORECAST_HOURS,
    "HEC-RAS": RAS_VERSION,
    "qpkit": package_version("qpkit"),
    "herbie-data": package_version("herbie-data"),
}]))
data source HRRR cycle (UTC) simulation end (UTC) forecast hours HEC-RAS qpkit herbie-data
0 NOAA / NCEP 2024-08-08 15:00:00+00:00 2024-08-09 09:00:00 18 7.0 0.1.0 2026.10.0

Extract and inspect the HEC-RAS example model

The Bald Eagle Creek model supplies a real 2D mesh, terrain, Curve Number infiltration, inflow hydrograph, downstream boundaries, and Sayers Dam structure. The same hydraulic inputs are used in both plans; precipitation is the only intentional difference.

Python
project_path = RasExamples.extract_project(
    "BaldEagleCrkMulti2D",
    output_path=run_root / "model",
    suffix="hrrr_qualification",
)
ras = init_ras_project(
    project_path,
    ras_version=RAS_VERSION,
    load_results_summary=False,
    hide_intro=True,
)

template_plan_row = ras.plan_df.loc[
    ras.plan_df["plan_number"].astype(str).str.zfill(2) == TEMPLATE_PLAN
].iloc[0]
assert str(template_plan_row["unsteady_number"]).zfill(2) == TEMPLATE_UNSTEADY
assert str(template_plan_row["geometry_number"]).zfill(2) == TEMPLATE_GEOMETRY

geometry_hdf = ras.project_folder / f"{ras.project_name}.g{TEMPLATE_GEOMETRY}.hdf"
template_unsteady_path = ras.project_folder / f"{ras.project_name}.u{TEMPLATE_UNSTEADY}"
project_bounds = tuple(float(v) for v in HdfProject.get_project_bounds_latlon(geometry_hdf, buffer_percent=0.0))
BOUNDS = tuple(float(v) for v in HdfProject.get_project_bounds_latlon(
    geometry_hdf, buffer_percent=PRECIPITATION_BUFFER_PERCENT
))
mesh_areas = HdfMesh.get_mesh_areas(geometry_hdf)
mesh_attributes = HdfMesh.get_mesh_area_attributes(geometry_hdf)
mesh_cell_count = int(np.asarray(mesh_attributes.loc["Cell Count", "Value"]).reshape(-1)[0])
mesh_cell_points = HdfMesh.get_mesh_cell_points(geometry_hdf, include_ghost_cells=False)
bc_lines = HdfBndry.get_bc_lines(geometry_hdf)
structures = HdfStruc.get_structures(geometry_hdf)
infiltration_stats = HdfInfiltration.get_preprocessed_infiltration_stats(
    geometry_hdf, mesh_name=MESH_NAME, variable="Curve Number"
)
inline_boundaries = RasUnsteady.get_inline_hydrograph_boundaries(
    template_unsteady_path, ras_object=ras
)
gate = RasUnsteady.get_gate_openings(TEMPLATE_UNSTEADY, ras_object=ras)
assert gate["gate_name"] == "Gate #1" and gate["count"] == 100
assert np.allclose(gate["values"], 2.0)

display(pd.DataFrame([{
    "model": ras.project_name,
    "mesh": MESH_NAME,
    "computational cells": mesh_cell_count,
    "result/center records": len(mesh_cell_points),
    "project WGS84 bounds": project_bounds,
    "buffered HRRR bounds": BOUNDS,
    "terrain": Path(ras.rasmap_df.iloc[0]["terrain_hdf_path"][0]).name,
    "infiltration": Path(ras.rasmap_df.iloc[0]["infiltration_hdf_path"][0]).name,
}]).T.rename(columns={0: "value"}))
display(infiltration_stats.round(3))

flow_rows = inline_boundaries.loc[inline_boundaries["bc_type"] == "Flow Hydrograph"]
assert not flow_rows.empty
flow_values = np.asarray(flow_rows.iloc[0]["values"], dtype=float)
flow_hours = np.arange(len(flow_values), dtype=float)

fig, (ax_model, ax_flow) = plt.subplots(1, 2, figsize=(15, 5.8), constrained_layout=True)
mesh_areas.boundary.plot(ax=ax_model, color="#263238", linewidth=1.2, label="2D flow area")
if not bc_lines.empty:
    bc_lines.plot(ax=ax_model, color="#d95f02", linewidth=2.0, label="Boundary-condition lines")
if not structures.empty:
    structures.plot(ax=ax_model, color="#6a3d9a", linewidth=2.0, label="SA/2D structure")
ax_model.set(title="Bald Eagle Creek Model", xlabel="Easting (ft)", ylabel="Northing (ft)")
ax_model.set_aspect("equal")
ax_model.legend(loc="best")

display_hours = min(FORECAST_HOURS, len(flow_values) - 1)
ax_flow.plot(flow_hours[:display_hours + 1], flow_values[:display_hours + 1],
             color="#2b6f9f", linewidth=2, marker="o", markersize=3)
ax_flow.set(title="Retained Synthetic Inflow", xlabel="Hours from simulation start", ylabel="Flow (cfs)")
ax_flow.grid(alpha=0.25)
plt.show()
value
model BaldEagleDamBrk
mesh BaldEagleCr
computational cells 18066
result/center records 18066
project WGS84 bounds (-77.759149, 40.960673, -77.327516, 41.184944)
buffered HRRR bounds (-77.867158, 40.904353, -77.219156, 41.240758)
terrain Terrain50.hdf
infiltration Infiltration.hdf
mesh_name n_cells min_val max_val mean_val median_val std_val variable
0 BaldEagleCr 18066 36.0 100.0 73.645 73.0 14.803 Curve Number

png

Download and spatially bound the archived HRRR grids

HRRR surface APCP is a running accumulation from the forecast initialization. The notebook requests only that GRIB2 message for forecast hours 0–18 (Herbie retrieves the message's bytes from the archived file, not the whole file). The copies cropped with GDAL to the buffered model footprint below serve the source audit and the disclosed qpkit comparison; the forcing is imported from the complete messages.

Python
archive_dir = run_root / "archive"
cropped_grib_dir = run_root / "cropped_grib"
archive_dir.mkdir(parents=True, exist_ok=True)
cropped_grib_dir.mkdir(parents=True, exist_ok=True)

gdal_data = Path(rasterio.__file__).resolve().parent / "gdal_data"
cropped_grib_files = []
cumulative_sources = []
source_rows = []
source_arrays = []
source_transform = None
source_crs = None

for forecast_hour in range(FORECAST_HOURS + 1):
    search = (
        ":APCP:surface:0-0 day acc fcst:"
        if forecast_hour == 0
        else rf":APCP:surface:0-{forecast_hour} hour acc fcst:"
    )
    with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
        hrrr = Herbie(
            simulation_start,
            model="hrrr",
            product="sfc",
            fxx=forecast_hour,
            save_dir=archive_dir,
            verbose=False,
        )
        source_path = Path(hrrr.download(search, save_dir=archive_dir, overwrite=False, verbose=False))

    target_path = cropped_grib_dir / (
        f"hrrr.{selected_cycle:%Y%m%d}.t{selected_cycle:%H}z."
        f"wrfsfc.f{forecast_hour:02d}.APCP.grib2"
    )
    with rasterio.open(source_path) as src:
        projected_bounds = transform_bounds("EPSG:4326", src.crs, *BOUNDS)
        window = from_bounds(*projected_bounds, transform=src.transform)
        window = Window(
            max(0, int(np.floor(window.col_off)) - 2),
            max(0, int(np.floor(window.row_off)) - 2),
            min(src.width, int(np.ceil(window.width)) + 4),
            min(src.height, int(np.ceil(window.height)) + 4),
        )
        values = src.read(1, window=window).astype("float64")
        transform = src.window_transform(window)
        crs = src.crs

    if not target_path.exists():
        profile = {
            "driver": "GRIB", "dtype": "float64", "count": 1,
            "width": values.shape[1], "height": values.shape[0],
            "crs": crs, "transform": transform,
        }
        with rasterio.Env(GDAL_DATA=str(gdal_data)):
            with rasterio.open(target_path, "w", **profile) as dst:
                dst.write(values, 1)

    cropped_grib_files.append(target_path)
    cumulative_sources.append(source_path)
    source_arrays.append(values)
    source_transform = transform
    source_crs = crs
    source_rows.append({
        "forecast_hour": forecast_hour,
        "valid_time_utc": simulation_start + timedelta(hours=forecast_hour),
        "mean_cumulative_mm": float(np.nanmean(values)),
        "maximum_cumulative_mm": float(np.nanmax(values)),
        "file_kib": target_path.stat().st_size / 1024,
    })

source_audit = pd.DataFrame(source_rows)
source_stack = np.stack(source_arrays)
source_hourly = np.diff(source_stack, axis=0)
assert len(cropped_grib_files) == FORECAST_HOURS + 1
assert np.isfinite(source_stack).all()
assert float(np.nanmin(source_hourly)) >= -0.01
assert float(np.nanmean(source_stack[-1])) > 0.0
display(source_audit.round({
    "mean_cumulative_mm": 3,
    "maximum_cumulative_mm": 3,
    "file_kib": 3,
}))

hourly_mean_mm = np.nanmean(source_hourly, axis=(1, 2))
hourly_max_mm = np.nanmax(source_hourly, axis=(1, 2))
fig, (ax_hyetograph, ax_map) = plt.subplots(1, 2, figsize=(14, 5), constrained_layout=True)
valid_ends = pd.date_range(simulation_start + timedelta(hours=1), periods=FORECAST_HOURS, freq="h")
ax_hyetograph.bar(valid_ends, hourly_mean_mm / 25.4, width=pd.Timedelta(minutes=45),
                  color="#2b6f9f", label="Footprint mean")
ax_hyetograph.plot(valid_ends, hourly_max_mm / 25.4, color="#b5453c", marker="o",
                   markersize=3, label="Footprint maximum")
ax_hyetograph.set(title="Archived HRRR Hourly APCP", xlabel="Interval end (UTC)", ylabel="Depth (in)")
ax_hyetograph.tick_params(axis="x", rotation=30)
ax_hyetograph.grid(axis="y", alpha=0.25)
ax_hyetograph.legend()

extent = [source_transform.c, source_transform.c + source_transform.a * source_stack.shape[2],
          source_transform.f + source_transform.e * source_stack.shape[1], source_transform.f]
image = ax_map.imshow(source_stack[-1] / 25.4, extent=extent, origin="upper", cmap="Blues")
ax_map.set(title="18-Hour HRRR Accumulation", xlabel="HRRR Lambert easting (m)", ylabel="HRRR Lambert northing (m)")
fig.colorbar(image, ax=ax_map, label="Cumulative depth (in)")
plt.show()

print(
    f"Source depth over the cropped HRRR window: mean={np.nanmean(source_stack[-1]) / 25.4:.3f} in, "
    f"range={np.nanmin(source_stack[-1]) / 25.4:.3f} to {np.nanmax(source_stack[-1]) / 25.4:.3f} in"
)
forecast_hour valid_time_utc mean_cumulative_mm maximum_cumulative_mm file_kib
0 0 2024-08-08 15:00:00 0.000 0.000 4.281
1 1 2024-08-08 16:00:00 0.743 4.325 4.281
2 2 2024-08-08 17:00:00 1.289 5.060 4.281
3 3 2024-08-08 18:00:00 1.536 5.070 4.281
4 4 2024-08-08 19:00:00 1.934 5.410 4.281
5 5 2024-08-08 20:00:00 2.038 5.520 4.281
6 6 2024-08-08 21:00:00 2.095 5.520 4.281
7 7 2024-08-08 22:00:00 2.108 5.530 4.281
8 8 2024-08-08 23:00:00 2.129 5.530 4.281
9 9 2024-08-09 00:00:00 2.157 5.530 4.281
10 10 2024-08-09 01:00:00 2.218 5.550 4.281
11 11 2024-08-09 02:00:00 2.373 6.100 4.281
12 12 2024-08-09 03:00:00 2.633 6.970 4.281
13 13 2024-08-09 04:00:00 4.317 12.950 4.281
14 14 2024-08-09 05:00:00 6.613 18.780 4.281
15 15 2024-08-09 06:00:00 10.906 25.900 4.281
16 16 2024-08-09 07:00:00 14.678 31.900 4.281
17 17 2024-08-09 08:00:00 20.172 37.860 4.281
18 18 2024-08-09 09:00:00 28.275 66.910 4.281

png

Text Only
Source depth over the cropped HRRR window: mean=1.113 in, range=0.386 to 2.634 in

Native hourly forcing (drives the hydraulic runs)

HRRR's surface file also carries the one-hour accumulation of each forecast hour (APCP from hour f−1 to f). Those 18 messages are imported with RasDssGrid.import_gridded into SHG grids of 3,000 m on a lattice that covers the buffered model footprint, with bilinear resampling. Each record is PER-CUM over its own hour. Every grid must be complete: a missing forcing cell stops the notebook instead of being read as zero rain.

Python
FORCING_CELL_M = 3000.0
precipitation_dir = ras.project_folder / "Precipitation" / "hrrr_forecast"
precipitation_dir.mkdir(parents=True, exist_ok=True)
hrrr_dss_file = precipitation_dir / "hrrr_native_hourly.dss"
if hrrr_dss_file.exists():
    hrrr_dss_file.unlink()

hourly_sources = []
for forecast_hour in range(1, FORECAST_HOURS + 1):
    with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
        hrrr = Herbie(simulation_start, model="hrrr", product="sfc", fxx=forecast_hour,
                      save_dir=archive_dir, verbose=False)
        hourly_sources.append(Path(hrrr.download(
            rf":APCP:surface:{forecast_hour - 1}-{forecast_hour} hour acc fcst:",
            save_dir=archive_dir, overwrite=False, verbose=False)))
print("one-hour HRRR messages:", len(hourly_sources), "|", RasDssGrid.list_variables(hourly_sources[-1]))

x0, y0, x1, y1 = gpd.GeoSeries([box(*BOUNDS)], crs="EPSG:4326").to_crs("EPSG:5070").total_bounds
forcing_lattice = (np.floor(x0 / FORCING_CELL_M) * FORCING_CELL_M, np.floor(y0 / FORCING_CELL_M) * FORCING_CELL_M,
                   np.ceil(x1 / FORCING_CELL_M) * FORCING_CELL_M, np.ceil(y1 / FORCING_CELL_M) * FORCING_CELL_M)
RasDssGrid.import_gridded(hourly_sources, hrrr_dss_file, product="total_precipitation",
                          envelope=forcing_lattice, cell_size=FORCING_CELL_M, method="bilinear",
                          parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "HRRR-APCP-1H"})
hrrr_catalog = RasDss.get_catalog(hrrr_dss_file)
assert len(hrrr_catalog) == FORECAST_HOURS

grid_rows = []
dss_arrays = []
first_grid = None
for pathname in hrrr_catalog["pathname"].astype(str):
    grid = RasDss.read_grid(hrrr_dss_file, pathname)
    values = np.asarray(grid["data"], dtype=float)
    missing = int(np.count_nonzero(~np.isfinite(values)))
    if missing:
        raise RuntimeError(f"{missing} forcing cells are missing in {pathname}; the lattice exceeds the source coverage")
    assert float(values.min()) >= 0.0
    assert grid["units"].upper() == "MM" and grid["data_type"].upper() == "PER-CUM"
    if first_grid is None:
        first_grid = grid
    dss_arrays.append(values)
    grid_rows.append({
        "start_time": pd.Timestamp(grid["start_time"]),
        "end_time": pd.Timestamp(grid["end_time"]),
        "missing_cells": missing,
        "mean_mm": float(values.mean()),
        "maximum_mm": float(values.max()),
        "rows": grid["shape"][0],
        "columns": grid["shape"][1],
        "cell_size_m": float(grid["cell_size"]),
    })

dss_audit = pd.DataFrame(grid_rows).sort_values("start_time").reset_index(drop=True)
order = np.argsort([pd.Timestamp(r["start_time"]) for r in grid_rows])
dss_stack = np.stack(dss_arrays)[order]
dss_total = dss_stack.sum(axis=0)  # every cell is finite (checked above)
expected_starts = pd.date_range(simulation_start, periods=FORECAST_HOURS, freq="h")
assert pd.DatetimeIndex(dss_audit["start_time"]).equals(expected_starts)
assert pd.DatetimeIndex(dss_audit["end_time"]).equals(expected_starts + pd.Timedelta(hours=1))
assert float(dss_total.mean()) > 0.0
display(dss_audit.round({"mean_mm": 3, "maximum_mm": 3, "cell_size_m": 1}))

fig, (ax_interval, ax_total) = plt.subplots(1, 2, figsize=(14, 5), constrained_layout=True)
ax_interval.bar(dss_audit["end_time"], dss_audit["mean_mm"] / 25.4,
                width=pd.Timedelta(minutes=45), color="#2b6f9f")
ax_interval.plot(dss_audit["end_time"], dss_audit["maximum_mm"] / 25.4,
                 color="#b5453c", marker="o", markersize=3)
ax_interval.set(title="Read-Back Native DSS Hourly APCP", xlabel="Interval end (UTC)", ylabel="Depth (in)")
ax_interval.tick_params(axis="x", rotation=30)
ax_interval.grid(axis="y", alpha=0.25)
image = ax_total.imshow(dss_total / 25.4, origin="lower", cmap="Blues")  # RasDss grid rows are south-first
ax_total.set(title="Native DSS 18-Hour Total", xlabel="SHG column", ylabel="SHG row")
fig.colorbar(image, ax=ax_total, label="Depth (in)")
plt.show()
Text Only
one-hour HRRR messages: 18 | ['Total_precipitation_surface_1_Hour_Accumulation']
start_time end_time missing_cells mean_mm maximum_mm rows columns cell_size_m
0 2024-08-08 15:00:00 2024-08-08 16:00:00 0 0.841 4.069 17 21 3000.0
1 2024-08-08 16:00:00 2024-08-08 17:00:00 0 0.525 2.120 17 21 3000.0
2 2024-08-08 17:00:00 2024-08-08 18:00:00 0 0.250 0.818 17 21 3000.0
3 2024-08-08 18:00:00 2024-08-08 19:00:00 0 0.479 1.429 17 21 3000.0
4 2024-08-08 19:00:00 2024-08-08 20:00:00 0 0.096 0.414 17 21 3000.0
5 2024-08-08 20:00:00 2024-08-08 21:00:00 0 0.057 0.568 17 21 3000.0
6 2024-08-08 21:00:00 2024-08-08 22:00:00 0 0.013 0.215 17 21 3000.0
7 2024-08-08 22:00:00 2024-08-08 23:00:00 0 0.021 0.161 17 21 3000.0
8 2024-08-08 23:00:00 2024-08-09 00:00:00 0 0.030 0.299 17 21 3000.0
9 2024-08-09 00:00:00 2024-08-09 01:00:00 0 0.068 0.716 17 21 3000.0
10 2024-08-09 01:00:00 2024-08-09 02:00:00 0 0.147 1.052 17 21 3000.0
11 2024-08-09 02:00:00 2024-08-09 03:00:00 0 0.213 1.195 17 21 3000.0
12 2024-08-09 03:00:00 2024-08-09 04:00:00 0 1.646 7.002 17 21 3000.0
13 2024-08-09 04:00:00 2024-08-09 05:00:00 0 2.143 6.947 17 21 3000.0
14 2024-08-09 05:00:00 2024-08-09 06:00:00 0 4.838 11.632 17 21 3000.0
15 2024-08-09 06:00:00 2024-08-09 07:00:00 0 3.619 10.012 17 21 3000.0
16 2024-08-09 07:00:00 2024-08-09 08:00:00 0 5.631 12.400 17 21 3000.0
17 2024-08-09 08:00:00 2024-08-09 09:00:00 0 9.008 33.199 17 21 3000.0

png

Validate the forcing against a full-source warp

Each hourly record is compared with an independent warp of its complete source message: Rasterio (GDAL) bilinear resampling onto the stored lattice (origin, cell size, rows and columns read back from the DSS record). The gate fails if any lattice cell lies outside the source coverage in either grid, or if any cell differs by more than 0.01 mm. Both implementations sample bilinearly at cell centres, so they agree to floating-point rounding; this is a check of placement, coverage and values, not of HRRR's skill.

Python
def full_source_warp(source_path, record, resampling=Resampling.bilinear):
    """GDAL warp of a whole GRIB message onto a stored DSS grid's lattice, south row first."""
    x_origin, y_origin = record["metadata"]["origin"]
    n_rows, n_columns = record["shape"]
    size = float(record["cell_size"])
    target = np.full((n_rows, n_columns), np.nan, dtype="float32")
    with rasterio.open(source_path) as source:
        reproject(source.read(1), target, src_transform=source.transform, src_crs=source.crs,
                  dst_transform=from_origin(x_origin, y_origin + n_rows * size, size, size),
                  dst_crs=record["crs"], resampling=resampling, dst_nodata=np.nan)
    return np.flipud(target).astype(float)


FORCING_TOLERANCE_MM = 0.01
oracle_rows = []
for source_path, pathname in zip(hourly_sources, sorted(hrrr_catalog["pathname"].astype(str),
                                                       key=lambda p: pd.Timestamp(RasDss.read_grid(hrrr_dss_file, p)["start_time"]))):
    record = RasDss.read_grid(hrrr_dss_file, pathname)
    stored = np.asarray(record["data"], dtype=float)
    reference = full_source_warp(source_path, record)
    difference = stored - reference
    oracle_rows.append({
        "hour ending": pd.Timestamp(record["end_time"]),
        "DSS missing cells": int(np.count_nonzero(~np.isfinite(stored))),
        "warp missing cells": int(np.count_nonzero(~np.isfinite(reference))),
        "RMSE (mm)": float(np.sqrt(np.mean(difference ** 2))),
        "max |difference| (mm)": float(np.max(np.abs(difference))),
    })
oracle_audit = pd.DataFrame(oracle_rows)
display(oracle_audit)
forcing_inside_source = bool((oracle_audit["DSS missing cells"] == 0).all() and (oracle_audit["warp missing cells"] == 0).all())
forcing_matches_warp = bool(oracle_audit["max |difference| (mm)"].max() <= FORCING_TOLERANCE_MM)
if not forcing_inside_source:
    raise RuntimeError("Forcing cells lie outside the HRRR source coverage")
if not forcing_matches_warp:
    raise RuntimeError("Native hourly forcing differs from the full-source warp")

running_total = full_source_warp(cumulative_sources[-1], first_grid)
print(f"Sum of the 18 one-hour grids against the warped 0-18 h accumulation: "
      f"max |difference| {np.max(np.abs(dss_total - running_total)):.4f} mm, "
      f"mean {dss_total.mean():.3f} mm against {running_total.mean():.3f} mm")
hour ending DSS missing cells warp missing cells RMSE (mm) max |difference| (mm)
0 2024-08-08 16:00:00 0 0 2.063541e-05 0.000095
1 2024-08-08 17:00:00 0 0 9.463915e-06 0.000048
2 2024-08-08 18:00:00 0 0 4.991016e-06 0.000017
3 2024-08-08 19:00:00 0 0 5.952392e-06 0.000019
4 2024-08-08 20:00:00 0 0 2.307025e-06 0.000012
5 2024-08-08 21:00:00 0 0 2.479833e-06 0.000014
6 2024-08-08 22:00:00 0 0 7.969182e-07 0.000006
7 2024-08-08 23:00:00 0 0 1.078369e-06 0.000004
8 2024-08-09 00:00:00 0 0 1.788583e-06 0.000010
9 2024-08-09 01:00:00 0 0 3.867094e-06 0.000024
10 2024-08-09 02:00:00 0 0 6.085983e-06 0.000033
11 2024-08-09 03:00:00 0 0 5.506271e-06 0.000031
12 2024-08-09 04:00:00 0 0 2.199022e-05 0.000109
13 2024-08-09 05:00:00 0 0 2.160909e-05 0.000122
14 2024-08-09 06:00:00 0 0 5.824731e-05 0.000238
15 2024-08-09 07:00:00 0 0 4.611430e-05 0.000146
16 2024-08-09 08:00:00 0 0 6.709775e-05 0.000262
17 2024-08-09 09:00:00 0 0 1.922543e-04 0.000769
Text Only
Sum of the 18 one-hour grids against the warped 0-18 h accumulation: max |difference| 0.0148 mm, mean 29.625 mm against 29.626 mm

Disclosed comparison: the qpkit route (fails spatial qualification)

Earlier versions of this notebook drove the forecast plan with this route: GDAL crops of the running accumulations, differenced, reprojected and written by qpkit 0.1.0. It is run here only for comparison, into a separate file that no plan uses. Missing cells are counted and kept missing; no sum or filter turns them into zeros. The route is compared with a full-source warp of the 0-18 h running accumulation on qpkit's own lattice, with both bilinear and nearest-neighbour resampling, because qpkit resamples by nearest neighbour.

Python
qpkit_dir = run_root / "qpkit_comparison"
qpkit_dir.mkdir(parents=True, exist_ok=True)
qpkit_dss_file = qpkit_dir / "hrrr_qpkit.dss"
grid_options = HRRRGridOptions.for_apcp(
    part_a="SHG", part_b="BALD_EAGLE_CREEK", part_c="PRECIPITATION", part_f="HRRR-APCP-1H-QPKIT",
    cell_size=3000.0, overwrite=True,
)
with rasterio.Env(GDAL_DATA=str(gdal_data)), QPKit(logger=logging.getLogger("qpkit")) as kit:
    with warnings.catch_warnings():
        warnings.filterwarnings("ignore", message="You will likely lose important projection information.*",
                                category=UserWarning, module=r"pyproj\.crs\.crs")
        qpkit_result = kit.hrrr_dss.write(tuple(cropped_grib_files), qpkit_dss_file,
                                          grid_options=grid_options, dss_version=7, interval=1)
qpkit_catalog = sorted(RasDss.get_catalog(qpkit_dss_file)["pathname"].astype(str))
qpkit_grids = [RasDss.read_grid(qpkit_dss_file, p) for p in qpkit_catalog]
qpkit_stack = np.stack([np.asarray(g["data"], dtype=float) for g in qpkit_grids])
missing_per_frame = np.count_nonzero(~np.isfinite(qpkit_stack), axis=(1, 2))
missing_every = int(np.count_nonzero((~np.isfinite(qpkit_stack)).all(axis=0)))
missing_any = int(np.count_nonzero((~np.isfinite(qpkit_stack)).any(axis=0)))
qpkit_total = qpkit_stack.sum(axis=0)  # NaN wherever any hour is missing
valid = np.isfinite(qpkit_total)
qpkit_rows = []
for label, resampling in (("bilinear", Resampling.bilinear), ("nearest", Resampling.nearest)):
    reference = full_source_warp(cumulative_sources[-1], qpkit_grids[0], resampling)
    both = valid & np.isfinite(reference)
    error = qpkit_total[both] - reference[both]
    qpkit_rows.append({"reference warp": label, "lattice cells": int(qpkit_total.size),
                       "cells missing in some hour": missing_any, "cells compared": int(both.sum()),
                       "RMSE (mm)": float(np.sqrt(np.mean(error ** 2))),
                       "max |difference| (mm)": float(np.max(np.abs(error)))})
print(f"qpkit wrote {len(qpkit_catalog)} records; missing cells per hour: {missing_per_frame.tolist()}; "
      f"missing in every hour: {missing_every}")
display(pd.DataFrame(qpkit_rows))
qpkit_spatially_qualified = missing_any == 0 and max(r["max |difference| (mm)"] for r in qpkit_rows) <= FORCING_TOLERANCE_MM
print("qpkit route spatially qualified:", qpkit_spatially_qualified, "(not used by any plan)")
Text Only
qpkit wrote 18 records; missing cells per hour: [69, 71, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69]; missing in every hour: 69
reference warp lattice cells cells missing in some hour cells compared RMSE (mm) max |difference| (mm)
0 bilinear 598 71 527 4.785354 17.503998
1 nearest 598 71 527 5.002228 20.129999
Text Only
qpkit route spatially qualified: False (not used by any plan)

Context: running accumulation and a Stage IV analysis

Two further checks on the sources. The native import of HRRR's running 0-18 h accumulation is compared with the hourly forcing above, and NCEP's Stage IV 24-hour analysis from NOAA's daily archive (12Z 8 August to 12Z 9 August) is imported to the same lattice. Both GRIB decodes are compared with GDAL's decode of the same file. Stage IV covers a different period than the forecast, so it is shown for context, not as an equality check: a forecast is not expected to equal an analysis.

Python
import gzip
import tarfile
import urllib.request

native_dir = run_root / "native_import"
native_dir.mkdir(parents=True, exist_ok=True)
running_dss = native_dir / "hrrr_native_running.dss"
if running_dss.exists():
    running_dss.unlink()
running = RasDssGrid.import_gridded(cumulative_sources[-1], running_dss, product="total_precipitation",
                                    envelope=forcing_lattice, cell_size=FORCING_CELL_M, method="bilinear",
                                    parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "HRRR-APCP-RUNNING"})
running_record = RasDss.read_grid(running_dss, running.pathnames[0])
native_running = np.asarray(running_record["data"], dtype=float)
print("running accumulation:", running.pathnames[0], running_record["data_type"])
print(f"hourly sum against the running record: max |difference| {np.max(np.abs(dss_total - native_running)):.4f} mm")
Text Only
running accumulation: /SHG/BALD_EAGLE_CREEK/PRECIPITATION/08AUG2024:1500/09AUG2024:0900/HRRR-APCP-RUNNING/ PER-CUM
hourly sum against the running record: max |difference| 0.0149 mm
Python
def fetch_stage4(day: str) -> Path:
    """The CONUS 24-hour Stage IV file (12Z to 12Z) from NOAA's daily source archive."""
    archive = native_dir / f"ncep_stage_iv_source_files_{day}.tar"
    if not archive.exists():
        url = ("https://water.noaa.gov/resources/downloads/precip/stageIV/"
               f"{day[:4]}/{day[4:6]}/{day[6:]}/{archive.name}")
        with urllib.request.urlopen(url, timeout=300) as response:
            archive.write_bytes(response.read())
    with tarfile.open(archive) as opened:
        member = next(m for m in opened.getmembers() if Path(m.name).name.startswith("conus_"))
        opened.extract(member, native_dir, filter="data")
    source = native_dir / member.name
    if source.suffix == ".gz":
        plain = source.with_suffix("")
        with gzip.open(source) as packed, plain.open("wb") as unpacked:
            shutil.copyfileobj(packed, unpacked)
        source = plain
    return source


stage4_source = fetch_stage4("20240809")
stage4_dss = native_dir / "stage4_native.dss"
if stage4_dss.exists():
    stage4_dss.unlink()
stage4 = RasDssGrid.import_gridded(stage4_source, stage4_dss, product="total_precipitation", envelope=forcing_lattice,
                                   cell_size=FORCING_CELL_M, method="bilinear",
                                   parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "STAGE IV 24H"})
stage4_total = np.asarray(RasDss.read_grid(stage4_dss, stage4.pathnames[0])["data"], dtype=float)
print("Stage IV:", stage4_source.name, "->", stage4.pathnames[0])

checks = []
for label, source in (("HRRR 0-18 h", cumulative_sources[-1]), ("HRRR 17-18 h", hourly_sources[-1]), ("Stage IV", stage4_source)):
    target = native_dir / f"{source.stem}.unwarped.dss"
    if target.exists():
        target.unlink()
    result = RasDssGrid.import_gridded(source, target, product="total_precipitation", target_crs=None,
                                       cell_size=None, align="vortex", parts={"A": "SOURCE", "F": "UNWARPED"})
    native_values = np.asarray(RasDss.read_grid(target, result.pathnames[0])["data"], dtype=float)
    with rasterio.open(source) as opened:
        gdal_values = np.flipud(opened.read(1, masked=True).astype(float).filled(np.nan))  # DSS rows are south-first
    both = np.isfinite(native_values) & np.isfinite(gdal_values)
    checks.append({"product": label, "grid": f"{native_values.shape[1]} x {native_values.shape[0]}",
                   "same valid cells": bool(np.array_equal(np.isfinite(native_values), np.isfinite(gdal_values))),
                   "max |native - GDAL| (mm)": float(np.max(np.abs(native_values[both] - gdal_values[both])))})
display(pd.DataFrame(checks))

fig, axes = plt.subplots(1, 3, figsize=(16, 4.5), constrained_layout=True)
top = max(float(dss_total.max()), float(np.nanmax(qpkit_total)), float(np.nanmax(stage4_total)))
qpkit_shown = np.ma.masked_invalid(qpkit_total)
for ax, values, title in (
    (axes[0], dss_total, "Native hourly forcing, 15Z 8 Aug + 18 h"),
    (axes[1], qpkit_shown, "qpkit comparison (grey: missing cells)"),
    (axes[2], stage4_total, "Stage IV analysis, 12Z 8 Aug to 12Z 9 Aug"),
):
    ax.set_facecolor("#9e9e9e")
    image = ax.imshow(values, origin="lower", cmap="Blues", vmin=0, vmax=top)
    ax.set(title=title, xlabel="SHG column", ylabel="SHG row")
fig.colorbar(image, ax=axes, label="Depth (mm)")
plt.show()
Text Only
Stage IV: conus_20240809_24h.grb2 -> /SHG/BALD_EAGLE_CREEK/PRECIPITATION/08AUG2024:1200/09AUG2024:1200/STAGE IV 24H/
product grid same valid cells max |native - GDAL| (mm)
0 HRRR 0-18 h 1799 x 1059 True 0.000015
1 HRRR 17-18 h 1799 x 1059 True 0.000008
2 Stage IV 1121 x 881 True 0.000015

png

Author baseline and forecast plans

Both plans are cloned from the same supplied plan and retain the same geometry, flow hydrographs, gate schedule, infiltration, and numerical settings. The baseline uses 0.0 in/hr constant precipitation; the forecast points to the DSS grid catalog created above. The DSS setter completes the three HEC-RAS-required meteorology modes while preserving explicit choices. The temporary/final HDF and runtime-message checks below remain authoritative.

Python
baseline_unsteady = RasPlan.clone_unsteady(
    TEMPLATE_UNSTEADY, new_title="HRRR No-Rain Baseline", ras_object=ras
)
forecast_unsteady = RasPlan.clone_unsteady(
    TEMPLATE_UNSTEADY, new_title="HRRR Forecast", ras_object=ras
)
plan_intervals = {"computation": COMPUTATION_INTERVAL, "output": "10MIN", "mapping": "10MIN"}
baseline_plan = RasPlan.clone_plan(
    TEMPLATE_PLAN, new_plan_shortid="HRRR No Rain", new_title="HRRR No-Rain Baseline",
    geometry=TEMPLATE_GEOMETRY, unsteady_flow=baseline_unsteady, num_cores=4,
    intervals=plan_intervals, ras_object=ras,
)
forecast_plan = RasPlan.clone_plan(
    TEMPLATE_PLAN, new_plan_shortid="HRRR Forecast", new_title="HRRR Forecast",
    geometry=TEMPLATE_GEOMETRY, unsteady_flow=forecast_unsteady, num_cores=4,
    intervals=plan_intervals, ras_object=ras,
)
for plan_number in (baseline_plan, forecast_plan):
    RasPlan.update_simulation_date(plan_number, simulation_start, simulation_end, ras_object=ras)

RasUnsteady.set_constant_precipitation(
    baseline_unsteady, value=0.0, units="in/hr", ras_object=ras
)
first_dss_pathname = str(hrrr_catalog.sort_values("pathname").iloc[0]["pathname"])
RasUnsteady.configure_gridded_dss_precipitation(
    forecast_unsteady,
    dss_filename=str(hrrr_dss_file),
    dss_pathname=first_dss_pathname,
    ras_object=ras,
    ratio=1.0,
    interpolation="Bilinear",
)
baseline_config = RasUnsteady.get_met_precipitation_config(baseline_unsteady, ras_object=ras)
forecast_config = RasUnsteady.get_met_precipitation_config(forecast_unsteady, ras_object=ras)
assert baseline_config["mode"] == "Constant" and baseline_config["constant_value"] == 0.0
assert forecast_config["mode"] == "Gridded" and forecast_config["source"] == "DSS"
display(pd.DataFrame([
    {"condition": "No-rain baseline", "plan": f"p{baseline_plan}", "unsteady": f"u{baseline_unsteady}",
     "precipitation": "Constant 0.0 in/hr"},
    {"condition": "Archived HRRR forecast", "plan": f"p{forecast_plan}", "unsteady": f"u{forecast_unsteady}",
     "precipitation": "18 hourly DSS grids"},
]))
condition plan unsteady precipitation
0 No-rain baseline p07 u04 Constant 0.0 in/hr
1 Archived HRRR forecast p08 u05 18 hourly DSS grids

Pre-compute validation: temporary plan HDF

HEC-RAS preprocessing creates the temporary plan HDF that the solver will use. This is the strongest pre-compute validation that gridded meteorology has been resolved onto the mesh. The check below requires finite, nonnegative, nonzero rainfall before a hydraulic compute is allowed to start.

Python
prepared = RasPreprocess.preprocess_plan(forecast_plan, ras_object=ras, max_wait=600)
if not bool(prepared):
    raise RuntimeError(prepared.error)
with h5py.File(prepared.tmp_hdf_path, "r") as hdf:
    solver_rainfall = hdf["Event Conditions/Meteorology/Precipitation/Values"][:]
    solver_timestamps = hdf["Event Conditions/Meteorology/Precipitation/Timestamp"][:]
assert solver_rainfall.ndim == 2 and solver_rainfall.shape[0] == len(solver_timestamps)
assert np.isfinite(solver_rainfall).all() and float(np.nanmin(solver_rainfall)) >= 0.0
assert float(np.nanmax(solver_rainfall)) > 0.0
solver_cumulative = np.cumsum(solver_rainfall, axis=0)
solver_total = solver_cumulative[-1]
dss_to_precompute_error = (
    solver_total - np.flipud(dss_total).reshape(-1) / 25.4
)
assert float(np.nanmax(np.abs(dss_to_precompute_error))) <= 5e-6

solver_time = pd.date_range(
    simulation_start + timedelta(hours=1),
    simulation_end,
    periods=solver_rainfall.shape[0],
)
fig, ax = plt.subplots(figsize=(11, 4.2))
ax.plot(solver_time, np.nanmean(solver_cumulative, axis=1), label="Raster mean", color="#2b6f9f")
ax.plot(solver_time, np.nanmax(solver_cumulative, axis=1), label="Raster maximum", color="#b5453c")
ax.set(title="HEC-RAS Temporary-HDF Cumulative Precipitation", xlabel="Model time (UTC)", ylabel="Cumulative depth (in)")
ax.grid(alpha=0.25)
ax.legend()
fig.autofmt_xdate()
plt.show()
precompute_summary = pd.DataFrame([{
    "temporary-HDF rows": solver_rainfall.shape[0],
    "DSS raster positions": solver_rainfall.shape[1],
    "final mean depth (in)": float(np.nanmean(solver_total)),
    "final maximum depth (in)": float(np.nanmax(solver_total)),
    "maximum DSS-to-precompute error (in)": float(np.nanmax(np.abs(dss_to_precompute_error))),
}])
display(precompute_summary.round({
    "final mean depth (in)": 4,
    "final maximum depth (in)": 4,
    "maximum DSS-to-precompute error (in)": 7,
}))

png

temporary-HDF rows DSS raster positions final mean depth (in) final maximum depth (in) maximum DSS-to-precompute error (in)
0 18 357 1.1663 2.2392 0.000001

Serial HEC-RAS computations

The no-rain plan is computed first, followed by the HRRR plan. Runtime messages, water-surface convergence, raster coverage, and volume accounting are checked after each run. This supplied instructional mesh retains a small numerical pocket near Sayers Dam under spatial rainfall. The automated gate therefore requires complete execution, no coverage error, no more than five cells above 0.1 ft WSEL error, a maximum below 1.0 ft, and explicit spatial disclosure of every outlier. That bounded exception is not approval for engineering use.

Python
hdf_output_variables = [
    "Cell Hydraulic Depth", "Water Surface", "Cell Precipitation Rate",
    "Cell Cumulative Precipitation Depth",
]
print(f"Computing p{baseline_plan} no-rain baseline in HEC-RAS {RAS_VERSION}...")
baseline_result = RasCmdr.compute_plan(
    baseline_plan, ras_object=ras, force_rerun=True, force_geompre=True, verify=True,
    use_optimal_hdf_settings=True, hdf_output_variables=hdf_output_variables,
)
if not bool(baseline_result):
    raise RuntimeError("No-rain baseline compute failed")
print(f"Baseline p{baseline_plan}: complete")

print(f"Computing p{forecast_plan} HRRR forecast in HEC-RAS {RAS_VERSION}...")
forecast_result = RasCmdr.compute_plan(
    forecast_plan, ras_object=ras, force_rerun=True, force_geompre=True, verify=True,
    use_optimal_hdf_settings=True, hdf_output_variables=hdf_output_variables,
)
if not bool(forecast_result):
    raise RuntimeError("HRRR forecast compute failed")
print(f"Forecast p{forecast_plan}: complete")

baseline_hdf = ras.project_folder / f"{ras.project_name}.p{baseline_plan}.hdf"
forecast_hdf = ras.project_folder / f"{ras.project_name}.p{forecast_plan}.hdf"

def compute_message_audit(condition: str, plan_number: str, hdf_path: Path) -> dict:
    messages = HdfResultsPlan.get_compute_messages_hdf_only(hdf_path)
    ws_error = HdfResultsMesh.get_mesh_max_ws_err(hdf_path)
    error_column = "cell_maximum_water_surface_error"
    volume = HdfResultsPlan.get_volume_accounting(hdf_path)
    lower = messages.lower()
    return {
        "condition": condition,
        "plan": f"p{plan_number}",
        "complete_process": "complete process" in lower,
        "precipitation_coverage_warning": "out-of-bounds" in lower and ("2d cells" in lower or "2d faces" in lower),
        "maximum_cell_wsel_error_ft": float(ws_error[error_column].max()),
        "cells_wsel_error_gt_0.01_ft": int(np.count_nonzero(ws_error[error_column] > 0.01)),
        "cells_wsel_error_gt_0.1_ft": int(np.count_nonzero(ws_error[error_column] > 0.1)),
        "volume_error_percent": float(volume.iloc[0]["Error Percent"]) if volume is not None and not volume.empty else np.nan,
    }

compute_audit = pd.DataFrame([
    compute_message_audit("No-rain baseline", baseline_plan, baseline_hdf),
    compute_message_audit("Archived HRRR forecast", forecast_plan, forecast_hdf),
])
display(compute_audit.round(6))
assert compute_audit["complete_process"].all()
assert not compute_audit["precipitation_coverage_warning"].any()
assert float(compute_audit["maximum_cell_wsel_error_ft"].max()) <= MAX_WSEL_ERROR_FT
assert int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()) <= MAX_CELLS_WSEL_ERROR_GT_0_1_FT
if int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()):
    print(
        "Localized WSEL convergence outliers remain in this instructional model; "
        "they are mapped below and require manual review."
    )
Text Only
Computing p07 no-rain baseline in HEC-RAS 7.0...


Baseline p07: complete
Computing p08 HRRR forecast in HEC-RAS 7.0...


Forecast p08: complete
condition plan complete_process precipitation_coverage_warning maximum_cell_wsel_error_ft cells_wsel_error_gt_0.01_ft cells_wsel_error_gt_0.1_ft volume_error_percent
0 No-rain baseline p07 True False 0.009990 0 0 0.000054
1 Archived HRRR forecast p08 True False 0.926102 4 3 0.001122
Text Only
Localized WSEL convergence outliers remain in this instructional model; they are mapped below and require manual review.

Final-HDF precipitation consistency

The completed plan HDF is the post-compute gold standard. The baseline must remain exactly dry. The forecast must contain positive precipitation at every result position and agree with the precompute HDF. A spatial nearest-grid comparison against the read-back DSS cumulative grid is also reported; small differences are expected because HEC-RAS uses bilinear interpolation.

Python
def precipitation_series(plan_hdf: Path):
    rate = HdfResultsMesh.get_mesh_timeseries(
        plan_hdf, MESH_NAME, "Cell Precipitation Rate", truncate=False
    )
    cumulative = HdfResultsMesh.get_mesh_timeseries(
        plan_hdf, MESH_NAME, "Cell Cumulative Precipitation Depth", truncate=False
    )
    return rate, cumulative

baseline_rate, baseline_cumulative = precipitation_series(baseline_hdf)
forecast_rate, forecast_cumulative = precipitation_series(forecast_hdf)
baseline_rate_values = np.asarray(baseline_rate.values, dtype=float)
baseline_cumulative_values = np.asarray(baseline_cumulative.values, dtype=float)
forecast_rate_values = np.asarray(forecast_rate.values, dtype=float)
forecast_cumulative_values = np.asarray(forecast_cumulative.values, dtype=float)
assert np.isclose(np.nanmax(baseline_rate_values), 0.0, atol=1e-9)
assert np.isclose(np.nanmax(baseline_cumulative_values), 0.0, atol=1e-9)
assert float(np.nanmax(forecast_rate_values)) > 0.0
assert float(np.nanmax(forecast_cumulative_values)) > 0.0

final_cumulative = forecast_cumulative_values[-1]

# Results exclude ghost cells by default; match them with physical cell centers.
mesh_points = HdfMesh.get_mesh_cell_points(forecast_hdf, include_ghost_cells=False)
mesh_points = mesh_points.loc[mesh_points["mesh_name"] == MESH_NAME].sort_values("cell_id").reset_index(drop=True)
assert len(mesh_points) == final_cumulative.size

origin_x, origin_y = first_grid["metadata"]["origin"]
cell_size = float(first_grid["cell_size"])
rows, columns = first_grid["shape"]
mesh_in_dss = mesh_points.to_crs(first_grid["crs"])
point_x = mesh_in_dss.geometry.x.to_numpy()
point_y = mesh_in_dss.geometry.y.to_numpy()
sample_columns = np.floor((point_x - origin_x) / cell_size).astype(int)
sample_rows_from_bottom = np.floor((point_y - origin_y) / cell_size).astype(int)
sample_rows = sample_rows_from_bottom
inside_dss = (
    (sample_columns >= 0) & (sample_columns < columns)
    & (sample_rows >= 0) & (sample_rows < rows)
)
sampled_dss_mm = np.full(len(mesh_in_dss), np.nan, dtype=float)
sampled_dss_mm[inside_dss] = dss_total[
    sample_rows[inside_dss], sample_columns[inside_dss]
]
sampled_dss_in = sampled_dss_mm / 25.4
mesh_on_valid_forcing = bool(inside_dss.all() and np.isfinite(sampled_dss_mm).all())
if not mesh_on_valid_forcing:
    raise RuntimeError(f"{int((~inside_dss | ~np.isfinite(sampled_dss_mm)).sum())} mesh cells sample no valid forcing cell")
active_mesh_mask = final_cumulative > 1e-6
comparison_mask = (
    inside_dss & active_mesh_mask
    & np.isfinite(sampled_dss_in) & np.isfinite(final_cumulative)
)
comparison_error = final_cumulative[comparison_mask] - sampled_dss_in[comparison_mask]
assert comparison_mask.any()
assert float(np.nanmean(final_cumulative)) > 0.0
assert int(np.count_nonzero(active_mesh_mask)) == mesh_cell_count
assert abs(float(np.nanmean(comparison_error))) <= 0.01
assert float(np.sqrt(np.nanmean(comparison_error ** 2))) <= 0.05

forcing_summary = pd.DataFrame([{
    "source mean (in)": float(np.nanmean(source_stack[-1]) / 25.4),
    "DSS sampled mean (in)": float(np.nanmean(sampled_dss_in[comparison_mask])),
    "temporary-HDF mean (in)": float(np.nanmean(solver_total)),
    "final-HDF active-cell mean (in)": float(np.nanmean(final_cumulative[comparison_mask])),
    "final-HDF all-position mean (in)": float(np.nanmean(final_cumulative)),
    "DSS-to-final mean error (in)": float(np.nanmean(comparison_error)),
    "DSS-to-final RMSE (in)": float(np.sqrt(np.nanmean(comparison_error ** 2))),
    "cells/result positions with positive rain": int(np.count_nonzero(final_cumulative > 1e-6)),
}])
display(forcing_summary.round(5))

fig, (ax_scatter, ax_hist) = plt.subplots(1, 2, figsize=(13, 5), constrained_layout=True)
ax_scatter.scatter(sampled_dss_in[comparison_mask], final_cumulative[comparison_mask], s=7, alpha=0.35)
limits = [float(np.nanmin(sampled_dss_in[comparison_mask])), float(np.nanmax(sampled_dss_in[comparison_mask]))]
ax_scatter.plot(limits, limits, color="black", linestyle="--", label="1:1")
ax_scatter.set(title="DSS vs Final-HDF Cumulative Depth", xlabel="Nearest DSS grid depth (in)", ylabel="HEC-RAS final depth (in)")
ax_scatter.grid(alpha=0.25)
ax_scatter.legend()
ax_hist.hist(comparison_error, bins=35, color="#2b6f9f", edgecolor="white")
ax_hist.axvline(0.0, color="black", linestyle="--")
ax_hist.set(title="Interpolation Difference", xlabel="Final HDF minus nearest DSS cell (in)", ylabel="Result positions")
plt.show()
source mean (in) DSS sampled mean (in) temporary-HDF mean (in) final-HDF active-cell mean (in) final-HDF all-position mean (in) DSS-to-final mean error (in) DSS-to-final RMSE (in) cells/result positions with positive rain
0 1.1132 0.99512 1.16633 0.99528 0.99528 0.00016 0.02793 18066

png

Hydraulic response and runtime diagnostics

The depth comparison isolates the consequence of precipitation by subtracting the no-rain baseline. Runtime messages and BCO files are scanned for actual warning/error/fatal/failed/missing diagnostics. Automated scans are helpful, but they do not replace manual review in HEC-RAS Mapper and the computation log.

Python
baseline_depth_ts = HdfResultsMesh.get_mesh_timeseries(
    baseline_hdf, MESH_NAME, "Cell Hydraulic Depth", truncate=False
)
forecast_depth_ts = HdfResultsMesh.get_mesh_timeseries(
    forecast_hdf, MESH_NAME, "Cell Hydraulic Depth", truncate=False
)
baseline_depth_values = np.asarray(baseline_depth_ts.values, dtype=float)
forecast_depth_values = np.asarray(forecast_depth_ts.values, dtype=float)
assert baseline_depth_values.shape == forecast_depth_values.shape
baseline_depth_values[baseline_depth_values < 0] = np.nan
forecast_depth_values[forecast_depth_values < 0] = np.nan
with warnings.catch_warnings():
    warnings.filterwarnings("ignore", message="All-NaN slice encountered", category=RuntimeWarning)
    baseline_depth_max = np.nanmax(baseline_depth_values, axis=0)
    forecast_depth_max = np.nanmax(forecast_depth_values, axis=0)

depth_comparison = mesh_points.copy()
depth_comparison["baseline_max_depth_ft"] = baseline_depth_max
depth_comparison["forecast_max_depth_ft"] = forecast_depth_max
depth_comparison["maximum_depth_envelope_change_ft"] = forecast_depth_max - baseline_depth_max
depth_delta = depth_comparison["maximum_depth_envelope_change_ft"].replace([np.inf, -np.inf], np.nan)
assert float(depth_delta.max()) > 0.0

representative_row = depth_comparison.loc[depth_delta.idxmax()]
representative_cell = int(representative_row["cell_id"])
baseline_times = pd.DatetimeIndex(pd.to_datetime(baseline_depth_ts.coords["time"].values))
forecast_times = pd.DatetimeIndex(pd.to_datetime(forecast_depth_ts.coords["time"].values))

hydraulic_summary = pd.DataFrame([{
    "baseline cells ever wet >0.1 ft": int(np.count_nonzero(baseline_depth_max > 0.1)),
    "forecast cells ever wet >0.1 ft": int(np.count_nonzero(forecast_depth_max > 0.1)),
    "newly wet cells": int(np.count_nonzero((forecast_depth_max > 0.1) & ~(baseline_depth_max > 0.1))),
    "maximum depth-envelope increase (ft)": float(depth_delta.max()),
    "cells with depth increase >0.1 ft": int(np.count_nonzero(depth_delta > 0.1)),
    "forecast cells with WSEL error >0.1 ft": int(
        compute_audit.loc[
            compute_audit["condition"] == "Archived HRRR forecast",
            "cells_wsel_error_gt_0.1_ft",
        ].iloc[0]
    ),
}])
display(hydraulic_summary.round(4).T.rename(columns={0: "value"}))

fig, (ax_ts, ax_delta) = plt.subplots(1, 2, figsize=(15, 5.8), constrained_layout=True)
ax_ts.plot(baseline_times, baseline_depth_values[:, representative_cell], color="#2b6f9f", linewidth=2, label="No-rain baseline")
ax_ts.plot(forecast_times, forecast_depth_values[:, representative_cell], color="#b5453c", linewidth=2, label="HRRR forecast")
ax_ts.set(title=f"Hydraulic Depth at Cell {representative_cell}", xlabel="Model time (UTC)", ylabel="Depth (ft)")
ax_ts.tick_params(axis="x", rotation=30)
ax_ts.grid(alpha=0.25)
ax_ts.legend()

finite_abs = np.abs(depth_delta.dropna())
color_limit = max(float(finite_abs.max()), 0.01)
depth_comparison.plot(ax=ax_delta, column="maximum_depth_envelope_change_ft", cmap="RdBu_r",
                      vmin=-color_limit, vmax=color_limit, markersize=4, legend=True,
                      legend_kwds={"label": "Forecast minus baseline max-depth envelope (ft)"})
mesh_areas.boundary.plot(ax=ax_delta, color="#263238", linewidth=0.8)
forecast_ws_error = HdfResultsMesh.get_mesh_max_ws_err(forecast_hdf)
high_error_points = forecast_ws_error.loc[
    forecast_ws_error["cell_maximum_water_surface_error"] > 0.1
]
if not high_error_points.empty:
    high_error_points.plot(
        ax=ax_delta,
        color="#ffcc00",
        edgecolor="black",
        marker="o",
        markersize=45,
        label="WSEL error > 0.1 ft",
    )
ax_delta.scatter(representative_row.geometry.x, representative_row.geometry.y,
                 color="black", marker="x", s=55, label=f"Cell {representative_cell}")
ax_delta.set(title="Calculated Hydraulic Response to HRRR Rainfall", xlabel="Easting (ft)", ylabel="Northing (ft)")
ax_delta.set_aspect("equal")
ax_delta.legend(loc="best")
plt.show()

diagnostic_pattern = re.compile(
    r"\b(?:warning|fatal|failed|missing)\b|"
    r"\berror(?::|\s+(?:reading|opening|processing|computing|occurred|encountered))",
    flags=re.IGNORECASE,
)
bco_rows = []
for condition, plan_number in (("No-rain baseline", baseline_plan), ("Archived HRRR forecast", forecast_plan)):
    bco_path = ras.project_folder / f"{ras.project_name}.bco{plan_number}"
    assert bco_path.exists()
    bco_text = bco_path.read_text(encoding="utf-8", errors="replace")
    problem_lines = [line.strip() for line in bco_text.splitlines() if diagnostic_pattern.search(line)]
    assert not problem_lines, problem_lines
    precip_matches = re.findall(r"Precipitation Excess \(inches\)\s+([-+0-9.]+)", bco_text)
    volume_matches = re.findall(
        r"Error\s+Percent Error\s+\*+\s+\*+\s+([-+0-9.]+)\s+([-+0-9.]+)",
        bco_text, flags=re.IGNORECASE | re.DOTALL,
    )
    bco_rows.append({
        "condition": condition,
        "diagnostic lines": len(problem_lines),
        "precipitation excess (in)": float(precip_matches[-1]) if precip_matches else np.nan,
        "volume error (acre-ft)": float(volume_matches[-1][0]) if volume_matches else np.nan,
        "volume error (%)": float(volume_matches[-1][1]) if volume_matches else np.nan,
    })
display(pd.DataFrame(bco_rows).round(6))
value
baseline cells ever wet >0.1 ft 1978.0000
forecast cells ever wet >0.1 ft 11895.0000
newly wet cells 9917.0000
maximum depth-envelope increase (ft) 2.3649
cells with depth increase >0.1 ft 10497.0000
forecast cells with WSEL error >0.1 ft 3.0000

png

condition diagnostic lines precipitation excess (in) volume error (acre-ft) volume error (%)
0 No-rain baseline 0 NaN 0.008607 0.000054
1 Archived HRRR forecast 0 0.2268 -0.184500 0.001122

Completion checklist

Passing this checklist means the saved outputs are internally consistent for this example and environment. It is not a substitute for model calibration, forecast verification, or an engineer's manual HEC-RAS review.

Python
completion_checks = pd.Series({
    "real archived HRRR grids downloaded": len(cropped_grib_files) == FORECAST_HOURS + 1,
    "source rainfall is finite and positive": np.isfinite(source_stack).all() and float(np.nanmax(source_stack[-1])) > 0,
    "18 native DSS interval grids read back": len(dss_audit) == FORECAST_HOURS,
    "forcing lattice lies inside the HRRR source coverage": forcing_inside_source,
    "native forcing matches the full-source warp (0.01 mm)": forcing_matches_warp,
    "every mesh cell samples a valid forcing cell": mesh_on_valid_forcing,
    "DSS timing exactly spans the plan": dss_audit.iloc[0]["start_time"] == pd.Timestamp(simulation_start) and dss_audit.iloc[-1]["end_time"] == pd.Timestamp(simulation_end),
    "temporary plan HDF contains nonzero rain": float(np.nanmax(solver_rainfall)) > 0,
    "baseline and forecast computes completed": bool(baseline_result) and bool(forecast_result),
    "baseline final HDF is dry": np.isclose(np.nanmax(baseline_cumulative_values), 0.0, atol=1e-9),
    "forecast final HDF is wet": float(np.nanmax(final_cumulative)) > 0,
    "DSS and temporary-HDF raster values agree": float(np.nanmax(np.abs(dss_to_precompute_error))) <= 5e-6,
    "DSS and final-HDF spatial depths agree": (
        abs(float(np.nanmean(comparison_error))) <= 0.01
        and float(np.sqrt(np.nanmean(comparison_error ** 2))) <= 0.05
    ),
    "forecast creates a positive hydraulic response": float(depth_delta.max()) > 0.0,
    "HEC-RAS completion and bounded local-convergence gates pass": (
        compute_audit["complete_process"].all()
        and float(compute_audit["maximum_cell_wsel_error_ft"].max()) <= MAX_WSEL_ERROR_FT
        and int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()) <= MAX_CELLS_WSEL_ERROR_GT_0_1_FT
    ),
}, name="passed").to_frame()
display(completion_checks)
assert completion_checks["passed"].all()

print("HRRR rain-on-grid qualification complete (native hourly forcing).")
print("qpkit comparison route spatially qualified:", qpkit_spatially_qualified, "- not used by any plan")
print(f"Cycle: {selected_cycle:%Y-%m-%d %H}Z; HEC-RAS {RAS_VERSION}; plans p{baseline_plan}/p{forecast_plan}")
print("Manual review of HEC-RAS Mapper, computation messages, mass balance, and hydraulic plausibility remains required.")
passed
real archived HRRR grids downloaded True
source rainfall is finite and positive True
18 native DSS interval grids read back True
forcing lattice lies inside the HRRR source coverage True
native forcing matches the full-source warp (0.01 mm) True
every mesh cell samples a valid forcing cell True
DSS timing exactly spans the plan True
temporary plan HDF contains nonzero rain True
baseline and forecast computes completed True
baseline final HDF is dry True
forecast final HDF is wet True
DSS and temporary-HDF raster values agree True
DSS and final-HDF spatial depths agree True
forecast creates a positive hydraulic response True
HEC-RAS completion and bounded local-convergence gates pass True
Text Only
HRRR rain-on-grid qualification complete (native hourly forcing).
qpkit comparison route spatially qualified: False - not used by any plan
Cycle: 2024-08-08 15Z; HEC-RAS 7.0; plans p07/p08
Manual review of HEC-RAS Mapper, computation messages, mass balance, and hydraulic plausibility remains required.