916 — HRRR Forecast to HEC-RAS: Executed Rain-on-Grid Qualification¶
This notebook performs a deterministic, end-to-end replay of a real NOAA HRRR
forecast through a native HEC-DSS gridded-precipitation boundary and HEC-RAS
7.0. It uses the archived 2024-08-08 15Z forecast cycle, selected because
it supplies positive but bounded rainfall over the Bald Eagle Creek example
model. The replay is deterministic; set RAS_COMMANDER_HRRR_CYCLE to another
YYYY-MM-DD HH:00 cycle to repeat the same workflow with a different archive
cycle.
The qualification gates are deliberately layered:
- download the HRRR APCP messages from the NOAA archive: the running accumulations, and the one-hour accumulation of each forecast hour;
- import the 18 one-hour messages with
RasDssGrid(the RAS Commander wrapper of the FAST-DSS importer) to 18 hourlyPER-CUMSHG grids covering the buffered model footprint; - validate every hourly grid against an independent full-source GDAL warp, and fail if any forcing cell lies outside the source coverage;
- create zero-rain and HRRR plans through public ras-commander APIs;
- inspect the temporary plan HDF created by HEC-RAS preprocessing;
- compute both plans in HEC-RAS 7.0;
- compare source, DSS, temporary-HDF, final-HDF, and hydraulic results; and
- scan runtime/BCO diagnostics and report mass-balance metrics.
Earlier versions of this notebook drove the forecast plan with grids from the
qpkit route (crop with GDAL, difference, reproject and write). On this cycle that
route leaves cells missing in every hourly grid and its values do not match a
full-source warp of the same HRRR messages, so it fails spatial qualification. It
is kept below only as a disclosed comparison; it no longer drives any run.
The model is instructional, not an operational forecast. Its supplied inflow, gate opening, terrain, infiltration, and diffusion-wave settings are retained. Always review the project, precipitation maps, runtime messages, and hydraulic results manually before using an automated workflow for engineering decisions.
Dependencies and execution policy¶
The forcing that drives the hydraulic runs is written by RasDssGrid, which needs
only ras-commander-fast-dss (installed by ras-commander[dss]): no qpkit, GDAL or
Java. herbie-data provides deterministic access to archived NOAA HRRR records, and
Rasterio (GDAL) provides the independent warp used to validate the forcing. The
disclosed comparison section also runs qpkit==0.1.0, a third-party package by
WEST Consultants, Inc. (Apache-2.0), whose release depends on the pydsstools
wheel. Install the notebook dependencies with:
pip install "numpy<2" "scipy<1.17" "ras-commander[notebooks,dss]" "qpkit==0.1.0" "herbie-data>=2024.8.0"
The notebook runs the two hydraulic plans serially. It does not launch concurrent HEC-RAS computations.
# Development mode: import the repository source when this notebook is run
# from the cloned repository. Set False to exercise an installed package.
USE_LOCAL_SOURCE = True
import sys
from pathlib import Path
repo_root = Path.cwd().parent if Path.cwd().name.lower() == "examples" else Path.cwd()
if USE_LOCAL_SOURCE and str(repo_root) not in sys.path:
sys.path.insert(0, str(repo_root))
import ras_commander
print(f"ras-commander {ras_commander.__version__} ({'repository source' if USE_LOCAL_SOURCE else 'installed package'})")
ras-commander 0.122.0 (repository source)
from contextlib import redirect_stderr, redirect_stdout
from datetime import datetime, timedelta, timezone
from importlib.metadata import version as package_version
from io import StringIO
import logging
import os
from pathlib import Path
import re
import shutil
import warnings
logging.getLogger("pydsstools.heclib.logging").setLevel(logging.WARNING)
warnings.filterwarnings(
"ignore",
message="A NumPy version >=2.0.0 and <2.8.0 is required for this version of SciPy.*",
category=UserWarning,
)
import geopandas as gpd
import h5py
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import rasterio
from rasterio.windows import Window, from_bounds
from rasterio.transform import from_origin
from rasterio.warp import Resampling, reproject, transform_bounds
from shapely.geometry import box
# Herbie prints a Unicode configuration message on first use. Redirecting it
# also keeps the executed notebook concise on Windows consoles.
with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
from herbie import Herbie
from qpkit import HRRRGridOptions, QPKit
from ras_commander import (
HdfBndry,
HdfInfiltration,
HdfMesh,
HdfProject,
HdfResultsMesh,
HdfResultsPlan,
RasDssGrid,
HdfStruc,
RasCmdr,
RasDss,
RasExamples,
RasPlan,
RasPreprocess,
RasUnsteady,
get_ras_exe,
init_ras_project,
)
from ras_commander.precip import PrecipHrrr
ipython = get_ipython()
if ipython is not None:
ipython.run_line_magic("matplotlib", "inline")
for logger_name in (
"httpx", "pydsstools", "pydsstools.heclib.logging", "qpkit",
"qpkit.services.dss.hrrr", "rasterio", "ras_commander",
):
logging.getLogger(logger_name).setLevel(logging.WARNING)
if package_version("qpkit") != "0.1.0":
raise RuntimeError(f"This qualified workflow requires qpkit 0.1.0; found {package_version('qpkit')}")
run_root = Path(os.environ.get(
"RAS_COMMANDER_EXAMPLE_RUN_ROOT",
repo_root / "working" / "example_916_hrrr_qualification",
)).resolve()
run_root.mkdir(parents=True, exist_ok=True)
def rel(path: Path) -> str:
path = Path(path).resolve()
try:
return path.relative_to(repo_root.resolve()).as_posix()
except ValueError:
return path.name
RAS_VERSION = "7.0"
FORECAST_HOURS = 18
PRECIPITATION_BUFFER_PERCENT = 50.0
COMPUTATION_INTERVAL = "5SEC"
MAX_WSEL_ERROR_FT = 1.00
MAX_CELLS_WSEL_ERROR_GT_0_1_FT = 5
MESH_NAME = "BaldEagleCr"
TEMPLATE_PLAN = "06"
TEMPLATE_UNSTEADY = "03"
TEMPLATE_GEOMETRY = "09"
ARCHIVE_CYCLE = os.environ.get("RAS_COMMANDER_HRRR_CYCLE", "2024-08-08 15:00")
selected_cycle = pd.Timestamp(ARCHIVE_CYCLE, tz="UTC").to_pydatetime()
simulation_start = selected_cycle.replace(tzinfo=None)
simulation_end = simulation_start + timedelta(hours=FORECAST_HOURS)
ras_exe = get_ras_exe(RAS_VERSION)
source_info = PrecipHrrr.get_info()
display(pd.DataFrame([{
"data source": source_info["provider"],
"HRRR cycle (UTC)": selected_cycle,
"simulation end (UTC)": simulation_end,
"forecast hours": FORECAST_HOURS,
"HEC-RAS": RAS_VERSION,
"qpkit": package_version("qpkit"),
"herbie-data": package_version("herbie-data"),
}]))
| data source | HRRR cycle (UTC) | simulation end (UTC) | forecast hours | HEC-RAS | qpkit | herbie-data | |
|---|---|---|---|---|---|---|---|
| 0 | NOAA / NCEP | 2024-08-08 15:00:00+00:00 | 2024-08-09 09:00:00 | 18 | 7.0 | 0.1.0 | 2026.10.0 |
Extract and inspect the HEC-RAS example model¶
The Bald Eagle Creek model supplies a real 2D mesh, terrain, Curve Number infiltration, inflow hydrograph, downstream boundaries, and Sayers Dam structure. The same hydraulic inputs are used in both plans; precipitation is the only intentional difference.
project_path = RasExamples.extract_project(
"BaldEagleCrkMulti2D",
output_path=run_root / "model",
suffix="hrrr_qualification",
)
ras = init_ras_project(
project_path,
ras_version=RAS_VERSION,
load_results_summary=False,
hide_intro=True,
)
template_plan_row = ras.plan_df.loc[
ras.plan_df["plan_number"].astype(str).str.zfill(2) == TEMPLATE_PLAN
].iloc[0]
assert str(template_plan_row["unsteady_number"]).zfill(2) == TEMPLATE_UNSTEADY
assert str(template_plan_row["geometry_number"]).zfill(2) == TEMPLATE_GEOMETRY
geometry_hdf = ras.project_folder / f"{ras.project_name}.g{TEMPLATE_GEOMETRY}.hdf"
template_unsteady_path = ras.project_folder / f"{ras.project_name}.u{TEMPLATE_UNSTEADY}"
project_bounds = tuple(float(v) for v in HdfProject.get_project_bounds_latlon(geometry_hdf, buffer_percent=0.0))
BOUNDS = tuple(float(v) for v in HdfProject.get_project_bounds_latlon(
geometry_hdf, buffer_percent=PRECIPITATION_BUFFER_PERCENT
))
mesh_areas = HdfMesh.get_mesh_areas(geometry_hdf)
mesh_attributes = HdfMesh.get_mesh_area_attributes(geometry_hdf)
mesh_cell_count = int(np.asarray(mesh_attributes.loc["Cell Count", "Value"]).reshape(-1)[0])
mesh_cell_points = HdfMesh.get_mesh_cell_points(geometry_hdf, include_ghost_cells=False)
bc_lines = HdfBndry.get_bc_lines(geometry_hdf)
structures = HdfStruc.get_structures(geometry_hdf)
infiltration_stats = HdfInfiltration.get_preprocessed_infiltration_stats(
geometry_hdf, mesh_name=MESH_NAME, variable="Curve Number"
)
inline_boundaries = RasUnsteady.get_inline_hydrograph_boundaries(
template_unsteady_path, ras_object=ras
)
gate = RasUnsteady.get_gate_openings(TEMPLATE_UNSTEADY, ras_object=ras)
assert gate["gate_name"] == "Gate #1" and gate["count"] == 100
assert np.allclose(gate["values"], 2.0)
display(pd.DataFrame([{
"model": ras.project_name,
"mesh": MESH_NAME,
"computational cells": mesh_cell_count,
"result/center records": len(mesh_cell_points),
"project WGS84 bounds": project_bounds,
"buffered HRRR bounds": BOUNDS,
"terrain": Path(ras.rasmap_df.iloc[0]["terrain_hdf_path"][0]).name,
"infiltration": Path(ras.rasmap_df.iloc[0]["infiltration_hdf_path"][0]).name,
}]).T.rename(columns={0: "value"}))
display(infiltration_stats.round(3))
flow_rows = inline_boundaries.loc[inline_boundaries["bc_type"] == "Flow Hydrograph"]
assert not flow_rows.empty
flow_values = np.asarray(flow_rows.iloc[0]["values"], dtype=float)
flow_hours = np.arange(len(flow_values), dtype=float)
fig, (ax_model, ax_flow) = plt.subplots(1, 2, figsize=(15, 5.8), constrained_layout=True)
mesh_areas.boundary.plot(ax=ax_model, color="#263238", linewidth=1.2, label="2D flow area")
if not bc_lines.empty:
bc_lines.plot(ax=ax_model, color="#d95f02", linewidth=2.0, label="Boundary-condition lines")
if not structures.empty:
structures.plot(ax=ax_model, color="#6a3d9a", linewidth=2.0, label="SA/2D structure")
ax_model.set(title="Bald Eagle Creek Model", xlabel="Easting (ft)", ylabel="Northing (ft)")
ax_model.set_aspect("equal")
ax_model.legend(loc="best")
display_hours = min(FORECAST_HOURS, len(flow_values) - 1)
ax_flow.plot(flow_hours[:display_hours + 1], flow_values[:display_hours + 1],
color="#2b6f9f", linewidth=2, marker="o", markersize=3)
ax_flow.set(title="Retained Synthetic Inflow", xlabel="Hours from simulation start", ylabel="Flow (cfs)")
ax_flow.grid(alpha=0.25)
plt.show()
| value | |
|---|---|
| model | BaldEagleDamBrk |
| mesh | BaldEagleCr |
| computational cells | 18066 |
| result/center records | 18066 |
| project WGS84 bounds | (-77.759149, 40.960673, -77.327516, 41.184944) |
| buffered HRRR bounds | (-77.867158, 40.904353, -77.219156, 41.240758) |
| terrain | Terrain50.hdf |
| infiltration | Infiltration.hdf |
| mesh_name | n_cells | min_val | max_val | mean_val | median_val | std_val | variable | |
|---|---|---|---|---|---|---|---|---|
| 0 | BaldEagleCr | 18066 | 36.0 | 100.0 | 73.645 | 73.0 | 14.803 | Curve Number |

Download and spatially bound the archived HRRR grids¶
HRRR surface APCP is a running accumulation from the forecast initialization. The notebook requests only that GRIB2 message for forecast hours 0–18 (Herbie retrieves the message's bytes from the archived file, not the whole file). The copies cropped with GDAL to the buffered model footprint below serve the source audit and the disclosed qpkit comparison; the forcing is imported from the complete messages.
archive_dir = run_root / "archive"
cropped_grib_dir = run_root / "cropped_grib"
archive_dir.mkdir(parents=True, exist_ok=True)
cropped_grib_dir.mkdir(parents=True, exist_ok=True)
gdal_data = Path(rasterio.__file__).resolve().parent / "gdal_data"
cropped_grib_files = []
cumulative_sources = []
source_rows = []
source_arrays = []
source_transform = None
source_crs = None
for forecast_hour in range(FORECAST_HOURS + 1):
search = (
":APCP:surface:0-0 day acc fcst:"
if forecast_hour == 0
else rf":APCP:surface:0-{forecast_hour} hour acc fcst:"
)
with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
hrrr = Herbie(
simulation_start,
model="hrrr",
product="sfc",
fxx=forecast_hour,
save_dir=archive_dir,
verbose=False,
)
source_path = Path(hrrr.download(search, save_dir=archive_dir, overwrite=False, verbose=False))
target_path = cropped_grib_dir / (
f"hrrr.{selected_cycle:%Y%m%d}.t{selected_cycle:%H}z."
f"wrfsfc.f{forecast_hour:02d}.APCP.grib2"
)
with rasterio.open(source_path) as src:
projected_bounds = transform_bounds("EPSG:4326", src.crs, *BOUNDS)
window = from_bounds(*projected_bounds, transform=src.transform)
window = Window(
max(0, int(np.floor(window.col_off)) - 2),
max(0, int(np.floor(window.row_off)) - 2),
min(src.width, int(np.ceil(window.width)) + 4),
min(src.height, int(np.ceil(window.height)) + 4),
)
values = src.read(1, window=window).astype("float64")
transform = src.window_transform(window)
crs = src.crs
if not target_path.exists():
profile = {
"driver": "GRIB", "dtype": "float64", "count": 1,
"width": values.shape[1], "height": values.shape[0],
"crs": crs, "transform": transform,
}
with rasterio.Env(GDAL_DATA=str(gdal_data)):
with rasterio.open(target_path, "w", **profile) as dst:
dst.write(values, 1)
cropped_grib_files.append(target_path)
cumulative_sources.append(source_path)
source_arrays.append(values)
source_transform = transform
source_crs = crs
source_rows.append({
"forecast_hour": forecast_hour,
"valid_time_utc": simulation_start + timedelta(hours=forecast_hour),
"mean_cumulative_mm": float(np.nanmean(values)),
"maximum_cumulative_mm": float(np.nanmax(values)),
"file_kib": target_path.stat().st_size / 1024,
})
source_audit = pd.DataFrame(source_rows)
source_stack = np.stack(source_arrays)
source_hourly = np.diff(source_stack, axis=0)
assert len(cropped_grib_files) == FORECAST_HOURS + 1
assert np.isfinite(source_stack).all()
assert float(np.nanmin(source_hourly)) >= -0.01
assert float(np.nanmean(source_stack[-1])) > 0.0
display(source_audit.round({
"mean_cumulative_mm": 3,
"maximum_cumulative_mm": 3,
"file_kib": 3,
}))
hourly_mean_mm = np.nanmean(source_hourly, axis=(1, 2))
hourly_max_mm = np.nanmax(source_hourly, axis=(1, 2))
fig, (ax_hyetograph, ax_map) = plt.subplots(1, 2, figsize=(14, 5), constrained_layout=True)
valid_ends = pd.date_range(simulation_start + timedelta(hours=1), periods=FORECAST_HOURS, freq="h")
ax_hyetograph.bar(valid_ends, hourly_mean_mm / 25.4, width=pd.Timedelta(minutes=45),
color="#2b6f9f", label="Footprint mean")
ax_hyetograph.plot(valid_ends, hourly_max_mm / 25.4, color="#b5453c", marker="o",
markersize=3, label="Footprint maximum")
ax_hyetograph.set(title="Archived HRRR Hourly APCP", xlabel="Interval end (UTC)", ylabel="Depth (in)")
ax_hyetograph.tick_params(axis="x", rotation=30)
ax_hyetograph.grid(axis="y", alpha=0.25)
ax_hyetograph.legend()
extent = [source_transform.c, source_transform.c + source_transform.a * source_stack.shape[2],
source_transform.f + source_transform.e * source_stack.shape[1], source_transform.f]
image = ax_map.imshow(source_stack[-1] / 25.4, extent=extent, origin="upper", cmap="Blues")
ax_map.set(title="18-Hour HRRR Accumulation", xlabel="HRRR Lambert easting (m)", ylabel="HRRR Lambert northing (m)")
fig.colorbar(image, ax=ax_map, label="Cumulative depth (in)")
plt.show()
print(
f"Source depth over the cropped HRRR window: mean={np.nanmean(source_stack[-1]) / 25.4:.3f} in, "
f"range={np.nanmin(source_stack[-1]) / 25.4:.3f} to {np.nanmax(source_stack[-1]) / 25.4:.3f} in"
)
| forecast_hour | valid_time_utc | mean_cumulative_mm | maximum_cumulative_mm | file_kib | |
|---|---|---|---|---|---|
| 0 | 0 | 2024-08-08 15:00:00 | 0.000 | 0.000 | 4.281 |
| 1 | 1 | 2024-08-08 16:00:00 | 0.743 | 4.325 | 4.281 |
| 2 | 2 | 2024-08-08 17:00:00 | 1.289 | 5.060 | 4.281 |
| 3 | 3 | 2024-08-08 18:00:00 | 1.536 | 5.070 | 4.281 |
| 4 | 4 | 2024-08-08 19:00:00 | 1.934 | 5.410 | 4.281 |
| 5 | 5 | 2024-08-08 20:00:00 | 2.038 | 5.520 | 4.281 |
| 6 | 6 | 2024-08-08 21:00:00 | 2.095 | 5.520 | 4.281 |
| 7 | 7 | 2024-08-08 22:00:00 | 2.108 | 5.530 | 4.281 |
| 8 | 8 | 2024-08-08 23:00:00 | 2.129 | 5.530 | 4.281 |
| 9 | 9 | 2024-08-09 00:00:00 | 2.157 | 5.530 | 4.281 |
| 10 | 10 | 2024-08-09 01:00:00 | 2.218 | 5.550 | 4.281 |
| 11 | 11 | 2024-08-09 02:00:00 | 2.373 | 6.100 | 4.281 |
| 12 | 12 | 2024-08-09 03:00:00 | 2.633 | 6.970 | 4.281 |
| 13 | 13 | 2024-08-09 04:00:00 | 4.317 | 12.950 | 4.281 |
| 14 | 14 | 2024-08-09 05:00:00 | 6.613 | 18.780 | 4.281 |
| 15 | 15 | 2024-08-09 06:00:00 | 10.906 | 25.900 | 4.281 |
| 16 | 16 | 2024-08-09 07:00:00 | 14.678 | 31.900 | 4.281 |
| 17 | 17 | 2024-08-09 08:00:00 | 20.172 | 37.860 | 4.281 |
| 18 | 18 | 2024-08-09 09:00:00 | 28.275 | 66.910 | 4.281 |

Source depth over the cropped HRRR window: mean=1.113 in, range=0.386 to 2.634 in
Native hourly forcing (drives the hydraulic runs)¶
HRRR's surface file also carries the one-hour accumulation of each forecast hour
(APCP from hour f−1 to f). Those 18 messages are imported with
RasDssGrid.import_gridded into SHG grids of 3,000 m on a lattice that covers
the buffered model footprint, with bilinear resampling. Each record is PER-CUM
over its own hour. Every grid must be complete: a missing forcing cell stops the
notebook instead of being read as zero rain.
FORCING_CELL_M = 3000.0
precipitation_dir = ras.project_folder / "Precipitation" / "hrrr_forecast"
precipitation_dir.mkdir(parents=True, exist_ok=True)
hrrr_dss_file = precipitation_dir / "hrrr_native_hourly.dss"
if hrrr_dss_file.exists():
hrrr_dss_file.unlink()
hourly_sources = []
for forecast_hour in range(1, FORECAST_HOURS + 1):
with redirect_stdout(StringIO()), redirect_stderr(StringIO()):
hrrr = Herbie(simulation_start, model="hrrr", product="sfc", fxx=forecast_hour,
save_dir=archive_dir, verbose=False)
hourly_sources.append(Path(hrrr.download(
rf":APCP:surface:{forecast_hour - 1}-{forecast_hour} hour acc fcst:",
save_dir=archive_dir, overwrite=False, verbose=False)))
print("one-hour HRRR messages:", len(hourly_sources), "|", RasDssGrid.list_variables(hourly_sources[-1]))
x0, y0, x1, y1 = gpd.GeoSeries([box(*BOUNDS)], crs="EPSG:4326").to_crs("EPSG:5070").total_bounds
forcing_lattice = (np.floor(x0 / FORCING_CELL_M) * FORCING_CELL_M, np.floor(y0 / FORCING_CELL_M) * FORCING_CELL_M,
np.ceil(x1 / FORCING_CELL_M) * FORCING_CELL_M, np.ceil(y1 / FORCING_CELL_M) * FORCING_CELL_M)
RasDssGrid.import_gridded(hourly_sources, hrrr_dss_file, product="total_precipitation",
envelope=forcing_lattice, cell_size=FORCING_CELL_M, method="bilinear",
parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "HRRR-APCP-1H"})
hrrr_catalog = RasDss.get_catalog(hrrr_dss_file)
assert len(hrrr_catalog) == FORECAST_HOURS
grid_rows = []
dss_arrays = []
first_grid = None
for pathname in hrrr_catalog["pathname"].astype(str):
grid = RasDss.read_grid(hrrr_dss_file, pathname)
values = np.asarray(grid["data"], dtype=float)
missing = int(np.count_nonzero(~np.isfinite(values)))
if missing:
raise RuntimeError(f"{missing} forcing cells are missing in {pathname}; the lattice exceeds the source coverage")
assert float(values.min()) >= 0.0
assert grid["units"].upper() == "MM" and grid["data_type"].upper() == "PER-CUM"
if first_grid is None:
first_grid = grid
dss_arrays.append(values)
grid_rows.append({
"start_time": pd.Timestamp(grid["start_time"]),
"end_time": pd.Timestamp(grid["end_time"]),
"missing_cells": missing,
"mean_mm": float(values.mean()),
"maximum_mm": float(values.max()),
"rows": grid["shape"][0],
"columns": grid["shape"][1],
"cell_size_m": float(grid["cell_size"]),
})
dss_audit = pd.DataFrame(grid_rows).sort_values("start_time").reset_index(drop=True)
order = np.argsort([pd.Timestamp(r["start_time"]) for r in grid_rows])
dss_stack = np.stack(dss_arrays)[order]
dss_total = dss_stack.sum(axis=0) # every cell is finite (checked above)
expected_starts = pd.date_range(simulation_start, periods=FORECAST_HOURS, freq="h")
assert pd.DatetimeIndex(dss_audit["start_time"]).equals(expected_starts)
assert pd.DatetimeIndex(dss_audit["end_time"]).equals(expected_starts + pd.Timedelta(hours=1))
assert float(dss_total.mean()) > 0.0
display(dss_audit.round({"mean_mm": 3, "maximum_mm": 3, "cell_size_m": 1}))
fig, (ax_interval, ax_total) = plt.subplots(1, 2, figsize=(14, 5), constrained_layout=True)
ax_interval.bar(dss_audit["end_time"], dss_audit["mean_mm"] / 25.4,
width=pd.Timedelta(minutes=45), color="#2b6f9f")
ax_interval.plot(dss_audit["end_time"], dss_audit["maximum_mm"] / 25.4,
color="#b5453c", marker="o", markersize=3)
ax_interval.set(title="Read-Back Native DSS Hourly APCP", xlabel="Interval end (UTC)", ylabel="Depth (in)")
ax_interval.tick_params(axis="x", rotation=30)
ax_interval.grid(axis="y", alpha=0.25)
image = ax_total.imshow(dss_total / 25.4, origin="lower", cmap="Blues") # RasDss grid rows are south-first
ax_total.set(title="Native DSS 18-Hour Total", xlabel="SHG column", ylabel="SHG row")
fig.colorbar(image, ax=ax_total, label="Depth (in)")
plt.show()
one-hour HRRR messages: 18 | ['Total_precipitation_surface_1_Hour_Accumulation']
| start_time | end_time | missing_cells | mean_mm | maximum_mm | rows | columns | cell_size_m | |
|---|---|---|---|---|---|---|---|---|
| 0 | 2024-08-08 15:00:00 | 2024-08-08 16:00:00 | 0 | 0.841 | 4.069 | 17 | 21 | 3000.0 |
| 1 | 2024-08-08 16:00:00 | 2024-08-08 17:00:00 | 0 | 0.525 | 2.120 | 17 | 21 | 3000.0 |
| 2 | 2024-08-08 17:00:00 | 2024-08-08 18:00:00 | 0 | 0.250 | 0.818 | 17 | 21 | 3000.0 |
| 3 | 2024-08-08 18:00:00 | 2024-08-08 19:00:00 | 0 | 0.479 | 1.429 | 17 | 21 | 3000.0 |
| 4 | 2024-08-08 19:00:00 | 2024-08-08 20:00:00 | 0 | 0.096 | 0.414 | 17 | 21 | 3000.0 |
| 5 | 2024-08-08 20:00:00 | 2024-08-08 21:00:00 | 0 | 0.057 | 0.568 | 17 | 21 | 3000.0 |
| 6 | 2024-08-08 21:00:00 | 2024-08-08 22:00:00 | 0 | 0.013 | 0.215 | 17 | 21 | 3000.0 |
| 7 | 2024-08-08 22:00:00 | 2024-08-08 23:00:00 | 0 | 0.021 | 0.161 | 17 | 21 | 3000.0 |
| 8 | 2024-08-08 23:00:00 | 2024-08-09 00:00:00 | 0 | 0.030 | 0.299 | 17 | 21 | 3000.0 |
| 9 | 2024-08-09 00:00:00 | 2024-08-09 01:00:00 | 0 | 0.068 | 0.716 | 17 | 21 | 3000.0 |
| 10 | 2024-08-09 01:00:00 | 2024-08-09 02:00:00 | 0 | 0.147 | 1.052 | 17 | 21 | 3000.0 |
| 11 | 2024-08-09 02:00:00 | 2024-08-09 03:00:00 | 0 | 0.213 | 1.195 | 17 | 21 | 3000.0 |
| 12 | 2024-08-09 03:00:00 | 2024-08-09 04:00:00 | 0 | 1.646 | 7.002 | 17 | 21 | 3000.0 |
| 13 | 2024-08-09 04:00:00 | 2024-08-09 05:00:00 | 0 | 2.143 | 6.947 | 17 | 21 | 3000.0 |
| 14 | 2024-08-09 05:00:00 | 2024-08-09 06:00:00 | 0 | 4.838 | 11.632 | 17 | 21 | 3000.0 |
| 15 | 2024-08-09 06:00:00 | 2024-08-09 07:00:00 | 0 | 3.619 | 10.012 | 17 | 21 | 3000.0 |
| 16 | 2024-08-09 07:00:00 | 2024-08-09 08:00:00 | 0 | 5.631 | 12.400 | 17 | 21 | 3000.0 |
| 17 | 2024-08-09 08:00:00 | 2024-08-09 09:00:00 | 0 | 9.008 | 33.199 | 17 | 21 | 3000.0 |

Validate the forcing against a full-source warp¶
Each hourly record is compared with an independent warp of its complete source message: Rasterio (GDAL) bilinear resampling onto the stored lattice (origin, cell size, rows and columns read back from the DSS record). The gate fails if any lattice cell lies outside the source coverage in either grid, or if any cell differs by more than 0.01 mm. Both implementations sample bilinearly at cell centres, so they agree to floating-point rounding; this is a check of placement, coverage and values, not of HRRR's skill.
def full_source_warp(source_path, record, resampling=Resampling.bilinear):
"""GDAL warp of a whole GRIB message onto a stored DSS grid's lattice, south row first."""
x_origin, y_origin = record["metadata"]["origin"]
n_rows, n_columns = record["shape"]
size = float(record["cell_size"])
target = np.full((n_rows, n_columns), np.nan, dtype="float32")
with rasterio.open(source_path) as source:
reproject(source.read(1), target, src_transform=source.transform, src_crs=source.crs,
dst_transform=from_origin(x_origin, y_origin + n_rows * size, size, size),
dst_crs=record["crs"], resampling=resampling, dst_nodata=np.nan)
return np.flipud(target).astype(float)
FORCING_TOLERANCE_MM = 0.01
oracle_rows = []
for source_path, pathname in zip(hourly_sources, sorted(hrrr_catalog["pathname"].astype(str),
key=lambda p: pd.Timestamp(RasDss.read_grid(hrrr_dss_file, p)["start_time"]))):
record = RasDss.read_grid(hrrr_dss_file, pathname)
stored = np.asarray(record["data"], dtype=float)
reference = full_source_warp(source_path, record)
difference = stored - reference
oracle_rows.append({
"hour ending": pd.Timestamp(record["end_time"]),
"DSS missing cells": int(np.count_nonzero(~np.isfinite(stored))),
"warp missing cells": int(np.count_nonzero(~np.isfinite(reference))),
"RMSE (mm)": float(np.sqrt(np.mean(difference ** 2))),
"max |difference| (mm)": float(np.max(np.abs(difference))),
})
oracle_audit = pd.DataFrame(oracle_rows)
display(oracle_audit)
forcing_inside_source = bool((oracle_audit["DSS missing cells"] == 0).all() and (oracle_audit["warp missing cells"] == 0).all())
forcing_matches_warp = bool(oracle_audit["max |difference| (mm)"].max() <= FORCING_TOLERANCE_MM)
if not forcing_inside_source:
raise RuntimeError("Forcing cells lie outside the HRRR source coverage")
if not forcing_matches_warp:
raise RuntimeError("Native hourly forcing differs from the full-source warp")
running_total = full_source_warp(cumulative_sources[-1], first_grid)
print(f"Sum of the 18 one-hour grids against the warped 0-18 h accumulation: "
f"max |difference| {np.max(np.abs(dss_total - running_total)):.4f} mm, "
f"mean {dss_total.mean():.3f} mm against {running_total.mean():.3f} mm")
| hour ending | DSS missing cells | warp missing cells | RMSE (mm) | max |difference| (mm) | |
|---|---|---|---|---|---|
| 0 | 2024-08-08 16:00:00 | 0 | 0 | 2.063541e-05 | 0.000095 |
| 1 | 2024-08-08 17:00:00 | 0 | 0 | 9.463915e-06 | 0.000048 |
| 2 | 2024-08-08 18:00:00 | 0 | 0 | 4.991016e-06 | 0.000017 |
| 3 | 2024-08-08 19:00:00 | 0 | 0 | 5.952392e-06 | 0.000019 |
| 4 | 2024-08-08 20:00:00 | 0 | 0 | 2.307025e-06 | 0.000012 |
| 5 | 2024-08-08 21:00:00 | 0 | 0 | 2.479833e-06 | 0.000014 |
| 6 | 2024-08-08 22:00:00 | 0 | 0 | 7.969182e-07 | 0.000006 |
| 7 | 2024-08-08 23:00:00 | 0 | 0 | 1.078369e-06 | 0.000004 |
| 8 | 2024-08-09 00:00:00 | 0 | 0 | 1.788583e-06 | 0.000010 |
| 9 | 2024-08-09 01:00:00 | 0 | 0 | 3.867094e-06 | 0.000024 |
| 10 | 2024-08-09 02:00:00 | 0 | 0 | 6.085983e-06 | 0.000033 |
| 11 | 2024-08-09 03:00:00 | 0 | 0 | 5.506271e-06 | 0.000031 |
| 12 | 2024-08-09 04:00:00 | 0 | 0 | 2.199022e-05 | 0.000109 |
| 13 | 2024-08-09 05:00:00 | 0 | 0 | 2.160909e-05 | 0.000122 |
| 14 | 2024-08-09 06:00:00 | 0 | 0 | 5.824731e-05 | 0.000238 |
| 15 | 2024-08-09 07:00:00 | 0 | 0 | 4.611430e-05 | 0.000146 |
| 16 | 2024-08-09 08:00:00 | 0 | 0 | 6.709775e-05 | 0.000262 |
| 17 | 2024-08-09 09:00:00 | 0 | 0 | 1.922543e-04 | 0.000769 |
Sum of the 18 one-hour grids against the warped 0-18 h accumulation: max |difference| 0.0148 mm, mean 29.625 mm against 29.626 mm
Disclosed comparison: the qpkit route (fails spatial qualification)¶
Earlier versions of this notebook drove the forecast plan with this route: GDAL
crops of the running accumulations, differenced, reprojected and written by
qpkit 0.1.0. It is run here only for comparison, into a separate file that no
plan uses. Missing cells are counted and kept missing; no sum or filter turns them
into zeros. The route is compared with a full-source warp of the 0-18 h running
accumulation on qpkit's own lattice, with both bilinear and nearest-neighbour
resampling, because qpkit resamples by nearest neighbour.
qpkit_dir = run_root / "qpkit_comparison"
qpkit_dir.mkdir(parents=True, exist_ok=True)
qpkit_dss_file = qpkit_dir / "hrrr_qpkit.dss"
grid_options = HRRRGridOptions.for_apcp(
part_a="SHG", part_b="BALD_EAGLE_CREEK", part_c="PRECIPITATION", part_f="HRRR-APCP-1H-QPKIT",
cell_size=3000.0, overwrite=True,
)
with rasterio.Env(GDAL_DATA=str(gdal_data)), QPKit(logger=logging.getLogger("qpkit")) as kit:
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message="You will likely lose important projection information.*",
category=UserWarning, module=r"pyproj\.crs\.crs")
qpkit_result = kit.hrrr_dss.write(tuple(cropped_grib_files), qpkit_dss_file,
grid_options=grid_options, dss_version=7, interval=1)
qpkit_catalog = sorted(RasDss.get_catalog(qpkit_dss_file)["pathname"].astype(str))
qpkit_grids = [RasDss.read_grid(qpkit_dss_file, p) for p in qpkit_catalog]
qpkit_stack = np.stack([np.asarray(g["data"], dtype=float) for g in qpkit_grids])
missing_per_frame = np.count_nonzero(~np.isfinite(qpkit_stack), axis=(1, 2))
missing_every = int(np.count_nonzero((~np.isfinite(qpkit_stack)).all(axis=0)))
missing_any = int(np.count_nonzero((~np.isfinite(qpkit_stack)).any(axis=0)))
qpkit_total = qpkit_stack.sum(axis=0) # NaN wherever any hour is missing
valid = np.isfinite(qpkit_total)
qpkit_rows = []
for label, resampling in (("bilinear", Resampling.bilinear), ("nearest", Resampling.nearest)):
reference = full_source_warp(cumulative_sources[-1], qpkit_grids[0], resampling)
both = valid & np.isfinite(reference)
error = qpkit_total[both] - reference[both]
qpkit_rows.append({"reference warp": label, "lattice cells": int(qpkit_total.size),
"cells missing in some hour": missing_any, "cells compared": int(both.sum()),
"RMSE (mm)": float(np.sqrt(np.mean(error ** 2))),
"max |difference| (mm)": float(np.max(np.abs(error)))})
print(f"qpkit wrote {len(qpkit_catalog)} records; missing cells per hour: {missing_per_frame.tolist()}; "
f"missing in every hour: {missing_every}")
display(pd.DataFrame(qpkit_rows))
qpkit_spatially_qualified = missing_any == 0 and max(r["max |difference| (mm)"] for r in qpkit_rows) <= FORCING_TOLERANCE_MM
print("qpkit route spatially qualified:", qpkit_spatially_qualified, "(not used by any plan)")
qpkit wrote 18 records; missing cells per hour: [69, 71, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69]; missing in every hour: 69
| reference warp | lattice cells | cells missing in some hour | cells compared | RMSE (mm) | max |difference| (mm) | |
|---|---|---|---|---|---|---|
| 0 | bilinear | 598 | 71 | 527 | 4.785354 | 17.503998 |
| 1 | nearest | 598 | 71 | 527 | 5.002228 | 20.129999 |
qpkit route spatially qualified: False (not used by any plan)
Context: running accumulation and a Stage IV analysis¶
Two further checks on the sources. The native import of HRRR's running 0-18 h accumulation is compared with the hourly forcing above, and NCEP's Stage IV 24-hour analysis from NOAA's daily archive (12Z 8 August to 12Z 9 August) is imported to the same lattice. Both GRIB decodes are compared with GDAL's decode of the same file. Stage IV covers a different period than the forecast, so it is shown for context, not as an equality check: a forecast is not expected to equal an analysis.
import gzip
import tarfile
import urllib.request
native_dir = run_root / "native_import"
native_dir.mkdir(parents=True, exist_ok=True)
running_dss = native_dir / "hrrr_native_running.dss"
if running_dss.exists():
running_dss.unlink()
running = RasDssGrid.import_gridded(cumulative_sources[-1], running_dss, product="total_precipitation",
envelope=forcing_lattice, cell_size=FORCING_CELL_M, method="bilinear",
parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "HRRR-APCP-RUNNING"})
running_record = RasDss.read_grid(running_dss, running.pathnames[0])
native_running = np.asarray(running_record["data"], dtype=float)
print("running accumulation:", running.pathnames[0], running_record["data_type"])
print(f"hourly sum against the running record: max |difference| {np.max(np.abs(dss_total - native_running)):.4f} mm")
running accumulation: /SHG/BALD_EAGLE_CREEK/PRECIPITATION/08AUG2024:1500/09AUG2024:0900/HRRR-APCP-RUNNING/ PER-CUM
hourly sum against the running record: max |difference| 0.0149 mm
def fetch_stage4(day: str) -> Path:
"""The CONUS 24-hour Stage IV file (12Z to 12Z) from NOAA's daily source archive."""
archive = native_dir / f"ncep_stage_iv_source_files_{day}.tar"
if not archive.exists():
url = ("https://water.noaa.gov/resources/downloads/precip/stageIV/"
f"{day[:4]}/{day[4:6]}/{day[6:]}/{archive.name}")
with urllib.request.urlopen(url, timeout=300) as response:
archive.write_bytes(response.read())
with tarfile.open(archive) as opened:
member = next(m for m in opened.getmembers() if Path(m.name).name.startswith("conus_"))
opened.extract(member, native_dir, filter="data")
source = native_dir / member.name
if source.suffix == ".gz":
plain = source.with_suffix("")
with gzip.open(source) as packed, plain.open("wb") as unpacked:
shutil.copyfileobj(packed, unpacked)
source = plain
return source
stage4_source = fetch_stage4("20240809")
stage4_dss = native_dir / "stage4_native.dss"
if stage4_dss.exists():
stage4_dss.unlink()
stage4 = RasDssGrid.import_gridded(stage4_source, stage4_dss, product="total_precipitation", envelope=forcing_lattice,
cell_size=FORCING_CELL_M, method="bilinear",
parts={"A": "SHG", "B": "BALD_EAGLE_CREEK", "F": "STAGE IV 24H"})
stage4_total = np.asarray(RasDss.read_grid(stage4_dss, stage4.pathnames[0])["data"], dtype=float)
print("Stage IV:", stage4_source.name, "->", stage4.pathnames[0])
checks = []
for label, source in (("HRRR 0-18 h", cumulative_sources[-1]), ("HRRR 17-18 h", hourly_sources[-1]), ("Stage IV", stage4_source)):
target = native_dir / f"{source.stem}.unwarped.dss"
if target.exists():
target.unlink()
result = RasDssGrid.import_gridded(source, target, product="total_precipitation", target_crs=None,
cell_size=None, align="vortex", parts={"A": "SOURCE", "F": "UNWARPED"})
native_values = np.asarray(RasDss.read_grid(target, result.pathnames[0])["data"], dtype=float)
with rasterio.open(source) as opened:
gdal_values = np.flipud(opened.read(1, masked=True).astype(float).filled(np.nan)) # DSS rows are south-first
both = np.isfinite(native_values) & np.isfinite(gdal_values)
checks.append({"product": label, "grid": f"{native_values.shape[1]} x {native_values.shape[0]}",
"same valid cells": bool(np.array_equal(np.isfinite(native_values), np.isfinite(gdal_values))),
"max |native - GDAL| (mm)": float(np.max(np.abs(native_values[both] - gdal_values[both])))})
display(pd.DataFrame(checks))
fig, axes = plt.subplots(1, 3, figsize=(16, 4.5), constrained_layout=True)
top = max(float(dss_total.max()), float(np.nanmax(qpkit_total)), float(np.nanmax(stage4_total)))
qpkit_shown = np.ma.masked_invalid(qpkit_total)
for ax, values, title in (
(axes[0], dss_total, "Native hourly forcing, 15Z 8 Aug + 18 h"),
(axes[1], qpkit_shown, "qpkit comparison (grey: missing cells)"),
(axes[2], stage4_total, "Stage IV analysis, 12Z 8 Aug to 12Z 9 Aug"),
):
ax.set_facecolor("#9e9e9e")
image = ax.imshow(values, origin="lower", cmap="Blues", vmin=0, vmax=top)
ax.set(title=title, xlabel="SHG column", ylabel="SHG row")
fig.colorbar(image, ax=axes, label="Depth (mm)")
plt.show()
Stage IV: conus_20240809_24h.grb2 -> /SHG/BALD_EAGLE_CREEK/PRECIPITATION/08AUG2024:1200/09AUG2024:1200/STAGE IV 24H/
| product | grid | same valid cells | max |native - GDAL| (mm) | |
|---|---|---|---|---|
| 0 | HRRR 0-18 h | 1799 x 1059 | True | 0.000015 |
| 1 | HRRR 17-18 h | 1799 x 1059 | True | 0.000008 |
| 2 | Stage IV | 1121 x 881 | True | 0.000015 |

Author baseline and forecast plans¶
Both plans are cloned from the same supplied plan and retain the same geometry, flow hydrographs, gate schedule, infiltration, and numerical settings. The baseline uses 0.0 in/hr constant precipitation; the forecast points to the DSS grid catalog created above. The DSS setter completes the three HEC-RAS-required meteorology modes while preserving explicit choices. The temporary/final HDF and runtime-message checks below remain authoritative.
baseline_unsteady = RasPlan.clone_unsteady(
TEMPLATE_UNSTEADY, new_title="HRRR No-Rain Baseline", ras_object=ras
)
forecast_unsteady = RasPlan.clone_unsteady(
TEMPLATE_UNSTEADY, new_title="HRRR Forecast", ras_object=ras
)
plan_intervals = {"computation": COMPUTATION_INTERVAL, "output": "10MIN", "mapping": "10MIN"}
baseline_plan = RasPlan.clone_plan(
TEMPLATE_PLAN, new_plan_shortid="HRRR No Rain", new_title="HRRR No-Rain Baseline",
geometry=TEMPLATE_GEOMETRY, unsteady_flow=baseline_unsteady, num_cores=4,
intervals=plan_intervals, ras_object=ras,
)
forecast_plan = RasPlan.clone_plan(
TEMPLATE_PLAN, new_plan_shortid="HRRR Forecast", new_title="HRRR Forecast",
geometry=TEMPLATE_GEOMETRY, unsteady_flow=forecast_unsteady, num_cores=4,
intervals=plan_intervals, ras_object=ras,
)
for plan_number in (baseline_plan, forecast_plan):
RasPlan.update_simulation_date(plan_number, simulation_start, simulation_end, ras_object=ras)
RasUnsteady.set_constant_precipitation(
baseline_unsteady, value=0.0, units="in/hr", ras_object=ras
)
first_dss_pathname = str(hrrr_catalog.sort_values("pathname").iloc[0]["pathname"])
RasUnsteady.configure_gridded_dss_precipitation(
forecast_unsteady,
dss_filename=str(hrrr_dss_file),
dss_pathname=first_dss_pathname,
ras_object=ras,
ratio=1.0,
interpolation="Bilinear",
)
baseline_config = RasUnsteady.get_met_precipitation_config(baseline_unsteady, ras_object=ras)
forecast_config = RasUnsteady.get_met_precipitation_config(forecast_unsteady, ras_object=ras)
assert baseline_config["mode"] == "Constant" and baseline_config["constant_value"] == 0.0
assert forecast_config["mode"] == "Gridded" and forecast_config["source"] == "DSS"
display(pd.DataFrame([
{"condition": "No-rain baseline", "plan": f"p{baseline_plan}", "unsteady": f"u{baseline_unsteady}",
"precipitation": "Constant 0.0 in/hr"},
{"condition": "Archived HRRR forecast", "plan": f"p{forecast_plan}", "unsteady": f"u{forecast_unsteady}",
"precipitation": "18 hourly DSS grids"},
]))
| condition | plan | unsteady | precipitation | |
|---|---|---|---|---|
| 0 | No-rain baseline | p07 | u04 | Constant 0.0 in/hr |
| 1 | Archived HRRR forecast | p08 | u05 | 18 hourly DSS grids |
Pre-compute validation: temporary plan HDF¶
HEC-RAS preprocessing creates the temporary plan HDF that the solver will use. This is the strongest pre-compute validation that gridded meteorology has been resolved onto the mesh. The check below requires finite, nonnegative, nonzero rainfall before a hydraulic compute is allowed to start.
prepared = RasPreprocess.preprocess_plan(forecast_plan, ras_object=ras, max_wait=600)
if not bool(prepared):
raise RuntimeError(prepared.error)
with h5py.File(prepared.tmp_hdf_path, "r") as hdf:
solver_rainfall = hdf["Event Conditions/Meteorology/Precipitation/Values"][:]
solver_timestamps = hdf["Event Conditions/Meteorology/Precipitation/Timestamp"][:]
assert solver_rainfall.ndim == 2 and solver_rainfall.shape[0] == len(solver_timestamps)
assert np.isfinite(solver_rainfall).all() and float(np.nanmin(solver_rainfall)) >= 0.0
assert float(np.nanmax(solver_rainfall)) > 0.0
solver_cumulative = np.cumsum(solver_rainfall, axis=0)
solver_total = solver_cumulative[-1]
dss_to_precompute_error = (
solver_total - np.flipud(dss_total).reshape(-1) / 25.4
)
assert float(np.nanmax(np.abs(dss_to_precompute_error))) <= 5e-6
solver_time = pd.date_range(
simulation_start + timedelta(hours=1),
simulation_end,
periods=solver_rainfall.shape[0],
)
fig, ax = plt.subplots(figsize=(11, 4.2))
ax.plot(solver_time, np.nanmean(solver_cumulative, axis=1), label="Raster mean", color="#2b6f9f")
ax.plot(solver_time, np.nanmax(solver_cumulative, axis=1), label="Raster maximum", color="#b5453c")
ax.set(title="HEC-RAS Temporary-HDF Cumulative Precipitation", xlabel="Model time (UTC)", ylabel="Cumulative depth (in)")
ax.grid(alpha=0.25)
ax.legend()
fig.autofmt_xdate()
plt.show()
precompute_summary = pd.DataFrame([{
"temporary-HDF rows": solver_rainfall.shape[0],
"DSS raster positions": solver_rainfall.shape[1],
"final mean depth (in)": float(np.nanmean(solver_total)),
"final maximum depth (in)": float(np.nanmax(solver_total)),
"maximum DSS-to-precompute error (in)": float(np.nanmax(np.abs(dss_to_precompute_error))),
}])
display(precompute_summary.round({
"final mean depth (in)": 4,
"final maximum depth (in)": 4,
"maximum DSS-to-precompute error (in)": 7,
}))

| temporary-HDF rows | DSS raster positions | final mean depth (in) | final maximum depth (in) | maximum DSS-to-precompute error (in) | |
|---|---|---|---|---|---|
| 0 | 18 | 357 | 1.1663 | 2.2392 | 0.000001 |
Serial HEC-RAS computations¶
The no-rain plan is computed first, followed by the HRRR plan. Runtime messages, water-surface convergence, raster coverage, and volume accounting are checked after each run. This supplied instructional mesh retains a small numerical pocket near Sayers Dam under spatial rainfall. The automated gate therefore requires complete execution, no coverage error, no more than five cells above 0.1 ft WSEL error, a maximum below 1.0 ft, and explicit spatial disclosure of every outlier. That bounded exception is not approval for engineering use.
hdf_output_variables = [
"Cell Hydraulic Depth", "Water Surface", "Cell Precipitation Rate",
"Cell Cumulative Precipitation Depth",
]
print(f"Computing p{baseline_plan} no-rain baseline in HEC-RAS {RAS_VERSION}...")
baseline_result = RasCmdr.compute_plan(
baseline_plan, ras_object=ras, force_rerun=True, force_geompre=True, verify=True,
use_optimal_hdf_settings=True, hdf_output_variables=hdf_output_variables,
)
if not bool(baseline_result):
raise RuntimeError("No-rain baseline compute failed")
print(f"Baseline p{baseline_plan}: complete")
print(f"Computing p{forecast_plan} HRRR forecast in HEC-RAS {RAS_VERSION}...")
forecast_result = RasCmdr.compute_plan(
forecast_plan, ras_object=ras, force_rerun=True, force_geompre=True, verify=True,
use_optimal_hdf_settings=True, hdf_output_variables=hdf_output_variables,
)
if not bool(forecast_result):
raise RuntimeError("HRRR forecast compute failed")
print(f"Forecast p{forecast_plan}: complete")
baseline_hdf = ras.project_folder / f"{ras.project_name}.p{baseline_plan}.hdf"
forecast_hdf = ras.project_folder / f"{ras.project_name}.p{forecast_plan}.hdf"
def compute_message_audit(condition: str, plan_number: str, hdf_path: Path) -> dict:
messages = HdfResultsPlan.get_compute_messages_hdf_only(hdf_path)
ws_error = HdfResultsMesh.get_mesh_max_ws_err(hdf_path)
error_column = "cell_maximum_water_surface_error"
volume = HdfResultsPlan.get_volume_accounting(hdf_path)
lower = messages.lower()
return {
"condition": condition,
"plan": f"p{plan_number}",
"complete_process": "complete process" in lower,
"precipitation_coverage_warning": "out-of-bounds" in lower and ("2d cells" in lower or "2d faces" in lower),
"maximum_cell_wsel_error_ft": float(ws_error[error_column].max()),
"cells_wsel_error_gt_0.01_ft": int(np.count_nonzero(ws_error[error_column] > 0.01)),
"cells_wsel_error_gt_0.1_ft": int(np.count_nonzero(ws_error[error_column] > 0.1)),
"volume_error_percent": float(volume.iloc[0]["Error Percent"]) if volume is not None and not volume.empty else np.nan,
}
compute_audit = pd.DataFrame([
compute_message_audit("No-rain baseline", baseline_plan, baseline_hdf),
compute_message_audit("Archived HRRR forecast", forecast_plan, forecast_hdf),
])
display(compute_audit.round(6))
assert compute_audit["complete_process"].all()
assert not compute_audit["precipitation_coverage_warning"].any()
assert float(compute_audit["maximum_cell_wsel_error_ft"].max()) <= MAX_WSEL_ERROR_FT
assert int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()) <= MAX_CELLS_WSEL_ERROR_GT_0_1_FT
if int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()):
print(
"Localized WSEL convergence outliers remain in this instructional model; "
"they are mapped below and require manual review."
)
Computing p07 no-rain baseline in HEC-RAS 7.0...
Baseline p07: complete
Computing p08 HRRR forecast in HEC-RAS 7.0...
Forecast p08: complete
| condition | plan | complete_process | precipitation_coverage_warning | maximum_cell_wsel_error_ft | cells_wsel_error_gt_0.01_ft | cells_wsel_error_gt_0.1_ft | volume_error_percent | |
|---|---|---|---|---|---|---|---|---|
| 0 | No-rain baseline | p07 | True | False | 0.009990 | 0 | 0 | 0.000054 |
| 1 | Archived HRRR forecast | p08 | True | False | 0.926102 | 4 | 3 | 0.001122 |
Localized WSEL convergence outliers remain in this instructional model; they are mapped below and require manual review.
Final-HDF precipitation consistency¶
The completed plan HDF is the post-compute gold standard. The baseline must remain exactly dry. The forecast must contain positive precipitation at every result position and agree with the precompute HDF. A spatial nearest-grid comparison against the read-back DSS cumulative grid is also reported; small differences are expected because HEC-RAS uses bilinear interpolation.
def precipitation_series(plan_hdf: Path):
rate = HdfResultsMesh.get_mesh_timeseries(
plan_hdf, MESH_NAME, "Cell Precipitation Rate", truncate=False
)
cumulative = HdfResultsMesh.get_mesh_timeseries(
plan_hdf, MESH_NAME, "Cell Cumulative Precipitation Depth", truncate=False
)
return rate, cumulative
baseline_rate, baseline_cumulative = precipitation_series(baseline_hdf)
forecast_rate, forecast_cumulative = precipitation_series(forecast_hdf)
baseline_rate_values = np.asarray(baseline_rate.values, dtype=float)
baseline_cumulative_values = np.asarray(baseline_cumulative.values, dtype=float)
forecast_rate_values = np.asarray(forecast_rate.values, dtype=float)
forecast_cumulative_values = np.asarray(forecast_cumulative.values, dtype=float)
assert np.isclose(np.nanmax(baseline_rate_values), 0.0, atol=1e-9)
assert np.isclose(np.nanmax(baseline_cumulative_values), 0.0, atol=1e-9)
assert float(np.nanmax(forecast_rate_values)) > 0.0
assert float(np.nanmax(forecast_cumulative_values)) > 0.0
final_cumulative = forecast_cumulative_values[-1]
# Results exclude ghost cells by default; match them with physical cell centers.
mesh_points = HdfMesh.get_mesh_cell_points(forecast_hdf, include_ghost_cells=False)
mesh_points = mesh_points.loc[mesh_points["mesh_name"] == MESH_NAME].sort_values("cell_id").reset_index(drop=True)
assert len(mesh_points) == final_cumulative.size
origin_x, origin_y = first_grid["metadata"]["origin"]
cell_size = float(first_grid["cell_size"])
rows, columns = first_grid["shape"]
mesh_in_dss = mesh_points.to_crs(first_grid["crs"])
point_x = mesh_in_dss.geometry.x.to_numpy()
point_y = mesh_in_dss.geometry.y.to_numpy()
sample_columns = np.floor((point_x - origin_x) / cell_size).astype(int)
sample_rows_from_bottom = np.floor((point_y - origin_y) / cell_size).astype(int)
sample_rows = sample_rows_from_bottom
inside_dss = (
(sample_columns >= 0) & (sample_columns < columns)
& (sample_rows >= 0) & (sample_rows < rows)
)
sampled_dss_mm = np.full(len(mesh_in_dss), np.nan, dtype=float)
sampled_dss_mm[inside_dss] = dss_total[
sample_rows[inside_dss], sample_columns[inside_dss]
]
sampled_dss_in = sampled_dss_mm / 25.4
mesh_on_valid_forcing = bool(inside_dss.all() and np.isfinite(sampled_dss_mm).all())
if not mesh_on_valid_forcing:
raise RuntimeError(f"{int((~inside_dss | ~np.isfinite(sampled_dss_mm)).sum())} mesh cells sample no valid forcing cell")
active_mesh_mask = final_cumulative > 1e-6
comparison_mask = (
inside_dss & active_mesh_mask
& np.isfinite(sampled_dss_in) & np.isfinite(final_cumulative)
)
comparison_error = final_cumulative[comparison_mask] - sampled_dss_in[comparison_mask]
assert comparison_mask.any()
assert float(np.nanmean(final_cumulative)) > 0.0
assert int(np.count_nonzero(active_mesh_mask)) == mesh_cell_count
assert abs(float(np.nanmean(comparison_error))) <= 0.01
assert float(np.sqrt(np.nanmean(comparison_error ** 2))) <= 0.05
forcing_summary = pd.DataFrame([{
"source mean (in)": float(np.nanmean(source_stack[-1]) / 25.4),
"DSS sampled mean (in)": float(np.nanmean(sampled_dss_in[comparison_mask])),
"temporary-HDF mean (in)": float(np.nanmean(solver_total)),
"final-HDF active-cell mean (in)": float(np.nanmean(final_cumulative[comparison_mask])),
"final-HDF all-position mean (in)": float(np.nanmean(final_cumulative)),
"DSS-to-final mean error (in)": float(np.nanmean(comparison_error)),
"DSS-to-final RMSE (in)": float(np.sqrt(np.nanmean(comparison_error ** 2))),
"cells/result positions with positive rain": int(np.count_nonzero(final_cumulative > 1e-6)),
}])
display(forcing_summary.round(5))
fig, (ax_scatter, ax_hist) = plt.subplots(1, 2, figsize=(13, 5), constrained_layout=True)
ax_scatter.scatter(sampled_dss_in[comparison_mask], final_cumulative[comparison_mask], s=7, alpha=0.35)
limits = [float(np.nanmin(sampled_dss_in[comparison_mask])), float(np.nanmax(sampled_dss_in[comparison_mask]))]
ax_scatter.plot(limits, limits, color="black", linestyle="--", label="1:1")
ax_scatter.set(title="DSS vs Final-HDF Cumulative Depth", xlabel="Nearest DSS grid depth (in)", ylabel="HEC-RAS final depth (in)")
ax_scatter.grid(alpha=0.25)
ax_scatter.legend()
ax_hist.hist(comparison_error, bins=35, color="#2b6f9f", edgecolor="white")
ax_hist.axvline(0.0, color="black", linestyle="--")
ax_hist.set(title="Interpolation Difference", xlabel="Final HDF minus nearest DSS cell (in)", ylabel="Result positions")
plt.show()
| source mean (in) | DSS sampled mean (in) | temporary-HDF mean (in) | final-HDF active-cell mean (in) | final-HDF all-position mean (in) | DSS-to-final mean error (in) | DSS-to-final RMSE (in) | cells/result positions with positive rain | |
|---|---|---|---|---|---|---|---|---|
| 0 | 1.1132 | 0.99512 | 1.16633 | 0.99528 | 0.99528 | 0.00016 | 0.02793 | 18066 |

Hydraulic response and runtime diagnostics¶
The depth comparison isolates the consequence of precipitation by subtracting the no-rain baseline. Runtime messages and BCO files are scanned for actual warning/error/fatal/failed/missing diagnostics. Automated scans are helpful, but they do not replace manual review in HEC-RAS Mapper and the computation log.
baseline_depth_ts = HdfResultsMesh.get_mesh_timeseries(
baseline_hdf, MESH_NAME, "Cell Hydraulic Depth", truncate=False
)
forecast_depth_ts = HdfResultsMesh.get_mesh_timeseries(
forecast_hdf, MESH_NAME, "Cell Hydraulic Depth", truncate=False
)
baseline_depth_values = np.asarray(baseline_depth_ts.values, dtype=float)
forecast_depth_values = np.asarray(forecast_depth_ts.values, dtype=float)
assert baseline_depth_values.shape == forecast_depth_values.shape
baseline_depth_values[baseline_depth_values < 0] = np.nan
forecast_depth_values[forecast_depth_values < 0] = np.nan
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message="All-NaN slice encountered", category=RuntimeWarning)
baseline_depth_max = np.nanmax(baseline_depth_values, axis=0)
forecast_depth_max = np.nanmax(forecast_depth_values, axis=0)
depth_comparison = mesh_points.copy()
depth_comparison["baseline_max_depth_ft"] = baseline_depth_max
depth_comparison["forecast_max_depth_ft"] = forecast_depth_max
depth_comparison["maximum_depth_envelope_change_ft"] = forecast_depth_max - baseline_depth_max
depth_delta = depth_comparison["maximum_depth_envelope_change_ft"].replace([np.inf, -np.inf], np.nan)
assert float(depth_delta.max()) > 0.0
representative_row = depth_comparison.loc[depth_delta.idxmax()]
representative_cell = int(representative_row["cell_id"])
baseline_times = pd.DatetimeIndex(pd.to_datetime(baseline_depth_ts.coords["time"].values))
forecast_times = pd.DatetimeIndex(pd.to_datetime(forecast_depth_ts.coords["time"].values))
hydraulic_summary = pd.DataFrame([{
"baseline cells ever wet >0.1 ft": int(np.count_nonzero(baseline_depth_max > 0.1)),
"forecast cells ever wet >0.1 ft": int(np.count_nonzero(forecast_depth_max > 0.1)),
"newly wet cells": int(np.count_nonzero((forecast_depth_max > 0.1) & ~(baseline_depth_max > 0.1))),
"maximum depth-envelope increase (ft)": float(depth_delta.max()),
"cells with depth increase >0.1 ft": int(np.count_nonzero(depth_delta > 0.1)),
"forecast cells with WSEL error >0.1 ft": int(
compute_audit.loc[
compute_audit["condition"] == "Archived HRRR forecast",
"cells_wsel_error_gt_0.1_ft",
].iloc[0]
),
}])
display(hydraulic_summary.round(4).T.rename(columns={0: "value"}))
fig, (ax_ts, ax_delta) = plt.subplots(1, 2, figsize=(15, 5.8), constrained_layout=True)
ax_ts.plot(baseline_times, baseline_depth_values[:, representative_cell], color="#2b6f9f", linewidth=2, label="No-rain baseline")
ax_ts.plot(forecast_times, forecast_depth_values[:, representative_cell], color="#b5453c", linewidth=2, label="HRRR forecast")
ax_ts.set(title=f"Hydraulic Depth at Cell {representative_cell}", xlabel="Model time (UTC)", ylabel="Depth (ft)")
ax_ts.tick_params(axis="x", rotation=30)
ax_ts.grid(alpha=0.25)
ax_ts.legend()
finite_abs = np.abs(depth_delta.dropna())
color_limit = max(float(finite_abs.max()), 0.01)
depth_comparison.plot(ax=ax_delta, column="maximum_depth_envelope_change_ft", cmap="RdBu_r",
vmin=-color_limit, vmax=color_limit, markersize=4, legend=True,
legend_kwds={"label": "Forecast minus baseline max-depth envelope (ft)"})
mesh_areas.boundary.plot(ax=ax_delta, color="#263238", linewidth=0.8)
forecast_ws_error = HdfResultsMesh.get_mesh_max_ws_err(forecast_hdf)
high_error_points = forecast_ws_error.loc[
forecast_ws_error["cell_maximum_water_surface_error"] > 0.1
]
if not high_error_points.empty:
high_error_points.plot(
ax=ax_delta,
color="#ffcc00",
edgecolor="black",
marker="o",
markersize=45,
label="WSEL error > 0.1 ft",
)
ax_delta.scatter(representative_row.geometry.x, representative_row.geometry.y,
color="black", marker="x", s=55, label=f"Cell {representative_cell}")
ax_delta.set(title="Calculated Hydraulic Response to HRRR Rainfall", xlabel="Easting (ft)", ylabel="Northing (ft)")
ax_delta.set_aspect("equal")
ax_delta.legend(loc="best")
plt.show()
diagnostic_pattern = re.compile(
r"\b(?:warning|fatal|failed|missing)\b|"
r"\berror(?::|\s+(?:reading|opening|processing|computing|occurred|encountered))",
flags=re.IGNORECASE,
)
bco_rows = []
for condition, plan_number in (("No-rain baseline", baseline_plan), ("Archived HRRR forecast", forecast_plan)):
bco_path = ras.project_folder / f"{ras.project_name}.bco{plan_number}"
assert bco_path.exists()
bco_text = bco_path.read_text(encoding="utf-8", errors="replace")
problem_lines = [line.strip() for line in bco_text.splitlines() if diagnostic_pattern.search(line)]
assert not problem_lines, problem_lines
precip_matches = re.findall(r"Precipitation Excess \(inches\)\s+([-+0-9.]+)", bco_text)
volume_matches = re.findall(
r"Error\s+Percent Error\s+\*+\s+\*+\s+([-+0-9.]+)\s+([-+0-9.]+)",
bco_text, flags=re.IGNORECASE | re.DOTALL,
)
bco_rows.append({
"condition": condition,
"diagnostic lines": len(problem_lines),
"precipitation excess (in)": float(precip_matches[-1]) if precip_matches else np.nan,
"volume error (acre-ft)": float(volume_matches[-1][0]) if volume_matches else np.nan,
"volume error (%)": float(volume_matches[-1][1]) if volume_matches else np.nan,
})
display(pd.DataFrame(bco_rows).round(6))
| value | |
|---|---|
| baseline cells ever wet >0.1 ft | 1978.0000 |
| forecast cells ever wet >0.1 ft | 11895.0000 |
| newly wet cells | 9917.0000 |
| maximum depth-envelope increase (ft) | 2.3649 |
| cells with depth increase >0.1 ft | 10497.0000 |
| forecast cells with WSEL error >0.1 ft | 3.0000 |

| condition | diagnostic lines | precipitation excess (in) | volume error (acre-ft) | volume error (%) | |
|---|---|---|---|---|---|
| 0 | No-rain baseline | 0 | NaN | 0.008607 | 0.000054 |
| 1 | Archived HRRR forecast | 0 | 0.2268 | -0.184500 | 0.001122 |
Completion checklist¶
Passing this checklist means the saved outputs are internally consistent for this example and environment. It is not a substitute for model calibration, forecast verification, or an engineer's manual HEC-RAS review.
completion_checks = pd.Series({
"real archived HRRR grids downloaded": len(cropped_grib_files) == FORECAST_HOURS + 1,
"source rainfall is finite and positive": np.isfinite(source_stack).all() and float(np.nanmax(source_stack[-1])) > 0,
"18 native DSS interval grids read back": len(dss_audit) == FORECAST_HOURS,
"forcing lattice lies inside the HRRR source coverage": forcing_inside_source,
"native forcing matches the full-source warp (0.01 mm)": forcing_matches_warp,
"every mesh cell samples a valid forcing cell": mesh_on_valid_forcing,
"DSS timing exactly spans the plan": dss_audit.iloc[0]["start_time"] == pd.Timestamp(simulation_start) and dss_audit.iloc[-1]["end_time"] == pd.Timestamp(simulation_end),
"temporary plan HDF contains nonzero rain": float(np.nanmax(solver_rainfall)) > 0,
"baseline and forecast computes completed": bool(baseline_result) and bool(forecast_result),
"baseline final HDF is dry": np.isclose(np.nanmax(baseline_cumulative_values), 0.0, atol=1e-9),
"forecast final HDF is wet": float(np.nanmax(final_cumulative)) > 0,
"DSS and temporary-HDF raster values agree": float(np.nanmax(np.abs(dss_to_precompute_error))) <= 5e-6,
"DSS and final-HDF spatial depths agree": (
abs(float(np.nanmean(comparison_error))) <= 0.01
and float(np.sqrt(np.nanmean(comparison_error ** 2))) <= 0.05
),
"forecast creates a positive hydraulic response": float(depth_delta.max()) > 0.0,
"HEC-RAS completion and bounded local-convergence gates pass": (
compute_audit["complete_process"].all()
and float(compute_audit["maximum_cell_wsel_error_ft"].max()) <= MAX_WSEL_ERROR_FT
and int(compute_audit["cells_wsel_error_gt_0.1_ft"].sum()) <= MAX_CELLS_WSEL_ERROR_GT_0_1_FT
),
}, name="passed").to_frame()
display(completion_checks)
assert completion_checks["passed"].all()
print("HRRR rain-on-grid qualification complete (native hourly forcing).")
print("qpkit comparison route spatially qualified:", qpkit_spatially_qualified, "- not used by any plan")
print(f"Cycle: {selected_cycle:%Y-%m-%d %H}Z; HEC-RAS {RAS_VERSION}; plans p{baseline_plan}/p{forecast_plan}")
print("Manual review of HEC-RAS Mapper, computation messages, mass balance, and hydraulic plausibility remains required.")
| passed | |
|---|---|
| real archived HRRR grids downloaded | True |
| source rainfall is finite and positive | True |
| 18 native DSS interval grids read back | True |
| forcing lattice lies inside the HRRR source coverage | True |
| native forcing matches the full-source warp (0.01 mm) | True |
| every mesh cell samples a valid forcing cell | True |
| DSS timing exactly spans the plan | True |
| temporary plan HDF contains nonzero rain | True |
| baseline and forecast computes completed | True |
| baseline final HDF is dry | True |
| forecast final HDF is wet | True |
| DSS and temporary-HDF raster values agree | True |
| DSS and final-HDF spatial depths agree | True |
| forecast creates a positive hydraulic response | True |
| HEC-RAS completion and bounded local-convergence gates pass | True |
HRRR rain-on-grid qualification complete (native hourly forcing).
qpkit comparison route spatially qualified: False - not used by any plan
Cycle: 2024-08-08 15Z; HEC-RAS 7.0; plans p07/p08
Manual review of HEC-RAS Mapper, computation messages, mass balance, and hydraulic plausibility remains required.