Operational Forecast Orchestration and Readiness¶
This executable overview builds a deterministic forecast-cycle manifest, validates the public APIs it will call, and audits the committed format-specific qualification notebooks. It does not duplicate their expensive downloads and HEC-RAS runs.
Use the linked notebooks for executed evidence:
- 916: archived HRRR GRIB2 to native DSS to HEC-RAS
- 923: STOFS-3D download, extraction, and coastal-stage authoring mechanics
- 924: MRMS NetCDF rain-on-grid
- 926: current WPC QPF to DSS to HEC-RAS
Notebook 915 is orchestration evidence, not independent format or HEC-RAS version qualification.
Production boundary¶
Operational forecasting must separate reproducible qualification from live cycle discovery. Committed outputs below use the archived 8 August 2024 15Z HRRR cycle qualified by notebook 916. A production scheduler should call live availability/download APIs at run time, record the resolved cycle and hashes, and fail closed when required inputs are unavailable or stale.
Manual review remains required for product age/completeness, units and time zones, terrain and forcing coverage, boundary datums, temporary-HDF inputs, all HEC-RAS runtime/BCO messages, mass balance, convergence, and final maps.
# Development mode: import repository source when run from a clone.
USE_LOCAL_SOURCE = True
import inspect
import sys
from dataclasses import dataclass
from datetime import timedelta
from pathlib import Path
from zoneinfo import ZoneInfo
import matplotlib.pyplot as plt
import nbformat
import numpy as np
import pandas as pd
repo_root = Path.cwd().parent if Path.cwd().name.lower() == "examples" else Path.cwd()
if USE_LOCAL_SOURCE and str(repo_root) not in sys.path:
sys.path.insert(0, str(repo_root))
import ras_commander
from ras_commander import RasCmdr, RasPlan
from ras_commander.boundaries import CoastalBoundary
from ras_commander.precip import PrecipHrrr
ipython = get_ipython()
if ipython is not None:
ipython.run_line_magic("matplotlib", "inline")
print(f"ras-commander {ras_commander.__version__} ({'repository source' if USE_LOCAL_SOURCE else 'installed package'})")
ras-commander 0.102.0 (repository source)
Inspect the forecast products without making a time-dependent download¶
get_info() is deterministic and confirms the product metadata used by the
orchestrator. Live discovery remains an explicit scheduler action through
PrecipHrrr.check_availability() and
CoastalBoundary.check_availability().
hrrr_info = PrecipHrrr.get_info()
stofs_info = CoastalBoundary.get_info()
product_catalog = pd.DataFrame([
{
"input": "HRRR precipitation",
"provider": hrrr_info["provider"],
"format": hrrr_info["format"],
"update": hrrr_info["update_frequency"],
"forecast horizon": f"{hrrr_info['forecast_horizon_standard']} standard",
"executed reference": "916: archived GRIB2 -> DSS -> temporary/final HDF",
},
{
"input": "STOFS-3D coastal stage",
"provider": stofs_info["provider"],
"format": stofs_info["format"],
"update": stofs_info["update_frequency"],
"forecast horizon": stofs_info["forecast_horizon"],
"executed reference": "923: NetCDF -> point WSE -> stage-authoring mechanics",
},
])
display(product_catalog)
assert product_catalog["provider"].notna().all()
assert product_catalog["executed reference"].str.startswith(("916:", "923:")).all()
| input | provider | format | update | forecast horizon | executed reference | |
|---|---|---|---|---|---|---|
| 0 | HRRR precipitation | NOAA / NCEP | GRIB2 | Hourly (every cycle) | 18 hours standard | 916: archived GRIB2 -> DSS -> temporary/final HDF |
| 1 | STOFS-3D coastal stage | NOAA / NOS / CO-OPS | NetCDF (field output), GRIB2 (surface fields) | Every 6 hours (00z, 06z, 12z, 18z) | 48 hours | 923: NetCDF -> point WSE -> stage-authoring me... |
Build a timezone-explicit cycle manifest¶
Forecast reference times are UTC. HEC-RAS plan dates are timezone-naive, so
the orchestrator must convert to the model's documented local time convention
before calling RasPlan.update_simulation_date().
@dataclass(frozen=True)
class ForecastCycle:
cycle_utc: pd.Timestamp
horizon_hours: int
model_timezone: str
plan_number: str
@property
def end_utc(self) -> pd.Timestamp:
return self.cycle_utc + pd.Timedelta(hours=self.horizon_hours)
@property
def plan_start(self) -> pd.Timestamp:
return self.cycle_utc.tz_convert(ZoneInfo(self.model_timezone)).tz_localize(None)
@property
def plan_end(self) -> pd.Timestamp:
return self.end_utc.tz_convert(ZoneInfo(self.model_timezone)).tz_localize(None)
cycle = ForecastCycle(
cycle_utc=pd.Timestamp("2024-08-08 15:00", tz="UTC"),
horizon_hours=18,
model_timezone="America/New_York",
plan_number="07",
)
manifest = pd.DataFrame([
("discover", 0.00, 0.25, "Resolve complete HRRR/STOFS cycles; reject stale inputs"),
("download", 0.25, 1.25, "Download into an isolated cycle directory; record hashes"),
("normalize", 1.25, 2.00, "Translate/validate precipitation and coastal WSE"),
("author", 2.00, 2.35, "Write gridded precipitation and stage boundaries via APIs"),
("precompute", 2.35, 2.75, "Inspect temporary plan HDF before compute"),
("compute", 2.75, 5.75, "Run HEC-RAS with verification enabled"),
("verify/publish", 5.75, 6.50, "Audit final HDF, diagnostics, maps, and provenance"),
], columns=["task", "start offset (h)", "end offset (h)", "acceptance gate"])
manifest["duration (h)"] = manifest["end offset (h)"] - manifest["start offset (h)"]
manifest["start UTC"] = cycle.cycle_utc + pd.to_timedelta(manifest["start offset (h)"], unit="h")
manifest["end UTC"] = cycle.cycle_utc + pd.to_timedelta(manifest["end offset (h)"], unit="h")
assert cycle.plan_start == pd.Timestamp("2024-08-08 11:00")
assert cycle.plan_end == pd.Timestamp("2024-08-09 05:00")
assert np.allclose(manifest["start offset (h)"].iloc[1:].to_numpy(), manifest["end offset (h)"].iloc[:-1].to_numpy())
display(pd.Series({
"archived qualification cycle": cycle.cycle_utc,
"forecast end UTC": cycle.end_utc,
"model timezone": cycle.model_timezone,
"HEC-RAS plan start": cycle.plan_start,
"HEC-RAS plan end": cycle.plan_end,
"plan": f"p{cycle.plan_number}",
}, name="value").to_frame())
display(manifest[["task", "start UTC", "end UTC", "acceptance gate"]])
fig, ax = plt.subplots(figsize=(12, 5.2), constrained_layout=True)
colors = plt.cm.Blues(np.linspace(0.38, 0.9, len(manifest)))
for (_, row), color in zip(manifest.iterrows(), colors):
ax.barh(
row["task"],
row["end offset (h)"] - row["start offset (h)"],
left=row["start offset (h)"],
color=color,
edgecolor="#263238",
)
ax.axvline(cycle.horizon_hours, color="#b5453c", linestyle="--", label="Forecast horizon")
ax.set(title="Deterministic Forecast-Cycle Orchestration Manifest", xlabel="Hours after cycle reference", ylabel="")
ax.invert_yaxis()
ax.grid(axis="x", alpha=0.25)
ax.legend(loc="lower right")
plt.show()
| value | |
|---|---|
| archived qualification cycle | 2024-08-08 15:00:00+00:00 |
| forecast end UTC | 2024-08-09 09:00:00+00:00 |
| model timezone | America/New_York |
| HEC-RAS plan start | 2024-08-08 11:00:00 |
| HEC-RAS plan end | 2024-08-09 05:00:00 |
| plan | p07 |
| task | start UTC | end UTC | acceptance gate | |
|---|---|---|---|---|
| 0 | discover | 2024-08-08 15:00:00+00:00 | 2024-08-08 15:15:00+00:00 | Resolve complete HRRR/STOFS cycles; reject sta... |
| 1 | download | 2024-08-08 15:15:00+00:00 | 2024-08-08 16:15:00+00:00 | Download into an isolated cycle directory; rec... |
| 2 | normalize | 2024-08-08 16:15:00+00:00 | 2024-08-08 17:00:00+00:00 | Translate/validate precipitation and coastal WSE |
| 3 | author | 2024-08-08 17:00:00+00:00 | 2024-08-08 17:21:00+00:00 | Write gridded precipitation and stage boundari... |
| 4 | precompute | 2024-08-08 17:21:00+00:00 | 2024-08-08 17:45:00+00:00 | Inspect temporary plan HDF before compute |
| 5 | compute | 2024-08-08 17:45:00+00:00 | 2024-08-08 20:45:00+00:00 | Run HEC-RAS with verification enabled |
| 6 | verify/publish | 2024-08-08 20:45:00+00:00 | 2024-08-08 21:30:00+00:00 | Audit final HDF, diagnostics, maps, and proven... |

Validate the public API contract¶
The signatures below are inspected from the installed source. This prevents
the overview from publishing stale pseudo-codeāthe previous notebook used an
obsolete keyword form for update_simulation_date().
api_contract = {
"PrecipHrrr.check_availability": PrecipHrrr.check_availability,
"PrecipHrrr.get_latest_forecast": PrecipHrrr.get_latest_forecast,
"CoastalBoundary.check_availability": CoastalBoundary.check_availability,
"CoastalBoundary.download_stofs3d": CoastalBoundary.download_stofs3d,
"CoastalBoundary.extract_wse_at_point": CoastalBoundary.extract_wse_at_point,
"CoastalBoundary.generate_stage_bc": CoastalBoundary.generate_stage_bc,
"RasPlan.update_simulation_date": RasPlan.update_simulation_date,
"RasCmdr.compute_plan": RasCmdr.compute_plan,
}
signature_table = pd.DataFrame([
{"public API": name, "signature": str(inspect.signature(function))}
for name, function in api_contract.items()
])
display(signature_table)
update_parameters = inspect.signature(RasPlan.update_simulation_date).parameters
assert tuple(update_parameters)[:3] == ("plan_number_or_path", "start_date", "end_date")
compute_parameters = inspect.signature(RasCmdr.compute_plan).parameters
assert {"force_rerun", "verify", "ras_object"}.issubset(compute_parameters)
| public API | signature | |
|---|---|---|
| 0 | PrecipHrrr.check_availability | (date: Optional[str] = None, cycle: Optional[i... |
| 1 | PrecipHrrr.get_latest_forecast | (output_dir: Union[str, pathlib.Path], hours: ... |
| 2 | CoastalBoundary.check_availability | (date: Optional[str] = None, cycle: Optional[i... |
| 3 | CoastalBoundary.download_stofs3d | (output_dir: Union[str, pathlib.Path], date: O... |
| 4 | CoastalBoundary.extract_wse_at_point | (stofs_dir: Union[str, pathlib.Path], lat: flo... |
| 5 | CoastalBoundary.generate_stage_bc | (wse_timeseries, unsteady_file: Union[str, pat... |
| 6 | RasPlan.update_simulation_date | (plan_number_or_path: Union[str, numbers.Numbe... |
| 7 | RasCmdr.compute_plan | (plan_number: Union[str, numbers.Number, pathl... |
Audit the executed component notebooks¶
The overview reads the committed notebook artifacts and verifies that every code cell ran, no notebook-level exception was stored, and review figures are present. Those checks demonstrate artifact completeness; their own numerical and manual-review gates remain authoritative for scientific qualification.
qualification_notebooks = {
"HRRR -> DSS -> HEC-RAS": ("916_hrrr_precipitation_forecast.ipynb", 6),
"STOFS-3D coastal stage": ("923_stofs3d_coastal_boundary.ipynb", 4),
"MRMS -> NetCDF -> HEC-RAS": ("924_mrms_netcdf_rain_on_grid.ipynb", 5),
"WPC QPF -> DSS -> HEC-RAS": ("926_wpc_qpf_precipitation_forecast.ipynb", 5),
}
examples_dir = repo_root / "examples"
qualification_rows = []
for route, (filename, minimum_figures) in qualification_notebooks.items():
component = nbformat.read(examples_dir / filename, as_version=4)
code_cells = [cell for cell in component.cells if cell.cell_type == "code"]
outputs = [output for cell in code_cells for output in cell.get("outputs", [])]
qualification_rows.append({
"route": route,
"notebook": filename.split("_", 1)[0],
"code cells": len(code_cells),
"executed cells": sum(cell.execution_count is not None for cell in code_cells),
"stored errors": sum(output.output_type == "error" for output in outputs),
"review figures": sum("image/png" in output.get("data", {}) for output in outputs),
"minimum figures": minimum_figures,
})
qualification = pd.DataFrame(qualification_rows)
qualification["all cells executed"] = qualification["code cells"] == qualification["executed cells"]
qualification["clean artifact"] = qualification["stored errors"] == 0
qualification["visual evidence"] = qualification["review figures"] >= qualification["minimum figures"]
display(qualification.drop(columns="minimum figures"))
assert qualification[["all cells executed", "clean artifact", "visual evidence"]].all().all()
fig, ax = plt.subplots(figsize=(12, 4.8), constrained_layout=True)
y = np.arange(len(qualification))
ax.barh(y - 0.18, qualification["executed cells"], height=0.34, label="Executed code cells", color="#2b6f9f")
ax.barh(y + 0.18, qualification["review figures"], height=0.34, label="Embedded review figures", color="#72b7b2")
ax.set_yticks(y, qualification["route"])
ax.set(title="Committed Executed Forecast-Component Artifacts", xlabel="Artifact count")
ax.invert_yaxis()
ax.grid(axis="x", alpha=0.25)
ax.legend()
plt.show()
| route | notebook | code cells | executed cells | stored errors | review figures | all cells executed | clean artifact | visual evidence | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | HRRR -> DSS -> HEC-RAS | 916 | 11 | 11 | 0 | 6 | True | True | True |
| 1 | STOFS-3D coastal stage | 923 | 25 | 25 | 0 | 4 | True | True | True |
| 2 | MRMS -> NetCDF -> HEC-RAS | 924 | 13 | 13 | 0 | 5 | True | True | True |
| 3 | WPC QPF -> DSS -> HEC-RAS | 926 | 11 | 11 | 0 | 5 | True | True | True |

Model-specific execution handoff¶
After live inputs pass their product-specific checks and have been authored to
an isolated model copy, the orchestration layer should update dates and compute
through the public APIs using actual datetime values:
RasPlan.update_simulation_date(
cycle.plan_number,
cycle.plan_start.to_pydatetime(),
cycle.plan_end.to_pydatetime(),
ras_object=ras,
)
result = RasCmdr.compute_plan(
cycle.plan_number,
ras_object=ras,
force_rerun=True,
verify=True,
use_optimal_hdf_settings=True,
)
if not result:
raise RuntimeError(result.error)
Before compute, inspect the temporary plan HDF as shown in 916/924/926. After compute, treat the completed plan HDF and the full runtime/BCO message set as the post-compute sources of truth.
Completion record¶
This notebook is complete when its manifest is internally consistent, current public signatures match the documented handoff, and every linked qualification artifact is fully executed, clean, and visual.
completion_checks = pd.Series({
"forecast cycle is timezone explicit": cycle.cycle_utc.tz is not None,
"plan window uses documented local timezone": cycle.plan_start == pd.Timestamp("2024-08-08 11:00"),
"orchestration tasks are contiguous": bool(np.allclose(manifest["start offset (h)"].iloc[1:], manifest["end offset (h)"].iloc[:-1])),
"public API signatures are current": tuple(update_parameters)[:3] == ("plan_number_or_path", "start_date", "end_date"),
"component notebooks are fully executed": bool(qualification["all cells executed"].all()),
"component notebooks contain no stored exceptions": bool(qualification["clean artifact"].all()),
"component notebooks retain visual evidence": bool(qualification["visual evidence"].all()),
"live availability is deferred to scheduler run time": True,
}, name="passed").to_frame()
display(completion_checks)
assert completion_checks["passed"].all()
print("Operational forecast orchestration/readiness audit complete.")
print("Notebook 915 is an executed orchestration artifact, not format or HEC-RAS version qualification.")
print("Run live availability/download checks at cycle time and record the resolved cycle, timestamps, and hashes.")
print("Review product completeness, units/time zones, temporary and final HDFs, all HEC-RAS messages, mass balance, convergence, and maps manually.")
| passed | |
|---|---|
| forecast cycle is timezone explicit | True |
| plan window uses documented local timezone | True |
| orchestration tasks are contiguous | True |
| public API signatures are current | True |
| component notebooks are fully executed | True |
| component notebooks contain no stored exceptions | True |
| component notebooks retain visual evidence | True |
| live availability is deferred to scheduler run time | True |
Operational forecast orchestration/readiness audit complete.
Notebook 915 is an executed orchestration artifact, not format or HEC-RAS version qualification.
Run live availability/download checks at cycle time and record the resolved cycle, timestamps, and hashes.
Review product completeness, units/time zones, temporary and final HDFs, all HEC-RAS messages, mass balance, convergence, and maps manually.