Skip to content

Operational Forecast Orchestration and Readiness

This executable overview builds a deterministic forecast-cycle manifest, validates the public APIs it will call, and audits the committed format-specific qualification notebooks. It does not duplicate their expensive downloads and HEC-RAS runs.

Use the linked notebooks for executed evidence:

  • 916: archived HRRR GRIB2 to native DSS to HEC-RAS
  • 923: STOFS-3D download, extraction, and coastal-stage authoring mechanics
  • 924: MRMS NetCDF rain-on-grid
  • 926: current WPC QPF to DSS to HEC-RAS

Notebook 915 is orchestration evidence, not independent format or HEC-RAS version qualification.

Production boundary

Operational forecasting must separate reproducible qualification from live cycle discovery. Committed outputs below use the archived 8 August 2024 15Z HRRR cycle qualified by notebook 916. A production scheduler should call live availability/download APIs at run time, record the resolved cycle and hashes, and fail closed when required inputs are unavailable or stale.

Manual review remains required for product age/completeness, units and time zones, terrain and forcing coverage, boundary datums, temporary-HDF inputs, all HEC-RAS runtime/BCO messages, mass balance, convergence, and final maps.

Python
# Development mode: import repository source when run from a clone.
USE_LOCAL_SOURCE = True

import inspect
import sys
from dataclasses import dataclass
from datetime import timedelta
from pathlib import Path
from zoneinfo import ZoneInfo

import matplotlib.pyplot as plt
import nbformat
import numpy as np
import pandas as pd

repo_root = Path.cwd().parent if Path.cwd().name.lower() == "examples" else Path.cwd()
if USE_LOCAL_SOURCE and str(repo_root) not in sys.path:
    sys.path.insert(0, str(repo_root))

import ras_commander
from ras_commander import RasCmdr, RasPlan
from ras_commander.boundaries import CoastalBoundary
from ras_commander.precip import PrecipHrrr

ipython = get_ipython()
if ipython is not None:
    ipython.run_line_magic("matplotlib", "inline")

print(f"ras-commander {ras_commander.__version__} ({'repository source' if USE_LOCAL_SOURCE else 'installed package'})")
Text Only
ras-commander 0.102.0 (repository source)

Inspect the forecast products without making a time-dependent download

get_info() is deterministic and confirms the product metadata used by the orchestrator. Live discovery remains an explicit scheduler action through PrecipHrrr.check_availability() and CoastalBoundary.check_availability().

Python
hrrr_info = PrecipHrrr.get_info()
stofs_info = CoastalBoundary.get_info()
product_catalog = pd.DataFrame([
    {
        "input": "HRRR precipitation",
        "provider": hrrr_info["provider"],
        "format": hrrr_info["format"],
        "update": hrrr_info["update_frequency"],
        "forecast horizon": f"{hrrr_info['forecast_horizon_standard']} standard",
        "executed reference": "916: archived GRIB2 -> DSS -> temporary/final HDF",
    },
    {
        "input": "STOFS-3D coastal stage",
        "provider": stofs_info["provider"],
        "format": stofs_info["format"],
        "update": stofs_info["update_frequency"],
        "forecast horizon": stofs_info["forecast_horizon"],
        "executed reference": "923: NetCDF -> point WSE -> stage-authoring mechanics",
    },
])
display(product_catalog)
assert product_catalog["provider"].notna().all()
assert product_catalog["executed reference"].str.startswith(("916:", "923:")).all()
input provider format update forecast horizon executed reference
0 HRRR precipitation NOAA / NCEP GRIB2 Hourly (every cycle) 18 hours standard 916: archived GRIB2 -> DSS -> temporary/final HDF
1 STOFS-3D coastal stage NOAA / NOS / CO-OPS NetCDF (field output), GRIB2 (surface fields) Every 6 hours (00z, 06z, 12z, 18z) 48 hours 923: NetCDF -> point WSE -> stage-authoring me...

Build a timezone-explicit cycle manifest

Forecast reference times are UTC. HEC-RAS plan dates are timezone-naive, so the orchestrator must convert to the model's documented local time convention before calling RasPlan.update_simulation_date().

Python
@dataclass(frozen=True)
class ForecastCycle:
    cycle_utc: pd.Timestamp
    horizon_hours: int
    model_timezone: str
    plan_number: str

    @property
    def end_utc(self) -> pd.Timestamp:
        return self.cycle_utc + pd.Timedelta(hours=self.horizon_hours)

    @property
    def plan_start(self) -> pd.Timestamp:
        return self.cycle_utc.tz_convert(ZoneInfo(self.model_timezone)).tz_localize(None)

    @property
    def plan_end(self) -> pd.Timestamp:
        return self.end_utc.tz_convert(ZoneInfo(self.model_timezone)).tz_localize(None)


cycle = ForecastCycle(
    cycle_utc=pd.Timestamp("2024-08-08 15:00", tz="UTC"),
    horizon_hours=18,
    model_timezone="America/New_York",
    plan_number="07",
)

manifest = pd.DataFrame([
    ("discover", 0.00, 0.25, "Resolve complete HRRR/STOFS cycles; reject stale inputs"),
    ("download", 0.25, 1.25, "Download into an isolated cycle directory; record hashes"),
    ("normalize", 1.25, 2.00, "Translate/validate precipitation and coastal WSE"),
    ("author", 2.00, 2.35, "Write gridded precipitation and stage boundaries via APIs"),
    ("precompute", 2.35, 2.75, "Inspect temporary plan HDF before compute"),
    ("compute", 2.75, 5.75, "Run HEC-RAS with verification enabled"),
    ("verify/publish", 5.75, 6.50, "Audit final HDF, diagnostics, maps, and provenance"),
], columns=["task", "start offset (h)", "end offset (h)", "acceptance gate"])
manifest["duration (h)"] = manifest["end offset (h)"] - manifest["start offset (h)"]
manifest["start UTC"] = cycle.cycle_utc + pd.to_timedelta(manifest["start offset (h)"], unit="h")
manifest["end UTC"] = cycle.cycle_utc + pd.to_timedelta(manifest["end offset (h)"], unit="h")

assert cycle.plan_start == pd.Timestamp("2024-08-08 11:00")
assert cycle.plan_end == pd.Timestamp("2024-08-09 05:00")
assert np.allclose(manifest["start offset (h)"].iloc[1:].to_numpy(), manifest["end offset (h)"].iloc[:-1].to_numpy())
display(pd.Series({
    "archived qualification cycle": cycle.cycle_utc,
    "forecast end UTC": cycle.end_utc,
    "model timezone": cycle.model_timezone,
    "HEC-RAS plan start": cycle.plan_start,
    "HEC-RAS plan end": cycle.plan_end,
    "plan": f"p{cycle.plan_number}",
}, name="value").to_frame())
display(manifest[["task", "start UTC", "end UTC", "acceptance gate"]])

fig, ax = plt.subplots(figsize=(12, 5.2), constrained_layout=True)
colors = plt.cm.Blues(np.linspace(0.38, 0.9, len(manifest)))
for (_, row), color in zip(manifest.iterrows(), colors):
    ax.barh(
        row["task"],
        row["end offset (h)"] - row["start offset (h)"],
        left=row["start offset (h)"],
        color=color,
        edgecolor="#263238",
    )
ax.axvline(cycle.horizon_hours, color="#b5453c", linestyle="--", label="Forecast horizon")
ax.set(title="Deterministic Forecast-Cycle Orchestration Manifest", xlabel="Hours after cycle reference", ylabel="")
ax.invert_yaxis()
ax.grid(axis="x", alpha=0.25)
ax.legend(loc="lower right")
plt.show()
value
archived qualification cycle 2024-08-08 15:00:00+00:00
forecast end UTC 2024-08-09 09:00:00+00:00
model timezone America/New_York
HEC-RAS plan start 2024-08-08 11:00:00
HEC-RAS plan end 2024-08-09 05:00:00
plan p07
task start UTC end UTC acceptance gate
0 discover 2024-08-08 15:00:00+00:00 2024-08-08 15:15:00+00:00 Resolve complete HRRR/STOFS cycles; reject sta...
1 download 2024-08-08 15:15:00+00:00 2024-08-08 16:15:00+00:00 Download into an isolated cycle directory; rec...
2 normalize 2024-08-08 16:15:00+00:00 2024-08-08 17:00:00+00:00 Translate/validate precipitation and coastal WSE
3 author 2024-08-08 17:00:00+00:00 2024-08-08 17:21:00+00:00 Write gridded precipitation and stage boundari...
4 precompute 2024-08-08 17:21:00+00:00 2024-08-08 17:45:00+00:00 Inspect temporary plan HDF before compute
5 compute 2024-08-08 17:45:00+00:00 2024-08-08 20:45:00+00:00 Run HEC-RAS with verification enabled
6 verify/publish 2024-08-08 20:45:00+00:00 2024-08-08 21:30:00+00:00 Audit final HDF, diagnostics, maps, and proven...

png

Validate the public API contract

The signatures below are inspected from the installed source. This prevents the overview from publishing stale pseudo-code—the previous notebook used an obsolete keyword form for update_simulation_date().

Python
api_contract = {
    "PrecipHrrr.check_availability": PrecipHrrr.check_availability,
    "PrecipHrrr.get_latest_forecast": PrecipHrrr.get_latest_forecast,
    "CoastalBoundary.check_availability": CoastalBoundary.check_availability,
    "CoastalBoundary.download_stofs3d": CoastalBoundary.download_stofs3d,
    "CoastalBoundary.extract_wse_at_point": CoastalBoundary.extract_wse_at_point,
    "CoastalBoundary.generate_stage_bc": CoastalBoundary.generate_stage_bc,
    "RasPlan.update_simulation_date": RasPlan.update_simulation_date,
    "RasCmdr.compute_plan": RasCmdr.compute_plan,
}
signature_table = pd.DataFrame([
    {"public API": name, "signature": str(inspect.signature(function))}
    for name, function in api_contract.items()
])
display(signature_table)

update_parameters = inspect.signature(RasPlan.update_simulation_date).parameters
assert tuple(update_parameters)[:3] == ("plan_number_or_path", "start_date", "end_date")
compute_parameters = inspect.signature(RasCmdr.compute_plan).parameters
assert {"force_rerun", "verify", "ras_object"}.issubset(compute_parameters)
public API signature
0 PrecipHrrr.check_availability (date: Optional[str] = None, cycle: Optional[i...
1 PrecipHrrr.get_latest_forecast (output_dir: Union[str, pathlib.Path], hours: ...
2 CoastalBoundary.check_availability (date: Optional[str] = None, cycle: Optional[i...
3 CoastalBoundary.download_stofs3d (output_dir: Union[str, pathlib.Path], date: O...
4 CoastalBoundary.extract_wse_at_point (stofs_dir: Union[str, pathlib.Path], lat: flo...
5 CoastalBoundary.generate_stage_bc (wse_timeseries, unsteady_file: Union[str, pat...
6 RasPlan.update_simulation_date (plan_number_or_path: Union[str, numbers.Numbe...
7 RasCmdr.compute_plan (plan_number: Union[str, numbers.Number, pathl...

Audit the executed component notebooks

The overview reads the committed notebook artifacts and verifies that every code cell ran, no notebook-level exception was stored, and review figures are present. Those checks demonstrate artifact completeness; their own numerical and manual-review gates remain authoritative for scientific qualification.

Python
qualification_notebooks = {
    "HRRR -> DSS -> HEC-RAS": ("916_hrrr_precipitation_forecast.ipynb", 6),
    "STOFS-3D coastal stage": ("923_stofs3d_coastal_boundary.ipynb", 4),
    "MRMS -> NetCDF -> HEC-RAS": ("924_mrms_netcdf_rain_on_grid.ipynb", 5),
    "WPC QPF -> DSS -> HEC-RAS": ("926_wpc_qpf_precipitation_forecast.ipynb", 5),
}
examples_dir = repo_root / "examples"
qualification_rows = []
for route, (filename, minimum_figures) in qualification_notebooks.items():
    component = nbformat.read(examples_dir / filename, as_version=4)
    code_cells = [cell for cell in component.cells if cell.cell_type == "code"]
    outputs = [output for cell in code_cells for output in cell.get("outputs", [])]
    qualification_rows.append({
        "route": route,
        "notebook": filename.split("_", 1)[0],
        "code cells": len(code_cells),
        "executed cells": sum(cell.execution_count is not None for cell in code_cells),
        "stored errors": sum(output.output_type == "error" for output in outputs),
        "review figures": sum("image/png" in output.get("data", {}) for output in outputs),
        "minimum figures": minimum_figures,
    })

qualification = pd.DataFrame(qualification_rows)
qualification["all cells executed"] = qualification["code cells"] == qualification["executed cells"]
qualification["clean artifact"] = qualification["stored errors"] == 0
qualification["visual evidence"] = qualification["review figures"] >= qualification["minimum figures"]
display(qualification.drop(columns="minimum figures"))
assert qualification[["all cells executed", "clean artifact", "visual evidence"]].all().all()

fig, ax = plt.subplots(figsize=(12, 4.8), constrained_layout=True)
y = np.arange(len(qualification))
ax.barh(y - 0.18, qualification["executed cells"], height=0.34, label="Executed code cells", color="#2b6f9f")
ax.barh(y + 0.18, qualification["review figures"], height=0.34, label="Embedded review figures", color="#72b7b2")
ax.set_yticks(y, qualification["route"])
ax.set(title="Committed Executed Forecast-Component Artifacts", xlabel="Artifact count")
ax.invert_yaxis()
ax.grid(axis="x", alpha=0.25)
ax.legend()
plt.show()
route notebook code cells executed cells stored errors review figures all cells executed clean artifact visual evidence
0 HRRR -> DSS -> HEC-RAS 916 11 11 0 6 True True True
1 STOFS-3D coastal stage 923 25 25 0 4 True True True
2 MRMS -> NetCDF -> HEC-RAS 924 13 13 0 5 True True True
3 WPC QPF -> DSS -> HEC-RAS 926 11 11 0 5 True True True

png

Model-specific execution handoff

After live inputs pass their product-specific checks and have been authored to an isolated model copy, the orchestration layer should update dates and compute through the public APIs using actual datetime values:

Python
RasPlan.update_simulation_date(
    cycle.plan_number,
    cycle.plan_start.to_pydatetime(),
    cycle.plan_end.to_pydatetime(),
    ras_object=ras,
)
result = RasCmdr.compute_plan(
    cycle.plan_number,
    ras_object=ras,
    force_rerun=True,
    verify=True,
    use_optimal_hdf_settings=True,
)
if not result:
    raise RuntimeError(result.error)

Before compute, inspect the temporary plan HDF as shown in 916/924/926. After compute, treat the completed plan HDF and the full runtime/BCO message set as the post-compute sources of truth.

Completion record

This notebook is complete when its manifest is internally consistent, current public signatures match the documented handoff, and every linked qualification artifact is fully executed, clean, and visual.

Python
completion_checks = pd.Series({
    "forecast cycle is timezone explicit": cycle.cycle_utc.tz is not None,
    "plan window uses documented local timezone": cycle.plan_start == pd.Timestamp("2024-08-08 11:00"),
    "orchestration tasks are contiguous": bool(np.allclose(manifest["start offset (h)"].iloc[1:], manifest["end offset (h)"].iloc[:-1])),
    "public API signatures are current": tuple(update_parameters)[:3] == ("plan_number_or_path", "start_date", "end_date"),
    "component notebooks are fully executed": bool(qualification["all cells executed"].all()),
    "component notebooks contain no stored exceptions": bool(qualification["clean artifact"].all()),
    "component notebooks retain visual evidence": bool(qualification["visual evidence"].all()),
    "live availability is deferred to scheduler run time": True,
}, name="passed").to_frame()
display(completion_checks)
assert completion_checks["passed"].all()

print("Operational forecast orchestration/readiness audit complete.")
print("Notebook 915 is an executed orchestration artifact, not format or HEC-RAS version qualification.")
print("Run live availability/download checks at cycle time and record the resolved cycle, timestamps, and hashes.")
print("Review product completeness, units/time zones, temporary and final HDFs, all HEC-RAS messages, mass balance, convergence, and maps manually.")
passed
forecast cycle is timezone explicit True
plan window uses documented local timezone True
orchestration tasks are contiguous True
public API signatures are current True
component notebooks are fully executed True
component notebooks contain no stored exceptions True
component notebooks retain visual evidence True
live availability is deferred to scheduler run time True
Text Only
Operational forecast orchestration/readiness audit complete.
Notebook 915 is an executed orchestration artifact, not format or HEC-RAS version qualification.
Run live availability/download checks at cycle time and record the resolved cycle, timestamps, and hashes.
Review product completeness, units/time zones, temporary and final HDFs, all HEC-RAS messages, mass balance, convergence, and maps manually.