DataFrame Reference¶
This page covers the current project-level DataFrames populated by
init_ras_project(). These tables are the stable entry point for most
automation, validation, and notebook workflows.
Tip
Use the DataFrames first for project metadata and file-path discovery. Use dedicated HDF readers for heavy result extraction, and use xarray-based result APIs when the data is naturally time-series or grid-shaped.
What Gets Created at Initialization¶
from ras_commander import init_ras_project, ras
init_ras_project(r"C:\Projects\MyModel", "6.6")
print(ras.plan_df.columns.tolist())
print(ras.boundaries_df.columns.tolist())
print(ras.rasmap_df.columns.tolist())
| DataFrame | Primary source | Typical use |
|---|---|---|
ras.plan_df |
.p## plan files plus .prj references |
execution targeting, geometry/flow linkage, HDF result discovery |
ras.geom_df |
.g## files and compiled .g##.hdf paths |
geometry inventory, HDF presence, geometry titles/descriptions |
ras.flow_df |
.f## files |
steady-flow inventory and descriptions |
ras.unsteady_df |
.u## files |
unsteady-flow inventory and descriptions |
ras.boundaries_df |
parsed unsteady boundary blocks | DSS audits, hydrograph inspection, boundary summaries |
ras.rasmap_df |
.rasmap project summary |
compact terrain / land-cover / projection path summary |
ras.results_df |
lightweight HDF summaries | completion, runtime, error/warning, and result-file status |
Related live notebooks:
- 101_project_initialization.ipynb
- 104_plan_parameter_operations.ipynb
- 111_executing_plan_sets.ipynb
- 122_rasmapper_spatial_review.ipynb
- 150_results_dataframe.ipynb
- 212_landcover_mannings_n_write.ipynb
- 611_validating_map_layers.ipynb
Project asset inventory¶
inspect_project_assets() returns one row per file reference or referenced
dataset. File and dataset state are separate: an available DSS container, for
example, does not prove that a pathname exists or covers a plan window.
The ordered schema is:
| Column group | Meaning |
|---|---|
inventory_schema_version, inventory_id, inspection_depth |
Snapshot identity and requested scope |
asset_id, parent_asset_id, asset_kind, asset_role |
Stable row identity, containment, and mechanical role |
plan_number, unsteady_number, required |
Plan/flow scope and whether the dependency is mechanically enabled |
owner_file, owner_sha256, reference_raw, resolved_path |
Exact reference provenance and resolved path |
path_scope, portable |
Internal/external/ambiguous path classification |
exists, is_file, is_dir, volume_id, file_id, size_bytes, mtime_ns, sha256 |
Filesystem and optional streamed-hash evidence |
dataset_name, expected_start, expected_end, available_start, available_end |
HDF/DSS/GDAL dataset and coverage evidence |
inspection_state, readiness, reason_code, detail, source_api |
Explicit observation state and parser provenance |
inspection_state is one of available, missing, ambiguous,
not_inspected, failed, or not_applicable. readiness is independently
ready, not_ready, unknown, or not_required. In particular,
not_inspected is never silently promoted to ready.
Controlled DSS 6/7 tests found existing-file reads byte-stable, but the normal
Java handle remained write-capable and several read APIs created a new DSS file
when the target was missing. Direct source pathname and coverage requests
therefore remain not_inspected with reason reader_not_source_immutable; the
container path is still inventoried normally. A future deeper reader will use a
verified disposable copy, OS-denied writes, must-exist open, and a short-lived
worker before returning snapshot-derived coverage.
Exact boundary-block inventory¶
RasUnsteady.inspect_boundary_blocks(staged, unsteady_number="01") returns one
row per exact Boundary Location= block in an owned StageProjectResult. This
is a mutation-evidence table, not a replacement for the broader
ras.boundaries_df summary. Its key groups are:
| Column group | Meaning |
|---|---|
inventory_schema_version, inventory_id, stage_operation_id |
Snapshot and owning stage identity |
owner_relative_path, owner_sha256, owner_size_bytes, owner_mtime_ns, volume_id, file_id |
Exact staged unsteady-file evidence |
boundary_index, occurrence_ordinal, boundary_count |
Raw order and duplicate disambiguation |
boundary_location_raw, location_kind, river, reach, river_station, area_2d, bc_line |
Exact and parsed location evidence |
bc_type, start_byte, end_byte_exclusive, block_length_bytes, block_sha256 |
Detected type and byte-splice evidence |
encoding, has_bom, newline, boundary_id |
File format and exact selector identity |
inspection_state, reason_code, detail |
Explicit success/failure evidence |
Every column has an exact Arrow dtype in DATAFRAME_SCHEMAS. A successful
mutation invalidates this snapshot; inspect a newly staged copy before a
subsequent edit.
Plan-Number Normalization¶
ras-commander normalizes RAS file numbers to a two-digit form before path construction. All of these inputs resolve to the same plan number:
from pathlib import Path
from ras_commander import RasUtils
RasUtils.normalize_ras_number(1) # "01"
RasUtils.normalize_ras_number("01") # "01"
RasUtils.normalize_ras_number("p01") # "01"
RasUtils.normalize_ras_number(Path("Model.p01")) # "01"
This matters when you pass prefixed plan numbers into RasCmdr,
RasProcess, HDF readers, or any helper that resolves .p## / .g## files
from project metadata.
plan_df¶
plan_df is the main execution and metadata table. It is assembled from the
.prj file, parsed .p## contents, and project-relative path resolution.
Key columns you can rely on (column names preserve the HEC-RAS plan-file keys
verbatim, including spaces and the HEC-RAS spelling of Reoccurance). Values
parsed from the plan text are strings unless noted:
| Column | Dtype | Meaning |
|---|---|---|
plan_number |
str | normalized two-digit plan id such as 01 |
geometry_number |
str | normalized geometry id used by the plan |
unsteady_number |
str / None | normalized unsteady-flow id when the plan is unsteady; None for steady plans |
quasi_unsteady_number |
str / None | normalized quasi-unsteady id when the plan references .q## |
flow_file_prefix |
str / None | exact normalized plan reference prefix: f, u, or q |
Plan Title |
str | HEC-RAS plan title (Plan Title=) |
Short Identifier |
str | HEC-RAS short id used by stored-map/result folders |
Geom File |
str | geometry reference number from the plan (Geom File=) |
Geom Path |
str | resolved absolute .g## path |
Flow File |
str | normalized steady, unsteady, or quasi-unsteady reference number (Flow File=) |
Flow Path |
str | resolved absolute .f##, .u##, or .q## path |
Sediment File |
str / None | normalized sediment-file number such as 01; None when no recognized s## reference is present |
sediment_number |
str / None | normalized .s## identifier; sediment is independent of flow regime |
Sediment Path |
str / None | expected absolute .s## path for the selected sediment file; file existence/readiness is reported separately by the asset inventory |
breach_definition_count |
nullable Int64 | number of successfully parsed stored Breach Loc definitions; zero means none and null means inspection failed |
breach_active_count |
nullable Int64 | number of stored definitions whose local RasBreach is_active flag is true; not evidence that a breach initiated during computation |
Computation Interval |
str | computation time step (Computation Interval=) |
Mapping Interval |
str | RAS Mapper output interval (Mapping Interval=) |
Simulation Date |
str | simulation date/time window (Simulation Date=) |
Run HTab / Run UNet / Run PostProcess / Run Sediment / Run WQNet |
str | run-flag toggles parsed from the plan |
UNET D1 Cores / UNET D2 Cores / PS Cores |
int / None | core counts; cast to int when present, else None |
Write IC File / IC Time |
str | restart / hot-start output-save settings when present |
Write IC File at Fixed DateTime |
str | restart-at-fixed-datetime flag when present |
Write IC File Reoccurance |
str | restart output recurrence interval (HEC-RAS spelling preserved) |
Write IC File at Sim End |
str | final-step restart output flag |
Program Version |
str | HEC-RAS version recorded in the plan |
description |
str / None | plan BEGIN DESCRIPTION block when present |
HDF_Results_Path |
str / None | resolved .p##.hdf path; None when results do not exist yet |
full_path |
str | resolved absolute .p## path |
flow_type |
str | Flow computation mode: "Unsteady", "Steady", "Quasi-Unsteady", or "Unknown" (derived from the plan's .u##, .f##, or .q## reference). Sediment, dam breach, and topology are separate features. |
geometry_type |
str | hydraulic inventory: 1D, 2D, 1D/2D, or Unknown |
plan_type |
str | finite execution class: steady_1d, unsteady_1d, unsteady_2d, unsteady_1d_2d, quasi_unsteady_1d, or unknown |
plan_classification_valid |
nullable bool | whether the plan maps to a supported execution class |
plan_classification_reason |
str / None | why classification failed closed |
geometry_metadata_source |
str | hdf, text, or unavailable provenance inherited from geom_df |
geometry_metadata_valid |
nullable bool | whether geometry metadata was successfully inspected |
geometry_metadata_error |
str / None | failed-source details, including HDF-to-text fallback |
Note
Most source columns derive directly from _parse_plan_file() in
ras_commander/RasPrj.py, so a plan key appears under its source name.
Sediment File is normalized for consistency with Geom File and Flow
File; the two breach counts are derived through the existing RasBreach
reader. Not every plan contains every source key.
The execution taxonomy is deliberately finite. HEC-RAS has no steady 2D
solver, so a steady plan that references 2D or 1D/2D geometry is
plan_type="unknown" with plan_classification_valid=False; no
steady_2d or steady_1d_2d value is emitted. Quasi-unsteady is anticipated
as quasi_unsteady_1d; quasi plans with a mesh also fail closed.
Sediment is orthogonal to flow classification. Both .u## and .q## plans
can reference .s##. Storage-area-only geometries with no cross sections or
2D areas currently classify as unknown. Pipe-network presence is also
orthogonal: an unsteady pipe-plus-mesh plan remains unsteady_2d based on its
hydraulic geometry inventory.
Common patterns:
# Plans that already have local HDF results
plans_with_results = ras.plan_df[ras.plan_df["HDF_Results_Path"].notna()]
# Plans using a specific geometry
g04_plans = ras.plan_df[ras.plan_df["geometry_number"] == "04"]
# Plans with successfully parsed active stored breach definitions
active_breach_plans = ras.plan_df[ras.plan_df["breach_active_count"].fillna(0) > 0]
# Plans linked to a sediment file (independent of the Run Sediment flag)
sediment_plans = ras.plan_df[ras.plan_df["Sediment File"].notna()]
# Quick lookup through RasPrj helpers
info = ras.get_plan_info("01")
paths = ras.get_hdf_paths("01")
geom_df¶
geom_df inventories geometry files and compiled HDF companions.
Columns (the metadata columns come from GeomMetadata, which prefers fast
HDF-based extraction when .g##.hdf exists and falls back to plain-text parsing):
| Column | Dtype | Meaning |
|---|---|---|
geom_file |
str | raw .g## reference token from the project (e.g. g01) |
geom_number |
str | normalized geometry id (e.g. 01) |
full_path |
str | resolved absolute .g## path |
hdf_path |
str | expected compiled .g##.hdf path |
geom_title |
str / None | parsed Geom Title= value when present |
description |
str / None | geometry BEGIN DESCRIPTION block when present |
has_1d_xs |
bool | True if the geometry has 1D cross sections |
has_2d_mesh |
nullable bool | True if HDF metadata identifies a mesh or text pairs Storage Area=<name> with Storage Area Is2D=-1; null when no source can be inspected |
geometry_type |
str | 1D, 2D, 1D/2D, or Unknown |
geometry_metadata_source |
str | successful source: hdf, text, or unavailable |
geometry_metadata_valid |
nullable bool | whether a geometry source was successfully inspected |
geometry_metadata_error |
str / None | failed source reads encountered before success, or terminal failure |
num_cross_sections |
int | count of 1D cross sections |
num_inline_structures |
int | total inline structures (bridges + culverts + weirs) |
num_bridges |
int | count of bridge structures |
num_culverts |
int | count of culvert structures |
num_weirs |
int | count of inline weir structures |
num_gates |
int | count of gate structures |
num_lateral_structures |
int | count of lateral structures |
num_sa_2d_connections |
int | count of SA/2D connections |
mesh_cell_count |
nullable Int64 | total HDF mesh cells; null for text-only geometry |
mesh_area_names |
list[str] | names of 2D flow areas |
When HDF inspection fails, GeomMetadata retries the plain geometry and keeps
the HDF error in geometry_metadata_error. When neither source is readable,
the classification booleans remain null and the geometry fails closed as
Unknown; an unreadable geometry never masquerades as a valid empty 1D model.
Text geometry can prove that a 2D area exists, but it cannot provide a mesh-cell
count until preprocessing materializes the HDF.
RasPlan.set_geom(), set_steady(), and set_unsteady() rewrite exactly one
top-level plan reference, preserve description text and newline style, verify
the write, and then refresh geometry, flow, and plan classification dataframes
in dependency order.
Classification tests pin the public RAS example archive by release, byte size, and SHA-256. Derived copies remove or corrupt HDFs without changing the immutable source. The private CLB qualification tier additionally covers real 6.31 text-only 2D, 6.70 pipe-plus-2D, and 7.00 1D HDF schemas. These gates qualify metadata classification only; they do not claim solver execution or numerical-result parity.
flow_df and unsteady_df¶
These inventory steady and unsteady flow files referenced by the project.
Both come from _parse_flow_file() / _parse_unsteady_file() in RasPrj.py.
flow_df (steady .f##):
| Column | Dtype | Meaning |
|---|---|---|
flow_number |
str | normalized steady-flow id |
unsteady_number |
None | always None for steady-flow rows |
full_path |
str | resolved absolute .f## path |
Flow Title |
str / None | parsed Flow Title= value |
Program Version |
str / None | HEC-RAS version recorded in the flow file |
description |
str | flow BEGIN DESCRIPTION block (empty string when absent) |
unsteady_df (unsteady .u##):
| Column | Dtype | Meaning |
|---|---|---|
unsteady_number |
str | normalized unsteady-flow id |
full_path |
str | resolved absolute .u## path |
Flow Title |
str / None | parsed Flow Title= value |
Program Version |
str / None | HEC-RAS version recorded in the file |
Use Restart |
str / None | restart/hot-start flag (Use Restart=) |
Restart Filename |
str / None | restart source file when restart is enabled |
Precipitation Mode / Wind Mode |
str / None | meteorology mode toggles when present |
Met BC=Precipitation\|... |
str / None | parsed gridded-precip / met-BC settings when present |
description |
str | unsteady BEGIN DESCRIPTION block (empty string when absent) |
Use these tables when you need to audit which .f## / .u## files exist
before editing them through RasPlan or RasUnsteady.
boundaries_df¶
boundaries_df is built from the unsteady boundary blocks and is the main
table for DSS-path audits and boundary-condition summaries.
Common columns (from _parse_boundary_condition() in RasPrj.py; DSS/typed
columns appear only when the source line is present):
| Column | Dtype | Meaning |
|---|---|---|
unsteady_number |
str | parent .u## file |
boundary_condition_number |
int | boundary sequence within the file (1-based) |
bc_type |
str | high-level boundary type (e.g. Flow Hydrograph, Normal Depth, Unknown) |
hydrograph_type |
str / None | hydrograph subtype when the BC is a hydrograph, else None |
river_reach_name |
str | 1D river/reach location field (may be empty string) |
river_station |
str | 1D river-station field (may be empty string) |
storage_area_name |
str | storage-area location field when present |
pump_station_name |
str | pump-station field when present |
area_2d |
str | 2D flow-area name field when present |
bc_line_name |
str | boundary-condition line name when present |
Interval |
str | time interval (Interval=) |
Use DSS |
str | "True" / "False" string (note: string, not bool) |
DSS File |
str | raw DSS file reference |
DSS Path |
str | raw DSS pathname |
dss_part_a … dss_part_f |
str | parsed DSS pathname components A–F |
Friction Slope |
str | raw friction-slope field for Normal Depth BCs |
friction_slope_value |
float / None | parsed friction-slope value |
critical_fallback_flag |
int / None | parsed critical-boundary fallback flag |
hydrograph_num_values |
int | number of inline hydrograph values (0 if none) |
hydrograph_values |
list[str] | inline hydrograph values when present |
Examples:
# DSS-backed boundaries only
dss_boundaries = ras.boundaries_df[ras.boundaries_df["Use DSS"] == "True"]
# Flow hydrographs only
flow_bcs = ras.boundaries_df[ras.boundaries_df["bc_type"] == "Flow Hydrograph"]
DSS boundary inventory¶
RasUnsteady.get_dss_boundaries(unsteady_file) returns the DSS-linked subset
from one unsteady-flow file with exact block identity and parsed pathname parts.
Its canonical 2D identity columns match boundaries_df:
| Column | Dtype | Meaning |
|---|---|---|
area_2d |
str | 2D flow-area name when present |
bc_line_name |
str | named 2D boundary-condition line when present |
sa_2d_name |
str | deprecated compatibility alias for area_2d |
bc_line |
str | deprecated compatibility alias for bc_line_name |
The aliases remain equal to their canonical columns during the compatibility
window. Machine-readable deprecation mappings are available from
inventory.attrs["deprecated_columns"].
Boundary mutation methods retain their existing compatibility keywords during
this window. Pass canonical inventory values explicitly, for example
sa_2d_name=row["area_2d"] and bc_line=row["bc_line_name"] when calling
RasUnsteady.set_boundary_dss_link().
rasmap_df¶
rasmap_df is a single-row compact summary of the project .rasmap. It has
one row for every outcome, including a missing or unreadable file. Several
columns contain lists because the dataframe is optimized for project overview,
not one-row-per-layer discovery.
Current default columns (each cell is one element because the DataFrame is a single row; "Dtype" describes the value inside that cell):
| Column | Dtype | Meaning |
|---|---|---|
projection_path |
str / None | project projection (.prj) reference |
profile_lines_path |
list[str] | profile/reference line paths |
soil_layer_path |
list[str] | soils sidecar paths |
infiltration_hdf_path |
list[str] | infiltration sidecar HDFs |
landcover_hdf_path |
list[str] | land-cover sidecar HDFs |
terrain_hdf_path |
list[str] | terrain HDFs |
reference_map_layer_names |
list[str] | reference map layer names |
reference_map_layer_path |
list[str] | reference map layer paths |
basemap_layer_names |
list[str] | basemap layer names |
basemap_layer_path |
list[str] | basemap layer paths |
current_settings |
dict | compact .rasmap settings summary |
rasmap_path |
str / None | requested or expected .rasmap path, including when absent |
rasmap_status |
str | absent, parsed, parsed_with_errors, or failed |
rasmap_error |
str / None | document-level read, parse, or initialization error |
rasmap_field_errors |
dict | extraction errors keyed by affected data column |
summary = ras.rasmap_df.iloc[0]
if summary["rasmap_status"] == "failed":
raise RuntimeError(summary["rasmap_error"])
if summary["rasmap_status"] == "absent":
print(f"No RASMapper file at {summary['rasmap_path']}")
else:
if summary["rasmap_field_errors"]:
print("Incomplete RASMapper fields:", summary["rasmap_field_errors"])
print(summary["terrain_hdf_path"])
print(summary["landcover_hdf_path"])
Do not use len(rasmap_df), rasmap_df.empty, or rasmap_df is not None as a
parse-success check. Those values are deliberately constant so callers can
safely use .iloc[0]; inspect rasmap_status instead. Missing optional
sections and an empty projection filename are valid parsed states. Missing
files referenced by otherwise valid XML are asset-validation findings, not
parser failures.
Use rasmap_df for quick project-level path inspection. When you need
discoverable layer names and per-layer metadata, prefer:
RasMap.list_terrain_layers()RasMap.list_landcover_layers()RasMap.list_soils_layers()RasMap.list_infiltration_layers()RasMap.list_map_layers()RasMap.list_geometry_layers()RasMap.list_result_layers()
For compiled HDF asset resolution, pair the .rasmap summary with
RasMap.get_hdf_geometry_association() on geometry or plan/result HDFs.
results_df¶
results_df is a lightweight summary table generated from plan HDFs through
ResultsSummary. It is intended for fast execution-status and runtime queries,
not heavy spatial extraction.
Columns (from ResultsSummary.summarize_plan() /
get_summary_columns() in ras_commander/results/ResultsSummary.py; identity,
health, and runtime columns are always present, None/0 when unavailable):
| Column | Dtype | Meaning |
|---|---|---|
plan_number |
str | copied plan id |
plan_title |
str / None | copied plan title |
flow_type |
str / None | Steady, Unsteady, Quasi-Unsteady, or Unknown flow computation mode |
hdf_path |
str | path to the .p##.hdf result file |
hdf_exists |
bool | whether the HDF result file exists |
hdf_file_modified |
datetime / None | HDF modification timestamp |
ras_version |
str / None | HEC-RAS version (Program Version) |
completed |
bool | completion flag parsed from compute metadata |
has_errors |
bool | summary error flag |
has_warnings |
bool | summary warning flag |
error_count |
int | parsed error-message count |
warning_count |
int | parsed warning-message count |
first_error_line |
str / None | first blocking compute-message line when present |
runtime_simulation_start |
datetime / None | simulation start time |
runtime_simulation_end |
datetime / None | simulation end time |
runtime_simulation_hours |
float / None | simulated duration (hours) |
runtime_complete_process_hours |
float / None | end-to-end wall-clock runtime (hours) |
runtime_unsteady_compute_hours |
float / None | unsteady compute runtime (hours) |
runtime_complete_process_speed |
float / None | normalized throughput (sim hr / wall hr) |
runtime_source |
str / None | 'hdf' or 'compute_messages' provenance |
vol_error |
float / None | volume-accounting error (unsteady only) |
vol_accounting_units |
str / None | volume units |
vol_error_percent |
float / None | volume error as percent |
vol_flux_in / vol_flux_out |
float / None | total inflow / outflow volume |
vol_starting / vol_ending |
float / None | starting / ending storage volume |
print(ras.results_df[[
"plan_number",
"completed",
"has_errors",
"runtime_complete_process_hours",
]])
# Refresh after new runs
ras.update_results_df(["01"])
HDF and Time-Series Data Are Not Always DataFrames¶
Project metadata belongs in pandas DataFrames. Heavy simulation outputs often do not. Prefer the dedicated HDF readers and xarray-backed APIs for:
- 1D cross-section time series
- 2D mesh cell or face time series
- reference lines and reference points
- profile-line flow and peak-Q extraction
- large raster or mesh-derived result families
Profile-Line Flow Outputs¶
HdfResultsMesh.get_profile_line_flow_timeseries() returns a profile/reference
line flow time series. The API uses native HDF reference-line internal faces
when present, then falls back to RAS Mapper profile-line geometry.
| Column | Meaning |
|---|---|
time |
Output timestamp |
flow |
Sum of selected face flows |
line_name |
Requested profile/reference line |
mesh_name |
2D flow area used for extraction |
direction |
absolute or signed aggregation mode |
face_count |
Count of selected mesh faces |
selection_source |
reference_line_internal_faces or profile_lines_geometry |
HdfResultsMesh.get_profile_line_peak_flow() returns one peak-Q row derived
from the time series.
| Column | Meaning |
|---|---|
line_name |
Requested profile/reference line |
mesh_name |
2D flow area used for extraction |
peak_time |
Timestamp of peak flow magnitude |
peak_flow |
Peak flow value; signed mode preserves native sign |
direction |
absolute or signed aggregation mode |
face_count |
Count of selected mesh faces |
selection_source |
reference_line_internal_faces or profile_lines_geometry |
See:
Practical Workflow¶
- Initialize the project and inspect
plan_df,boundaries_df, andrasmap_df. - Normalize any user-supplied plan or geometry numbers before constructing paths.
- Use
results_dffor quick execution and health checks. - Use
RasMap.list_*_layers()andget_hdf_geometry_association()for per-layer RASMapper and compiled-HDF QA. - Move to dedicated HDF or geometry APIs only after the project metadata tables tell you which files and plans matter.