Project Archive¶
The archive and inspect commands treat an entire HEC-RAS project as a single unit — discovering all geometry configurations, plan runs, and terrain rasters and exporting them to a structured, cloud-native archive.
Why Project-Level Archival?¶
A HEC-RAS project typically contains:
- Multiple geometry configurations (
.g01,.g02, …) — each may define different mesh resolutions, channel cross-sections, or land cover representations - Multiple plan runs (
.p01,.p02, …) — dam-break scenarios, design storms, calibration runs, etc. - Terrain rasters (
Terrain/*.tif) — the underlying DEM used to compute depths - Plan result HDFs (
.p##.hdf) — contain a copy of the geometry plus computed summary variables
Results not exported by default
Plan HDF files embed a full copy of the geometry they used. Exporting plan results without a clear need doubles your storage. Use --results explicitly when you want result variables (maximum depth, velocity, etc.).
Output Directory Structure¶
Each geometry source and plan produces one consolidated GeoParquet file with a layer column
to distinguish different geometry types or result variables. This enables efficient DuckDB queries
with WHERE layer = 'mesh_cells' and avoids directory proliferation.
{output_dir}/
├── manifest.json # Project catalog (schema v2.7, index metadata)
├── {ProjectName}.parquet # Project metadata (RasPrj dataframes, _table column)
├── {ProjectName}.g01.parquet # All geometry from g01 (HDF + text layers)
├── {ProjectName}.g06.parquet # All geometry from g06
├── {ProjectName}.p01.parquet # All results from p01, layer column (--results)
└── terrain/ # Only with --terrain
└── Terrain50_cog.tif
Large 2D projects can also use a partitioned results layout:
{output_dir}/
├── {ProjectName}.g01.parquet
└── results/
└── p01/
├── maximum_depth.parquet
└── maximum_water_surface.parquet
Querying consolidated files¶
-- Filter by layer column for homogeneous geometry types
SELECT * FROM 'BaldEagle.g01.parquet' WHERE layer = 'mesh_cells'
SELECT * FROM 'BaldEagle.g01.parquet' WHERE layer = 'bc_lines'
SELECT * FROM 'BaldEagle.g01.parquet' WHERE layer = 'cross_sections_text'
-- Results by variable
SELECT * FROM 'BaldEagle.p01.parquet' WHERE layer = 'maximum_depth'
-- Project metadata
SELECT * FROM 'BaldEagle.parquet' WHERE _table = 'plan_df'
-- List available layers in a file
SELECT DISTINCT layer FROM 'BaldEagle.g01.parquet'
-- Bounding-box filter on the GeoParquet 1.1 bbox struct (manifest schema 2.7+).
-- Rows are Hilbert ordered within each layer, so DuckDB skips row groups
-- whose bbox statistics miss the window.
SELECT * FROM 'BaldEagle.g01.parquet'
WHERE layer = 'mesh_cells'
AND bbox.xmin <= 2050000 AND bbox.xmax >= 2040000
AND bbox.ymin <= 345000 AND bbox.ymax >= 335000
-- Archives written before schema 2.7 have flat columns instead:
-- bbox_xmin <= 2050000 AND bbox_xmax >= 2040000 ...
GeoParquet features¶
- ZSTD compression — Better compression ratios than snappy
- GeoParquet 1.1 bbox covering — a nullable
bboxstruct column (bbox.xmin,bbox.ymin,bbox.xmax,bbox.ymax; null for null or empty geometry) withcoveringmetadata, which GeoParquet 1.1 validators accept. Archives written before manifest schema 2.7 carry four flatbbox_xmin...bbox_ymaxcolumns instead; every ras2cng reader (query,sync,spatial-index,maplibre,pmtiles) handles both, andspatial-indexrewrites a file in the form it already has - Layer row groups — rows are grouped by
layerand a row group ends every 65,536 rows and at every layer boundary, so alayerfilter or a bbox filter skips whole row groups - Encoding — ZSTD level 3, dictionary encoding on string columns only (unique numeric columns gain nothing from a per-row-group dictionary). On Bald Eagle p07 (15 geometries plus results) the archive is 50.5 MB, against 58.0 MB from ras2cng 0.12's single-row-group files
hilbert_indexwritten once — with sorting on, the writer stores the Hilbert key itself and the spatial-index pass leaves the file as written; it only rewrites files from older archives or files written without the key- Hilbert order — within each layer, rows are ordered by DuckDB spatial
ST_Hilbertof each bbox centre over the layer's extent (16 bits per axis). The default post-processing pass stores that key ashilbert_index(disable both with--no-sort). Before 0.13 the in-writer sort called a DuckDB function that does not exist and silently did nothing; only the post-processing pass ordered rows - Native writer (default): terrio's
write_geoparquet_layers(..., profile="archive"), the one Rust implementation of the archive profile, writes files identical to the Python engine (rows, order, schema,geometadata, row groups). fast-hdf's formerexport_geoparquet_ipc_fileswas retired in its favour (terrio owns file formats).RAS2CNG_GEOPARQUET_WRITER=auto|native|prefer-native|python:auto(default;prefer-nativeis the same) uses terrio when the installed build haswrite_geoparquet_layers, and otherwise (or when terrio refuses a file) writes it with pyarrow, warning once per reason and recording the reason in the manifest'sgeoparquet.fallback_reason. terrio takes any Arrow column type, Z coordinates and geometry collections; only an empty table goes to pyarrow.native(aliasesterrio,rust; the oldfast-hdfvalue is accepted) requires terrio and raises when it cannot write a file;pythonnever tries it.- Measured on Bald Eagle p07 (16 files, fresh process per file, loaded host): 10.4 s against 12.7 s for pyarrow, peak memory 561 MB against 1184 MB. Files are -5.4% to +2.2% the size of pyarrow's: terrio keeps min/max statistics on large WKB values, which pyarrow drops. Set
RAS2CNG_GEOPARQUET_WRITER=pythonto keep the pyarrow writer.
- Direct path (default with a capable terrio): geometry and
--resultsgo through terrio'swrite_hecras_archive, straight from the HDF with no GeoDataFrame for the expensive layers.RAS2CNG_ARCHIVE_ENGINEor--archive-engine=auto|prefer-native|native|python:auto(default;prefer-nativeis the same, except thatautoalso stays on Python underRAS2CNG_GEOPARQUET_WRITER=python) uses it when the installed terrio has it, and otherwise writes exactly what earlier versions wrote, as manifest 2.7.native(aliasesdirect,terrio,rust) raises rather than sending a whole file to the Python path.pythonnever tries it.- What comes from where:
mesh_cells,mesh_facesand 2D summary variables are native. All other geometry layers and all*_textlayers come from Python, because the engine does not hold all their columns. A layer or variable the engine refuses is filled from Python; if Python cannot read it either, it is absent from both paths and recorded withfilled: false. Any other problem sends that file to the Python path.--no-sort,--results-layout variable,--results-geometry point|noneand steady plans use the Python path. - Measured (
--results, median of three, shared host): Bald Eagle p07 29.1 s to 12.5 s, peak 4.0 GB to 0.76 GB; Spring (HEC-RAS 5.0.7) 22.4 s to 16.2 s, 3.0 GB to 2.0 GB; Spring_BLE (6.1) 27.9 s to 24.7 s, 4.0 GB to 3.1 GB, because its plan stores no summaryLocationand stays on Python. Those figures are with terrio 0.5.0.3 and older, which link ras-commander-fast-hdf 0.3.0. terrio 0.5.0.4 links fast-hdf 0.5.0, which reads summaries without a storedLocation, so these plans take the direct path too: Spring p01 has 5 native variables and none from Python, Spring_BLE p02 8 native;archive --resultsmeasured 20.4 s / 2,257 MB to 13.7 s / 1,367 MB (Spring) and 25.4 s / 3,354 MB to 13.3 s / 1,441 MB (Spring_BLE) against 0.5.0.3. Layers, rows and non-null columns are equal to the Python path; geometry files are byte-identical. Plan files no longer carry the all-null columns of geometry layers (191 columns to 20 on p07).
- Result join indexes — Geometryless result tables get
join_index; when matching mesh geometry exists, they inherithilbert_indexthroughmesh_namepluscell_idorface_id - Text layer suffix — Text geometry layers get a
_textsuffix (e.g.,cross_sections_text) to avoid collision with HDF layers
Commands¶
inspect — Discover project structure¶
Prints a summary of what's in a project without extracting any data.
ras2cng inspect path/to/BaldEagleCrkMulti2D
ras2cng inspect path/to/BaldEagleCrkMulti2D.prj
ras2cng inspect path/to/project --json # JSON output for scripting
Output includes: - Project name, CRS, units - Table of geometry files (id, file, type, size) - Table of plan files (id, geometry, flow file, HDF status) - Terrain file count and total size
archive — Export project to consolidated GeoParquet archive¶
# Geometry only (default — safe, fast)
ras2cng archive path/to/BaldEagleCrkMulti2D /output/bald_eagle
# Include plan result variables
ras2cng archive path/to/project /output/archive --results
# Include terrain COG conversion
ras2cng archive path/to/project /output/archive --terrain
# Also extract geometry copy from plan HDF files
ras2cng archive path/to/project /output/archive --plan-geometry
# Specific plans only
ras2cng archive path/to/project /output/archive --results --plans p01,p02
# Large-project-safe selected results: one table per plan/variable, no repeated geometry
ras2cng archive path/to/project /output/archive --results \
--result-variables maximum_depth,maximum_water_surface \
--results-layout variable \
--results-geometry none \
--no-sort
# Full archive
ras2cng archive path/to/project /output/archive --results --terrain
# Generate result rasters (WSE, Depth, Velocity) alongside the archive
ras2cng archive path/to/project /output/archive --results --map
# Generate rasters with a specific render mode
ras2cng archive path/to/project /output/archive --results --map --render-mode sloping
# Create one authoritative COG per named RASMapper terrain
ras2cng archive path/to/project /output/archive --terrain --consolidate-terrain
# Set an explicit target for a mixed-resolution named terrain
ras2cng archive path/to/project /output/archive --terrain --consolidate-terrain \
--terrain-resolution "Existing Terrain=6"
# Opt in: simplify the polygons of the geometry layers (never mesh cells/faces) at 0.1 ft,
# RAS Mapper's own filter. The archive is the data of record and is off by default.
ras2cng archive /path/to/project /output/archive --simplify-tolerance 0.1
# Skip spatial post-processing during extraction
ras2cng archive path/to/project /output/archive --no-sort
# Post-process an existing archive later
ras2cng spatial-index /output/archive
# Fail fast on any extraction error (default: skip and continue)
ras2cng archive path/to/project /output/archive --fail-fast
Large 2D Result Archives¶
The default --results mode writes one GeoParquet per plan and repeats geometry
inside each result variable. That is convenient for direct map rendering, but it
can create large memory spikes on multi-million-cell 2D meshes.
For large projects, prefer:
--results-layout variable— writes one parquet per plan/variable, so ras2cng does not hold every variable for a plan in memory at once.--results-geometry none— writes attribute tables keyed bymesh_namepluscell_idorface_id. Mesh geometry remains in the geometry parquet once.--result-variables ...— restricts export to display or analysis variables that are actually needed.--no-sort— skips the default post-processing pass when the extraction worker is memory constrained. Runras2cng spatial-index /output/archivelater on a worker with enough RAM to addhilbert_indexto geometry andjoin_indexor inheritedhilbert_indexto geometryless result tables without re-extracting the model.
Geometryless result tables are not directly renderable by MapLibre as a layer.
They are the canonical DRY storage form. A viewer or tile service should join
them to the geometry parquet on mesh_name, cell_id for cell variables or
mesh_name, face_id for face variables. If a standalone renderable layer is
required, use --results-geometry point or --results-geometry polygon for a
small selected subset of variables, or generate derived map rasters/tiles.
When spatial post-processing is enabled (the default for archive_project()
unless sort=False or --no-sort is used), geometry parquet files are sorted
by layer,hilbert_index (layers in their written order) and keep their
GeoParquet bbox covering metadata and layer row groups.
Geometryless result tables are sorted by matching geometry hilbert_index when
the archive has a joinable mesh_cells or mesh_faces layer; otherwise they
fall back to deterministic mesh_name plus cell/face key ordering. The manifest
records index_status=spatial_join for inherited spatial ordering and
index_status=join_key for key-only ordering.
Auxiliary raw result summaries are enabled by default with --results. They cover reference
points/lines, SA/2D connections, pumps, pipe conduits/nodes, and 1D structures when those
datasets exist. The exporter reduces time-series datasets in bounded HDF chunks and records
the source geometry join instead of duplicating geometry. Use --mesh-results-only only for
an intentionally limited archive.
Named Terrain Policy¶
Terrain names in the RASMapper configuration are separate surfaces. ras2cng never merges
different named terrains implicitly. With --consolidate-terrain, every named terrain gets
its own authoritative COG, source-inventory provenance JSON, source TIFF footprint
GeoParquet, and any available terrain-modification construction vectors.
The default publication resolution follows a no-upsample policy:
- Native cells finer than 5 ft are reduced to the smallest whole-number native-cell multiple at or above 5 ft.
- Native cells at or above 5 ft retain their native resolution, including 30-ft terrain.
- A named terrain containing mixed native cell sizes requires an explicit
--terrain-resolution NAME=VALUEdecision. The target must be a whole-number multiple of the coarsest source cell. This avoids upsampling without forcing an impractical common multiple for mosaics that mix grids such as 2-foot and 1-meter tiles. - Every source resolution and resampling factor is retained in the terrain provenance JSON.
- The merge is target-grid and windowed, with the first RASMapper source winning where TIFFs overlap. Memory use scales with the processing block rather than the full mosaic.
The output uses transparent nodata, tiled COG storage, ZSTD compression, and overviews. This keeps the numerical terrain usable for Identify and analysis while supporting byte-range web delivery.
1D Steady Result Archives¶
Computed steady-flow plan HDFs are archived as raw cross-section records rather than as
interpolated surfaces. Each row retains the source river, reach, node_id, and profile,
with the HDF values available at that element, such as WSEL, flow, total top width, and total
flow area. Optional variables that are not stored in a row-aligned HDF dataset remain null.
Use the same memory-safe archive command for a 1D steady model:
ras2cng archive path/to/SteadyModel /output/archive --results \
--results-layout variable --results-geometry none --crs EPSG:2249
The result table is stored as results/pNN/steady_cross_sections.parquet. Its manifest entry
declares the exact viewer join: River -> river, Reach -> reach, and RS -> node_id.
No terrain or floodplain surface is fabricated. RASMapper Stored Map COGs are the separate,
authoritative delivery path when an interpolated raster display is available.
Use --crs only when that CRS has been independently verified for the project. It stamps the
source geometry GeoParquet and archive manifest when a public release omits a .rasmap file or
HDF projection record; it does not reproject or infer coordinates.
Python API¶
from pathlib import Path
from ras2cng import archive_project, inspect_project, ProjectInfo
# Inspect without extracting
info = inspect_project(Path("BaldEagleCrkMulti2D"))
print(info.name, info.crs, info.units)
for g in info.geom_files:
print(g.geom_id, g.hdf_path)
# Archive geometry only
manifest = archive_project(
Path("BaldEagleCrkMulti2D"),
Path("/output/archive"),
)
print(f"Wrote {len(manifest.geometry)} geometry entries")
print(f"manifest.json at /output/archive/manifest.json")
# Archive with results
manifest = archive_project(
Path("project.prj"),
Path("/output/archive"),
include_results=True,
plans=["p01", "p03"], # None = all plans
)
# Archive with terrain COG conversion
manifest = archive_project(
Path("project.prj"),
Path("/output/archive"),
include_terrain=True,
)
# Archive with result raster generation
manifest = archive_project(
Path("project.prj"),
Path("/output/archive"),
include_results=True,
map_results=True,
render_mode="horizontal", # or "sloping", "slopingPretty"
)
# Archive with custom options
manifest = archive_project(
Path("project.prj"),
Path("/output/archive"),
include_results=True,
include_terrain=True,
include_plan_geometry=True, # Also extract geometry from plan HDFs
sort=False, # Skip spatial post-processing
skip_errors=False, # Fail fast on errors
)
manifest.json Schema¶
Every archive includes a manifest.json that catalogs all exported layers for downstream tooling (DuckDB, PostGIS sync, PMTiles generation, etc.). Schema 3.0 adds an archive record to geometry and plan entries when the direct path is active (path, fallback_reason, native_layers, python_layers, refused, crs_source, seconds, and omitted when the engine reports it: rows it left out of a layer it wrote, each with layer, source_row, point_count and reason; empty for the layers ras2cng asks the engine for today); layers, columns and the bbox/hilbert_index layout are unchanged, so 2.7 readers keep working. Schema 2.7 switches archive GeoParquet to the bbox struct covering and adds a geoparquet writer record to geometry and plan entries (engine, fallback_reason, hilbert, bbox_covering, rows, row_groups). Schema 2.5 added spatial post-processing metadata, composite raw-result joins, named-terrain provenance, terrain source footprints, and terrain-modification vectors.
{
"schema_version": "3.0",
"project": {
"name": "BaldEagleDamBrk",
"prj_file": "BaldEagleDamBrk.prj",
"source_path": "/abs/path/to/project",
"archive_path": "/abs/path/to/archive",
"created_at": "2026-03-04T15:30:00Z",
"crs": "EPSG:2271",
"units": "US Survey Feet",
"plan_count": 6,
"geom_count": 13
},
"project_parquet": "BaldEagleDamBrk.parquet",
"geometry": [
{
"geom_id": "g01",
"source_file": "BaldEagleDamBrk.g01.hdf",
"file_type": "hdf+text",
"parquet": "BaldEagleDamBrk.g01.parquet",
"plans_using": ["p01", "p05"],
"layers": [
{
"layer": "mesh_cells",
"filter_value": "mesh_cells",
"rows": 87039,
"geometry_type": "Polygon",
"crs": "EPSG:2271",
"hilbert_index": "hilbert_index",
"sort_order": "layer,hilbert_index",
"bbox_columns": ["bbox.xmin", "bbox.ymin", "bbox.xmax", "bbox.ymax"],
"bbox_covering": "struct"
},
{
"layer": "bc_lines",
"filter_value": "bc_lines",
"rows": 12,
"geometry_type": "LineString",
"crs": "EPSG:2271"
},
{
"layer": "cross_sections_text",
"filter_value": "cross_sections_text",
"rows": 192,
"geometry_type": "LineString",
"crs": "EPSG:2271"
}
],
"size_bytes": 2621440
}
],
"results": [
{
"plan_id": "p01",
"plan_title": "Dam Break Scenario",
"geom_id": "g01",
"flow_id": "u01",
"hdf_exists": true,
"completed": true,
"parquet": "",
"layout": "variable",
"geometry_mode": "none",
"variables": [
{
"variable": "maximum_depth",
"filter_value": "maximum_depth",
"rows": 87039,
"parquet": "results/p01/maximum_depth.parquet",
"geometry_mode": "none",
"index_column": "cell_id",
"geometry_filter": "mesh_cells",
"join_columns": {},
"profile_column": "",
"source": "",
"hilbert_index": "hilbert_index",
"join_index": "join_index",
"sort_order": "hilbert_index",
"index_status": "spatial_join",
"size_bytes": 3145728
}
],
"size_bytes": 3145728
}
],
"terrain": [
{
"source_file": "Terrain/Terrain50.tif",
"cog_file": "terrain/Terrain50_cog.tif",
"size_bytes": 12582912,
"crs": "EPSG:2271",
"terrain_name": "Terrain50",
"source_files": ["Terrain/Terrain50.tile-01.tif"],
"target_resolution": 5.0,
"horizontal_units": "Feet",
"provenance_file": "terrain/Terrain50_terrain-provenance.json",
"authoritative": true
}
],
"terrain_sources": [
{
"terrain_name": "Terrain50",
"layers": [{"layer": "terrain_source_footprints", "parquet": "terrain/sources/terrain50/terrain_source_footprints.parquet"}]
}
],
"terrain_modifications": [
{
"terrain_name": "Terrain50",
"layers": [{"layer": "terrain_modification_lines", "parquet": "terrain/modifications/terrain50/terrain_modification_lines.parquet"}]
}
],
"postprocessing": {
"spatial_index": {
"hilbert_column": "hilbert_index",
"join_index_column": "join_index",
"hilbert_level": 16,
"geometry_file_count": 1,
"result_file_count": 1,
"error_count": 0
}
}
}
errors appears only when an export step failed and was skipped (the default
skip_errors=True). Each entry names the step, the file it would have written,
and the error, for example
{"step": "project_metadata", "file": "BaldEagleDamBrk.parquet", "error": "OSError: ..."}.
A failed step leaves no file behind: the project metadata Parquet is staged and
renamed into place only after a complete write, and project_parquet stays
null. The final console line then reads "Archive completed with N error(s)".
Project metadata Parquet¶
{ProjectName}.parquet is the union of ras-commander's plan_df, geom_df,
flow_df, unsteady_df, boundaries_df, results_df, and rasmap_df, with a
_table column naming the source frame. List columns are Parquet lists and
dict columns such as current_settings are Parquet structs.
rasmap_field_errors (failed RASMapper field -> error message) is a Parquet
MAP<string, string>, because its keys vary by project and are usually empty,
and a struct with no fields cannot be stored in Parquet. Read it back as Python
dicts with pq.read_table(path).to_pandas(maps_as_pydicts="strict").
Using the manifest with Python¶
from ras2cng import Manifest
m = Manifest.load(Path("/output/archive/manifest.json"))
print(m.geom_ids) # ['g01', 'g06', ...]
print(m.plan_ids) # ['p01', 'p02', ...]
# List geometry and result parquet files
print(m.layer_paths()) # ['BaldEagle.g01.parquet', ...]
print(m.result_paths()) # ['BaldEagle.p01.parquet', 'results/p01/maximum_depth.parquet', ...]
# Browse geometry layers within each consolidated file
for entry in m.geometry:
print(f"{entry['geom_id']}: {entry['parquet']}")
for layer in entry["layers"]:
print(f" WHERE layer = '{layer['filter_value']}' → {layer['rows']} rows")
When to Use Each Flag¶
| Flag | When to use |
|---|---|
| (default, no flags) | Archiving geometry for GIS analysis, tile generation, or multi-project inventory |
--results |
You need flood depth/velocity maps and the model has been run |
--terrain |
You need the DEM in cloud-optimized format for raster tile generation |
--map |
Generate result rasters (WSE, Depth, Velocity) via RasStoreMapHelper |
--consolidate-terrain |
Merge multiple terrain TIFFs into a single COG |
--render-mode |
Set water surface render mode: horizontal, sloping, or slopingPretty |
--plan-geometry |
Also extract the geometry copy embedded in plan HDF files |
--plans p01,p02 |
Only specific scenarios are relevant (saves time/space on large projects) |
--no-sort |
Skip default spatial post-processing; run ras2cng spatial-index ARCHIVE_DIR later |
--fail-fast |
Debugging extraction issues; default is to skip and continue |
--ras-version |
Specify HEC-RAS version for RasProcess mapping |
--rasprocess |
Path to HEC-RAS install directory (required on Linux/Wine) |
Querying with DuckDB¶
After archiving, query consolidated files using the layer column:
from ras2cng import DuckSession
with DuckSession() as db:
db.register_parquet("/output/archive/BaldEagle.g01.parquet")
# List available layers
df = db.query("SELECT DISTINCT layer FROM _")
print(df)
# Query a specific layer
df = db.query("""
SELECT COUNT(*) as n, AVG(ST_Area(geometry)) as avg_area
FROM _ WHERE layer = 'mesh_cells'
""")
print(df)
See DuckDB Queries for more examples.