Cloud-Native Geometry Export with ras2cng¶
Export HEC-RAS geometry to cloud-native GeoParquet format using ras2cng, query with DuckDB, generate PMTiles for web maps, and display interactively in the notebook.
USE_LOCAL_SOURCE = False
if USE_LOCAL_SOURCE:
import sys
from pathlib import Path
local_path = str(Path.cwd().parent)
if local_path not in sys.path:
sys.path.insert(0, local_path)
print(f"LOCAL SOURCE MODE: Loading from {local_path}/ras_commander")
else:
print("PIP PACKAGE MODE: Loading installed ras-commander")
from ras_commander import RasExamples, init_ras_project, ras
from pathlib import Path
import geopandas as gpd
import ras_commander
print(f"ras-commander: {ras_commander.__file__}")
import ras2cng
print(f"ras2cng: {ras2cng.__file__}")
Prerequisites¶
- ras-commander installed:
pip install ras-commander - ras2cng installed:
pip install "ras2cng[duckdb]" - No HEC-RAS required: Geometry export works from preprocessed HDF files
- Optional:
tippecanoe+pmtilesCLIs for PMTiles generation (conda install -c conda-forge tippecanoe pmtiles)
What You'll Learn¶
- Export 2D mesh geometry (cells, boundaries, structures) to GeoParquet
- Export 1D cross sections from text geometry files
- Query GeoParquet with DuckDB SQL
- Display geometry interactively in the notebook
- Generate PMTiles for serverless web maps
Related Notebooks¶
- 410_2d_hdf_data_extraction.ipynb — Extract 2D mesh data with ras-commander
- 201_1d_plaintext_geometry.ipynb — Parse 1D geometry
- 961_cloud_native_results_export.ipynb — Export simulation results with ras2cng
Parameters¶
PROJECT_2D = "BaldEagleCrkMulti2D"
PROJECT_1D = "Muncie"
OUTPUT_DIR = Path("out/960_cloud_native_geometry")
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
Example 1: 2D Mesh Geometry (BaldEagleCrkMulti2D)¶
Export mesh cell polygons, boundary condition lines, and breaklines from a 2D HEC-RAS geometry HDF file.
project_path = RasExamples.extract_project(PROJECT_2D)
print(f"Project: {project_path}")
geom_hdfs = sorted(project_path.glob("*.g??.hdf"))
print(f"Geometry HDF files: {[f.name for f in geom_hdfs]}")
geom_hdf = geom_hdfs[0]
print(f"Using: {geom_hdf.name} ({geom_hdf.stat().st_size / 1e6:.1f} MB)")
Export Mesh Cells to GeoParquet¶
The mesh_cells layer contains the computational mesh — each cell is a polygon with its cell ID, mesh name, and area.
from ras2cng import export_geometry_layers
mesh_out = OUTPUT_DIR / "mesh_cells.parquet"
export_geometry_layers(geom_hdf, mesh_out, layer="mesh_cells")
gdf_mesh = gpd.read_parquet(mesh_out)
print(f"Exported {len(gdf_mesh)} mesh cells")
print(f"CRS: {gdf_mesh.crs}")
print(f"File size: {mesh_out.stat().st_size / 1e3:.1f} KB")
gdf_mesh.head()
Export Additional Geometry Layers¶
ras2cng can export 10 HDF layers and 3 text geometry layers. Export boundary condition lines and breaklines for a complete picture.
from ras2cng import export_all_hdf_layers, HDF_LAYERS
import pandas as pd
print(f"Available HDF layers: {HDF_LAYERS}")
all_layers_dir = OUTPUT_DIR / "all_layers"
layer_paths = export_all_hdf_layers(geom_hdf, all_layers_dir)
print(f"\nExported {len(layer_paths)} layers:")
for layer_name, pq_path in layer_paths.items():
gdf_layer = gpd.read_parquet(pq_path)
print(f" {layer_name}: {len(gdf_layer)} features ({pq_path.stat().st_size / 1e3:.1f} KB)")
print(f"\nAll layer parquets in: {all_layers_dir}")
DuckDB Analytics¶
Query the GeoParquet file with SQL using DuckDB. The table alias is always _.
from ras2cng import query_parquet, DuckSession
print("Layer statistics:")
for layer_name, pq_path in layer_paths.items():
gdf_l = gpd.read_parquet(pq_path)
print(f" {layer_name:25s} {len(gdf_l):>6} features")
df_mesh_summary = query_parquet(
mesh_out,
"""
SELECT
mesh_name,
COUNT(*) AS cell_count,
ROUND(MIN(ST_Area(geometry)), 1) AS min_area,
ROUND(AVG(ST_Area(geometry)), 1) AS avg_area,
ROUND(MAX(ST_Area(geometry)), 1) AS max_area
FROM _
GROUP BY mesh_name
"""
)
print("Mesh cell area statistics (sq ft):")
df_mesh_summary
Interactive Map¶
Display the mesh geometry interactively using GeoPandas .explore() (uses Folium/Leaflet under the hood).
try:
gdf_mesh_wgs84 = gdf_mesh.to_crs("EPSG:4326")
gdf_mesh_wgs84.explore(
column="mesh_name",
cmap="Set2",
tooltip=["mesh_name", "cell_id"],
style_kwds={"weight": 0.5, "fillOpacity": 0.3},
tiles="CartoDB positron",
)
except ImportError as e:
print(f"Interactive map requires folium and mapclassify: {e}")
print("Install with: pip install folium mapclassify")
print(f"\nMesh summary: {len(gdf_mesh)} cells, CRS={gdf_mesh.crs}")
Example 2: 1D Cross Sections (Muncie)¶
Export cross section cut lines from a plain text geometry file (.g##). This demonstrates ras2cng's text geometry support.
import re
muncie_path = RasExamples.extract_project(PROJECT_1D)
print(f"Project: {muncie_path}")
text_geoms = [
f for f in sorted(muncie_path.iterdir())
if re.match(r".*\.g\d{2}$", f.name)
]
print(f"Text geometry files: {[f.name for f in text_geoms]}")
geom_hdfs_1d = sorted(muncie_path.glob("*.g??.hdf"))
print(f"Geometry HDF files: {[f.name for f in geom_hdfs_1d]}")
from ras2cng import export_geometry_layers, ALL_TEXT_LAYERS
print(f"Text geometry layers: {ALL_TEXT_LAYERS}")
if text_geoms:
text_geom = text_geoms[0]
xs_out = OUTPUT_DIR / "muncie_cross_sections.parquet"
export_geometry_layers(text_geom, xs_out, layer="cross_sections")
gdf_xs = gpd.read_parquet(xs_out)
print(f"\nExported {len(gdf_xs)} cross sections from {text_geom.name}")
print(f"Columns: {list(gdf_xs.columns)}")
gdf_xs.head()
elif geom_hdfs_1d:
geom_hdf_1d = geom_hdfs_1d[0]
xs_out = OUTPUT_DIR / "muncie_cross_sections.parquet"
export_geometry_layers(geom_hdf_1d, xs_out, layer="cross_sections")
gdf_xs = gpd.read_parquet(xs_out)
print(f"\nExported {len(gdf_xs)} cross sections from {geom_hdf_1d.name}")
gdf_xs.head()
else:
print("No geometry files found in Muncie project")
gdf_xs = None
if xs_out.exists():
df_xs_stats = query_parquet(
xs_out,
"""
SELECT
COUNT(*) AS total_xs,
ROUND(MIN(ST_Length(geometry)), 1) AS min_length,
ROUND(AVG(ST_Length(geometry)), 1) AS avg_length,
ROUND(MAX(ST_Length(geometry)), 1) AS max_length
FROM _
"""
)
print("Cross section statistics:")
print(df_xs_stats.to_string(index=False))
if gdf_xs is not None and len(gdf_xs) > 0:
try:
if gdf_xs.crs is None:
print("No CRS on text geometry — displaying without reprojection")
gdf_xs.explore(
color="blue",
tooltip=gdf_xs.columns.drop("geometry").tolist()[:5],
style_kwds={"weight": 2},
)
else:
gdf_xs_wgs84 = gdf_xs.to_crs("EPSG:4326")
gdf_xs_wgs84.explore(
color="blue",
tooltip=gdf_xs_wgs84.columns.drop("geometry").tolist()[:5],
style_kwds={"weight": 2},
tiles="CartoDB positron",
)
except ImportError as e:
print(f"Interactive map requires folium and mapclassify: {e}")
print(f"\nCross section summary: {len(gdf_xs)} features")
PMTiles Generation (Optional)¶
Generate vector PMTiles from the GeoParquet files for serverless web map deployment. Requires tippecanoe and pmtiles CLIs on PATH.
import shutil
has_tippecanoe = bool(shutil.which("tippecanoe"))
has_pmtiles_cli = bool(shutil.which("pmtiles"))
has_tools = has_tippecanoe and has_pmtiles_cli
print(f"tippecanoe: {'found' if has_tippecanoe else 'NOT FOUND'}")
print(f"pmtiles: {'found' if has_pmtiles_cli else 'NOT FOUND'}")
if not has_tools:
print("\nPMTiles generation requires tippecanoe + pmtiles CLIs.")
print("Install: conda install -c conda-forge tippecanoe pmtiles")
print("Skipping PMTiles generation (GeoParquet export still works).")
if has_tools:
from ras2cng import generate_pmtiles_from_input
pmtiles_out = OUTPUT_DIR / "mesh_cells.pmtiles"
generate_pmtiles_from_input(
mesh_out,
pmtiles_out,
layer_name="mesh_cells",
min_zoom=8,
max_zoom=14,
)
print(f"PMTiles: {pmtiles_out.name} ({pmtiles_out.stat().st_size / 1e6:.1f} MB)")
print("\nServe from any static host (S3, GitHub Pages, Cloudflare R2).")
print("No tile server needed — PMTiles are self-contained.")
else:
print("Skipped — install tippecanoe + pmtiles to generate tiles.")
print("\nCLI equivalent:")
print(f" ras2cng pmtiles {mesh_out} {OUTPUT_DIR / 'mesh_cells.pmtiles'} --layer mesh_cells --min-zoom 8 --max-zoom 14")
Displaying PMTiles in a Web Map¶
Once generated, PMTiles can be displayed in any MapLibre GL JS application with the PMTiles protocol adapter. Here's a minimal HTML example:
<script src="https://unpkg.com/maplibre-gl/dist/maplibre-gl.js"></script>
<script src="https://unpkg.com/pmtiles/dist/pmtiles.js"></script>
<script>
let protocol = new pmtiles.Protocol();
maplibregl.addProtocol("pmtiles", protocol.tile);
const map = new maplibregl.Map({
container: "map",
style: {
version: 8,
sources: {
mesh: {
type: "vector",
url: "pmtiles://./mesh_cells.pmtiles",
},
},
layers: [
{
id: "mesh-fill",
source: "mesh",
"source-layer": "mesh_cells",
type: "fill",
paint: { "fill-color": "#0080ff", "fill-opacity": 0.3 },
},
],
},
});
</script>
No tile server is needed — PMTiles serve directly from S3, GitHub Pages, or any static file host.
Output Summary¶
print("Output files:")
for f in sorted(OUTPUT_DIR.glob("*")):
print(f" {f.name:40s} {f.stat().st_size / 1e3:>8.1f} KB")
Key Takeaways¶
- ras2cng exports HEC-RAS geometry to GeoParquet — cloud-optimized, ZSTD-compressed, spatially sorted
- 10 HDF layers + 3 text layers available: mesh cells, BC lines, breaklines, cross sections, structures, and more
- DuckDB provides SQL analytics on GeoParquet files with full spatial function support
- PMTiles enable serverless web maps — no tile server required, serve from static hosting
- GeoPandas
.explore()gives instant interactive maps in the notebook
CLI Equivalents¶
# Export mesh cells
ras2cng geometry model.g01.hdf mesh_cells.parquet --layer mesh_cells
# Export all layers to consolidated file
ras2cng archive path/to/project/ ./archive/
# Query with DuckDB
ras2cng query mesh_cells.parquet "SELECT mesh_name, COUNT(*) FROM _ GROUP BY mesh_name"
# Generate PMTiles
ras2cng pmtiles mesh_cells.parquet mesh.pmtiles --layer mesh_cells
Next Steps¶
- 961_cloud_native_results_export.ipynb — Export simulation results and build flood depth maps
- ras2cng documentation — Complete API reference
- ras2cng GitHub — Source code and issues