# RAS Commander Documentation > Python library for automating HEC-RAS operations # LLM Agent Guide # LLM Agent Guide This page is the canonical, copy-runnable landing page for **LLM agents** building on RAS Commander. It is intentionally dense: install, the cardinal rules, and complete recipes you can paste and run. Machine-readable docs A curated index of this site is published at [`https://rascommander.info/llms.txt`](https://rascommander.info/llms.txt), with the full concatenated corpus at [`https://rascommander.info/llms-full.txt`](https://rascommander.info/llms-full.txt). Most pages are also mirrored as raw markdown (append `.md` to the page URL). ## Install Bash ```bash pip install ras-commander ``` RAS Commander automates an **installed HEC-RAS** (`Ras.exe`). Execution recipes require HEC-RAS on Windows; parsing/extraction recipes work anywhere the result files exist. ## Cardinal rules 1. **Static classes — call, don't instantiate.** `RasCmdr`, `RasPlan`, `RasGeometry`, `RasUnsteady`, and every `Hdf*` class are static namespaces. Call methods directly: `RasCmdr.compute_plan("01")`, **not** `RasCmdr().compute_plan(...)`. The only object you instantiate (indirectly) is the project object via `init_ras_project()`. 1. **DataFrame-first.** After `init_ras_project()`, the project metadata lives in pandas DataFrames: `plan_df`, `geom_df`, `flow_df`, `unsteady_df`, `boundaries_df`, `rasmap_df`, `results_df`. Use them as the source of truth for file paths, plan/geometry linkage, and boundary inventory instead of globbing the folder. See the [DataFrame Reference](https://rascommander.info/reference/dataframe-reference/index.md) for every column. 1. **`ras_object=` discipline for multi-project work.** A global `ras` object is convenient for a single project. The moment you work with more than one project, capture the object returned by `init_ras_project(...)` and pass it explicitly: `RasCmdr.compute_plan("01", ras_object=my_ras)`. Mixing the global with a second project is the most common agent bug. 1. **`RasExamples.extract_project(...)` for real test data.** Do not invent or mock HEC-RAS projects. Extract a bundled, real project. Use `suffix=` to get an isolated copy you can mutate without colliding with other runs: `RasExamples.extract_project("Muncie", suffix="_run1")`. 1. **Detect steady vs unsteady before extracting results.** Steady and unsteady HDF results have different structures. Call `HdfResultsPlan.is_steady_plan(hdf_path)` first. 1. **Normalize plan/geometry numbers.** `"1"`, `"01"`, `"p01"`, and `Path("Model.p01")` all resolve to plan `01`. Most APIs accept any of these; `RasUtils.normalize_ras_number()` makes it explicit. ## Recipe 1 — Initialize and inspect plans Python ```python from ras_commander import init_ras_project, RasExamples # Extract a real bundled project to an isolated copy project_path = RasExamples.extract_project("Muncie", suffix="_inspect") # init_ras_project returns the project object (also sets the global `ras`) my_ras = init_ras_project(project_path, "6.6") # DataFrame-first inspection print(my_ras.plan_df[["plan_number", "Plan Title", "geometry_number", "unsteady_number", "flow_type"]]) print(my_ras.geom_df[["geom_number", "geom_title", "has_2d_mesh", "num_cross_sections"]]) # Which plans already have computed HDF results? computed = my_ras.plan_df[my_ras.plan_df["HDF_Results_Path"].notna()] print(f"{len(computed)} of {len(my_ras.plan_df)} plans have HDF results") ``` ## Recipe 2 — Compute a plan, then verify success Python ```python from ras_commander import init_ras_project, RasExamples, RasCmdr project_path = RasExamples.extract_project("Muncie", suffix="_compute") my_ras = init_ras_project(project_path, "6.6") # Execute to a separate destination folder so the source stays immutable success = RasCmdr.compute_plan( "01", dest_folder=str(project_path) + "_results", overwrite_dest=True, ras_object=my_ras, ) print(f"Execution {'succeeded' if success else 'failed'}") # Refresh and read the results summary (completion, errors, runtime) my_ras.update_results_df(["01"]) print(my_ras.results_df[["plan_number", "completed", "has_errors", "runtime_complete_process_hours"]]) ``` ## Recipe 3 — Extract maximum WSE from a 2D mesh Python ```python from ras_commander import init_ras_project, RasExamples from ras_commander.hdf import HdfResultsPlan, HdfResultsMesh # A 2D example with computed results project_path = RasExamples.extract_project("BaldEagleCrkMulti2D", suffix="_wse") my_ras = init_ras_project(project_path, "6.6") # Pick a plan that has results plan_number = my_ras.plan_df[my_ras.plan_df["HDF_Results_Path"].notna()] \ ["plan_number"].iloc[0] # Steady and unsteady results differ — detect first hdf_path = my_ras.plan_df.set_index("plan_number") \ .loc[plan_number, "HDF_Results_Path"] if HdfResultsPlan.is_steady_plan(hdf_path): raise RuntimeError("This recipe expects an unsteady 2D plan") # Maximum water surface elevation per mesh cell (GeoDataFrame, one row per cell) max_ws = HdfResultsMesh.get_mesh_max_ws(plan_number, ras_object=my_ras) print(max_ws.columns.tolist()) print(f"Peak WSE across mesh: {max_ws['maximum_water_surface'].max():.2f} ft") ``` ## Recipe 4 — Audit boundary conditions and DSS paths Python ```python from ras_commander import init_ras_project, RasExamples project_path = RasExamples.extract_project("BaldEagleCrkMulti2D", suffix="_bc") my_ras = init_ras_project(project_path, "6.6") bc = my_ras.boundaries_df # Inventory by type print(bc["bc_type"].value_counts()) # DSS-backed boundaries with their parsed pathname parts dss = bc[bc["Use DSS"] == "True"] print(dss[["unsteady_number", "bc_type", "river_reach_name", "dss_part_b", "dss_part_c", "DSS File"]]) ``` ## Where to go next - [DataFrame Reference](https://rascommander.info/reference/dataframe-reference/index.md) — every column of every project DataFrame, with dtypes and meanings (the #1 thing an integrating agent cannot introspect without running HEC-RAS). - [HDF Data Extraction](https://rascommander.info/user-guide/hdf-data-extraction/index.md) — mesh/cross-section result APIs. - [Plan Execution](https://rascommander.info/user-guide/plan-execution/index.md) — parallel compute, destination folders. - [API Reference](https://rascommander.info/api/index.md) — full class/method signatures from docstrings. # Getting Started # Installation ## Requirements - **Python**: 3.10 or higher (3.13 recommended for new installations) - **HEC-RAS**: 6.0+ recommended (3.x-5.x supported via RasControl) - **Operating System**: Windows (for full HEC-RAS execution), Linux (map generation via Wine), Mac (HDF analysis only) ## Prerequisites ### Install Astral uv (Recommended) uv is a fast Python package manager required for Claude Code agent operations and recommended for all users: PowerShell ```powershell irm https://astral.sh/uv/install.ps1 | iex ``` Bash ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` Verify installation: Bash ```bash uv --version # Should show: uv 0.5.x or later ``` Why uv? - 10-100x faster than pip for package installation - Required for Claude Code agent scripts and skills - Enables one-off script execution with `uvx` - Better dependency resolution ### Install Anaconda (Recommended for Notebooks) For Jupyter notebook work, we recommend Anaconda: - **Download**: - **Verify**: `conda --version` ## Standard Environment Names RAS Commander uses two standard environment names: | Environment | Purpose | Install Type | | ------------------- | ------------------------- | ---------------- | | **`RasCommander`** | Standard user environment | pip package | | **`rascmdr_local`** | Development environment | editable install | ## Install from PyPI (Most Users) **Environment name**: `RasCommander` **Use this if**: You're using ras-commander, NOT editing its source code. Bash ```bash # Create environment conda create -n RasCommander python=3.13 conda activate RasCommander # Install ras-commander pip install ras-commander # Install Jupyter (for notebooks) pip install jupyter ipykernel python -m ipykernel install --user --name RasCommander --display-name "Python (RasCommander)" ``` Quick Install For a quick install without Jupyter: Bash ```bash pip install --upgrade ras-commander ``` ## Core Dependencies These are installed automatically with `pip install ras-commander`: Bash ```bash h5py numpy pandas requests tqdm scipy xarray geopandas matplotlib shapely rasterstats rtree pywin32 psutil ``` ## Optional Dependencies ### Notebook Examples Additional packages for running the example notebooks: Bash ```bash pip install rasterio pyproj ``` ### DSS File Operations For reading HEC-DSS boundary condition files: Bash ```bash pip install pyjnius ``` Java Required DSS operations require Java 8+ (JRE or JDK) installed on your system. ### Linux/Wine (Headless RasProcess Workflows) On Linux, `RasProcess.exe` can run under Wine for documented RasProcess workflows such as stored map generation (WSE, Depth, Velocity TIFs) and native reference validation of geometry associations. This enables headless CI/CD pipelines and cloud-based map or QA/QC workflows. **Requirements**: - Wine 8.0+ (64-bit prefix) - .NET Framework 4.8 (installed via winetricks) - HEC-RAS DLLs copied from a Windows installation Bash ```bash # Step 1: Install Wine sudo dpkg --add-architecture i386 sudo apt update sudo apt install -y wine wine64 wine32 winetricks cabextract # Step 2: Create Wine prefix with .NET 4.8 export WINEPREFIX=/opt/hecras-wine export WINEARCH=win64 wineboot --init winetricks -q dotnet48 # ~15 min, installs .NET Framework 4.8 winetricks -q gdiplus # Native GDI+ (required for System.Drawing) winetricks -q corefonts # Arial, Times New Roman, etc. # Step 3: Copy HEC-RAS DLLs from a Windows machine # From C:\Program Files (x86)\HEC\HEC-RAS\6.6\ copy all DLLs and the GDAL/ folder # to /opt/hecras-wine/drive_c/HEC-RAS/6.6/ ``` Detailed Setup Instructions Run `RasProcess.setup_wine_environment()` in Python to print the complete list of required DLLs and step-by-step instructions. Validated Environment Verified on the CLB07 Proxmox container `ras2cng-wine` (Debian 13, Wine 11.0). Custom `wine_executable` paths or wrappers are supported; helper tools such as `winepath` are resolved automatically. Scope Wine support covers wrapped `RasProcess.exe` workflows. Full HEC-RAS simulation (`Ras.exe`) still requires Windows. HDF analysis and Python-native geometry association writes work on Linux without Wine. **Verify installation**: Python ```python from ras_commander import RasProcess # Check Wine environment is configured correctly status = RasProcess.check_wine_environment() print(status) ``` **Usage** (no code changes needed -- auto-detected): Python ```python from ras_commander import RasProcess # Same API as Windows -- Wine wrapping is automatic results = RasProcess.store_maps(plan_number="01", wse=True, depth=True) ``` Python ```python # Optional native reference validator. This mutates the supplied HDF in place, # so use it on a disposable copy or intentional validation target. validation = RasProcess.validate_geometry_association_cli( "MyModel.g01.hdf", terrain_hdf_path="Terrain/ExistingTerrain.hdf", ras_version="7.0", ) print(validation["passed"]) ``` ### Remote Execution For distributed computation across multiple machines: Bash ```bash # Individual backends pip install paramiko # SSH remote execution pip install pywinrm # WinRM remote execution pip install docker # Docker container execution pip install boto3 # AWS EC2 execution pip install azure-identity azure-mgmt-compute # Azure execution # Or install all at once pip install ras-commander[remote-all] ``` ## Development Installation **Environment name**: `rascmdr_local` **Use this if**: You're editing ras-commander source code or contributing to the library. Bash ```bash # Clone the repository git clone https://github.com/gpt-cmdr/ras-commander.git cd ras-commander # Create development environment conda create -n rascmdr_local python=3.13 conda activate rascmdr_local # Install in editable mode pip install -e . # Install Jupyter (for notebooks) pip install jupyter ipykernel python -m ipykernel install --user --name rascmdr_local --display-name "Python (rascmdr_local)" ``` Using uv for Development For faster package installation during development: Bash ```bash uv venv .venv .venv\Scripts\activate # Windows uv pip install -e . ``` ### Flexible Import Pattern Scripts in the repository use this pattern to work both with installed packages and development mode: Python ```python from pathlib import Path import sys try: from ras_commander import init_ras_project, RasCmdr except ImportError: # Add parent directory to path for development sys.path.append(str(Path(__file__).resolve().parent.parent)) from ras_commander import init_ras_project, RasCmdr ``` ## Verifying Installation Python ```python import ras_commander print(f"RAS Commander version: {ras_commander.__version__}") # Test basic imports from ras_commander import ( init_ras_project, RasCmdr, RasPlan, RasExamples, ras ) print("All imports successful!") ``` ## Troubleshooting ### Dependency Conflicts with NumPy If you encounter numpy-related errors: Bash ```bash # Clear local pip packages # Windows: Delete C:\Users\\AppData\Roaming\Python\ # Create fresh environment python -m venv fresh_env fresh_env\Scripts\activate pip install ras-commander ``` ### HEC-RAS Not Found If HEC-RAS execution fails: 1. Verify HEC-RAS is installed (default: `C:\Program Files\HEC\HEC-RAS\7.x\`) 1. Specify the full path to Ras.exe: Python ```python init_ras_project("/path/to/project", r"D:\Programs\HEC\HEC-RAS\6.5\Ras.exe") ``` ### pywin32 Errors For COM interface issues on Windows: Bash ```bash pip uninstall pywin32 pip install pywin32 python -c "import win32com.client" # Test import ``` # Quick Start This guide covers the essential operations to get started with RAS Commander. ## Initialize a Project Python ```python from ras_commander import init_ras_project, ras # Initialize with version number (uses default installation path) init_ras_project(r"C:\Projects\MyRASProject", "6.5") # Or specify full path to Ras.exe init_ras_project(r"C:\Projects\MyRASProject", r"D:\HEC-RAS\6.5\Ras.exe") ``` The global `ras` object now contains your project data. ## Explore Project Structure Python ```python # View available plans print(ras.plan_df) # View geometry files print(ras.geom_df) # View unsteady flow files print(ras.unsteady_df) # View boundary conditions print(ras.boundaries_df) # View HDF result files print(ras.get_hdf_entries()) ``` ## Execute Plans ### Single Plan Python ```python from ras_commander import RasCmdr # Execute plan 01 success = RasCmdr.compute_plan("01") print(f"Execution {'succeeded' if success else 'failed'}") ``` ### Execute to Destination Folder Python ```python # Execute with results saved to separate folder success = RasCmdr.compute_plan( "01", dest_folder=r"C:\Results\Run1", overwrite_dest=True ) ``` ### Parallel Execution Python ```python # Run multiple plans simultaneously results = RasCmdr.compute_parallel( plan_number=["01", "02", "03"], max_workers=3, num_cores=2 ) for plan, success in results.items(): print(f"Plan {plan}: {'OK' if success else 'FAILED'}") ``` ## Extract HDF Results Python ```python from ras_commander import HdfResultsMesh, HdfResultsXsec # Get path to HDF file hdf_path = ras.plan_df.loc[ras.plan_df['plan_number'] == '01', 'hdf_path'].iloc[0] # Extract maximum water surface elevation (2D) max_wse = HdfResultsMesh.get_mesh_max_ws(hdf_path) print(max_wse.head()) # Extract cross-section results (1D) xsec_wse = HdfResultsXsec.get_xsec_timeseries(hdf_path, "Water Surface") print(xsec_wse.head()) ``` ## Modify Plan Parameters Python ```python from ras_commander import RasPlan # Set number of compute cores RasPlan.set_num_cores("01", 4) # Change geometry file RasPlan.set_geom("01", "02") # Use geometry file g02 # Update computation interval RasPlan.set_computation_interval("01", "5MIN") # Update description RasPlan.set_description("01", "Modified run with 5-minute interval") ``` ## Work with Example Projects Python ```python from ras_commander import RasExamples # List available example projects all_projects = RasExamples.list_projects() print(all_projects) # Extract a project for testing path = RasExamples.extract_project("Muncie") print(f"Extracted to: {path}") # Initialize the extracted project init_ras_project(path, "6.5") ``` ## Multiple Projects Python ```python from ras_commander import RasPrj, init_ras_project, RasCmdr # Create separate project instances project1 = RasPrj() project2 = RasPrj() # Initialize each init_ras_project(r"C:\Project1", "6.5", ras_object=project1) init_ras_project(r"C:\Project2", "6.5", ras_object=project2) # Execute plans specifying which project RasCmdr.compute_plan("01", ras_object=project1) RasCmdr.compute_plan("01", ras_object=project2) ``` ## Next Steps - **[Project Initialization](https://rascommander.info/getting-started/project-initialization/index.md)**: Detailed project setup options - **[Plan Execution](https://rascommander.info/user-guide/plan-execution/index.md)**: Advanced execution modes - **[HDF Data Extraction](https://rascommander.info/user-guide/hdf-data-extraction/index.md)**: Working with results - **[Example Notebooks](https://rascommander.info/examples/index.md)**: 30+ complete examples # Project Initialization Understanding how to properly initialize HEC-RAS projects is fundamental to using RAS Commander effectively. ## Basic Initialization The `init_ras_project()` function discovers project files and populates the RAS object with DataFrames containing project metadata. Python ```python from ras_commander import init_ras_project, ras # Basic initialization with version number init_ras_project(r"C:\Projects\MyProject", "6.5") ``` ## HEC-RAS Executable Specification You can specify the HEC-RAS executable in several ways: ### Version Number Use a version string to reference the default installation path: Python ```python # Standard installation on C: drive init_ras_project(r"C:\Projects\MyProject", "6.5") # Also accepts formats like "65", "6.5", "6.5.0" init_ras_project(r"C:\Projects\MyProject", "66") ``` ### Full Executable Path For non-standard installations or HEC-RAS on other drives: Python ```python init_ras_project( r"C:\Projects\MyProject", r"D:\Programs\HEC\HEC-RAS\6.5\Ras.exe" ) ``` ### Auto-Detection Omit the version to attempt detection from plan files: Python ```python init_ras_project(r"C:\Projects\MyProject") ``` Warning Auto-detection may fail if plan files don't contain version information. ## Understanding the RAS Object After initialization, the `ras` object contains: ### DataFrames | Attribute | Description | | ------------------- | -------------------------------------------------------------- | | `ras.plan_df` | Plan files (.p01, .p02, etc.) with paths, titles, linked files | | `ras.geom_df` | Geometry files (.g01, .g02, etc.) with basic metadata | | `ras.flow_df` | Steady flow files (.f01, .f02, etc.) | | `ras.unsteady_df` | Unsteady flow files (.u01, .u02, etc.) | | `ras.boundaries_df` | Boundary conditions extracted from unsteady files | | `ras.rasmap_df` | RASMapper configuration paths (terrain, land cover) | ### Properties | Attribute | Description | | -------------------- | -------------------------------- | | `ras.project_folder` | Path to project directory | | `ras.project_name` | Project name (without extension) | | `ras.prj_file` | Path to .prj file | | `ras.ras_exe_path` | Path to HEC-RAS executable | ### Example: Exploring a Project Python ```python from ras_commander import init_ras_project, ras init_ras_project(r"C:\Projects\Muncie", "6.5") print(f"Project: {ras.project_name}") print(f"Folder: {ras.project_folder}") print(f"HEC-RAS: {ras.ras_exe_path}") print("\n=== Plans ===") print(ras.plan_df[['plan_number', 'plan_title', 'geom_number', 'hdf_path']]) print("\n=== Geometry Files ===") print(ras.geom_df[['geom_number', 'file_path']]) print("\n=== Boundary Conditions ===") print(ras.boundaries_df[['Name', 'Type', 'Interval']]) ``` ## DataFrame Reference Detailed information about each DataFrame available after project initialization. ### plan_df Columns | Column | Type | Description | | -------------------------------- | ---- | --------------------------------- | | `plan_number` | str | Plan identifier ("01", "02", ...) | | `full_path` | Path | Full path to .p## file | | `Short Identifier` | str | User-defined short name | | `Plan Title` | str | User-defined full title | | `Geom File` | str | Geometry file reference (g##) | | `Flow File` | str | Flow file reference (f## or u##) | | `unsteady_number` | str | Unsteady flow number if used | | `geometry_number` | str | Geometry number used | | `Simulation Date` | str | Start/end dates string | | `Computation Interval` | str | Time step (e.g., "2MIN") | | `Run HTab`, `Run UNet` | int | Run flags | | `UNET D1 Cores`, `UNET D2 Cores` | int | Core settings | | `HDF_Results_Path` | Path | Path to results HDF if exists | ### boundaries_df Columns | Column | Type | Description | | --------------------------- | ---- | ------------------------- | | `unsteady_number` | str | Links to .u## file | | `boundary_condition_number` | int | Sequential ID within file | | `river_reach_name` | str | River/reach location | | `river_station` | str | Station location | | `storage_area_name` | str | SA name (if applicable) | | `bc_type` | str | Boundary type | | `hydrograph_type` | str | Specific hydrograph type | | `Interval` | str | Time interval | | `hydrograph_num_values` | int | Number of data points | ### Common Query Patterns Python ```python # Find plans using specific geometry g01_plans = ras.plan_df[ras.plan_df['geometry_number'] == '01'] # Get plans with completed results completed = ras.plan_df[ras.plan_df['HDF_Results_Path'].notna()] # Count boundary conditions by type bc_counts = ras.boundaries_df['bc_type'].value_counts() # Get flow hydrographs only flow_bcs = ras.boundaries_df[ras.boundaries_df['bc_type'] == 'Flow Hydrograph'] # Find unsteady files using restart restart_files = ras.unsteady_df[ras.unsteady_df['Use Restart'] == 'True'] ``` ### HDF Entries Helper Python ```python # Get only plans with HDF results hdf_entries = ras.get_hdf_entries() print(f"Plans with results: {len(hdf_entries)}") ``` ## Working with Multiple Projects For scripts that need to work with multiple HEC-RAS projects simultaneously: ### Using Named Instances Python ```python from ras_commander import RasPrj, init_ras_project, RasCmdr # Create separate instances upstream_model = RasPrj() downstream_model = RasPrj() # Initialize each init_ras_project(r"C:\Projects\Upstream", "6.5", ras_object=upstream_model) init_ras_project(r"C:\Projects\Downstream", "6.5", ras_object=downstream_model) # Access data from each print(f"Upstream plans: {upstream_model.plan_df['plan_number'].tolist()}") print(f"Downstream plans: {downstream_model.plan_df['plan_number'].tolist()}") # Execute specifying the project RasCmdr.compute_plan("01", ras_object=upstream_model) RasCmdr.compute_plan("01", ras_object=downstream_model) ``` ### Return Value Pattern `init_ras_project()` returns the initialized RAS object: Python ```python project = init_ras_project(r"C:\Projects\MyProject", "6.5", ras_object=RasPrj()) print(project.plan_df) ``` ## Refreshing Project Data If project files change externally (e.g., after running HEC-RAS manually), re-initialize to refresh: Python ```python # After external changes init_ras_project(ras.project_folder, "6.5") # Or for a named instance init_ras_project(project.project_folder, "6.5", ras_object=project) ``` ## Best Practices 1. **Single Project Scripts**: Use the global `ras` object for simplicity 1. **Multiple Projects**: Always use named `RasPrj` instances and pass `ras_object` to all functions 1. **Consistency**: Don't mix global `ras` and named instances in the same logical section 1. **Version Specification**: Explicitly specify version rather than relying on auto-detection 1. **Path Handling**: Use raw strings (`r"..."`) or forward slashes for Windows paths ## Common Issues ### Project File Not Found Python ```python # Ensure the path contains a .prj file from pathlib import Path project_path = Path(r"C:\Projects\MyProject") prj_files = list(project_path.glob("*.prj")) if not prj_files: print("No .prj file found in directory") else: init_ras_project(project_path, "6.5") ``` ### Missing HEC-RAS Installation Python ```python from pathlib import Path ras_exe = Path(r"C:\Program Files\HEC\HEC-RAS\6.5\Ras.exe") if not ras_exe.exists(): print(f"HEC-RAS not found at {ras_exe}") # Try alternative location ras_exe = Path(r"D:\Programs\HEC\HEC-RAS\6.5\Ras.exe") init_ras_project(r"C:\Projects\MyProject", str(ras_exe)) ``` # Reference # DataFrame Reference This page covers the **current project-level DataFrames** populated by `init_ras_project()`. These tables are the stable entry point for most automation, validation, and notebook workflows. Tip Use the DataFrames first for project metadata and file-path discovery. Use dedicated HDF readers for heavy result extraction, and use xarray-based result APIs when the data is naturally time-series or grid-shaped. ## What Gets Created at Initialization Python ```python from ras_commander import init_ras_project, ras init_ras_project(r"C:\Projects\MyModel", "6.6") print(ras.plan_df.columns.tolist()) print(ras.boundaries_df.columns.tolist()) print(ras.rasmap_df.columns.tolist()) ``` | DataFrame | Primary source | Typical use | | ------------------- | ------------------------------------------ | ---------------------------------------------------------------- | | `ras.plan_df` | `.p##` plan files plus `.prj` references | execution targeting, geometry/flow linkage, HDF result discovery | | `ras.geom_df` | `.g##` files and compiled `.g##.hdf` paths | geometry inventory, HDF presence, geometry titles/descriptions | | `ras.flow_df` | `.f##` files | steady-flow inventory and descriptions | | `ras.unsteady_df` | `.u##` files | unsteady-flow inventory and descriptions | | `ras.boundaries_df` | parsed unsteady boundary blocks | DSS audits, hydrograph inspection, boundary summaries | | `ras.rasmap_df` | `.rasmap` project summary | compact terrain / land-cover / projection path summary | | `ras.results_df` | lightweight HDF summaries | completion, runtime, error/warning, and result-file status | Related live notebooks: - [101_project_initialization.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/101_project_initialization.ipynb) - [104_plan_parameter_operations.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/104_plan_parameter_operations.ipynb) - [111_executing_plan_sets.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/111_executing_plan_sets.ipynb) - [122_rasmapper_spatial_review.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/122_rasmapper_spatial_review.ipynb) - [150_results_dataframe.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/150_results_dataframe.ipynb) - [212_landcover_mannings_n_write.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/212_landcover_mannings_n_write.ipynb) - [611_validating_map_layers.ipynb](https://github.com/gpt-cmdr/ras-commander/blob/main/examples/611_validating_map_layers.ipynb) ## Plan-Number Normalization ras-commander normalizes RAS file numbers to a two-digit form before path construction. All of these inputs resolve to the same plan number: Python ```python from pathlib import Path from ras_commander import RasUtils RasUtils.normalize_ras_number(1) # "01" RasUtils.normalize_ras_number("01") # "01" RasUtils.normalize_ras_number("p01") # "01" RasUtils.normalize_ras_number(Path("Model.p01")) # "01" ``` This matters when you pass prefixed plan numbers into `RasCmdr`, `RasProcess`, HDF readers, or any helper that resolves `.p##` / `.g##` files from project metadata. ## `plan_df` `plan_df` is the main execution and metadata table. It is assembled from the `.prj` file, parsed `.p##` contents, and project-relative path resolution. Key columns you can rely on (column names preserve the HEC-RAS plan-file keys verbatim, including spaces and the HEC-RAS spelling of `Reoccurance`). Values parsed from the plan text are strings unless noted: | Column | Dtype | Meaning | | -------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------ | | `plan_number` | str | normalized two-digit plan id such as `01` | | `geometry_number` | str | normalized geometry id used by the plan | | `unsteady_number` | str / None | normalized unsteady-flow id when the plan is unsteady; `None` for steady plans | | `Plan Title` | str | HEC-RAS plan title (`Plan Title=`) | | `Short Identifier` | str | HEC-RAS short id used by stored-map/result folders | | `Geom File` | str | geometry reference number from the plan (`Geom File=`) | | `Geom Path` | str | resolved absolute `.g##` path | | `Flow File` | str | steady/unsteady flow reference number (`Flow File=`) | | `Flow Path` | str | resolved absolute `.f##` / `.u##` path | | `Computation Interval` | str | computation time step (`Computation Interval=`) | | `Mapping Interval` | str | RAS Mapper output interval (`Mapping Interval=`) | | `Simulation Date` | str | simulation date/time window (`Simulation Date=`) | | `Run HTab` / `Run UNet` / `Run PostProcess` / `Run Sediment` / `Run WQNet` | str | run-flag toggles parsed from the plan | | `UNET D1 Cores` / `UNET D2 Cores` / `PS Cores` | int / None | core counts; cast to `int` when present, else `None` | | `Write IC File` / `IC Time` | str | restart / hot-start output-save settings when present | | `Write IC File at Fixed DateTime` | str | restart-at-fixed-datetime flag when present | | `Write IC File Reoccurance` | str | restart output recurrence interval (HEC-RAS spelling preserved) | | `Write IC File at Sim End` | str | final-step restart output flag | | `Program Version` | str | HEC-RAS version recorded in the plan | | `description` | str / None | plan `BEGIN DESCRIPTION` block when present | | `HDF_Results_Path` | str / None | resolved `.p##.hdf` path; `None` when results do not exist yet | | `full_path` | str | resolved absolute `.p##` path | | `flow_type` | str | `"Unsteady"`, `"Steady"`, or `"Unknown"` (derived from `unsteady_number`) | Note Columns derive directly from `_parse_plan_file()` in `ras_commander/RasPrj.py`, so any plan key present in the file appears as a same-named column. Not every plan contains every key; missing keys are simply absent or `None`. Common patterns: Python ```python # Plans that already have local HDF results plans_with_results = ras.plan_df[ras.plan_df["HDF_Results_Path"].notna()] # Plans using a specific geometry g04_plans = ras.plan_df[ras.plan_df["geometry_number"] == "04"] # Quick lookup through RasPrj helpers info = ras.get_plan_info("01") paths = ras.get_hdf_paths("01") ``` ## `geom_df` `geom_df` inventories geometry files and compiled HDF companions. Columns (the metadata columns come from `GeomMetadata`, which prefers fast HDF-based extraction when `.g##.hdf` exists and falls back to plain-text parsing): | Column | Dtype | Meaning | | ------------------------ | ---------- | -------------------------------------------------------- | | `geom_file` | str | raw `.g##` reference token from the project (e.g. `g01`) | | `geom_number` | str | normalized geometry id (e.g. `01`) | | `full_path` | str | resolved absolute `.g##` path | | `hdf_path` | str | expected compiled `.g##.hdf` path | | `geom_title` | str / None | parsed `Geom Title=` value when present | | `description` | str / None | geometry `BEGIN DESCRIPTION` block when present | | `has_1d_xs` | bool | `True` if the geometry has 1D cross sections | | `has_2d_mesh` | bool | `True` if the geometry has 2D mesh / flow areas | | `num_cross_sections` | int | count of 1D cross sections | | `num_inline_structures` | int | total inline structures (bridges + culverts + weirs) | | `num_bridges` | int | count of bridge structures | | `num_culverts` | int | count of culvert structures | | `num_weirs` | int | count of inline weir structures | | `num_gates` | int | count of gate structures | | `num_lateral_structures` | int | count of lateral structures | | `num_sa_2d_connections` | int | count of SA/2D connections | | `mesh_cell_count` | int | total 2D mesh cells across all areas | | `mesh_area_names` | list[str] | names of 2D flow areas | When metadata extraction fails for a geometry, counts default to `0`, booleans to `False`, and `mesh_area_names` to an empty list. Use this table for geometry discovery first, then move to `RasGeometry`, `Geom*`, or HDF readers for detail. ## `flow_df` and `unsteady_df` These inventory steady and unsteady flow files referenced by the project. Both come from `_parse_flow_file()` / `_parse_unsteady_file()` in `RasPrj.py`. `flow_df` (steady `.f##`): | Column | Dtype | Meaning | | ----------------- | ---------- | --------------------------------------------------------- | | `flow_number` | str | normalized steady-flow id | | `unsteady_number` | None | always `None` for steady-flow rows | | `full_path` | str | resolved absolute `.f##` path | | `Flow Title` | str / None | parsed `Flow Title=` value | | `Program Version` | str / None | HEC-RAS version recorded in the flow file | | `description` | str | flow `BEGIN DESCRIPTION` block (empty string when absent) | `unsteady_df` (unsteady `.u##`): | Column | Dtype | Meaning | | ---------------------------------- | ---------- | ------------------------------------------------------------- | | `unsteady_number` | str | normalized unsteady-flow id | | `full_path` | str | resolved absolute `.u##` path | | `Flow Title` | str / None | parsed `Flow Title=` value | | `Program Version` | str / None | HEC-RAS version recorded in the file | | `Use Restart` | str / None | restart/hot-start flag (`Use Restart=`) | | `Restart Filename` | str / None | restart source file when restart is enabled | | `Precipitation Mode` / `Wind Mode` | str / None | meteorology mode toggles when present | | `Met BC=Precipitation\|...` | str / None | parsed gridded-precip / met-BC settings when present | | `description` | str | unsteady `BEGIN DESCRIPTION` block (empty string when absent) | Use these tables when you need to audit which `.f##` / `.u##` files exist before editing them through `RasPlan` or `RasUnsteady`. ## `boundaries_df` `boundaries_df` is built from the unsteady boundary blocks and is the main table for DSS-path audits and boundary-condition summaries. Common columns (from `_parse_boundary_condition()` in `RasPrj.py`; DSS/typed columns appear only when the source line is present): | Column | Dtype | Meaning | | --------------------------- | ------------ | ---------------------------------------------------------------------------- | | `unsteady_number` | str | parent `.u##` file | | `boundary_condition_number` | int | boundary sequence within the file (1-based) | | `bc_type` | str | high-level boundary type (e.g. `Flow Hydrograph`, `Normal Depth`, `Unknown`) | | `hydrograph_type` | str / None | hydrograph subtype when the BC is a hydrograph, else `None` | | `river_reach_name` | str | 1D river/reach location field (may be empty string) | | `river_station` | str | 1D river-station field (may be empty string) | | `storage_area_name` | str | storage-area location field when present | | `pump_station_name` | str | pump-station field when present | | `area_2d` | str | 2D flow-area name field when present | | `bc_line_name` | str | boundary-condition line name when present | | `Interval` | str | time interval (`Interval=`) | | `Use DSS` | str | `"True"` / `"False"` string (note: string, not bool) | | `DSS File` | str | raw DSS file reference | | `DSS Path` | str | raw DSS pathname | | `dss_part_a` … `dss_part_f` | str | parsed DSS pathname components A–F | | `Friction Slope` | str | raw friction-slope field for `Normal Depth` BCs | | `friction_slope_value` | float / None | parsed friction-slope value | | `critical_fallback_flag` | int / None | parsed critical-boundary fallback flag | | `hydrograph_num_values` | int | number of inline hydrograph values (0 if none) | | `hydrograph_values` | list[str] | inline hydrograph values when present | Examples: Python ```python # DSS-backed boundaries only dss_boundaries = ras.boundaries_df[ras.boundaries_df["Use DSS"] == "True"] # Flow hydrographs only flow_bcs = ras.boundaries_df[ras.boundaries_df["bc_type"] == "Flow Hydrograph"] ``` ## `rasmap_df` `rasmap_df` is a **single-row compact summary** of the project `.rasmap`. Several columns contain lists because the dataframe is optimized for project overview, not one-row-per-layer discovery. Current default columns (each cell is one element because the DataFrame is a single row; "Dtype" describes the value inside that cell): | Column | Dtype | Meaning | | --------------------------- | ---------- | ------------------------------------- | | `projection_path` | str / None | project projection (`.prj`) reference | | `profile_lines_path` | list[str] | profile/reference line paths | | `soil_layer_path` | list[str] | soils sidecar paths | | `infiltration_hdf_path` | list[str] | infiltration sidecar HDFs | | `landcover_hdf_path` | list[str] | land-cover sidecar HDFs | | `terrain_hdf_path` | list[str] | terrain HDFs | | `reference_map_layer_names` | list[str] | reference map layer names | | `reference_map_layer_path` | list[str] | reference map layer paths | | `basemap_layer_names` | list[str] | basemap layer names | | `basemap_layer_path` | list[str] | basemap layer paths | | `current_settings` | dict | compact `.rasmap` settings summary | Python ```python summary = ras.rasmap_df.iloc[0] print(summary["terrain_hdf_path"]) print(summary["landcover_hdf_path"]) ``` Use `rasmap_df` for quick project-level path inspection. When you need discoverable layer names and per-layer metadata, prefer: - `RasMap.list_terrain_layers()` - `RasMap.list_landcover_layers()` - `RasMap.list_soils_layers()` - `RasMap.list_infiltration_layers()` - `RasMap.list_map_layers()` - `RasMap.list_geometry_layers()` - `RasMap.list_result_layers()` For compiled HDF asset resolution, pair the `.rasmap` summary with `RasMap.get_hdf_geometry_association()` on geometry or plan/result HDFs. ## `results_df` `results_df` is a lightweight summary table generated from plan HDFs through `ResultsSummary`. It is intended for fast execution-status and runtime queries, not heavy spatial extraction. Columns (from `ResultsSummary.summarize_plan()` / `get_summary_columns()` in `ras_commander/results/ResultsSummary.py`; identity, health, and runtime columns are always present, `None`/`0` when unavailable): | Column | Dtype | Meaning | | -------------------------------- | --------------- | ------------------------------------------------ | | `plan_number` | str | copied plan id | | `plan_title` | str / None | copied plan title | | `flow_type` | str / None | `Steady` / `Unsteady` classification | | `hdf_path` | str | path to the `.p##.hdf` result file | | `hdf_exists` | bool | whether the HDF result file exists | | `hdf_file_modified` | datetime / None | HDF modification timestamp | | `ras_version` | str / None | HEC-RAS version (`Program Version`) | | `completed` | bool | completion flag parsed from compute metadata | | `has_errors` | bool | summary error flag | | `has_warnings` | bool | summary warning flag | | `error_count` | int | parsed error-message count | | `warning_count` | int | parsed warning-message count | | `first_error_line` | str / None | first blocking compute-message line when present | | `runtime_simulation_start` | datetime / None | simulation start time | | `runtime_simulation_end` | datetime / None | simulation end time | | `runtime_simulation_hours` | float / None | simulated duration (hours) | | `runtime_complete_process_hours` | float / None | end-to-end wall-clock runtime (hours) | | `runtime_unsteady_compute_hours` | float / None | unsteady compute runtime (hours) | | `runtime_complete_process_speed` | float / None | normalized throughput (sim hr / wall hr) | | `runtime_source` | str / None | `'hdf'` or `'compute_messages'` provenance | | `vol_error` | float / None | volume-accounting error (unsteady only) | | `vol_accounting_units` | str / None | volume units | | `vol_error_percent` | float / None | volume error as percent | | `vol_flux_in` / `vol_flux_out` | float / None | total inflow / outflow volume | | `vol_starting` / `vol_ending` | float / None | starting / ending storage volume | Python ```python print(ras.results_df[[ "plan_number", "completed", "has_errors", "runtime_complete_process_hours", ]]) # Refresh after new runs ras.update_results_df(["01"]) ``` ## HDF and Time-Series Data Are Not Always DataFrames Project metadata belongs in pandas DataFrames. Heavy simulation outputs often do not. Prefer the dedicated HDF readers and xarray-backed APIs for: - 1D cross-section time series - 2D mesh cell or face time series - reference lines and reference points - profile-line flow and peak-Q extraction - large raster or mesh-derived result families ### Profile-Line Flow Outputs `HdfResultsMesh.get_profile_line_flow_timeseries()` returns a profile/reference line flow time series. The API uses native HDF reference-line internal faces when present, then falls back to RAS Mapper profile-line geometry. | Column | Meaning | | ------------------ | ----------------------------------------------------------- | | `time` | Output timestamp | | `flow` | Sum of selected face flows | | `line_name` | Requested profile/reference line | | `mesh_name` | 2D flow area used for extraction | | `direction` | `absolute` or `signed` aggregation mode | | `face_count` | Count of selected mesh faces | | `selection_source` | `reference_line_internal_faces` or `profile_lines_geometry` | `HdfResultsMesh.get_profile_line_peak_flow()` returns one peak-Q row derived from the time series. | Column | Meaning | | ------------------ | ----------------------------------------------------------- | | `line_name` | Requested profile/reference line | | `mesh_name` | 2D flow area used for extraction | | `peak_time` | Timestamp of peak flow magnitude | | `peak_flow` | Peak flow value; signed mode preserves native sign | | `direction` | `absolute` or `signed` aggregation mode | | `face_count` | Count of selected mesh faces | | `selection_source` | `reference_line_internal_faces` or `profile_lines_geometry` | See: - [HDF Data Extraction](https://rascommander.info/user-guide/hdf-data-extraction/index.md) - [Spatial Data & RASMapper](https://rascommander.info/user-guide/spatial-data/index.md) - [API Reference](https://rascommander.info/api/index.md) ## Practical Workflow 1. Initialize the project and inspect `plan_df`, `boundaries_df`, and `rasmap_df`. 1. Normalize any user-supplied plan or geometry numbers before constructing paths. 1. Use `results_df` for quick execution and health checks. 1. Use `RasMap.list_*_layers()` and `get_hdf_geometry_association()` for per-layer RASMapper and compiled-HDF QA. 1. Move to dedicated HDF or geometry APIs only after the project metadata tables tell you which files and plans matter. # Quick Reference Fast lookup tables for common HEC-RAS keywords, paths, and formats. ## Geometry File Keywords ### Cross Section Keywords | Keyword | Description | Example | | ------------------------- | ----------------------------- | --------------------------------------------- | | `River Reach=` | Start of river/reach block | `River Reach=River1,Reach1` | | `Type RM Length L Ch R =` | Cross section header | `Type RM Length L Ch R = 1 ,1000,500,500,500` | | `#Sta/Elev=` | Station-elevation pairs count | `#Sta/Elev= 40` | | `#Mann=` | Manning's n segments | `#Mann= 3 , 0 , 0` | | `Bank Sta=` | Bank station locations | `Bank Sta=190,375` | | `XS GIS Cut Line=` | Cross section GIS coordinates | `XS GIS Cut Line= 5` | | `Levee=` | Levee data | `Levee= 12 , 0` | | `Ineffective=` | Ineffective flow areas | `Ineffective= 2 , 0 , 0` | | `Blocked=` | Blocked obstruction | `Blocked= 2 , 0` | ### 2D Flow Area Keywords | Keyword | Description | | -------------------- | ----------------------- | | `Storage Area=` | Storage area definition | | `2D Flow Area=` | 2D mesh area definition | | `Storage Area Conn=` | SA/2D connection | | `Connection=` | Generic connection | | `BC Line Name=` | Boundary condition line | ### Structure Keywords | Keyword | Description | | --------------------------- | ------------------ | | `Type RM Length L Ch R = 2` | Bridge | | `Type RM Length L Ch R = 3` | Inline weir | | `Culvert=` | Culvert definition | | `Lateral Weir=` | Lateral structure | | `Inline Structure=` | Inline structure | ## Plan File Keywords | Keyword | Description | Example | | ----------------------- | ------------------------- | --------------------------- | | `Plan Title=` | Plan name | `Plan Title=Base Run` | | `Short Identifier=` | Short ID | `Short Identifier=p01` | | `Geom File=` | Geometry reference | `Geom File=g01` | | `Flow File=` | Flow file reference | `Flow File=u01` | | `Run HTab=` | Run hydraulic tables flag | `Run HTab= -1` | | `Run UNet=` | Run unsteady flag | `Run UNet= -1` | | `Run PostProcess=` | Run RASMapper flag | `Run PostProcess= 0` | | `Computation Interval=` | Time step | `Computation Interval=2MIN` | | `Output Interval=` | Output frequency | `Output Interval=15MIN` | | `UNET D1 Cores=` | 1D cores | `UNET D1 Cores= 1` | | `UNET D2 Cores=` | 2D cores | `UNET D2 Cores= 4` | ## HDF Path Quick Reference ### Most Common Paths Python ```python # Plan Info "/Plan Data/Plan Information" # 2D Max Results "/Results/Unsteady/Output/Output Blocks/Base Output/Summary Output/2D Flow Areas/{area}/Maximum Water Surface" # 2D Time Series "/Results/Unsteady/Output/Output Blocks/Base Output/Unsteady Time Series/2D Flow Areas/{area}/Water Surface" # 1D Cross Section Results "/Results/Unsteady/Output/Output Blocks/Base Output/Unsteady Time Series/Cross Sections/{river}_{reach}/Water Surface" # Timestamps "/Results/Unsteady/Output/Output Blocks/Base Output/Unsteady Time Series/Time Date Stamp" # Mesh Geometry "/Geometry/2D Flow Areas/{area}/Cells Center Coordinate" # Steady Results "/Results/Steady/Output/Geometry/Cross Sections/{river}_{reach}/Water Surface" ``` ### Path Variables | Variable | Description | Example | | ----------------- | ------------------------- | -------------------- | | `{area}` | 2D flow area name | `"Floodplain"` | | `{river}` | River name | `"White River"` | | `{reach}` | Reach name | `"Upper"` | | `{river}_{reach}` | Combined (spaces removed) | `"WhiteRiver_Upper"` | ## Data Format Cheat Sheet ### Fixed-Width Columns | Data Type | Width | Per Line | Example | | -------------- | -------- | --------------------- | -------------- | | 1D Sta/Elev | 8 chars | 10 values | `0 660.41` | | 2D Coordinates | 16 chars | 4 values | `648224.43125` | | Manning's n | 8 chars | 9 values (3 triplets) | `0 .06 0` | ### Count Interpretation | Keyword | Count Means | Total Values | | ---------------------------- | ------------ | ------------ | | `#Sta/Elev=` | PAIRS | count × 2 | | `#Mann=` | SEGMENTS | count × 3 | | `Reach XY=` | PAIRS | count × 2 | | `Storage Area Surface Line=` | POINTS | count × 2 | | `Levee=` | TOTAL values | count | ### Time Formats | Source | Format | Example | | ------------- | -------------------- | ------------------------ | | HDF timestamp | `%d%b%Y %H:%M:%S:%f` | `01JAN2000 00:00:00:000` | | COM timestamp | `%d%b%Y %H%M` | `01JAN2000 0000` | | Plan file | `ddMMMYYYY,HHMM` | `01Jan2000,0000` | ## Computation Interval Strings | String | Interval | | ------- | ---------- | | `1SEC` | 1 second | | `2SEC` | 2 seconds | | `5SEC` | 5 seconds | | `10SEC` | 10 seconds | | `15SEC` | 15 seconds | | `30SEC` | 30 seconds | | `1MIN` | 1 minute | | `2MIN` | 2 minutes | | `5MIN` | 5 minutes | | `10MIN` | 10 minutes | | `15MIN` | 15 minutes | | `30MIN` | 30 minutes | | `1HOUR` | 1 hour | ## Culvert Shape Codes | Code | Shape | | ---- | ----------------- | | 1 | Circular | | 2 | Box | | 3 | Pipe Arch | | 4 | Ellipse | | 5 | Arch | | 6 | Semi-Circle | | 7 | Low Profile Arch | | 8 | High Profile Arch | | 9 | Con Span | ## Run Flag Values | Value | Meaning | | ----- | -------------- | | `-1` | True/Enabled | | `0` | False/Disabled | RASMapper Inversion `Run PostProcess=` uses **inverted logic**: `0` = True, `-1` = False ## See Also - [Geometry Parsing](https://rascommander.info/reference/geometry-parsing/index.md) - Detailed parsing reference - [HDF Structure](https://rascommander.info/reference/hdf-structure/index.md) - Complete HDF documentation - [File Formats](https://rascommander.info/reference/file-formats/index.md) - File naming conventions # HEC-RAS File Formats Reference for HEC-RAS file types and naming conventions. Related Documentation ras-commander parses these file formats into pandas DataFrames for programmatic access. See the [DataFrame Reference](https://rascommander.info/reference/dataframe-reference/index.md) for complete documentation of the DataFrame structures returned by each parsing function. ## Project Files | Extension | Description | | --------- | --------------------------------------------------- | | `.prj` | Project file - master index of all project files | | `.rasmap` | RASMapper configuration (terrain, land cover paths) | ## Plan Files | Extension | Description | | ------------------ | ------------------------------------ | | `.p##` | Plan file (e.g., `.p01`, `.p02`) | | `.p##.hdf` | Plan results HDF file | | `.p##.tmp.hdf` | Temporary results during computation | | `.computeMsgs.txt` | Computation messages log | ## Geometry Files | Extension | Description | | ---------- | -------------------------------------------- | | `.g##` | Geometry file (e.g., `.g01`, `.g02`) | | `.g##.hdf` | Preprocessed geometry HDF (hydraulic tables) | | `.c##` | Geometry preprocessor cache files | ## Flow Files | Extension | Description | | --------- | ---------------------------------- | | `.f##` | Steady flow file | | `.u##` | Unsteady flow file | | `.b##` | Boundary conditions (older format) | ## File Numbering Files use two-digit numbering: `01` through `99`. Text Only ```text MyProject.prj # Project file MyProject.p01 # Plan 01 MyProject.p01.hdf # Plan 01 results MyProject.g01 # Geometry 01 MyProject.g01.hdf # Geometry 01 preprocessed MyProject.u01 # Unsteady flow 01 ``` ## Plan File Structure Plan files (`.p##`) are ASCII text with key-value pairs: Text Only ```text Plan Title=My Simulation Plan Short Identifier=Plan01 Geom File=g01 Flow File=u01 Computation Interval=5MIN Output Interval=15MIN Run HTab=1 Run UNet=1 Run SedTran=0 ``` ## Geometry File Structure Geometry files (`.g##`) use FORTRAN-style fixed-width formatting: Text Only ```text River Reach=Big Creek,Upper Type RM Length L Ch R = 1 ,1000 ,500 ,300 ,400 XS GIS Cut Line=2 -100.0 500.0 100.0 500.0 #Sta/Elev= 5 0 105 50 100 100 98 150 100 200 105 ``` ### Cross Section Format - 8-character fixed-width fields - Station-elevation pairs - Maximum 450 points per cross section - Bank stations require interpolation ## Unsteady Flow File Structure Unsteady flow files (`.u##`) contain boundary condition definitions: Text Only ```text Flow Title=100-Year Event Program Version=6.50 Boundary Location=Big Creek,Upper,1000, , , , , Interval=15MIN Flow Hydrograph= 10 0 100 0.5 200 1 500 1.5 1000 2 2000 3 1500 4 1000 5 500 6 300 7 200 ``` ## HDF File Structure HEC-RAS 6.x+ stores results in HDF5 format: Text Only ```text / ├── Plan Data/ │ ├── Plan Information/ │ └── Plan Parameters/ ├── Geometry/ │ ├── Cross Sections/ │ ├── 2D Flow Areas/ │ └── Structures/ └── Results/ ├── Unsteady/ │ └── Output/ │ ├── Output Blocks/ │ │ └── Base Output/ │ │ ├── Unsteady Time Series/ │ │ └── Summary Output/ │ └── Geometry Info/ └── Summary/ ├── Compute Messages (text) └── Volume Accounting/ ``` ### Key HDF Paths | Path | Description | | ---------------------------------------------------------------------------------------- | -------------------- | | `/Results/Unsteady/Output/Output Blocks/Base Output/Unsteady Time Series/2D Flow Areas/` | 2D mesh time series | | `/Results/Unsteady/Output/Output Blocks/Base Output/Summary Output/2D Flow Areas/` | 2D mesh summary | | `/Geometry/2D Flow Areas/{area}/Cells Center Coordinate` | Cell center points | | `/Results/Summary/Compute Messages (text)` | Computation messages | ## RASMapper Configuration `.rasmap` files are XML format: XML ```xml depth_legend.xml .\Terrain\terrain.hdf ``` # User Guide # Architecture Overview RAS Commander provides a comprehensive Python API for HEC-RAS automation. This page describes the library's architecture and key design patterns. ## The Standard Workflow Every RAS Commander script follows this pattern: ``` flowchart LR A["1️⃣ Initialize"] --> B["2️⃣ Configure"] B --> C["3️⃣ Execute"] C --> D["4️⃣ Extract"] A1["init_ras_project()"] -.-> A B1["RasPlan, Geom*"] -.-> B C1["RasCmdr.compute_plan()"] -.-> C D1["HdfResults*"] -.-> D style A fill:#4CAF50,color:#fff style B fill:#2196F3,color:#fff style C fill:#FF9800,color:#fff style D fill:#9C27B0,color:#fff ``` Python ```python # The 4-step pattern in code: from ras_commander import init_ras_project, RasCmdr, HdfResultsMesh, ras init_ras_project("/path/to/project", "6.5") # 1. Initialize # Optionally modify: RasPlan.set_num_cores("01", 4) # 2. Configure RasCmdr.compute_plan("01") # 3. Execute max_wse = HdfResultsMesh.get_mesh_max_ws("01") # 4. Extract ``` ## The Two Worlds: Config vs Results **This is the key insight** for understanding which class to use: ``` flowchart TB subgraph BEFORE["📝 BEFORE Execution"] TXT["Plain text files
.p##, .g##, .u##"] RAS["Ras*, Geom* classes"] TXT <--> RAS end subgraph AFTER["📊 AFTER Execution"] HDF[".p##.hdf files"] HDFC["Hdf* classes"] HDF --> HDFC end BEFORE -->|"RasCmdr.compute_plan()"| AFTER style BEFORE fill:#E3F2FD,stroke:#1976D2 style AFTER fill:#FFF3E0,stroke:#F57C00 ``` | I want to... | Use | Example | | -------------------- | --------------- | --------------------------------------------- | | Modify plan settings | `RasPlan` | `RasPlan.set_num_cores("01", 4)` | | Parse geometry | `Geom*` classes | `GeomCrossSection.get_station_elevation(...)` | | Run a simulation | `RasCmdr` | `RasCmdr.compute_plan("01")` | | Extract results | `Hdf*` classes | `HdfResultsMesh.get_mesh_max_ws("01")` | ## Class Categories ### Project Management | Class | Description | | -------------------- | ------------------------------------------------------------- | | `RasPrj` | Manages HEC-RAS project state, file discovery, and DataFrames | | `init_ras_project()` | Initialize projects and set up RAS objects | | `RasExamples` | Download and manage HEC-RAS example projects | ### Plan Execution | Class | Description | | ------------ | ------------------------------------------------------------- | | `RasCmdr` | Execute plans via command line (single, sequential, parallel) | | `RasControl` | Legacy COM interface for HEC-RAS 3.x-6.x | ### File Operations | Class | Description | | ------------- | ---------------------------------------------------------- | | `RasPlan` | Plan file operations (cloning, parameters, descriptions) | | `RasGeo` | Geometry file operations (2D Manning's n land cover) | | `RasGeometry` | 1D geometry parsing (cross sections, storage, connections) | | `RasStruct` | Inline structure parsing (bridges, culverts, weirs) | | `RasBreach` | Breach parameter modification in plan files | | `RasUnsteady` | Unsteady flow file management | | `RasUtils` | General utility functions | | `RasMap` | RASMapper configuration parsing | | `RasDss` | DSS file operations for boundary conditions | | `RasFixit` | Geometry repair (blocked obstructions) | ### HDF Data Access | Class | Description | | --------------------------- | ---------------------------------------------- | | `HdfBase` | Core HDF operations (time parsing, attributes) | | `HdfPlan` | Plan-level information from HDF files | | `HdfMesh` | Mesh geometry data (cells, faces, points) | | `HdfResultsMesh` | Mesh results (WSE, velocity, depth) | | `HdfResultsPlan` | Plan results (volume accounting, runtime) | | `HdfResultsXsec` | 1D cross-section results | | `HdfStruc` | Structure geometry and SA/2D connections | | `HdfResultsBreach` | Dam breach results extraction | | `HdfHydraulicTables` | Cross section property tables (HTAB) | | `HdfPipe` | Pipe network analysis | | `HdfPump` | Pump station analysis | | `HdfFluvialPluvial` | Fluvial-pluvial boundary analysis | | `HdfBndry` | Boundary condition geometry features | | `HdfPlot`, `HdfResultsPlot` | Visualization utilities | ## Design Patterns ### Static Class Pattern Most RAS Commander classes use static methods with no instantiation required: Python ```python # Correct - static method call RasCmdr.compute_plan("01") RasPlan.set_num_cores("01", 4) # Incorrect - don't instantiate # cmd = RasCmdr() # Not needed ``` The `@log_call` decorator provides automatic logging: Python ```python from ras_commander import RasCmdr # Call is automatically logged at DEBUG level success = RasCmdr.compute_plan("01") ``` ### Global vs Named RAS Objects For single projects, use the global `ras` object: Python ```python from ras_commander import init_ras_project, ras init_ras_project("/path/to/project", "6.5") print(ras.plan_df) # Global object populated ``` For multiple projects, create named instances: Python ```python from ras_commander import RasPrj, init_ras_project project1 = RasPrj() init_ras_project("/path/project1", "6.5", ras_object=project1) project2 = RasPrj() init_ras_project("/path/project2", "6.5", ras_object=project2) # Always specify which project RasCmdr.compute_plan("01", ras_object=project1) ``` ### Input Standardization The `@standardize_input` decorator accepts multiple input types: Python ```python from ras_commander import HdfResultsMesh # All of these work: HdfResultsMesh.get_mesh_max_ws("01") # Plan number HdfResultsMesh.get_mesh_max_ws(1) # Integer HdfResultsMesh.get_mesh_max_ws(Path("project.p01.hdf")) # Path HdfResultsMesh.get_mesh_max_ws("/full/path.hdf") # String path ``` ### Plain Text vs HDF Separation The library separates plain text file operations (Ras *classes) from HDF operations (Hdf* classes): - **Ras\* classes**: Read/write plan files (.p##), geometry files (.g##), unsteady files (.u##) - **Hdf\* classes**: Read HDF results files (.p##.hdf) and preprocessed geometry (.g##.hdf) Example with dam breach: Python ```python from ras_commander import RasBreach, HdfResultsBreach # Plain text: modify breach parameters in plan file RasBreach.update_breach_block("01", "Dam1", start_time=10.0) # HDF: extract breach results after computation summary = HdfResultsBreach.get_breach_summary("01") ``` ## Data Flow Text Only ```text Project Folder │ ├── .prj file ─────────────────► init_ras_project() │ │ ├── .p## plan files ───────────────────►├── ras.plan_df ├── .g## geometry files ───────────────►├── ras.geom_df ├── .f## steady flow files ────────────►├── ras.flow_df ├── .u## unsteady flow files ──────────►├── ras.unsteady_df │ └── ras.boundaries_df │ ├── RasCmdr.compute_plan() ─────────────► HEC-RAS Execution │ └── .p##.hdf result files ──────────────► Hdf* classes │ ├── HdfResultsMesh ├── HdfResultsXsec └── HdfResultsPlan ``` ## Module Organization Text Only ```text ras_commander/ ├── __init__.py # Main exports ├── RasPrj.py # Project management ├── RasCmdr.py # Plan execution ├── RasPlan.py # Plan file operations ├── RasGeo.py # Geometry operations ├── RasUnsteady.py # Unsteady flow files ├── RasUtils.py # Utilities ├── RasExamples.py # Example project management ├── RasMap.py # RASMapper parsing ├── RasControl.py # Legacy COM interface │ ├── hdf/ # HDF submodule │ ├── HdfBase.py │ ├── HdfPlan.py │ ├── HdfMesh.py │ ├── HdfResults*.py │ └── ... │ ├── geom/ # Geometry parsing submodule │ ├── RasGeometry.py │ ├── RasStruct.py │ └── ... │ ├── dss/ # DSS file operations │ └── RasDss.py │ ├── fixit/ # Geometry repair │ ├── RasFixit.py │ ├── obstructions.py │ └── log_parser.py │ └── remote/ # Remote execution ├── LocalWorker.py ├── PsexecWorker.py ├── DockerWorker.py └── ... ``` ## Function Naming Conventions Function names follow the conventions of their underlying data source: ### RasControl (Legacy COM) Uses abbreviated names matching HECRASController: Python ```python RasControl.get_comp_msgs() # Matches .comp_msgs.txt file naming ``` ### HdfResultsPlan (Modern HDF) Uses descriptive names matching HDF structure: Python ```python HdfResultsPlan.get_compute_messages() # Matches HDF dataset naming ``` This is intentional - each reflects the conventions of its technology source. ## Error Handling The library uses Python exceptions with informative messages: Python ```python from ras_commander import init_ras_project try: init_ras_project("/nonexistent/path", "6.5") except FileNotFoundError as e: print(f"Project not found: {e}") except ValueError as e: print(f"Invalid parameter: {e}") ``` All operations are logged via the centralized `LoggingConfig`: Python ```python import logging # Increase verbosity logging.getLogger('ras_commander').setLevel(logging.DEBUG) ``` # Plan Execution RAS Commander provides three modes for executing HEC-RAS plans, each optimized for different workflows. ## Single Plan Execution Execute one plan with full parameter control using `RasCmdr.compute_plan()`. Python ```python from ras_commander import init_ras_project, RasCmdr init_ras_project("/path/to/project", "6.5") # Basic execution success = RasCmdr.compute_plan("01") ``` ### Parameters | Parameter | Type | Description | | ---------------- | -------- | ------------------------------------------- | | `plan_number` | str | Plan identifier ("01", "02", etc.) | | `dest_folder` | str/Path | Directory for computation (None = in-place) | | `ras_object` | RasPrj | Project object (default: global `ras`) | | `clear_geompre` | bool | Clear geometry preprocessor files first | | `num_cores` | int | Number of CPU cores to use | | `overwrite_dest` | bool | Overwrite destination if exists | ### Examples Python ```python # Execute with specific core count success = RasCmdr.compute_plan("01", num_cores=4) # Execute to separate folder (preserves original) success = RasCmdr.compute_plan( "01", dest_folder="/results/run1", overwrite_dest=True ) # Force geometry preprocessing success = RasCmdr.compute_plan("01", clear_geompre=True) ``` ## Sequential Execution Run multiple plans in order using `RasCmdr.compute_test_mode()`. Plans execute in a copy of the project. Python ```python results = RasCmdr.compute_test_mode( plan_number=["01", "02", "03"], dest_folder_suffix="[Test]" ) for plan, success in results.items(): print(f"Plan {plan}: {'OK' if success else 'FAILED'}") ``` ### When to Use - Plans have dependencies (e.g., plan 02 needs results from 01) - Controlled resource usage is needed - Debugging complex multi-plan workflows ### Parameters | Parameter | Type | Description | | -------------------- | ---- | ----------------------------------- | | `plan_number` | list | Plans to run in order | | `dest_folder_suffix` | str | Suffix for test folder name | | `clear_geompre` | bool | Clear preprocessor before each plan | | `num_cores` | int | Cores per plan | | `overwrite_dest` | bool | Overwrite test folder | ## Parallel Execution Run multiple independent plans simultaneously using `RasCmdr.compute_parallel()`. Creates temporary worker folders. Python ```python results = RasCmdr.compute_parallel( plan_number=["01", "02", "03"], max_workers=3, num_cores=2, dest_folder="/results/parallel_run" ) ``` ### Resource Optimization Balance workers and cores based on your system: Python ```python import psutil # Calculate optimal configuration physical_cores = psutil.cpu_count(logical=False) cores_per_worker = 2 max_workers = physical_cores // cores_per_worker # Also consider RAM (each HEC-RAS instance needs 2-4GB+) available_ram_gb = psutil.virtual_memory().available / (1024**3) ram_limited_workers = int(available_ram_gb // 4) # Use the more restrictive limit optimal_workers = min(max_workers, ram_limited_workers) results = RasCmdr.compute_parallel( plan_number=["01", "02", "03", "04"], max_workers=optimal_workers, num_cores=cores_per_worker ) ``` ### Parameters | Parameter | Type | Description | | ---------------- | -------- | ------------------------------------ | | `plan_number` | list | Plans to run concurrently | | `max_workers` | int | Maximum parallel HEC-RAS instances | | `num_cores` | int | Cores assigned to each worker | | `dest_folder` | str/Path | Final results location | | `clear_geompre` | bool | Clear preprocessor in worker folders | | `overwrite_dest` | bool | Overwrite destination folder | ### How It Works 1. Creates temporary worker folders (copies of project) 1. Assigns plans to workers 1. Executes plans in parallel 1. Consolidates results to destination folder 1. Cleans up worker folders ## Execution Mode Comparison | Feature | Single | Sequential | Parallel | | -------------- | ------------------ | --------------- | -------------------- | | Speed | Fast (1 plan) | Moderate | Fastest (many plans) | | Resource Usage | Low | Low | High | | Dependencies | N/A | Supported | Not supported | | Disk Space | Low | Medium | High (temp folders) | | Use Case | Testing, debugging | Dependent plans | Batch processing | ## Plan Modification Before Execution Modify plan parameters programmatically before running: Python ```python from ras_commander import RasPlan, RasCmdr # Clone and modify new_plan = RasPlan.clone_plan("01", new_plan_shortid="Modified Run") # Change parameters RasPlan.set_num_cores(new_plan, 4) RasPlan.set_computation_interval(new_plan, "5MIN") RasPlan.set_description(new_plan, "Run with finer timestep") # Execute modified plan success = RasCmdr.compute_plan(new_plan) ``` ## Native Flow Hydrograph Optimization HEC-RAS native automated flow optimization scales selected flow hydrographs until a stage or flow target is met at a reference point or line. RAS Commander exposes this through `RasFlowOptimization` while keeping execution inside `RasCmdr`. Python ```python from ras_commander import init_ras_project, RasFlowOptimization, RasCmdr init_ras_project("/path/to/tutorial-13-project", "6.5") # Copy a plan and enable native HEC-RAS flow optimization on the copy. opt_plan = RasFlowOptimization.copy_plan_with_optimization( "01", new_plan_shortid="Native Flow Opt", mode="stage", reference_location="Yosemite Falls Vantage Point", target_value=3963.5, tolerance=0.1, hydrographs=["BCLine: Inflow"], min_ratio=0.5, max_ratio=1.0, max_iterations=10, ) # Execute through RAS Commander, not a direct Ras.exe call. success = RasCmdr.compute_plan(opt_plan) # After compute, read native trial output from the plan HDF or compute messages. trials = RasFlowOptimization.get_trial_results(opt_plan) print(trials[["trial", "ratio", "difference", "target", "computed"]]) ``` Use `RasFlowOptimization.get_settings()` to audit an existing plan and `RasFlowOptimization.list_flow_hydrographs()` to discover flow hydrographs that can be selected for scaling. `RasFlowOptimization.compute_plan_and_get_trials()` is a convenience wrapper around `RasCmdr.compute_plan()` followed by trial extraction. See `examples/301_flow_hydrograph_optimization.ipynb` for the Tutorial 13 workflow and fallback guidance. `RasCalibrate` remains the ras-commander-native calibration/search API for broader parameter sweeps, arbitrary model edits, and custom objective metrics. Native flow optimization is narrower: it delegates HEC-RAS' built-in flow-ratio trial loop to HEC-RAS and reports the resulting trial table where available. See the [official HEC-RAS flow hydrograph optimization tutorial](https://www.hec.usace.army.mil/confluence/rasdocs/hgt/latest/tutorials/2d-unsteady-flow/flow-hydrograph-optimization) for the corresponding GUI workflow. ## Checking Results After execution, verify results were generated and the run completed without errors. This is critical for determining if the simulation succeeded. ### Quick Verification Python ```python from ras_commander import init_ras_project, ras, HdfResultsPlan # Refresh project data to see new results init_ras_project(ras.project_folder, "6.5") # Check for HDF results hdf_entries = ras.get_hdf_entries() print(f"Found {len(hdf_entries)} HDF result files") ``` ### Check Compute Messages for Errors The compute messages contain the HEC-RAS log output. Always check for errors: Python ```python from ras_commander import HdfResultsPlan # Get computation messages (accepts plan number or HDF path) msgs = HdfResultsPlan.get_compute_messages("01") if msgs: # Check for error keywords if 'ERROR' in msgs.upper() or 'FAILED' in msgs.upper(): print("ERRORS DETECTED in computation!") # Print error lines for line in msgs.split('\n'): if 'ERROR' in line.upper() or 'FAILED' in line.upper(): print(f" {line}") else: print("Computation completed without errors") else: print("No compute messages - run may not have completed") ``` ### Check Volume Accounting Volume accounting verifies mass conservation. A large imbalance indicates problems: Python ```python from ras_commander import HdfResultsPlan volume_df = HdfResultsPlan.get_volume_accounting("01") if volume_df is not None: print("Volume Accounting Data:") # Display as transposed for readability print(volume_df.T) else: print("No volume accounting - run may have failed") ``` ### Check Unsteady Results Exist Verify unsteady results were written to the HDF: Python ```python from ras_commander import HdfResultsPlan # Basic unsteady info try: info = HdfResultsPlan.get_unsteady_info("01") print("Unsteady results present") print(info.T) except KeyError: print("No unsteady results found") # Detailed unsteady summary try: summary = HdfResultsPlan.get_unsteady_summary("01") print("\nUnsteady Summary:") print(summary.T) except KeyError: print("No unsteady summary - check if run completed") ``` ### Check Runtime Performance Review computation timing and performance: Python ```python from ras_commander import HdfResultsPlan runtime = HdfResultsPlan.get_runtime_data("01") if runtime is not None: print(f"Plan: {runtime['Plan Name'].iloc[0]}") print(f"Simulation: {runtime['Simulation Time (hr)'].iloc[0]:.1f} hr") print(f"Compute Time: {runtime['Complete Process (hr)'].iloc[0]:.3f} hr") print(f"Speed: {runtime['Complete Process Speed (hr/hr)'].iloc[0]:.0f}x realtime") ``` ### Complete Verification Example Python ```python from ras_commander import init_ras_project, HdfResultsPlan init_ras_project("/path/to/project", "6.5") def check_run_success(plan_number): """Check if a plan run was successful.""" print(f"\n{'='*50}") print(f"Verifying Plan {plan_number}") print('='*50) success = True # 1. Check compute messages msgs = HdfResultsPlan.get_compute_messages(plan_number) if msgs: has_errors = any(kw in msgs.upper() for kw in ['ERROR', 'FAILED', 'UNSTABLE']) if has_errors: print("[FAIL] Errors found in compute messages") success = False else: print("[OK] No errors in compute messages") else: print("[FAIL] No compute messages found") success = False # 2. Check volume accounting volume = HdfResultsPlan.get_volume_accounting(plan_number) if volume is not None: print("[OK] Volume accounting data present") else: print("[WARN] No volume accounting data") # 3. Check unsteady results try: HdfResultsPlan.get_unsteady_summary(plan_number) print("[OK] Unsteady results present") except: print("[WARN] No unsteady summary") # 4. Runtime data runtime = HdfResultsPlan.get_runtime_data(plan_number) if runtime is not None: speed = runtime['Complete Process Speed (hr/hr)'].iloc[0] print(f"[INFO] Compute speed: {speed:.0f}x realtime") print(f"\nOverall: {'SUCCESS' if success else 'NEEDS REVIEW'}") return success # Usage check_run_success("01") ``` See [Workflows and Patterns](https://rascommander.info/user-guide/workflows-and-patterns/#verifying-run-success) for more detailed verification patterns including batch verification. ## Detailed Compute Message Logging Monitor HEC-RAS execution in real-time with stream callbacks. This provides live feedback during computation and enables automated error detection. ### Stream Callbacks The `stream_callback` parameter accepts callback objects for real-time monitoring: Python ```python from ras_commander import RasCmdr from ras_commander.callbacks import ConsoleCallback # Monitor execution with console output success = RasCmdr.compute_plan( "01", stream_callback=ConsoleCallback(verbose=True) ) ``` ### Available Callbacks Python ```python from ras_commander.callbacks import ( ConsoleCallback, # Print to console FileLoggerCallback, # Log to file ProgressBarCallback, # Show progress bar (requires tqdm) SynchronizedCallback # Thread-safe wrapper for parallel execution ) # Console callback RasCmdr.compute_plan("01", stream_callback=ConsoleCallback(verbose=True)) # File logger callback RasCmdr.compute_plan("01", stream_callback=FileLoggerCallback("run.log")) # Progress bar (requires: pip install tqdm) RasCmdr.compute_plan("01", stream_callback=ProgressBarCallback()) ``` ### Custom Callbacks Create custom callbacks for specialized monitoring: Python ```python from ras_commander.callbacks import ExecutionCallback class ErrorDetectionCallback(ExecutionCallback): """Callback that stops execution on first error.""" def on_exec_message(self, message): # Check for error keywords if any(kw in message.upper() for kw in ['ERROR', 'FAILED', 'UNSTABLE']): print(f"❌ ERROR DETECTED: {message}") # Could raise exception, send alert, etc. elif 'warning' in message.lower(): print(f"⚠ WARNING: {message}") else: # Normal message print(f"ℹ {message}") # Use custom callback RasCmdr.compute_plan("01", stream_callback=ErrorDetectionCallback()) ``` ### Callback Methods Custom callbacks can implement these methods: Python ```python class MyCallback(ExecutionCallback): def on_start(self, plan_number): """Called when execution starts.""" print(f"Starting plan {plan_number}") def on_exec_message(self, message): """Called for each HEC-RAS message during execution.""" print(f"HEC-RAS: {message}") def on_complete(self, success): """Called when execution completes.""" if success: print("✓ Execution completed successfully") else: print("✗ Execution failed") def on_error(self, error): """Called if exception occurs.""" print(f"Exception: {error}") ``` ### Parallel Execution with Callbacks Use `SynchronizedCallback` wrapper for thread-safe logging: Python ```python from ras_commander.callbacks import ConsoleCallback, SynchronizedCallback # Wrap callback for thread safety safe_callback = SynchronizedCallback(ConsoleCallback(verbose=True)) # Use with parallel execution results = RasCmdr.compute_parallel( plan_number=["01", "02", "03"], max_workers=3, stream_callback=safe_callback # Thread-safe logging ) ``` ### Post-Execution Message Review Review compute messages after execution completes: Python ```python from ras_commander import HdfResultsPlan # Get all compute messages msgs = HdfResultsPlan.get_compute_messages("01") # Parse for specific information for line in msgs.split('\n'): if 'time step' in line.lower(): print(line) # Timestep information elif 'iterations' in line.lower(): print(line) # Iteration counts elif 'error' in line.lower(): print(f"⚠ {line}") # Errors ``` ## Error Handling Python ```python from ras_commander import RasCmdr import logging # Enable debug logging logging.getLogger('ras_commander').setLevel(logging.DEBUG) try: success = RasCmdr.compute_plan("01") if not success: print("Plan execution failed - check HEC-RAS logs") except FileNotFoundError as e: print(f"Plan file not found: {e}") except ValueError as e: print(f"Invalid parameter: {e}") ``` ## Best Practices 1. **Test first**: Use `compute_plan()` with `dest_folder` to test without modifying original 1. **Monitor resources**: Watch CPU and RAM during parallel execution 1. **Clear preprocessor**: Use `clear_geompre=True` after geometry changes 1. **Check return values**: Always verify execution success 1. **Use logging**: Enable DEBUG level for troubleshooting # Workflows and Patterns Common workflow patterns for automating HEC-RAS with ras-commander. ## Clone-Modify-Execute Pattern The most common workflow: clone a plan, modify parameters, execute, analyze results. Python ```python from ras_commander import init_ras_project, RasCmdr, RasPlan, HdfResultsMesh from pathlib import Path # 1. Initialize project init_ras_project("/path/to/project", "6.5") # 2. Clone the base plan new_plan = RasPlan.clone_plan("01", "sensitivity_01") # 3. Modify parameters RasPlan.set_num_cores(new_plan, 8) RasPlan.set_computation_interval(new_plan, "1MIN") # 4. Execute success = RasCmdr.compute_plan(new_plan) # 5. Analyze results if success: max_wse = HdfResultsMesh.get_mesh_max_ws(new_plan) print(f"Max WSE: {max_wse['max_ws'].max():.2f} ft") ``` ## Batch Parameter Sensitivity Run multiple scenarios with different parameters: Python ```python from ras_commander import init_ras_project, RasCmdr, RasPlan, RasGeo init_ras_project("/path/to/project", "6.5") # Define sensitivity parameters manning_n_values = [0.03, 0.04, 0.05, 0.06] results = {} for n in manning_n_values: # Clone plan for this scenario plan_name = f"mann_n_{n:.2f}" new_plan = RasPlan.clone_plan("01", plan_name) # Modify Manning's n (using geometry operations) # ... geometry modification code ... # Execute success = RasCmdr.compute_plan(new_plan, num_cores=4) # Store result results[n] = { 'plan': new_plan, 'success': success } # Summary for n, result in results.items(): print(f"n={n:.2f}: {'Success' if result['success'] else 'Failed'}") ``` ## Parallel Batch Processing Execute many plans efficiently using parallel workers: Python ```python from ras_commander import init_ras_project, RasCmdr, RasPlan init_ras_project("/path/to/project", "6.5") # Create multiple plan variations plans = [] for i in range(1, 11): new_plan = RasPlan.clone_plan("01", f"scenario_{i:02d}") # Modify parameters for each scenario... plans.append(new_plan) # Execute in parallel (local machine) results = RasCmdr.compute_parallel( plan_number=plans, num_workers=4, # 4 parallel workers num_cores=4 # 4 cores per worker ) # Check results successful = sum(1 for v in results.values() if v) print(f"Completed: {successful}/{len(plans)} plans") ``` ## Boundary Condition Modification Modify unsteady flow boundary conditions: Python ```python from ras_commander import init_ras_project, RasUnsteady, RasCmdr import pandas as pd init_ras_project("/path/to/project", "6.5") # 1. Extract current boundary condition table tables = RasUnsteady.extract_tables("u01") flow_hydrograph = tables['Flow Hydrograph='] print(f"Original flow table: {len(flow_hydrograph)} values") # 2. Modify the hydrograph (e.g., scale by 1.2x) modified_flow = flow_hydrograph * 1.2 # 3. Write back to file RasUnsteady.write_table_to_file( "u01", "Flow Hydrograph=", modified_flow ) # 4. Execute with modified boundary success = RasCmdr.compute_plan("01") ``` ## Results Extraction Pipeline Extract and analyze results systematically: Python ```python from ras_commander import ( init_ras_project, HdfResultsMesh, HdfResultsPlan, HdfXsec ) import pandas as pd init_ras_project("/path/to/project", "6.5") plans = ["01", "02", "03"] all_results = [] for plan in plans: # Get plan metadata runtime = HdfResultsPlan.get_runtime_data(plan) # Get 2D mesh results max_wse = HdfResultsMesh.get_mesh_max_ws(plan) max_depth = HdfResultsMesh.get_mesh_max_depth(plan) # Aggregate statistics stats = { 'plan': plan, 'runtime_seconds': runtime.get('Compute Time (s)', 0), 'max_wse': max_wse['max_ws'].max(), 'max_depth': max_depth['max_depth'].max(), 'avg_depth': max_depth['max_depth'].mean() } all_results.append(stats) # Create summary DataFrame summary_df = pd.DataFrame(all_results) print(summary_df) summary_df.to_csv("results_summary.csv", index=False) ``` ## Working with Multiple Projects Handle multiple HEC-RAS projects simultaneously: Python ```python from ras_commander import init_ras_project, RasCmdr, RasPrj # Create separate project objects project1 = RasPrj() project2 = RasPrj() # Initialize each init_ras_project("/path/to/project1", "6.5", ras_object=project1) init_ras_project("/path/to/project2", "6.5", ras_object=project2) # Work with project 1 print(f"Project 1 has {len(project1.plan_df)} plans") RasCmdr.compute_plan("01", ras_object=project1) # Work with project 2 print(f"Project 2 has {len(project2.plan_df)} plans") RasCmdr.compute_plan("01", ras_object=project2) ``` ## Error Handling Pattern Robust error handling for automation scripts: Python ```python from ras_commander import init_ras_project, RasCmdr, RasPlan import logging # Setup logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) def safe_execute_plan(plan_number, **kwargs): """Execute plan with error handling.""" try: success = RasCmdr.compute_plan(plan_number, **kwargs) if success: logger.info(f"Plan {plan_number}: completed successfully") return True else: logger.warning(f"Plan {plan_number}: execution returned False") return False except FileNotFoundError as e: logger.error(f"Plan {plan_number}: file not found - {e}") return False except Exception as e: logger.error(f"Plan {plan_number}: unexpected error - {e}") return False # Usage init_ras_project("/path/to/project", "6.5") plans = ["01", "02", "03"] results = {} for plan in plans: results[plan] = safe_execute_plan(plan, num_cores=4) # Report failed = [p for p, s in results.items() if not s] if failed: logger.warning(f"Failed plans: {failed}") ``` ## Geometry Preprocessing Optimization For parallel runs, preprocess geometry once: Python ```python from ras_commander import init_ras_project, RasCmdr, RasGeo init_ras_project("/path/to/project", "6.5") # Step 1: Run single plan to build geometry preprocessor files print("Building geometry preprocessor files...") RasCmdr.compute_plan("01", clear_geompre=False) # Step 2: Run parallel without clearing preprocessor print("Running parallel with cached geometry...") plans = ["01", "02", "03", "04", "05", "06"] results = RasCmdr.compute_parallel( plan_number=plans, num_workers=4, num_cores=4, clear_geompre=False # Don't clear cached geometry! ) print(f"Completed: {sum(results.values())}/{len(plans)}") ``` ## Verifying Run Success After executing HEC-RAS plans, it's critical to verify the run completed successfully and without runtime errors. The HDF file contains three key pieces of information for validation: ### Complete Verification Workflow Python ```python from ras_commander import init_ras_project, HdfResultsPlan, RasCmdr init_ras_project("/path/to/project", "6.5") def verify_run_success(plan_number: str) -> dict: """ Comprehensive verification of HEC-RAS run success. Returns dict with success status and diagnostic information. """ result = { 'plan': plan_number, 'success': False, 'has_results': False, 'volume_balance_ok': False, 'no_errors': False, 'messages': [] } # 1. Check compute messages for errors compute_msgs = HdfResultsPlan.get_compute_messages(plan_number) if compute_msgs: result['has_results'] = True # Check for error indicators in messages error_keywords = ['ERROR', 'FAILED', 'UNSTABLE', 'ABORTED'] has_errors = any(kw in compute_msgs.upper() for kw in error_keywords) result['no_errors'] = not has_errors if has_errors: # Extract error lines for reporting for line in compute_msgs.split('\n'): if any(kw in line.upper() for kw in error_keywords): result['messages'].append(line.strip()) # 2. Check volume accounting (mass balance) volume_df = HdfResultsPlan.get_volume_accounting(plan_number) if volume_df is not None: # Volume accounting exists - check for balance result['volume_accounting'] = volume_df.to_dict('records')[0] # Typical check: cumulative error should be small relative to total volume result['volume_balance_ok'] = True # Customize threshold as needed # 3. Check unsteady summary for completion status try: unsteady_summary = HdfResultsPlan.get_unsteady_summary(plan_number) result['unsteady_summary'] = unsteady_summary.to_dict('records')[0] except KeyError: result['messages'].append("No unsteady summary found - run may not have completed") # Overall success determination result['success'] = ( result['has_results'] and result['no_errors'] and result['volume_balance_ok'] ) return result # Usage result = verify_run_success("01") if result['success']: print(f"Plan {result['plan']}: Run completed successfully") else: print(f"Plan {result['plan']}: Run had issues") for msg in result['messages']: print(f" - {msg}") ``` ### Checking Compute Messages Compute messages contain the HEC-RAS computation log with warnings, errors, and performance information: Python ```python from ras_commander import HdfResultsPlan # Get computation messages msgs = HdfResultsPlan.get_compute_messages("01") # Display messages if msgs: print("="*60) print("COMPUTATION MESSAGES") print("="*60) for line in msgs.split('\n'): if line.strip(): # Highlight errors and warnings if 'ERROR' in line.upper(): print(f"[ERROR] {line}") elif 'WARNING' in line.upper(): print(f"[WARN] {line}") else: print(f" {line}") else: print("No compute messages found - run may not have completed") ``` ### Checking Volume Accounting Volume accounting verifies mass conservation in the hydraulic simulation: Python ```python from ras_commander import HdfResultsPlan # Get volume accounting volume_df = HdfResultsPlan.get_volume_accounting("01") if volume_df is not None: print("Volume Accounting:") print(volume_df.T) # Transpose for readability # Key attributes to check (available attributes vary by model): # - Boundary Conditions In/Out # - Precipitation In # - Storage Area volumes # - Cumulative error else: print("No volume accounting data - check if run completed") ``` ### Checking Unsteady Results Existence Verify that unsteady results were generated: Python ```python from ras_commander import HdfResultsPlan # Check unsteady info (basic attributes) try: unsteady_info = HdfResultsPlan.get_unsteady_info("01") print("Unsteady Results Found:") print(unsteady_info.T) except KeyError: print("No unsteady results - plan may not have run or is steady flow") # Check unsteady summary (detailed summary) try: unsteady_summary = HdfResultsPlan.get_unsteady_summary("01") print("\nUnsteady Summary:") print(unsteady_summary.T) except KeyError: print("No unsteady summary data") ``` ### Checking Runtime Performance Monitor computation performance and timing: Python ```python from ras_commander import HdfResultsPlan runtime = HdfResultsPlan.get_runtime_data("01") if runtime is not None: print("Runtime Statistics:") print(f" Plan: {runtime['Plan Name'].iloc[0]}") print(f" Simulation Duration: {runtime['Simulation Time (hr)'].iloc[0]:.2f} hours") print(f" Compute Time: {runtime['Complete Process (hr)'].iloc[0]:.4f} hours") print(f" Speed: {runtime['Complete Process Speed (hr/hr)'].iloc[0]:.1f}x realtime") # Check individual process times if runtime['Unsteady Flow Computations (hr)'].iloc[0] != 'N/A': print(f" Unsteady Compute: {runtime['Unsteady Flow Computations (hr)'].iloc[0]:.4f} hours") ``` ### Batch Verification Pattern For parallel or batch runs, verify all plans systematically: Python ```python from ras_commander import init_ras_project, HdfResultsPlan, RasCmdr import pandas as pd init_ras_project("/path/to/project", "6.5") # Run multiple plans plans = ["01", "02", "03", "04"] results = RasCmdr.compute_parallel(plans, max_workers=4) # Verify all runs verification_results = [] for plan, executed in results.items(): if not executed: verification_results.append({ 'plan': plan, 'status': 'EXECUTION_FAILED', 'errors': ['Did not execute'] }) continue # Check compute messages msgs = HdfResultsPlan.get_compute_messages(plan) has_errors = 'ERROR' in msgs.upper() if msgs else True # Check volume accounting volume_df = HdfResultsPlan.get_volume_accounting(plan) has_volume = volume_df is not None # Get runtime runtime = HdfResultsPlan.get_runtime_data(plan) compute_time = runtime['Complete Process (hr)'].iloc[0] if runtime is not None else None verification_results.append({ 'plan': plan, 'status': 'OK' if (not has_errors and has_volume) else 'ERRORS', 'has_errors': has_errors, 'has_volume_accounting': has_volume, 'compute_time_hr': compute_time }) # Summary report summary_df = pd.DataFrame(verification_results) print("\n" + "="*60) print("BATCH RUN VERIFICATION SUMMARY") print("="*60) print(summary_df.to_string(index=False)) # Identify failures failures = summary_df[summary_df['status'] != 'OK'] if not failures.empty: print(f"\n{len(failures)} plans require attention!") ``` ## Data Source Strategy Choosing the right data source for your workflow: | Task | Recommended Source | Method | | ------------------------ | ----------------------- | ------------------------------------- | | Read boundary conditions | Plain text (.u##) | `RasUnsteady.extract_tables()` | | Modify Manning's n | Plain text (.g##) | `RasGeo.set_mannings_baseoverrides()` | | Extract max WSE | HDF results (.p##.hdf) | `HdfResultsMesh.get_mesh_max_ws()` | | Read mesh geometry | HDF geometry (.g##.hdf) | `HdfMesh.get_mesh_cell_polygons()` | | Read pipe networks | HDF only | `HdfPipe.get_pipe_conduits()` | ## Related - [Plan Execution](https://rascommander.info/user-guide/plan-execution/index.md) - Detailed execution options - [HDF Data Extraction](https://rascommander.info/user-guide/hdf-data-extraction/index.md) - Working with results - [Remote Parallel](https://rascommander.info/parallel-compute/remote-parallel/index.md) - Distributed execution # HDF Data Extraction RAS Commander provides comprehensive access to HEC-RAS HDF result files through specialized classes. ## Overview HEC-RAS 6.x stores results in HDF5 format (`.p##.hdf` files). RAS Commander's `Hdf*` classes provide: - **HdfResultsMesh**: 2D mesh results (WSE, velocity, depth) - **HdfResultsXsec**: 1D cross-section results - **HdfResultsPlan**: Plan-level results (volume accounting, runtime) - **HdfMesh**: Mesh geometry (cells, faces, points) - **HdfStruc**: Structure data and connections ## Getting HDF File Paths Python ```python from ras_commander import init_ras_project, ras, RasPlan init_ras_project("/path/to/project", "6.5") # From plan_df hdf_path = ras.plan_df.loc[ ras.plan_df['plan_number'] == '01', 'hdf_path' ].iloc[0] # Or using RasPlan hdf_path = RasPlan.get_results_path("01") ``` ## 2D Mesh Results ### Maximum Values Python ```python from ras_commander import HdfResultsMesh # Maximum water surface elevation max_wse = HdfResultsMesh.get_mesh_max_ws(hdf_path) print(max_wse[['cell_id', 'max_ws', 'geometry']].head()) # Maximum velocity at cell faces max_vel = HdfResultsMesh.get_mesh_max_face_v(hdf_path) # Maximum depth max_depth = HdfResultsMesh.get_mesh_max_depth(hdf_path) # Time of maximum WSE max_wse_time = HdfResultsMesh.get_mesh_max_ws_time(hdf_path) ``` ### Time Series Python ```python from ras_commander import HdfResultsMesh, HdfMesh # Get mesh area names mesh_names = HdfMesh.get_mesh_area_names(hdf_path) first_mesh = mesh_names[0] # Water surface time series (returns xarray DataArray) wse_ts = HdfResultsMesh.get_mesh_timeseries( hdf_path, first_mesh, "Water Surface" ) print(wse_ts) # Available variables: "Water Surface", "Velocity", "Depth" ``` ### Cell and Face Data Python ```python from ras_commander import HdfResultsMesh # Cell time series for specific cells cell_ts = HdfResultsMesh.get_mesh_cells_timeseries( hdf_path, mesh_name="2D Flow Area", cell_ids=[0, 1, 2, 3], var="Water Surface" ) # Face time series (flow, velocity) face_ts = HdfResultsMesh.get_mesh_faces_timeseries( hdf_path, mesh_name="2D Flow Area", face_ids=[10, 11, 12], var="Face Velocity" ) ``` ### Profile-Line Flow and Peak Q Use the profile-line helpers when you need modeled Q across a named RAS Mapper profile/reference line. The API uses native HDF reference-line internal faces when present. If those are absent, pass a RAS Mapper Profile Lines feature file or initialize a project whose `rasmap_df` resolves `profile_lines_path`. Python ```python from ras_commander import HdfResultsMesh flow_ts = HdfResultsMesh.get_profile_line_flow_timeseries( hdf_path, line_name="Upstream", mesh_name="Perimeter 1", profile_lines_path="Features/Profile Lines.shp", direction="absolute", ) peak_q = HdfResultsMesh.get_profile_line_peak_flow( hdf_path, line_name="Upstream", mesh_name="Perimeter 1", profile_lines_path="Features/Profile Lines.shp", ) ``` `direction="absolute"` sums absolute face flows to avoid cancellation from opposing face-normal signs. `direction="signed"` preserves native HEC-RAS face signs, so the line orientation and face normals control the sign convention. ## 1D Cross-Section Results Python ```python from ras_commander import HdfResultsXsec # All cross-section results (returns xarray Dataset) xsec_results = HdfResultsXsec.get_xsec_timeseries(hdf_path) print(xsec_results) # Available variables typically include: # - Water_Surface # - Flow # - Velocity_Channel # - Velocity_Total # Extract specific cross-section xs_name = xsec_results['cross_section'][0].item() wse_xs = xsec_results['Water_Surface'].sel(cross_section=xs_name) ``` ## Plan-Level Results Plan-level results contain critical information for verifying simulation success and diagnosing issues. These methods are essential for automated workflows and quality control. ### Compute Messages (Error Checking) The compute messages contain the full HEC-RAS computation log. **This is the primary source for detecting runtime errors:** Python ```python from ras_commander import HdfResultsPlan # Get computation messages messages = HdfResultsPlan.get_compute_messages(hdf_path) if messages: # Check for errors error_keywords = ['ERROR', 'FAILED', 'UNSTABLE', 'ABORTED'] has_errors = any(kw in messages.upper() for kw in error_keywords) if has_errors: print("ERRORS DETECTED:") for line in messages.split('\n'): if any(kw in line.upper() for kw in error_keywords): print(f" {line}") else: print("Run completed without errors") # Check for warnings if 'WARNING' in messages.upper(): print("\nWarnings found - review compute messages") else: print("No compute messages - run may not have completed") ``` **Common error patterns to look for:** - `"ERROR"` - General computation errors - `"FAILED"` - Component failures - `"UNSTABLE"` - Numerical instability - `"ABORTED"` - Run terminated early ### Volume Accounting (Mass Balance) Volume accounting verifies mass conservation in the simulation. Large imbalances indicate numerical issues: Python ```python from ras_commander import HdfResultsPlan volume = HdfResultsPlan.get_volume_accounting(hdf_path) if volume is not None: print("Volume Accounting:") print(volume.T) # Transpose for readability # Volume accounting attributes may include: # - Boundary Conditions In/Out # - Precipitation In # - Infiltration Out # - Storage Area volumes # - SA/2D In/Out # - Cumulative error percentage else: print("No volume accounting - check if run completed successfully") ``` ### Unsteady Results Information Check that unsteady results were properly generated: Python ```python from ras_commander import HdfResultsPlan # Basic unsteady attributes try: info = HdfResultsPlan.get_unsteady_info(hdf_path) print("Unsteady Info:") print(info.T) except KeyError: print("No unsteady results found") # Detailed unsteady summary try: summary = HdfResultsPlan.get_unsteady_summary(hdf_path) print("\nUnsteady Summary:") print(summary.T) except KeyError: print("No unsteady summary available") ``` ### Runtime Statistics Monitor computation performance: Python ```python from ras_commander import HdfResultsPlan runtime = HdfResultsPlan.get_runtime_data(hdf_path) if runtime is not None: print("Runtime Statistics:") print(f" Plan: {runtime['Plan Name'].iloc[0]}") print(f" File: {runtime['File Name'].iloc[0]}") print(f" Simulation Start: {runtime['Simulation Start Time'].iloc[0]}") print(f" Simulation End: {runtime['Simulation End Time'].iloc[0]}") print(f" Simulation Duration: {runtime['Simulation Time (hr)'].iloc[0]:.2f} hr") print(f" Total Compute Time: {runtime['Complete Process (hr)'].iloc[0]:.4f} hr") print(f" Compute Speed: {runtime['Complete Process Speed (hr/hr)'].iloc[0]:.0f}x realtime") # Process breakdown if runtime['Unsteady Flow Computations (hr)'].iloc[0] != 'N/A': print(f" Geometry Processing: {runtime['Completing Geometry (hr)'].iloc[0]:.4f} hr") print(f" Unsteady Compute: {runtime['Unsteady Flow Computations (hr)'].iloc[0]:.4f} hr") ``` ### Complete Verification Function Combine all checks into a reusable verification function: Python ```python from ras_commander import HdfResultsPlan def verify_hdf_results(hdf_path_or_plan): """ Comprehensive verification of HDF results. Returns dict with verification status and details. """ result = { 'valid': False, 'has_compute_msgs': False, 'has_errors': False, 'has_volume_accounting': False, 'has_unsteady_results': False, 'runtime_hours': None, 'errors': [] } # 1. Check compute messages msgs = HdfResultsPlan.get_compute_messages(hdf_path_or_plan) if msgs: result['has_compute_msgs'] = True error_kw = ['ERROR', 'FAILED', 'UNSTABLE', 'ABORTED'] if any(kw in msgs.upper() for kw in error_kw): result['has_errors'] = True for line in msgs.split('\n'): if any(kw in line.upper() for kw in error_kw): result['errors'].append(line.strip()) # 2. Check volume accounting volume = HdfResultsPlan.get_volume_accounting(hdf_path_or_plan) result['has_volume_accounting'] = volume is not None # 3. Check unsteady results try: HdfResultsPlan.get_unsteady_summary(hdf_path_or_plan) result['has_unsteady_results'] = True except: pass # 4. Get runtime runtime = HdfResultsPlan.get_runtime_data(hdf_path_or_plan) if runtime is not None: result['runtime_hours'] = runtime['Complete Process (hr)'].iloc[0] # Determine overall validity result['valid'] = ( result['has_compute_msgs'] and not result['has_errors'] and result['has_volume_accounting'] ) return result # Usage status = verify_hdf_results("01") print(f"Valid: {status['valid']}") if status['errors']: print(f"Errors: {status['errors']}") ``` ## Mesh Geometry Python ```python from ras_commander import HdfMesh # Cell polygons as GeoDataFrame cells = HdfMesh.get_mesh_cell_polygons(hdf_path) print(cells[['cell_id', 'geometry']].head()) # Cell face lines faces = HdfMesh.get_mesh_cell_faces(hdf_path) # Cell center points points = HdfMesh.get_mesh_cell_points(hdf_path) # Mesh area perimeter perimeter = HdfMesh.get_mesh_perimeter(hdf_path) ``` ## Structure Data Python ```python from ras_commander import HdfStruc # SA/2D Connections connections = HdfStruc.get_connection_list(hdf_path) print(connections) # Connection profiles profile = HdfStruc.get_connection_profile(hdf_path, "Connection 1") # Gate data gates = HdfStruc.get_connection_gates(hdf_path, "Connection 1") ``` ## Pipe Networks Python ```python from ras_commander import HdfPipe # Pipe conduit geometry conduits = HdfPipe.get_pipe_conduits(hdf_path) # Pipe node locations nodes = HdfPipe.get_pipe_nodes(hdf_path) # Node depth time series node_depth = HdfPipe.get_pipe_network_timeseries( hdf_path, "Nodes/Depth" ) # Pipe network summary summary = HdfPipe.get_pipe_network_summary(hdf_path) ``` ## Pump Stations Python ```python from ras_commander import HdfPump # Pump station locations stations = HdfPump.get_pump_stations(hdf_path) # Pump group details groups = HdfPump.get_pump_groups(hdf_path) # Station time series pump_ts = HdfPump.get_pump_station_timeseries( hdf_path, "Pump Station 1" ) # Pump operation history operation = HdfPump.get_pump_operation_timeseries( hdf_path, "Pump Station 1" ) ``` ## Exploring HDF Structure Python ```python from ras_commander import HdfBase # Print HDF file structure HdfBase.get_dataset_info(hdf_path) # Explore specific group HdfBase.get_dataset_info( hdf_path, group_path="/Results/Unsteady/Output" ) ``` ## Working with xarray Results Many methods return xarray DataArrays or Datasets: Python ```python import matplotlib.pyplot as plt # Get time series wse_ts = HdfResultsMesh.get_mesh_timeseries( hdf_path, "2D Flow Area", "Water Surface" ) # Select specific cell cell_0_wse = wse_ts.sel(cell=0) # Plot cell_0_wse.plot() plt.title("Water Surface at Cell 0") plt.show() # Convert to pandas wse_df = wse_ts.to_dataframe() ``` ## Working with GeoDataFrames Geometry methods return GeoPandas GeoDataFrames: Python ```python import matplotlib.pyplot as plt # Get max WSE with geometry max_wse = HdfResultsMesh.get_mesh_max_ws(hdf_path) # Plot fig, ax = plt.subplots(figsize=(10, 8)) max_wse.plot( column='max_ws', cmap='Blues', legend=True, ax=ax ) plt.title("Maximum Water Surface Elevation") plt.show() # Export to file max_wse.to_file("max_wse.geojson", driver="GeoJSON") ``` ## Performance Tips 1. **Use specific methods**: `get_mesh_max_ws()` is faster than extracting all time series 1. **Limit cell/face selections**: Specify `cell_ids` or `face_ids` when possible 1. **Close files**: HDF files are closed automatically, but avoid keeping many open 1. **Memory**: Large models may require chunked processing ## Common Issues ### HDF File Not Found Python ```python from pathlib import Path hdf_path = RasPlan.get_results_path("01") if hdf_path is None or not Path(hdf_path).exists(): print("No HDF results - run the plan first") ``` ### Missing Data Python ```python try: max_wse = HdfResultsMesh.get_mesh_max_ws(hdf_path) except KeyError as e: print(f"Dataset not found in HDF: {e}") # Check if plan was fully computed ``` # Geometry Operations RAS Commander provides comprehensive geometry parsing and modification for HEC-RAS projects. ## Overview | Class | Purpose | | -------------------- | ---------------------------------------------------------------- | | `RasGeometry` | 1D geometry parsing (cross sections, storage areas, connections) | | `RasGeometryUtils` | Parsing utilities (fixed-width, count interpretation) | | `RasStruct` | Inline structure parsing (bridges, culverts, weirs) | | `RasGeo` | 2D Manning's n land cover operations | | `HdfHydraulicTables` | Cross section property tables (HTAB) from HDF | ## Cross Sections ### List Cross Sections Python ```python from ras_commander import RasGeometry, init_ras_project init_ras_project("/path/to/project", "6.5") # Get all cross sections xs_df = RasGeometry.get_cross_sections("01") # geometry number print(xs_df[['river', 'reach', 'station', 'description']]) ``` ### Station-Elevation Data Python ```python # Get station-elevation for a specific cross section river = "Big Creek" reach = "Upper" station = "1000" sta_elev = RasGeometry.get_station_elevation("01", river, reach, station) print(sta_elev) # DataFrame with 'station' and 'elevation' columns ``` ### Manning's n Values Python ```python # Get Manning's n for a cross section mannings = RasGeometry.get_mannings_n("01", river, reach, station) print(mannings) # Returns LOB, Channel, ROB values ``` ### Modify Cross Sections Python ```python import pandas as pd # Create modified station-elevation new_sta_elev = pd.DataFrame({ 'station': [0, 50, 100, 150, 200], 'elevation': [105, 100, 98, 100, 105] }) # Update the cross section RasGeometry.set_station_elevation( "01", river, reach, station, new_sta_elev ) ``` Critical Limits - Maximum 450 points per cross section - Bank stations are automatically interpolated if not on existing points - Always verify results after modification ## Storage Areas Python ```python # List all storage areas sa_df = RasGeometry.get_storage_areas("01") print(sa_df[['name', 'max_elevation']]) # Get elevation-volume curve sa_name = "Storage Area 1" elev_vol = RasGeometry.get_storage_elevation_volume("01", sa_name) print(elev_vol) # DataFrame with elevation, area, volume ``` ## Lateral Structures Python ```python # List lateral structures lat_df = RasGeometry.get_lateral_structures("01") print(lat_df) # Get weir profile for a lateral structure profile = RasGeometry.get_lateral_weir_profile("01", "Lateral Weir 1") print(profile) # Station and elevation ``` ## SA/2D Connections Python ```python # List connections conn_df = RasGeometry.get_connections("01") print(conn_df) # Get weir profile weir_profile = RasGeometry.get_connection_weir_profile("01", "SA-2D Conn 1") # Get gate data gates = RasGeometry.get_connection_gates("01", "SA-2D Conn 1") ``` ## Inline Structures ### Inline Weirs Python ```python from ras_commander import RasStruct # List inline weirs weirs = RasStruct.get_inline_weirs("01") print(weirs) # Get weir profile profile = RasStruct.get_inline_weir_profile("01", river, reach, station) print(profile) # Get gate data gates = RasStruct.get_inline_weir_gates("01", river, reach, station) ``` ### Bridges Python ```python # List bridges bridges = RasStruct.get_bridges("01") print(bridges) # Get bridge deck profile deck = RasStruct.get_bridge_deck("01", river, reach, station) # Get pier data piers = RasStruct.get_bridge_piers("01", river, reach, station) # Get abutment data abutment = RasStruct.get_bridge_abutment("01", river, reach, station) # Get approach sections approach = RasStruct.get_bridge_approach_sections("01", river, reach, station) # Get bridge coefficients coeffs = RasStruct.get_bridge_coefficients("01", river, reach, station) # Get HTAB settings htab = RasStruct.get_bridge_htab("01", river, reach, station) ``` ### Culverts Python ```python # List all culverts culverts = RasStruct.get_culverts("01") print(culverts) # Get detailed culvert data for all at a location all_culverts = RasStruct.get_all_culverts("01", river, reach, station) ``` **Culvert Shape Codes:** | Code | Shape | | ---- | ----------------- | | 1 | Circular | | 2 | Box | | 3 | Pipe Arch | | 4 | Ellipse | | 5 | Arch | | 6 | Semi-Circle | | 7 | Low Profile Arch | | 8 | High Profile Arch | | 9 | Con Span | ## 2D Manning's n (Land Cover) Python ```python from ras_commander import RasGeo # Get base Manning's n table base_n = RasGeo.get_base_mannings_table("01") print(base_n) # Get regional overrides regional = RasGeo.get_regional_mannings("01", "2D Flow Area") # Update Manning's n RasGeo.set_base_mannings_table("01", updated_table) ``` ## Hydraulic Tables (HTAB) Extract property tables from preprocessed geometry HDF: Python ```python from ras_commander import HdfHydraulicTables # Get geometry HDF path geom_hdf = "/path/to/project.g01.hdf" # Get cross section HTAB htab = HdfHydraulicTables.get_xs_htab(geom_hdf, river, reach, station) print(htab) # Contains: elevation, area, conveyance, wetted_perimeter, top_width ``` This enables rating curve generation without re-running HEC-RAS. ## Geometry Preprocessor Files Clear `.c##` files to force HEC-RAS to recalculate hydraulic tables: Python ```python from ras_commander import RasGeo, RasPlan # Clear for specific plan plan_path = RasPlan.get_plan_path("01") RasGeo.clear_geompre_files(plan_path) # Or clear for all plans RasGeo.clear_geompre_files() ``` ## File Format Notes HEC-RAS geometry files use FORTRAN-style fixed-width formatting: - 8-character fields (common) - Comma-separated values (some sections) - Bank stations require interpolation to match points The `RasGeometryUtils` class handles these formats internally. ## Best Practices 1. **Backup first**: Always backup geometry files before modification 1. **Clear preprocessor**: Run `clear_geompre_files()` after geometry changes 1. **Validate changes**: Re-open in HEC-RAS GUI to verify modifications 1. **Point limits**: Keep cross sections under 450 points 1. **Bank stations**: Let the library handle interpolation automatically ## Example: Modify Cross Section Elevations Python ```python from ras_commander import RasGeometry, RasGeo, RasCmdr, init_ras_project import pandas as pd init_ras_project("/path/to/project", "6.5") # Get current data river, reach, station = "Big Creek", "Upper", "1000" sta_elev = RasGeometry.get_station_elevation("01", river, reach, station) # Lower the channel by 2 feet sta_elev['elevation'] = sta_elev['elevation'] - 2.0 # Update geometry RasGeometry.set_station_elevation("01", river, reach, station, sta_elev) # Clear preprocessor and recompute RasGeo.clear_geompre_files() success = RasCmdr.compute_plan("01", dest_folder="./modified_run") ``` # Working with Boundary Conditions Boundary conditions define the upstream and downstream hydraulic conditions for HEC-RAS models. The ras-commander library provides tools to access, analyze, and modify boundary conditions through the `boundaries_df` dataframe and the `RasUnsteady` class. ## Overview Boundary conditions in HEC-RAS can include: - **Flow Hydrographs**: Time-series flow data at upstream locations - **Stage Hydrographs**: Time-series water surface elevation data - **Normal Depth**: Downstream boundary defined by slope - **Rating Curves**: Stage-discharge relationships - **Gate Operations**: Time-varying gate openings - **Lateral Inflows**: Distributed flow inputs - **Storage Area Connections**: Time-varying elevations or flows Read-Only Access The `boundaries_df` dataframe provides read-only access to boundary condition metadata. To modify boundary condition data, use the `RasUnsteady` class methods. ## Accessing Boundary Conditions After initializing a RAS project, boundary condition metadata is available through `ras.boundaries_df`: Python ```python from ras_commander import init_ras_project, ras # Initialize project init_ras_project(r"C:\HEC\projects\MyProject\MyProject.prj", "6.5") # Access boundary conditions boundary_conditions = ras.boundaries_df if boundary_conditions is not None and not boundary_conditions.empty: print(f"Found {len(boundary_conditions)} boundary conditions:") print(boundary_conditions.head()) else: print("No boundary conditions found or unsteady flow files not present") ``` Project Requirements Boundary conditions are only available for projects with unsteady flow files (.u##). Steady flow models do not have boundary conditions in the same format. ## Understanding the Boundaries DataFrame The `boundaries_df` dataframe contains comprehensive metadata about each boundary condition: ### Key Columns | Column | Description | Example Values | | --------------------------- | ---------------------------------- | ------------------------------------- | | `unsteady_number` | Links to unsteady flow file (.u##) | "01", "02" | | `boundary_condition_number` | Sequential ID for each BC | 1, 2, 3... | | `river_reach_name` | River/reach name (for river BCs) | "Muncie", "Tributary" | | `river_station` | River station (for river BCs) | "10000", "5280.5" | | `storage_area_name` | Storage area name (for SA BCs) | "Detention Basin 1" | | `pump_station_name` | Pump station name (for pump BCs) | "Pump 1" | | `bc_type` | Boundary condition type | "Flow Hydrograph", "Stage Hydrograph" | | `hydrograph_type` | Specific hydrograph format | "1", "2", "3" | | `Interval` | Time interval for data | "1HOUR", "15MIN", "1DAY" | | `hydrograph_num_values` | Number of data points | 48, 96, 168 | | `hydrograph_name` | Optional name/description | "100-Year Event" | | `DSS_path` | Path to DSS file (if applicable) | "/A/B/C/01JAN2000/1HOUR/F/" | ### Example Output Python ```python print(boundary_conditions.columns.tolist()) # Output: # ['unsteady_number', 'boundary_condition_number', 'river_reach_name', # 'river_station', 'storage_area_name', 'pump_station_name', 'bc_type', # 'hydrograph_type', 'Interval', 'hydrograph_num_values', 'hydrograph_name', # 'DSS_path', ...] ``` ## Filtering Boundary Conditions by Type Use pandas filtering to isolate specific boundary condition types: ### Flow Hydrographs Python ```python # Get all flow hydrographs flow_hydrographs = boundary_conditions[ boundary_conditions['bc_type'] == 'Flow Hydrograph' ] print(f"\nFlow Hydrographs ({len(flow_hydrographs)}):") print(flow_hydrographs[[ 'river_reach_name', 'river_station', 'hydrograph_num_values', 'Interval' ]]) ``` ### Stage Hydrographs Python ```python # Get all stage hydrographs stage_hydrographs = boundary_conditions[ boundary_conditions['bc_type'] == 'Stage Hydrograph' ] print(f"\nStage Hydrographs ({len(stage_hydrographs)}):") print(stage_hydrographs[[ 'river_reach_name', 'river_station', 'hydrograph_num_values' ]]) ``` ### Normal Depth Boundaries Python ```python # Get normal depth boundaries (downstream) normal_depth = boundary_conditions[ boundary_conditions['bc_type'] == 'Normal Depth' ] print(f"\nNormal Depth Boundaries ({len(normal_depth)}):") print(normal_depth[['river_reach_name', 'river_station']]) ``` ### DSS-Linked Boundaries Python ```python # Find boundaries linked to DSS files dss_boundaries = boundary_conditions[ boundary_conditions['DSS_path'].notna() ] print(f"\nDSS-Linked Boundaries ({len(dss_boundaries)}):") print(dss_boundaries[[ 'river_reach_name', 'river_station', 'bc_type', 'DSS_path' ]]) ``` ## Analyzing Boundary Data ### Summary Statistics Python ```python import pandas as pd # Count boundary types bc_type_counts = boundary_conditions['bc_type'].value_counts() print("\nBoundary Condition Type Summary:") print(bc_type_counts) # Analyze time intervals interval_counts = boundary_conditions['Interval'].value_counts() print("\nTime Interval Distribution:") print(interval_counts) # Summary by unsteady file print("\nBoundary Conditions by Unsteady File:") print(boundary_conditions.groupby('unsteady_number')['bc_type'].value_counts()) ``` ### Finding Specific Boundaries Python ```python # Find boundaries at a specific river station station_boundaries = boundary_conditions[ boundary_conditions['river_station'] == '10000' ] # Find boundaries for a specific river/reach river_boundaries = boundary_conditions[ boundary_conditions['river_reach_name'].str.contains('Muncie', na=False) ] # Find boundaries with most data points max_values = boundary_conditions['hydrograph_num_values'].max() largest_boundaries = boundary_conditions[ boundary_conditions['hydrograph_num_values'] == max_values ] ``` ## Modifying Boundary Conditions To modify boundary condition data values, use the `RasUnsteady` class: ### Reading Boundary Data Python ```python from ras_commander import RasUnsteady # Extract all tables from unsteady flow file tables = RasUnsteady.extract_tables("u01") # Available table types (keys vary by project) print("Available tables:") for key in tables.keys(): print(f" {key}") # Access specific boundary data if 'Flow Hydrograph=' in tables: flow_hydrograph = tables['Flow Hydrograph='] print("\nFlow Hydrograph Data:") print(flow_hydrograph.head()) ``` Table Key Format Table keys include the equals sign (e.g., `'Flow Hydrograph='`, `'Stage Hydrograph='`). This matches the format in HEC-RAS unsteady flow files. ### Modifying Boundary Data Python ```python # Example: Scale flow hydrograph by 20% if 'Flow Hydrograph=' in tables: original_flow = tables['Flow Hydrograph='] # Modify the data (keep time column, scale flow column) modified_flow = original_flow.copy() modified_flow.iloc[:, 1] = modified_flow.iloc[:, 1] * 1.2 # Write back to file RasUnsteady.write_table_to_file( "u01", "Flow Hydrograph=", modified_flow ) print("Flow hydrograph scaled by 1.2x") ``` ### Creating New Boundary Data Python ```python import pandas as pd import numpy as np # Create a new flow hydrograph time_hours = np.arange(0, 48, 1) # 48 hours base_flow = 100 # cfs peak_flow = 5000 # cfs # Simple triangular hydrograph flows = np.concatenate([ np.linspace(base_flow, peak_flow, 12), # Rising limb np.linspace(peak_flow, base_flow, 36) # Falling limb ]) new_hydrograph = pd.DataFrame({ 'Time': time_hours, 'Flow': flows }) # Write to unsteady file RasUnsteady.write_table_to_file( "u02", "Flow Hydrograph=", new_hydrograph ) ``` File Backup Always backup your unsteady flow files (.u##) before modifying them. Incorrect modifications can corrupt the file and make it unreadable by HEC-RAS. ## Visualizing Boundary Conditions ### Plot Flow Hydrograph Python ```python import matplotlib.pyplot as plt # Extract and plot flow hydrograph tables = RasUnsteady.extract_tables("u01") if 'Flow Hydrograph=' in tables: flow_data = tables['Flow Hydrograph='] plt.figure(figsize=(12, 6)) plt.plot(flow_data.iloc[:, 0], flow_data.iloc[:, 1], 'b-', linewidth=2) plt.xlabel('Time (hours)', fontsize=12) plt.ylabel('Flow (cfs)', fontsize=12) plt.title('Upstream Flow Hydrograph', fontsize=14, fontweight='bold') plt.grid(True, alpha=0.3) plt.tight_layout() plt.show() ``` ### Plot Stage Hydrograph Python ```python # Extract and plot stage hydrograph if 'Stage Hydrograph=' in tables: stage_data = tables['Stage Hydrograph='] plt.figure(figsize=(12, 6)) plt.plot(stage_data.iloc[:, 0], stage_data.iloc[:, 1], 'r-', linewidth=2) plt.xlabel('Time (hours)', fontsize=12) plt.ylabel('Stage (ft)', fontsize=12) plt.title('Downstream Stage Hydrograph', fontsize=14, fontweight='bold') plt.grid(True, alpha=0.3) plt.tight_layout() plt.show() ``` ### Compare Multiple Boundaries Python ```python # Plot multiple flow hydrographs fig, axes = plt.subplots(2, 1, figsize=(12, 10)) tables_u01 = RasUnsteady.extract_tables("u01") tables_u02 = RasUnsteady.extract_tables("u02") if 'Flow Hydrograph=' in tables_u01: flow_u01 = tables_u01['Flow Hydrograph='] axes[0].plot(flow_u01.iloc[:, 0], flow_u01.iloc[:, 1], 'b-', label='Plan 01') if 'Flow Hydrograph=' in tables_u02: flow_u02 = tables_u02['Flow Hydrograph='] axes[0].plot(flow_u02.iloc[:, 0], flow_u02.iloc[:, 1], 'r-', label='Plan 02') axes[0].set_xlabel('Time (hours)') axes[0].set_ylabel('Flow (cfs)') axes[0].set_title('Flow Hydrograph Comparison') axes[0].legend() axes[0].grid(True, alpha=0.3) # Add stage comparison if available if 'Stage Hydrograph=' in tables_u01: stage_u01 = tables_u01['Stage Hydrograph='] axes[1].plot(stage_u01.iloc[:, 0], stage_u01.iloc[:, 1], 'b-', label='Plan 01') if 'Stage Hydrograph=' in tables_u02: stage_u02 = tables_u02['Stage Hydrograph='] axes[1].plot(stage_u02.iloc[:, 0], stage_u02.iloc[:, 1], 'r-', label='Plan 02') axes[1].set_xlabel('Time (hours)') axes[1].set_ylabel('Stage (ft)') axes[1].set_title('Stage Hydrograph Comparison') axes[1].legend() axes[1].grid(True, alpha=0.3) plt.tight_layout() plt.show() ``` ## Complete Workflow Example Python ```python from ras_commander import init_ras_project, ras, RasUnsteady import matplotlib.pyplot as plt import pandas as pd # 1. Initialize project init_ras_project(r"C:\HEC\projects\FloodStudy\FloodStudy.prj", "6.5") # 2. Analyze boundary conditions boundaries = ras.boundaries_df print(f"Total boundary conditions: {len(boundaries)}") print(f"\nBoundary types:\n{boundaries['bc_type'].value_counts()}") # 3. Filter for flow hydrographs flow_bcs = boundaries[boundaries['bc_type'] == 'Flow Hydrograph'] print(f"\n{len(flow_bcs)} flow hydrographs found") # 4. Extract and modify boundary data tables = RasUnsteady.extract_tables("u01") if 'Flow Hydrograph=' in tables: original_flow = tables['Flow Hydrograph='] # Create 1.5x scaled scenario scaled_flow = original_flow.copy() scaled_flow.iloc[:, 1] = scaled_flow.iloc[:, 1] * 1.5 # Write to new unsteady file RasUnsteady.write_table_to_file("u02", "Flow Hydrograph=", scaled_flow) # 5. Visualize comparison fig, ax = plt.subplots(figsize=(12, 6)) ax.plot(original_flow.iloc[:, 0], original_flow.iloc[:, 1], 'b-', linewidth=2, label='Original') ax.plot(scaled_flow.iloc[:, 0], scaled_flow.iloc[:, 1], 'r-', linewidth=2, label='1.5x Scaled') ax.set_xlabel('Time (hours)', fontsize=12) ax.set_ylabel('Flow (cfs)', fontsize=12) ax.set_title('Boundary Condition Comparison', fontsize=14, fontweight='bold') ax.legend() ax.grid(True, alpha=0.3) plt.tight_layout() plt.show() print("\nModified boundary condition created and visualized") ``` ## Limitations and Considerations Read-Only DataFrame The `boundaries_df` dataframe is for metadata access only. Modifying values in this dataframe will not affect the actual HEC-RAS project files. Use `RasUnsteady` methods to modify boundary data. Unsteady Flow Only Boundary conditions are specific to unsteady flow models. Steady flow models use flow files (.f##) with a different structure accessed through `ras.flow_df`. DSS File Integration For boundaries linked to DSS files, use the `RasDss` class to extract and analyze time series data. See the [DSS Operations](https://rascommander.info/user-guide/dss-operations/index.md) guide for details. ### Common Issues 1. **Empty boundaries_df**: Occurs when no unsteady flow files exist or cannot be parsed 1. **Missing table keys**: Not all unsteady files contain all boundary types 1. **Time interval mismatch**: Ensure modified data matches the original time interval 1. **Column format**: Boundary data tables typically have two columns (time, value) ## Best Practices 1. Always check if `boundaries_df` exists and is not empty before accessing 1. Use descriptive variable names when filtering boundary types 1. Backup unsteady files before modifying 1. Verify modifications by re-reading the file 1. Document units and coordinate systems in comments 1. Use version control for boundary condition modifications 1. Test modified boundaries with small test runs before full simulations ## Related Documentation - [Common Workflows and Patterns](https://rascommander.info/user-guide/workflows-and-patterns/index.md) - Complete analysis workflows - [DSS Operations](https://rascommander.info/user-guide/dss-operations/index.md) - Working with DSS boundary condition files - [Plan Execution](https://rascommander.info/user-guide/plan-execution/index.md) - Executing unsteady flow plans - [Project Initialization](https://rascommander.info/getting-started/project-initialization/index.md) - Setting up RAS projects ## API Reference For detailed API documentation, see: - `RasUnsteady.extract_tables()` - Extract boundary data tables - `RasUnsteady.write_table_to_file()` - Write modified boundary data - `RasPrj.boundaries_df` - Access boundary condition metadata - `RasDss` class - DSS file operations for boundary conditions # API Reference # API Reference This section provides documentation for the RAS Commander Python API. ## Core Classes Primary classes for project management and execution: - [`RasPrj`](https://rascommander.info/api/core/#rasprj) - Project management and data structures - [`RasCmdr`](https://rascommander.info/api/core/#rascmdr) - Plan execution (single, parallel, sequential) - [`RasPlan`](https://rascommander.info/api/core/#rasplan) - Plan file operations - [`RasFlowOptimization`](https://rascommander.info/api/core/#rasflowoptimization) - Native HEC-RAS flow hydrograph optimization settings and trial results - [`RasGeo`](https://rascommander.info/api/core/#rasgeo) - Geometry file operations - [`RasUnsteady`](https://rascommander.info/api/core/#rasunsteady) - Unsteady flow file management - [`RasSteady`](https://rascommander.info/api/core/#rassteady) - Steady flow file authoring and parsing - [`RasUtils`](https://rascommander.info/api/core/#rasutils) - Utility functions - [`RasExamples`](https://rascommander.info/api/core/#rasexamples) - Example project management - [`RasMap`](https://rascommander.info/api/core/#rasmap) - RASMapper configuration, layer discovery, and geometry HDF associations - [`RasProcess`](https://rascommander.info/api/core/#rasprocess) - RasProcess.exe CLI automation, stored maps, and native reference validators - [`RasControl`](https://rascommander.info/api/core/#rascontrol) - Legacy COM interface ## HDF Modules Classes for reading HDF result files: - [`HdfBase`](https://rascommander.info/api/hdf/#hdfbase) - Core HDF operations - [`HdfPlan`](https://rascommander.info/api/hdf/#hdfplan) - Plan information - [`HdfMesh`](https://rascommander.info/api/hdf/#hdfmesh) - Mesh geometry - [`HdfResultsMesh`](https://rascommander.info/api/hdf/#hdfresultsmesh) - 2D mesh results - [`HdfResultsPlan`](https://rascommander.info/api/hdf/#hdfresultsplan) - Plan-level results - [`HdfResultsXsec`](https://rascommander.info/api/hdf/#hdfresultsxsec) - 1D cross-section results - [`HdfStruc`](https://rascommander.info/api/hdf/#hdfstruc) - Structure data - [`HdfResultsBreach`](https://rascommander.info/api/hdf/#hdfresultsbreach) - Dam breach results - [`HdfHydraulicTables`](https://rascommander.info/api/hdf/#hdfhydraulictables) - Cross section HTAB data - [`HdfStorageArea`](https://rascommander.info/api/hdf/#hdfstoragearea) - Storage area volume-elevation curves - [`HdfChannelCapacity`](https://rascommander.info/api/hdf/#hdfchannelcapacity) - 1D channel capacity analysis - [`HdfStruc1D`](https://rascommander.info/api/hdf/#hdfstruc1d) - 1D inline structure data - [`HdfPipe`](https://rascommander.info/api/hdf/#hdfpipe) - Pipe network analysis - [`HdfPump`](https://rascommander.info/api/hdf/#hdfpump) - Pump station analysis ## Geometry Modules Classes for parsing and authoring geometry files: - [`RasGeometry`](https://rascommander.info/api/geometry/#rasgeometry) - Cross sections, storage, connections - [`GeomCrossSection`](https://rascommander.info/api/geometry/#geomcrosssection) - Cross-section builder and blocked obstructions - [`GeomBridge`](https://rascommander.info/api/geometry/#geombridge) - Bridge geometry authoring - [`GeomBcLines`](https://rascommander.info/api/geometry/#geombclines) - 2D boundary condition line authoring - [`GeomLateral`](https://rascommander.info/api/geometry/#geomlateral) - Lateral structure parsing - [`GeomStorage`](https://rascommander.info/api/geometry/#geomstorage) - Storage area and 2D flow area writing - [`GeomLevee`](https://rascommander.info/api/geometry/#geomlevee) - Levee read/write - [`RasGeometryUtils`](https://rascommander.info/api/geometry/#rasgeometryutils) - Parsing utilities - [`RasStruct`](https://rascommander.info/api/geometry/#rasstruct) - Inline structures - [`RasBreach`](https://rascommander.info/api/geometry/#rasbreach) - Breach parameters ## Terrain Modules Classes for terrain creation, modification writing, and terrain-modification analysis: - [`RasTerrain`](https://rascommander.info/api/terrain/#rasterrain) - Terrain HDF creation from rasters - [`RasTerrainModWriter`](https://rascommander.info/api/terrain/#rasterrainmodwriter) / `RasTerrainModification` - Line and polygon terrain modification HDF/.rasmap writing - [`RasTerrainMod`](https://rascommander.info/api/terrain/#rasterrainmod) - Terrain profile and volume comparison with modifications applied ## Fixit Module Automated geometry repair: - [`RasFixit`](https://rascommander.info/api/fixit/#rasfixit) - Fix blocked obstruction overlaps - [`FixResults`](https://rascommander.info/api/fixit/#fixresults) - Fix operation results - [`log_parser`](https://rascommander.info/api/fixit/#log-parser) - HEC-RAS log parsing for error detection ## DSS Modules Classes for reading DSS files: - [`RasDss`](https://rascommander.info/api/dss/#rasdss) - DSS file operations ## Remote Modules Classes for distributed execution: - [`LocalWorker`](https://rascommander.info/api/remote/#localworker) - Local parallel execution - [`PsexecWorker`](https://rascommander.info/api/remote/#psexecworker) - Windows remote execution - [`DockerWorker`](https://rascommander.info/api/remote/#dockerworker) - Container execution - [`init_ras_worker`](https://rascommander.info/api/remote/#factory-function) - Factory function - [`compute_parallel_remote`](https://rascommander.info/api/remote/#execution) - Distributed execution ## Usage Pattern All primary classes use static methods: Python ```python # No instantiation needed from ras_commander import RasCmdr, RasPlan # Direct static method calls RasCmdr.compute_plan("01") RasPlan.set_num_cores("01", 4) ``` ## Decorators RAS Commander uses two key decorators that affect method behavior: ### @standardize_input Automatically converts various input types to the correct HDF file path. This decorator is applied to all HDF methods. **Accepted Input Types:** | Input Type | Example | Behavior | | ----------------- | ---------------- | ---------------------------------- | | Plan number (str) | `"01"`, `"p01"` | Looks up HDF path in `ras.plan_df` | | Plan number (int) | `1`, `2` | Converted to string, then lookup | | Path object | `Path("x.hdf")` | Used directly if file exists | | String path | `"/path/to.hdf"` | Converted to Path, used directly | | h5py.File | `hdf_file` | Extracts filename from object | **file_type Parameter:** Python ```python @standardize_input(file_type='plan_hdf') # Default - looks for .p##.hdf @standardize_input(file_type='geom_hdf') # Looks for .g##.hdf @standardize_input(file_type='plan') # Looks for .p## (plain text) ``` **Usage Examples:** Python ```python from ras_commander import HdfResultsMesh, init_ras_project init_ras_project("/path/to/project", "6.5") # All of these are equivalent: HdfResultsMesh.get_mesh_max_ws("01") # Plan number string HdfResultsMesh.get_mesh_max_ws(1) # Integer HdfResultsMesh.get_mesh_max_ws("p01") # With 'p' prefix HdfResultsMesh.get_mesh_max_ws(Path("x.hdf")) # Path object HdfResultsMesh.get_mesh_max_ws("/path/to.hdf") # String path ``` Project Initialization Required When using plan/geometry numbers (not direct paths), you must first call `init_ras_project()` to populate the `ras.plan_df` lookup table. ### @log_call Automatic logging decorator applied to most methods. Logs function entry/exit at DEBUG level. Python ```python @log_call def my_function(): ... # Logs: "Calling my_function" # Logs: "Finished my_function" ``` Enable debug logging to see these messages: Python ```python import logging logging.getLogger('ras_commander').setLevel(logging.DEBUG) ``` # Core Classes Core classes for HEC-RAS project management and execution. ## Important Notes Static Class Pattern All primary classes use static methods - do NOT instantiate: Python ```python # Correct RasCmdr.compute_plan("01") # Wrong - will fail cmd = RasCmdr() cmd.compute_plan("01") ``` RASMapper Flag Inversion When using `RasPlan.update_run_flags()`, note that RASMapper flags have **inverted logic**: - Standard flags: `True = -1`, `False = 0` - RASMapper flag: `True = 0`, `False = -1` This is a HEC-RAS quirk, not a library bug. Input Flexibility Most methods accept multiple input types via `@standardize_input`: Python ```python # All valid for HDF methods: HdfResultsMesh.get_mesh_max_ws("01") # Plan number HdfResultsMesh.get_mesh_max_ws(1) # Integer HdfResultsMesh.get_mesh_max_ws(Path("x.hdf")) # Path object ``` ## Project Management ### init_ras_project ### init_ras_project Python ```python init_ras_project(ras_project_folder, ras_version=None, ras_object=None, load_results_summary=True, hide_intro=False) -> RasPrj ``` Initialize a RAS project for use with the ras-commander library. This is the primary function for setting up a HEC-RAS project. It: 1. Finds the project file (.prj) in the specified folder OR uses the provided .prj file 1. Validates .prj files by checking for "Proj Title=" marker 1. Identifies the appropriate HEC-RAS executable 1. Loads project data (plans, geometries, flows) 1. Creates dataframes containing project components 1. Loads HDF results summaries (if load_results_summary=True) Parameters: | Name | Type | Description | Default | | ---------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- | | `ras_project_folder` | `str or Path` | Path to the RAS project folder OR direct path to a .prj file. If a .prj file is provided: - File is validated to have .prj extension - File content is checked for "Proj Title=" marker - Parent folder is used as the project folder | *required* | | `ras_version` | `str` | The version of RAS to use (e.g., "7.0") OR a full path to the Ras.exe file (e.g., "D:/Programs/HEC/HEC-RAS/6.6/Ras.exe"). If None, will attempt to detect from plan files. | `None` | | `ras_object` | `RasPrj` | If None, updates the global 'ras' object. If a RasPrj instance, updates that instance. If any other value, creates and returns a new RasPrj instance. | `None` | | `load_results_summary` | `bool, default=True` | If True, populate results_df with lightweight HDF results summaries at initialization. This enables quick queries of execution status and basic results via ras.results_df without needing to re-scan HDF files. Set to False for faster initialization when results are not needed. | `True` | | `hide_intro` | `bool, default=False` | If True, suppress the agent intro banner that is printed after initialization. The banner provides API guidance for AI agents using the library. | `False` | Returns: | Name | Type | Description | | -------- | -------- | ------------------------------- | | `RasPrj` | `RasPrj` | An initialized RasPrj instance. | Raises: | Type | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `FileNotFoundError` | If the specified project folder or .prj file doesn't exist. | | `ValueError` | If the provided file is not a .prj file, does not contain "Proj Title=", or if no HEC-RAS project file is found in the folder. | Example > > > #### Initialize using project folder (existing behavior) > > > > > > init_ras_project("/path/to/project", "7.0") print(f"Initialized project: {ras.project_name}") > > > > > > #### Initialize using direct .prj file path (new feature) > > > > > > init_ras_project("/path/to/project/MyModel.prj", "7.0") print(f"Initialized project: {ras.project_name}") > > > > > > #### Create a new RasPrj instance with .prj file > > > > > > my_project = init_ras_project("/path/to/project/MyModel.prj", "7.0", "new") print(f"Created project instance: {my_project.project_name}") > > > > > > #### Skip results loading for faster initialization > > > > > > init_ras_project("/path/to/project", "7.0", load_results_summary=False) Source code in `ras_commander/RasPrj.py` ```python @log_call def init_ras_project( ras_project_folder, ras_version=None, ras_object=None, load_results_summary=True, hide_intro=False ) -> 'RasPrj': """ Initialize a RAS project for use with the ras-commander library. This is the primary function for setting up a HEC-RAS project. It: 1. Finds the project file (.prj) in the specified folder OR uses the provided .prj file 2. Validates .prj files by checking for "Proj Title=" marker 3. Identifies the appropriate HEC-RAS executable 4. Loads project data (plans, geometries, flows) 5. Creates dataframes containing project components 6. Loads HDF results summaries (if load_results_summary=True) Args: ras_project_folder (str or Path): Path to the RAS project folder OR direct path to a .prj file. If a .prj file is provided: - File is validated to have .prj extension - File content is checked for "Proj Title=" marker - Parent folder is used as the project folder ras_version (str, optional): The version of RAS to use (e.g., "7.0") OR a full path to the Ras.exe file (e.g., "D:/Programs/HEC/HEC-RAS/6.6/Ras.exe"). If None, will attempt to detect from plan files. ras_object (RasPrj, optional): If None, updates the global 'ras' object. If a RasPrj instance, updates that instance. If any other value, creates and returns a new RasPrj instance. load_results_summary (bool, default=True): If True, populate results_df with lightweight HDF results summaries at initialization. This enables quick queries of execution status and basic results via ras.results_df without needing to re-scan HDF files. Set to False for faster initialization when results are not needed. hide_intro (bool, default=False): If True, suppress the agent intro banner that is printed after initialization. The banner provides API guidance for AI agents using the library. Returns: RasPrj: An initialized RasPrj instance. Raises: FileNotFoundError: If the specified project folder or .prj file doesn't exist. ValueError: If the provided file is not a .prj file, does not contain "Proj Title=", or if no HEC-RAS project file is found in the folder. Example: >>> # Initialize using project folder (existing behavior) >>> init_ras_project("/path/to/project", "7.0") >>> print(f"Initialized project: {ras.project_name}") >>> >>> # Initialize using direct .prj file path (new feature) >>> init_ras_project("/path/to/project/MyModel.prj", "7.0") >>> print(f"Initialized project: {ras.project_name}") >>> >>> # Create a new RasPrj instance with .prj file >>> my_project = init_ras_project("/path/to/project/MyModel.prj", "7.0", "new") >>> print(f"Created project instance: {my_project.project_name}") >>> >>> # Skip results loading for faster initialization >>> init_ras_project("/path/to/project", "7.0", load_results_summary=False) """ # Convert to Path object for consistent handling # Use safe_resolve to preserve drive letters on Windows mapped network drives from .RasUtils import RasUtils input_path = RasUtils.safe_resolve(Path(ras_project_folder)) # Detect if input is a file or folder if input_path.is_file(): # User provided a .prj file path if input_path.suffix.lower() != '.prj': error_msg = f"The provided file is not a HEC-RAS project file (.prj): {input_path}" logger.error(error_msg) raise ValueError(f"{error_msg}. Please provide either a project folder or a .prj file.") # Enhanced validation: Check if file contains "Proj Title=" to verify it's a HEC-RAS project file try: content, encoding = read_file_with_fallback_encoding(input_path) if content is None or "Proj Title=" not in content: error_msg = f"The file does not appear to be a valid HEC-RAS project file (missing 'Proj Title='): {input_path}" logger.error(error_msg) raise ValueError(f"{error_msg}. Please provide a valid HEC-RAS .prj file.") logger.debug(f"Validated .prj file contains 'Proj Title=' marker") except Exception as e: error_msg = f"Error validating .prj file: {e}" logger.error(error_msg) raise ValueError(f"{error_msg}. Please ensure the file is a valid HEC-RAS project file.") # Extract the parent folder to use as project_folder project_folder = input_path.parent specified_prj_file = input_path # Store for optimization logger.debug(f"User provided .prj file: {input_path}") logger.debug(f"Using parent folder as project_folder: {project_folder}") elif input_path.is_dir(): # User provided a folder path (existing behavior) project_folder = input_path specified_prj_file = None logger.debug(f"User provided folder path: {project_folder}") else: # Path doesn't exist if input_path.suffix.lower() == '.prj': error_msg = f"The specified .prj file does not exist: {input_path}" logger.error(error_msg) raise FileNotFoundError( f"{error_msg}. Please check the path and try again. " f"See: https://rascommander.info/getting-started/project-initialization/" ) else: error_msg = f"The specified RAS project folder does not exist: {input_path}" logger.error(error_msg) raise FileNotFoundError( f"{error_msg}. Please check the path and try again. " f"See: https://rascommander.info/getting-started/project-initialization/" ) # Determine which RasPrj instance to use if ras_object is None: # Use the global 'ras' object logger.debug("Initializing global 'ras' object via init_ras_project function.") ras_object = ras elif not isinstance(ras_object, RasPrj): # Create a new RasPrj instance logger.debug("Creating a new RasPrj instance.") ras_object = RasPrj() ras_exe_path = None # Use version specified by user if provided if ras_version is not None: ras_exe_path = get_ras_exe(ras_version) if ras_exe_path == "Ras.exe" and ras_version != "Ras.exe": logger.warning( f"HEC-RAS Version {ras_version} was not found. Running HEC-RAS will fail. " f"See: https://rascommander.info/getting-started/installation/" ) else: # No version specified, try to detect from plan files detected_version = None logger.debug("No HEC-RAS version specified. Detecting from plan files.") # Look for .pXX files in project folder logger.debug(f"Searching for plan files in {project_folder}") # Search for any file with .p01 through .p99 extension, regardless of base name plan_files = list(project_folder.glob("*.p[0-9][0-9]")) if not plan_files: logger.debug(f"No plan files found in {project_folder}") for plan_file in plan_files: logger.debug(f"Found plan file: {plan_file.name}") content, encoding = read_file_with_fallback_encoding(plan_file) if not content: logger.debug(f"Could not read content from {plan_file.name}") continue logger.debug(f"Successfully read plan file with {encoding} encoding") # Look for Program Version in plan file for line in content.splitlines(): if line.startswith("Program Version="): version = line.split("=")[1].strip() logger.debug(f"Found Program Version={version} in {plan_file.name}") # Replace 00 in version string if present if "00" in version: version = version.replace("00", "0") # Try to get RAS executable for this version test_exe_path = get_ras_exe(version) logger.debug(f"Checking RAS executable path: {test_exe_path}") if test_exe_path != "Ras.exe": detected_version = version ras_exe_path = test_exe_path logger.info(f"Detected HEC-RAS version {version} from {plan_file.name}") break else: logger.debug(f"Version {version} not found in default installation path") if detected_version: break if not detected_version: logger.error("No valid HEC-RAS version found in any plan files.") ras_exe_path = "Ras.exe" logger.warning( "No valid HEC-RAS version was detected. Running HEC-RAS will fail. " "See: https://rascommander.info/getting-started/installation/" ) # Initialize or re-initialize with the determined executable path # Pass specified_prj_file to avoid re-searching when user provided .prj file directly if specified_prj_file is not None: ras_object.initialize(project_folder, ras_exe_path, prj_file=specified_prj_file, load_results_summary=load_results_summary) else: ras_object.initialize(project_folder, ras_exe_path, load_results_summary=load_results_summary) # Store version for RasControl (legacy COM interface support) ras_object.ras_version = ras_version if ras_version else detected_version # NOTE: Removed automatic global ras update for thread-safety # When ras_object is explicitly passed, we should NOT modify the global ras object # This allows multiple threads to use separate RasPrj instances without conflicts # # Previous behavior (removed): # if ras_object is not ras: # ras.initialize(project_folder, ras_exe_path, ...) # This was NOT thread-safe # # If you need to update the global ras object, call init_ras_project without ras_object: # init_ras_project(path, version) # Updates global ras # Or explicitly pass ras_object=None (default behavior) # Log CLB Engineering branding banner with version and docs links from . import __version__ logger.info( f"ras-commander v{__version__} | " f"An open-source project of CLB Engineering Corporation (https://clbengineering.com/) | " f"Docs: https://rascommander.info | " f"GitHub: https://github.com/gpt-cmdr/ras-commander" ) logger.info(f"Project initialized: {ras_object.project_name} | Folder: {ras_object.project_folder}") logger.info(f"Using HEC-RAS executable: {ras_exe_path}") if not hide_intro: _obj = "ras" if ras_object is ras else "ras_object" logger.info( "\n" "═══════════════════════════════════════════════════════════════════════\n" "ras-commander | HEC-RAS Automation Library\n" "Docs: https://rascommander.info/\n" "Repo: https://github.com/gpt-cmdr/ras-commander\n" "LLM agents: https://rascommander.info/llms.txt\n" "═══════════════════════════════════════════════════════════════════════\n" "\n" "PROJECT DATAFRAMES (single source of truth — use these, not file globbing):\n" f" {_obj}.plan_df Plans, HDF paths, geometry/flow associations\n" f" {_obj}.geom_df Geometry files and HDF preprocessor paths\n" f" {_obj}.flow_df Steady flow files\n" f" {_obj}.unsteady_df Unsteady flow files and configurations\n" f" {_obj}.boundaries_df Boundary conditions (type, name, location)\n" f" {_obj}.results_df Lightweight HDF results summaries\n" f" {_obj}.rasmap_df RASMapper layers, terrain, land cover paths\n" "\n" "KEY APIS (static classes — call directly, never instantiate):\n" " Execution: RasCmdr.compute_plan() / compute_parallel() / compute_test_mode()\n" " Plan Files: RasPlan.clone_plan() / clone_geom() / set_geom()\n" " Unsteady: RasUnsteady — IC/BC management, gate openings, precipitation\n" " Geometry: GeomCrossSection, GeomBridge, GeomStorage, GeomLateral, GeomMesh\n" " HDF Results: HdfResultsPlan.get_wse() / get_compute_messages()\n" " HdfResultsMesh.get_mesh_max_ws() / get_mesh_cells_timeseries()\n" " HdfMesh.get_mesh_cell_points()\n" " QA/QC: RasCheck.run_check() / RasFixit (geometry repair)\n" " DSS: RasDss.get_timeseries() / check_pathname()\n" " USGS: UsgsGaugeSpatial, GaugeMatcher, RasUsgsBoundaryGeneration\n" " Precipitation: StormGenerator, Atlas14Storm, PrecipAorc, Atlas14Variance\n" " Terrain: RasTerrain.create_terrain_hdf() / RasTerrainMod\n" "\n" "MULTI-PROJECT: Pass ras_object= to all API calls when using local RasPrj instances.\n" "\n" "EXAMPLES: 100+ notebooks in examples/ (100s=execution, 200s=geometry, 300s=unsteady,\n" " 400s=HDF results, 500s=remote, 800s=QA/QC, 900s=data integration).\n" " Review relevant notebooks before assembling new workflows.\n" "\n" "PLATFORM: Most HEC-RAS operations require Windows. Linux/Wine support for\n" " headless execution, data access, geometry modification, and preprocessing\n" " is available via RasProcess (HEC-RAS 6.6+). See ras_commander/RasProcess.py.\n" " Remote distributed execution: ras_commander/remote/ (PsExec, Docker, SSH, cloud).\n" "═══════════════════════════════════════════════════════════════════════" ) return ras_object ``` ### RasPrj ### RasPrj RasPrj.py - Manages HEC-RAS projects within the ras-commander library This module provides a class for managing HEC-RAS projects. Classes: | Name | Description | | -------- | -------------------------------------- | | `RasPrj` | A class for managing HEC-RAS projects. | Functions: | Name | Description | | ------------------ | --------------------------------------------------------- | | `init_ras_project` | Initialize a RAS project. | | `get_ras_exe` | Determine the HEC-RAS executable path based on the input. | DEVELOPER NOTE: This class is used to initialize a RAS project and is used in conjunction with the RasCmdr class to manage the execution of RAS plans. By default, the RasPrj class is initialized with the global 'ras' object. However, you can create multiple RasPrj instances to manage multiple projects. Do not mix and match global 'ras' object instances and custom instances of RasPrj - it will cause errors. This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). Example @log_call def my_function(): Text Only ```text logger.debug("Additional debug information") # Function logic here ``` ______________________________________________________________________ All of the methods in this class are class methods and are designed to be used with instances of the class. List of Functions in RasPrj: - initialize() - \_load_project_data() - \_get_geom_file_for_plan() - \_parse_plan_file() - \_parse_unsteady_file() - \_parse_flow_file() - \_get_prj_entries() - \_parse_boundary_condition() - is_initialized (property) - check_initialized() - find_ras_prj() - get_project_name() - get_prj_entries() - get_plan_entries() - get_flow_entries() - get_unsteady_entries() - get_geom_entries() - get_hdf_entries() - print_data() - get_plan_value() - get_boundary_conditions() - update_results_df() - get_results_entries() Functions in RasPrj that are not part of the class: - init_ras_project() - get_ras_exe() ## Plan Execution ### RasCmdr ### RasCmdr RasCmdr - Execution operations for running HEC-RAS simulations This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). Example @log_call def my_function(): Text Only ```text logger.debug("Additional debug information") # Function logic here ``` ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasCmdr: - compute_plan() - compute_parallel() - compute_test_mode() #### Real-Time Execution Monitoring (v0.88.0+) The `stream_callback` parameter enables real-time progress monitoring during HEC-RAS execution: Python ```python from ras_commander import RasCmdr from ras_commander.callbacks import ConsoleCallback # Simple console monitoring callback = ConsoleCallback(verbose=True) RasCmdr.compute_plan("01", stream_callback=callback) ``` **Output:** Text Only ```text [Plan 01] Starting execution... [Plan 01] Geometry Preprocessor Version 6.6 [Plan 01] Computing Cross Section HTAB's [Plan 01] Starting Unsteady Flow Computations [Plan 01] Time: 01JAN2020 0600 [ 1.25% Done] [Plan 01] SUCCESS in 45.2s ``` ##### Callback Lifecycle Callbacks receive notifications at key execution points: 1. `on_prep_start(plan_number)` - Before geometry preprocessing 1. `on_prep_complete(plan_number)` - After preprocessing 1. `on_exec_start(plan_number, command)` - When HEC-RAS subprocess starts 1. `on_exec_message(plan_number, message)` - Each .bco file message (real-time) 1. `on_exec_complete(plan_number, success, duration)` - After execution 1. `on_verify_result(plan_number, verified)` - After HDF verification (if `verify=True`) ##### Built-in Callbacks ###### ConsoleCallback Simple callback that prints execution progress to console. This is the simplest possible callback implementation, suitable for: - Interactive sessions - Debugging - Quick scripts Thread Safety Uses print() with file argument for atomic writes. Safe for concurrent use in compute_parallel(). Example > > > from ras_commander import RasCmdr from ras_commander.callbacks import ConsoleCallback > > > > > > callback = ConsoleCallback() RasCmdr.compute_plan("01", stream_callback=callback) [Plan 01] Starting execution... [Plan 01] Geometry Preprocessor Version 6.6 [Plan 01] SUCCESS in 45.2s Source code in `ras_commander/callbacks.py` ```python class ConsoleCallback: """ Simple callback that prints execution progress to console. This is the simplest possible callback implementation, suitable for: - Interactive sessions - Debugging - Quick scripts Thread Safety: Uses print() with file argument for atomic writes. Safe for concurrent use in compute_parallel(). Example: >>> from ras_commander import RasCmdr >>> from ras_commander.callbacks import ConsoleCallback >>> >>> callback = ConsoleCallback() >>> RasCmdr.compute_plan("01", stream_callback=callback) [Plan 01] Starting execution... [Plan 01] Geometry Preprocessor Version 6.6 [Plan 01] SUCCESS in 45.2s """ def __init__(self, verbose: bool = False): """ Initialize console callback. Args: verbose: If True, print all messages. If False, only print start/complete. """ self.verbose = verbose def on_exec_start(self, plan_number: str, command: str) -> None: """Print execution start message.""" print(f"[Plan {plan_number}] Starting execution...", file=sys.stdout, flush=True) if self.verbose: print(f"[Plan {plan_number}] Command: {command}", file=sys.stdout, flush=True) def on_exec_message(self, plan_number: str, message: str) -> None: """Print execution messages (if verbose mode enabled).""" if self.verbose: print(f"[Plan {plan_number}] {message.strip()}", file=sys.stdout, flush=True) def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Print execution completion message.""" status = "SUCCESS" if success else "FAILED" print(f"[Plan {plan_number}] {status} in {duration:.1f}s", file=sys.stdout, flush=True) ``` ###### __init__ Python ```python __init__(verbose: bool = False) ``` Initialize console callback. Parameters: | Name | Type | Description | Default | | --------- | ------ | ----------------------------------------------------------------- | ------- | | `verbose` | `bool` | If True, print all messages. If False, only print start/complete. | `False` | Source code in `ras_commander/callbacks.py` ```python def __init__(self, verbose: bool = False): """ Initialize console callback. Args: verbose: If True, print all messages. If False, only print start/complete. """ self.verbose = verbose ``` ###### on_exec_start Python ```python on_exec_start(plan_number: str, command: str) -> None ``` Print execution start message. Source code in `ras_commander/callbacks.py` ```python def on_exec_start(self, plan_number: str, command: str) -> None: """Print execution start message.""" print(f"[Plan {plan_number}] Starting execution...", file=sys.stdout, flush=True) if self.verbose: print(f"[Plan {plan_number}] Command: {command}", file=sys.stdout, flush=True) ``` ###### on_exec_message Python ```python on_exec_message(plan_number: str, message: str) -> None ``` Print execution messages (if verbose mode enabled). Source code in `ras_commander/callbacks.py` ```python def on_exec_message(self, plan_number: str, message: str) -> None: """Print execution messages (if verbose mode enabled).""" if self.verbose: print(f"[Plan {plan_number}] {message.strip()}", file=sys.stdout, flush=True) ``` ###### on_exec_complete Python ```python on_exec_complete(plan_number: str, success: bool, duration: float) -> None ``` Print execution completion message. Source code in `ras_commander/callbacks.py` ```python def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Print execution completion message.""" status = "SUCCESS" if success else "FAILED" print(f"[Plan {plan_number}] {status} in {duration:.1f}s", file=sys.stdout, flush=True) ``` ###### FileLoggerCallback Callback that writes execution progress to per-plan log files. Creates a separate log file for each plan, enabling: - Detailed execution logs - Post-execution analysis - Archival records Thread Safety Uses threading.Lock to ensure thread-safe file operations. Safe for concurrent use in compute_parallel(). Example > > > from pathlib import Path from ras_commander.callbacks import FileLoggerCallback > > > > > > callback = FileLoggerCallback(output_dir=Path("logs")) RasCmdr.compute_plan("01", stream_callback=callback) ###### Creates logs/plan_01_execution.log with full details Source code in `ras_commander/callbacks.py` ```python class FileLoggerCallback: """ Callback that writes execution progress to per-plan log files. Creates a separate log file for each plan, enabling: - Detailed execution logs - Post-execution analysis - Archival records Thread Safety: Uses threading.Lock to ensure thread-safe file operations. Safe for concurrent use in compute_parallel(). Example: >>> from pathlib import Path >>> from ras_commander.callbacks import FileLoggerCallback >>> >>> callback = FileLoggerCallback(output_dir=Path("logs")) >>> RasCmdr.compute_plan("01", stream_callback=callback) # Creates logs/plan_01_execution.log with full details """ def __init__(self, output_dir: Path): """ Initialize file logger callback. Args: output_dir: Directory for log files (created if doesn't exist) """ self.output_dir = Path(output_dir) self.output_dir.mkdir(parents=True, exist_ok=True) self.log_handles = {} self.lock = Lock() logger.info(f"FileLoggerCallback initialized: {self.output_dir}") def on_exec_start(self, plan_number: str, command: str) -> None: """Open log file for this plan.""" with self.lock: log_file = self.output_dir / f"plan_{plan_number}_execution.log" self.log_handles[plan_number] = open(log_file, 'w', encoding='utf-8') self.log_handles[plan_number].write(f"=== Plan {plan_number} Execution Log ===\n") self.log_handles[plan_number].write(f"Command: {command}\n") self.log_handles[plan_number].write("=" * 80 + "\n\n") self.log_handles[plan_number].flush() def on_exec_message(self, plan_number: str, message: str) -> None: """Write message to plan's log file.""" with self.lock: if plan_number in self.log_handles: self.log_handles[plan_number].write(message + '\n') self.log_handles[plan_number].flush() def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Write completion message and close log file.""" with self.lock: if plan_number in self.log_handles: status = "SUCCESS" if success else "FAILED" self.log_handles[plan_number].write("\n" + "=" * 80 + "\n") self.log_handles[plan_number].write(f"Execution {status} in {duration:.1f} seconds\n") self.log_handles[plan_number].close() del self.log_handles[plan_number] def __del__(self): """Cleanup: close any remaining open file handles.""" with self.lock: for handle in self.log_handles.values(): try: handle.close() except: pass ``` ###### __init__ Python ```python __init__(output_dir: Path) ``` Initialize file logger callback. Parameters: | Name | Type | Description | Default | | ------------ | ------ | -------------------------------------------------- | ---------- | | `output_dir` | `Path` | Directory for log files (created if doesn't exist) | *required* | Source code in `ras_commander/callbacks.py` ```python def __init__(self, output_dir: Path): """ Initialize file logger callback. Args: output_dir: Directory for log files (created if doesn't exist) """ self.output_dir = Path(output_dir) self.output_dir.mkdir(parents=True, exist_ok=True) self.log_handles = {} self.lock = Lock() logger.info(f"FileLoggerCallback initialized: {self.output_dir}") ``` ###### on_exec_start Python ```python on_exec_start(plan_number: str, command: str) -> None ``` Open log file for this plan. Source code in `ras_commander/callbacks.py` ```python def on_exec_start(self, plan_number: str, command: str) -> None: """Open log file for this plan.""" with self.lock: log_file = self.output_dir / f"plan_{plan_number}_execution.log" self.log_handles[plan_number] = open(log_file, 'w', encoding='utf-8') self.log_handles[plan_number].write(f"=== Plan {plan_number} Execution Log ===\n") self.log_handles[plan_number].write(f"Command: {command}\n") self.log_handles[plan_number].write("=" * 80 + "\n\n") self.log_handles[plan_number].flush() ``` ###### on_exec_message Python ```python on_exec_message(plan_number: str, message: str) -> None ``` Write message to plan's log file. Source code in `ras_commander/callbacks.py` ```python def on_exec_message(self, plan_number: str, message: str) -> None: """Write message to plan's log file.""" with self.lock: if plan_number in self.log_handles: self.log_handles[plan_number].write(message + '\n') self.log_handles[plan_number].flush() ``` ###### on_exec_complete Python ```python on_exec_complete(plan_number: str, success: bool, duration: float) -> None ``` Write completion message and close log file. Source code in `ras_commander/callbacks.py` ```python def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Write completion message and close log file.""" with self.lock: if plan_number in self.log_handles: status = "SUCCESS" if success else "FAILED" self.log_handles[plan_number].write("\n" + "=" * 80 + "\n") self.log_handles[plan_number].write(f"Execution {status} in {duration:.1f} seconds\n") self.log_handles[plan_number].close() del self.log_handles[plan_number] ``` ###### __del__ Python ```python __del__() ``` Cleanup: close any remaining open file handles. Source code in `ras_commander/callbacks.py` ```python def __del__(self): """Cleanup: close any remaining open file handles.""" with self.lock: for handle in self.log_handles.values(): try: handle.close() except: pass ``` ###### ProgressBarCallback Callback that displays tqdm progress bars for execution. Shows real-time progress with: - Per-plan progress bar - Last message displayed - Execution time tracking Thread Safety Uses threading.Lock to ensure thread-safe tqdm operations. Safe for concurrent use in compute_parallel(). Requirements Requires tqdm package: pip install tqdm Example > > > from ras_commander.callbacks import ProgressBarCallback > > > > > > callback = ProgressBarCallback() RasCmdr.compute_plan("01", stream_callback=callback) Plan 01: 100%|████████████| 1234/1234 [00:45\<00:00, 27.42msg/s] Source code in `ras_commander/callbacks.py` ```python class ProgressBarCallback: """ Callback that displays tqdm progress bars for execution. Shows real-time progress with: - Per-plan progress bar - Last message displayed - Execution time tracking Thread Safety: Uses threading.Lock to ensure thread-safe tqdm operations. Safe for concurrent use in compute_parallel(). Requirements: Requires tqdm package: pip install tqdm Example: >>> from ras_commander.callbacks import ProgressBarCallback >>> >>> callback = ProgressBarCallback() >>> RasCmdr.compute_plan("01", stream_callback=callback) Plan 01: 100%|████████████| 1234/1234 [00:45<00:00, 27.42msg/s] """ def __init__(self): """Initialize progress bar callback.""" if not TQDM_AVAILABLE: raise ImportError( "ProgressBarCallback requires tqdm package. " "Install with: pip install tqdm" ) self.pbars = {} self.lock = Lock() def on_exec_start(self, plan_number: str, command: str) -> None: """Create progress bar for this plan.""" with self.lock: self.pbars[plan_number] = tqdm( desc=f"Plan {plan_number}", unit="msg", dynamic_ncols=True ) def on_exec_message(self, plan_number: str, message: str) -> None: """Update progress bar with new message.""" with self.lock: if plan_number in self.pbars: # Show last 50 characters of message short_message = message.strip()[-50:] self.pbars[plan_number].set_postfix_str(short_message) self.pbars[plan_number].update(1) def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Close progress bar and display final status.""" with self.lock: if plan_number in self.pbars: status = "SUCCESS" if success else "FAILED" self.pbars[plan_number].set_postfix_str(f"{status} in {duration:.1f}s") self.pbars[plan_number].close() del self.pbars[plan_number] ``` ###### __init__ Python ```python __init__() ``` Initialize progress bar callback. Source code in `ras_commander/callbacks.py` ```python def __init__(self): """Initialize progress bar callback.""" if not TQDM_AVAILABLE: raise ImportError( "ProgressBarCallback requires tqdm package. " "Install with: pip install tqdm" ) self.pbars = {} self.lock = Lock() ``` ###### on_exec_start Python ```python on_exec_start(plan_number: str, command: str) -> None ``` Create progress bar for this plan. Source code in `ras_commander/callbacks.py` ```python def on_exec_start(self, plan_number: str, command: str) -> None: """Create progress bar for this plan.""" with self.lock: self.pbars[plan_number] = tqdm( desc=f"Plan {plan_number}", unit="msg", dynamic_ncols=True ) ``` ###### on_exec_message Python ```python on_exec_message(plan_number: str, message: str) -> None ``` Update progress bar with new message. Source code in `ras_commander/callbacks.py` ```python def on_exec_message(self, plan_number: str, message: str) -> None: """Update progress bar with new message.""" with self.lock: if plan_number in self.pbars: # Show last 50 characters of message short_message = message.strip()[-50:] self.pbars[plan_number].set_postfix_str(short_message) self.pbars[plan_number].update(1) ``` ###### on_exec_complete Python ```python on_exec_complete(plan_number: str, success: bool, duration: float) -> None ``` Close progress bar and display final status. Source code in `ras_commander/callbacks.py` ```python def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Close progress bar and display final status.""" with self.lock: if plan_number in self.pbars: status = "SUCCESS" if success else "FAILED" self.pbars[plan_number].set_postfix_str(f"{status} in {duration:.1f}s") self.pbars[plan_number].close() del self.pbars[plan_number] ``` ###### SynchronizedCallback Thread-safe wrapper for any callback implementation. Wraps an existing callback to add thread-safety using locks. Useful when: - Using a callback that isn't thread-safe - Working with compute_parallel() - Sharing state across callbacks Thread Safety Wraps all callback methods with threading.Lock. Guarantees only one thread executes callback at a time. Example > > > class MyCallback: ... def **init**(self): ... self.messages = [] # Not thread-safe! ... def on_exec_message(self, plan_number, message): ... self.messages.append((plan_number, message)) > > > > > > unsafe_callback = MyCallback() safe_callback = SynchronizedCallback(unsafe_callback) RasCmdr.compute_parallel(["01", "02"], stream_callback=safe_callback) Source code in `ras_commander/callbacks.py` ```python class SynchronizedCallback: """ Thread-safe wrapper for any callback implementation. Wraps an existing callback to add thread-safety using locks. Useful when: - Using a callback that isn't thread-safe - Working with compute_parallel() - Sharing state across callbacks Thread Safety: Wraps all callback methods with threading.Lock. Guarantees only one thread executes callback at a time. Example: >>> class MyCallback: ... def __init__(self): ... self.messages = [] # Not thread-safe! ... def on_exec_message(self, plan_number, message): ... self.messages.append((plan_number, message)) >>> >>> unsafe_callback = MyCallback() >>> safe_callback = SynchronizedCallback(unsafe_callback) >>> RasCmdr.compute_parallel(["01", "02"], stream_callback=safe_callback) """ def __init__(self, callback: ExecutionCallback): """ Wrap a callback with thread-safety. Args: callback: Callback object to wrap (must implement ExecutionCallback methods) """ self._callback = callback self._lock = Lock() def on_prep_start(self, plan_number: str) -> None: """Thread-safe wrapper for on_prep_start.""" with self._lock: if hasattr(self._callback, 'on_prep_start'): self._callback.on_prep_start(plan_number) def on_prep_complete(self, plan_number: str) -> None: """Thread-safe wrapper for on_prep_complete.""" with self._lock: if hasattr(self._callback, 'on_prep_complete'): self._callback.on_prep_complete(plan_number) def on_exec_start(self, plan_number: str, command: str) -> None: """Thread-safe wrapper for on_exec_start.""" with self._lock: if hasattr(self._callback, 'on_exec_start'): self._callback.on_exec_start(plan_number, command) def on_exec_message(self, plan_number: str, message: str) -> None: """Thread-safe wrapper for on_exec_message.""" with self._lock: if hasattr(self._callback, 'on_exec_message'): self._callback.on_exec_message(plan_number, message) def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Thread-safe wrapper for on_exec_complete.""" with self._lock: if hasattr(self._callback, 'on_exec_complete'): self._callback.on_exec_complete(plan_number, success, duration) def on_verify_result(self, plan_number: str, verified: bool) -> None: """Thread-safe wrapper for on_verify_result.""" with self._lock: if hasattr(self._callback, 'on_verify_result'): self._callback.on_verify_result(plan_number, verified) ``` ###### __init__ Python ```python __init__(callback: ExecutionCallback) ``` Wrap a callback with thread-safety. Parameters: | Name | Type | Description | Default | | ---------- | ------------------- | ------------------------------------------------------------------ | ---------- | | `callback` | `ExecutionCallback` | Callback object to wrap (must implement ExecutionCallback methods) | *required* | Source code in `ras_commander/callbacks.py` ```python def __init__(self, callback: ExecutionCallback): """ Wrap a callback with thread-safety. Args: callback: Callback object to wrap (must implement ExecutionCallback methods) """ self._callback = callback self._lock = Lock() ``` ###### on_prep_start Python ```python on_prep_start(plan_number: str) -> None ``` Thread-safe wrapper for on_prep_start. Source code in `ras_commander/callbacks.py` ```python def on_prep_start(self, plan_number: str) -> None: """Thread-safe wrapper for on_prep_start.""" with self._lock: if hasattr(self._callback, 'on_prep_start'): self._callback.on_prep_start(plan_number) ``` ###### on_prep_complete Python ```python on_prep_complete(plan_number: str) -> None ``` Thread-safe wrapper for on_prep_complete. Source code in `ras_commander/callbacks.py` ```python def on_prep_complete(self, plan_number: str) -> None: """Thread-safe wrapper for on_prep_complete.""" with self._lock: if hasattr(self._callback, 'on_prep_complete'): self._callback.on_prep_complete(plan_number) ``` ###### on_exec_start Python ```python on_exec_start(plan_number: str, command: str) -> None ``` Thread-safe wrapper for on_exec_start. Source code in `ras_commander/callbacks.py` ```python def on_exec_start(self, plan_number: str, command: str) -> None: """Thread-safe wrapper for on_exec_start.""" with self._lock: if hasattr(self._callback, 'on_exec_start'): self._callback.on_exec_start(plan_number, command) ``` ###### on_exec_message Python ```python on_exec_message(plan_number: str, message: str) -> None ``` Thread-safe wrapper for on_exec_message. Source code in `ras_commander/callbacks.py` ```python def on_exec_message(self, plan_number: str, message: str) -> None: """Thread-safe wrapper for on_exec_message.""" with self._lock: if hasattr(self._callback, 'on_exec_message'): self._callback.on_exec_message(plan_number, message) ``` ###### on_exec_complete Python ```python on_exec_complete(plan_number: str, success: bool, duration: float) -> None ``` Thread-safe wrapper for on_exec_complete. Source code in `ras_commander/callbacks.py` ```python def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """Thread-safe wrapper for on_exec_complete.""" with self._lock: if hasattr(self._callback, 'on_exec_complete'): self._callback.on_exec_complete(plan_number, success, duration) ``` ###### on_verify_result Python ```python on_verify_result(plan_number: str, verified: bool) -> None ``` Thread-safe wrapper for on_verify_result. Source code in `ras_commander/callbacks.py` ```python def on_verify_result(self, plan_number: str, verified: bool) -> None: """Thread-safe wrapper for on_verify_result.""" with self._lock: if hasattr(self._callback, 'on_verify_result'): self._callback.on_verify_result(plan_number, verified) ``` ##### Custom Callbacks Create custom callbacks by implementing the `ExecutionCallback` protocol: Python ```python class CustomCallback: """Minimal custom callback - implement only what you need.""" def on_exec_complete(self, plan_number, success, duration): status = "SUCCESS" if success else "FAILED" print(f"Plan {plan_number}: {status} in {duration:.1f}s") # Use it RasCmdr.compute_plan("01", stream_callback=CustomCallback()) ``` Thread Safety for Parallel Execution Callbacks used with `compute_parallel()` must be thread-safe. Use `threading.Lock` for shared state: Python ```python from threading import Lock class ThreadSafeCallback: def __init__(self): self.lock = Lock() self.results = {} def on_exec_complete(self, plan_number, success, duration): with self.lock: self.results[plan_number] = (success, duration) ``` ##### BcoMonitor Utility ###### BcoMonitor Monitor HEC-RAS .bco file for execution signals and messages. The .bco file is created when 'Write Detailed= 1' is set in the plan file. It contains detailed computation messages written incrementally during execution. This class enables: - Real-time progress monitoring via .bco file polling - Message streaming to callbacks - Signal detection for early termination - Thread-safe operation (no shared state) Attributes: | Name | Type | Description | | ------------------ | --------------------------------- | ----------------------------------------------- | | `project_path` | `Path` | Path to HEC-RAS project folder | | `plan_number` | `str` | Plan number (e.g., "01", "02") | | `project_name` | `str` | Project name (without extension) | | `signal_string` | `str` | String to detect in .bco for early termination | | `check_interval` | `float` | Seconds between .bco file polls (default: 0.5) | | `max_wait_seconds` | `int` | Maximum wait time before timeout (default: 300) | | `message_callback` | `Optional[Callable[[str], None]]` | Optional callback for new messages | Example > > > monitor = BcoMonitor( ... project_path=Path("/path/to/project"), ... plan_number="01", ... project_name="MyProject", ... message_callback=lambda msg: print(msg) ... ) process = subprocess.Popen(["RAS.exe", ...]) signal_detected = monitor.monitor_until_signal(process) Source code in `ras_commander/RasBco.py` ```python @dataclass class BcoMonitor: """ Monitor HEC-RAS .bco file for execution signals and messages. The .bco file is created when 'Write Detailed= 1' is set in the plan file. It contains detailed computation messages written incrementally during execution. This class enables: - Real-time progress monitoring via .bco file polling - Message streaming to callbacks - Signal detection for early termination - Thread-safe operation (no shared state) Attributes: project_path: Path to HEC-RAS project folder plan_number: Plan number (e.g., "01", "02") project_name: Project name (without extension) signal_string: String to detect in .bco for early termination check_interval: Seconds between .bco file polls (default: 0.5) max_wait_seconds: Maximum wait time before timeout (default: 300) message_callback: Optional callback for new messages Example: >>> monitor = BcoMonitor( ... project_path=Path("/path/to/project"), ... plan_number="01", ... project_name="MyProject", ... message_callback=lambda msg: print(msg) ... ) >>> process = subprocess.Popen(["RAS.exe", ...]) >>> signal_detected = monitor.monitor_until_signal(process) """ # Configuration project_path: Path plan_number: str project_name: str signal_string: str = "Starting Unsteady Flow Computations" check_interval: float = 0.5 max_wait_seconds: int = 300 # Optional callback for streaming messages message_callback: Optional[Callable[[str], None]] = None # Internal state (initialized in __post_init__) bco_file: Path = field(init=False) execution_start_time: Optional[float] = field(default=None, init=False) _last_file_position: int = field(default=0, init=False) def __post_init__(self): """Initialize paths and validate configuration.""" self.bco_file = self.project_path / f"{self.project_name}.bco{self.plan_number}" logger.debug(f"BcoMonitor initialized for {self.bco_file.name}") @staticmethod def enable_detailed_logging(plan_file_path: Path) -> bool: """ Enable detailed logging in a plan file by setting 'Write Detailed= 1'. This creates a .bcoXX file during HEC-RAS execution that can be monitored for the "Starting Unsteady Flow Computations" signal and other messages. Args: plan_file_path: Path to the plan file (.pXX) Returns: bool: True if successful, False otherwise Note: This modifies the plan file in-place. The modification is safe and follows HEC-RAS plan file format conventions. """ try: content = plan_file_path.read_text(encoding='utf-8', errors='ignore') # Check if Write Detailed line exists if "Write Detailed=" in content: # Replace existing setting new_content = re.sub( r'Write Detailed=\s*\d+', 'Write Detailed= 1', content ) if new_content != content: plan_file_path.write_text(new_content, encoding='utf-8') logger.debug(f"Enabled detailed logging in {plan_file_path.name}") else: # Add the setting after Run HTab line or at the end if "Run HTab=" in content: new_content = content.replace( "Run HTab=", "Write Detailed= 1\nRun HTab=" ) else: new_content = content + "\nWrite Detailed= 1\n" plan_file_path.write_text(new_content, encoding='utf-8') logger.debug(f"Added detailed logging setting to {plan_file_path.name}") return True except Exception as e: logger.warning(f"Could not enable detailed logging: {e}") return False def monitor_until_signal(self, process: subprocess.Popen) -> bool: """ Monitor .bco file until signal string appears or process completes. This method polls the .bco file at regular intervals, checking for: 1. Process completion (normal exit or crash) 2. Signal string detection (for early termination) 3. New messages (streamed to callback if provided) Args: process: Running subprocess.Popen instance Returns: bool: True if signal detected, False if process completed without signal Note: - This is a blocking call that returns when signal appears or process exits - Callbacks are invoked from the calling thread (not a new thread) - File is read incrementally to minimize I/O overhead """ self.execution_start_time = time.time() start_time = time.time() logger.info(f"Monitoring {self.bco_file.name} for '{self.signal_string}' signal...") while time.time() - start_time < self.max_wait_seconds: # Check if process died if process.poll() is not None: logger.info(f"Process exited with code {process.returncode}") # Read any final messages if self.bco_file.exists(): self._read_and_callback_new_content() return False # Check for .bco file with signal detection if self.bco_file.exists(): # Verify file was modified after we started execution file_mtime = self.bco_file.stat().st_mtime if file_mtime >= self.execution_start_time: # Read new content and check for signal content = self._read_and_callback_new_content() if content and self.signal_string in content: logger.info(f"Detected '{self.signal_string}' in {self.bco_file.name}") return True time.sleep(self.check_interval) logger.warning(f"Monitoring timed out after {self.max_wait_seconds}s") return False def get_final_messages(self) -> Optional[str]: """ Read complete .bco file content after execution. Returns: Optional[str]: Full .bco file content, or None if file doesn't exist Note: Uses encoding resilience (errors='ignore') to handle partially written files. """ if not self.bco_file.exists(): return None try: return self.bco_file.read_text(encoding='utf-8', errors='ignore') except Exception as e: logger.debug(f"Could not read .bco file: {e}") return None def _read_and_callback_new_content(self) -> Optional[str]: """ Read new content since last position and invoke callback if provided. Returns: Optional[str]: New content read, or None if error/no new content Note: Updates internal _last_file_position to track reading progress. """ try: # Get current file size file_size = self.bco_file.stat().st_size # No new content since last read if file_size <= self._last_file_position: return None # Read from last position to current end with open(self.bco_file, 'r', encoding='utf-8', errors='ignore') as f: f.seek(self._last_file_position) new_content = f.read() self._last_file_position = f.tell() # Invoke callback with new content if provided if new_content and self.message_callback: # Split into lines and call back for each non-empty line for line in new_content.splitlines(): if line.strip(): try: self.message_callback(line) except Exception as e: logger.warning(f"Callback error: {e}") return new_content except Exception as e: logger.debug(f"Could not read new .bco content: {e}") return None def has_signal(self) -> bool: """ Check if signal string exists in .bco file. Returns: bool: True if signal detected, False otherwise """ content = self.get_final_messages() return content is not None and self.signal_string in content ``` ###### enable_detailed_logging Python ```python enable_detailed_logging(plan_file_path: Path) -> bool ``` Enable detailed logging in a plan file by setting 'Write Detailed= 1'. This creates a .bcoXX file during HEC-RAS execution that can be monitored for the "Starting Unsteady Flow Computations" signal and other messages. Parameters: | Name | Type | Description | Default | | ---------------- | ------ | ---------------------------- | ---------- | | `plan_file_path` | `Path` | Path to the plan file (.pXX) | *required* | Returns: | Name | Type | Description | | ------ | ------ | ----------------------------------- | | `bool` | `bool` | True if successful, False otherwise | Note This modifies the plan file in-place. The modification is safe and follows HEC-RAS plan file format conventions. Source code in `ras_commander/RasBco.py` ```python @staticmethod def enable_detailed_logging(plan_file_path: Path) -> bool: """ Enable detailed logging in a plan file by setting 'Write Detailed= 1'. This creates a .bcoXX file during HEC-RAS execution that can be monitored for the "Starting Unsteady Flow Computations" signal and other messages. Args: plan_file_path: Path to the plan file (.pXX) Returns: bool: True if successful, False otherwise Note: This modifies the plan file in-place. The modification is safe and follows HEC-RAS plan file format conventions. """ try: content = plan_file_path.read_text(encoding='utf-8', errors='ignore') # Check if Write Detailed line exists if "Write Detailed=" in content: # Replace existing setting new_content = re.sub( r'Write Detailed=\s*\d+', 'Write Detailed= 1', content ) if new_content != content: plan_file_path.write_text(new_content, encoding='utf-8') logger.debug(f"Enabled detailed logging in {plan_file_path.name}") else: # Add the setting after Run HTab line or at the end if "Run HTab=" in content: new_content = content.replace( "Run HTab=", "Write Detailed= 1\nRun HTab=" ) else: new_content = content + "\nWrite Detailed= 1\n" plan_file_path.write_text(new_content, encoding='utf-8') logger.debug(f"Added detailed logging setting to {plan_file_path.name}") return True except Exception as e: logger.warning(f"Could not enable detailed logging: {e}") return False ``` ###### monitor_until_signal Python ```python monitor_until_signal(process: Popen) -> bool ``` Monitor .bco file until signal string appears or process completes. This method polls the .bco file at regular intervals, checking for: 1. Process completion (normal exit or crash) 1. Signal string detection (for early termination) 1. New messages (streamed to callback if provided) Parameters: | Name | Type | Description | Default | | --------- | ------- | --------------------------------- | ---------- | | `process` | `Popen` | Running subprocess.Popen instance | *required* | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------------------------------ | | `bool` | `bool` | True if signal detected, False if process completed without signal | Note - This is a blocking call that returns when signal appears or process exits - Callbacks are invoked from the calling thread (not a new thread) - File is read incrementally to minimize I/O overhead Source code in `ras_commander/RasBco.py` ```python def monitor_until_signal(self, process: subprocess.Popen) -> bool: """ Monitor .bco file until signal string appears or process completes. This method polls the .bco file at regular intervals, checking for: 1. Process completion (normal exit or crash) 2. Signal string detection (for early termination) 3. New messages (streamed to callback if provided) Args: process: Running subprocess.Popen instance Returns: bool: True if signal detected, False if process completed without signal Note: - This is a blocking call that returns when signal appears or process exits - Callbacks are invoked from the calling thread (not a new thread) - File is read incrementally to minimize I/O overhead """ self.execution_start_time = time.time() start_time = time.time() logger.info(f"Monitoring {self.bco_file.name} for '{self.signal_string}' signal...") while time.time() - start_time < self.max_wait_seconds: # Check if process died if process.poll() is not None: logger.info(f"Process exited with code {process.returncode}") # Read any final messages if self.bco_file.exists(): self._read_and_callback_new_content() return False # Check for .bco file with signal detection if self.bco_file.exists(): # Verify file was modified after we started execution file_mtime = self.bco_file.stat().st_mtime if file_mtime >= self.execution_start_time: # Read new content and check for signal content = self._read_and_callback_new_content() if content and self.signal_string in content: logger.info(f"Detected '{self.signal_string}' in {self.bco_file.name}") return True time.sleep(self.check_interval) logger.warning(f"Monitoring timed out after {self.max_wait_seconds}s") return False ``` ##### ExecutionCallback Protocol ###### ExecutionCallback ExecutionCallback - Protocol for HEC-RAS execution progress callbacks. This module defines the callback interface for monitoring HEC-RAS computation lifecycle events. Callbacks enable real-time progress tracking, logging, and UI updates during long-running simulations. The Protocol pattern allows partial implementation - classes only need to implement the callback methods they care about. ###### ExecutionCallback Bases: `Protocol` Protocol for execution progress callbacks. This defines the interface for monitoring HEC-RAS computation lifecycle. Implementations can provide any subset of these methods - all are optional. Lifecycle Order 1. on_prep_start() - Before geometry preprocessing 1. on_prep_complete() - After preprocessing 1. on_exec_start() - HEC-RAS subprocess started 1. on_exec_message() - During execution (potentially many calls) 1. on_exec_complete() - HEC-RAS subprocess finished 1. on_verify_result() - After HDF verification (if verify=True) Thread Safety When used with compute_parallel(), callbacks are invoked from worker threads concurrently. Implementations MUST be thread-safe. Use locks, thread-local storage, or atomic operations as needed. Example - Simple Console Logging > > > class ConsoleCallback: ... def on_exec_start(self, plan_number, command): ... print(f"[{plan_number}] Starting...") ... def on_exec_message(self, plan_number, message): ... print(f"[{plan_number}] {message}") ... def on_exec_complete(self, plan_number, success, duration): ... print(f"[{plan_number}] Done in {duration:.1f}s") Example - Thread-Safe File Logging > > > from threading import Lock class FileCallback: ... def **init**(self): ... self.lock = Lock() ... self.files = {} ... def on_exec_start(self, plan_number, command): ... with self.lock: ... self.files[plan_number] = open(f"plan\_{plan_number}.log", 'w') ... def on_exec_message(self, plan_number, message): ... with self.lock: ... if plan_number in self.files: ... self.files[plan_number].write(message + '\\n') Source code in `ras_commander/ExecutionCallback.py` ```python @runtime_checkable class ExecutionCallback(Protocol): """ Protocol for execution progress callbacks. This defines the interface for monitoring HEC-RAS computation lifecycle. Implementations can provide any subset of these methods - all are optional. Lifecycle Order: 1. on_prep_start() - Before geometry preprocessing 2. on_prep_complete() - After preprocessing 3. on_exec_start() - HEC-RAS subprocess started 4. on_exec_message() - During execution (potentially many calls) 5. on_exec_complete() - HEC-RAS subprocess finished 6. on_verify_result() - After HDF verification (if verify=True) Thread Safety: When used with compute_parallel(), callbacks are invoked from worker threads concurrently. Implementations MUST be thread-safe. Use locks, thread-local storage, or atomic operations as needed. Example - Simple Console Logging: >>> class ConsoleCallback: ... def on_exec_start(self, plan_number, command): ... print(f"[{plan_number}] Starting...") ... def on_exec_message(self, plan_number, message): ... print(f"[{plan_number}] {message}") ... def on_exec_complete(self, plan_number, success, duration): ... print(f"[{plan_number}] Done in {duration:.1f}s") Example - Thread-Safe File Logging: >>> from threading import Lock >>> class FileCallback: ... def __init__(self): ... self.lock = Lock() ... self.files = {} ... def on_exec_start(self, plan_number, command): ... with self.lock: ... self.files[plan_number] = open(f"plan_{plan_number}.log", 'w') ... def on_exec_message(self, plan_number, message): ... with self.lock: ... if plan_number in self.files: ... self.files[plan_number].write(message + '\\n') """ def on_prep_start(self, plan_number: str) -> None: """ Called before geometry preprocessing and core setup. This is invoked before: - Geometry preprocessor file clearing (if clear_geompre=True) - Number of cores configuration (if num_cores specified) Args: plan_number: Plan identifier (e.g., "01", "02") Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... def on_prep_complete(self, plan_number: str) -> None: """ Called after geometry preprocessing and core setup complete. This is invoked after: - Geometry preprocessor files cleared (if applicable) - Number of cores set in plan file (if applicable) - Just before HEC-RAS subprocess starts Args: plan_number: Plan identifier (e.g., "01", "02") Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... def on_exec_start(self, plan_number: str, command: str) -> None: """ Called when HEC-RAS subprocess starts. This is invoked immediately before subprocess execution begins. The command includes the full command line that will be executed. Args: plan_number: Plan identifier (e.g., "01", "02") command: Full command line (e.g., '"C:/RAS/RAS.exe" -c project.prj plan.p01') Note: At this point the subprocess has been constructed but not yet started. This is the last callback before HEC-RAS begins running. Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... def on_exec_message(self, plan_number: str, message: str) -> None: """ Called for each new .bco file message during execution. This is invoked repeatedly as HEC-RAS writes to the .bco file. Messages are streamed line-by-line in near real-time (polling interval: 0.5s). Args: plan_number: Plan identifier (e.g., "01", "02") message: Single line from .bco file (newline stripped) Frequency: - Called potentially hundreds or thousands of times per plan - Frequency depends on HEC-RAS computation complexity - Polling interval: 0.5 seconds (configurable in BcoMonitor) Performance: - Keep callback implementation FAST (< 1ms recommended) - Avoid blocking I/O, network calls, or heavy computation - For expensive operations, queue messages and process in separate thread Thread Safety: May be called concurrently for different plans in compute_parallel(). CRITICAL: Implement proper locking if writing to shared resources. Example Messages: - "Geometry Preprocessor Version 6.6" - "Computing Cross Section HTAB's" - "Starting Unsteady Flow Computations" - "Time: 01JAN2020 0600 [ 1.25% Done]" """ ... def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """ Called when HEC-RAS execution finishes. This is invoked immediately after subprocess completes (successfully or not). Args: plan_number: Plan identifier (e.g., "01", "02") success: True if subprocess exited with code 0, False otherwise duration: Execution time in seconds (floating point) Note: - success=True does NOT guarantee HEC-RAS succeeded (it may have errors) - Use on_verify_result() to check if HDF contains "Complete Process" - duration is wall-clock time, not CPU time Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... def on_verify_result(self, plan_number: str, verified: bool) -> None: """ Called after HDF verification (only if verify=True parameter used). This is invoked after checking HDF file for "Complete Process" message. Args: plan_number: Plan identifier (e.g., "01", "02") verified: True if HDF contains "Complete Process", False otherwise Note: - Only called when RasCmdr.compute_plan(..., verify=True) - verified=True is the strongest guarantee that HEC-RAS succeeded - verified=False may indicate computation errors or incomplete results Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` ###### on_prep_start Python ```python on_prep_start(plan_number: str) -> None ``` Called before geometry preprocessing and core setup. This is invoked before: - Geometry preprocessor file clearing (if clear_geompre=True) - Number of cores configuration (if num_cores specified) Parameters: | Name | Type | Description | Default | | ------------- | ----- | ---------------------------------- | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | Thread Safety May be called concurrently for different plans in compute_parallel(). Source code in `ras_commander/ExecutionCallback.py` ```python def on_prep_start(self, plan_number: str) -> None: """ Called before geometry preprocessing and core setup. This is invoked before: - Geometry preprocessor file clearing (if clear_geompre=True) - Number of cores configuration (if num_cores specified) Args: plan_number: Plan identifier (e.g., "01", "02") Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` ###### on_prep_complete Python ```python on_prep_complete(plan_number: str) -> None ``` Called after geometry preprocessing and core setup complete. This is invoked after: - Geometry preprocessor files cleared (if applicable) - Number of cores set in plan file (if applicable) - Just before HEC-RAS subprocess starts Parameters: | Name | Type | Description | Default | | ------------- | ----- | ---------------------------------- | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | Thread Safety May be called concurrently for different plans in compute_parallel(). Source code in `ras_commander/ExecutionCallback.py` ```python def on_prep_complete(self, plan_number: str) -> None: """ Called after geometry preprocessing and core setup complete. This is invoked after: - Geometry preprocessor files cleared (if applicable) - Number of cores set in plan file (if applicable) - Just before HEC-RAS subprocess starts Args: plan_number: Plan identifier (e.g., "01", "02") Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` ###### on_exec_start Python ```python on_exec_start(plan_number: str, command: str) -> None ``` Called when HEC-RAS subprocess starts. This is invoked immediately before subprocess execution begins. The command includes the full command line that will be executed. Parameters: | Name | Type | Description | Default | | ------------- | ----- | -------------------------------------------------------------------- | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | | `command` | `str` | Full command line (e.g., '"C:/RAS/RAS.exe" -c project.prj plan.p01') | *required* | Note At this point the subprocess has been constructed but not yet started. This is the last callback before HEC-RAS begins running. Thread Safety May be called concurrently for different plans in compute_parallel(). Source code in `ras_commander/ExecutionCallback.py` ```python def on_exec_start(self, plan_number: str, command: str) -> None: """ Called when HEC-RAS subprocess starts. This is invoked immediately before subprocess execution begins. The command includes the full command line that will be executed. Args: plan_number: Plan identifier (e.g., "01", "02") command: Full command line (e.g., '"C:/RAS/RAS.exe" -c project.prj plan.p01') Note: At this point the subprocess has been constructed but not yet started. This is the last callback before HEC-RAS begins running. Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` ###### on_exec_message Python ```python on_exec_message(plan_number: str, message: str) -> None ``` Called for each new .bco file message during execution. This is invoked repeatedly as HEC-RAS writes to the .bco file. Messages are streamed line-by-line in near real-time (polling interval: 0.5s). Parameters: | Name | Type | Description | Default | | ------------- | ----- | --------------------------------------------- | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | | `message` | `str` | Single line from .bco file (newline stripped) | *required* | Frequency - Called potentially hundreds or thousands of times per plan - Frequency depends on HEC-RAS computation complexity - Polling interval: 0.5 seconds (configurable in BcoMonitor) Performance - Keep callback implementation FAST (< 1ms recommended) - Avoid blocking I/O, network calls, or heavy computation - For expensive operations, queue messages and process in separate thread Thread Safety May be called concurrently for different plans in compute_parallel(). CRITICAL: Implement proper locking if writing to shared resources. Example Messages - "Geometry Preprocessor Version 6.6" - "Computing Cross Section HTAB's" - "Starting Unsteady Flow Computations" - "Time: 01JAN2020 0600 [ 1.25% Done]" Source code in `ras_commander/ExecutionCallback.py` ```python def on_exec_message(self, plan_number: str, message: str) -> None: """ Called for each new .bco file message during execution. This is invoked repeatedly as HEC-RAS writes to the .bco file. Messages are streamed line-by-line in near real-time (polling interval: 0.5s). Args: plan_number: Plan identifier (e.g., "01", "02") message: Single line from .bco file (newline stripped) Frequency: - Called potentially hundreds or thousands of times per plan - Frequency depends on HEC-RAS computation complexity - Polling interval: 0.5 seconds (configurable in BcoMonitor) Performance: - Keep callback implementation FAST (< 1ms recommended) - Avoid blocking I/O, network calls, or heavy computation - For expensive operations, queue messages and process in separate thread Thread Safety: May be called concurrently for different plans in compute_parallel(). CRITICAL: Implement proper locking if writing to shared resources. Example Messages: - "Geometry Preprocessor Version 6.6" - "Computing Cross Section HTAB's" - "Starting Unsteady Flow Computations" - "Time: 01JAN2020 0600 [ 1.25% Done]" """ ... ``` ###### on_exec_complete Python ```python on_exec_complete(plan_number: str, success: bool, duration: float) -> None ``` Called when HEC-RAS execution finishes. This is invoked immediately after subprocess completes (successfully or not). Parameters: | Name | Type | Description | Default | | ------------- | ------- | ------------------------------------------------------ | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | | `success` | `bool` | True if subprocess exited with code 0, False otherwise | *required* | | `duration` | `float` | Execution time in seconds (floating point) | *required* | Note - success=True does NOT guarantee HEC-RAS succeeded (it may have errors) - Use on_verify_result() to check if HDF contains "Complete Process" - duration is wall-clock time, not CPU time Thread Safety May be called concurrently for different plans in compute_parallel(). Source code in `ras_commander/ExecutionCallback.py` ```python def on_exec_complete(self, plan_number: str, success: bool, duration: float) -> None: """ Called when HEC-RAS execution finishes. This is invoked immediately after subprocess completes (successfully or not). Args: plan_number: Plan identifier (e.g., "01", "02") success: True if subprocess exited with code 0, False otherwise duration: Execution time in seconds (floating point) Note: - success=True does NOT guarantee HEC-RAS succeeded (it may have errors) - Use on_verify_result() to check if HDF contains "Complete Process" - duration is wall-clock time, not CPU time Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` ###### on_verify_result Python ```python on_verify_result(plan_number: str, verified: bool) -> None ``` Called after HDF verification (only if verify=True parameter used). This is invoked after checking HDF file for "Complete Process" message. Parameters: | Name | Type | Description | Default | | ------------- | ------ | -------------------------------------------------------- | ---------- | | `plan_number` | `str` | Plan identifier (e.g., "01", "02") | *required* | | `verified` | `bool` | True if HDF contains "Complete Process", False otherwise | *required* | Note - Only called when RasCmdr.compute_plan(..., verify=True) - verified=True is the strongest guarantee that HEC-RAS succeeded - verified=False may indicate computation errors or incomplete results Thread Safety May be called concurrently for different plans in compute_parallel(). Source code in `ras_commander/ExecutionCallback.py` ```python def on_verify_result(self, plan_number: str, verified: bool) -> None: """ Called after HDF verification (only if verify=True parameter used). This is invoked after checking HDF file for "Complete Process" message. Args: plan_number: Plan identifier (e.g., "01", "02") verified: True if HDF contains "Complete Process", False otherwise Note: - Only called when RasCmdr.compute_plan(..., verify=True) - verified=True is the strongest guarantee that HEC-RAS succeeded - verified=False may indicate computation errors or incomplete results Thread Safety: May be called concurrently for different plans in compute_parallel(). """ ... ``` All callback methods are **optional** - implement only what you need. The protocol uses `@runtime_checkable` for flexible duck-typing. ### RasControl ### RasControl RasControl - HECRASController API Wrapper (ras-commander style) Provides ras-commander style API for legacy HEC-RAS versions (3.x-4.x) that use HECRASController COM interface instead of HDF files. Includes robust process management with session tracking, orphan detection, and optional watchdog protection for Jupyter kernel restarts. Public functions (HEC-RAS Operations): - RasControl.run_plan(plan, ras_object=None, force_recompute=False, use_watchdog=True, max_runtime=3600) -> Tuple\[bool, List[str]\] - RasControl.get_steady_results(plan, ras_object=None) -> pandas.DataFrame - RasControl.get_unsteady_results(plan, max_times=None, ras_object=None) -> pandas.DataFrame - RasControl.get_output_times(plan, ras_object=None) -> List[str] - RasControl.get_plans(plan, ras_object=None) -> List[dict] - RasControl.set_current_plan(plan, ras_object=None) -> bool - RasControl.get_comp_msgs(plan, ras_object=None) -> str Public functions (Process Management): - RasControl.list_processes(show_all=False) -> pandas.DataFrame - RasControl.scan_orphans() -> List[SessionLock] - RasControl.cleanup_orphans(interactive=True, dry_run=False) -> int - RasControl.force_cleanup_all() -> int Private functions: - \_terminate_ras_process() -> None - \_is_ras_running() -> bool - RasControl.\_normalize_version(version: str) -> str - RasControl.\_get_project_info(plan, ras_object=None) -> Tuple\[Path, str, Optional[str], Optional[str]\] - RasControl.\_com_open_close(project_path: Path, version: str, operation_func: Callable\[[Any], Any\]) -> Any Session tracking infrastructure: - SessionLock dataclass - Tracks active COM sessions with lock files - Module-level \_active_sessions dict - Tracks all active sessions - atexit handler - Emergency cleanup on Python exit - Watchdog support - Optional independent process for kernel restart protection #### RasControl Details Open-Operate-Close Pattern Unlike other ras-commander classes, RasControl opens HEC-RAS, performs one operation, then closes it. This prevents conflicts with modern workflows and ensures clean resource management. ##### Supported Versions | Version | Registry Key | HEC-RAS Years | | ---------------------------------- | ------------ | ------------- | | `"31"` | 3.1 | Legacy | | `"41"` | 4.1 | ~2008-2014 | | `"501"`, `"503"`, `"505"`, `"506"` | 5.0.x | 2015-2019 | | `"60"` | 6.0 | 2020 | | `"63"` | 6.3 | 2021-2022 | | `"66"` | 6.6 | 2023-2024 | | `"70"` | 7.0 | 2025+ | ##### RasControl vs RasCmdr | Aspect | RasControl | RasCmdr | | -------------------- | ------------------------- | ----------------------- | | **HEC-RAS Versions** | 3.x - 7.x (COM) | 5.x+ (command line) | | **Data Source** | Live COM extraction | HDF file results | | **Requires GUI** | Yes (HEC-RAS installed) | Yes (HEC-RAS installed) | | **Use Case** | Legacy models, validation | Modern automation | | **Returns** | pandas DataFrame | bool / dict | ##### Understanding "Max WS" in Unsteady Results When extracting unsteady results, the **first row per cross section** (time_index=1) contains "Max WS" - the maximum at ANY computational timestep: Python ```python # Unsteady results include special "Max WS" row df = RasControl.get_unsteady_results("01") # time_index=1 is "Max WS" (maximum at any timestep) df_max = df[df['time_string'] == 'Max WS'] # time_index=2+ are actual output intervals df_timeseries = df[df['time_string'] != 'Max WS'] # Parse datetime for analysis df_timeseries['datetime'] = pd.to_datetime( df_timeseries['time_string'], format='%d%b%Y %H%M' ) ``` Max WS vs Output Interval Maximums "Max WS" captures peaks that may occur BETWEEN output intervals. This is critical for design applications - always use "Max WS" for peak values, not `max()` of output intervals. ##### Result Columns **Steady Results** (`get_steady_results`): | Column | Type | Description | | ----------- | ----- | ------------------------- | | `river` | str | River name | | `reach` | str | Reach name | | `node_id` | str | Cross section station | | `profile` | str | Profile name | | `wsel` | float | Water surface elevation | | `velocity` | float | Total velocity | | `flow` | float | Total flow | | `froude` | float | Froude number | | `energy` | float | Energy grade elevation | | `max_depth` | float | Maximum channel depth | | `min_ch_el` | float | Minimum channel elevation | **Unsteady Results** (`get_unsteady_results`): Same columns plus `time_index`, `time_string`, `datetime`. ##### Compute Messages Fallback The `get_comp_msgs()` method attempts to read computation messages from multiple sources: 1. First tries `.computeMsgs.txt` (modern format) 1. Falls back to `.comp_msgs.txt` (legacy format) 1. Returns empty string if neither exists ## File Operations ### RasPlan ### RasPlan RasPlan - Operations for handling plan files in HEC-RAS projects This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). 1. Obtain the logger using: logger = logging.getLogger(**name**) Example @log_call def my_function(): logger = logging.getLogger(**name**) logger.debug("Additional debug information") # Function logic here ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasPlan: - set_geom(): Set the geometry for a specified plan - set_steady(): Apply a steady flow file to a plan file - set_unsteady(): Apply an unsteady flow file to a plan file - set_num_cores(): Update the maximum number of cores to use - set_geom_preprocessor(): Update geometry preprocessor settings - clone_plan(): Create a new plan file based on a template - clone_unsteady(): Copy unsteady flow files from a template - clone_steady(): Copy steady flow files from a template - clone_geom(): Copy geometry files from a template - get_next_number(): Determine the next available number from a list - get_plan_value(): Retrieve a specific value from a plan file - get_results_path(): Get the results file path for a plan - get_plan_path(): Get the full path for a plan number - get_flow_path(): Get the full path for a flow number - get_unsteady_path(): Get the full path for an unsteady number - get_geom_path(): Get the full path for a geometry number - update_run_flags(): Update various run flags in a plan file - update_plan_intervals(): Update computation and output intervals - update_plan_description(): Update the description in a plan file - read_plan_description(): Read the description from a plan file - read_geom_description(): Read the description from a geometry file - update_geom_description(): Update the description in a geometry file - read_flow_description(): Read the description from a steady flow file - update_flow_description(): Update the description in a steady flow file - update_simulation_date(): Update simulation start and end dates - get_restart_output_settings(): Parse restart/Hot Start output settings - set_restart_output_settings(): Configure restart/Hot Start output settings - get_shortid(): Get the Short Identifier from a plan file - set_shortid(): Set the Short Identifier in a plan file - get_plan_title(): Get the Plan Title from a plan file - set_plan_title(): Set the Plan Title in a plan file - delete_plan(): Delete a plan and its associated files - renumber_plan(): Renumber a plan file and update references - delete_geom(): Delete a geometry file and its associated files - renumber_geom(): Renumber a geometry file and update references - delete_unsteady(): Delete an unsteady flow file - renumber_unsteady(): Renumber an unsteady flow file and update references - delete_steady(): Delete a steady flow file - renumber_steady(): Renumber a steady flow file and update references ### RasFlowOptimization ### RasFlowOptimization RasFlowOptimization - Native HEC-RAS flow hydrograph optimization helpers. This module configures HEC-RAS Automated Flow Optimization using the native `Flow Ratio ...` plan-file parameters and extracts trial summaries from HDF results or computation messages after execution. All methods are static and are designed to be used without instantiation. ### RasGeo ### RasGeo RasGeo - Operations for handling geometry files in HEC-RAS projects DEPRECATION NOTICE This class is deprecated and will be removed before v1.0. Please migrate to the new geometry subpackage classes: - GeomPreprocessor.clear_geompre_files() - replaces RasGeo.clear_geompre_files() - GeomLandCover.get_base_mannings_n() - replaces RasGeo.get_mannings_baseoverrides() - GeomLandCover.set_base_mannings_n() - replaces RasGeo.set_mannings_baseoverrides() - GeomLandCover.get_region_mannings_n() - replaces RasGeo.get_mannings_regionoverrides() - GeomLandCover.set_region_mannings_n() - replaces RasGeo.set_mannings_regionoverrides() This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasGeo: - clear_geompre_files(): Clears geometry preprocessor files for specified plan files [DEPRECATED] - get_mannings_baseoverrides(): Reads base Manning's n table from a geometry file [DEPRECATED] - get_mannings_regionoverrides(): Reads Manning's n region overrides from a geometry file [DEPRECATED] - set_mannings_baseoverrides(): Writes base Manning's n values to a geometry file [DEPRECATED] - set_mannings_regionoverrides(): Writes regional Manning's n overrides to a geometry file [DEPRECATED] - clone_geom(): Copy geometry files from a template with optional title and description ### RasUnsteady ### RasUnsteady RasUnsteady - Operations for handling unsteady flow files in HEC-RAS projects. This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). Example @log_call def my_function(): logger.debug("Additional debug information") # Function logic here ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasUnsteady: - update_flow_title() - read_unsteady_description() - update_unsteady_description() - update_restart_settings() - set_restart_settings() - get_restart_settings() - set_hydrograph_fixed_start_time() - extract_boundary_and_tables() - print_boundaries_and_tables() - identify_tables() - parse_fixed_width_table() - extract_tables() - write_table_to_file() - get_met_precipitation_config() - set_precipitation_hyetograph() - set_constant_precipitation() - set_gridded_precipitation() - configure_gridded_dss_precipitation() - set_meteorological_station() - get_meteorological_stations() - set_point_evapotranspiration() - get_point_evapotranspiration() Precipitation Functions: - get_met_precipitation_config() - Read Meteorological Data tab precipitation settings - set_precipitation_hyetograph() - Write hyetograph DataFrame to unsteady file - set_gridded_precipitation() - Configure GDAL raster precipitation - configure_gridded_dss_precipitation() - Configure gridded DSS precipitation Meteorological Point Data Functions: - set_meteorological_station() - Create/update meteorological station metadata - get_meteorological_stations() - Parse meteorological station metadata - set_point_evapotranspiration() - Write point ET series from a DataFrame - get_point_evapotranspiration() - Parse point ET series into a DataFrame DSS Boundary Condition Functions: - get_dss_boundaries() - Extract all DSS-linked BCs with full path info - get_inline_hydrograph_boundaries() - Extract inline table BCs with time series data - delete_boundary() - Remove one Boundary Location block from an unsteady file - update_dss_run_identifier() - Update DSS path F-part for new scenarios - set_boundary_dss_link() - Convert inline BC to DSS-linked (complete state transition) - set_boundary_inline_hydrograph() - Write inline hydrograph, convert DSS to inline - set_flow_hydrograph_slope() - Add or update `Flow Hydrograph Slope=` (EG slope) for a Flow Hydrograph BC - set_normal_depth_boundary() - Add or update Normal Depth (Friction Slope=) for a 1D river or 2D BC line boundary - get_unique_dss_subbasins() - Get unique HMS subbasin names from DSS paths - update_dss_path_by_station() - Update DSS A-part for specific river station - update_flow_multiplier_by_station() - Update/insert QMult for specific river station - update_boundary_dss_paths() - Batch update DSS paths and multipliers - get_rating_curve() - Read Rating Curve (stage, discharge) pairs from a boundary - set_rating_curve() - Write or replace Rating Curve data on a boundary Stage/Flow Hydrograph Functions (Internal Boundary): - get_stage_flow_hydrograph() - Read observed stage/flow pairs from an internal BC - set_stage_flow_hydrograph() - Write observed stage/flow pairs to an internal BC Lateral Inflow Hydrograph Functions: - get_lateral_inflow_hydrograph() - Read lateral inflow hydrograph data from a boundary - set_lateral_inflow_hydrograph() - Write lateral inflow hydrograph data to a boundary Uniform Lateral Inflow Hydrograph Functions: - get_uniform_lateral_inflow_hydrograph() - Read uniform lateral inflow data (reach-based BC) - set_uniform_lateral_inflow_hydrograph() - Write uniform lateral inflow data (reach-based BC) Initial Conditions Method Selection: - get_initial_flow_method() - Determine which IC method is active (restart_file, prior_ws, initial_flow_distribution, none) - set_initial_flow_method() - Set the IC method selection (restart_file, prior_ws, initial_flow_distribution, none) - get_prior_ws_filename() - Read Prior WS Filename and Profile from unsteady file - set_prior_ws_filename() - Write Prior WS Filename and Profile to unsteady file Initial Flow Distribution Table: - get_initial_conditions() - Read all IC entries (flow, storage, rrr) as DataFrame - set_initial_conditions() - Write IC entries from list of dicts or DataFrame (auto-sets IC method) - validate_initial_flow_stations() - Check IC flow stations match geometry cross sections Non-Newtonian Method Selection: - get_non_newtonian_method() - Read the Non-Newtonian method integer and return name - set_non_newtonian_method() - Set the Non-Newtonian method by integer or name ### RasSteady ### RasSteady RasSteady - Read and author HEC-RAS steady flow files (.f##). All methods are static and are designed to be used without instantiation. ## Utilities ### RasUtils ### RasUtils RasUtils - Utility functions for the ras-commander library This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). Example @log_call def my_function(): logger.debug("Additional debug information") # Function logic here ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasUtils: - create_directory() - safe_resolve() - find_files_by_extension() - get_file_size() - get_file_modification_time() - normalize_ras_number() - get_plan_path() - remove_with_retry() - update_plan_file() - check_file_access() - convert_to_dataframe() - save_to_excel() - calculate_rmse() - calculate_percent_bias() - calculate_error_metrics() - update_file() - get_next_number() - clone_file() - update_project_file() - remove_prj_entry() - rename_prj_entry() - decode_byte_strings() - perform_kdtree_query() - find_nearest_neighbors() - consolidate_dataframe() - find_nearest_value() - horizontal_distance() - find_valid_ras_folders() - is_valid_ras_folder() - safe_write_geometry() # Phase 2.1 - Atomic file write with backup - rollback_geometry() # Phase 2.1 - Restore from backup - validate_geometry_file_basic() # Phase 2.1 - Basic validation - backup_files() # Move files to timestamped Backup folder (safe deletion) - \_read_description_block() # Internal - Read BEGIN DESCRIPTION / END DESCRIPTION block - \_write_description_block() # Internal - Write BEGIN DESCRIPTION / END DESCRIPTION block #### RasUtils A class containing utility functions for the ras-commander library. When integrating new functions that do not clearly fit into other classes, add them here. Source code in `ras_commander/RasUtils.py` ```python class RasUtils: """ A class containing utility functions for the ras-commander library. When integrating new functions that do not clearly fit into other classes, add them here. """ @staticmethod @log_call def create_directory(directory_path: Path, ras_object=None) -> Path: """ Ensure that a directory exists, creating it if necessary. Parameters: directory_path (Path): Path to the directory ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Path to the ensured directory Example: >>> ensured_dir = RasUtils.create_directory(Path("output")) >>> print(f"Directory ensured: {ensured_dir}") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(directory_path) try: path.mkdir(parents=True, exist_ok=True) logger.debug(f"Directory ensured: {path}") except Exception as e: logger.error(f"Failed to create directory {path}: {e}") raise return path @staticmethod def safe_resolve(path: Path) -> Path: """ Resolve path while preserving Windows drive letters. On Windows with mapped network drives, Path.resolve() converts drive letters (H:\\) to UNC paths (\\\\server\\share). HEC-RAS cannot read from UNC paths, so we preserve the drive letter format. This function: - On non-Windows: Uses standard resolve() - On Windows with local drives: Uses standard resolve() - On Windows with mapped drives: Falls back to absolute() to preserve drive letter Parameters: path (Path): Path to resolve Returns: Path: Resolved path with drive letter preserved if applicable Example: >>> from pathlib import Path >>> from ras_commander import RasUtils >>> # Local drive - normal resolution >>> resolved = RasUtils.safe_resolve(Path("C:/Projects/Model.prj")) >>> # Mapped drive - preserves H: instead of converting to UNC >>> resolved = RasUtils.safe_resolve(Path("H:/Projects/Model.prj")) """ # Ensure we have a Path object path = Path(path) # On non-Windows, use standard resolve if os.name != 'nt': return path.resolve() original_str = str(path) resolved = path.resolve() # Check if original had drive letter but resolved became UNC path # Drive letter format: "X:..." where X is a letter # UNC format: "\\..." (starts with double backslash) has_drive_letter = len(original_str) >= 2 and original_str[1] == ':' is_unc = str(resolved).startswith('\\\\') if has_drive_letter and is_unc: # Mapped network drive detected - use absolute() to preserve drive letter logger.debug( f"Mapped drive detected: {original_str} would resolve to UNC {resolved}. " f"Using absolute() to preserve drive letter." ) return path.absolute() return resolved # Windows reserved device names (case-insensitive, without extensions) _WINDOWS_RESERVED_NAMES = frozenset({ 'CON', 'PRN', 'AUX', 'NUL', 'COM1', 'COM2', 'COM3', 'COM4', 'COM5', 'COM6', 'COM7', 'COM8', 'COM9', 'LPT1', 'LPT2', 'LPT3', 'LPT4', 'LPT5', 'LPT6', 'LPT7', 'LPT8', 'LPT9', }) @staticmethod def ignore_windows_reserved( directory: str | Path, contents: list[str], ) -> set[str]: """ Ignore function for shutil.copytree that skips Windows reserved device names. Windows lists virtual device names (NUL, CON, PRN, etc.) in directory listings even though they are not real files. shutil.copytree fails when it tries to copy them. This function filters them out. Parameters: directory: The directory being copied (provided by copytree) contents: List of names in the directory (provided by copytree) Returns: set: Names to ignore (Windows reserved device names) """ ignored = set() for name in contents: stem = Path(name).stem.upper() if stem in RasUtils._WINDOWS_RESERVED_NAMES: logger.debug(f"Skipping Windows reserved name: {name} in {directory}") ignored.add(name) return ignored @staticmethod def is_windows_reserved_name(name: str) -> bool: """ Check if a filename is a Windows reserved device name. Parameters: name: Filename to check Returns: bool: True if the name is a reserved device name """ stem = Path(name).stem.upper() return stem in RasUtils._WINDOWS_RESERVED_NAMES @staticmethod @log_call def find_files_by_extension(extension: str, ras_object=None) -> list: """ List all files in the project directory with a specific extension. Parameters: extension (str): File extension to filter (e.g., '.prj') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: list: List of file paths matching the extension Example: >>> prj_files = RasUtils.find_files_by_extension('.prj') >>> print(f"Found {len(prj_files)} .prj files") """ ras_obj = ras_object or ras ras_obj.check_initialized() try: files = list(ras_obj.project_folder.glob(f"*{extension}")) file_list = [str(file) for file in files] logger.info(f"Found {len(file_list)} files with extension '{extension}' in {ras_obj.project_folder}") return file_list except Exception as e: logger.error(f"Failed to find files with extension '{extension}': {e}") raise @staticmethod @log_call def get_file_size(file_path: Path, ras_object=None) -> Optional[int]: """ Get the size of a file in bytes. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional[int]: Size of the file in bytes, or None if the file does not exist Example: >>> size = RasUtils.get_file_size(Path("project.prj")) >>> print(f"File size: {size} bytes") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(file_path) if path.exists(): try: size = path.stat().st_size logger.debug(f"Size of {path}: {size} bytes") return size except Exception as e: logger.error(f"Failed to get size for {path}: {e}") raise else: logger.warning(f"File not found: {path}") return None @staticmethod @log_call def get_file_modification_time(file_path: Path, ras_object=None) -> Optional[float]: """ Get the last modification time of a file. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional[float]: Last modification time as a timestamp, or None if the file does not exist Example: >>> mtime = RasUtils.get_file_modification_time(Path("project.prj")) >>> print(f"Last modified: {mtime}") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(file_path) if path.exists(): try: mtime = path.stat().st_mtime logger.debug(f"Last modification time of {path}: {mtime}") return mtime except Exception as e: logger.exception(f"Failed to get modification time for {path}") raise else: logger.warning(f"File not found: {path}") return None @staticmethod @log_call def normalize_ras_number(ras_number: Union[str, int, float, Path, Number]) -> str: """ Normalize RAS file numbers to two-digit string format. HEC-RAS uses two-digit file extensions for plans (.p01), geometries (.g02), flows (.f03), etc. This function standardizes various input formats to ensure consistent file path construction. Parameters: ras_number (Union[str, int, float, Path, Number]): Input number in various formats: - int: 1, 2, 3, etc. - str: "1", "01", "001", "p01", ".p01", "project.p01", etc. - float: 1.0, 2.0 (must be whole numbers) - Path: Path("project.p05") - extracts number from extension - Number: numpy.int64(1), etc. Returns: str: Normalized two-digit format ("01", "02", ..., "99") Raises: ValueError: If the number is not between 1 and 99, or cannot be converted TypeError: If the input type is invalid Examples: >>> RasUtils.normalize_ras_number(1) '01' >>> RasUtils.normalize_ras_number("1") '01' >>> RasUtils.normalize_ras_number("01") '01' >>> RasUtils.normalize_ras_number("001") '01' >>> RasUtils.normalize_ras_number("p01") '01' >>> RasUtils.normalize_ras_number(np.int64(5)) '05' >>> RasUtils.normalize_ras_number(Path("project.p02")) '02' Notes: - Used for plan numbers, geometry numbers, flow file numbers, etc. - Ensures consistent handling across all RAS file types - Prevents file path construction errors from unnormalized inputs """ # Handle Path objects - extract number from file extension if isinstance(ras_number, Path): # Extract from extensions like .p01, .g02, .f03, etc. suffix = ras_number.suffix # e.g., ".p01" if len(suffix) >= 2 and suffix[0] == '.': # Try to extract number after the letter (e.g., "01" from ".p01") number_part = suffix[2:] # Skip "." and letter if number_part.isdigit(): ras_number = number_part else: raise ValueError( f"Cannot extract RAS number from Path extension: {ras_number}. " f"Expected format like 'project.p01' or 'geom.g02'" ) else: raise ValueError( f"Cannot extract RAS number from Path: {ras_number}. " f"Expected file with RAS extension like .p01, .g02, etc." ) # Convert to integer for validation try: # Handle string inputs including bare prefixed forms ("p01") and # filename/path strings ("project.p01"). if isinstance(ras_number, str): text = ras_number.strip() path_suffix = Path(text).suffix if ( len(path_suffix) >= 3 and path_suffix[0] == "." and path_suffix[1].isalpha() and path_suffix[2:].isdigit() ): text = path_suffix[2:] elif ( len(text) >= 2 and text[0] == "." and text[1].isalpha() and text[2:].isdigit() ): text = text[2:] elif len(text) >= 2 and text[0].isalpha() and text[1:].isdigit(): text = text[1:] stripped = text.lstrip('0') if not stripped or not stripped.isdigit(): # Handle edge cases like "0", "00", or non-numeric strings if not stripped: # Was all zeros ras_int = 0 else: raise ValueError(f"Cannot convert '{ras_number}' to integer") else: ras_int = int(stripped) else: # Handle numeric types (int, float, numpy types, etc.) ras_int = int(ras_number) # Check if float had decimal component if isinstance(ras_number, (float, np.floating)) and ras_number != ras_int: raise ValueError( f"RAS numbers must be integers, got float with decimals: {ras_number}" ) except (ValueError, TypeError) as e: raise ValueError( f"Cannot convert RAS number '{ras_number}' (type: {type(ras_number).__name__}) " f"to integer: {e}" ) from e # Validate range (1-99 for HEC-RAS files) if not 1 <= ras_int <= 99: raise ValueError( f"RAS file number must be between 1 and 99, got: {ras_int}. " f"See: https://rascommander.info/user-guide/plan-execution/" ) # Return normalized two-digit format normalized = f"{ras_int:02d}" logger.debug(f"Normalized RAS number '{ras_number}' to '{normalized}'") return normalized @staticmethod @log_call def get_plan_path(current_plan_number_or_path: Union[str, Number, Path], ras_object=None) -> Path: """ Get the path for a plan file with a given plan number or path. Parameters: current_plan_number_or_path (Union[str, Number, Path]): The plan number (e.g., '01', 1, or 1.0) or full path to the plan file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Full path to the plan file Raises: ValueError: If plan number is not between 1 and 99 TypeError: If input type is invalid FileNotFoundError: If the plan file does not exist Example: >>> plan_path = RasUtils.get_plan_path(1) >>> print(f"Plan file path: {plan_path}") >>> plan_path = RasUtils.get_plan_path("01") >>> print(f"Plan file path: {plan_path}") >>> plan_path = RasUtils.get_plan_path("path/to/plan.p01") >>> print(f"Plan file path: {plan_path}") """ # Validate RAS object ras_obj = ras_object or ras ras_obj.check_initialized() # Handle direct file path input plan_path = Path(current_plan_number_or_path) if plan_path.is_file(): logger.debug(f"Using provided plan file path: {plan_path}") return plan_path # Handle plan number input - use centralized normalization try: current_plan_number = RasUtils.normalize_ras_number(current_plan_number_or_path) logger.debug(f"Normalized plan number to: {current_plan_number}") except (ValueError, TypeError) as e: logger.error(f"Invalid plan number: {current_plan_number_or_path}. {e}") raise # Construct and validate plan path plan_name = f"{ras_obj.project_name}.p{current_plan_number}" full_plan_path = ras_obj.project_folder / plan_name if not full_plan_path.exists(): logger.error(f"Plan file does not exist: {full_plan_path}") raise FileNotFoundError(f"Plan file does not exist: {full_plan_path}") logger.debug(f"Constructed plan file path: {full_plan_path}") return full_plan_path @staticmethod @log_call def remove_with_retry( path: Path, max_attempts: int = 5, initial_delay: float = 1.0, is_folder: bool = True, ras_object=None ) -> bool: """ Attempts to remove a file or folder with retry logic and exponential backoff. Parameters: path (Path): Path to the file or folder to be removed. max_attempts (int): Maximum number of removal attempts. initial_delay (float): Initial delay between attempts in seconds. is_folder (bool): If True, the path is treated as a folder; if False, it's treated as a file. ras_object (RasPrj, optional): Accepted for backward compatibility. The cleanup does not require an initialized RAS project, so it can be used before project extraction or during worker-folder cleanup. Returns: bool: True if the file or folder was successfully removed, False otherwise. Example: >>> success = RasUtils.remove_with_retry(Path("temp_folder"), is_folder=True) >>> print(f"Removal successful: {success}") """ path = Path(path) for attempt in range(1, max_attempts + 1): try: if path.exists(): if is_folder: shutil.rmtree(path) logger.debug(f"Folder removed: {path}") else: path.unlink() logger.debug(f"File removed: {path}") else: logger.debug(f"Path does not exist, nothing to remove: {path}") return True except PermissionError as pe: if attempt < max_attempts: delay = initial_delay * (2 ** (attempt - 1)) # Exponential backoff logger.warning( f"PermissionError on attempt {attempt} to remove {path}: {pe}. " f"Retrying in {delay} seconds..." ) time.sleep(delay) else: logger.error( f"Failed to remove {path} after {max_attempts} attempts due to PermissionError: {pe}. Skipping." ) return False except Exception as e: logger.exception(f"Failed to remove {path} on attempt {attempt}") return False return False @staticmethod @log_call def update_plan_file( plan_number_or_path: Union[str, Path], file_type: str, entry_number: int, ras_object=None ) -> None: """ Update a plan file with a new file reference. Parameters: plan_number_or_path (Union[str, Path]): The plan number (1 to 99) or full path to the plan file file_type (str): Type of file to update ('Geom', 'Flow', or 'Unsteady') entry_number (int): Number (from 1 to 99) to set ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Raises: ValueError: If an invalid file_type is provided FileNotFoundError: If the plan file doesn't exist Example: >>> RasUtils.update_plan_file(1, "Geom", 2) >>> RasUtils.update_plan_file("path/to/plan.p01", "Geom", 2) """ ras_obj = ras_object or ras ras_obj.check_initialized() valid_file_types = {'Geom': 'g', 'Flow': 'f', 'Unsteady': 'u'} if file_type not in valid_file_types: logger.error( f"Invalid file_type '{file_type}'. Expected one of: {', '.join(valid_file_types.keys())}" ) raise ValueError( f"Invalid file_type. Expected one of: {', '.join(valid_file_types.keys())}" ) plan_file_path = Path(plan_number_or_path) if not plan_file_path.is_file(): plan_file_path = RasUtils.get_plan_path(plan_number_or_path, ras_object) if not plan_file_path.exists(): logger.error(f"Plan file not found: {plan_file_path}") raise FileNotFoundError(f"Plan file not found: {plan_file_path}") file_prefix = valid_file_types[file_type] search_pattern = f"{file_type} File=" formatted_entry_number = f"{int(entry_number):02d}" # Ensure two-digit format try: RasUtils.check_file_access(plan_file_path, 'r') with plan_file_path.open('r') as file: lines = file.readlines() except Exception as e: logger.exception(f"Failed to read plan file {plan_file_path}") raise updated = False for i, line in enumerate(lines): if line.startswith(search_pattern): lines[i] = f"{search_pattern}{file_prefix}{formatted_entry_number}\n" logger.info( f"Updated {file_type} File in {plan_file_path} to {file_prefix}{formatted_entry_number}" ) updated = True break if not updated: logger.warning( f"Search pattern '{search_pattern}' not found in {plan_file_path}. No update performed." ) try: with plan_file_path.open('w') as file: file.writelines(lines) logger.info(f"Successfully updated plan file: {plan_file_path}") except Exception as e: logger.exception(f"Failed to write updates to plan file {plan_file_path}") raise # Refresh RasPrj dataframes try: ras_obj.plan_df = ras_obj.get_plan_entries() ras_obj.geom_df = ras_obj.get_geom_entries() ras_obj.flow_df = ras_obj.get_flow_entries() ras_obj.unsteady_df = ras_obj.get_unsteady_entries() logger.debug("RAS object dataframes have been refreshed.") except Exception as e: logger.exception("Failed to refresh RasPrj dataframes") raise @staticmethod @log_call def check_file_access(file_path: Path, mode: str = 'r') -> None: """ Check if the file can be accessed with the specified mode. Parameters: file_path (Path): Path to the file mode (str): Mode to check ('r' for read, 'w' for write, etc.) Raises: FileNotFoundError: If the file does not exist PermissionError: If the required permissions are not met """ path = Path(file_path) if not path.exists(): logger.error(f"File not found: {file_path}") raise FileNotFoundError(f"File not found: {file_path}") if mode in ('r', 'rb'): if not os.access(path, os.R_OK): logger.error(f"Read permission denied for file: {file_path}") raise PermissionError(f"Read permission denied for file: {file_path}") else: logger.debug(f"Read access granted for file: {file_path}") if mode in ('w', 'wb', 'a', 'ab'): parent_dir = path.parent if not os.access(parent_dir, os.W_OK): logger.error(f"Write permission denied for directory: {parent_dir}") raise PermissionError(f"Write permission denied for directory: {parent_dir}") else: logger.debug(f"Write access granted for directory: {parent_dir}") @staticmethod @log_call def convert_to_dataframe( data_source: Union[pd.DataFrame, Path], **kwargs: Any ) -> pd.DataFrame: """ Converts input to a pandas DataFrame. Supports existing DataFrames or file paths (CSV, Excel, TSV, Parquet). Args: data_source (Union[pd.DataFrame, Path]): The input to convert to a DataFrame. Can be a file path or an existing DataFrame. **kwargs: Additional keyword arguments to pass to pandas read functions. Returns: pd.DataFrame: The resulting DataFrame. Raises: NotImplementedError: If the file type is unsupported or input type is invalid. Example: >>> df = RasUtils.convert_to_dataframe(Path("data.csv")) >>> print(type(df)) """ if isinstance(data_source, pd.DataFrame): logger.debug("Input is already a DataFrame, returning a copy.") return data_source.copy() elif isinstance(data_source, Path): ext = data_source.suffix.replace('.', '', 1) logger.debug(f"Converting file with extension '{ext}' to DataFrame.") if ext == 'csv': return pd.read_csv(data_source, **kwargs) elif ext.startswith('x'): return pd.read_excel(data_source, **kwargs) elif ext == "tsv": return pd.read_csv(data_source, sep="\t", **kwargs) elif ext in ["parquet", "pq", "parq"]: return pd.read_parquet(data_source, **kwargs) else: logger.error(f"Unsupported file type: {ext}") raise NotImplementedError(f"Unsupported file type {ext}. Should be one of csv, tsv, parquet, or xlsx.") else: logger.error(f"Unsupported input type: {type(data_source)}") raise NotImplementedError(f"Unsupported type {type(data_source)}. Only file path / existing DataFrame supported at this time") @staticmethod @log_call def save_to_excel( dataframe: pd.DataFrame, excel_path: Path, **kwargs: Any ) -> None: """ Saves a pandas DataFrame to an Excel file with retry functionality. Args: dataframe (pd.DataFrame): The DataFrame to save. excel_path (Path): The path to the Excel file where the DataFrame will be saved. **kwargs: Additional keyword arguments passed to `DataFrame.to_excel()`. Raises: IOError: If the file cannot be saved after multiple attempts. Example: >>> df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}) >>> RasUtils.save_to_excel(df, Path('output.xlsx')) """ saved = False max_attempts = 3 attempt = 0 while not saved and attempt < max_attempts: try: dataframe.to_excel(excel_path, **kwargs) logger.info(f'DataFrame successfully saved to {excel_path}') saved = True except IOError as e: attempt += 1 if attempt < max_attempts: logger.warning(f"Error saving file. Attempt {attempt} of {max_attempts}. Please close the Excel document if it's open.") else: logger.error(f"Failed to save {excel_path} after {max_attempts} attempts.") raise IOError(f"Failed to save {excel_path} after {max_attempts} attempts. Last error: {str(e)}") @staticmethod @log_call def calculate_rmse(observed_values: np.ndarray, predicted_values: np.ndarray, normalized: bool = True) -> float: """ Calculate the Root Mean Squared Error (RMSE) between observed and predicted values. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. normalized (bool, optional): Whether to normalize RMSE to a percentage of observed_values. Defaults to True. Returns: float: The calculated RMSE value. Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_rmse(observed, predicted) 0.06396394 """ rmse = np.sqrt(np.mean((predicted_values - observed_values) ** 2)) if normalized: rmse = rmse / np.abs(np.mean(observed_values)) logger.debug(f"Calculated RMSE: {rmse}") return rmse @staticmethod @log_call def calculate_percent_bias(observed_values: np.ndarray, predicted_values: np.ndarray, as_percentage: bool = False) -> float: """ Calculate the Percent Bias between observed and predicted values. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. as_percentage (bool, optional): If True, return bias as a percentage. Defaults to False. Returns: float: The calculated Percent Bias. Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_percent_bias(observed, predicted, as_percentage=True) 3.33333333 """ multiplier = 100 if as_percentage else 1 obs_mean = np.mean(observed_values) if obs_mean == 0: logger.warning("Percent bias undefined: mean of observed values is zero") return np.nan percent_bias = multiplier * (np.mean(predicted_values) - obs_mean) / obs_mean logger.debug(f"Calculated Percent Bias: {percent_bias}") return percent_bias @staticmethod @log_call def calculate_error_metrics(observed_values: np.ndarray, predicted_values: np.ndarray) -> Dict[str, float]: """ Compute a trio of error metrics: correlation, RMSE, and Percent Bias. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. Returns: Dict[str, float]: A dictionary containing correlation ('cor'), RMSE ('rmse'), and Percent Bias ('pb'). Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_error_metrics(observed, predicted) {'cor': 0.9993, 'rmse': 0.06396, 'pb': 0.03333} """ correlation = np.corrcoef(observed_values, predicted_values)[0, 1] rmse = RasUtils.calculate_rmse(observed_values, predicted_values) percent_bias = RasUtils.calculate_percent_bias(observed_values, predicted_values) metrics = {'cor': correlation, 'rmse': rmse, 'pb': percent_bias} logger.debug(f"Calculated error metrics: {metrics}") return metrics @staticmethod @log_call def update_file(file_path: Path, update_function: Callable, *args) -> None: """ Generic method to update a file. Parameters: file_path (Path): Path to the file to be updated update_function (Callable): Function to update the file contents *args: Additional arguments to pass to the update_function Raises: Exception: If there's an error updating the file Example: >>> def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines >>> RasUtils.update_file(Path("example.txt"), update_content, "Hello") """ try: with open(file_path, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() updated_lines = update_function(lines, *args) if args else update_function(lines) with open(file_path, 'w', encoding='utf-8', errors='replace') as f: f.writelines(updated_lines) logger.info(f"Successfully updated file: {file_path}") except Exception as e: logger.exception(f"Failed to update file {file_path}") raise @staticmethod @log_call def get_next_number(existing_numbers: list) -> str: """ Determine the next available number from a list of existing numbers. Parameters: existing_numbers (list): List of existing numbers as strings Returns: str: Next available number as a zero-padded string Example: >>> RasUtils.get_next_number(["01", "02", "04"]) "05" """ existing_numbers = sorted(int(num) for num in existing_numbers) next_number = max(existing_numbers, default=0) + 1 return f"{next_number:02d}" @staticmethod @log_call def clone_file(template_path: Path, new_path: Path, update_function: Optional[Callable] = None, *args) -> None: """ Generic method to clone a file and optionally update it. Parameters: template_path (Path): Path to the template file new_path (Path): Path where the new file will be created update_function (Optional[Callable]): Function to update the cloned file *args: Additional arguments to pass to the update_function Raises: FileNotFoundError: If the template file doesn't exist Example: >>> def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines >>> RasUtils.clone_file(Path("template.txt"), Path("new.txt"), update_content, "Hello") """ if not template_path.exists(): logger.error(f"Template file '{template_path}' does not exist.") raise FileNotFoundError(f"Template file '{template_path}' does not exist.") shutil.copy(template_path, new_path) logger.info(f"File cloned from {template_path} to {new_path}") if update_function: RasUtils.update_file(new_path, update_function, *args) @staticmethod @log_call def update_project_file(prj_file: Path, file_type: str, new_num: str, ras_object=None) -> None: """ Update the project file with a new entry. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file being added (e.g., 'Plan', 'Geom') new_num (str): Number of the new file entry ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.update_project_file(Path("project.prj"), "Plan", "02") """ ras_obj = ras_object or ras ras_obj.check_initialized() try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() new_line = f"{file_type} File={file_type[0].lower()}{new_num}\n" lines.append(new_line) with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(lines) logger.info(f"Project file updated with new {file_type} entry: {new_num}") except Exception as e: logger.exception(f"Failed to update project file {prj_file}") raise # NOTE: remove_prj_entry() and rename_prj_entry() are awaiting maintainer review @staticmethod @log_call def remove_prj_entry(prj_file: Path, file_type: str, number: str, ras_object=None) -> None: """ Remove a file entry from the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') number (str): Two-digit number of the entry to remove (e.g., '05') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.remove_prj_entry(Path("project.prj"), "Plan", "05") # Removes the line "Plan File=p05" from the .prj file """ ras_obj = ras_object or ras ras_obj.check_initialized() prefix = file_type[0].lower() target = f"{file_type} File={prefix}{number}" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() new_lines = [line for line in lines if line.strip() != target] if len(new_lines) == len(lines): logger.warning(f"Entry '{target}' not found in {prj_file}") return with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(new_lines) logger.info(f"Removed {file_type} entry {number} from project file") except Exception as e: logger.exception(f"Failed to remove entry from project file {prj_file}") raise @staticmethod @log_call def rename_prj_entry(prj_file: Path, file_type: str, old_number: str, new_number: str, ras_object=None) -> None: """ Rename a file entry in the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') old_number (str): Current two-digit number (e.g., '05') new_number (str): New two-digit number (e.g., '02') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.rename_prj_entry(Path("project.prj"), "Plan", "05", "02") # Changes "Plan File=p05" to "Plan File=p02" in the .prj file """ ras_obj = ras_object or ras ras_obj.check_initialized() prefix = file_type[0].lower() old_line = f"{file_type} File={prefix}{old_number}" new_line_content = f"{file_type} File={prefix}{new_number}" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() found = False for i, line in enumerate(lines): if line.strip() == old_line: lines[i] = new_line_content + '\n' found = True break if not found: logger.warning(f"Entry '{old_line}' not found in {prj_file}") return with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(lines) logger.info(f"Renamed {file_type} entry {old_number} to {new_number} in project file") except Exception as e: logger.exception(f"Failed to rename entry in project file {prj_file}") raise @staticmethod @log_call def backup_files( files: List[Union[Path, str]], project_folder: Union[Path, str], operation_label: str = "deleted", ) -> Optional[Path]: """ Move files to a timestamped Backup folder inside the project. Creates {project_folder}/Backup/{YYYY-MM-DD_HHMMSS}_{operation_label}/ and moves each existing file into that folder. Non-existent files are silently skipped. Parameters: files (List[Union[Path, str]]): File paths to back up (str or Path). project_folder (Union[Path, str]): Project root where Backup/ will be created. operation_label (str): Label appended to timestamp folder name (e.g., "deleted_p05"). Returns: Optional[Path]: Path to backup folder if any files were moved, None otherwise. Example: >>> files = [Path("Muncie.p05"), Path("Muncie.p05.hdf")] >>> backup_dir = RasUtils.backup_files(files, project_folder, "deleted_p05") """ files = [Path(f) for f in files] existing_files = [f for f in files if f.exists()] if not existing_files: return None timestamp = datetime.datetime.now().strftime("%Y-%m-%d_%H%M%S") backup_folder = Path(project_folder) / "Backup" / f"{timestamp}_{operation_label}" backup_folder.mkdir(parents=True, exist_ok=True) for f in existing_files: shutil.move(str(f), str(backup_folder / f.name)) logger.info(f"Backed up {f.name} to {backup_folder}") return backup_folder # From FunkShuns @staticmethod @log_call def decode_byte_strings(dataframe: pd.DataFrame) -> pd.DataFrame: """ Decodes byte strings in a DataFrame to regular string objects. This function converts columns with byte-encoded strings (e.g., b'string') into UTF-8 decoded strings. Args: dataframe (pd.DataFrame): The DataFrame containing byte-encoded string columns. Returns: pd.DataFrame: The DataFrame with byte strings decoded to regular strings. Example: >>> df = pd.DataFrame({'A': [b'hello', b'world'], 'B': [1, 2]}) >>> decoded_df = RasUtils.decode_byte_strings(df) >>> print(decoded_df) A B 0 hello 1 1 world 2 """ str_df = dataframe.select_dtypes(['object']) str_df = str_df.stack().str.decode('utf-8').unstack() for col in str_df: dataframe[col] = str_df[col] return dataframe @staticmethod @log_call def perform_kdtree_query( reference_points: np.ndarray, query_points: np.ndarray, max_distance: float = 2.0 ) -> np.ndarray: """ Performs a KDTree query between two datasets and returns indices with distances exceeding max_distance set to -1. Args: reference_points (np.ndarray): The reference dataset for KDTree. query_points (np.ndarray): The query dataset to search against KDTree of reference_points. max_distance (float, optional): The maximum distance threshold. Indices with distances greater than this are set to -1. Defaults to 2.0. Returns: np.ndarray: Array of indices from reference_points that are nearest to each point in query_points. Indices with distances > max_distance are set to -1. Example: >>> ref_points = np.array([[0, 0], [1, 1], [2, 2]]) >>> query_points = np.array([[0.5, 0.5], [3, 3]]) >>> result = RasUtils.perform_kdtree_query(ref_points, query_points) >>> print(result) array([ 0, -1]) """ dist, snap = KDTree(reference_points).query(query_points, distance_upper_bound=max_distance) snap[dist > max_distance] = -1 return snap @staticmethod @log_call def find_nearest_neighbors(points: np.ndarray, max_distance: float = 2.0) -> np.ndarray: """ Creates a self KDTree for dataset points and finds nearest neighbors excluding self, with distances above max_distance set to -1. Args: points (np.ndarray): The dataset to build the KDTree from and query against itself. max_distance (float, optional): The maximum distance threshold. Indices with distances greater than max_distance are set to -1. Defaults to 2.0. Returns: np.ndarray: Array of indices representing the nearest neighbor in points for each point in points. Indices with distances > max_distance or self-matches are set to -1. Example: >>> points = np.array([[0, 0], [1, 1], [2, 2], [10, 10]]) >>> result = RasUtils.find_nearest_neighbors(points) >>> print(result) array([1, 0, 1, -1]) """ dist, snap = KDTree(points).query(points, k=2, distance_upper_bound=max_distance) snap[dist > max_distance] = -1 snp = pd.DataFrame(snap, index=np.arange(len(snap))) snp = snp.replace(-1, np.nan) snp.loc[snp[0] == snp.index, 0] = np.nan snp.loc[snp[1] == snp.index, 1] = np.nan filled = snp[0].fillna(snp[1]) snapped = filled.fillna(-1).astype(np.int64).to_numpy() return snapped @staticmethod @log_call def consolidate_dataframe( dataframe: pd.DataFrame, group_by: Optional[Union[str, List[str]]] = None, pivot_columns: Optional[Union[str, List[str]]] = None, level: Optional[int] = None, n_dimensional: bool = False, aggregation_method: Union[str, Callable] = 'list' ) -> pd.DataFrame: """ Consolidate rows in a DataFrame by merging duplicate values into lists or using a specified aggregation function. Args: dataframe (pd.DataFrame): The DataFrame to consolidate. group_by (Optional[Union[str, List[str]]]): Columns or indices to group by. pivot_columns (Optional[Union[str, List[str]]]): Columns to pivot. level (Optional[int]): Level of multi-index to group by. n_dimensional (bool): If True, use a pivot table for N-Dimensional consolidation. aggregation_method (Union[str, Callable]): Aggregation method, e.g., 'list' to aggregate into lists. Returns: pd.DataFrame: The consolidated DataFrame. Example: >>> df = pd.DataFrame({'A': [1, 1, 2], 'B': [4, 5, 6], 'C': [7, 8, 9]}) >>> result = RasUtils.consolidate_dataframe(df, group_by='A') >>> print(result) B C A 1 [4, 5] [7, 8] 2 [6] [9] """ if aggregation_method == 'list': agg_func = lambda x: tuple(x) else: agg_func = aggregation_method if n_dimensional: result = dataframe.pivot_table(group_by, pivot_columns, aggfunc=agg_func) else: result = dataframe.groupby(group_by, level=level).agg(agg_func).map(list) return result @staticmethod @log_call def find_nearest_value(array: Union[list, np.ndarray], target_value: Union[int, float]) -> Union[int, float]: """ Finds the nearest value in a NumPy array to the specified target value. Args: array (Union[list, np.ndarray]): The array to search within. target_value (Union[int, float]): The value to find the nearest neighbor to. Returns: Union[int, float]: The nearest value in the array to the specified target value. Example: >>> arr = np.array([1, 3, 5, 7, 9]) >>> result = RasUtils.find_nearest_value(arr, 6) >>> print(result) 5 """ array = np.asarray(array) idx = (np.abs(array - target_value)).argmin() return array[idx] @staticmethod @log_call def horizontal_distance(coord1: np.ndarray, coord2: np.ndarray) -> float: """ Calculate the horizontal distance between two coordinate points. Args: coord1 (np.ndarray): First coordinate point [X, Y]. coord2 (np.ndarray): Second coordinate point [X, Y]. Returns: float: Horizontal distance. Example: >>> distance = RasUtils.horizontal_distance(np.array([0, 0]), np.array([3, 4])) >>> print(distance) 5.0 """ return np.linalg.norm(coord2 - coord1) @staticmethod def find_valid_ras_folders( search_path: Union[str, Path], max_depth: Optional[int] = None, return_project_info: bool = False ) -> Union[List[Path], List[Dict[str, Any]]]: """ Recursively search for valid HEC-RAS project folders. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line (HEC-RAS project file) 2. At least one .pXX file where XX is 01-99 (plan files) This function does NOT require the global ras object to be initialized, making it suitable for discovery operations before project initialization. Args: search_path (Union[str, Path]): Root directory to search for HEC-RAS projects. max_depth (Optional[int]): Maximum folder depth to search. None means unlimited. Depth 0 = search_path only, 1 = immediate subdirectories, etc. return_project_info (bool): If True, return list of dicts with folder path, project name, prj file path, and plan count. If False, return list of Paths. Returns: Union[List[Path], List[Dict[str, Any]]]: - If return_project_info=False: List of Path objects for valid HEC-RAS folders - If return_project_info=True: List of dicts with keys: - 'folder': Path to the project folder - 'project_name': Name extracted from .prj filename - 'prj_file': Path to the .prj file - 'plan_count': Number of plan files found - 'plan_numbers': List of plan numbers (e.g., ['01', '02', '15']) Example: >>> # Find all valid HEC-RAS project folders >>> folders = RasUtils.find_valid_ras_folders("C:/Projects/Hydrology") >>> for folder in folders: ... print(f"Found project: {folder}") >>> # Get detailed info about each project >>> projects = RasUtils.find_valid_ras_folders( ... "C:/Projects", ... max_depth=3, ... return_project_info=True ... ) >>> for proj in projects: ... print(f"{proj['project_name']}: {proj['plan_count']} plans") Note: This function distinguishes HEC-RAS .prj files from ESRI projection files by checking for "Proj Title=" on the first line of the file. """ search_path = Path(search_path) if not search_path.exists(): logger.warning(f"Search path does not exist: {search_path}") return [] if not search_path.is_dir(): logger.warning(f"Search path is not a directory: {search_path}") return [] valid_folders = [] def is_valid_ras_prj(prj_file: Path) -> bool: """Check if a .prj file is a valid HEC-RAS project file.""" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: first_line = f.readline() return first_line.strip().startswith("Proj Title=") except Exception as e: logger.debug(f"Could not read .prj file {prj_file}: {e}") return False def get_plan_files(folder: Path) -> List[Tuple[str, Path]]: """Get all valid plan files (.p01 to .p99) in a folder.""" plan_files = [] for i in range(1, 100): plan_num = f"{i:02d}" # Look for files matching *.pXX pattern for pfile in folder.glob(f"*.p{plan_num}"): plan_files.append((plan_num, pfile)) return plan_files def check_folder(folder: Path) -> Optional[Dict[str, Any]]: """Check if a folder is a valid HEC-RAS project folder.""" # Find .prj files prj_files = list(folder.glob("*.prj")) if not prj_files: return None # Find valid HEC-RAS .prj file (not ESRI projection file) valid_prj = None for prj_file in prj_files: if is_valid_ras_prj(prj_file): valid_prj = prj_file break if valid_prj is None: return None # Check for plan files plan_files = get_plan_files(folder) if not plan_files: return None # This is a valid HEC-RAS project folder return { 'folder': folder, 'project_name': valid_prj.stem, 'prj_file': valid_prj, 'plan_count': len(plan_files), 'plan_numbers': [pn for pn, _ in plan_files] } def scan_directory(current_path: Path, current_depth: int): """Recursively scan directories for HEC-RAS projects.""" # Check if we've exceeded max depth if max_depth is not None and current_depth > max_depth: return # Check current folder result = check_folder(current_path) if result: valid_folders.append(result) # Don't search subdirectories of a valid project folder # (nested projects are uncommon and would cause confusion) return # Scan subdirectories try: for item in current_path.iterdir(): if item.is_dir() and not item.name.startswith('.'): scan_directory(item, current_depth + 1) except PermissionError: logger.debug(f"Permission denied accessing: {current_path}") except Exception as e: logger.debug(f"Error scanning {current_path}: {e}") # Start scanning logger.info(f"Searching for HEC-RAS projects in: {search_path}") scan_directory(search_path, 0) logger.info(f"Found {len(valid_folders)} valid HEC-RAS project folders") if return_project_info: return valid_folders else: return [info['folder'] for info in valid_folders] @staticmethod def is_valid_ras_folder(folder_path: Union[str, Path]) -> bool: """ Check if a single folder is a valid HEC-RAS project folder. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line 2. At least one .pXX file where XX is 01-99 This function does NOT require the global ras object to be initialized. Args: folder_path (Union[str, Path]): Path to the folder to check. Returns: bool: True if the folder is a valid HEC-RAS project folder. Example: >>> if RasUtils.is_valid_ras_folder("C:/Projects/MyRASModel"): ... print("This is a valid HEC-RAS project folder") ... else: ... print("Not a valid HEC-RAS project folder") """ folder_path = Path(folder_path) if not folder_path.exists() or not folder_path.is_dir(): return False # Find .prj files prj_files = list(folder_path.glob("*.prj")) if not prj_files: return False # Check if any .prj file is a valid HEC-RAS project file def is_valid_ras_prj(prj_file: Path) -> bool: try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: first_line = f.readline() return first_line.strip().startswith("Proj Title=") except Exception: return False has_valid_prj = any(is_valid_ras_prj(pf) for pf in prj_files) if not has_valid_prj: return False # Check for at least one plan file (.p01 to .p99) for i in range(1, 100): plan_num = f"{i:02d}" if list(folder_path.glob(f"*.p{plan_num}")): return True return False # ============================================================================= # Atomic File Write Infrastructure (Phase 2.1 - HTAB Modification) # ============================================================================= @staticmethod @log_call def safe_write_geometry( geom_file: Union[str, Path], modified_lines: List[str], create_backup: bool = True ) -> Optional[Path]: """ Atomically write geometry file with backup for safe file modification. This function implements safe file modification for HEC-RAS geometry files, ensuring data integrity through atomic operations and optional backup creation. Process: 1. Create timestamped backup: geom_file.YYYYMMDD_HHMMSS.bak 2. Write to temp file: geom_file.tmp 3. Basic validation (line count reasonable, file size reasonable) 4. Atomic rename temp -> original (os.replace) 5. Return backup path Parameters: geom_file (Union[str, Path]): Path to the geometry file to write. modified_lines (List[str]): List of lines to write to the file. Each line should include newline characters if needed. create_backup (bool): If True, create timestamped backup before modification. Defaults to True for safety. Returns: Optional[Path]: Path to backup file if create_backup=True and successful, None if create_backup=False or file didn't exist before. Raises: FileNotFoundError: If the geometry file doesn't exist (for modification). PermissionError: If write access is denied to the file or directory. ValueError: If modified_lines is empty or validation fails. IOError: If atomic rename fails. Example: >>> from ras_commander import RasUtils >>> from pathlib import Path >>> >>> # Read geometry file >>> geom_file = Path("project/geometry.g01") >>> with open(geom_file, 'r') as f: ... lines = f.readlines() >>> >>> # Modify HTAB parameters (example) >>> modified_lines = modify_htab_params(lines, starting_el=580.0) >>> >>> # Safe write with backup >>> backup_path = RasUtils.safe_write_geometry(geom_file, modified_lines) >>> print(f"Backup created at: {backup_path}") Notes: - This function uses os.replace() for atomic rename, which is atomic on both Windows (NTFS) and Unix filesystems. - Backup files use format: filename.YYYYMMDD_HHMMSS.bak - If validation fails, temp file is deleted and original remains unchanged. - For rollback, use rollback_geometry() with the returned backup path. See Also: - rollback_geometry: Restore from backup after failed modification - .claude/rules/python/path-handling.md: Path handling patterns """ geom_file = Path(geom_file) backup_path = None temp_path = None # Validate inputs if not modified_lines: raise ValueError("modified_lines cannot be empty") # Verify original file exists (we're modifying, not creating) if not geom_file.exists(): raise FileNotFoundError(f"Geometry file not found: {geom_file}") # Check write permissions if not os.access(geom_file.parent, os.W_OK): raise PermissionError(f"Write permission denied for directory: {geom_file.parent}") try: # Read original file for validation comparison original_size = geom_file.stat().st_size with open(geom_file, 'r', encoding='utf-8', errors='replace') as f: original_line_count = sum(1 for _ in f) # Step 1: Create timestamped backup if create_backup: timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") backup_path = geom_file.parent / f"{geom_file.name}.{timestamp}.bak" # Copy original to backup shutil.copy2(geom_file, backup_path) logger.info(f"Backup created: {backup_path}") # Step 2: Write to temp file temp_path = geom_file.parent / f"{geom_file.name}.tmp" with open(temp_path, 'w', encoding='utf-8', newline='') as f: f.writelines(modified_lines) logger.debug(f"Temp file written: {temp_path}") # Step 3: Basic validation temp_size = temp_path.stat().st_size new_line_count = len(modified_lines) # Validation: File shouldn't be empty if temp_size == 0: raise ValueError("Modified file would be empty - validation failed") # Validation: Line count shouldn't change drastically (>50% reduction suspicious) if new_line_count < original_line_count * 0.5: raise ValueError( f"Line count reduced drastically ({original_line_count} -> {new_line_count}). " f"This may indicate data corruption. Aborting." ) # Validation: File size shouldn't shrink too much (>80% reduction suspicious) if temp_size < original_size * 0.2: raise ValueError( f"File size reduced drastically ({original_size} -> {temp_size} bytes). " f"This may indicate data corruption. Aborting." ) logger.debug( f"Validation passed: {new_line_count} lines, {temp_size} bytes " f"(original: {original_line_count} lines, {original_size} bytes)" ) # Step 4: Atomic rename temp -> original # os.replace() is atomic on both Windows (NTFS) and Unix os.replace(temp_path, geom_file) temp_path = None # Mark as successfully moved logger.info(f"Geometry file atomically updated: {geom_file}") return backup_path except Exception as e: # Clean up temp file if it exists if temp_path and temp_path.exists(): try: temp_path.unlink() logger.debug(f"Cleaned up temp file: {temp_path}") except Exception as cleanup_error: logger.warning(f"Failed to clean up temp file {temp_path}: {cleanup_error}") logger.error(f"Failed to write geometry file {geom_file}: {e}") raise @staticmethod @log_call def rollback_geometry( geom_file: Union[str, Path], backup_path: Union[str, Path] ) -> None: """ Restore geometry file from backup after failed modification. This function restores a geometry file from a previously created backup, typically used when a modification operation fails or produces incorrect results. Process: 1. Verify backup file exists 2. Copy backup -> original (preserves backup for safety) 3. Log restoration Parameters: geom_file (Union[str, Path]): Path to the geometry file to restore. backup_path (Union[str, Path]): Path to the backup file created by safe_write_geometry(). Returns: None Raises: FileNotFoundError: If backup file doesn't exist. PermissionError: If write access is denied. IOError: If copy operation fails. Example: >>> from ras_commander import RasUtils >>> from pathlib import Path >>> >>> # Attempt modification >>> try: ... backup = RasUtils.safe_write_geometry(geom_file, modified_lines) ... # Run HEC-RAS to validate ... RasCmdr.compute_plan("01", clear_geompre=True) ... except Exception as e: ... # Modification failed - rollback ... if backup: ... RasUtils.rollback_geometry(geom_file, backup) ... print("Geometry file restored from backup") ... raise Notes: - This function copies the backup to original, preserving the backup. - Use shutil.copy2() to preserve file metadata (timestamps, permissions). - After successful rollback, you may want to delete the backup manually if no longer needed. See Also: - safe_write_geometry: Create backup and safely write modifications """ geom_file = Path(geom_file) backup_path = Path(backup_path) # Verify backup exists if not backup_path.exists(): raise FileNotFoundError(f"Backup file not found: {backup_path}") # Check write permissions if geom_file.exists() and not os.access(geom_file, os.W_OK): raise PermissionError(f"Write permission denied for file: {geom_file}") if not os.access(geom_file.parent, os.W_OK): raise PermissionError(f"Write permission denied for directory: {geom_file.parent}") try: # Copy backup to original (preserves backup for safety) shutil.copy2(backup_path, geom_file) logger.info(f"Geometry file restored from backup: {geom_file} <- {backup_path}") except Exception as e: logger.error(f"Failed to restore geometry file {geom_file} from {backup_path}: {e}") raise @staticmethod @log_call def validate_geometry_file_basic( geom_file: Union[str, Path], min_lines: int = 10, required_patterns: Optional[List[str]] = None ) -> bool: """ Perform basic validation on a geometry file. This function checks that a geometry file meets basic structural requirements, useful for pre-modification validation or post-write verification. Parameters: geom_file (Union[str, Path]): Path to the geometry file to validate. min_lines (int): Minimum number of lines expected. Defaults to 10. required_patterns (Optional[List[str]]): List of strings that must appear somewhere in the file. Defaults to ["River Reach="] for HEC-RAS geometry. Returns: bool: True if validation passes, False otherwise. Example: >>> if RasUtils.validate_geometry_file_basic(geom_file): ... print("Geometry file appears valid") >>> >>> # Custom validation >>> if RasUtils.validate_geometry_file_basic( ... geom_file, ... required_patterns=["River Reach=", "Type RM Length"] ... ): ... print("Geometry file has cross sections") Notes: - This is a basic structural check, not a full HEC-RAS validation. - For comprehensive validation, use HEC-RAS geometric preprocessor. """ geom_file = Path(geom_file) if required_patterns is None: # Default: Check for River Reach definition (present in most geometry files) required_patterns = ["River Reach="] if not geom_file.exists(): logger.warning(f"Geometry file does not exist: {geom_file}") return False try: with open(geom_file, 'r', encoding='utf-8', errors='replace') as f: content = f.read() lines = content.splitlines() # Check minimum line count if len(lines) < min_lines: logger.warning( f"Geometry file has too few lines: {len(lines)} < {min_lines}" ) return False # Check required patterns for pattern in required_patterns: if pattern not in content: logger.warning(f"Required pattern not found in geometry file: {pattern}") return False logger.debug(f"Geometry file validation passed: {geom_file}") return True except Exception as e: logger.error(f"Error validating geometry file {geom_file}: {e}") return False @staticmethod def _read_description_block(file_path: Union[str, Path]) -> str: """ Read the BEGIN DESCRIPTION / END DESCRIPTION block from any HEC-RAS text file. HEC-RAS uses the same description block format in plan (.p##), geometry (.g##), unsteady (.u##), and steady flow (.f##) files. Parameters: file_path (Union[str, Path]): Path to the HEC-RAS text file. Returns: str: The description text, or empty string if no description block found. """ file_path = Path(file_path) if not file_path.exists(): logger.warning(f"File not found for description read: {file_path}") return "" try: with open(file_path, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() except IOError as e: logger.error(f"Error reading file {file_path}: {e}") return "" description_lines = [] in_description = False for line in lines: stripped_upper = line.strip().upper() if stripped_upper in ('BEGIN DESCRIPTION:', 'BEGIN DESCRIPTION'): in_description = True elif stripped_upper in ('END DESCRIPTION:', 'END DESCRIPTION'): break elif in_description: description_lines.append(line.strip()) return '\n'.join(description_lines) @staticmethod def _write_description_block( file_path: Union[str, Path], description: str, title_keyword: str ) -> bool: """ Write a BEGIN DESCRIPTION / END DESCRIPTION block into any HEC-RAS text file. If an existing description block is found, it is replaced in place. If no description block exists, a new one is inserted after the title and Program Version lines. Parameters: file_path (Union[str, Path]): Path to the HEC-RAS text file. description (str): Description text to write. title_keyword (str): The title keyword for this file type, e.g. 'Plan Title', 'Geom Title', 'Flow Title'. Used to determine insertion point when no existing description block is found. Returns: bool: True if successful, False otherwise. """ file_path = Path(file_path) if not file_path.exists(): logger.error(f"File not found for description write: {file_path}") return False try: with open(file_path, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() except IOError as e: logger.error(f"Error reading file {file_path}: {e}") return False # Find existing description block desc_start_idx = None desc_end_idx = None for i, line in enumerate(lines): stripped_upper = line.strip().upper() if stripped_upper.startswith('BEGIN DESCRIPTION'): desc_start_idx = i elif stripped_upper.startswith('END DESCRIPTION'): desc_end_idx = i break # Prepare the new description block description_clean = description.rstrip() description_block = [ 'BEGIN DESCRIPTION:\n', description_clean + '\n', 'END DESCRIPTION:\n' ] if desc_start_idx is not None and desc_end_idx is not None: # Replace existing description block in place new_lines = lines[:desc_start_idx] + description_block + lines[desc_end_idx + 1:] else: # Find insertion point: after title_keyword= and Program Version= lines last_header_idx = 0 for i, line in enumerate(lines): stripped = line.strip() if stripped.startswith(f'{title_keyword}=') or stripped.startswith('Program Version='): last_header_idx = max(last_header_idx, i) insertion_idx = last_header_idx + 1 new_lines = lines[:insertion_idx] + description_block + lines[insertion_idx:] try: with open(file_path, 'w', encoding='utf-8') as f: f.writelines(new_lines) logger.info(f"Updated description in {file_path}") return True except IOError as e: logger.error(f"Error writing description to {file_path}: {e}") return False @staticmethod @log_call def dos2unix(project_dir: Union[str, Path], extensions: Optional[List[str]] = None) -> int: """ Convert CRLF line endings to LF in HEC-RAS text files. Processes .b## and .g## files by default (boundary and geometry text files that need LF endings for Linux HEC-RAS execution). Done in-place using pure Python (no shell dependency). Attribution: Implementation pattern derived from ras-agent (https://github.com/gheistand/ras-agent) by Glenn Heistand / CHAMP — Illinois State Water Survey. See runner.py:_dos2unix_dir(). Parameters: project_dir (Union[str, Path]): Path to the HEC-RAS project directory. extensions (Optional[List[str]]): Custom regex patterns for file extensions to process. Defaults to [r'\\.(b|g)\\d+$'] which matches .b01, .g01, etc. Returns: int: Number of files modified. Example: >>> from ras_commander import RasUtils >>> count = RasUtils.dos2unix(Path("/project/dir")) >>> print(f"Converted {count} files") """ import re project_dir = Path(project_dir) if not project_dir.is_dir(): raise FileNotFoundError(f"Directory not found: {project_dir}") if extensions is None: patterns = [re.compile(r'\.(b|g)\d+$', re.IGNORECASE)] else: patterns = [re.compile(ext, re.IGNORECASE) for ext in extensions] modified_count = 0 for fpath in project_dir.iterdir(): if not fpath.is_file(): continue if not any(p.search(fpath.name) for p in patterns): continue try: raw = fpath.read_bytes() if b'\r' in raw: fpath.write_bytes(raw.replace(b'\r\n', b'\n').replace(b'\r', b'\n')) modified_count += 1 logger.debug(f"dos2unix: {fpath.name}") except (OSError, PermissionError) as exc: logger.warning(f"dos2unix skipped {fpath.name}: {exc}") logger.info(f"dos2unix: converted {modified_count} files in {project_dir}") return modified_count @staticmethod def _scan_native_linux_ras(roots) -> Dict[str, Path]: """Scan native-Linux HEC-RAS install roots for RasUnsteady solver binaries. A native Linux install has no Ras.exe; the executable is the RasUnsteady solver (under ``bin_ras/`` for some 5.0.x layouts). Returns a mapping of ``{version-folder-name: Path(RasUnsteady)}``. Platform-agnostic so it is directly unit-testable (CLB-883). """ found: Dict[str, Path] = {} for root in roots: root = Path(root) if not root.exists(): continue for child in sorted(root.iterdir()): if not child.is_dir(): continue exe = None for binname in ("RasUnsteady", "rasUnsteady", "rasUnsteady64"): for cand in (child / binname, child / "bin_ras" / binname): if cand.is_file(): exe = cand break if exe is not None: break if exe is not None: found.setdefault(child.name, exe) return found @staticmethod @log_call def discover_ras_versions() -> Dict[str, Path]: """ Discover installed HEC-RAS versions by scanning Windows Registry, filesystem, and Wine prefixes (on Linux). Resolution order: 1. Windows Registry (HKLM, WOW6432Node, HKCU) -- Windows only 2. Standard filesystem paths (Program Files) -- Windows only 3. Native Linux installs (/opt/hecras/, /opt/HEC-RAS/, ~/hecras/, or $RAS_COMMANDER_LINUX_RAS_ROOT) -- Linux only 4. Wine prefix paths (~/.wine, /opt/hecras-wine, etc.) -- Linux only Returns: Dict[str, Path]: Mapping of version string -> Path to the executable. On Windows/Wine this is ``Ras.exe``; for native Linux installs there is no Ras.exe, so it maps to the ``RasUnsteady`` solver binary (use ``.parent`` as ``ras_exe_dir`` for ``RasCmdr.compute_plan_linux``). Example: {"6.6": Path("C:/Program Files (x86)/HEC/HEC-RAS/6.6/Ras.exe")} """ discovered: Dict[str, Path] = {} # Version folder names matching RasPrj.get_ras_exe() ras_version_folders = [ "7.0", "6.7 Beta 5", "6.7 Beta 4", "6.6", "6.5", "6.4.1", "6.3.1", "6.3", "6.2", "6.1", "6.0", "5.0.7", "5.0.6", "5.0.5", "5.0.4", "5.0.3", "5.0.1", "5.0", "4.1.0", "4.0" ] version_aliases = { "4.1": "4.1.0", "41": "4.1.0", "410": "4.1.0", "40": "4.0", "50": "5.0", "501": "5.0.1", "503": "5.0.3", "504": "5.0.4", "505": "5.0.5", "506": "5.0.6", "507": "5.0.7", "60": "6.0", "61": "6.1", "62": "6.2", "63": "6.3", "631": "6.3.1", "6.4": "6.4.1", "64": "6.4.1", "641": "6.4.1", "65": "6.5", "66": "6.6", "6.7": "6.7 Beta 5", "67": "6.7 Beta 5", "70": "7.0", } def _normalize_version(raw: str, install_dir: Optional[Path] = None) -> str: v = str(raw).strip() if v in version_aliases: return version_aliases[v] if install_dir is not None: fn = install_dir.name.strip() if fn in version_aliases: return version_aliases[fn] if fn in ras_version_folders: return fn return v def _add(version: str, exe_path: Path, source: str) -> None: if version in discovered: logger.debug(f"Skipping duplicate HEC-RAS {version} from {source}") return discovered[version] = exe_path logger.info(f"Discovered HEC-RAS {version} at {exe_path} via {source}") def _scan_root(root_dir: Path, source_label: str) -> None: """Scan a directory containing versioned HEC-RAS subfolders.""" if not root_dir.exists(): return # Check known folder names first for folder_name in ras_version_folders: exe = root_dir / folder_name / "Ras.exe" if exe.is_file(): v = _normalize_version(folder_name, exe.parent) _add(v, exe, source_label) # Glob for any other folders with Ras.exe try: for exe in sorted(root_dir.glob("*/Ras.exe")): v = _normalize_version(exe.parent.name, exe.parent) _add(v, exe, source_label) except OSError as exc: logger.warning(f"Filesystem scan failed for {root_dir}: {exc}") # --- Windows: Registry + Program Files --- if os.name == 'nt': # Registry scan try: import winreg def _is_no_more(exc: OSError) -> bool: return getattr(exc, "winerror", None) == 259 hive_map = { "HKLM": winreg.HKEY_LOCAL_MACHINE, "HKCU": winreg.HKEY_CURRENT_USER, } registry_locations = [ ("HKLM", r"SOFTWARE\HEC\HEC-RAS"), ("HKLM", r"SOFTWARE\WOW6432Node\HEC\HEC-RAS"), ("HKCU", r"SOFTWARE\HEC\HEC-RAS"), ] install_value_names = ( "InstallDir", "InstallPath", "Install Path", "Path", "ExePath", "RasExePath", ) for hive_name, subkey_path in registry_locations: try: with winreg.OpenKey(hive_map[hive_name], subkey_path) as root_key: idx = 0 while True: try: vk_name = winreg.EnumKey(root_key, idx) except OSError as exc: if _is_no_more(exc): break break idx += 1 try: with winreg.OpenKey(root_key, vk_name) as vk: install_val = None for val_name in install_value_names: try: val, _ = winreg.QueryValueEx(vk, val_name) if val: install_val = str(val) break except (FileNotFoundError, OSError): continue if install_val: p = Path(os.path.expandvars(install_val.strip().strip('"'))) if p.suffix.lower() != '.exe': p = p / "Ras.exe" if p.name.lower() == "ras.exe" and p.is_file(): v = _normalize_version(vk_name, p.parent) _add(v, p, f"registry {hive_name}\\{subkey_path}") except (FileNotFoundError, OSError): continue except (FileNotFoundError, OSError): continue except ImportError: logger.debug("winreg not available, skipping registry scan") # Filesystem scan (standard Windows paths) _scan_root(Path("C:/Program Files (x86)/HEC/HEC-RAS"), "filesystem (x86)") _scan_root(Path("C:/Program Files/HEC/HEC-RAS"), "filesystem") # --- Linux: native install scan --- else: # Native Linux HEC-RAS installs have no Ras.exe; the RasUnsteady # solver binary is the executable. Maps version -> RasUnsteady path # (callers for compute_plan_linux use ``.parent`` as ras_exe_dir). # Roots are configurable via $RAS_COMMANDER_LINUX_RAS_ROOT (CLB-883). linux_native_roots = [ Path(os.path.expanduser("~/hecras")), Path("/opt/hecras"), Path("/opt/HEC-RAS"), ] env_root = os.environ.get("RAS_COMMANDER_LINUX_RAS_ROOT") if env_root: linux_native_roots.insert(0, Path(env_root)) for _folder, _exe in RasUtils._scan_native_linux_ras(linux_native_roots).items(): _add(_normalize_version(_folder, _exe.parent), _exe, "linux native") # --- Linux: Wine prefix scan --- wine_prefix_candidates = [ Path(os.path.expanduser("~/.wine")), Path("/opt/hecras-wine"), Path(os.path.expanduser("~/hecras-wine")), ] # Also check WINEPREFIX env var env_prefix = os.environ.get("WINEPREFIX") if env_prefix: wine_prefix_candidates.insert(0, Path(env_prefix)) for prefix in wine_prefix_candidates: drive_c = prefix / "drive_c" if not drive_c.exists(): continue logger.debug(f"Scanning Wine prefix: {prefix}") # Standard HEC-RAS locations under drive_c _scan_root(drive_c / "Program Files (x86)" / "HEC" / "HEC-RAS", f"wine {prefix}") _scan_root(drive_c / "Program Files" / "HEC" / "HEC-RAS", f"wine {prefix}") _scan_root(drive_c / "HEC-RAS", f"wine {prefix}") logger.info(f"Discovered {len(discovered)} installed HEC-RAS version(s)") return discovered ``` ##### create_directory Python ```python create_directory(directory_path: Path, ras_object=None) -> Path ``` Ensure that a directory exists, creating it if necessary. Parameters: directory_path (Path): Path to the directory ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Path to the ensured directory Example: > > > ensured_dir = RasUtils.create_directory(Path("output")) print(f"Directory ensured: {ensured_dir}") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def create_directory(directory_path: Path, ras_object=None) -> Path: """ Ensure that a directory exists, creating it if necessary. Parameters: directory_path (Path): Path to the directory ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Path to the ensured directory Example: >>> ensured_dir = RasUtils.create_directory(Path("output")) >>> print(f"Directory ensured: {ensured_dir}") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(directory_path) try: path.mkdir(parents=True, exist_ok=True) logger.debug(f"Directory ensured: {path}") except Exception as e: logger.error(f"Failed to create directory {path}: {e}") raise return path ``` ##### safe_resolve Python ```python safe_resolve(path: Path) -> Path ``` Resolve path while preserving Windows drive letters. On Windows with mapped network drives, Path.resolve() converts drive letters (H:) to UNC paths (\\server\\share). HEC-RAS cannot read from UNC paths, so we preserve the drive letter format. This function: - On non-Windows: Uses standard resolve() - On Windows with local drives: Uses standard resolve() - On Windows with mapped drives: Falls back to absolute() to preserve drive letter Parameters: | Name | Type | Description | Default | | ------ | ------ | --------------- | ---------- | | `path` | `Path` | Path to resolve | *required* | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------------------- | | `Path` | `Path` | Resolved path with drive letter preserved if applicable | Example > > > from pathlib import Path from ras_commander import RasUtils > > > > > > ###### Local drive - normal resolution > > > > > > resolved = RasUtils.safe_resolve(Path("C:/Projects/Model.prj")) > > > > > > ###### Mapped drive - preserves H: instead of converting to UNC > > > > > > resolved = RasUtils.safe_resolve(Path("H:/Projects/Model.prj")) Source code in `ras_commander/RasUtils.py` ```python @staticmethod def safe_resolve(path: Path) -> Path: """ Resolve path while preserving Windows drive letters. On Windows with mapped network drives, Path.resolve() converts drive letters (H:\\) to UNC paths (\\\\server\\share). HEC-RAS cannot read from UNC paths, so we preserve the drive letter format. This function: - On non-Windows: Uses standard resolve() - On Windows with local drives: Uses standard resolve() - On Windows with mapped drives: Falls back to absolute() to preserve drive letter Parameters: path (Path): Path to resolve Returns: Path: Resolved path with drive letter preserved if applicable Example: >>> from pathlib import Path >>> from ras_commander import RasUtils >>> # Local drive - normal resolution >>> resolved = RasUtils.safe_resolve(Path("C:/Projects/Model.prj")) >>> # Mapped drive - preserves H: instead of converting to UNC >>> resolved = RasUtils.safe_resolve(Path("H:/Projects/Model.prj")) """ # Ensure we have a Path object path = Path(path) # On non-Windows, use standard resolve if os.name != 'nt': return path.resolve() original_str = str(path) resolved = path.resolve() # Check if original had drive letter but resolved became UNC path # Drive letter format: "X:..." where X is a letter # UNC format: "\\..." (starts with double backslash) has_drive_letter = len(original_str) >= 2 and original_str[1] == ':' is_unc = str(resolved).startswith('\\\\') if has_drive_letter and is_unc: # Mapped network drive detected - use absolute() to preserve drive letter logger.debug( f"Mapped drive detected: {original_str} would resolve to UNC {resolved}. " f"Using absolute() to preserve drive letter." ) return path.absolute() return resolved ``` ##### ignore_windows_reserved Python ```python ignore_windows_reserved(directory: str | Path, contents: list[str]) -> set[str] ``` Ignore function for shutil.copytree that skips Windows reserved device names. Windows lists virtual device names (NUL, CON, PRN, etc.) in directory listings even though they are not real files. shutil.copytree fails when it tries to copy them. This function filters them out. Parameters: | Name | Type | Description | Default | | ----------- | ----------- | ----------------------------------------------------- | ------------------------------------------------- | | `directory` | \`str | Path\` | The directory being copied (provided by copytree) | | `contents` | `list[str]` | List of names in the directory (provided by copytree) | *required* | Returns: | Name | Type | Description | | ----- | ---------- | ----------------------------------------------- | | `set` | `set[str]` | Names to ignore (Windows reserved device names) | Source code in `ras_commander/RasUtils.py` ```python @staticmethod def ignore_windows_reserved( directory: str | Path, contents: list[str], ) -> set[str]: """ Ignore function for shutil.copytree that skips Windows reserved device names. Windows lists virtual device names (NUL, CON, PRN, etc.) in directory listings even though they are not real files. shutil.copytree fails when it tries to copy them. This function filters them out. Parameters: directory: The directory being copied (provided by copytree) contents: List of names in the directory (provided by copytree) Returns: set: Names to ignore (Windows reserved device names) """ ignored = set() for name in contents: stem = Path(name).stem.upper() if stem in RasUtils._WINDOWS_RESERVED_NAMES: logger.debug(f"Skipping Windows reserved name: {name} in {directory}") ignored.add(name) return ignored ``` ##### is_windows_reserved_name Python ```python is_windows_reserved_name(name: str) -> bool ``` Check if a filename is a Windows reserved device name. Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------------- | ---------- | | `name` | `str` | Filename to check | *required* | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------ | | `bool` | `bool` | True if the name is a reserved device name | Source code in `ras_commander/RasUtils.py` ```python @staticmethod def is_windows_reserved_name(name: str) -> bool: """ Check if a filename is a Windows reserved device name. Parameters: name: Filename to check Returns: bool: True if the name is a reserved device name """ stem = Path(name).stem.upper() return stem in RasUtils._WINDOWS_RESERVED_NAMES ``` ##### find_files_by_extension Python ```python find_files_by_extension(extension: str, ras_object=None) -> list ``` List all files in the project directory with a specific extension. Parameters: extension (str): File extension to filter (e.g., '.prj') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: list: List of file paths matching the extension Example: > > > prj_files = RasUtils.find_files_by_extension('.prj') print(f"Found {len(prj_files)} .prj files") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def find_files_by_extension(extension: str, ras_object=None) -> list: """ List all files in the project directory with a specific extension. Parameters: extension (str): File extension to filter (e.g., '.prj') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: list: List of file paths matching the extension Example: >>> prj_files = RasUtils.find_files_by_extension('.prj') >>> print(f"Found {len(prj_files)} .prj files") """ ras_obj = ras_object or ras ras_obj.check_initialized() try: files = list(ras_obj.project_folder.glob(f"*{extension}")) file_list = [str(file) for file in files] logger.info(f"Found {len(file_list)} files with extension '{extension}' in {ras_obj.project_folder}") return file_list except Exception as e: logger.error(f"Failed to find files with extension '{extension}': {e}") raise ``` ##### get_file_size Python ```python get_file_size(file_path: Path, ras_object=None) -> Optional[int] ``` Get the size of a file in bytes. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional\[int\]: Size of the file in bytes, or None if the file does not exist Example: > > > size = RasUtils.get_file_size(Path("project.prj")) print(f"File size: {size} bytes") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def get_file_size(file_path: Path, ras_object=None) -> Optional[int]: """ Get the size of a file in bytes. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional[int]: Size of the file in bytes, or None if the file does not exist Example: >>> size = RasUtils.get_file_size(Path("project.prj")) >>> print(f"File size: {size} bytes") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(file_path) if path.exists(): try: size = path.stat().st_size logger.debug(f"Size of {path}: {size} bytes") return size except Exception as e: logger.error(f"Failed to get size for {path}: {e}") raise else: logger.warning(f"File not found: {path}") return None ``` ##### get_file_modification_time Python ```python get_file_modification_time(file_path: Path, ras_object=None) -> Optional[float] ``` Get the last modification time of a file. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional\[float\]: Last modification time as a timestamp, or None if the file does not exist Example: > > > mtime = RasUtils.get_file_modification_time(Path("project.prj")) print(f"Last modified: {mtime}") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def get_file_modification_time(file_path: Path, ras_object=None) -> Optional[float]: """ Get the last modification time of a file. Parameters: file_path (Path): Path to the file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Optional[float]: Last modification time as a timestamp, or None if the file does not exist Example: >>> mtime = RasUtils.get_file_modification_time(Path("project.prj")) >>> print(f"Last modified: {mtime}") """ ras_obj = ras_object or ras ras_obj.check_initialized() path = Path(file_path) if path.exists(): try: mtime = path.stat().st_mtime logger.debug(f"Last modification time of {path}: {mtime}") return mtime except Exception as e: logger.exception(f"Failed to get modification time for {path}") raise else: logger.warning(f"File not found: {path}") return None ``` ##### normalize_ras_number Python ```python normalize_ras_number(ras_number: Union[str, int, float, Path, Number]) -> str ``` Normalize RAS file numbers to two-digit string format. HEC-RAS uses two-digit file extensions for plans (.p01), geometries (.g02), flows (.f03), etc. This function standardizes various input formats to ensure consistent file path construction. ras_number (Union[str, int, float, Path, Number]): Input number in various formats: - int: 1, 2, 3, etc. - str: "1", "01", "001", "p01", ".p01", "project.p01", etc. - float: 1.0, 2.0 (must be whole numbers) - Path: Path("project.p05") - extracts number from extension - Number: numpy.int64(1), etc. Returns: str: Normalized two-digit format ("01", "02", ..., "99") Raises: ValueError: If the number is not between 1 and 99, or cannot be converted TypeError: If the input type is invalid Examples: > > > RasUtils.normalize_ras_number(1) '01' RasUtils.normalize_ras_number("1") '01' RasUtils.normalize_ras_number("01") '01' RasUtils.normalize_ras_number("001") '01' RasUtils.normalize_ras_number("p01") '01' RasUtils.normalize_ras_number(np.int64(5)) '05' RasUtils.normalize_ras_number(Path("project.p02")) '02' Notes: - Used for plan numbers, geometry numbers, flow file numbers, etc. - Ensures consistent handling across all RAS file types - Prevents file path construction errors from unnormalized inputs Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def normalize_ras_number(ras_number: Union[str, int, float, Path, Number]) -> str: """ Normalize RAS file numbers to two-digit string format. HEC-RAS uses two-digit file extensions for plans (.p01), geometries (.g02), flows (.f03), etc. This function standardizes various input formats to ensure consistent file path construction. Parameters: ras_number (Union[str, int, float, Path, Number]): Input number in various formats: - int: 1, 2, 3, etc. - str: "1", "01", "001", "p01", ".p01", "project.p01", etc. - float: 1.0, 2.0 (must be whole numbers) - Path: Path("project.p05") - extracts number from extension - Number: numpy.int64(1), etc. Returns: str: Normalized two-digit format ("01", "02", ..., "99") Raises: ValueError: If the number is not between 1 and 99, or cannot be converted TypeError: If the input type is invalid Examples: >>> RasUtils.normalize_ras_number(1) '01' >>> RasUtils.normalize_ras_number("1") '01' >>> RasUtils.normalize_ras_number("01") '01' >>> RasUtils.normalize_ras_number("001") '01' >>> RasUtils.normalize_ras_number("p01") '01' >>> RasUtils.normalize_ras_number(np.int64(5)) '05' >>> RasUtils.normalize_ras_number(Path("project.p02")) '02' Notes: - Used for plan numbers, geometry numbers, flow file numbers, etc. - Ensures consistent handling across all RAS file types - Prevents file path construction errors from unnormalized inputs """ # Handle Path objects - extract number from file extension if isinstance(ras_number, Path): # Extract from extensions like .p01, .g02, .f03, etc. suffix = ras_number.suffix # e.g., ".p01" if len(suffix) >= 2 and suffix[0] == '.': # Try to extract number after the letter (e.g., "01" from ".p01") number_part = suffix[2:] # Skip "." and letter if number_part.isdigit(): ras_number = number_part else: raise ValueError( f"Cannot extract RAS number from Path extension: {ras_number}. " f"Expected format like 'project.p01' or 'geom.g02'" ) else: raise ValueError( f"Cannot extract RAS number from Path: {ras_number}. " f"Expected file with RAS extension like .p01, .g02, etc." ) # Convert to integer for validation try: # Handle string inputs including bare prefixed forms ("p01") and # filename/path strings ("project.p01"). if isinstance(ras_number, str): text = ras_number.strip() path_suffix = Path(text).suffix if ( len(path_suffix) >= 3 and path_suffix[0] == "." and path_suffix[1].isalpha() and path_suffix[2:].isdigit() ): text = path_suffix[2:] elif ( len(text) >= 2 and text[0] == "." and text[1].isalpha() and text[2:].isdigit() ): text = text[2:] elif len(text) >= 2 and text[0].isalpha() and text[1:].isdigit(): text = text[1:] stripped = text.lstrip('0') if not stripped or not stripped.isdigit(): # Handle edge cases like "0", "00", or non-numeric strings if not stripped: # Was all zeros ras_int = 0 else: raise ValueError(f"Cannot convert '{ras_number}' to integer") else: ras_int = int(stripped) else: # Handle numeric types (int, float, numpy types, etc.) ras_int = int(ras_number) # Check if float had decimal component if isinstance(ras_number, (float, np.floating)) and ras_number != ras_int: raise ValueError( f"RAS numbers must be integers, got float with decimals: {ras_number}" ) except (ValueError, TypeError) as e: raise ValueError( f"Cannot convert RAS number '{ras_number}' (type: {type(ras_number).__name__}) " f"to integer: {e}" ) from e # Validate range (1-99 for HEC-RAS files) if not 1 <= ras_int <= 99: raise ValueError( f"RAS file number must be between 1 and 99, got: {ras_int}. " f"See: https://rascommander.info/user-guide/plan-execution/" ) # Return normalized two-digit format normalized = f"{ras_int:02d}" logger.debug(f"Normalized RAS number '{ras_number}' to '{normalized}'") return normalized ``` ##### get_plan_path Python ```python get_plan_path(current_plan_number_or_path: Union[str, Number, Path], ras_object=None) -> Path ``` Get the path for a plan file with a given plan number or path. Parameters: current_plan_number_or_path (Union[str, Number, Path]): The plan number (e.g., '01', 1, or 1.0) or full path to the plan file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Full path to the plan file Raises: ValueError: If plan number is not between 1 and 99 TypeError: If input type is invalid FileNotFoundError: If the plan file does not exist Example: > > > plan_path = RasUtils.get_plan_path(1) print(f"Plan file path: {plan_path}") plan_path = RasUtils.get_plan_path("01") print(f"Plan file path: {plan_path}") plan_path = RasUtils.get_plan_path("path/to/plan.p01") print(f"Plan file path: {plan_path}") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def get_plan_path(current_plan_number_or_path: Union[str, Number, Path], ras_object=None) -> Path: """ Get the path for a plan file with a given plan number or path. Parameters: current_plan_number_or_path (Union[str, Number, Path]): The plan number (e.g., '01', 1, or 1.0) or full path to the plan file ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Returns: Path: Full path to the plan file Raises: ValueError: If plan number is not between 1 and 99 TypeError: If input type is invalid FileNotFoundError: If the plan file does not exist Example: >>> plan_path = RasUtils.get_plan_path(1) >>> print(f"Plan file path: {plan_path}") >>> plan_path = RasUtils.get_plan_path("01") >>> print(f"Plan file path: {plan_path}") >>> plan_path = RasUtils.get_plan_path("path/to/plan.p01") >>> print(f"Plan file path: {plan_path}") """ # Validate RAS object ras_obj = ras_object or ras ras_obj.check_initialized() # Handle direct file path input plan_path = Path(current_plan_number_or_path) if plan_path.is_file(): logger.debug(f"Using provided plan file path: {plan_path}") return plan_path # Handle plan number input - use centralized normalization try: current_plan_number = RasUtils.normalize_ras_number(current_plan_number_or_path) logger.debug(f"Normalized plan number to: {current_plan_number}") except (ValueError, TypeError) as e: logger.error(f"Invalid plan number: {current_plan_number_or_path}. {e}") raise # Construct and validate plan path plan_name = f"{ras_obj.project_name}.p{current_plan_number}" full_plan_path = ras_obj.project_folder / plan_name if not full_plan_path.exists(): logger.error(f"Plan file does not exist: {full_plan_path}") raise FileNotFoundError(f"Plan file does not exist: {full_plan_path}") logger.debug(f"Constructed plan file path: {full_plan_path}") return full_plan_path ``` ##### remove_with_retry Python ```python remove_with_retry(path: Path, max_attempts: int = 5, initial_delay: float = 1.0, is_folder: bool = True, ras_object=None) -> bool ``` Attempts to remove a file or folder with retry logic and exponential backoff. Parameters: path (Path): Path to the file or folder to be removed. max_attempts (int): Maximum number of removal attempts. initial_delay (float): Initial delay between attempts in seconds. is_folder (bool): If True, the path is treated as a folder; if False, it's treated as a file. ras_object (RasPrj, optional): Accepted for backward compatibility. The cleanup does not require an initialized RAS project, so it can be used before project extraction or during worker-folder cleanup. Returns: bool: True if the file or folder was successfully removed, False otherwise. Example: > > > success = RasUtils.remove_with_retry(Path("temp_folder"), is_folder=True) print(f"Removal successful: {success}") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def remove_with_retry( path: Path, max_attempts: int = 5, initial_delay: float = 1.0, is_folder: bool = True, ras_object=None ) -> bool: """ Attempts to remove a file or folder with retry logic and exponential backoff. Parameters: path (Path): Path to the file or folder to be removed. max_attempts (int): Maximum number of removal attempts. initial_delay (float): Initial delay between attempts in seconds. is_folder (bool): If True, the path is treated as a folder; if False, it's treated as a file. ras_object (RasPrj, optional): Accepted for backward compatibility. The cleanup does not require an initialized RAS project, so it can be used before project extraction or during worker-folder cleanup. Returns: bool: True if the file or folder was successfully removed, False otherwise. Example: >>> success = RasUtils.remove_with_retry(Path("temp_folder"), is_folder=True) >>> print(f"Removal successful: {success}") """ path = Path(path) for attempt in range(1, max_attempts + 1): try: if path.exists(): if is_folder: shutil.rmtree(path) logger.debug(f"Folder removed: {path}") else: path.unlink() logger.debug(f"File removed: {path}") else: logger.debug(f"Path does not exist, nothing to remove: {path}") return True except PermissionError as pe: if attempt < max_attempts: delay = initial_delay * (2 ** (attempt - 1)) # Exponential backoff logger.warning( f"PermissionError on attempt {attempt} to remove {path}: {pe}. " f"Retrying in {delay} seconds..." ) time.sleep(delay) else: logger.error( f"Failed to remove {path} after {max_attempts} attempts due to PermissionError: {pe}. Skipping." ) return False except Exception as e: logger.exception(f"Failed to remove {path} on attempt {attempt}") return False return False ``` ##### update_plan_file Python ```python update_plan_file(plan_number_or_path: Union[str, Path], file_type: str, entry_number: int, ras_object=None) -> None ``` Update a plan file with a new file reference. Parameters: plan_number_or_path (Union[str, Path]): The plan number (1 to 99) or full path to the plan file file_type (str): Type of file to update ('Geom', 'Flow', or 'Unsteady') entry_number (int): Number (from 1 to 99) to set ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Raises: ValueError: If an invalid file_type is provided FileNotFoundError: If the plan file doesn't exist Example: > > > RasUtils.update_plan_file(1, "Geom", 2) RasUtils.update_plan_file("path/to/plan.p01", "Geom", 2) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def update_plan_file( plan_number_or_path: Union[str, Path], file_type: str, entry_number: int, ras_object=None ) -> None: """ Update a plan file with a new file reference. Parameters: plan_number_or_path (Union[str, Path]): The plan number (1 to 99) or full path to the plan file file_type (str): Type of file to update ('Geom', 'Flow', or 'Unsteady') entry_number (int): Number (from 1 to 99) to set ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Raises: ValueError: If an invalid file_type is provided FileNotFoundError: If the plan file doesn't exist Example: >>> RasUtils.update_plan_file(1, "Geom", 2) >>> RasUtils.update_plan_file("path/to/plan.p01", "Geom", 2) """ ras_obj = ras_object or ras ras_obj.check_initialized() valid_file_types = {'Geom': 'g', 'Flow': 'f', 'Unsteady': 'u'} if file_type not in valid_file_types: logger.error( f"Invalid file_type '{file_type}'. Expected one of: {', '.join(valid_file_types.keys())}" ) raise ValueError( f"Invalid file_type. Expected one of: {', '.join(valid_file_types.keys())}" ) plan_file_path = Path(plan_number_or_path) if not plan_file_path.is_file(): plan_file_path = RasUtils.get_plan_path(plan_number_or_path, ras_object) if not plan_file_path.exists(): logger.error(f"Plan file not found: {plan_file_path}") raise FileNotFoundError(f"Plan file not found: {plan_file_path}") file_prefix = valid_file_types[file_type] search_pattern = f"{file_type} File=" formatted_entry_number = f"{int(entry_number):02d}" # Ensure two-digit format try: RasUtils.check_file_access(plan_file_path, 'r') with plan_file_path.open('r') as file: lines = file.readlines() except Exception as e: logger.exception(f"Failed to read plan file {plan_file_path}") raise updated = False for i, line in enumerate(lines): if line.startswith(search_pattern): lines[i] = f"{search_pattern}{file_prefix}{formatted_entry_number}\n" logger.info( f"Updated {file_type} File in {plan_file_path} to {file_prefix}{formatted_entry_number}" ) updated = True break if not updated: logger.warning( f"Search pattern '{search_pattern}' not found in {plan_file_path}. No update performed." ) try: with plan_file_path.open('w') as file: file.writelines(lines) logger.info(f"Successfully updated plan file: {plan_file_path}") except Exception as e: logger.exception(f"Failed to write updates to plan file {plan_file_path}") raise # Refresh RasPrj dataframes try: ras_obj.plan_df = ras_obj.get_plan_entries() ras_obj.geom_df = ras_obj.get_geom_entries() ras_obj.flow_df = ras_obj.get_flow_entries() ras_obj.unsteady_df = ras_obj.get_unsteady_entries() logger.debug("RAS object dataframes have been refreshed.") except Exception as e: logger.exception("Failed to refresh RasPrj dataframes") raise ``` ##### check_file_access Python ```python check_file_access(file_path: Path, mode: str = 'r') -> None ``` Check if the file can be accessed with the specified mode. Parameters: file_path (Path): Path to the file mode (str): Mode to check ('r' for read, 'w' for write, etc.) Raises: FileNotFoundError: If the file does not exist PermissionError: If the required permissions are not met Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def check_file_access(file_path: Path, mode: str = 'r') -> None: """ Check if the file can be accessed with the specified mode. Parameters: file_path (Path): Path to the file mode (str): Mode to check ('r' for read, 'w' for write, etc.) Raises: FileNotFoundError: If the file does not exist PermissionError: If the required permissions are not met """ path = Path(file_path) if not path.exists(): logger.error(f"File not found: {file_path}") raise FileNotFoundError(f"File not found: {file_path}") if mode in ('r', 'rb'): if not os.access(path, os.R_OK): logger.error(f"Read permission denied for file: {file_path}") raise PermissionError(f"Read permission denied for file: {file_path}") else: logger.debug(f"Read access granted for file: {file_path}") if mode in ('w', 'wb', 'a', 'ab'): parent_dir = path.parent if not os.access(parent_dir, os.W_OK): logger.error(f"Write permission denied for directory: {parent_dir}") raise PermissionError(f"Write permission denied for directory: {parent_dir}") else: logger.debug(f"Write access granted for directory: {parent_dir}") ``` ##### convert_to_dataframe Python ```python convert_to_dataframe(data_source: Union[DataFrame, Path], **kwargs: Any) -> pd.DataFrame ``` Converts input to a pandas DataFrame. Supports existing DataFrames or file paths (CSV, Excel, TSV, Parquet). Parameters: | Name | Type | Description | Default | | ------------- | ------------------------ | --------------------------------------------------------------------------------- | ---------- | | `data_source` | `Union[DataFrame, Path]` | The input to convert to a DataFrame. Can be a file path or an existing DataFrame. | *required* | | `**kwargs` | `Any` | Additional keyword arguments to pass to pandas read functions. | `{}` | Returns: | Type | Description | | ----------- | -------------------------------------- | | `DataFrame` | pd.DataFrame: The resulting DataFrame. | Raises: | Type | Description | | --------------------- | --------------------------------------------------------- | | `NotImplementedError` | If the file type is unsupported or input type is invalid. | Example > > > df = RasUtils.convert_to_dataframe(Path("data.csv")) print(type(df)) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def convert_to_dataframe( data_source: Union[pd.DataFrame, Path], **kwargs: Any ) -> pd.DataFrame: """ Converts input to a pandas DataFrame. Supports existing DataFrames or file paths (CSV, Excel, TSV, Parquet). Args: data_source (Union[pd.DataFrame, Path]): The input to convert to a DataFrame. Can be a file path or an existing DataFrame. **kwargs: Additional keyword arguments to pass to pandas read functions. Returns: pd.DataFrame: The resulting DataFrame. Raises: NotImplementedError: If the file type is unsupported or input type is invalid. Example: >>> df = RasUtils.convert_to_dataframe(Path("data.csv")) >>> print(type(df)) """ if isinstance(data_source, pd.DataFrame): logger.debug("Input is already a DataFrame, returning a copy.") return data_source.copy() elif isinstance(data_source, Path): ext = data_source.suffix.replace('.', '', 1) logger.debug(f"Converting file with extension '{ext}' to DataFrame.") if ext == 'csv': return pd.read_csv(data_source, **kwargs) elif ext.startswith('x'): return pd.read_excel(data_source, **kwargs) elif ext == "tsv": return pd.read_csv(data_source, sep="\t", **kwargs) elif ext in ["parquet", "pq", "parq"]: return pd.read_parquet(data_source, **kwargs) else: logger.error(f"Unsupported file type: {ext}") raise NotImplementedError(f"Unsupported file type {ext}. Should be one of csv, tsv, parquet, or xlsx.") else: logger.error(f"Unsupported input type: {type(data_source)}") raise NotImplementedError(f"Unsupported type {type(data_source)}. Only file path / existing DataFrame supported at this time") ``` ##### save_to_excel Python ```python save_to_excel(dataframe: DataFrame, excel_path: Path, **kwargs: Any) -> None ``` Saves a pandas DataFrame to an Excel file with retry functionality. Parameters: | Name | Type | Description | Default | | ------------ | ----------- | ------------------------------------------------------------- | ---------- | | `dataframe` | `DataFrame` | The DataFrame to save. | *required* | | `excel_path` | `Path` | The path to the Excel file where the DataFrame will be saved. | *required* | | `**kwargs` | `Any` | Additional keyword arguments passed to DataFrame.to_excel(). | `{}` | Raises: | Type | Description | | --------- | ---------------------------------------------------- | | `IOError` | If the file cannot be saved after multiple attempts. | Example > > > df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}) RasUtils.save_to_excel(df, Path('output.xlsx')) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def save_to_excel( dataframe: pd.DataFrame, excel_path: Path, **kwargs: Any ) -> None: """ Saves a pandas DataFrame to an Excel file with retry functionality. Args: dataframe (pd.DataFrame): The DataFrame to save. excel_path (Path): The path to the Excel file where the DataFrame will be saved. **kwargs: Additional keyword arguments passed to `DataFrame.to_excel()`. Raises: IOError: If the file cannot be saved after multiple attempts. Example: >>> df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}) >>> RasUtils.save_to_excel(df, Path('output.xlsx')) """ saved = False max_attempts = 3 attempt = 0 while not saved and attempt < max_attempts: try: dataframe.to_excel(excel_path, **kwargs) logger.info(f'DataFrame successfully saved to {excel_path}') saved = True except IOError as e: attempt += 1 if attempt < max_attempts: logger.warning(f"Error saving file. Attempt {attempt} of {max_attempts}. Please close the Excel document if it's open.") else: logger.error(f"Failed to save {excel_path} after {max_attempts} attempts.") raise IOError(f"Failed to save {excel_path} after {max_attempts} attempts. Last error: {str(e)}") ``` ##### calculate_rmse Python ```python calculate_rmse(observed_values: ndarray, predicted_values: ndarray, normalized: bool = True) -> float ``` Calculate the Root Mean Squared Error (RMSE) between observed and predicted values. Parameters: | Name | Type | Description | Default | | ------------------ | --------- | ------------------------------------------------------------------------------- | ---------- | | `observed_values` | `ndarray` | Actual observations time series. | *required* | | `predicted_values` | `ndarray` | Estimated/predicted time series. | *required* | | `normalized` | `bool` | Whether to normalize RMSE to a percentage of observed_values. Defaults to True. | `True` | Returns: | Name | Type | Description | | ------- | ------- | -------------------------- | | `float` | `float` | The calculated RMSE value. | Example > > > observed = np.array([1, 2, 3]) predicted = np.array([1.1, 2.2, 2.9]) RasUtils.calculate_rmse(observed, predicted) 0.06396394 Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def calculate_rmse(observed_values: np.ndarray, predicted_values: np.ndarray, normalized: bool = True) -> float: """ Calculate the Root Mean Squared Error (RMSE) between observed and predicted values. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. normalized (bool, optional): Whether to normalize RMSE to a percentage of observed_values. Defaults to True. Returns: float: The calculated RMSE value. Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_rmse(observed, predicted) 0.06396394 """ rmse = np.sqrt(np.mean((predicted_values - observed_values) ** 2)) if normalized: rmse = rmse / np.abs(np.mean(observed_values)) logger.debug(f"Calculated RMSE: {rmse}") return rmse ``` ##### calculate_percent_bias Python ```python calculate_percent_bias(observed_values: ndarray, predicted_values: ndarray, as_percentage: bool = False) -> float ``` Calculate the Percent Bias between observed and predicted values. Parameters: | Name | Type | Description | Default | | ------------------ | --------- | -------------------------------------------------------- | ---------- | | `observed_values` | `ndarray` | Actual observations time series. | *required* | | `predicted_values` | `ndarray` | Estimated/predicted time series. | *required* | | `as_percentage` | `bool` | If True, return bias as a percentage. Defaults to False. | `False` | Returns: | Name | Type | Description | | ------- | ------- | ---------------------------- | | `float` | `float` | The calculated Percent Bias. | Example > > > observed = np.array([1, 2, 3]) predicted = np.array([1.1, 2.2, 2.9]) RasUtils.calculate_percent_bias(observed, predicted, as_percentage=True) 3.33333333 Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def calculate_percent_bias(observed_values: np.ndarray, predicted_values: np.ndarray, as_percentage: bool = False) -> float: """ Calculate the Percent Bias between observed and predicted values. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. as_percentage (bool, optional): If True, return bias as a percentage. Defaults to False. Returns: float: The calculated Percent Bias. Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_percent_bias(observed, predicted, as_percentage=True) 3.33333333 """ multiplier = 100 if as_percentage else 1 obs_mean = np.mean(observed_values) if obs_mean == 0: logger.warning("Percent bias undefined: mean of observed values is zero") return np.nan percent_bias = multiplier * (np.mean(predicted_values) - obs_mean) / obs_mean logger.debug(f"Calculated Percent Bias: {percent_bias}") return percent_bias ``` ##### calculate_error_metrics Python ```python calculate_error_metrics(observed_values: ndarray, predicted_values: ndarray) -> Dict[str, float] ``` Compute a trio of error metrics: correlation, RMSE, and Percent Bias. Parameters: | Name | Type | Description | Default | | ------------------ | --------- | -------------------------------- | ---------- | | `observed_values` | `ndarray` | Actual observations time series. | *required* | | `predicted_values` | `ndarray` | Estimated/predicted time series. | *required* | Returns: | Type | Description | | ------------------ | -------------------------------------------------------------------------------------------------------- | | `Dict[str, float]` | Dict\[str, float\]: A dictionary containing correlation ('cor'), RMSE ('rmse'), and Percent Bias ('pb'). | Example > > > observed = np.array([1, 2, 3]) predicted = np.array([1.1, 2.2, 2.9]) RasUtils.calculate_error_metrics(observed, predicted) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def calculate_error_metrics(observed_values: np.ndarray, predicted_values: np.ndarray) -> Dict[str, float]: """ Compute a trio of error metrics: correlation, RMSE, and Percent Bias. Args: observed_values (np.ndarray): Actual observations time series. predicted_values (np.ndarray): Estimated/predicted time series. Returns: Dict[str, float]: A dictionary containing correlation ('cor'), RMSE ('rmse'), and Percent Bias ('pb'). Example: >>> observed = np.array([1, 2, 3]) >>> predicted = np.array([1.1, 2.2, 2.9]) >>> RasUtils.calculate_error_metrics(observed, predicted) {'cor': 0.9993, 'rmse': 0.06396, 'pb': 0.03333} """ correlation = np.corrcoef(observed_values, predicted_values)[0, 1] rmse = RasUtils.calculate_rmse(observed_values, predicted_values) percent_bias = RasUtils.calculate_percent_bias(observed_values, predicted_values) metrics = {'cor': correlation, 'rmse': rmse, 'pb': percent_bias} logger.debug(f"Calculated error metrics: {metrics}") return metrics ``` ##### update_file Python ```python update_file(file_path: Path, update_function: Callable, *args) -> None ``` Generic method to update a file. Parameters: file_path (Path): Path to the file to be updated update_function (Callable): Function to update the file contents \*args: Additional arguments to pass to the update_function Raises: Exception: If there's an error updating the file Example: > > > def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines RasUtils.update_file(Path("example.txt"), update_content, "Hello") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def update_file(file_path: Path, update_function: Callable, *args) -> None: """ Generic method to update a file. Parameters: file_path (Path): Path to the file to be updated update_function (Callable): Function to update the file contents *args: Additional arguments to pass to the update_function Raises: Exception: If there's an error updating the file Example: >>> def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines >>> RasUtils.update_file(Path("example.txt"), update_content, "Hello") """ try: with open(file_path, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() updated_lines = update_function(lines, *args) if args else update_function(lines) with open(file_path, 'w', encoding='utf-8', errors='replace') as f: f.writelines(updated_lines) logger.info(f"Successfully updated file: {file_path}") except Exception as e: logger.exception(f"Failed to update file {file_path}") raise ``` ##### get_next_number Python ```python get_next_number(existing_numbers: list) -> str ``` Determine the next available number from a list of existing numbers. Parameters: existing_numbers (list): List of existing numbers as strings Returns: str: Next available number as a zero-padded string Example: > > > RasUtils.get_next_number(["01", "02", "04"]) "05" Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def get_next_number(existing_numbers: list) -> str: """ Determine the next available number from a list of existing numbers. Parameters: existing_numbers (list): List of existing numbers as strings Returns: str: Next available number as a zero-padded string Example: >>> RasUtils.get_next_number(["01", "02", "04"]) "05" """ existing_numbers = sorted(int(num) for num in existing_numbers) next_number = max(existing_numbers, default=0) + 1 return f"{next_number:02d}" ``` ##### clone_file Python ```python clone_file(template_path: Path, new_path: Path, update_function: Optional[Callable] = None, *args) -> None ``` Generic method to clone a file and optionally update it. Parameters: template_path (Path): Path to the template file new_path (Path): Path where the new file will be created update_function (Optional[Callable]): Function to update the cloned file \*args: Additional arguments to pass to the update_function Raises: FileNotFoundError: If the template file doesn't exist Example: > > > def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines RasUtils.clone_file(Path("template.txt"), Path("new.txt"), update_content, "Hello") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def clone_file(template_path: Path, new_path: Path, update_function: Optional[Callable] = None, *args) -> None: """ Generic method to clone a file and optionally update it. Parameters: template_path (Path): Path to the template file new_path (Path): Path where the new file will be created update_function (Optional[Callable]): Function to update the cloned file *args: Additional arguments to pass to the update_function Raises: FileNotFoundError: If the template file doesn't exist Example: >>> def update_content(lines, new_value): ... lines[0] = f"New value: {new_value}\\n" ... return lines >>> RasUtils.clone_file(Path("template.txt"), Path("new.txt"), update_content, "Hello") """ if not template_path.exists(): logger.error(f"Template file '{template_path}' does not exist.") raise FileNotFoundError(f"Template file '{template_path}' does not exist.") shutil.copy(template_path, new_path) logger.info(f"File cloned from {template_path} to {new_path}") if update_function: RasUtils.update_file(new_path, update_function, *args) ``` ##### update_project_file Python ```python update_project_file(prj_file: Path, file_type: str, new_num: str, ras_object=None) -> None ``` Update the project file with a new entry. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file being added (e.g., 'Plan', 'Geom') new_num (str): Number of the new file entry ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: > > > RasUtils.update_project_file(Path("project.prj"), "Plan", "02") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def update_project_file(prj_file: Path, file_type: str, new_num: str, ras_object=None) -> None: """ Update the project file with a new entry. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file being added (e.g., 'Plan', 'Geom') new_num (str): Number of the new file entry ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.update_project_file(Path("project.prj"), "Plan", "02") """ ras_obj = ras_object or ras ras_obj.check_initialized() try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() new_line = f"{file_type} File={file_type[0].lower()}{new_num}\n" lines.append(new_line) with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(lines) logger.info(f"Project file updated with new {file_type} entry: {new_num}") except Exception as e: logger.exception(f"Failed to update project file {prj_file}") raise ``` ##### remove_prj_entry Python ```python remove_prj_entry(prj_file: Path, file_type: str, number: str, ras_object=None) -> None ``` Remove a file entry from the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') number (str): Two-digit number of the entry to remove (e.g., '05') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: > > > RasUtils.remove_prj_entry(Path("project.prj"), "Plan", "05") ###### Removes the line "Plan File=p05" from the .prj file Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def remove_prj_entry(prj_file: Path, file_type: str, number: str, ras_object=None) -> None: """ Remove a file entry from the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') number (str): Two-digit number of the entry to remove (e.g., '05') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.remove_prj_entry(Path("project.prj"), "Plan", "05") # Removes the line "Plan File=p05" from the .prj file """ ras_obj = ras_object or ras ras_obj.check_initialized() prefix = file_type[0].lower() target = f"{file_type} File={prefix}{number}" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() new_lines = [line for line in lines if line.strip() != target] if len(new_lines) == len(lines): logger.warning(f"Entry '{target}' not found in {prj_file}") return with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(new_lines) logger.info(f"Removed {file_type} entry {number} from project file") except Exception as e: logger.exception(f"Failed to remove entry from project file {prj_file}") raise ``` ##### rename_prj_entry Python ```python rename_prj_entry(prj_file: Path, file_type: str, old_number: str, new_number: str, ras_object=None) -> None ``` Rename a file entry in the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') old_number (str): Current two-digit number (e.g., '05') new_number (str): New two-digit number (e.g., '02') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: > > > RasUtils.rename_prj_entry(Path("project.prj"), "Plan", "05", "02") ###### Changes "Plan File=p05" to "Plan File=p02" in the .prj file Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def rename_prj_entry(prj_file: Path, file_type: str, old_number: str, new_number: str, ras_object=None) -> None: """ Rename a file entry in the .prj file. Parameters: prj_file (Path): Path to the project file file_type (str): Type of file entry ('Plan', 'Geom', 'Unsteady', or 'Flow') old_number (str): Current two-digit number (e.g., '05') new_number (str): New two-digit number (e.g., '02') ras_object (RasPrj, optional): RAS object to use. If None, uses the default ras object. Example: >>> RasUtils.rename_prj_entry(Path("project.prj"), "Plan", "05", "02") # Changes "Plan File=p05" to "Plan File=p02" in the .prj file """ ras_obj = ras_object or ras ras_obj.check_initialized() prefix = file_type[0].lower() old_line = f"{file_type} File={prefix}{old_number}" new_line_content = f"{file_type} File={prefix}{new_number}" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: lines = f.readlines() found = False for i, line in enumerate(lines): if line.strip() == old_line: lines[i] = new_line_content + '\n' found = True break if not found: logger.warning(f"Entry '{old_line}' not found in {prj_file}") return with open(prj_file, 'w', encoding='utf-8', errors='replace') as f: f.writelines(lines) logger.info(f"Renamed {file_type} entry {old_number} to {new_number} in project file") except Exception as e: logger.exception(f"Failed to rename entry in project file {prj_file}") raise ``` ##### backup_files Python ```python backup_files(files: List[Union[Path, str]], project_folder: Union[Path, str], operation_label: str = 'deleted') -> Optional[Path] ``` Move files to a timestamped Backup folder inside the project. Creates {project_folder}/Backup/{YYYY-MM-DD_HHMMSS}\_{operation_label}/ and moves each existing file into that folder. Non-existent files are silently skipped. Parameters: files (List\[Union[Path, str]\]): File paths to back up (str or Path). project_folder (Union[Path, str]): Project root where Backup/ will be created. operation_label (str): Label appended to timestamp folder name (e.g., "deleted_p05"). Returns: Optional\[Path\]: Path to backup folder if any files were moved, None otherwise. Example: > > > files = [Path("Muncie.p05"), Path("Muncie.p05.hdf")] backup_dir = RasUtils.backup_files(files, project_folder, "deleted_p05") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def backup_files( files: List[Union[Path, str]], project_folder: Union[Path, str], operation_label: str = "deleted", ) -> Optional[Path]: """ Move files to a timestamped Backup folder inside the project. Creates {project_folder}/Backup/{YYYY-MM-DD_HHMMSS}_{operation_label}/ and moves each existing file into that folder. Non-existent files are silently skipped. Parameters: files (List[Union[Path, str]]): File paths to back up (str or Path). project_folder (Union[Path, str]): Project root where Backup/ will be created. operation_label (str): Label appended to timestamp folder name (e.g., "deleted_p05"). Returns: Optional[Path]: Path to backup folder if any files were moved, None otherwise. Example: >>> files = [Path("Muncie.p05"), Path("Muncie.p05.hdf")] >>> backup_dir = RasUtils.backup_files(files, project_folder, "deleted_p05") """ files = [Path(f) for f in files] existing_files = [f for f in files if f.exists()] if not existing_files: return None timestamp = datetime.datetime.now().strftime("%Y-%m-%d_%H%M%S") backup_folder = Path(project_folder) / "Backup" / f"{timestamp}_{operation_label}" backup_folder.mkdir(parents=True, exist_ok=True) for f in existing_files: shutil.move(str(f), str(backup_folder / f.name)) logger.info(f"Backed up {f.name} to {backup_folder}") return backup_folder ``` ##### decode_byte_strings Python ```python decode_byte_strings(dataframe: DataFrame) -> pd.DataFrame ``` Decodes byte strings in a DataFrame to regular string objects. This function converts columns with byte-encoded strings (e.g., b'string') into UTF-8 decoded strings. Parameters: | Name | Type | Description | Default | | ----------- | ----------- | ----------------------------------------------------- | ---------- | | `dataframe` | `DataFrame` | The DataFrame containing byte-encoded string columns. | *required* | Returns: | Type | Description | | ----------- | ------------------------------------------------------------------------- | | `DataFrame` | pd.DataFrame: The DataFrame with byte strings decoded to regular strings. | Example > > > df = pd.DataFrame({'A': [b'hello', b'world'], 'B': [1, 2]}) decoded_df = RasUtils.decode_byte_strings(df) print(decoded_df) A B 0 hello 1 1 world 2 Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def decode_byte_strings(dataframe: pd.DataFrame) -> pd.DataFrame: """ Decodes byte strings in a DataFrame to regular string objects. This function converts columns with byte-encoded strings (e.g., b'string') into UTF-8 decoded strings. Args: dataframe (pd.DataFrame): The DataFrame containing byte-encoded string columns. Returns: pd.DataFrame: The DataFrame with byte strings decoded to regular strings. Example: >>> df = pd.DataFrame({'A': [b'hello', b'world'], 'B': [1, 2]}) >>> decoded_df = RasUtils.decode_byte_strings(df) >>> print(decoded_df) A B 0 hello 1 1 world 2 """ str_df = dataframe.select_dtypes(['object']) str_df = str_df.stack().str.decode('utf-8').unstack() for col in str_df: dataframe[col] = str_df[col] return dataframe ``` ##### perform_kdtree_query Python ```python perform_kdtree_query(reference_points: ndarray, query_points: ndarray, max_distance: float = 2.0) -> np.ndarray ``` Performs a KDTree query between two datasets and returns indices with distances exceeding max_distance set to -1. Parameters: | Name | Type | Description | Default | | ------------------ | --------- | -------------------------------------------------------------------------------------------------------- | ---------- | | `reference_points` | `ndarray` | The reference dataset for KDTree. | *required* | | `query_points` | `ndarray` | The query dataset to search against KDTree of reference_points. | *required* | | `max_distance` | `float` | The maximum distance threshold. Indices with distances greater than this are set to -1. Defaults to 2.0. | `2.0` | Returns: | Type | Description | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ndarray` | np.ndarray: Array of indices from reference_points that are nearest to each point in query_points. Indices with distances > max_distance are set to -1. | Example > > > ref_points = np.array(\[[0, 0], [1, 1], [2, 2]\]) query_points = np.array(\[[0.5, 0.5], [3, 3]\]) result = RasUtils.perform_kdtree_query(ref_points, query_points) print(result) array([ 0, -1]) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def perform_kdtree_query( reference_points: np.ndarray, query_points: np.ndarray, max_distance: float = 2.0 ) -> np.ndarray: """ Performs a KDTree query between two datasets and returns indices with distances exceeding max_distance set to -1. Args: reference_points (np.ndarray): The reference dataset for KDTree. query_points (np.ndarray): The query dataset to search against KDTree of reference_points. max_distance (float, optional): The maximum distance threshold. Indices with distances greater than this are set to -1. Defaults to 2.0. Returns: np.ndarray: Array of indices from reference_points that are nearest to each point in query_points. Indices with distances > max_distance are set to -1. Example: >>> ref_points = np.array([[0, 0], [1, 1], [2, 2]]) >>> query_points = np.array([[0.5, 0.5], [3, 3]]) >>> result = RasUtils.perform_kdtree_query(ref_points, query_points) >>> print(result) array([ 0, -1]) """ dist, snap = KDTree(reference_points).query(query_points, distance_upper_bound=max_distance) snap[dist > max_distance] = -1 return snap ``` ##### find_nearest_neighbors Python ```python find_nearest_neighbors(points: ndarray, max_distance: float = 2.0) -> np.ndarray ``` Creates a self KDTree for dataset points and finds nearest neighbors excluding self, with distances above max_distance set to -1. Parameters: | Name | Type | Description | Default | | -------------- | --------- | ---------------------------------------------------------------------------------------------------------------- | ---------- | | `points` | `ndarray` | The dataset to build the KDTree from and query against itself. | *required* | | `max_distance` | `float` | The maximum distance threshold. Indices with distances greater than max_distance are set to -1. Defaults to 2.0. | `2.0` | Returns: | Type | Description | | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ndarray` | np.ndarray: Array of indices representing the nearest neighbor in points for each point in points. Indices with distances > max_distance or self-matches are set to -1. | Example > > > points = np.array(\[[0, 0], [1, 1], [2, 2], [10, 10]\]) result = RasUtils.find_nearest_neighbors(points) print(result) array([1, 0, 1, -1]) Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def find_nearest_neighbors(points: np.ndarray, max_distance: float = 2.0) -> np.ndarray: """ Creates a self KDTree for dataset points and finds nearest neighbors excluding self, with distances above max_distance set to -1. Args: points (np.ndarray): The dataset to build the KDTree from and query against itself. max_distance (float, optional): The maximum distance threshold. Indices with distances greater than max_distance are set to -1. Defaults to 2.0. Returns: np.ndarray: Array of indices representing the nearest neighbor in points for each point in points. Indices with distances > max_distance or self-matches are set to -1. Example: >>> points = np.array([[0, 0], [1, 1], [2, 2], [10, 10]]) >>> result = RasUtils.find_nearest_neighbors(points) >>> print(result) array([1, 0, 1, -1]) """ dist, snap = KDTree(points).query(points, k=2, distance_upper_bound=max_distance) snap[dist > max_distance] = -1 snp = pd.DataFrame(snap, index=np.arange(len(snap))) snp = snp.replace(-1, np.nan) snp.loc[snp[0] == snp.index, 0] = np.nan snp.loc[snp[1] == snp.index, 1] = np.nan filled = snp[0].fillna(snp[1]) snapped = filled.fillna(-1).astype(np.int64).to_numpy() return snapped ``` ##### consolidate_dataframe Python ```python consolidate_dataframe(dataframe: DataFrame, group_by: Optional[Union[str, List[str]]] = None, pivot_columns: Optional[Union[str, List[str]]] = None, level: Optional[int] = None, n_dimensional: bool = False, aggregation_method: Union[str, Callable] = 'list') -> pd.DataFrame ``` Consolidate rows in a DataFrame by merging duplicate values into lists or using a specified aggregation function. Parameters: | Name | Type | Description | Default | | -------------------- | --------------------------------- | ----------------------------------------------------------- | ---------- | | `dataframe` | `DataFrame` | The DataFrame to consolidate. | *required* | | `group_by` | `Optional[Union[str, List[str]]]` | Columns or indices to group by. | `None` | | `pivot_columns` | `Optional[Union[str, List[str]]]` | Columns to pivot. | `None` | | `level` | `Optional[int]` | Level of multi-index to group by. | `None` | | `n_dimensional` | `bool` | If True, use a pivot table for N-Dimensional consolidation. | `False` | | `aggregation_method` | `Union[str, Callable]` | Aggregation method, e.g., 'list' to aggregate into lists. | `'list'` | Returns: | Type | Description | | ----------- | ----------------------------------------- | | `DataFrame` | pd.DataFrame: The consolidated DataFrame. | Example > > > df = pd.DataFrame({'A': [1, 1, 2], 'B': [4, 5, 6], 'C': [7, 8, 9]}) result = RasUtils.consolidate_dataframe(df, group_by='A') print(result) B C A\ > > > 1 [4, 5] [7, 8] 2 [6] [9] Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def consolidate_dataframe( dataframe: pd.DataFrame, group_by: Optional[Union[str, List[str]]] = None, pivot_columns: Optional[Union[str, List[str]]] = None, level: Optional[int] = None, n_dimensional: bool = False, aggregation_method: Union[str, Callable] = 'list' ) -> pd.DataFrame: """ Consolidate rows in a DataFrame by merging duplicate values into lists or using a specified aggregation function. Args: dataframe (pd.DataFrame): The DataFrame to consolidate. group_by (Optional[Union[str, List[str]]]): Columns or indices to group by. pivot_columns (Optional[Union[str, List[str]]]): Columns to pivot. level (Optional[int]): Level of multi-index to group by. n_dimensional (bool): If True, use a pivot table for N-Dimensional consolidation. aggregation_method (Union[str, Callable]): Aggregation method, e.g., 'list' to aggregate into lists. Returns: pd.DataFrame: The consolidated DataFrame. Example: >>> df = pd.DataFrame({'A': [1, 1, 2], 'B': [4, 5, 6], 'C': [7, 8, 9]}) >>> result = RasUtils.consolidate_dataframe(df, group_by='A') >>> print(result) B C A 1 [4, 5] [7, 8] 2 [6] [9] """ if aggregation_method == 'list': agg_func = lambda x: tuple(x) else: agg_func = aggregation_method if n_dimensional: result = dataframe.pivot_table(group_by, pivot_columns, aggfunc=agg_func) else: result = dataframe.groupby(group_by, level=level).agg(agg_func).map(list) return result ``` ##### find_nearest_value Python ```python find_nearest_value(array: Union[list, ndarray], target_value: Union[int, float]) -> Union[int, float] ``` Finds the nearest value in a NumPy array to the specified target value. Parameters: | Name | Type | Description | Default | | -------------- | ---------------------- | ------------------------------------------ | ---------- | | `array` | `Union[list, ndarray]` | The array to search within. | *required* | | `target_value` | `Union[int, float]` | The value to find the nearest neighbor to. | *required* | Returns: | Type | Description | | ------------------- | ---------------------------------------------------------------------------------- | | `Union[int, float]` | Union\[int, float\]: The nearest value in the array to the specified target value. | Example > > > arr = np.array([1, 3, 5, 7, 9]) result = RasUtils.find_nearest_value(arr, 6) print(result) 5 Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def find_nearest_value(array: Union[list, np.ndarray], target_value: Union[int, float]) -> Union[int, float]: """ Finds the nearest value in a NumPy array to the specified target value. Args: array (Union[list, np.ndarray]): The array to search within. target_value (Union[int, float]): The value to find the nearest neighbor to. Returns: Union[int, float]: The nearest value in the array to the specified target value. Example: >>> arr = np.array([1, 3, 5, 7, 9]) >>> result = RasUtils.find_nearest_value(arr, 6) >>> print(result) 5 """ array = np.asarray(array) idx = (np.abs(array - target_value)).argmin() return array[idx] ``` ##### horizontal_distance Python ```python horizontal_distance(coord1: ndarray, coord2: ndarray) -> float ``` Calculate the horizontal distance between two coordinate points. Parameters: | Name | Type | Description | Default | | -------- | --------- | ------------------------------- | ---------- | | `coord1` | `ndarray` | First coordinate point [X, Y]. | *required* | | `coord2` | `ndarray` | Second coordinate point [X, Y]. | *required* | Returns: | Name | Type | Description | | ------- | ------- | -------------------- | | `float` | `float` | Horizontal distance. | Example > > > distance = RasUtils.horizontal_distance(np.array([0, 0]), np.array([3, 4])) print(distance) 5.0 Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def horizontal_distance(coord1: np.ndarray, coord2: np.ndarray) -> float: """ Calculate the horizontal distance between two coordinate points. Args: coord1 (np.ndarray): First coordinate point [X, Y]. coord2 (np.ndarray): Second coordinate point [X, Y]. Returns: float: Horizontal distance. Example: >>> distance = RasUtils.horizontal_distance(np.array([0, 0]), np.array([3, 4])) >>> print(distance) 5.0 """ return np.linalg.norm(coord2 - coord1) ``` ##### find_valid_ras_folders Python ```python find_valid_ras_folders(search_path: Union[str, Path], max_depth: Optional[int] = None, return_project_info: bool = False) -> Union[List[Path], List[Dict[str, Any]]] ``` Recursively search for valid HEC-RAS project folders. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line (HEC-RAS project file) 1. At least one .pXX file where XX is 01-99 (plan files) This function does NOT require the global ras object to be initialized, making it suitable for discovery operations before project initialization. Parameters: | Name | Type | Description | Default | | --------------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------- | ---------- | | `search_path` | `Union[str, Path]` | Root directory to search for HEC-RAS projects. | *required* | | `max_depth` | `Optional[int]` | Maximum folder depth to search. None means unlimited. Depth 0 = search_path only, 1 = immediate subdirectories, etc. | `None` | | `return_project_info` | `bool` | If True, return list of dicts with folder path, project name, prj file path, and plan count. If False, return list of Paths. | `False` | Returns: | Type | Description | | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Union[List[Path], List[Dict[str, Any]]]` | Union\[List[Path], List\[Dict[str, Any]\]\]: - If return_project_info=False: List of Path objects for valid HEC-RAS folders - If return_project_info=True: List of dicts with keys: - 'folder': Path to the project folder - 'project_name': Name extracted from .prj filename - 'prj_file': Path to the .prj file - 'plan_count': Number of plan files found - 'plan_numbers': List of plan numbers (e.g., ['01', '02', '15']) | Example > > > ###### Find all valid HEC-RAS project folders > > > > > > folders = RasUtils.find_valid_ras_folders("C:/Projects/Hydrology") for folder in folders: ... print(f"Found project: {folder}") > > > > > > ###### Get detailed info about each project > > > > > > projects = RasUtils.find_valid_ras_folders( ... "C:/Projects", ... max_depth=3, ... return_project_info=True ... ) for proj in projects: ... print(f"{proj['project_name']}: {proj['plan_count']} plans") Note This function distinguishes HEC-RAS .prj files from ESRI projection files by checking for "Proj Title=" on the first line of the file. Source code in `ras_commander/RasUtils.py` ```python @staticmethod def find_valid_ras_folders( search_path: Union[str, Path], max_depth: Optional[int] = None, return_project_info: bool = False ) -> Union[List[Path], List[Dict[str, Any]]]: """ Recursively search for valid HEC-RAS project folders. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line (HEC-RAS project file) 2. At least one .pXX file where XX is 01-99 (plan files) This function does NOT require the global ras object to be initialized, making it suitable for discovery operations before project initialization. Args: search_path (Union[str, Path]): Root directory to search for HEC-RAS projects. max_depth (Optional[int]): Maximum folder depth to search. None means unlimited. Depth 0 = search_path only, 1 = immediate subdirectories, etc. return_project_info (bool): If True, return list of dicts with folder path, project name, prj file path, and plan count. If False, return list of Paths. Returns: Union[List[Path], List[Dict[str, Any]]]: - If return_project_info=False: List of Path objects for valid HEC-RAS folders - If return_project_info=True: List of dicts with keys: - 'folder': Path to the project folder - 'project_name': Name extracted from .prj filename - 'prj_file': Path to the .prj file - 'plan_count': Number of plan files found - 'plan_numbers': List of plan numbers (e.g., ['01', '02', '15']) Example: >>> # Find all valid HEC-RAS project folders >>> folders = RasUtils.find_valid_ras_folders("C:/Projects/Hydrology") >>> for folder in folders: ... print(f"Found project: {folder}") >>> # Get detailed info about each project >>> projects = RasUtils.find_valid_ras_folders( ... "C:/Projects", ... max_depth=3, ... return_project_info=True ... ) >>> for proj in projects: ... print(f"{proj['project_name']}: {proj['plan_count']} plans") Note: This function distinguishes HEC-RAS .prj files from ESRI projection files by checking for "Proj Title=" on the first line of the file. """ search_path = Path(search_path) if not search_path.exists(): logger.warning(f"Search path does not exist: {search_path}") return [] if not search_path.is_dir(): logger.warning(f"Search path is not a directory: {search_path}") return [] valid_folders = [] def is_valid_ras_prj(prj_file: Path) -> bool: """Check if a .prj file is a valid HEC-RAS project file.""" try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: first_line = f.readline() return first_line.strip().startswith("Proj Title=") except Exception as e: logger.debug(f"Could not read .prj file {prj_file}: {e}") return False def get_plan_files(folder: Path) -> List[Tuple[str, Path]]: """Get all valid plan files (.p01 to .p99) in a folder.""" plan_files = [] for i in range(1, 100): plan_num = f"{i:02d}" # Look for files matching *.pXX pattern for pfile in folder.glob(f"*.p{plan_num}"): plan_files.append((plan_num, pfile)) return plan_files def check_folder(folder: Path) -> Optional[Dict[str, Any]]: """Check if a folder is a valid HEC-RAS project folder.""" # Find .prj files prj_files = list(folder.glob("*.prj")) if not prj_files: return None # Find valid HEC-RAS .prj file (not ESRI projection file) valid_prj = None for prj_file in prj_files: if is_valid_ras_prj(prj_file): valid_prj = prj_file break if valid_prj is None: return None # Check for plan files plan_files = get_plan_files(folder) if not plan_files: return None # This is a valid HEC-RAS project folder return { 'folder': folder, 'project_name': valid_prj.stem, 'prj_file': valid_prj, 'plan_count': len(plan_files), 'plan_numbers': [pn for pn, _ in plan_files] } def scan_directory(current_path: Path, current_depth: int): """Recursively scan directories for HEC-RAS projects.""" # Check if we've exceeded max depth if max_depth is not None and current_depth > max_depth: return # Check current folder result = check_folder(current_path) if result: valid_folders.append(result) # Don't search subdirectories of a valid project folder # (nested projects are uncommon and would cause confusion) return # Scan subdirectories try: for item in current_path.iterdir(): if item.is_dir() and not item.name.startswith('.'): scan_directory(item, current_depth + 1) except PermissionError: logger.debug(f"Permission denied accessing: {current_path}") except Exception as e: logger.debug(f"Error scanning {current_path}: {e}") # Start scanning logger.info(f"Searching for HEC-RAS projects in: {search_path}") scan_directory(search_path, 0) logger.info(f"Found {len(valid_folders)} valid HEC-RAS project folders") if return_project_info: return valid_folders else: return [info['folder'] for info in valid_folders] ``` ##### is_valid_ras_folder Python ```python is_valid_ras_folder(folder_path: Union[str, Path]) -> bool ``` Check if a single folder is a valid HEC-RAS project folder. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line 1. At least one .pXX file where XX is 01-99 This function does NOT require the global ras object to be initialized. Parameters: | Name | Type | Description | Default | | ------------- | ------------------ | ---------------------------- | ---------- | | `folder_path` | `Union[str, Path]` | Path to the folder to check. | *required* | Returns: | Name | Type | Description | | ------ | ------ | ----------------------------------------------------- | | `bool` | `bool` | True if the folder is a valid HEC-RAS project folder. | Example > > > if RasUtils.is_valid_ras_folder("C:/Projects/MyRASModel"): ... print("This is a valid HEC-RAS project folder") ... else: ... print("Not a valid HEC-RAS project folder") Source code in `ras_commander/RasUtils.py` ```python @staticmethod def is_valid_ras_folder(folder_path: Union[str, Path]) -> bool: """ Check if a single folder is a valid HEC-RAS project folder. A valid HEC-RAS project folder contains: 1. A .prj file with "Proj Title=" on the first line 2. At least one .pXX file where XX is 01-99 This function does NOT require the global ras object to be initialized. Args: folder_path (Union[str, Path]): Path to the folder to check. Returns: bool: True if the folder is a valid HEC-RAS project folder. Example: >>> if RasUtils.is_valid_ras_folder("C:/Projects/MyRASModel"): ... print("This is a valid HEC-RAS project folder") ... else: ... print("Not a valid HEC-RAS project folder") """ folder_path = Path(folder_path) if not folder_path.exists() or not folder_path.is_dir(): return False # Find .prj files prj_files = list(folder_path.glob("*.prj")) if not prj_files: return False # Check if any .prj file is a valid HEC-RAS project file def is_valid_ras_prj(prj_file: Path) -> bool: try: with open(prj_file, 'r', encoding='utf-8', errors='replace') as f: first_line = f.readline() return first_line.strip().startswith("Proj Title=") except Exception: return False has_valid_prj = any(is_valid_ras_prj(pf) for pf in prj_files) if not has_valid_prj: return False # Check for at least one plan file (.p01 to .p99) for i in range(1, 100): plan_num = f"{i:02d}" if list(folder_path.glob(f"*.p{plan_num}")): return True return False ``` ##### safe_write_geometry Python ```python safe_write_geometry(geom_file: Union[str, Path], modified_lines: List[str], create_backup: bool = True) -> Optional[Path] ``` Atomically write geometry file with backup for safe file modification. This function implements safe file modification for HEC-RAS geometry files, ensuring data integrity through atomic operations and optional backup creation. Process 1. Create timestamped backup: geom_file.YYYYMMDD_HHMMSS.bak 1. Write to temp file: geom_file.tmp 1. Basic validation (line count reasonable, file size reasonable) 1. Atomic rename temp -> original (os.replace) 1. Return backup path Parameters: | Name | Type | Description | Default | | ---------------- | ------------------ | ------------------------------------------------------------------------------------------ | ---------- | | `geom_file` | `Union[str, Path]` | Path to the geometry file to write. | *required* | | `modified_lines` | `List[str]` | List of lines to write to the file. Each line should include newline characters if needed. | *required* | | `create_backup` | `bool` | If True, create timestamped backup before modification. Defaults to True for safety. | `True` | Returns: | Type | Description | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `Optional[Path]` | Optional\[Path\]: Path to backup file if create_backup=True and successful, None if create_backup=False or file didn't exist before. | Raises: | Type | Description | | ------------------- | ------------------------------------------------------ | | `FileNotFoundError` | If the geometry file doesn't exist (for modification). | | `PermissionError` | If write access is denied to the file or directory. | | `ValueError` | If modified_lines is empty or validation fails. | | `IOError` | If atomic rename fails. | Example > > > from ras_commander import RasUtils from pathlib import Path > > > > > > ###### Read geometry file > > > > > > geom_file = Path("project/geometry.g01") with open(geom_file, 'r') as f: ... lines = f.readlines() > > > > > > ###### Modify HTAB parameters (example) > > > > > > modified_lines = modify_htab_params(lines, starting_el=580.0) > > > > > > ###### Safe write with backup > > > > > > backup_path = RasUtils.safe_write_geometry(geom_file, modified_lines) print(f"Backup created at: {backup_path}") Notes - This function uses os.replace() for atomic rename, which is atomic on both Windows (NTFS) and Unix filesystems. - Backup files use format: filename.YYYYMMDD_HHMMSS.bak - If validation fails, temp file is deleted and original remains unchanged. - For rollback, use rollback_geometry() with the returned backup path. See Also - rollback_geometry: Restore from backup after failed modification - .claude/rules/python/path-handling.md: Path handling patterns Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def safe_write_geometry( geom_file: Union[str, Path], modified_lines: List[str], create_backup: bool = True ) -> Optional[Path]: """ Atomically write geometry file with backup for safe file modification. This function implements safe file modification for HEC-RAS geometry files, ensuring data integrity through atomic operations and optional backup creation. Process: 1. Create timestamped backup: geom_file.YYYYMMDD_HHMMSS.bak 2. Write to temp file: geom_file.tmp 3. Basic validation (line count reasonable, file size reasonable) 4. Atomic rename temp -> original (os.replace) 5. Return backup path Parameters: geom_file (Union[str, Path]): Path to the geometry file to write. modified_lines (List[str]): List of lines to write to the file. Each line should include newline characters if needed. create_backup (bool): If True, create timestamped backup before modification. Defaults to True for safety. Returns: Optional[Path]: Path to backup file if create_backup=True and successful, None if create_backup=False or file didn't exist before. Raises: FileNotFoundError: If the geometry file doesn't exist (for modification). PermissionError: If write access is denied to the file or directory. ValueError: If modified_lines is empty or validation fails. IOError: If atomic rename fails. Example: >>> from ras_commander import RasUtils >>> from pathlib import Path >>> >>> # Read geometry file >>> geom_file = Path("project/geometry.g01") >>> with open(geom_file, 'r') as f: ... lines = f.readlines() >>> >>> # Modify HTAB parameters (example) >>> modified_lines = modify_htab_params(lines, starting_el=580.0) >>> >>> # Safe write with backup >>> backup_path = RasUtils.safe_write_geometry(geom_file, modified_lines) >>> print(f"Backup created at: {backup_path}") Notes: - This function uses os.replace() for atomic rename, which is atomic on both Windows (NTFS) and Unix filesystems. - Backup files use format: filename.YYYYMMDD_HHMMSS.bak - If validation fails, temp file is deleted and original remains unchanged. - For rollback, use rollback_geometry() with the returned backup path. See Also: - rollback_geometry: Restore from backup after failed modification - .claude/rules/python/path-handling.md: Path handling patterns """ geom_file = Path(geom_file) backup_path = None temp_path = None # Validate inputs if not modified_lines: raise ValueError("modified_lines cannot be empty") # Verify original file exists (we're modifying, not creating) if not geom_file.exists(): raise FileNotFoundError(f"Geometry file not found: {geom_file}") # Check write permissions if not os.access(geom_file.parent, os.W_OK): raise PermissionError(f"Write permission denied for directory: {geom_file.parent}") try: # Read original file for validation comparison original_size = geom_file.stat().st_size with open(geom_file, 'r', encoding='utf-8', errors='replace') as f: original_line_count = sum(1 for _ in f) # Step 1: Create timestamped backup if create_backup: timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") backup_path = geom_file.parent / f"{geom_file.name}.{timestamp}.bak" # Copy original to backup shutil.copy2(geom_file, backup_path) logger.info(f"Backup created: {backup_path}") # Step 2: Write to temp file temp_path = geom_file.parent / f"{geom_file.name}.tmp" with open(temp_path, 'w', encoding='utf-8', newline='') as f: f.writelines(modified_lines) logger.debug(f"Temp file written: {temp_path}") # Step 3: Basic validation temp_size = temp_path.stat().st_size new_line_count = len(modified_lines) # Validation: File shouldn't be empty if temp_size == 0: raise ValueError("Modified file would be empty - validation failed") # Validation: Line count shouldn't change drastically (>50% reduction suspicious) if new_line_count < original_line_count * 0.5: raise ValueError( f"Line count reduced drastically ({original_line_count} -> {new_line_count}). " f"This may indicate data corruption. Aborting." ) # Validation: File size shouldn't shrink too much (>80% reduction suspicious) if temp_size < original_size * 0.2: raise ValueError( f"File size reduced drastically ({original_size} -> {temp_size} bytes). " f"This may indicate data corruption. Aborting." ) logger.debug( f"Validation passed: {new_line_count} lines, {temp_size} bytes " f"(original: {original_line_count} lines, {original_size} bytes)" ) # Step 4: Atomic rename temp -> original # os.replace() is atomic on both Windows (NTFS) and Unix os.replace(temp_path, geom_file) temp_path = None # Mark as successfully moved logger.info(f"Geometry file atomically updated: {geom_file}") return backup_path except Exception as e: # Clean up temp file if it exists if temp_path and temp_path.exists(): try: temp_path.unlink() logger.debug(f"Cleaned up temp file: {temp_path}") except Exception as cleanup_error: logger.warning(f"Failed to clean up temp file {temp_path}: {cleanup_error}") logger.error(f"Failed to write geometry file {geom_file}: {e}") raise ``` ##### rollback_geometry Python ```python rollback_geometry(geom_file: Union[str, Path], backup_path: Union[str, Path]) -> None ``` Restore geometry file from backup after failed modification. This function restores a geometry file from a previously created backup, typically used when a modification operation fails or produces incorrect results. Process 1. Verify backup file exists 1. Copy backup -> original (preserves backup for safety) 1. Log restoration Parameters: | Name | Type | Description | Default | | ------------- | ------------------ | --------------------------------------------------------- | ---------- | | `geom_file` | `Union[str, Path]` | Path to the geometry file to restore. | *required* | | `backup_path` | `Union[str, Path]` | Path to the backup file created by safe_write_geometry(). | *required* | Returns: | Type | Description | | ------ | ----------- | | `None` | None | Raises: | Type | Description | | ------------------- | ----------------------------- | | `FileNotFoundError` | If backup file doesn't exist. | | `PermissionError` | If write access is denied. | | `IOError` | If copy operation fails. | Example > > > from ras_commander import RasUtils from pathlib import Path > > > > > > ###### Attempt modification > > > > > > try: ... backup = RasUtils.safe_write_geometry(geom_file, modified_lines) ... # Run HEC-RAS to validate ... RasCmdr.compute_plan("01", clear_geompre=True) ... except Exception as e: ... # Modification failed - rollback ... if backup: ... RasUtils.rollback_geometry(geom_file, backup) ... print("Geometry file restored from backup") ... raise Notes - This function copies the backup to original, preserving the backup. - Use shutil.copy2() to preserve file metadata (timestamps, permissions). - After successful rollback, you may want to delete the backup manually if no longer needed. See Also - safe_write_geometry: Create backup and safely write modifications Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def rollback_geometry( geom_file: Union[str, Path], backup_path: Union[str, Path] ) -> None: """ Restore geometry file from backup after failed modification. This function restores a geometry file from a previously created backup, typically used when a modification operation fails or produces incorrect results. Process: 1. Verify backup file exists 2. Copy backup -> original (preserves backup for safety) 3. Log restoration Parameters: geom_file (Union[str, Path]): Path to the geometry file to restore. backup_path (Union[str, Path]): Path to the backup file created by safe_write_geometry(). Returns: None Raises: FileNotFoundError: If backup file doesn't exist. PermissionError: If write access is denied. IOError: If copy operation fails. Example: >>> from ras_commander import RasUtils >>> from pathlib import Path >>> >>> # Attempt modification >>> try: ... backup = RasUtils.safe_write_geometry(geom_file, modified_lines) ... # Run HEC-RAS to validate ... RasCmdr.compute_plan("01", clear_geompre=True) ... except Exception as e: ... # Modification failed - rollback ... if backup: ... RasUtils.rollback_geometry(geom_file, backup) ... print("Geometry file restored from backup") ... raise Notes: - This function copies the backup to original, preserving the backup. - Use shutil.copy2() to preserve file metadata (timestamps, permissions). - After successful rollback, you may want to delete the backup manually if no longer needed. See Also: - safe_write_geometry: Create backup and safely write modifications """ geom_file = Path(geom_file) backup_path = Path(backup_path) # Verify backup exists if not backup_path.exists(): raise FileNotFoundError(f"Backup file not found: {backup_path}") # Check write permissions if geom_file.exists() and not os.access(geom_file, os.W_OK): raise PermissionError(f"Write permission denied for file: {geom_file}") if not os.access(geom_file.parent, os.W_OK): raise PermissionError(f"Write permission denied for directory: {geom_file.parent}") try: # Copy backup to original (preserves backup for safety) shutil.copy2(backup_path, geom_file) logger.info(f"Geometry file restored from backup: {geom_file} <- {backup_path}") except Exception as e: logger.error(f"Failed to restore geometry file {geom_file} from {backup_path}: {e}") raise ``` ##### validate_geometry_file_basic Python ```python validate_geometry_file_basic(geom_file: Union[str, Path], min_lines: int = 10, required_patterns: Optional[List[str]] = None) -> bool ``` Perform basic validation on a geometry file. This function checks that a geometry file meets basic structural requirements, useful for pre-modification validation or post-write verification. Parameters: | Name | Type | Description | Default | | ------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------- | ---------- | | `geom_file` | `Union[str, Path]` | Path to the geometry file to validate. | *required* | | `min_lines` | `int` | Minimum number of lines expected. Defaults to 10. | `10` | | `required_patterns` | `Optional[List[str]]` | List of strings that must appear somewhere in the file. Defaults to ["River Reach="] for HEC-RAS geometry. | `None` | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------- | | `bool` | `bool` | True if validation passes, False otherwise. | Example > > > if RasUtils.validate_geometry_file_basic(geom_file): ... print("Geometry file appears valid") > > > > > > ###### Custom validation > > > > > > if RasUtils.validate_geometry_file_basic( ... geom_file, ... required_patterns=["River Reach=", "Type RM Length"] ... ): ... print("Geometry file has cross sections") Notes - This is a basic structural check, not a full HEC-RAS validation. - For comprehensive validation, use HEC-RAS geometric preprocessor. Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def validate_geometry_file_basic( geom_file: Union[str, Path], min_lines: int = 10, required_patterns: Optional[List[str]] = None ) -> bool: """ Perform basic validation on a geometry file. This function checks that a geometry file meets basic structural requirements, useful for pre-modification validation or post-write verification. Parameters: geom_file (Union[str, Path]): Path to the geometry file to validate. min_lines (int): Minimum number of lines expected. Defaults to 10. required_patterns (Optional[List[str]]): List of strings that must appear somewhere in the file. Defaults to ["River Reach="] for HEC-RAS geometry. Returns: bool: True if validation passes, False otherwise. Example: >>> if RasUtils.validate_geometry_file_basic(geom_file): ... print("Geometry file appears valid") >>> >>> # Custom validation >>> if RasUtils.validate_geometry_file_basic( ... geom_file, ... required_patterns=["River Reach=", "Type RM Length"] ... ): ... print("Geometry file has cross sections") Notes: - This is a basic structural check, not a full HEC-RAS validation. - For comprehensive validation, use HEC-RAS geometric preprocessor. """ geom_file = Path(geom_file) if required_patterns is None: # Default: Check for River Reach definition (present in most geometry files) required_patterns = ["River Reach="] if not geom_file.exists(): logger.warning(f"Geometry file does not exist: {geom_file}") return False try: with open(geom_file, 'r', encoding='utf-8', errors='replace') as f: content = f.read() lines = content.splitlines() # Check minimum line count if len(lines) < min_lines: logger.warning( f"Geometry file has too few lines: {len(lines)} < {min_lines}" ) return False # Check required patterns for pattern in required_patterns: if pattern not in content: logger.warning(f"Required pattern not found in geometry file: {pattern}") return False logger.debug(f"Geometry file validation passed: {geom_file}") return True except Exception as e: logger.error(f"Error validating geometry file {geom_file}: {e}") return False ``` ##### dos2unix Python ```python dos2unix(project_dir: Union[str, Path], extensions: Optional[List[str]] = None) -> int ``` Convert CRLF line endings to LF in HEC-RAS text files. Processes .b## and .g## files by default (boundary and geometry text files that need LF endings for Linux HEC-RAS execution). Done in-place using pure Python (no shell dependency). Attribution: Implementation pattern derived from ras-agent (https://github.com/gheistand/ras-agent) by Glenn Heistand / CHAMP — Illinois State Water Survey. See runner.py:\_dos2unix_dir(). Parameters: | Name | Type | Description | Default | | ------------- | --------------------- | ------------------------------------------------------------------------- | ----------------------------------------- | | `project_dir` | `Union[str, Path]` | Path to the HEC-RAS project directory. | *required* | | `extensions` | `Optional[List[str]]` | Custom regex patterns for file extensions to process. Defaults to \[r'.(b | g)\\d+$'\] which matches .b01, .g01, etc. | Returns: | Name | Type | Description | | ----- | ----- | ------------------------- | | `int` | `int` | Number of files modified. | Example > > > from ras_commander import RasUtils count = RasUtils.dos2unix(Path("/project/dir")) print(f"Converted {count} files") Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def dos2unix(project_dir: Union[str, Path], extensions: Optional[List[str]] = None) -> int: """ Convert CRLF line endings to LF in HEC-RAS text files. Processes .b## and .g## files by default (boundary and geometry text files that need LF endings for Linux HEC-RAS execution). Done in-place using pure Python (no shell dependency). Attribution: Implementation pattern derived from ras-agent (https://github.com/gheistand/ras-agent) by Glenn Heistand / CHAMP — Illinois State Water Survey. See runner.py:_dos2unix_dir(). Parameters: project_dir (Union[str, Path]): Path to the HEC-RAS project directory. extensions (Optional[List[str]]): Custom regex patterns for file extensions to process. Defaults to [r'\\.(b|g)\\d+$'] which matches .b01, .g01, etc. Returns: int: Number of files modified. Example: >>> from ras_commander import RasUtils >>> count = RasUtils.dos2unix(Path("/project/dir")) >>> print(f"Converted {count} files") """ import re project_dir = Path(project_dir) if not project_dir.is_dir(): raise FileNotFoundError(f"Directory not found: {project_dir}") if extensions is None: patterns = [re.compile(r'\.(b|g)\d+$', re.IGNORECASE)] else: patterns = [re.compile(ext, re.IGNORECASE) for ext in extensions] modified_count = 0 for fpath in project_dir.iterdir(): if not fpath.is_file(): continue if not any(p.search(fpath.name) for p in patterns): continue try: raw = fpath.read_bytes() if b'\r' in raw: fpath.write_bytes(raw.replace(b'\r\n', b'\n').replace(b'\r', b'\n')) modified_count += 1 logger.debug(f"dos2unix: {fpath.name}") except (OSError, PermissionError) as exc: logger.warning(f"dos2unix skipped {fpath.name}: {exc}") logger.info(f"dos2unix: converted {modified_count} files in {project_dir}") return modified_count ``` ##### discover_ras_versions Python ```python discover_ras_versions() -> Dict[str, Path] ``` Discover installed HEC-RAS versions by scanning Windows Registry, filesystem, and Wine prefixes (on Linux). Resolution order: 1. Windows Registry (HKLM, WOW6432Node, HKCU) -- Windows only 1. Standard filesystem paths (Program Files) -- Windows only 1. Native Linux installs (/opt/hecras/, /opt/HEC-RAS/, ~/hecras/, or $RAS_COMMANDER_LINUX_RAS_ROOT) -- Linux only 1. Wine prefix paths (~/.wine, /opt/hecras-wine, etc.) -- Linux only Returns: | Name | Type | Description | | --------- | ----------------- | ----------------------------------------------------------------------- | | | `Dict[str, Path]` | Dict\[str, Path\]: Mapping of version string -> Path to the executable. | | | `Dict[str, Path]` | On Windows/Wine this is Ras.exe; for native Linux installs there | | | `Dict[str, Path]` | is no Ras.exe, so it maps to the RasUnsteady solver binary (use | | | `Dict[str, Path]` | .parent as ras_exe_dir for RasCmdr.compute_plan_linux). | | `Example` | `Dict[str, Path]` | {"6.6": Path("C:/Program Files (x86)/HEC/HEC-RAS/6.6/Ras.exe")} | Source code in `ras_commander/RasUtils.py` ```python @staticmethod @log_call def discover_ras_versions() -> Dict[str, Path]: """ Discover installed HEC-RAS versions by scanning Windows Registry, filesystem, and Wine prefixes (on Linux). Resolution order: 1. Windows Registry (HKLM, WOW6432Node, HKCU) -- Windows only 2. Standard filesystem paths (Program Files) -- Windows only 3. Native Linux installs (/opt/hecras/, /opt/HEC-RAS/, ~/hecras/, or $RAS_COMMANDER_LINUX_RAS_ROOT) -- Linux only 4. Wine prefix paths (~/.wine, /opt/hecras-wine, etc.) -- Linux only Returns: Dict[str, Path]: Mapping of version string -> Path to the executable. On Windows/Wine this is ``Ras.exe``; for native Linux installs there is no Ras.exe, so it maps to the ``RasUnsteady`` solver binary (use ``.parent`` as ``ras_exe_dir`` for ``RasCmdr.compute_plan_linux``). Example: {"6.6": Path("C:/Program Files (x86)/HEC/HEC-RAS/6.6/Ras.exe")} """ discovered: Dict[str, Path] = {} # Version folder names matching RasPrj.get_ras_exe() ras_version_folders = [ "7.0", "6.7 Beta 5", "6.7 Beta 4", "6.6", "6.5", "6.4.1", "6.3.1", "6.3", "6.2", "6.1", "6.0", "5.0.7", "5.0.6", "5.0.5", "5.0.4", "5.0.3", "5.0.1", "5.0", "4.1.0", "4.0" ] version_aliases = { "4.1": "4.1.0", "41": "4.1.0", "410": "4.1.0", "40": "4.0", "50": "5.0", "501": "5.0.1", "503": "5.0.3", "504": "5.0.4", "505": "5.0.5", "506": "5.0.6", "507": "5.0.7", "60": "6.0", "61": "6.1", "62": "6.2", "63": "6.3", "631": "6.3.1", "6.4": "6.4.1", "64": "6.4.1", "641": "6.4.1", "65": "6.5", "66": "6.6", "6.7": "6.7 Beta 5", "67": "6.7 Beta 5", "70": "7.0", } def _normalize_version(raw: str, install_dir: Optional[Path] = None) -> str: v = str(raw).strip() if v in version_aliases: return version_aliases[v] if install_dir is not None: fn = install_dir.name.strip() if fn in version_aliases: return version_aliases[fn] if fn in ras_version_folders: return fn return v def _add(version: str, exe_path: Path, source: str) -> None: if version in discovered: logger.debug(f"Skipping duplicate HEC-RAS {version} from {source}") return discovered[version] = exe_path logger.info(f"Discovered HEC-RAS {version} at {exe_path} via {source}") def _scan_root(root_dir: Path, source_label: str) -> None: """Scan a directory containing versioned HEC-RAS subfolders.""" if not root_dir.exists(): return # Check known folder names first for folder_name in ras_version_folders: exe = root_dir / folder_name / "Ras.exe" if exe.is_file(): v = _normalize_version(folder_name, exe.parent) _add(v, exe, source_label) # Glob for any other folders with Ras.exe try: for exe in sorted(root_dir.glob("*/Ras.exe")): v = _normalize_version(exe.parent.name, exe.parent) _add(v, exe, source_label) except OSError as exc: logger.warning(f"Filesystem scan failed for {root_dir}: {exc}") # --- Windows: Registry + Program Files --- if os.name == 'nt': # Registry scan try: import winreg def _is_no_more(exc: OSError) -> bool: return getattr(exc, "winerror", None) == 259 hive_map = { "HKLM": winreg.HKEY_LOCAL_MACHINE, "HKCU": winreg.HKEY_CURRENT_USER, } registry_locations = [ ("HKLM", r"SOFTWARE\HEC\HEC-RAS"), ("HKLM", r"SOFTWARE\WOW6432Node\HEC\HEC-RAS"), ("HKCU", r"SOFTWARE\HEC\HEC-RAS"), ] install_value_names = ( "InstallDir", "InstallPath", "Install Path", "Path", "ExePath", "RasExePath", ) for hive_name, subkey_path in registry_locations: try: with winreg.OpenKey(hive_map[hive_name], subkey_path) as root_key: idx = 0 while True: try: vk_name = winreg.EnumKey(root_key, idx) except OSError as exc: if _is_no_more(exc): break break idx += 1 try: with winreg.OpenKey(root_key, vk_name) as vk: install_val = None for val_name in install_value_names: try: val, _ = winreg.QueryValueEx(vk, val_name) if val: install_val = str(val) break except (FileNotFoundError, OSError): continue if install_val: p = Path(os.path.expandvars(install_val.strip().strip('"'))) if p.suffix.lower() != '.exe': p = p / "Ras.exe" if p.name.lower() == "ras.exe" and p.is_file(): v = _normalize_version(vk_name, p.parent) _add(v, p, f"registry {hive_name}\\{subkey_path}") except (FileNotFoundError, OSError): continue except (FileNotFoundError, OSError): continue except ImportError: logger.debug("winreg not available, skipping registry scan") # Filesystem scan (standard Windows paths) _scan_root(Path("C:/Program Files (x86)/HEC/HEC-RAS"), "filesystem (x86)") _scan_root(Path("C:/Program Files/HEC/HEC-RAS"), "filesystem") # --- Linux: native install scan --- else: # Native Linux HEC-RAS installs have no Ras.exe; the RasUnsteady # solver binary is the executable. Maps version -> RasUnsteady path # (callers for compute_plan_linux use ``.parent`` as ras_exe_dir). # Roots are configurable via $RAS_COMMANDER_LINUX_RAS_ROOT (CLB-883). linux_native_roots = [ Path(os.path.expanduser("~/hecras")), Path("/opt/hecras"), Path("/opt/HEC-RAS"), ] env_root = os.environ.get("RAS_COMMANDER_LINUX_RAS_ROOT") if env_root: linux_native_roots.insert(0, Path(env_root)) for _folder, _exe in RasUtils._scan_native_linux_ras(linux_native_roots).items(): _add(_normalize_version(_folder, _exe.parent), _exe, "linux native") # --- Linux: Wine prefix scan --- wine_prefix_candidates = [ Path(os.path.expanduser("~/.wine")), Path("/opt/hecras-wine"), Path(os.path.expanduser("~/hecras-wine")), ] # Also check WINEPREFIX env var env_prefix = os.environ.get("WINEPREFIX") if env_prefix: wine_prefix_candidates.insert(0, Path(env_prefix)) for prefix in wine_prefix_candidates: drive_c = prefix / "drive_c" if not drive_c.exists(): continue logger.debug(f"Scanning Wine prefix: {prefix}") # Standard HEC-RAS locations under drive_c _scan_root(drive_c / "Program Files (x86)" / "HEC" / "HEC-RAS", f"wine {prefix}") _scan_root(drive_c / "Program Files" / "HEC" / "HEC-RAS", f"wine {prefix}") _scan_root(drive_c / "HEC-RAS", f"wine {prefix}") logger.info(f"Discovered {len(discovered)} installed HEC-RAS version(s)") return discovered ``` #### Method Categories ##### File Operations | Method | Description | | -------------------------------------- | ----------------------------------------- | | `create_directory(path)` | Ensure directory exists, create if needed | | `find_files_by_extension(folder, ext)` | Find all files with given extension | | `get_file_size(path)` | Get file size in bytes | | `get_file_modification_time(path)` | Get file modification timestamp | | `clone_file(src, dest)` | Copy file to new location | | `update_file(path, content)` | Write content to file | | `remove_with_retry(path, retries=3)` | Delete file with retry logic | | `check_file_access(path, mode)` | Verify file access permissions | ##### Plan/Project Helpers | Method | Description | | ---------------------------------------- | --------------------------------------- | | `normalize_ras_number(number)` | Convert "1", "01", "p01" to "01" format | | `get_plan_path(plan_number)` | Get full path to plan file | | `get_next_number(folder, prefix)` | Find next available plan/geom number | | `update_plan_file(path, key, value)` | Update single key in plan file | | `update_project_file(prj_path, updates)` | Batch update .prj file | ##### Data Conversion | Method | Description | | ------------------------------------- | ----------------------------------------- | | `convert_to_dataframe(path)` | Load CSV/Excel to DataFrame | | `save_to_excel(df, path, sheet)` | Save DataFrame to Excel | | `decode_byte_strings(data)` | Decode HDF byte strings to Python strings | | `consolidate_dataframe(df, group_by)` | Group and aggregate DataFrame rows | ##### Statistical Analysis | Method | Description | | ------------------------------------- | ---------------------------------- | | `calculate_rmse(observed, predicted)` | Root Mean Square Error | | `calculate_percent_bias(obs, pred)` | Percent bias metric | | `calculate_error_metrics(obs, pred)` | All metrics (RMSE, NSE, PBIAS, R²) | Python ```python from ras_commander import RasUtils import numpy as np observed = np.array([100, 120, 140, 160, 180]) predicted = np.array([105, 125, 135, 165, 175]) metrics = RasUtils.calculate_error_metrics(observed, predicted) print(f"RMSE: {metrics['rmse']:.2f}") print(f"NSE: {metrics['nse']:.3f}") print(f"PBIAS: {metrics['pbias']:.1f}%") ``` ##### Spatial Operations | Method | Description | | ----------------------------------------------- | ------------------------------------ | | `perform_kdtree_query(points, query, max_dist)` | Find nearest points using KDTree | | `find_nearest_neighbors(points, max_dist)` | Find nearest neighbor for each point | | `find_nearest_value(array, target)` | Find value closest to target | | `horizontal_distance(p1, p2)` | Calculate 2D distance between points | Python ```python from ras_commander import RasUtils import numpy as np # Find nearest mesh cell for a list of query points mesh_centroids = np.array([[0, 0], [10, 10], [20, 20]]) query_points = np.array([[5, 5], [15, 15]]) indices = RasUtils.perform_kdtree_query( mesh_centroids, query_points, max_distance=10.0 ) # Returns [1, 2] - nearest cell indices ``` ### RasExamples ### RasExamples RasExamples - Manage and load HEC-RAS example projects for testing and development This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Logs are written to both console and a rotating file handler. - The default log file is 'ras_commander.log' in the 'logs' directory. - The default log level is INFO. To use logging in this module: 1. Use the @log_call decorator for automatic function call logging. 1. For additional logging, use logger.level calls (e.g., logger.info(), logger.debug()). 1. Obtain the logger using: logger = logging.getLogger(**name**) Example @log_call def my_function(): logger = logging.getLogger(**name**) logger.debug("Additional debug information") # Function logic here ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasExamples: - get_example_projects() - list_categories() - list_projects() - extract_project() - is_project_extracted() - clean_projects_directory() ### RasMap ### RasMap RasMap - Parses HEC-RAS mapper configuration files (.rasmap) This module provides functionality to extract and organize information from HEC-RAS mapper configuration files, including paths to terrain, soil, and land cover data. It also includes functions to automate the post-processing of stored maps. This module is part of the ras-commander library and uses a centralized logging configuration. Logging Configuration: - The logging is set up in the logging_config.py file. - A @log_call decorator is available to automatically log function calls. - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL Classes: | Name | Description | | -------- | ------------------------------------------------------------- | | `RasMap` | Class for parsing and accessing HEC-RAS mapper configuration. | ______________________________________________________________________ All of the methods in this class are static and are designed to be used without instantiation. List of Functions in RasMap: - parse_rasmap(): Parse a .rasmap file and extract relevant information - get_rasmap_path(): Get the path to the .rasmap file based on the current project - initialize_rasmap_df(): Initialize the rasmap_df as part of project initialization - get_terrain_names(): Extracts terrain layer names from a given .rasmap file - list_map_layers(): List all map layers in the RASMapper configuration file - list_reference_map_layers(): List shapefile/GeoJSON reference map layers - list_basemap_layers(): List standard basemap layers registered in .rasmap - set_map_layer_visibility(): Toggle reference, basemap, and land-classification map layers - add_reference_map_layer(): Add a shapefile/GeoJSON reference map layer - add_basemap_layer(): Add a standard basemap layer - add_map_layer(): Add a map layer to the RASMapper configuration file - remove_map_layer(): Remove a map layer from the RASMapper configuration file - list_geometry_layers(): List top-level geometries and child geometry elements - list_geometry_features(): List HDF geometry features inside a layer - list_land_classification_polygons(): List sidecar classification polygon overrides - add_land_classification_polygon(): Add sidecar classification polygon override - update_land_classification_polygon(): Update sidecar classification polygon override - delete_land_classification_polygon(): Delete sidecar classification polygon override - set_geometry_layer_visibility(): Toggle child geometry elements such as mesh, XS, and structures - list_result_layers(): List RASMapper result plan and child layers - set_result_layer_visibility(): Toggle result plan and result child layers - get_current_view(): Read the RASMapper CurrentView bounds - set_current_view(): Write the RASMapper CurrentView bounds - set_terrain_layer_visibility(): Toggle terrain layers for RASMapper inspection - list_terrain_display_settings(): List persisted terrain display settings - get_terrain_display_settings(): Read persisted terrain display settings for one terrain - set_terrain_display_settings(): Write persisted terrain display settings - set_update_legend_with_view(): Enable viewport-updated legends on raster surface layers - zoom_to_geometry_layer(): Zoom CurrentView to HDF-derived geometry element extents - get_geometry_feature_bounds(): Get HDF-derived extents for a selected feature - open_rasmapper(): Launch standalone RasMapper.exe against the project .rasmap - capture_rasmapper_snapshot(): Capture a visible RASMapper window screenshot - create_spatial_review_package(): Build a RASMapper QA/QC evidence bundle - postprocess_stored_maps(): Automates the generation of stored floodplain map outputs via GUI automation - store_all_maps(): Headless stored map generation using RasStoreMapHelper.exe (no GUI required) - get_results_folder(): Get the folder path containing raster results for a specified plan - get_results_raster(): Get the .vrt file path for a specified plan and variable name - set_water_surface_render_mode(): Set the water surface rendering mode (horizontal or sloped) - get_water_surface_render_mode(): Get the current water surface rendering mode - add_terrain_layer(): Add terrain layer to RASMapper configuration - list_results_plans(): List all plan result layers in the RASMapper configuration - ensure_results_plan_layer(): Register a plan HDF as a RASMapper result layer - ensure_2d_encroachment_plan_layers(): Register editable 2D encroachment plan layers - list_results_map_layers(): List RASMapper result map layers - add_results_map_layer(): Add a RASMapper result map layer - list_calculated_layers(): List all calculated layers across all plan results - add_calculated_layer(): Add a calculated layer with .rasscript to the RASMapper configuration - remove_calculated_layer(): Remove a calculated layer from the RASMapper configuration - add_wse_comparison_layers(): Batch add WSE comparison layers for existing/proposed plan pairs #### RASMapper Layer Discovery The layer-list methods read the `.rasmap` file and return one dataframe row per layer. Use these when a workflow needs discoverable layer names and resolved paths instead of the compact, list-valued `ras.rasmap_df` project summary. Python ```python from ras_commander import RasMap terrain_layers = RasMap.list_terrain_layers(project_path) terrain_display = RasMap.list_terrain_display_settings(project_path) landcover_layers = RasMap.list_landcover_layers(project_path) soils_layers = RasMap.list_soils_layers(project_path) infiltration_layers = RasMap.list_infiltration_layers(project_path) ``` `list_land_classification_layers()` is the broad parser for RASMapper `Type="LandCoverLayer"` entries. The land-cover, soils, and infiltration methods are filtered convenience wrappers around that catalog. #### Classification Polygon Overrides `RasMap.add_land_classification_polygon()` authors the RAS Mapper `Classification Polygons` sidecar group used by land-cover, soils, and infiltration layers. It also upserts the affected `Raster Map` and `Variables` class rows when those datasets exist. Python ```python from shapely.geometry import box from ras_commander import RasMap polygons = RasMap.add_land_classification_polygon( "Land Classification/LandCover.hdf", box(2083000, 370500, 2083500, 371000), class_name="Parking Lot", class_id=99, variable_values={ "mannings_n": 0.105, "percent_impervious": 95.0, }, ) ``` Use `list_land_classification_polygons()`, `update_land_classification_polygon()`, and `delete_land_classification_polygon()` for extraction and maintenance. After editing sidecar polygons for an already-associated geometry, rerun preprocessing/property-table workflows so compiled geometry HDFs consume the new override. #### Terrain Display Settings `RasMap.list_terrain_display_settings()`, `RasMap.get_terrain_display_settings()`, and `RasMap.set_terrain_display_settings()` expose RASMapper terrain display controls persisted in `.rasmap` XML. They cover hillshade display and Z factor, contour display and interval, and terrain stitch-edge plot options such as `Plot stitch TIN edges`. Python ```python RasMap.set_terrain_display_settings( project_path, terrain_name="TerrainWithChannel", hillshade_enabled=True, hillshade_z_factor=2.0, contour_enabled=True, contour_interval=5.0, stitch_edges_enabled=True, ) ``` CLB-272 owns these terrain display toggles. CLB-253 remains the separate terrain-modification gap for generating terrain changes such as channel modifications and interpolated cross-section terrain products. #### Geometry HDF Layer Associations `RasMap.get_hdf_geometry_association()` reads `/Geometry` association attributes from geometry HDFs and plan/result HDFs without mutation. `RasMap.associate_geometry_layers()` writes the geometry HDF attributes through Python-native `h5py`. Python ```python association = RasMap.get_hdf_geometry_association("MyModel.g01.hdf") print(association["terrain_hdf_path"]) RasMap.associate_geometry_layers( project_path, "MyModel.g01.hdf", terrain_hdf_path="Terrain/ExistingTerrain.hdf", landcover_hdf_path="Land Classification/LandCover.hdf", infiltration_hdf_path="Land Classification/Infiltration.hdf", ) ``` Compiled HDF only `associate_geometry_layers()` updates an existing `.g##.hdf`. It does not compile plain-text `.g##` geometry into HDF or create missing geometry datasets. ### RasProcess ### RasProcess RasProcess - Wrapper for RasProcess.exe CLI automation This module provides functionality to automate HEC-RAS mapping operations through the undocumented RasProcess.exe command-line interface. It enables headless generation of stored maps, preprocessing, and other RASMapper operations. Key Features: - Generate stored maps (WSE, Depth, Velocity, etc.) without GUI - Support for Max/Min profiles and specific timesteps - Automatic .rasmap modification for stored map configuration - Georeferencing fix for StoreMap command bug - **Linux support via Wine** (headless, no display required) Classes: | Name | Description | | ------------ | ---------------------------------------------- | | `RasProcess` | Static class for RasProcess.exe CLI operations | Note RasProcess.exe is an undocumented CLI tool bundled with HEC-RAS that exposes RASMapper automation functionality. See rasmapper_docs/16_rasprocess_cli_reference.md for complete documentation. Linux/Wine Support: On Linux, RasProcess.exe runs under Wine with .NET Framework 4.8. Requirements: - Wine 8.0+ (64-bit prefix) - winetricks: dotnet48, gdiplus, corefonts - HEC-RAS DLLs copied into Wine prefix (see setup_wine_environment()) Text Only ```text The module auto-detects Linux and wraps commands with Wine automatically. No code changes needed for users — just install the Wine environment. ``` #### RasProcess Details RasProcess.exe CLI RasProcess.exe is an undocumented command-line interface bundled with HEC-RAS that enables headless automation of RASMapper operations. The `RasProcess` class wraps this CLI for programmatic access. ##### Geometry Association Validator `validate_geometry_association_cli()` runs the native `RasProcess.exe SetGeometryAssociation` command and compares the resulting `/Geometry` attributes against ras-commander's expected HEC-RAS-style attributes. Python ```python from ras_commander import RasProcess result = RasProcess.validate_geometry_association_cli( "MyModel.g01.hdf", terrain_hdf_path="Terrain/ExistingTerrain.hdf", landcover_hdf_path="Land Classification/LandCover.hdf", ras_version="7.0", ) print(result["passed"]) print(result["return_code"]) print(result["mismatches"]) ``` The returned dictionary includes the native command arguments, return code, stdout/stderr, before/after attributes, expected attributes, mismatch list, and `passed`. In-place mutation This method mutates the supplied HDF. It exists as a native reference validator for disposable copies or intentional validation runs. Normal workflows should call `RasMap.associate_geometry_layers()`. ##### Supported Map Types | Parameter | XML Type | Display Name | Default | | --------------------- | -------------------------- | ------------ | ------- | | `wse` | elevation | WSE | True | | `depth` | depth | Depth | True | | `velocity` | velocity | Velocity | True | | `froude` | froude | Froude | False | | `shear_stress` | Shear | Shear Stress | False | | `depth_x_velocity` | depth and velocity | D * V | False | | `depth_x_velocity_sq` | depth and velocity squared | D * V² | False | ##### Profile Selection The `profile` parameter accepts: - `"Max"` - Maximum values across all timesteps (default) - `"Min"` - Minimum values across all timesteps - Specific timestamp string from `get_plan_timestamps()` (e.g., `"10SEP2018 02:30:00"`) ##### Basic Usage Python ```python from ras_commander import init_ras_project, RasProcess # Initialize project init_ras_project("path/to/project", "7.0") # Generate default maps (WSE, Depth, Velocity) results = RasProcess.store_maps( plan_number="01", profile="Max", wse=True, depth=True, velocity=True ) # Results is a dict: {'wse': [Path(...)], 'depth': [...], ...} for map_type, files in results.items(): print(f"{map_type}: {len(files)} file(s)") ``` ##### Batch Processing Python ```python # Generate maps for ALL plans with HDF results all_results = RasProcess.store_all_maps( profile="Max", wse=True, depth=True, velocity=True, froude=True ) for plan_num, files in all_results.items(): print(f"Plan {plan_num}: {sum(len(f) for f in files.values())} files") ``` ##### Timestep Maps Python ```python # Get available timestamps timestamps = RasProcess.get_plan_timestamps("01") print(f"Available: {timestamps[:3]}...") # ['10SEP2018 00:00:00', ...] # Generate map for specific time results = RasProcess.store_maps( plan_number="01", profile=timestamps[10], # 10th timestep wse=True ) ``` Georeferencing Fix RasProcess.exe has a known bug where generated TIFs may lack proper CRS information. Set `fix_georef=True` (default) to automatically apply the CRS from the project's projection file using rasterio. ##### Custom Output Path By default, RasProcess.exe writes to `//`. Use the `output_path` parameter to redirect output to any directory: Python ```python # Output to custom location results = RasProcess.store_maps( plan_number="01", output_path="C:/Exports/FloodMaps", depth=True, wse=True ) # Files moved to C:/Exports/FloodMaps/ after generation ``` How output_path Works The default `StoreAllMaps` command hardcodes output to `/`. When `output_path` is specified, individual `StoreMap` XML commands are used instead, with an absolute `OutputBaseFilename` that bypasses the ShortID prefix via C#'s `Path.Combine()` behavior. Relative paths are resolved against the project folder. # HDF Modules Classes for reading and processing HEC-RAS HDF result files. ## Core Classes ### HdfBase Base functionality for HDF file operations. - `get_dataset_info(hdf_path, group_path=None)` - Print HDF structure - `get_attrs(hdf_path, path)` - Get attributes at path - `get_projection(hdf_path)` - Get coordinate system - `parse_ras_datetime(datetime_str)` - Parse HEC-RAS datetime string - `parse_ras_datetime_ms(datetime_bytes)` - Parse datetime with milliseconds ### HdfPlan Plan-level information from HDF files. - `get_plan_info(hdf_path)` - Get plan metadata - `get_simulation_times(hdf_path)` - Get start/end times - `get_plan_parameters(hdf_path)` - Get computation parameters - `get_2d_flow_options(hdf_path)` - Get 2D equation set, initial condition time, tolerances, and solver options from computed HDF output ## Mesh Operations ### HdfMesh Mesh geometry data. - `get_mesh_area_names(hdf_path)` - List 2D flow areas - `get_mesh_cell_polygons(hdf_path)` - Get cell polygons as GeoDataFrame - `get_mesh_cell_faces(hdf_path)` - Get cell face lines - `get_mesh_cell_points(hdf_path)` - Get cell center points - `get_mesh_perimeter(hdf_path)` - Get mesh perimeter polygon - `get_mesh_cell_count(hdf_path)` - Get number of cells - `get_nearest_cell(hdf_path, point)` - Find nearest cell to point - `get_nearest_face(hdf_path, point)` - Find nearest face to point ### HdfResultsMesh 2D mesh results. - `get_mesh_max_ws(hdf_path)` - Maximum water surface elevation - `get_mesh_max_ws_time(hdf_path)` - Time of maximum WSE - `get_mesh_max_depth(hdf_path)` - Maximum depth - `get_mesh_max_face_v(hdf_path)` - Maximum face velocity - `get_mesh_timeseries(hdf_path, mesh, var)` - Time series for mesh - `get_mesh_cells_timeseries(hdf_path, mesh, cell_ids, var)` - Cell time series - `get_mesh_faces_timeseries(hdf_path, mesh, face_ids, var)` - Face time series - `get_profile_line_flow_timeseries(hdf_path, line_name, mesh_name=None, profile_lines_path=None, direction="absolute")` - Flow time series across a RAS Mapper profile/reference line - `get_profile_line_peak_flow(hdf_path, line_name, mesh_name=None, profile_lines_path=None, direction="absolute")` - Peak Q and peak time for a profile/reference line ## Plan Results ### HdfResultsPlan Plan-level results. - `get_runtime_data(hdf_path)` - Runtime statistics - `get_volume_accounting(hdf_path)` - Volume accounting data - `get_compute_messages(hdf_path)` - Computation messages - `get_compute_options(hdf_path)` - Computation options used - `is_steady_plan(hdf_path)` - Check if steady state - `get_steady_profile_names(hdf_path)` - Get steady profile names - `get_steady_wse(hdf_path)` - Get steady water surface elevations - `get_steady_info(hdf_path)` - Get steady flow metadata ### HdfResultsXsec 1D cross-section results. - `get_xsec_timeseries(hdf_path)` - All cross-section time series - `get_xsec_summary(hdf_path)` - Cross-section summary data ## 1D Geometry ### HdfXsec Cross-section and river geometry extraction from HDF. - `get_cross_sections(hdf_path)` - Extract cross-section geometries as GeoDataFrame - `get_river_centerlines(hdf_path)` - Extract river centerlines - `get_river_stationing(hdf_path)` - Calculate river stationing along centerlines - `get_river_reaches(hdf_path)` - Return model 1D river reach lines - `get_river_edge_lines(hdf_path)` - Return river edge lines - `get_river_bank_lines(hdf_path)` - Extract river bank lines ## Structure Data ### HdfStruc Structure geometry and SA/2D connections. - `get_connection_list(hdf_path)` - List SA/2D connections - `get_connection_profile(hdf_path, name)` - Get connection profile - `get_connection_gates(hdf_path, name)` - Get gate data ### HdfResultsBreach Dam breach results. - `get_breach_timeseries(hdf_path, structure)` - Breach time series - `get_breach_summary(hdf_path, structure)` - Breach summary statistics - `get_breaching_variables(hdf_path, structure)` - Breach geometry evolution - `get_structure_variables(hdf_path, structure)` - Structure flow variables ### HdfStorageArea Storage area volume-elevation curve extraction from HDF. - `get_volume_elevation_curve(hdf_path, sa_name)` - Get volume-elevation curve for a storage area - `get_storage_area_names(hdf_path)` - List storage areas in HDF ### HdfChannelCapacity 1D channel capacity analysis (multi-AEP). - `get_channel_capacity(hdf_path, river=None, reach=None)` - Compute channel capacity from cross-section geometry and results - `get_multi_aep_capacity(hdf_paths, aep_labels)` - Compare capacity across multiple AEP simulations ### HdfStruc1D 1D inline structure data extraction from HDF. - `get_inline_structure_data(hdf_path)` - Extract inline structure geometry and results - `get_structure_flow_timeseries(hdf_path, structure_name)` - Get flow time series through a structure ### HdfHydraulicTables Cross section property tables (HTAB). - `get_xs_htab(hdf_path, river, reach, station)` - Get HTAB data ## Infrastructure ### HdfPipe Pipe network analysis. - `get_pipe_conduits(hdf_path)` - Get conduit geometry - `get_pipe_nodes(hdf_path)` - Get node locations - `get_pipe_network_timeseries(hdf_path, var)` - Network time series - `get_pipe_network_summary(hdf_path)` - Network summary - `get_pipe_profile(hdf_path, conduit_id)` - Get conduit profile ### HdfPump Pump station analysis. - `get_pump_stations(hdf_path)` - Get station locations - `get_pump_groups(hdf_path)` - Get pump groups - `get_pump_station_timeseries(hdf_path, name)` - Station time series - `get_pump_station_summary(hdf_path)` - Station summary - `get_pump_operation_timeseries(hdf_path, name)` - Operation history ## Analysis ### HdfFluvialPluvial Fluvial-pluvial boundary analysis. - `calculate_fluvial_pluvial_boundary(hdf_path, delta_t)` - Calculate boundary ### HdfInfiltration Infiltration parameter management from HDF geometry files. **Geometry File Operations:** - `get_infiltration_baseoverrides(hdf_path)` - Retrieve infiltration parameters from geometry HDF - `set_infiltration_baseoverrides(hdf_path, data)` - Set infiltration parameters **Raster and Layer Operations:** - `get_infiltration_layer_data(hdf_path)` - Get infiltration layer data from HDF - `get_classification_polygons(hdf_path)` - Read infiltration sidecar classification polygon overrides - `get_infiltration_map(hdf_path)` - Read infiltration raster map - `calculate_soil_statistics(hdf_path)` - Process zonal statistics for soil analysis **Soil Analysis:** - `get_significant_mukeys(hdf_path, threshold)` - Identify mukeys above percentage threshold - `calculate_total_significant_percentage(hdf_path)` - Compute total coverage - `get_infiltration_parameters(hdf_path, mukey)` - Get parameters for specific mukey - `calculate_weighted_parameters(hdf_path)` - Compute weighted average parameters **Data Export:** - `save_statistics(data, path)` - Export soil statistics to CSV ### HdfLandCover Land-cover sidecar and final Manning's n extraction. - `get_landcover_raster_map(hdf_path)` - Read land-cover class IDs, names, and Manning's n values - `get_classification_polygons(hdf_path)` - Read land-cover sidecar classification polygon overrides - `get_preprocessed_mannings_n(hdf_path)` - Read preprocessed cell-center Manning's n values from geometry HDF ### HdfBndry Boundary condition geometry. - `get_bc_lines(hdf_path)` - Get BC lines - `get_breaklines(hdf_path)` - Get breaklines ## Utilities ### HdfUtils Utility class for HDF file operations. **Data Conversion:** - `convert_ras_string(value)` - Convert RAS HDF strings to Python objects - `convert_ras_hdf_value(value)` - Convert general HDF values to Python objects - `convert_df_datetimes_to_str(df)` - Convert DataFrame datetime columns to strings - `convert_hdf5_attrs_to_dict(attrs)` - Convert HDF5 attributes to dictionary - `convert_timesteps_to_datetimes(timesteps)` - Convert timesteps to datetime objects **Spatial Operations:** - `perform_kdtree_query(source, target)` - KDTree search between datasets - `find_nearest_neighbors(data, k)` - Find nearest neighbors within dataset **DateTime Parsing:** - `parse_ras_datetime(datetime_str)` - Parse RAS datetime (ddMMMYYYY HH:MM:SS) - `parse_ras_window_datetime(datetime_str)` - Parse simulation window datetime - `parse_duration(duration_str)` - Parse duration strings (HH:MM:SS) - `parse_ras_datetime_ms(datetime_bytes)` - Parse datetime with milliseconds - `parse_run_time_window(window_str)` - Parse time window strings ## Visualization ### HdfPlot & HdfResultsPlot Basic plotting utilities. - `plot_results_max_wsel(gdf)` - Plot maximum WSE map ## Usage Example Python ```python from ras_commander import HdfResultsMesh, HdfResultsPlan, init_ras_project init_ras_project("/path/to/project", "6.5") # Get HDF path hdf_path = ras.plan_df.loc[ras.plan_df['plan_number'] == '01', 'hdf_path'].iloc[0] # Extract max WSE max_wse = HdfResultsMesh.get_mesh_max_ws(hdf_path) # Get runtime stats runtime = HdfResultsPlan.get_runtime_data(hdf_path) ``` # Geometry Modules Classes for parsing and modifying HEC-RAS geometry files. ## RasGeometry Comprehensive 1D geometry parsing and modification. ### Cross Section Methods - `get_cross_sections(geom)` - List all cross sections - `build_cross_section(input_spec=None, **kwargs)` - Build a complete Type 1 cross-section geometry entry from station/elevation, terrain, adjacent XS, bank, Manning's n, and reach-length inputs - `get_station_elevation(geom, river, reach, station)` - Get station-elevation pairs - `set_station_elevation(geom, river, reach, station, sta_elev)` - Modify station-elevation - `get_mannings_n(geom, river, reach, station)` - Get Manning's n values - `get_bank_stations(geom, river, reach, station)` - Get bank station locations ### Cross Section Builder `GeomCrossSection.build_cross_section()` returns a `CrossSectionBuildResult` with resolved station/elevation, bank stations, Manning's n breakpoints, reach lengths, fallback messages, and formatted `.g##` geometry lines. The method accepts either keyword arguments or a `CrossSectionBuildInput` dataclass. Python ```python from ras_commander import ( CrossSectionBankStations, CrossSectionManningsN, CrossSectionReachLengths, GeomCrossSection, ) result = GeomCrossSection.build_cross_section( river="Example River", reach="Main", rs="1000", terrain_profile=terrain_df, # columns: station/elevation or Station/Elevation cut_line=[(0.0, 0.0), (500.0, 0.0)], river_centerline=[(250.0, -50.0), (250.0, 50.0)], ) entry_text = result.text ``` Fallback behavior is intentionally visible. Every fallback logs at `ERROR` level with `river|reach|RS` and also appears in `result.fallback_messages`. The builder always writes required `Bank Sta=`, `#Sta/Elev=`, and `#Mann=` records when enough station/elevation data can be resolved. Resolution order: - Station/elevation: explicit `station_elevation`, terrain profile or `RasTerrainMod.get_terrain_profile()`, then adjacent XS interpolation. - Bank stations: explicit station/elevation, explicit stations with terrain elevations, river-centerline intersection with default 20-unit main-channel width, then profile-interpolated bank elevations when terrain is unavailable. - Manning's n: controlled by `mannings_strategy`. `auto` prefers land cover, neighboring XS interpolation, user values, then defaults (`MC=0.06`, `LOB=ROB=0.08`). Strategies `landcover`, `neighbor`, `user`, and `default` make a source preferred. - Point count: station/elevation output is capped at 500 points using a Douglas-Peucker-style reducer that preserves endpoints, banks, the thalweg, and major slope breaks. Fully specified inputs avoid fallbacks: Python ```python result = GeomCrossSection.build_cross_section( river="Example River", reach="Main", rs="1000", station_elevation=survey_df, bank_stations=CrossSectionBankStations(120.0, 180.0, 534.2, 533.8), mannings_n=CrossSectionManningsN(lob=0.08, channel=0.05, rob=0.08), reach_lengths=CrossSectionReachLengths(left=400.0, channel=390.0, right=410.0), ) assert result.fallback_messages == [] ``` ### Storage Area Methods - `get_storage_areas(geom)` - List storage areas - `get_storage_elevation_volume(geom, name)` - Get elevation-volume curve ### Lateral Structure Methods - `get_lateral_structures(geom)` - List lateral structures - `get_lateral_weir_profile(geom, name)` - Get weir profile ### Connection Methods - `get_connections(geom)` - List SA/2D connections - `get_connection_weir_profile(geom, name)` - Get connection weir profile - `get_connection_gates(geom, name)` - Get gate data ## RasGeometryUtils Parsing utilities for HEC-RAS geometry files. ### Methods - `parse_fixed_width(line, width=8)` - Parse fixed-width formatted line - `parse_count_line(line)` - Parse count header line - `interpolate_bank_station(sta_elev, bank)` - Interpolate bank station elevation ## GeomReferenceFeatures Reference line and reference point helpers for 2D calibration and native reference-line output. ### Reference Line Methods - `add_reference_lines(geom_file, lines, storage_area)` - Insert manually supplied reference lines into a `.g##` file - `generate_reference_lines_from_longitudinal_line(...)` - Generate transverse reference-line dictionaries at regular station intervals along a named longitudinal line - `add_reference_lines_from_longitudinal_line(...)` - Generate and write transverse reference lines through the existing `.g##` writer - `get_reference_lines(geom_file)` - Read reference lines from a `.g##` file ### Automated Reference Lines Python ```python from ras_commander import GeomReferenceFeatures reference_lines = GeomReferenceFeatures.generate_reference_lines_from_longitudinal_line( centerlines_gdf, longitudinal_line_name="Main River", spacing=500.0, line_length=1500.0, name_template="MainRiver_{station_int}", ) GeomReferenceFeatures.add_reference_lines( "MyModel.g01", reference_lines, storage_area="Perimeter 1", ) ``` For result-guided orientation, pass `orientation="velocity"` or `orientation="depth_velocity"` with `orientation_plan_hdf`. Generated lines fall back to normal-to-line orientation unless `orientation_fallback="raise"` is set. ## GeomMesh Headless 2D mesh generation helpers and compiled geometry HDF refinement-region utilities. ### Refinement Region Methods - `add_refinement_region(geom_number, polygon, spacing_dx, ...)` - Add one refinement polygon to an existing compiled geometry HDF. - `add_flowline_refinement_regions(geom_number, flowlines, buffer_width, ...)` - Buffer GeoDataFrame or LineString channel flowlines into refinement-region polygons, optionally simplify/trim them, write them through `add_refinement_region()`, and return FID/name/spacing mappings. - `get_refinement_regions(geom_number)` - Read refinement-region FID, name, and spacing values from a compiled geometry HDF. - `set_refinement_region_spacing(geom_number, spacing_dx, ...)` - Update spacing for one or more existing refinement regions. - `set_refinement_region_name(geom_number, new_name, ...)` - Rename an existing refinement region. ## RasStruct Inline structure parsing. ### Inline Weir Methods - `get_inline_weirs(geom)` - List inline weirs - `get_inline_weir_profile(geom, river, reach, station)` - Get weir profile - `get_inline_weir_gates(geom, river, reach, station)` - Get gate data ### Bridge Methods - `get_bridges(geom)` - List bridges - `get_bridge_deck(geom, river, reach, station)` - Get deck profile - `get_bridge_piers(geom, river, reach, station)` - Get pier data - `get_bridge_abutment(geom, river, reach, station)` - Get abutment data - `get_bridge_approach_sections(geom, river, reach, station)` - Get approach sections - `get_bridge_coefficients(geom, river, reach, station)` - Get coefficients - `get_hydraulic_methods(geom, river, reach, station)` - Get bridge low-flow/high-flow method selections from `Bridge Culvert-`, `Deck Dist Width WeirC`, `BR Coef=`, and `WSPro=` records - `set_hydraulic_methods(geom, river, reach, station, low_flow_method=..., high_flow_method=..., weir_coefficient=...)` - Set bridge modeling approach method selections and related coefficients - `get_bridge_htab(geom, river, reach, station)` - Get HTAB settings Accepted `low_flow_method` values are `energy`, `momentum`, `yarnell`, and `wspro`. Accepted `high_flow_method` values are `energy` and `pressure_weir`. Optional compute flags are `use_energy`, `use_momentum`, `use_yarnell`, and `use_wspro`. Optional coefficient fields include `momentum_cd`, `yarnell_k`, `pressure_flow_submerged_inlet_cd`, `pressure_flow_submerged_inlet_outlet_cd`, and positive `weir_coefficient`. Unsupported combinations, such as disabling the selected low-flow method or selecting Momentum/Yarnell without an existing or supplied coefficient, raise `ValueError`. ### Culvert Methods - `GeomCulvert.get_culverts(geom, river, reach, station)` - Get all culverts at a bridge/culvert structure - `GeomCulvert.get_all(geom, river=None, reach=None)` - Get all culverts in a geometry file - `GeomCulvert.set_culverts(geom, river, reach, station, culverts)` - Replace culvert records at an existing bridge/culvert structure - `GeomCulvert.set_culvert(geom, river, reach, station, culvert=None, culvert_index=None, culvert_name=None, **kwargs)` - Update one culvert by index/name or append a new one - `GeomCulvert.get_adjacent_cross_sections(geom, river, reach, station)` - Find the nearest upstream and downstream cross sections around a structure - `GeomCulvert.set_adjacent_ineffective_flow(geom, river, reach, station, upstream_ineffective=None, downstream_ineffective=None, ...)` - Coordinate ineffective-flow writes on adjacent cross sections `set_culverts()` accepts a DataFrame, list of dictionaries, or one dictionary. Shape can be supplied as `Shape` code or `ShapeName` for any taxonomy-backed HEC-RAS culvert shape: Circular, Box, Pipe Arch, Ellipse, Arch, Semi-Circle, Low Profile Arch, High Profile Arch, or Con Span. Required fields are validated against `culvert_taxonomy.json`, including shape-specific dimensions, positive/nonnegative numeric ranges, `Chart #`/`Scale#` combinations, a maximum of 10 culvert groups per crossing, and a maximum of 25 identical barrels per group. The API preserves legacy field names `InletType` and `OutletType` for HEC-RAS `Chart #` and `Scale#`; `ChartID` and `ScaleID` aliases are also accepted. Single-barrel records require `UpstreamStation` and `DownstreamStation`. Multi-barrel records require `NumBarrels` and matching `BarrelStations` pairs. Python ```python from ras_commander.geom.GeomCulvert import GeomCulvert GeomCulvert.set_culverts( "model.g01", "River", "Reach", "1000", [ { "ShapeName": "Circular", "Span": 6, "Length": 50, "ManningsN": 0.013, "EntranceLoss": 0.5, "ExitLoss": 1.0, "InletType": 1, "OutletType": 1, "UpstreamInvert": 25.1, "UpstreamStation": 996, "DownstreamInvert": 25.0, "DownstreamStation": 996, "CulvertName": "Culvert #1", }, { "ShapeName": "Pipe Arch", "Span": 7, "Rise": 5, "Length": 48, "ManningsN": 0.024, "EntranceLoss": 0.4, "ExitLoss": 1.0, "ChartID": 34, "ScaleID": 1, "UpstreamInvert": 26.2, "UpstreamStation": 1000, "DownstreamInvert": 25.8, "DownstreamStation": 1000, "CulvertName": "Pipe Arch", }, { "ShapeName": "Box", "Span": 4, "Rise": 4, "Length": 55, "ManningsN": 0.015, "EntranceLoss": 0.3, "ExitLoss": 1.0, "InletType": 8, "OutletType": 1, "UpstreamInvert": 27.5, "DownstreamInvert": 27.0, "NumBarrels": 2, "BarrelStations": [(980, 980), (1020, 1020)], "CulvertName": "Twin Box", }, ], ) ``` ## GeomCrossSection Cross-section authoring and blocked-obstruction management. ### Cross Section Builder - `build_cross_section(input_spec=None, **kwargs)` - Build complete cross-section geometry entry from terrain, survey, or adjacent XS data - `get_blocked_obstructions(geom_file, river, reach, rs)` - Read blocked obstructions for a cross section - `set_blocked_obstructions(geom_file, river, reach, rs, obstructions)` - Write blocked obstructions See the [Cross Section Builder](#cross-section-builder) section above for resolution order and fallback behavior. ## GeomBridge Bridge geometry authoring (deck profiles, piers, abutments, approach sections). ### Methods - `build_bridge(geom_file, river, reach, rs, **bridge_params)` - Author complete bridge geometry - `get_bridge_deck(geom_file, river, reach, rs)` - Read bridge deck profile - `set_bridge_deck(geom_file, river, reach, rs, deck_data)` - Write bridge deck profile ## GeomBcLines 2D boundary condition line geometry authoring. ### Methods - `add_bc_line(geom_file, flow_area, name, coordinates, bc_type)` - Add BC line to 2D flow area - `get_bc_lines(geom_file, flow_area=None)` - Read existing BC lines - `remove_bc_line(geom_file, flow_area, name)` - Remove a BC line ## GeomLateral Lateral structure parsing and modification. ### Methods - `get_lateral_structures(geom_file)` - List lateral structures - `get_lateral_weir_profile(geom_file, name)` - Get weir profile data ## GeomStorage Storage area and 2D flow area geometry parsing and writing. ### Methods - `get_storage_areas(geom_file)` - List storage areas with elevation-volume data - `get_2d_flow_areas(geom_file)` - List 2D flow areas with settings - `get_2d_flow_area_settings(geom_file)` - Read 2D flow area computation settings - `set_2d_flow_area_settings(geom_file, area_name, **settings)` - Write 2D flow area settings (subgrid sampling, composite classification) - `write_2d_flow_area_perimeter(geom_file, area_name, coordinates, ...)` - Write 2D flow area perimeter ## GeomLevee Levee station-elevation parsing and modification. ### Methods - `get_levees(geom_file, river=None, reach=None, rs=None)` - Read levee data for cross sections - `set_levees(geom_file, river, reach, rs, levee_data)` - Write levee station-elevation data ## RasBreach Breach parameter modification in plan files. ### Methods - `list_breach_structures_plan(plan)` - List structures with breach data - `read_breach_block(plan, structure)` - Read breach parameters - `update_breach_block(plan, structure, **params)` - Modify breach parameters ## Usage Examples ### Cross Section Modification Python ```python from ras_commander import RasGeometry, init_ras_project init_ras_project("/path/to/project", "6.5") # Get station-elevation sta_elev = RasGeometry.get_station_elevation("01", "River", "Reach", "1000") # Modify and save sta_elev['elevation'] = sta_elev['elevation'] - 2.0 RasGeometry.set_station_elevation("01", "River", "Reach", "1000", sta_elev) ``` ### Breach Parameter Update Python ```python from ras_commander import RasBreach # Update breach parameters RasBreach.update_breach_block( "01", "Dam1", formation_time=2.0, bottom_width=100.0 ) ```