Remote Modules¶
Classes and functions for distributed HEC-RAS execution across local, PsExec, and Docker workers.
Factory Function¶
init_ras_worker¶
Create workers with init_ras_worker() and the options for the selected worker
type. The factory returns a LocalWorker, PsexecWorker, or DockerWorker for
the implemented backends.
Worker Classes¶
LocalWorker¶
Run plans in isolated folders on the control machine:
from ras_commander.remote import init_ras_worker
local = init_ras_worker(
"local",
worker_folder=r"C:\RasRemote",
cores_total=8,
cores_per_plan=4,
)
PsexecWorker¶
Run plans on a Windows machine through PsExec and an accessible network share:
from ras_commander.remote import init_ras_worker
remote = init_ras_worker(
"psexec",
hostname="WORKSTATION-01",
share_path=r"\\WORKSTATION-01\RasRemote",
worker_folder=r"C:\RasRemote",
ras_exe_path=r"C:\Program Files\HEC\HEC-RAS\6.6\Ras.exe",
session_id=2,
cores_total=16,
cores_per_plan=4,
)
session_id must be a positive integer. PsExec always targets that desktop
session with -i <session_id>. When system_account=True, the command uses both
-s and -i <session_id>; SYSTEM remains unsuitable for most interactive
HEC-RAS runs.
The PsExec worker uses the command-line interface and requires HEC-RAS 5 or newer. It rejects 4.x before staging because those versions require COM. For 5.x, it sets Current Plan only in the staged project and uses the project-first command form. Result copyback publishes the selected result and exact fresh completion-message sidecars together, removes stale destination messages, and retains the worker copy if publication fails.
DockerWorker¶
Run plans with a local Docker daemon and an image implementing the
DockerWorker shell-runner contract. hecras:6.6 below is an illustrative
local image tag, not a public Docker Hub reference. For the published paired
images, use RasDocker and the
container catalog:
from ras_commander.remote import init_ras_worker
docker = init_ras_worker(
"docker",
docker_image="hecras:6.6",
staging_directory=r"C:\RasDocker",
cores_total=8,
cores_per_plan=4,
)
Supported worker types¶
init_ras_worker() accepts "psexec", "local", and "docker". Any other
worker type raises ValueError. Slurm/HPC execution is not a worker backend:
use the RasSlurm portable-request adapter below
(Slurm Portable Execution) or
RasApptainer, a static API for one native-Linux unsteady solve per Slurm
allocation with an optional Wine prepare job
(Slurm + Apptainer Execution).
Execution¶
compute_parallel_remote¶
Execute queued plans across the worker pool:
from ras_commander.remote import compute_parallel_remote
results = compute_parallel_remote(
plan_numbers=["01", "02", "03", "04"],
workers=[local, remote],
num_cores=4,
force_rerun=False,
max_concurrent=None,
autoclean=True,
copy_geometry_outputs=True,
)
num_cores must be at least 1. The scheduler enforces each worker's effective
capacity as the smaller of its configured max_parallel_plans and
cores_total // num_cores. Plans remain queued until a real worker slot is free,
so a slow host cannot be oversubscribed and a faster host can accept later plans.
Workers with lower queue_priority values are preferred when capacity is
available.
For PsExec runs, the staged plan is rewritten to use num_cores; the source plan
and source project dataframes are not changed. For local and PsExec workers, set
copy_geometry_outputs=False to copy the plan-result HDF back without copying
geometry HDF and preprocessor outputs. The default remains True.
Concurrent geometry copyback
copy_geometry_outputs=True preserves the previous behavior, but concurrent
local or PsExec plans that share a geometry can race while copying the same
geometry outputs. For concurrent scenario ensembles using already-preprocessed
shared geometry, set copy_geometry_outputs=False.
The return value maps each plan number to an ExecutionResult with success,
worker_id, hdf_path, result_path, result_format, error_message, and
execution_time fields. result_path identifies the selected HDF or legacy
output artifact; result_format is "hdf" or "legacy" when one result
family is available.
The progress watchdog stops queued submissions when no plan finishes within the
slowest worker's max_runtime_minutes plus a staging/copy-back margin. It then
waits for already-started worker tasks before returning so those tasks cannot
continue mutating copied project outputs after the API call has returned.
Portable Plan Execution¶
Use the portable Slurm tutorial
for site configuration, rendering, staging, submission, and collection.
The backend comparison explains how
this contract differs from RasDocker, DockerWorker, and RasApptainer.
Portable execution runs one prepared steady or unsteady plan per request,
always with one CPU core, inside a pinned container. Requests and receipts are
versioned JSON (ras-commander-execution-request/v1,
ras-commander-execution-receipt/v1), and every path in a request is relative
to the request file's directory, so the same bundle runs through Docker or
Slurm/Apptainer without changes.
from ras_commander.remote import (
RasExecutionRequest,
RasPortableDocker,
validate_execution_receipt,
)
from ras_commander.RasSlurm import RasSlurm, SlurmSiteConfig, SlurmTransportConfig
request = RasExecutionRequest.create(
execution_id="reach-001",
request_directory="bundles/reach-001",
source_project_path="input/Model.prj",
plan_number="02",
output_directory="results",
ras_executable=r"C:\Program Files (x86)\HEC\HEC-RAS\6.6\Ras.exe",
container_identity="registry.example/hecras-steady@sha256:<64 hex>",
)
request_path = request.write("bundles/reach-001/request.json")
docker_result = RasPortableDocker.execute_request(request_path)
RasExecutionRequest.create()records the SHA-256 of the project file and of the whole project tree.read()andfrom_dict()reject unknown fields.request.payload_sha256(also available as the backward-compatiblerequest.digest) identifies the retained JSON request. Optional fields that were omitted stay omitted, while supplied values are emitted in their validated portable spelling. A new receipt therefore binds the caller's retained field set rather than silently inserting defaults.request.specification_sha256identifies the normalized execution meaning. Requests that differ only by omission of default-valued optional fields have different payload identities but the same specification identity. Use this second digest for cache keys or governed retry comparisons; continue using the payload identity to authenticate a particular retained request. Receipt validation also accepts the normalized digest written by older v1 executors, preserving validation of existing results.- In the container,
python -m ras_commander.remote.execute_request execute-request /job/request.jsoncopies the project to<output>/runtime_project, runsRasCmdr.compute_plan(num_cores=1, max_runtime=timeout_seconds), parses the compute messages, runsvalidate_steady_results, and writes<output>/execution_receipt.json. On Linux, a Windowsras_executableis run through Windows Python under Wine. This needs the image variablesRAS_COMMANDER_WINE_PREFIX_SEEDandRAS_COMMANDER_WINE_PYTHON, plusRAS_COMMANDER_WINE_ARCH(optional, defaultwin64) andRAS_COMMANDER_WINE_CPU_SHIM(optional absolute path to the CPU-numbering shim, preloaded for Wine processes; empty or0disables it). The executor also disables the Wine debugger (winedbg.exe=d) so a crash exits instead of waiting. The Wine executor image sets all four. Each request gets its own copy of the Wine prefix in Python's temporary directory, selected through the processTMPDIRwhere applicable. Fleet runners should bind that directory to writable node-local scratch. The temporary parent must be an absolute regular directory outside the request bundle and image seed. The executor maps the actual copied prefix into Windows, removes it after the attempt, and keeps project results and receipts in the declared output directory. Runtime staging messages identify prefix-copy start, completion, and delegated execution start. Solver deadlines and the requirement for an absent or empty output directory remain unchanged. RasPortableDocker.execute_request()andexecute_pool(max_concurrent=8)mount the bundle read-only and only the output directory writable.RasSlurm.render_submission(),stage(),submit(),status(),cancel(),collect(), andsubmit_batch()run requests as one exclusive-node pool (execution_mode="pool") or a throttled job array (execution_mode="array",array_throttle). Each step runsapptainer exec --contain --cleanenvwith a private<node_scratch_root>/<jobid>/ras-commander-<execution_id>-*bound as/tmp.SlurmTransportConfigsupportsmode="local"ormode="ssh", withtransfer_mode="rsync"or"scp".validate_execution_receipt()checks that a receipt belongs to its request and container image. Passverify_result_hdf_digest=Trueafter results cross a transfer boundary.
Stored maps after hydraulic validation¶
A request can also ask for RASMapper steady-profile stored maps:
request = RasExecutionRequest.create(
..., # the same arguments as above
stored_maps={
"terrain_name": "FIM 3DEP Terrain",
"profiles": None, # every profile, or exact names / 0-based indexes
"map_types": ["depth"], # default
"inundation_boundary": False, # default
"timeout_seconds": 1800, # default
},
)
- The block is validated by
StoredMapsRequest. It records its own schema (ras-commander-stored-maps-request/v1) and rejects unknown fields. Without the block, request and receipt JSON is exactly the original v1 contract, and request digests don't change. Executors from before this block reject a request that has one, because they don't accept unknown fields. - The executor runs
RasProcess.store_maps_at_steady_profiles()only after the steady results pass hydraulic validation. It runs on the runtime project copy with a newly initializedRasPrjand writes to<output>/maps. - The receipt gets a
stored_mapssection with these fields: requested: the block from the requeststatus:passed,failed, orskippedreason_codeelapsed_secondsoutput_directory(alwaysmaps)products: one row per product, withprofile_index,profile_name,map_type,primary_path(relative to the output directory), andfile_counterror- The reason codes are:
| Code | Meaning |
|---|---|
STORED_MAPS_COMPLETED |
Every requested product was produced. |
STORED_MAPS_SKIPPED_HYDRAULICS_NOT_VALIDATED |
Hydraulic validation did not pass, so maps were not run. |
STORED_MAPS_FAILED |
Map generation raised an error. |
STORED_MAPS_TIMEOUT |
Map generation ran past timeout_seconds. |
STORED_MAPS_OUTPUT_INVALID |
The product table returned is missing required columns. |
STORED_MAPS_NO_PRODUCTS |
No products were returned. |
STORED_MAPS_PRODUCT_MISSING |
A requested product was not produced (StoredMapProductsIncompleteError, or a primary file is missing). The products that were generated are still listed in products. |
STORED_MAPS_OUTPUT_OUTSIDE_RESULTS |
A product was written outside maps/. |
STORED_MAPS_RESULT_HDF_CHANGED |
Mapping changed the result HDF after it was validated. |
- Success rule: a mapping failure never changes
solver_verified,hydraulic_validated,result_validation, orresult_hdf_sha256. When maps are requested, receiptsuccessrequires hydraulic success and apassedstored_mapssection.validate_execution_receipt()also checks that the section matches the request and that every product's primary file exists under the output directory. - Receipt consistency: a
passedsection must list exactly one row for each requested profile and map type, plus a boundary row only when one was requested.skippedappears exactly when hydraulic validation did not pass. nullis rejected: leave the block out instead of writing"stored_maps": null.- Timeouts: each executor's time limit is
timeout_seconds + stored_maps.timeout_seconds(RasExecutionRequest.execution_timeout_seconds). Each executor adds its own margin on top of that: 120 s for Docker, 300 s for the Wine handoff, and 420 s for the Slurm step launcher. The launcher script is unchanged. Only the StoreAllMaps helper run itself is bounded bystored_maps.timeout_seconds. Setting up the project, fixing georeferencing, and re-hashing the result HDF use the executor margin.SlurmSiteConfig.time_limitis not checked against the step budgets, so set it high enough for all of them.
Installation¶
# Base package, including local and PsExec workers
pip install ras-commander
# Docker and all optional remote backends
pip install ras-commander[remote-docker]
pip install ras-commander[remote-all]
Complete source reference¶
init_ras_worker source reference¶
Initialize and validate a remote execution worker.
This factory function creates worker objects of various types, validates connectivity, and ensures HEC-RAS is available on the target system.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
worker_type
|
str
|
Type of worker - "psexec", "local", or "docker". For HPC clusters use RasSlurm or RasApptainer instead of a worker. |
required |
ras_object
|
RasPrj
|
RasPrj object to get ras_exe_path from. If None, uses global ras. |
None
|
**kwargs
|
Any
|
Worker-type specific configuration parameters |
{}
|
Common kwargs (all worker types): ras_exe_path: Path to HEC-RAS.exe on target machine (optional - obtained from ras object if not provided) worker_id: Unique identifier (auto-generated if not provided)
PsExec-specific kwargs
hostname: Remote machine hostname or IP (required) share_path: UNC path to network share (required, e.g., \hostname\RasRemote) worker_folder: Local path on remote machine corresponding to share_path (optional). If not specified, defaults to C:{share_name} (e.g., C:\RasRemote). Set this if your share points to a different local path. credentials: Dict with 'username' and 'password' (OPTIONAL - recommended to omit). When omitted, uses Windows authentication which avoids GUI access issues. Only provide if your network requires explicit authentication. session_id: Session ID to run in (default 2). Use "query user" on remote to check. process_priority: OS process priority for HEC-RAS execution. Valid values: "low" (default), "below normal", "normal". Recommended: "low" to minimize impact on remote user operations. queue_priority: Execution queue priority (0-9). Lower values execute first. Workers at queue level 0 are filled before queue level 1, etc. Default: 0. Use for tiered bursting (local=0, remote=1, cloud=2). system_account: Run as SYSTEM (default False) psexec_path: Path to PsExec.exe (auto-detected if not provided)
Returns:
| Type | Description |
|---|---|
RasWorker
|
Initialized and validated worker object ready for execution |
Raises:
| Type | Description |
|---|---|
ValueError
|
Invalid worker_type or missing required parameters |
ConnectionError
|
Cannot connect to remote machine |
FileNotFoundError
|
PsExec.exe or RAS.exe not found |
PermissionError
|
Insufficient permissions for remote execution |
Example
Initialize PsExec worker - ras_exe_path obtained from ras object¶
worker = init_ras_worker( "psexec", hostname="WORKSTATION-01", share_path=r"\WORKSTATION-01\RasRemote", session_id=2 # Check with "query user" on remote machine )
Or with explicit ras_object¶
worker = init_ras_worker( "psexec", ras_object=my_ras_project, hostname="WORKSTATION-01", share_path=r"\WORKSTATION-01\RasRemote" )
load_workers_from_json source reference¶
Load worker configurations from a JSON file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
json_path
|
Union[str, Path]
|
Path to JSON file containing worker configurations |
required |
ras_object
|
RasPrj
|
RasPrj object to get ras_exe_path from. If None, uses global ras. |
None
|
enabled_only
|
bool
|
If True, only load workers with "enabled": true (default True) |
True
|
Returns:
| Type | Description |
|---|---|
List[RasWorker]
|
List of initialized worker objects |
JSON Format
{ "workers": [ { "name": "Local Compute", "worker_type": "local", "worker_folder": "C:\RasRemote", "process_priority": "low", "queue_priority": 0, "cores_total": 4, "cores_per_plan": 2, "enabled": true }, { "name": "Remote PC", "worker_type": "psexec", "hostname": "192.168.1.100", "share_path": "\192.168.1.100\RasRemote", "worker_folder": "C:\RasRemote", "username": ".\user", "password": "password", "session_id": 2, "process_priority": "low", "queue_priority": 0, "cores_total": 8, "cores_per_plan": 2, "enabled": true } ] }
Example
workers = load_workers_from_json("RemoteWorkers.json") results = compute_parallel_remote(["01", "02"], workers=workers)
compute_parallel_remote source reference¶
Execute HEC-RAS plans in parallel across multiple remote workers.
Plans are assigned as worker capacity becomes available, respecting each worker's queue_priority (lower values execute first).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
plan_numbers
|
Union[str, List[str]]
|
Single plan number or list of plan numbers to execute |
required |
workers
|
List[RasWorker]
|
List of initialized worker objects (from init_ras_worker) |
required |
ras_object
|
RasPrj
|
RasPrj object for the project. If None, uses global ras. |
None
|
num_cores
|
int
|
Number of cores to allocate per plan execution |
4
|
clear_geompre
|
bool
|
Clear geometry preprocessor files (.c## files) before execution |
False
|
force_geompre
|
bool
|
Clear cached geometry products in place and force execution |
False
|
force_rerun
|
bool
|
Force execution even if results are current. When False (default), checks file modification times and skips if results are current. |
False
|
max_concurrent
|
Optional[int]
|
Maximum concurrent executions (default: sum of all worker slots) |
None
|
autoclean
|
bool
|
Delete temporary worker folders after execution (default True). Set to False for debugging to preserve worker folders. |
True
|
copy_geometry_outputs
|
bool
|
Copy geometry outputs back after local or PsExec execution (default True for backward compatibility). Concurrent plans sharing a geometry can race during copyback; use False for scenario ensembles that share preprocessed geometry. |
True
|
Returns:
| Type | Description |
|---|---|
Dict[str, ExecutionResult]
|
Dict mapping plan_number to ExecutionResult |
Example
Initialize workers¶
worker1 = init_ras_worker("psexec", hostname="PC1", ...) worker2 = init_ras_worker("psexec", hostname="PC2", ...)
Execute plans¶
results = compute_parallel_remote( ["01", "02", "03", "04"], workers=[worker1, worker2], ras_object=ras )
Check results¶
for plan_num, result in results.items(): if result.success: print( f"Plan {plan_num} succeeded " f"({result.execution_time:.1f}s)" ) else: print(f"Plan {plan_num} failed: {result.error_message}")
Scheduling
Workers are sorted by queue_priority (ascending). Plans are submitted only when a worker has an available capacity slot, so faster workers can take later plans without oversubscribing slower hosts.
Multi-Core Workers
Effective capacity is the smaller of the worker's configured max_parallel_plans and cores_total // num_cores. For example, a worker configured for 4 plans but given num_cores=8 and cores_total=16 runs at most 2 plans simultaneously.
get_worker_status source reference¶
Get status summary for a list of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workers
|
List[RasWorker]
|
List of worker instances |
required |
Returns:
| Type | Description |
|---|---|
Dict mapping worker_id to status dict with keys
|
|
-worker_type
|
Type of worker |
-hostname
|
Target hostname |
-queue_priority
|
Queue priority level |
-max_parallel_plans
|
Max concurrent plans |
-available
|
True if worker is available for execution |
LocalWorker source reference¶
Bases: RasWorker
Local parallel execution worker (uses RasCmdr.compute_plan internally).
IMPLEMENTATION STATUS: ✓ FULLY IMPLEMENTED
This worker provides a unified interface for local execution that matches the remote worker interface. It creates temporary worker folders, copies projects, executes using RasCmdr.compute_plan(), and copies results back.
Attributes:
| Name | Type | Description |
|---|---|---|
worker_folder |
str
|
Local path where worker folders are created (e.g., C:\RasRemote) |
process_priority |
str
|
OS process priority ("low", "below normal", "normal") |
queue_priority |
int
|
Execution queue priority (0-9, lower executes first) |
cores_total |
int
|
Total CPU cores available for this worker |
cores_per_plan |
int
|
Cores to allocate per HEC-RAS plan |
max_parallel_plans |
int
|
Max plans to run in parallel (calculated: cores_total/cores_per_plan) |
Example
worker = init_ras_worker( "local", worker_folder=r"C:\RasRemote", process_priority="low", queue_priority=0, cores_total=8, cores_per_plan=2 )
PsexecWorker source reference¶
Bases: RasWorker
PsExec-based Windows remote execution worker.
Uses Microsoft Sysinternals PsExec to execute HEC-RAS on remote Windows machines via network share deployment.
IMPLEMENTATION STATUS: ✓ FULLY IMPLEMENTED
Attributes:
| Name | Type | Description |
|---|---|---|
share_path |
str
|
UNC path to accessible network share (e.g., \hostname\RasRemote) |
worker_folder |
str
|
Local path on remote machine that corresponds to share_path. This is the actual folder path on the remote machine's filesystem. Example: If share_path is \hostname\RasRemote and the share points to C:\RasRemote on the remote machine, set worker_folder="C:\RasRemote". If not specified, defaults to "C:{share_name}" (e.g., C:\RasRemote). |
credentials |
Dict[str, str]
|
Dict with 'username' and 'password' for remote authentication. OPTIONAL for trusted networks. When omitted (empty dict or None), PsExec uses the current user's Windows authentication, which: - Works on domain-joined machines with proper trust - Avoids the "secondary logon" issue that prevents GUI access - Is RECOMMENDED for most internal network setups When credentials ARE provided, the specified user must be the same user logged into the remote desktop session, or have "Replace a process level token" (SeAssignPrimaryTokenPrivilege) right. |
session_id |
int
|
Session ID to run in (default 2 - typical for single-user workstations) |
process_priority |
str
|
OS process priority for HEC-RAS execution on remote machine. Valid values: "low" (default), "below normal", "normal". Recommended: "low" to minimize impact on remote user operations. Note: Higher priorities (above normal, high, realtime) are NOT supported to avoid impacting remote user operations. |
queue_priority |
int
|
Execution queue priority level (0-9). Lower values execute first. Workers at queue level 0 are fully utilized before queue level 1. Default: 0. Use for tiered bursting (local=0, remote=1, cloud=2). |
system_account |
bool
|
Run as SYSTEM account (default False) |
psexec_path |
str
|
Path to PsExec.exe (auto-detected from PATH if not specified) |
remote_temp_folder |
str
|
Temporary folder name on remote machine |
cores_total |
int
|
Total CPU cores available on remote machine (optional) |
cores_per_plan |
int
|
Cores to allocate per HEC-RAS plan (default 4) |
max_parallel_plans |
int
|
Max plans to run in parallel (calculated: cores_total/cores_per_plan) |
CRITICAL: HEC-RAS is a GUI application and REQUIRES session-based execution. - system_account=False (default) - Runs in the configured user desktop session - system_account=True - Uses SYSTEM in the configured interactive session; generally not recommended for HEC-RAS
Multi-Core Parallelism: - Set cores_total (e.g., 16) and cores_per_plan (e.g., 4) for parallel execution - Worker will run cores_total/cores_per_plan plans simultaneously (e.g., 4 plans) - Each plan gets cores_per_plan cores allocated - If not specified, executes plans sequentially (legacy behavior)
Session-based execution requires additional Group Policy configuration on the remote machine. See REMOTE_WORKER_SETUP_GUIDE.md for complete setup instructions.
Example
RECOMMENDED: No credentials (uses Windows authentication, avoids GUI issues)¶
worker = init_ras_worker( "psexec", hostname="WORKSTATION-01", share_path=r"\WORKSTATION-01\RasRemote", worker_folder=r"C:\RasRemote", ras_exe_path=r"C:\Program Files\HEC\HEC-RAS\6.3\RAS.exe", session_id=2 )
With explicit credentials (only if required by network policy)¶
worker = init_ras_worker( "psexec", hostname="WORKSTATION-01", share_path=r"\WORKSTATION-01\RasRemote", worker_folder=r"C:\RasRemote", credentials={"username": "DOMAIN\user", "password": "YOUR_PASSWORD"}, ras_exe_path=r"C:\Program Files\HEC\HEC-RAS\6.3\RAS.exe", session_id=2, process_priority="low", queue_priority=0 )
DockerWorker source reference¶
Bases: RasWorker
Docker containerized HEC-RAS execution worker.
Uses Rocky Linux 8 container with native HEC-RAS 6.6 Linux binaries. Requires two-step workflow: 1. Preprocess on Windows (creates .tmp.hdf files) 2. Execute simulation in Linux container
Attributes:
| Name | Type | Description |
|---|---|---|
docker_image |
str
|
Docker image name/tag (e.g., "hecras:6.6") |
docker_host |
Optional[str]
|
Docker daemon URL (None for local, or "tcp://host:2375") |
container_input_path |
str
|
Mount point for project files in container |
container_output_path |
str
|
Mount point for results in container |
container_script_path |
str
|
Path to run_ras.sh in container |
max_runtime_minutes |
int
|
Timeout for simulation (default 480 = 8 hours) |
preprocess_on_host |
bool
|
Whether to run preprocessing on Windows host first |
cpu_limit |
Optional[str]
|
CPU limit for container (e.g., "4" or "0.5") |
memory_limit |
Optional[str]
|
Memory limit (e.g., "8g", "4096m") |
Example
worker = init_docker_worker( ... docker_image="hecras:6.6", ... cores_total=8, ... cores_per_plan=4, ... preprocess_on_host=True ... )
ExecutionResult source reference¶
Result of a single plan execution.
Attributes:
| Name | Type | Description |
|---|---|---|
plan_number |
str
|
Plan number that was executed |
worker_id |
str
|
ID of worker that executed the plan |
success |
bool
|
True if execution completed successfully |
hdf_path |
Optional[str]
|
Path to the output HDF when |
result_path |
Optional[str]
|
Path to the selected HDF or legacy output artifact |
result_format |
Optional[str]
|
|
error_message |
Optional[str]
|
Error message (if failed) |
execution_time |
float
|
Time in seconds for execution |
PreprocessPolicy source reference¶
RasExecutionRequest source reference¶
Immutable request for one steady or unsteady plan using one CPU core.
source_project_path and output_directory are relative to the JSON
request's directory. The request records source identities for one
pre-execution validation pass. The output directory must be outside that
source tree.
create
classmethod
¶
create(*, execution_id: str, request_directory: Union[str, Path], source_project_path: Union[str, Path], plan_number: Union[str, int], output_directory: Union[str, Path], ras_executable: str, container_identity: str, preprocess_policy: Union[str, PreprocessPolicy] = PreprocessPolicy.REBUILD, timeout_seconds: int = 3600, flow_tolerance_absolute: float = 0.01, flow_tolerance_relative: float = 1e-06, stored_maps: Optional[Union[StoredMapsRequest, Mapping[str, Any]]] = None) -> 'RasExecutionRequest'
Create a request and bind it to the current immutable input tree.
stored_maps optionally requests steady-profile stored maps after
hydraulic validation (see :class:StoredMapsRequest). Omitting it
produces a request identical to the original v1 contract.
from_dict
classmethod
¶
Validate and retain a request mapping, rejecting unknown fields.
The retained mapping remains the authoritative payload for request identity. Validation may supply dataclass defaults for omitted optional fields, but those defaults must not silently change the request digest recorded by an execution receipt.
read
classmethod
¶
Read and validate a request JSON file.
resolve_paths
¶
Resolve and confine input/output paths relative to one request file.
to_dict
¶
Return the retained serializable request.
stored_maps is omitted when unset so v1 request bytes and digests
are unchanged. Requests loaded from JSON also preserve omission of
other optional fields; use :attr:specification_sha256 when a
normalized semantic identity is required.
RasExecutionReceipt source reference¶
StoredMapsRequest source reference¶
Optional steady-profile stored-map products for one execution request.
Maps are generated by
:meth:ras_commander.RasProcess.store_maps_at_steady_profiles on the
isolated runtime copy, only after the steady results pass hydraulic
validation, into <output_directory>/maps.
Attributes:
| Name | Type | Description |
|---|---|---|
terrain_name |
str
|
Registered RASMapper terrain layer to map against. |
profiles |
Optional[tuple[Union[str, int], ...]]
|
|
map_types |
tuple[str, ...]
|
Profile-dependent products; defaults to |
inundation_boundary |
bool
|
Also generate one boundary polygon for the final
selected profile. Defaults to |
timeout_seconds |
int
|
Aggregate StoreAllMaps helper timeout. |
mode |
str
|
Only |
schema |
str
|
|
validate_execution_receipt source reference¶
Validate receipt identity and output placement.
Source bytes are not re-read: execution already validated them once before
copying to its isolated runtime directory. Result bytes are rehashed only
when verify_result_hdf_digest is requested at a real transfer boundary.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request_path
|
Union[str, Path]
|
The request JSON the receipt answers. |
required |
receipt_path
|
Union[str, Path]
|
Must be the canonical
|
required |
expected_runtime_identity
|
Optional[str]
|
Container identity the
executor is known to have run (for example |
None
|
verify_result_hdf_digest
|
bool
|
Rehash the result HDF bytes. |
False
|
Returns:
| Type | Description |
|---|---|
RasExecutionReceipt
|
The validated receipt. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the receipt path is not canonical, or its execution ID, request digest, container identities, source-tree evidence, result HDF placement/digest, or requested stored-map section and product files do not match. |
Examples:
validate_steady_results source reference¶
Validate steady HDF rows and the effective authored flow schedule.
Flow changes are applied downstream in descending river-station order for
each river/reach. Every steady result cross section must map to an authored
effective flow for its profile within max(abs_tol, rel_tol * |flow|).
The function never raises for unreadable or inconsistent inputs; it fails
closed with passed=False and one or more reason codes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
hdf_path
|
Union[str, Path]
|
Computed steady plan HDF ( |
required |
flow_path
|
Union[str, Path]
|
Steady flow file ( |
required |
absolute_tolerance
|
float
|
Absolute flow tolerance (model units). |
0.01
|
relative_tolerance
|
float
|
Relative flow tolerance. |
1e-06
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Summary with |
dict[str, Any]
|
(list of str), |
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
tolerances, and |
codes
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
Examples:
validate_unsteady_results source reference¶
Validate an unsteady result from HEC-RAS's own volume accounting.
Model-agnostic across 1D, 2D, and combined models: HEC-RAS writes whole-model
volume accounting to Results/Unsteady/Summary/Volume Accounting for
every unsteady run. The result passes when that block exists, its values are
finite, and the absolute volume error is within
volume_error_percent_limit.
When every inflow is an inline hydrograph -- no DSS-sourced inflow, no
stage-driven boundary, no fixed start time, and precipitation disabled --
the authored inflow volume is integrated over the simulation window and
must match HEC-RAS's Total Boundary Flux of Water In within
inflow_relative_tolerance. Otherwise that reconciliation is recorded as
not_applicable with its reasons; it is never assumed to pass.
The function never raises for unreadable or inconsistent inputs; it fails
closed with passed=False and one or more reason codes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
hdf_path
|
Union[str, Path]
|
Computed unsteady plan HDF ( |
required |
flow_path
|
Union[str, Path]
|
Unsteady flow file ( |
required |
plan_path
|
Union[str, Path]
|
Plan file ( |
required |
volume_error_percent_limit
|
float
|
Largest acceptable
|
UNSTEADY_VOLUME_ERROR_PERCENT_LIMIT
|
inflow_relative_tolerance
|
float
|
Relative tolerance for inflow reconciliation. |
UNSTEADY_INFLOW_RELATIVE_TOLERANCE
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
|
dict[str, Any]
|
|
``inflow_reconciliation`` section. Reason codes
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
Examples:
RasPortableDocker source reference¶
Static Docker execution API for independent prepared projects.
build_execute_command
staticmethod
¶
build_execute_command(request_path: Union[str, Path], *, image: Optional[str] = None, docker_executable: str = 'docker', python_executable: str = 'python', memory: Optional[str] = None, pull: str = 'missing', security_options: Sequence[str] = ()) -> tuple[str, ...]
Render a shell-free, one-CPU Docker command for one request bundle.
The bundle directory is mounted read-only at /job and only the
request's output directory is mounted writable. The output directory
is created (empty) as a side effect.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request_path
|
Union[str, Path]
|
|
required |
image
|
Optional[str]
|
Image to run. Must equal the request's
immutable OCI |
None
|
docker_executable
|
str
|
Docker CLI executable. |
'docker'
|
python_executable
|
str
|
Python executable inside the image. |
'python'
|
memory
|
Optional[str]
|
Optional Docker |
None
|
pull
|
str
|
Docker |
'missing'
|
security_options
|
Sequence[str]
|
Docker |
()
|
Returns:
| Type | Description |
|---|---|
tuple[str, ...]
|
The Docker argument vector. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the image differs from |
execute_pool
staticmethod
¶
execute_pool(request_paths: Sequence[Union[str, Path]], *, max_concurrent: int = 8, image: Optional[str] = None, docker_executable: str = 'docker', python_executable: str = 'python', memory: Optional[str] = None, pull: str = 'missing', security_options: Sequence[str] = ()) -> PortableDockerPoolResult
Execute independent project requests with bounded concurrency.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request_paths
|
Sequence[Union[str, Path]]
|
Request files; each
|
required |
max_concurrent
|
int
|
Containers run at once (1-256). |
8
|
image
|
Optional[str]
|
Container image passed to |
None
|
docker_executable
|
str
|
Docker command passed to |
'docker'
|
python_executable
|
str
|
Python command passed to |
'python'
|
memory
|
Optional[str]
|
Memory limit passed to |
None
|
pull
|
str
|
Image pull policy passed to |
'missing'
|
security_options
|
Sequence[str]
|
Security options passed to |
()
|
Returns:
| Type | Description |
|---|---|
PortableDockerPoolResult
|
Results keyed by |
PortableDockerPoolResult
|
request order. A request that raises is recorded as a failed |
PortableDockerPoolResult
|
result rather than aborting the pool. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
execute_request
staticmethod
¶
execute_request(request_path: Union[str, Path], *, image: Optional[str] = None, docker_executable: str = 'docker', python_executable: str = 'python', memory: Optional[str] = None, pull: str = 'missing', security_options: Sequence[str] = ()) -> PortableDockerExecutionResult
Run one request in Docker and load its hydraulic receipt.
Docker receives one CPU. The bundle is mounted read-only with only the declared output directory over-mounted writable. The input model is therefore immutable at both the host contract and container boundary.
PortableDockerExecutionResult source reference¶
PortableDockerPoolResult source reference¶
RasSlurm source reference¶
Render, transport, and manage a Slurm request pool or job array.
cancel
staticmethod
¶
Cancel a submitted allocation through its configured transport.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
submission
|
SlurmSubmission
|
Submitted handle. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If |
collect
staticmethod
¶
Retrieve remote outputs only, then bind receipts to frozen requests.
For SSH transports each request's output directory is copied back and
result HDF digests are re-verified. Every receipt is checked with
:func:validate_execution_receipt against the SIF identity, and each
rc-### scheduler step must be COMPLETED with exit 0:0.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
submission
|
SlurmSubmission
|
Submitted handle. |
required |
Returns:
| Type | Description |
|---|---|
SlurmCollection
|
Receipts ( |
SlurmCollection
|
errors, and scheduler accounting. Collection is best effort per |
SlurmCollection
|
request; one failed transfer or receipt does not raise. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If rendered files no longer match their digests. |
render_submission
staticmethod
¶
render_submission(request_paths: Sequence[Union[str, Path]], submission_directory: Union[str, Path], *, site: SlurmSiteConfig, submission_id: Optional[str] = None) -> SlurmSubmission
Stage self-contained bundles and write a safe static sbatch script.
Each request is copied to bundles/<execution_id>/request.json with
its project tree, so the submission directory holds no workstation
paths. Writes slurm_batch.json, portable_slurm_launcher.py,
submit.sbatch, and slurm_submission.json. Nothing is submitted.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request_paths
|
Sequence[Union[str, Path]]
|
Request files to run in one exclusive-node allocation or as one array task each. |
required |
submission_directory
|
Union[str, Path]
|
Absent or empty directory. |
required |
site
|
SlurmSiteConfig
|
Scheduler, image, and memory settings. |
required |
submission_id
|
Optional[str]
|
Allocation name; random if omitted. |
None
|
Returns:
| Type | Description |
|---|---|
SlurmSubmission
|
Rendered handle with |
Raises:
| Type | Description |
|---|---|
ValueError
|
If no requests are given, execution IDs or outputs
collide, a request's |
stage
staticmethod
¶
Stage locally or transfer the complete bundle over SSH.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
submission
|
SlurmSubmission
|
Handle from :meth: |
required |
transport
|
SlurmTransportConfig
|
|
required |
Returns:
| Type | Description |
|---|---|
SlurmSubmission
|
Handle bound to |
SlurmSubmission
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If rendered files no longer match their digests. |
RuntimeError
|
If the remote |
status
staticmethod
¶
Read allocation and per-step state, exit, elapsed, and MaxRSS.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
submission
|
SlurmSubmission
|
Submitted handle. |
required |
Returns:
| Type | Description |
|---|---|
SlurmStatus
|
|
SlurmStatus
|
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If |
submit
staticmethod
¶
Submit a staged batch without invoking a local shell.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
submission
|
SlurmSubmission
|
Staged handle without a |
required |
Returns:
| Type | Description |
|---|---|
SlurmSubmission
|
Handle with the numeric |
SlurmSubmission
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If files were tampered with, the handle already has a job ID, or an SSH handle was not staged. |
RuntimeError
|
If |
submit_batch
staticmethod
¶
submit_batch(request_paths: Sequence[Union[str, Path]], submission_directory: Union[str, Path], *, site: SlurmSiteConfig, transport: SlurmTransportConfig = SlurmTransportConfig(), submission_id: Optional[str] = None) -> SlurmSubmission
Render, stage, and submit one request pool or job array.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request_paths
|
Sequence[Union[str, Path]]
|
Request files. |
required |
submission_directory
|
Union[str, Path]
|
Absent or empty directory. |
required |
site
|
SlurmSiteConfig
|
Scheduler, image, and memory settings. |
required |
transport
|
SlurmTransportConfig
|
Local or SSH transport. |
SlurmTransportConfig()
|
submission_id
|
Optional[str]
|
Allocation name; random if omitted. |
None
|
Returns:
| Type | Description |
|---|---|
SlurmSubmission
|
Submitted handle with |
Raises:
| Type | Description |
|---|---|
ValueError
|
As raised by :meth: |
RuntimeError
|
As raised by :meth: |
RasApptainer source reference¶
Static namespace for native HEC-RAS unsteady execution via Slurm + Apptainer.
cancel
staticmethod
¶
cancel(job: ApptainerJob, transport: ApptainerTransport, profile: ApptainerSiteProfile) -> CommandResult
Cancel this explicitly identified allocation; no cluster contact occurs otherwise.
check_solver_ready
staticmethod
¶
Return pre-submit solver-readiness findings for a completed tmp.hdf.
collect
staticmethod
¶
collect(job: ApptainerJob, transport: ApptainerTransport, profile: ApptainerSiteProfile, destination: Optional[Union[str, Path]] = None) -> ApptainerCollection
Download a terminal job's out/<jobid> folder and verify it.
success requires: scheduler state COMPLETED, a schema-valid receipt bound to this
request and Slurm job, a succeeded receipt, and every output hash matching locally.
Failed jobs are collected too so their evidence is retained.
collect_prepare
staticmethod
¶
collect_prepare(job: ApptainerJob, transport: ApptainerTransport, profile: ApptainerSiteProfile, project_destination: Union[str, Path], destination: Optional[Union[str, Path]] = None) -> ApptainerPrepareCollection
Download a terminal prepare job, verify it, and assemble the prepared project.
success requires scheduler state COMPLETED, a matching
SHA256SUMS, and a succeeded prepare receipt bound to this request
whose .p##.tmp.hdf/.b##/.x## hashes match the downloaded
files. Only then is project_destination (new or empty) written: the
staged project overlaid with every changed or new file.
load_profile
staticmethod
¶
Load and validate a site profile JSON; unknown fields are rejected.
pull_command
staticmethod
¶
The one-time, administrator-run command that creates the shared SIF (not executed here).
Run it on a node, then record the printed SHA-256 in the profile
(apptainer_image_sha256 and container_identity: sif:sha256:<digest>).
render_job
staticmethod
¶
render_job(project_folder: Union[str, Path], project_name: str, plan_number: Union[str, int], profile: ApptainerSiteProfile, job_directory: Union[str, Path], check_inputs: bool = True) -> ApptainerJob
Stage Windows-preprocessed artifacts and write the sbatch script. No SSH.
With check_inputs (default) the local tmp.hdf is first verified solver-ready
(:func:check_solver_ready); InputCheckError is raised before anything is staged.
Writes job_directory containing inputs/ (tmp.hdf, .b, .x with LF line
endings, any declared external gridded-DSS input, and SHA256SUMS),
engine.sh, job.sh and job.json. The source project is never
modified.
For cfile releases (before 6.3.1: 5.0.7, 6.1) the non-empty computed-geometry
<project>.c<geom> file is also required and staged byte-for-byte (it is binary),
the compiled-HDF core attributes are left untouched (OMP/MKL_NUM_THREADS still set
the threads), and the engine runs rasUnsteady64 <project>.c<geom> b<plan>
(bin_ras layout, 5.0.7) or RasUnsteady <project>.c<geom> b<plan>
(canonical layout, 6.1) in the job directory.
render_prepare_job
staticmethod
¶
render_prepare_job(project_folder: Union[str, Path], project_name: str, plan_number: Union[str, int], profile: ApptainerSiteProfile, job_directory: Union[str, Path]) -> ApptainerJob
Stage a complete project for Wine preprocessing on Slurm. No SSH.
The job runs RasPreprocess.preprocess_plan under Windows Python in
the profile's prepare_image (a Wine executor SIF), with the plan's
solver cores set to profile.num_cores. Submit it with
:meth:submit, then use :meth:collect_prepare and
:meth:render_job on the prepared project it returns.
Every regular file below project_folder is staged with its relative
path (terrain and other subfolders included); external dependencies
outside the folder are not followed. Writes inputs/ with
SHA256SUMS, job.sh and job.json.
Raises:
| Type | Description |
|---|---|
ApptainerProfileError
|
If the profile has no |
ValueError
|
If names are unsafe or a staged path is unsupported. |
FileNotFoundError
|
If the project or plan file is missing. |
FileExistsError
|
If |
status
staticmethod
¶
status(job: ApptainerJob, transport: ApptainerTransport, profile: ApptainerSiteProfile) -> ApptainerStatus
sacct-based status; falls back to squeue, then UNKNOWN until accounting appears.
submit
staticmethod
¶
submit(job: ApptainerJob, transport: Optional[ApptainerTransport] = None, profile: Optional[ApptainerSiteProfile] = None, dry_run: bool = True) -> ApptainerJob
Transfer the job folder and sbatch it. dry_run=True (default) never contacts SSH.
The submission intent is persisted (state SUBMITTING) before sbatch; a job in
that state is never resubmitted automatically because an ambiguous SSH failure may
already have created the allocation. Reconcile on the cluster, then render a new job.
RasContainerImages source reference¶
Static namespace for the image catalog and the local SIF cache.
applicable_roles
class-attribute
instance-attribute
¶
normalize_version
class-attribute
instance-attribute
¶
cache_directory
staticmethod
¶
Return the SIF cache: cache_dir, $RAS_COMMANDER_IMAGE_CACHE, or
~/.cache/ras-commander/images. On a cluster, point it at shared storage
visible to compute nodes so one copy serves every job.
catalog
staticmethod
¶
Return every published image in the packaged catalog.
ensure_sif_command
staticmethod
¶
ensure_sif_command(role: str, version: Union[str, float, int], cache_dir: str, *, apptainer_executable: str = 'apptainer', pull: bool = True, work_dir: Optional[str] = None) -> str
Return a bash snippet that reuses or fills a shared cluster cache.
Run it on a login host (for example over the site's SSH transport). It
prints <sha256> <path> for a verified cached SIF, pulling it first
when absent (unless pull=False). Use the printed hash in the site
configuration. The snippet never overwrites a verified file.
work_dir (default <cache_dir>/.apptainer-work) holds Apptainer's
build temp and layer cache unless APPTAINER_TMPDIR /
APPTAINER_CACHEDIR are already set. Use node-local disk when the
cache is on network storage; unpacking image layers there is slow.
get
staticmethod
¶
Return the published image for a role and release.
Raises:
| Type | Description |
|---|---|
ValueError
|
If the role does not apply to that release. |
LookupError
|
If it applies but no image has been published; build
one ( |
resolve_sif
staticmethod
¶
resolve_sif(role: str, version: Union[str, float, int], *, sif_path: Optional[Union[str, Path]] = None, expected_sha256: Optional[str] = None, cache_dir: Optional[Union[str, Path]] = None, pull: str = 'missing', apptainer_executable: str = 'apptainer') -> ResolvedSif
Return a verified SIF for a role and HEC-RAS release.
Order: sif_path (a local or site-built image, used as-is after its
hash is computed and, when given, checked against expected_sha256),
then the cache, then a registry pull into the cache. pull is
"never" (offline; fail if not cached), "missing" (default), or
"always" (refresh the cached copy).
Cached files are named by registry digest and verified on every use:
against the catalog's sif_sha256 for published SIFs, otherwise
against the .sha256 sidecar written when the file was pulled
(docker:// conversions are site-specific, so their hash is
recorded rather than predicted).
Returns:
| Type | Description |
|---|---|
ResolvedSif
|
|
ResolvedSif
|
|
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
|
ValueError
|
A hash does not match. |
LookupError
|
No published image exists for that role and release. |
RuntimeError
|
|
SlurmSiteConfig source reference¶
Scheduler-only settings kept outside hydraulic request manifests.
execution_mode="pool" (default) reserves one exclusive node and runs up
to slots_per_node one-core requests concurrently. "array" submits one
single-core array task per request, at most array_throttle at once.
slurm_memory is the memory for each request step; pool mode reserves it
once per slot. Each step binds a private directory below
node_scratch_root (node-local disk; /tmp by default) to /tmp in
a --contain container, so Wine prefixes and temporary files never share
host /tmp or the request bundle.
stage_bundles_to_node_scratch=True additionally copies each request
bundle to private scratch on the sole allocated node. Runtime model I/O
stays there; verified byte copies of outputs are published back afterward.
Scratch is retained if publication fails. The default preserves shared
runtime-project placement.
SlurmTransportConfig source reference¶
SlurmSubmission source reference¶
SlurmStatus source reference¶
SlurmTaskAccounting source reference¶
SlurmCollection source reference¶
Wine request path translation¶
On POSIX hosts, the portable CLI checks the copied task-private prefix's Z:
symlink. When it resolves to the POSIX root, an absolute request path without
parent traversal or Windows drive/separator syntax in a POSIX component is translated directly to the equivalent Z: path. This
avoids a separate cold Wine startup just to translate a path. Prefixes with
other drive mappings or incompatible component names retain winepath conversion and its error handling.
Delegated Windows Python execution, request validation, execution deadlines,
source immutability and task-private wineserver shutdown stay unchanged.