Skip to content

Container implementation comparison and TACC design

The implementation comparison was checked September 11, 2026. This page retains the proposed TACC/PyLauncher integration, with implementation status updated October 13, 2026. RasApptainer implements one native unsteady plan per Slurm allocation and an optional Wine prepare job; RasSlurm implements portable request pools and job arrays. Their contracts differ from the proposed worker integration below.

The Docker image pairs have Docker qualification. CLB has run both Slurm APIs live with site-built SIFs (see execution backends); the published native 6.6 SIF and an executor built from this repository were run live through both APIs on October 13, 2026. None of these images has been qualified on TACC.

Useful patterns from public implementations

Implementation Implemented behavior inspected Application to this workflow
Andy Carter's ras2fim command generator Bounded Windows Docker jobs; single-node Slurm/Apptainer shell pool; per-job logs; configurable threads per run. Keep scheduling outside containers and preserve individual model folders. Add matched solver settings and resource allocation.
FEMA FFRD ras-runner Action-based input/output transfer and an HDF completion check after native execution. Retain explicit input/output contracts and verified results. Cloud object-store transfer can remain a separate adapter.
slawler/ras-docker One S3 model payload per run, output upload, live solver output and progress parsing. Expose live output and callbacks without adding a scheduler inside the image.
dhardestylewis/hec_ras_docker Native runtime image and documented TACC Singularity pull/exec examples. Use the same distributed image under an HPC container runtime; qualify mounts and writable paths.
neeraip/hecras-v66-linux ZIP staging and native geometry/unsteady commands. Isolated staging is useful; do not copy shell failure handling or assume its hand-built preprocessing tables replace vendor preprocessing.

These are observed source features, not independent successful builds or hydraulic qualifications of those projects. Public availability also does not establish permission to copy code; implementation here uses our own API and does not vendor their source.

Two cores per model, controlled outside the image

The default is two cores per container, configurable from one through eight. On Docker, RasDocker sets --cpus=N; the native worker passes num_cores=N to RasCmdr.compute_plan_linux(). The Wine worker applies the same count through RasPlan.set_num_cores() and RasPlan.set_2d_flow_options() before preprocessing. Both stage receipts record the effective count. Docker's quota and the solver's thread count serve different purposes and should agree. The quota alone does not prevent the solver from creating many threads, and a thread setting alone is not a container resource limit. Docker resource documentation explains quota versus CPU affinity.

Proposed TACC execution

TACC documents Apptainer for HPC containers and PyLauncher for dispatching independent commands within a Slurm allocation. Apptainer can convert a Docker Hub image into a SIF file. Pull that file once before a campaign, then reuse it instead of contacting Docker Hub for every model. TACC container guide, Apptainer OCI support.

flowchart TD
    P["Prepare complete models with RasDocker.preprocess_plan()"] --> I["Prepared project folders and receipts"]
    S["Slurm allocation"] --> L["PyLauncher: two assigned cores per command"]
    I --> L
    L --> A["One Apptainer instance per selected model/plan"]
    A --> R["Native worker: RasCmdr.compute_plan_linux(num_cores=2)"]
    R --> O["Validated final HDF, logs and receipt on host storage"]

The host RasDocker class constructs Docker commands. The proposed TACC worker adapter would invoke the same native Python worker and preserve its request, receipt, and validation behavior. That adapter is distinct from the implemented RasApptainer API, which stages solver-ready artifacts and invokes the native engine, and RasSlurm, which uses the portable Python executor. No Docker daemon or MPI pool is needed inside the solver container.

Configure PyLauncher with cores=2 per command and verify CPU binding on the target system. Its documentation says that Slurm -n and --tasks-per-node do not set its task capacity; it queries allocated node cores. Do not assume those options alone prevent oversubscription. Reserve more capacity or reduce simultaneous jobs when memory and scratch use are the limiting factors. TACC PyLauncher guide.

Apptainer runs as the submitting user and can inherit scheduler cgroup limits. The target site's allocation and binding policy must enforce the resource boundary; num_cores configures the solver inside that boundary. Apptainer resource limits.

Each model needs a separate writable /job bind and private scratch outside that model directory. Keep terrain and other shared dependencies read-only. Stage active jobs on the system's scratch storage and collect outputs before node-local temporary storage is cleared. TACC I/O guidance.

The current images contain x86-64 vendor binaries. The target must provide compatible x86 nodes; Vista's ARM nodes cannot run those binaries natively. TACC Vista architecture. Choose the exact system, partition and allocation settings with Andy before writing a production Slurm example.

Required ras2fim handoff changes

The existing staging code copies only the temporary plan HDF, boundary file and geometry input. The new native worker requires the complete prepared project plus its preparation receipt. Preserve the .ras-commander receipt tree when transferring it.

The existing result collector collects a modified .p01.tmp.hdf and renames it. Our native worker preserves that prepared input and publishes a separate .p01.hdf. Collection must select the final HDF named by a successful compute receipt. Merely changing the Docker image would leave the collector reading uncomputed input.

Qualify native Apptainer execution first using a prepared model. Wine preparation can then move to the cluster with RasApptainer.render_prepare_job, which uses the Wine executor image: the SIF stays read-only, and each task copies the baked prefix into its own node-scratch /tmp under apptainer exec --contain --cleanenv. This replaces relying on the pre-compute image's default /run/ras-job directory, which an arbitrary host user cannot write under Apptainer. Wine needs a CPU-numbering shim when a Slurm CPU set does not start at CPU 0; see the CPU shim.