Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
e72f2cd
added updated modules for Oscar supercomputer
mrodrig6 Dec 9, 2025
8df72f8
fix to oscar mako file
mrodrig6 Dec 9, 2025
0056072
merger
mrodrig6 Dec 9, 2025
4d5b58b
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 10, 2025
19156ec
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 12, 2025
93bc50a
Oscar mako to account for srun
mrodrig6 Dec 12, 2025
55d57fb
Merge branch 'merger' of https://github.com/ComputationalFlowGroup/MF…
mrodrig6 Dec 12, 2025
c579caa
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 12, 2025
85c5b1b
Merge branch 'merger' of https://github.com/ComputationalFlowGroup/MF…
mrodrig6 Dec 12, 2025
517dbc1
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 15, 2025
49fb9db
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 16, 2025
0a27a21
Merge branch 'merger' of https://github.com/ComputationalFlowGroup/MF…
mrodrig6 Dec 16, 2025
5b58358
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 19, 2025
0a71bc3
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 21, 2025
7e3ad87
Merge branch 'MFlowCode:master' into merger
mrodrig6 Dec 28, 2025
8953c2f
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 11, 2026
46719ad
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 16, 2026
51e1a9c
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 17, 2026
7f5a037
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 20, 2026
94cd299
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 22, 2026
1052bd2
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 25, 2026
e1ad0ac
Merge branch 'MFlowCode:master' into merger
mrodrig6 Jan 28, 2026
9a99858
added Purdue Anvil
mrodrig6 Jan 29, 2026
18f4dd9
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 1, 2026
4b07b68
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 3, 2026
376fd81
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 4, 2026
3e5a88d
added Purdue Anvil mako file
mrodrig6 Feb 4, 2026
3d5252a
Merge remote-tracking branch 'refs/remotes/origin/merger' into merger
mrodrig6 Feb 4, 2026
294d749
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 5, 2026
7484d57
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 5, 2026
dc1678d
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 9, 2026
d3940db
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 10, 2026
d933d7e
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 13, 2026
c6b1b74
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 13, 2026
8108486
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 15, 2026
fb9f2cd
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 17, 2026
2f5bf9b
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 17, 2026
c208547
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 18, 2026
b6650d9
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 19, 2026
a533da6
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 20, 2026
3287ce9
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 23, 2026
58a88d4
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 23, 2026
93279c4
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 24, 2026
f5d4e05
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 24, 2026
ac28e64
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 25, 2026
efbaaad
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 25, 2026
9e67b91
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 26, 2026
23e88bb
Merge branch 'MFlowCode:master' into merger
mrodrig6 Feb 27, 2026
152405b
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 1, 2026
689d632
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 2, 2026
698f2c1
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 3, 2026
d55fef6
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 3, 2026
5019fbe
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 4, 2026
2e2ddb6
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 5, 2026
c35ea81
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 7, 2026
8f38bf4
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 9, 2026
f679f6b
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 9, 2026
eb3dd4d
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 11, 2026
d0a1693
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 12, 2026
bce528d
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 13, 2026
37ddf02
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 14, 2026
b12086b
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 16, 2026
ae2f3bb
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 17, 2026
140dbb7
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 17, 2026
00617ed
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 19, 2026
ac240ed
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 19, 2026
1036caf
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 21, 2026
b3c3679
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 27, 2026
9bd4fcb
Merge branch 'MFlowCode:master' into merger
mrodrig6 Mar 31, 2026
61302ac
Merge branch 'MFlowCode:master' into merger
mrodrig6 Apr 1, 2026
9ce66ec
Merge branch 'MFlowCode:master' into merger
mrodrig6 Apr 1, 2026
7eaf0a0
Merge branch 'MFlowCode:master' into merger
mrodrig6 Apr 2, 2026
12a830f
Merge branch 'MFlowCode:master' into merger
mrodrig6 Apr 2, 2026
da17a24
updating mako and modules for Brown Oscar
mrodrig6 Apr 3, 2026
66cb389
Merge branch 'merger' of https://github.com/ComputationalFlowGroup/MF…
mrodrig6 Apr 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion toolchain/bootstrap/modules.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ if [ -v $u_c ]; then
log "$C""ACCESS$W: Bridges2 (b) | Expanse (e) | Delta (d) | DeltaAI (dai)"
log "$Y""Gatech$W: Phoenix (p)"
log "$R""Caltech$W: Richardson (r)"
log "$BR""Brown$W: Oscar (o)"
log "$BR""Brown$W: Oscar (o)"
log "$BR""Purdue$W: Anvil (pa)"
log "$B""DoD$W: Carpenter Cray (cc) | Carpenter GNU (c) | Nautilus (n)"
log "$OR""Florida$W: HiPerGator (h)"
log_n "($G""a$W/$G""f$W/$G""s$W/$G""w$W/$B""tuo$W/$C""b$W/$C""e$CR/$C""d/$C""dai$CR/$Y""p$CR/$R""r$CR/$B""cc$CR/$B""c$CR/$B""n$CR/$BR""o"$CR"/$OR""h"$CR"): "
Expand Down
10 changes: 9 additions & 1 deletion toolchain/modules
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,18 @@ d-gpu MFC_CUDA_CC=80,86

o Brown Oscar
o-all python
o-cpu hpcx-mpi python/3.13.10s
o-cpu hpcx-mpi python/3.13.5
o-gpu nvhpc cuda/12.3.0 cmake/3.26.3
o-gpu CC=nvc CXX=nvc++ FC=nvfortran

pa Purdue Anvil
pa-cpu gcc/11.2.0
pa-cpu openmpi/4.1.6
pa-cpu python/3.9.5
pa-cpu fftw/3.3.8
pa-cpu cmake/3.20.0
pa-all PYTHONPATH=""

dai NCSA DeltaAI
dai-all python cmake nvidia/25.5
dai-all CC=nvc CXX=nvc++ FC=nvfortran
Expand Down
50 changes: 50 additions & 0 deletions toolchain/templates/anvil.mako
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env bash

<%namespace name="helpers" file="helpers.mako"/>

% if engine == 'batch':
#SBATCH --nodes=${nodes}
#SBATCH --ntasks-per-node=${tasks_per_node}
#SBATCH --cpus-per-task=1
#SBATCH --job-name="${name}"
#SBATCH --time=${walltime}
% if partition:
##SBATCH --partition=${partition}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug

% endif
Comment on lines +11 to +13
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Partition sbatch is commented 🐞 Bug ≡ Correctness

In toolchain/templates/anvil.mako the partition directive is written as "##SBATCH", so it is treated
as a comment even when partition is provided. This makes user-specified partitions ineffective and
can lead to jobs scheduling on the wrong/default partition or not scheduling as expected.
Agent Prompt
## Issue description
The Anvil batch template uses `##SBATCH --partition=...`, which comments out the partition directive even when the user provides a partition.

## Issue Context
This causes the `partition` CLI/template variable to have no effect.

## Fix Focus Areas
- toolchain/templates/anvil.mako[11-13]

## Suggested fix
Change `##SBATCH --partition=${partition}` to `#SBATCH --partition=${partition}`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +11 to +13
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Double hash comments out the partition directive.

Line 12 uses ##SBATCH instead of #SBATCH, which effectively comments out the partition directive. This means the partition will never be applied even when specified by the user.

Proposed fix
 % if partition:
-##SBATCH --partition=${partition}
+#SBATCH --partition=${partition}
 % endif
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
% if partition:
##SBATCH --partition=${partition}
% endif
% if partition:
`#SBATCH` --partition=${partition}
% endif

% if account:
#SBATCH --account="${account}"
% endif
#SBATCH --output="${name}.out"
#SBATCH --error="${name}.err"
#SBATCH --export=ALL
% if email:
#SBATCH --mail-user=${email}
#SBATCH --mail-type="BEGIN, END, FAIL"
% endif
% endif

${helpers.template_prologue()}

ok ":) Loading modules:\n"
cd "${MFC_ROOT_DIR}"
. ./mfc.sh load -c pa -m ${'g' if gpu_enabled else 'c'}
cd - > /dev/null
Comment on lines +29 to +31
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

GPU mode loads modules that don't exist.

When gpu_enabled is true, this loads with -m g (GPU mode), but toolchain/modules only defines pa-cpu entries—no pa-gpu modules exist. This will result in no GPU-specific modules being loaded.

If GPU support is intentionally deferred (CPU-only for now), consider adding a guard or comment. Otherwise, add pa-gpu definitions to toolchain/modules.

echo

% for target in targets:
${helpers.run_prologue(target)}

% if not mpi:
(set -x; ${profiler} "${target.get_install_binpath(case)}")
% else:
(set -x; ${profiler} \
mpirun -np ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}")
% endif

${helpers.run_epilogue(target)}

echo
% endfor

${helpers.template_epilogue()}
3 changes: 2 additions & 1 deletion toolchain/templates/oscar.mako
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#SBATCH --nodes=${nodes}
#SBATCH --tasks-per-node=${tasks_per_node}
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=4g
#SBATCH --job-name="${name}"
#SBATCH --time=${walltime}
% if partition:
Expand Down Expand Up @@ -42,7 +43,7 @@ echo
(set -x; ${profiler} "${target.get_install_binpath(case)}")
% else:
(set -x; ${profiler} \
mpirun -np ${nodes*tasks_per_node} \
srun \
"${target.get_install_binpath(case)}")
Comment on lines 44 to 47
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Mpi task count dropped 🐞 Bug ≡ Correctness

In toolchain/templates/oscar.mako the MPI path now invokes plain "srun" without using
nodes*tasks_per_node, so the configured task count no longer influences how many ranks are launched.
This can under-launch MPI ranks and make MPI runs behave differently than intended for the same
nodes/tasks_per_node settings.
Agent Prompt
## Issue description
Oscar’s batch template switched from `mpirun -np ${nodes*tasks_per_node}` to `srun` but no longer passes any task-count arguments derived from `nodes`/`tasks_per_node`. As a result, the configured MPI size is not enforced by the launcher.

## Issue Context
Other Slurm templates in this repo that use `srun` explicitly pass `--ntasks ${nodes*tasks_per_node}`.

## Fix Focus Areas
- toolchain/templates/oscar.mako[42-48]

## Suggested fix
Update the MPI branch to something like:
- `srun --ntasks ${nodes*tasks_per_node} "${target.get_install_binpath(case)}"`

(Optionally also align the SBATCH header to consistently use `--ntasks-per-node=${tasks_per_node}` if that’s the intended directive for this cluster.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

% endif

Expand Down
Loading