Skip to content

fix script to init EESSI for csh shells, fixes also the CI for csh and speeds up CI #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ocaisa merged 21 commits into from
Oct 9, 2025
Merged

fix script to init EESSI for csh shells, fixes also the CI for csh and speeds up CI #102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3e8d2fe
also test init for csh
truib Oct 5, 2025
fbd2689
install (t)csh too
truib Oct 5, 2025
5120a61
format csh file to follow syntax + 4 spaces indentation
truib Oct 5, 2025
a952a16
use backticks in csh + some polishing
truib Oct 5, 2025
5b5bc10
avoid some apt triggers
truib Oct 5, 2025
0b914fd
ensure trigger for man-db is not triggered earlier
truib Oct 5, 2025
b15b4f7
(t)csh uses slightly different shell redirection syntax
truib Oct 5, 2025
b56da62
using module alias in non-interactive csh is a little tricky
truib Oct 5, 2025
c315cdc
fix syntax glitch
truib Oct 5, 2025
05016f6
show test script to be run
truib Oct 5, 2025
e4fa51d
create, cat and test .cshrc in HOME dir
truib Oct 5, 2025
56cb0d7
use csh workaround for other tests
truib Oct 5, 2025
90f3e8d
discard some error messages + use here-string in last test too
truib Oct 5, 2025
bb8ac18
fix stderr redirection for .cshrc
truib Oct 5, 2025
056d6be
clean up some debug output and add new for test 4
truib Oct 5, 2025
0c60458
fix string quoting
truib Oct 5, 2025
a9c5dc3
discard output when loading EasyBuild
truib Oct 5, 2025
cac1100
try running in non-login shell
truib Oct 5, 2025
5b93cd8
clean up some debug output and rearrange output processing
truib Oct 5, 2025
9da3863
replacing tabs with spaces and final polishing/cleanup
truib Oct 5, 2025
40bc520
don't need to show workflow file
truib Oct 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 59 additions & 37 deletions .github/workflows/scripts/test_init_scripts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,59 +2,81 @@
export LMOD_PAGER=cat

if [ -z ${EESSI_VERSION} ] || [ ! -d /cvmfs/software.eessi.io/versions/${EESSI_VERSION} ]; then
echo "\$EESSI_VERSION has to be set to a valid EESSI version."
exit 1
echo "\$EESSI_VERSION has to be set to a valid EESSI version."
exit 1
fi

if [ -z ${EXPECTED_EASYBUILD_VERSION} ]; then
echo "\$EXPECTED_EASYBUILD_VERSION has to be set to an EasyBuild version that is expected to be available in EESSI version ${EESSI_VERSION}."
exit 1
echo "\$EXPECTED_EASYBUILD_VERSION has to be set to an EasyBuild version that is expected to be available in EESSI version ${EESSI_VERSION}."
exit 1
fi

# initialize assert framework
if [ ! -d assert.sh ]; then
echo "assert.sh not cloned."
echo ""
echo "run \`git clone https://github.com/lehmannro/assert.sh.git\`"
exit 1
echo "assert.sh not cloned."
echo ""
echo "run \`git clone https://github.com/lehmannro/assert.sh.git\`"
exit 1
fi
. assert.sh/assert.sh

TEST_SHELLS=("bash" "zsh" "fish" "ksh")
TEST_SHELLS=("bash" "zsh" "fish" "ksh" "csh")
SHELLS=$@

for shell in ${SHELLS[@]}; do
echo = | awk 'NF += (OFS = $_) + 100'
echo RUNNING TESTS FOR SHELL: $shell
echo = | awk 'NF += (OFS = $_) + 100'
echo = | awk 'NF += (OFS = $_) + 100'
echo RUNNING TESTS FOR SHELL: $shell
echo = | awk 'NF += (OFS = $_) + 100'
if [[ ! " ${TEST_SHELLS[*]} " =~ [[:space:]]${shell}[[:space:]] ]]; then
### EXCEPTION FOR CSH ###
echo -e "\033[33mWe don't now how to test the shell '$shell', PRs are Welcome.\033[0m"
### EXCEPTION FOR CSH ###
echo -e "\033[33mWe don't now how to test the shell '$shell', PRs are Welcome.\033[0m"
else
# TEST 1: Source Script and check Module Output
assert "$shell -c 'source init/lmod/$shell' 2>&1 " "Module for EESSI/$EESSI_VERSION loaded successfully"
# TEST 2: Check if module overviews first section is the loaded EESSI module
MODULE_SECTIONS=($($shell -c "source init/lmod/$shell 2>/dev/null; module ov 2>&1 | grep -e '---'"))
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/modules/all"
assert_raises 'echo "${MODULE_SECTIONS[1]}" | grep -E "$PATTERN"'
# TEST 3: Check if module overviews second section is the EESSI init module
assert "echo ${MODULE_SECTIONS[4]}" "/cvmfs/software.eessi.io/versions/$EESSI_VERSION/init/modules"
# Test 4: Load EasyBuild module and check version
# eb --version outputs: "This is EasyBuild 5.1.1 (framework: 5.1.1, easyblocks: 5.1.1) on host ..."
command="$shell -c 'source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; eb --version | cut -d \" \" -f4'"
assert "$command" "$EXPECTED_EASYBUILD_VERSION"
# Test 5: Load EasyBuild module and check path
EASYBUILD_PATH=$($shell -c "source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; which eb")
# escape the dots in ${EASYBUILD_VERSION}
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/software/EasyBuild/${EXPECTED_EASYBUILD_VERSION//./\\.}/bin/eb"
echo "$EASYBUILD_PATH" | grep -E "$PATTERN"
assert_raises 'echo "$EASYBUILD_PATH" | grep -E "$PATTERN"'

#End Test Suite
assert_end "source_eessi_$shell"
fi
done
# TEST 1: Source Script and check Module Output
expected="Module for EESSI/$EESSI_VERSION loaded successfully"
assert "$shell -c 'source init/lmod/$shell' 2>&1 " "${expected}"

# TEST 2: Check if module overviews first section is the loaded EESSI module
if [ "$shell" = "csh" ]; then
# module is defined as alias, but aliases are only retained in interactive
# shells we work around this by creating a .cshrc file (which sources the
# init script), and then simply run the remaining commands
echo "source init/lmod/$shell" > ~/.cshrc
MODULE_SECTIONS=($($shell -c "module ov" 2>&1 | grep -e '---'))
else
MODULE_SECTIONS=($($shell -c "source init/lmod/$shell 2>/dev/null; module ov 2>&1 | grep -e '---'"))
fi
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/modules/all"
assert_raises 'echo "${MODULE_SECTIONS[1]}" | grep -E "$PATTERN"'

# TEST 3: Check if module overviews second section is the EESSI init module
assert "echo ${MODULE_SECTIONS[4]}" "/cvmfs/software.eessi.io/versions/$EESSI_VERSION/init/modules"

# TEST 4: Load EasyBuild module and check version
# eb --version outputs: "This is EasyBuild 5.1.1 (framework: 5.1.1, easyblocks: 5.1.1) on host ..."
if [ "$shell" = "csh" ]; then
echo "source init/lmod/$shell" > ~/.cshrc
command="$shell -c 'module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; eb --version' | cut -d \" \" -f4"
else
command="$shell -c 'source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; eb --version | cut -d \" \" -f4'"
fi
assert "$command" "$EXPECTED_EASYBUILD_VERSION"

# TEST 5: Load EasyBuild module and check path
if [ "$shell" = "csh" ]; then
echo "source init/lmod/$shell" > ~/.cshrc
EASYBUILD_PATH=$($shell -c "module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; which eb")
else
EASYBUILD_PATH=$($shell -c "source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; which eb")
fi
# escape the dots in ${EASYBUILD_VERSION}
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/software/EasyBuild/${EXPECTED_EASYBUILD_VERSION//./\\.}/bin/eb"
echo "$EASYBUILD_PATH" | grep -E "$PATTERN"
assert_raises 'echo "$EASYBUILD_PATH" | grep -E "$PATTERN"'

# End Test Suite
assert_end "source_eessi_$shell"
fi
done

# RESET PAGER
export LMOD_PAGER=
8 changes: 7 additions & 1 deletion .github/workflows/tests_init_module.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ jobs:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Avoid processing man-db trigger
Copy link
Member

@ocaisa ocaisa Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, I'm going to steal this for the EESSI GitHub Action

Copy link
Contributor

@MaKaNu MaKaNu Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the slow-down caused by the sudo apt install zsh ksh fish tcsh below?

Copy link
Member

@ocaisa ocaisa Oct 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a general thing, slow is relative, it typically 20-30 second for updating man-db, which we will never use so might as well disable it.

run: |
# avoid processing trigger for man-db (seems to slow down installation)
echo "set man-db/auto-update false" | sudo debconf-communicate
sudo dpkg-reconfigure man-db

- name: Mount EESSI CernVM-FS pilot repository
uses: cvmfs-contrib/github-action-cvmfs@55899ca74cf78ab874bdf47f5a804e47c198743c # v4.0
with:
Expand All @@ -46,7 +52,7 @@ jobs:
- name: Install missing shells
run: |
sudo apt update
sudo apt install zsh ksh fish
sudo apt install zsh ksh fish tcsh
echo "# INIT ZSH" > ~/.zshrc

- name: Run tests for available shells
Expand Down
23 changes: 14 additions & 9 deletions init/lmod/csh
View file
Open in desktop
Copy link
Contributor

@MaKaNu MaKaNu Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trz42 it seems you are way more talented than me, to handle csh error message.

After all the "Illegal variable name." and "then: then/endif not found" error messages, I am happy to see this script finally working correctly.

Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
# Choose an EESSI CVMFS repository
if (! $?EESSI_CVMFS_REPO) then; set EESSI_CVMFS_REPO = "/cvmfs/software.eessi.io"; endif
if (! $?EESSI_CVMFS_REPO) then
set EESSI_CVMFS_REPO = "/cvmfs/software.eessi.io"
endif
# Choose an EESSI version
setenv EESSI_VERSION_DEFAULT "__EESSI_VERSION_DEFAULT__"
if (! $?EESSI_VERSION) then; set EESSI_VERSION = "${EESSI_VERSION_DEFAULT}"; endif
if (! $?EESSI_VERSION) then
set EESSI_VERSION = "${EESSI_VERSION_DEFAULT}"
endif
# Path to top-level module tree
setenv MODULEPATH "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/modules"
source "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/$(uname -m)/usr/share/Lmod/init/csh"
source "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/`uname -m`/usr/share/Lmod/init/csh"

if (! $?__Init_Default_Modules ) then
setenv __Init_Default_Modules 1;
setenv __Init_Default_Modules 1

## ability to predefine elsewhere the default list
if (! $?LMOD_SYSTEM_DEFAULT_MODULES) then; setenv LMOD_SYSTEM_DEFAULT_MODULES "EESSI/$EESSI_VERSION"; endif
module --initial_load --no_redirect restore
# ability to predefine elsewhere the default list
if (! $?LMOD_SYSTEM_DEFAULT_MODULES) then
setenv LMOD_SYSTEM_DEFAULT_MODULES "EESSI/${EESSI_VERSION}"
endif
module --initial_load --no_redirect restore
else
module refresh
module refresh
endif