|
1 | 1 | # Jupyter Notebook with Conda/Singularity in OOD |
| 2 | +Please note that Greene and Torch organize overlay files and Singularity images in different directories. |
2 | 3 |
|
| 4 | +## Greene Directories |
| 5 | +Overlay Files: |
| 6 | +``` |
| 7 | +/scratch/work/public/overlay-fs-ext3/ |
| 8 | +``` |
| 9 | +Singularity Files: |
| 10 | +``` |
| 11 | +/scratch/work/public/singularity/ |
| 12 | +``` |
| 13 | + |
| 14 | +## Torch Directories |
| 15 | +Overlay Files: |
| 16 | +``` |
| 17 | +/share/apps/overlay-fs-ext3 |
| 18 | +``` |
| 19 | +Singularity Files: |
| 20 | +``` |
| 21 | +/share/apps/images/ |
| 22 | +``` |
3 | 23 | ## OOD + Singularity + conda |
4 | 24 | This page describes how to use your Singularity with conda environment in OOD GUI at Torch. |
5 | 25 |
|
@@ -38,16 +58,16 @@ The above code automatically makes your environment look for the default shared |
38 | 58 | ```bash |
39 | 59 | [NetID@log-1 ~]$ mkdir /scratch/$USER/my_env |
40 | 60 | [NetID@log-1 ~]$ cd /scratch/$USER/my_env |
41 | | -[NetID@log-1 my_env]$ cp -rp /scratch/work/public/overlay-fs-ext3/overlay-15GB-500K.ext3.gz . |
| 61 | +[NetID@log-1 my_env]$ cp -rp /share/apps/overlay-fs-ext3/overlay-15GB-500K.ext3.gz . |
42 | 62 | [NetID@log-1 my_env]$ gunzip overlay-15GB-500K.ext3.gz |
43 | 63 | ``` |
44 | 64 | Above we used the overlay file `overlay-15GB-500K.ext3.gz` which will contain all of the installed packages. There are more optional overlay files. You can find instructions on the following pages: [Singularity with Conda](../07_containers/03_singularity_with_conda.md), [Squash File System and Singularity](../07_containers/04_squash_file_system_and_singularity.md). |
45 | 65 |
|
46 | 66 | ### Launch Singularity Environment for Installation |
47 | 67 | ```bash |
48 | | -[NetID@log-1 ~]$ singularity exec --overlay /scratch/$USER/my_env/overlay-15GB-500K.ext3:rw /scratch/work/public/singularity/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif /bin/bash |
| 68 | +[NetID@log-1 ~]$ singularity exec --overlay /scratch/$USER/my_env/overlay-15GB-500K.ext3:rw /share/apps/images/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif /bin/bash |
49 | 69 | ``` |
50 | | -Above we used the Singularity OS image `cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif` which provides the base operating system environment for the conda environment. There are other Singularity OS images available at `/scratch/work/public/singularity` |
| 70 | +Above we used the Singularity OS image `cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif` which provides the base operating system environment for the conda environment. There are other Singularity OS images available at `/share/apps/images` |
51 | 71 |
|
52 | 72 | Launching Singularity with the `--overlay` flag mounts the overlay file to a new directory: `/ext3` - you will notice that when not using Singularity `/ext3` is not available. |
53 | 73 |
|
@@ -125,7 +145,7 @@ Singularity> exit |
125 | 145 |
|
126 | 146 | # wait to be assigned a node |
127 | 147 |
|
128 | | -[NetID@cm001 my_env]$ singularity exec --overlay /scratch/$USER/my_env/overlay-15GB-500K.ext3:rw /scratch/work/public/singularity/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif /bin/bash |
| 148 | +[NetID@cm001 my_env]$ singularity exec --overlay /scratch/$USER/my_env/overlay-15GB-500K.ext3:rw /share/apps/images/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif /bin/bash |
129 | 149 |
|
130 | 150 | Singularity> source /ext3/env.sh |
131 | 151 | # activate the environment |
@@ -153,11 +173,11 @@ At the bottom of the file we have the template singularity command. |
153 | 173 | ```bash |
154 | 174 | singularity exec $nv \ |
155 | 175 | --overlay /scratch/$USER/my_env/overlay-15GB-500K.ext3:ro \ |
156 | | - /scratch/work/public/singularity/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif \ |
| 176 | + /share/apps/images/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif \ |
157 | 177 | /bin/bash -c "source /ext3/env.sh; $cmd $args" |
158 | 178 | ``` |
159 | 179 | :::warning |
160 | | -If you used a different overlay (/scratch/$USER/my_env/overlay-15GB-500K.ext3 shown above) or .sif file (/scratch/work/public/singularity/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif shown above), you MUST change those lines in the command above to the files you used. |
| 180 | +If you used a different overlay (/scratch/$USER/my_env/overlay-15GB-500K.ext3 shown above) or .sif file (/share/apps/images/cuda12.3.2-cudnn9.0.0-ubuntu-22.04.4.sif shown above), you MUST change those lines in the command above to the files you used. |
161 | 181 | ::: |
162 | 182 |
|
163 | 183 | Edit the default `kernel.json` file by setting PYTHON_LOCATION and KERNEL_DISPLAY_NAME using a text editor like nano/vim. |
|
0 commit comments