AMD_instructions.txt

Setting up the Ollama container and downloading models

docker run -d --restart always --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

docker exec -it ollama bash

ollama pull {model name}



Running OpenWebUI (Not essential, but I would recommend it)

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main



Setting up the ROCM docker container (thanks to Trade Mamba for the initial instruction https://medium.com/@trademamba/pytorch-using-amd-gpus-for-the-average-data-science-joe-389bf0878e95)

#You can use this for initial setup, then stop and start it using just the name. Adding a volume means that your work will be preserved, even if the container needs to be updated or rebuilt from scratch. When you build the container, and once you're logged into the Jupyter server, there will be a MachineLearning folder, which is where all of our files will be saved, though if you rebuild the container you will need to re-install any packages.

docker volume create rocmjupytervolume

docker run -d \
    -p 8888:8888 \
    --cap-add=SYS_PTRACE \
    --security-opt seccomp=unconfined \
    --device=/dev/kfd \
    --device=/dev/dri \
    --group-add video \
    --net=host \
    --ipc=host \
    --shm-size 8G \
    --volume rocmjupytervolume:/MachineLearning \
    --name rocmjupyter \
    rocm/pytorch:latest \
    bash -c "sleep infinity"

docker exec -it rocmjupyter bash

#This only needs to be run once
pip install jupyterlab

#I'd suggest adding the following to a bash script, so you can quickly get it running
nohup jupyter-lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root --NotebookApp.token='mytoken'

#The link your browser will use to connect
http://<your-server-ip>:8888/?token=mytoken