Skip to content

Commit 0f3750f

Browse files
committed
update RPC docs
1 parent e93d69c commit 0f3750f

1 file changed

Lines changed: 12 additions & 9 deletions

File tree

docs/rpc.md

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Building and Using the RPC Server with `stable-diffusion.cpp`
22

3-
This guide covers how to build a version of the RPC server from `llama.cpp` that is compatible with your version of `stable-diffusion.cpp` to manage multi-backends setups. RPC allows you to offload specific model components to a remote server.
3+
This guide covers how to build a version of [the RPC server from `llama.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md) that is compatible with your version of `stable-diffusion.cpp` to manage multi-backends setups. RPC allows you to offload specific model components to a remote server.
44

55
> **Note on Model Location:** The model files (e.g., `.safetensors` or `.gguf`) remain on the **Client** machine. The client parses the file and transmits the necessary tensor data and computational graphs to the server. The server does not need to store the model files locally.
66
@@ -16,7 +16,7 @@ cmake .. \
1616
cmake --build . --config Release -j $(nproc)
1717
```
1818

19-
> **Note:** Ensure you add the other flags you would normally use (e.g., `-DSD_VULKAN=ON`, `-DSD_CUDA=ON`, `-DSD_HIPBLAS=ON`, or `-DGGML_METAL=ON`), for more information about building `stable-diffusion.cpp` from source, please refer to the `build.md` documentation.
19+
> **Note:** Ensure you add the other flags you would normally use (e.g., `-DSD_VULKAN=ON`, `-DSD_CUDA=ON`, `-DSD_HIPBLAS=ON`, or `-DGGML_METAL=ON`), for more information about building `stable-diffusion.cpp` from source, please refer to the [docs/build.md](docs%build.md) documentation.
2020
2121
## 2. Ensure `llama.cpp` is at the correct commit
2222

@@ -39,8 +39,7 @@ cmake --build . --config Release -j $(nproc)
3939
cd llama.cpp
4040
git checkout $HASH
4141
```
42-
43-
To save on download time and storage, you can use a shallow clone to download only the target commit:
42+
To save on download time and storage, you can use a shallow clone to download only the target commit:
4443
```bash
4544
mkdir -p llama.cpp
4645
cd llama.cpp
@@ -54,7 +53,7 @@ To save on download time and storage, you can use a shallow clone to download on
5453

5554
The RPC server acts as the worker. You must explicitly enable the **backend** (the hardware interface, such as CUDA for Nvidia, Metal for Apple Silicon, or Vulkan) when building, otherwise the server will default to using only the CPU.
5655

57-
To find the correct flags, refer to the official documentation for the `llama.cpp` repository.
56+
To find the correct flags, refer to the official documentation for the [`llama.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) repository.
5857

5958
> **Crucial:** You must include the compiler flags required to satisfy the API compatibility with `stable-diffusion.cpp` (`-DGGML_MAX_NAME=128`). Without this flag, `GGML_MAX_NAME` will default to `64` for the server, and data transfers between the client and server will fail. Of course, `-DGGML_RPC` must also be enabled.
6059
>
@@ -167,18 +166,22 @@ Example: A main machine (192.168.1.10) with 3 GPUs, with one GPU running CUDA an
167166

168167
**Terminal 1 (CUDA):**
169168
```bash
170-
# Linux / macOS / WSL
169+
# Linux / WSL
171170
export CUDA_VISIBLE_DEVICES=0
172-
./rpc-server-cuda --host 0.0.0.0
171+
cd ./build_cuda/bin/Release
172+
./rpc-server --host 0.0.0.0
173173

174174
# Windows PowerShell
175175
$env:CUDA_VISIBLE_DEVICES="0"
176-
./rpc-server-cuda --host 0.0.0.0
176+
cd .\build_cuda\bin\Release
177+
./rpc-server --host 0.0.0.0
177178
```
178179

179180
**Terminal 2 (Vulkan):**
180181
```bash
181-
./rpc-server-vulkan --host 0.0.0.0 --port 50053 -d Vulkan1,Vulkan2
182+
cd ./build_vulkan/bin/Release
183+
# ignore the first GPU (used by CUDA server)
184+
./rpc-server --host 0.0.0.0 --port 50053 -d Vulkan1,Vulkan2
182185
```
183186

184187
**On the second machine:**

0 commit comments

Comments
 (0)