Skip to content

chore(versions): vLLM 0.19.0#839

Open
doringeman wants to merge 1 commit intodocker:mainfrom
doringeman:vllm-0.19.0
Open

chore(versions): vLLM 0.19.0#839
doringeman wants to merge 1 commit intodocker:mainfrom
doringeman:vllm-0.19.0

Conversation

@doringeman
Copy link
Copy Markdown
Contributor

Will help #832 for vLLM on Linux and Windows.
Not yet for vllm-metal on macOS.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Signed-off-by: Dorin Geman <dorin.geman@docker.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the vLLM version from 0.17.0 to 0.19.0. A critical issue was identified where this upgrade breaks the backend because the --use-v2-block-manager flag, which is still explicitly appended in the Go implementation, has been removed in vLLM 0.19.0 and will cause process startup failures.

@@ -1,5 +1,5 @@
GO_VERSION=1.25
VLLM_VERSION=0.17.0
VLLM_VERSION=0.19.0
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Upgrading to vLLM 0.19.0 is a breaking change for the current backend implementation. vLLM 0.19.0 removed the --use-v2-block-manager flag (as the V2 block manager is now the default and only option), but the code in pkg/inference/backends/vllm/vllm.go:172 still explicitly appends this flag when speculative decoding is enabled. This will cause the vLLM process to fail on startup with an 'unrecognized arguments' error on Linux.

You should update the Go backend to handle this flag conditionally based on the version, or remove it if the older version (0.17.1) used for macOS also supports the V2 block manager without the explicit flag. Merging this version bump without the corresponding code change will break speculative decoding functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant