Skip to content

Fix stop parameter check to allow explicit None value#2482

Open
sam-baumann wants to merge 1 commit intoMaartenGr:masterfrom
sam-baumann:master
Open

Fix stop parameter check to allow explicit None value#2482
sam-baumann wants to merge 1 commit intoMaartenGr:masterfrom
sam-baumann:master

Conversation

@sam-baumann
Copy link
Copy Markdown

Previously, not self.generator_kwargs.get("stop") would overwrite a user-supplied stop=None (which disables the stop sequence). Use "stop" not in to only set the default when the key is absent.

Thinking models, at least gemma4:e2b in Ollama, will stop their reasoning chain on \n even if they shouldn't complete, so this gives a way to overwrite that

What does this PR do?

checks if "stop" is a member of the generator_kwargs dict instead of just checking it as a boolean, as passing in None caused it to evaluate to False and it sets stop as \n anyway

Fixes #2481

Before submitting

  • This PR fixes a typo or improves the docs (if yes, ignore all other checks!).
  • Did you read the contributor guideline?
  • Was this discussed/approved via a Github issue? Please add a link to it if that's the case. Thinking models in ollama exit early #2481 submitted alongside this PR
  • Did you make sure to update the documentation with your changes (if applicable)? N/A
  • Did you write any new necessary tests? N/A

Previously, `not self.generator_kwargs.get("stop")` would overwrite a
user-supplied `stop=None` (which disables the stop sequence). Use `"stop"
not in` to only set the default when the key is absent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Thinking models in ollama exit early

1 participant