Problem
Some models' chat template are like this: "{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n<think>\n' }}". But API client won't see that <think> tag thus breaking parsing and UI.
Solution
Maybe make it optional, but add the ability to detect this and "compensate" that tag at the start of the response.
Acknowledgements