Skip to content

feat(ai-proxy): add defaults field for fallback model options#12895

Open
sihyeonn wants to merge 3 commits intoapache:masterfrom
sihyeonn:feat/sh-defaults
Open

feat(ai-proxy): add defaults field for fallback model options#12895
sihyeonn wants to merge 3 commits intoapache:masterfrom
sihyeonn:feat/sh-defaults

Conversation

@sihyeonn
Copy link

@sihyeonn sihyeonn commented Jan 13, 2026

Description

Add defaults field to ai-proxy and ai-proxy-multi plugins to provide fallback values for model options. Currently, options field always overrides user request values. This PR introduces a new defaults field that only applies when the user hasn't set a value in their request. Behavior:

  • options: Always override user values (e.g., enforce specific model)
  • defaults: Apply only when not present in user request (e.g., default max_tokens)

Example configuration:

{
"options": { "model": "gpt-4" },
"defaults": { "max_tokens": 512, "temperature": 0.7 }
}

User Request Result
{"model": "gpt-3.5"} {"model": "gpt-4", "max_tokens": 512, "temperature": 0.7}
{"max_tokens": 100} {"model": "gpt-4", "max_tokens": 100, "temperature": 0.7}

Which issue(s) this PR fixes:

Fixes #

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Separate options and defaults behavior:
- options: always override user request values
- defaults: apply only when not set in user request

This allows more flexible configuration where administrators can
enforce certain values (via options) while providing sensible
defaults for optional parameters.

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jan 13, 2026
Copy link
Contributor

@Baoyuantop Baoyuantop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Supplement Documentation - Update ai-proxy.md and ai-proxy.md, adding documentation for the defaults field
  2. Supplement ai-proxy Plugin Tests - Add test cases to ai-proxy.openai-compatible.t or relevant test files
  3. Link to Related Issues or Supplement Requirement Context - Specify the source of use cases for this feature

@Baoyuantop Baoyuantop added the wait for update wait for the author's response in this issue/PR label Jan 26, 2026
- Add defaults field description to ai-proxy.md (en/zh)
- Add instances.defaults field description to ai-proxy-multi.md (en/zh)
- Add test cases for defaults field in ai-proxy.openai-compatible.t
@sihyeonn
Copy link
Author

Thanks for the review! I've addressed your feedback:

  1. Added documentation for the defaults field in both ai-proxy.md and ai-proxy-multi.md (en/zh)
  2. Added test cases to ai-proxy.openai-compatible.t
  3. This feature addresses a common use case where admins want to set fallback values (like max_tokens, temperature) without overriding user preferences. Unlike options which always overrides, defaults only applies when users don't specify values in their requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files. user responded wait for update wait for the author's response in this issue/PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants