Skip to content

Conversation

@SkyeYoung
Copy link
Member

@SkyeYoung SkyeYoung commented Jan 22, 2026

Description

Refer to the changes in #12565 to add support for vertex ai.

Already tested in real world:

# config.yaml
deployment:
  role: data_plane
  role_data_plane:
    config_provider: yaml
routes:
  - id: 1
    uri: /anything
    plugins:
      ai-proxy:
        ssl_verify: false
        provider: vertex-ai
        provider_conf:
          project_id: evident-xxx
          region: us-central1
        auth:
          gcp:
            service_account_json: |
              {
                ...
              }
        options:
          model: google/gemini-2.5-flash
#END

❯ curl -X POST http://localhost:9080/anything -H "Content-Type: application/json" -d '{"messages": [{"role": "user", "content": "Hello"}]}'
{"model":"google/gemini-2.5-flash","id":"IthxaZicNsq4qsMPjJeJqAI","object":"chat.completion","usage":{"completion_tokens_details":{"reasoning_tokens":17},"extra_properties":{"google":{"traffic_type":"ON_DEMAND"}},"total_tokens":28,"prompt_tokens":1,"completion_tokens":10},"created":1769068578,"system_fingerprint":"","choices":[{"index":0,"finish_reason":"stop","message":{"content":"Hello there! How can I help you today?","role":"assistant"},"logprobs":null}]}%                         

Which issue(s) this PR fixes:

Fixes #

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. enhancement New feature or request labels Jan 22, 2026
membphis
membphis previously approved these changes Jan 23, 2026
Copy link
Member

@membphis membphis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one minor issue

local function fetch_gcp_access_token(ctx, name, gcp_conf)
local key = core.lrucache.plugin_ctx_id(ctx, name)
local access_token = gcp_access_token_cache:get(key)
if not access_token then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another style:

if access_token then
    return access_token
end

-- generate access token
... ...

Copy link
Member Author

@SkyeYoung SkyeYoung Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check it when I have time.

AlinsRan
AlinsRan previously approved these changes Jan 23, 2026
@SkyeYoung SkyeYoung dismissed stale reviews from AlinsRan and membphis via c2e1a40 January 25, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants