feat(ai-proxy): add native support for Anthropic provider #12924

uniprewan · 2026-01-19T12:43:25Z

Background

Currently, the APISIX AI Proxy plugin offers OpenAI compatibility. However, native support for the Anthropic Messages API is incomplete, often resulting in simple request forwarding without leveraging Anthropic's specific protocol features (like the structured Messages API and dedicated system prompt handling).

This feature is crucial for users who want to:

Utilize Anthropic's unique models (e.g., Claude 3 family) with their native protocol structure.
Ensure proper handling of system prompts and message roles as defined by the Anthropic Messages API.
Improve the efficiency and reliability of the Anthropic provider integration within APISIX.

What I Did

This PR introduces native support for the Anthropic provider by implementing a refactored architecture with abstraction.

Architecture Refactoring: Introduced a common base driver (ai-driver-base.lua) to provide shared functionalities for all AI drivers.
Anthropic Native Driver: Implemented a dedicated anthropic.lua driver that inherits from the base class.
Protocol Translation: Implemented comprehensive protocol translation logic for requests, responses, and SSE streaming.
OpenAI Refactoring: Refactored openai-base.lua to inherit from the new base driver for consistency.

Checklist

I have followed the APISIX contribution guidelines.
I have verified the changes locally.
I have added documentation and test cases.

Baoyuantop · 2026-01-20T02:00:55Z

Hi @uniprewan, could you explain in detail the background of your PR submission? Is the existing anthropic provider not working?

Copilot

Pull request overview

This PR attempts to add native Anthropic protocol support to the ai-proxy plugin by introducing a common base driver pattern. However, the implementation has critical flaws that prevent it from working.

Changes:

Introduced a new base driver (ai-driver-base.lua) intended to share common logic across AI providers
Refactored openai-base.lua to use the new base driver pattern
Implemented a new anthropic.lua driver with protocol translation for Anthropic's Messages API
Moved Anthropic from the middle to the end of the provider list in schema.lua

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 29 comments.

File	Description
apisix/plugins/ai-drivers/schema.lua	Reordered provider list, moving Anthropic to the end
apisix/plugins/ai-drivers/ai-driver-base.lua	New base driver with critical implementation flaws
apisix/plugins/ai-drivers/openai-base.lua	Refactored to use base driver but breaks existing functionality
apisix/plugins/ai-drivers/anthropic.lua	New Anthropic driver with protocol translation issues

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-20T03:04:32Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+    local res, err = httpc:request({
+        method = "POST",
+        url = upstream_url,
+        headers = headers,
+        body = core.json.encode(request_table ),
+        ssl_verify = false, -- 生产环境应为 true
+        timeout = conf.timeout or 60000,


The timeout is only used in the request() call but not in the HTTP client setup. The original implementation calls httpc:set_timeout(conf.timeout) before making the connection. The resty.http library's request() method timeout parameter may not work as expected without proper client initialization.

Suggested change

local res, err = httpc:request({

method = "POST",

url = upstream_url,

headers = headers,

body = core.json.encode(request_table ),

ssl_verify = false, -- 生产环境应为 true

timeout = conf.timeout or 60000,

httpc:set_timeout(conf.timeout or 60000)

local res, err = httpc:request({

method = "POST",

url = upstream_url,

headers = headers,

body = core.json.encode(request_table ),

ssl_verify = false, -- 生产环境应为 true

Copilot · 2026-01-20T03:04:32Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+    -- 4. 处理响应
+    local is_stream = request_table.stream
+    local content_type = res.headers["Content-Type"]
+
+    if is_stream and core.string.find(content_type, "text/event-stream") then
+        -- 流式响应处理
+        return self:handle_stream_response(ctx, res, conf)
+    else
+        -- 非流式响应处理
+        return self:handle_non_stream_response(ctx, res, conf)
+    end


The request method doesn't handle retry scenarios for 429 (rate limit) or 5xx errors. The original openai-base.lua implementation returns the status code for these errors to allow the ai-proxy base.lua to retry with another instance. Without this, the retry mechanism in ai-proxy-multi won't work properly.

Copilot · 2026-01-20T03:04:32Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+function _M.request(self, ctx, conf, request_table, extra_opts)
+    -- 1. 协议转换（如果驱动提供了 transform_request）
+    if self.transform_request then
+        request_table = self.transform_request(request_table)
+    end
+
+    -- 2. 构造上游请求
+    local upstream_url = self.scheme .. "://" .. self.host .. ":" .. self.port .. self.path
+    local headers = {
+        ["Host"] = self.host,
+        ["Content-Type"] = "application/json",
+        -- 认证头由具体驱动在 transform_request 中添加或在 conf 中获取
+    }
+
+    -- 3. 发送请求
+    local httpc = http.new( )
+    local res, err = httpc:request({
+        method = "POST",
+        url = upstream_url,
+        headers = headers,
+        body = core.json.encode(request_table ),
+        ssl_verify = false, -- 生产环境应为 true
+        timeout = conf.timeout or 60000,
+    })
+
+    if not res then
+        core.log.error("failed to send request to LLM server: ", err)
+        return self:handle_error(err)
+    end
+
+    -- 4. 处理响应
+    local is_stream = request_table.stream
+    local content_type = res.headers["Content-Type"]
+
+    if is_stream and core.string.find(content_type, "text/event-stream") then
+        -- 流式响应处理
+        return self:handle_stream_response(ctx, res, conf)
+    else
+        -- 非流式响应处理
+        return self:handle_non_stream_response(ctx, res, conf)
+    end
+end
+
+-- 处理非流式响应
+function _M.handle_non_stream_response(self, ctx, res, conf)
+    local raw_res_body = res:read_body()
+    if not raw_res_body then
+        core.log.warn("failed to read response body: ", res.err)
+        return self:handle_error(res.err)
+    end
+
+    -- 协议转换（如果驱动提供了 transform_response）
+    if self.transform_response then
+        raw_res_body = self.transform_response(raw_res_body)
+    end
+
+    -- 设置响应头和状态码
+    core.response.set_header(ctx, "Content-Type", "application/json")
+    core.response.set_status(ctx, res.status)
+    core.response.set_body(ctx, raw_res_body)
+    core.response.send_response(ctx)
+end
+
+-- 处理流式响应
+function _M.handle_stream_response(self, ctx, res, conf)
+    core.response.set_header(ctx, "Content-Type", "text/event-stream")
+    core.response.set_status(ctx, res.status)
+    core.response.send_http_header(ctx )
+
+    local body_reader = res.body_reader
+    local chunk
+    while true do
+        chunk, err = body_reader()
+        if not chunk then
+            break
+        end
+
+        -- 协议转换（如果驱动提供了 process_sse_chunk）
+        if self.process_sse_chunk then
+            chunk = self.process_sse_chunk(chunk)
+        end
+
+        core.response.write(ctx, chunk)
+    end
+
+    if err then
+        core.log.error("failed to read stream body: ", err)
+    end
+
+    core.response.close(ctx)
+end


The ai-driver-base.lua doesn't set any of the context variables that are used for metrics and logging, such as ctx.llm_request_start_time, ctx.var.llm_time_to_first_token, ctx.var.apisix_upstream_response_time, ctx.ai_token_usage, etc. These are essential for the logging and prometheus metrics functionality. The original implementation properly tracks all these metrics.

Copilot · 2026-01-20T03:04:33Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+
+local core = require("apisix.core")
+local plugin = require("apisix.plugin")
+local http = require("resty.http" )


There is an extra space before the closing parenthesis in the require statement. This should be 'require("resty.http")' without the extra space.

Suggested change

local http = require("resty.http" )

local http = require("resty.http")

Copilot · 2026-01-20T03:04:33Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+local http = require("resty.http" )
+local url = require("socket.url")
+-- 假设 sse 模块存在于 apisix.plugins.ai-drivers.sse
+local sse = require("apisix.plugins.ai-drivers.sse") 


There is an extra space before the closing parenthesis. This is inconsistent with the rest of the codebase.

Suggested change

local http = require("resty.http" )

local url = require("socket.url")

-- 假设 sse 模块存在于 apisix.plugins.ai-drivers.sse

local sse = require("apisix.plugins.ai-drivers.sse")

local http = require("resty.http")

local url = require("socket.url")

-- 假设 sse 模块存在于 apisix.plugins.ai-drivers.sse

local sse = require("apisix.plugins.ai-drivers.sse")

Copilot · 2026-01-20T03:04:38Z

apisix/plugins/ai-drivers/openai-base.lua

+        process_sse_chunk = _M.process_sse_chunk,
+        parse_token_usage = _M.parse_token_usage,
+        -- transform_request 和 transform_response 在 OpenAI 兼容层中通常不需要
+    } )


There is an extra space before the closing parenthesis. This is inconsistent with the rest of the codebase.

Suggested change

} )

})

Copilot · 2026-01-20T03:04:38Z

apisix/plugins/ai-drivers/openai-base.lua


-
-return _M
+return _M.new({})


The refactored openai-base.lua returns _M.new({}) at the end, which means every OpenAI-compatible driver that requires this module will get the same singleton instance with empty opts. This breaks the pattern where each driver (openai.lua, deepseek.lua, etc.) calls .new() with their specific host/port/path configuration. This line should return just _M, not _M.new({}).

Suggested change

return _M.new({})

return _M

Copilot · 2026-01-20T03:04:38Z

apisix/plugins/ai-drivers/anthropic.lua

+return driver_base.new({
+    host = "api.anthropic.com",
+    port = 443,
+    path = "/v1/messages",
+    transform_request = _M.transform_request,
+    transform_response = _M.transform_response,
+    process_sse_chunk = _M.process_sse_chunk
+})


The Anthropic driver is missing the validate_request method which is called by ai-proxy/base.lua. This will cause a runtime error when the Anthropic provider is used. The validate_request method needs to be implemented or the driver needs to properly inherit it from the base driver.

Copilot · 2026-01-20T03:04:39Z

apisix/plugins/ai-drivers/ai-driver-base.lua

+        method = "POST",
+        url = upstream_url,
+        headers = headers,
+        body = core.json.encode(request_table ),


There is an extra space before the closing parenthesis. This is inconsistent with the rest of the codebase.

Copilot · 2026-01-20T03:04:39Z

apisix/plugins/ai-drivers/anthropic.lua

+function _M.process_sse_chunk(chunk)
+    local events = sse.decode(chunk)
+    local out = {}
+
+    for _, e in ipairs(events) do
+        if e.type == "message" then
+            local d = core.json.decode(e.data)
+            if d.type == "content_block_delta" then
+                -- 转换为 OpenAI 兼容的流式格式
+                table.insert(out, "data: " .. core.json.encode({
+                    choices = {
+                        {
+                            delta = {
+                                content = d.delta.text
+                            }
+                        }
+                    }
+                }) .. "\n")
+            elseif d.type == "message_stop" then
+                table.insert(out, "data: [DONE]\n")
+            end
+        end


The process_sse_chunk function doesn't handle JSON decode errors properly. If core.json.decode(e.data) fails at line 54, the variable 'd' will be nil but the code continues to access d.type without checking, which will cause a runtime error.

uniprewan · 2026-01-20T06:44:30Z

你好@uniprewan您能否详细说明一下您提交公关稿的背景？现有的慈善机构是否无法正常工作？

Hi @Baoyuantop,
Great question! Currently, the ai-proxy plugin's support for Anthropic is limited to simple request forwarding. It doesn't handle the protocol differences between OpenAI and Anthropic natively.
For example, Anthropic's Messages API requires a separate system field for system prompts and has different message role structures. My PR introduces a native driver that performs proper protocol translation, allowing users to use Anthropic models seamlessly with OpenAI-compatible requests.
I've also refactored the driver architecture to make it easier to add other providers in the future. I've just updated the PR with English comments, documentation, and test cases. Thanks!

Baoyuantop · 2026-01-22T03:19:23Z

Hi @uniprewan, the current code introduces many breaking changes that may cause existing AI providers to become inoperable. Please maintain the existing implementation or refactor it in a fully backward-compatible manner, ensuring that all CI operations pass after the code changes.

uniprewan · 2026-01-22T06:32:21Z

Hi @uniprewan, could you explain in detail the background of your PR submission? Is the existing anthropic provider not working?

feat(ai-proxy): add native support for Anthropic provider

7001773

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Jan 19, 2026

Baoyuantop marked this pull request as draft January 20, 2026 02:01

moonming requested a review from Copilot January 20, 2026 02:54

Copilot started reviewing on behalf of moonming January 20, 2026 02:55 View session

Copilot AI reviewed Jan 20, 2026

View reviewed changes

uniprewan added 2 commits January 20, 2026 14:19

docs: add anthropic driver doc and test cases

a948e94

chore: translate all comments to English

86f03ba

uniprewan closed this Jan 20, 2026

uniprewan reopened this Jan 20, 2026

fix: correct openai-base implementation and inheritance

2f45c9f

uniprewan closed this Jan 20, 2026

uniprewan added 2 commits January 20, 2026 15:02

refactor: final clean up and fix schema

f6d9726

style: remove unused import and use ngx.time()

61da135

uniprewan reopened this Jan 20, 2026

test: add comprehensive test cases and clean up comments

c677ccb

uniprewan marked this pull request as ready for review January 21, 2026 07:22

uniprewan added 2 commits January 22, 2026 11:27

fix: resolve lint (unused core) and license issues in test file

0525585

fix: optimize http request implementation in base driver

332d024

uniprewan force-pushed the feat/ai-proxy-anthropic-native branch from 788517d to 332d024 Compare January 22, 2026 06:26

uniprewan closed this Jan 22, 2026

uniprewan reopened this Jan 22, 2026

uniprewan closed this Jan 22, 2026

uniprewan deleted the feat/ai-proxy-anthropic-native branch January 22, 2026 07:08

uniprewan mentioned this pull request Jan 23, 2026

feat(ai-proxy): add native Anthropic endpoint support #12936

Closed

5 tasks

	local http = require("resty.http" )
	local http = require("resty.http")

feat(ai-proxy): add native support for Anthropic provider #12924

feat(ai-proxy): add native support for Anthropic provider #12924

Conversation

uniprewan commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

What I Did

Checklist

Uh oh!

Baoyuantop commented Jan 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

uniprewan commented Jan 20, 2026

Uh oh!

Baoyuantop commented Jan 22, 2026

Uh oh!

uniprewan commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

uniprewan commented Jan 19, 2026 •

edited

Loading