fix(anthropic): token limits for streaming with tool calls

[icecrasher321]

Summary

• Fix tool calls to have non streaming limits since we don't stream them
• Remove these changes for bedrock since it handles it on it's own

Type of Change

• Bug fix

Testing

Tested manually

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jan 30, 2026 7:55pm

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes token limit handling for Anthropic provider tool calls to prevent SDK timeout errors.

Key Changes:

• Applied non-streaming token limits (8192 default vs 64000 max) to tool call execution in apps/sim/providers/anthropic/index.ts when !shouldStreamToolCalls
• Removed maxOutputTokens configuration from Bedrock model definitions since AWS SDK handles timeouts differently
• Improved documentation in models.ts to clarify that maxOutputTokens only applies to direct Anthropic SDK calls
• Added .cursor/commands/council.md for agent spawning workflow

Context:
The Anthropic SDK throws errors for non-streaming requests that may exceed 10 minutes. When tool calls are executed in non-streaming mode (even within a streaming request), they should use the conservative default token limit (8192) rather than the maximum (64000) to prevent timeout issues. This doesn't apply to Bedrock since it uses AWS SDK which has different timeout handling.

Confidence Score: 4/5

• Safe to merge with one minor consideration about token limit edge cases
• The fix correctly addresses the Anthropic SDK timeout issue by applying non-streaming token limits to tool execution paths. The Bedrock revert is appropriate since AWS SDK handles timeouts differently. However, there's a potential edge case where users explicitly requesting high maxTokens might not expect the limit to be capped at 8192 during tool calls.
• Consider reviewing apps/sim/providers/anthropic/index.ts to verify the token limit logic aligns with expected behavior

Important Files Changed

Filename	Overview
apps/sim/providers/anthropic/index.ts	Applied non-streaming token limits to tool call execution paths to prevent Anthropic SDK timeouts
apps/sim/providers/bedrock/index.ts	Reverted maxOutputTokens changes for Bedrock models - AWS SDK handles token limits differently
apps/sim/providers/models.ts	Removed maxOutputTokens from Bedrock model definitions and improved documentation clarity

Sequence Diagram

sequenceDiagram
    participant Client
    participant AnthropicProvider
    participant AnthropicSDK
    participant Tools

    Note over Client,AnthropicProvider: Request with stream=true, tools present
    Client->>AnthropicProvider: execute(request)
    
    alt shouldStreamToolCalls = false
        Note over AnthropicProvider: Non-streaming tool execution path
        AnthropicProvider->>AnthropicProvider: Create nonStreamingMaxTokens<br/>(uses default: 8192 instead of max: 64000)
        AnthropicProvider->>AnthropicProvider: Create intermediatePayload with<br/>non-streaming token limit
        
        loop Tool execution iterations
            AnthropicProvider->>AnthropicSDK: messages.create(intermediatePayload)
            Note over AnthropicSDK: Non-streaming call avoids<br/>SDK timeout (>10 min limit)
            AnthropicSDK-->>AnthropicProvider: response with tool_use
            AnthropicProvider->>Tools: Execute tool calls
            Tools-->>AnthropicProvider: Tool results
            AnthropicProvider->>AnthropicProvider: Update messages with results
        end
        
        AnthropicProvider-->>Client: Stream final response to client
    else shouldStreamToolCalls = true
        Note over AnthropicProvider: Regular streaming path
        AnthropicProvider->>AnthropicSDK: messages.stream()
        AnthropicSDK-->>Client: Stream response
    end

Loading

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[icecrasher321]

@cursor review

[icecrasher321]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR