feat(langsmith): add langsmith tools for logging, output selector use tool-aware listing

[icecrasher321]

Summary

• Langsmith tools to log
• Output selector to use tool-aware listing (in deploy chat + client chat window)

Type of Change

• Bug fix
• New feature

Testing

Tested manually

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
docs	Ready	Preview, Comment	Jan 15, 2026 0:10am

[greptile-apps]

Greptile Summary

Added LangSmith integration for workflow observability with two core tools (langsmith_create_run and langsmith_create_runs_batch) that forward execution data to LangSmith's API. Improved output selector and tag dropdown to use tool-aware output listing by extracting tool-specific outputs based on the selected operation, enabling proper autocomplete for blocks with multiple tool operations.

Key Changes:

• Created new Langsmith block with operation dropdown supporting single run creation and batch operations
• Implemented utility functions for normalizing run payloads with auto-generated UUIDs, trace IDs, and compact timestamps
• Refactored output resolution logic by consolidating duplicated generateOutputPaths functions into centralized utilities in block-outputs.ts
• Updated output selector in deploy chat and tag dropdown to call getToolOutputs() when a tool operation is selected

Issues Found:

• Documentation has parameter name mismatches (camelCase vs snake_case) and incorrectly states create_run produces no outputs

Confidence Score: 4/5

• Safe to merge after fixing documentation parameter names and output descriptions
• Implementation is solid with proper type safety, error handling, and normalization logic. The refactoring to centralize output path generation is a good improvement. However, documentation has several inaccuracies that should be corrected before merge
• Pay attention to apps/docs/content/docs/en/tools/langsmith.mdx - needs parameter name corrections

Important Files Changed

Filename	Overview
apps/sim/tools/langsmith/create_run.ts	Added single run creation tool with proper API key handling, payload normalization, and response transformation
apps/sim/blocks/blocks/langsmith.ts	Block configuration with operation dropdown, comprehensive subblocks for both single and batch operations
apps/sim/lib/workflows/blocks/block-outputs.ts	Added getToolOutputs, getToolOutputPaths, getToolOutputType, and getOutputPathsFromSchema functions for tool-aware output handling
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/chat/components/output-select/output-select.tsx	Updated to use getToolOutputs for tool-aware output listing in chat deploy mode
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/tag-dropdown.tsx	Refactored to use centralized utility functions from block-outputs.ts, removed duplicated code
apps/docs/content/docs/en/tools/langsmith.mdx	Documentation added for langsmith tools with incomplete output documentation

Sequence Diagram

sequenceDiagram
    participant User
    participant Block as Langsmith Block
    participant Tool as Langsmith Tool
    participant Utils as normalizeLangsmithRunPayload
    participant API as LangSmith API

    User->>Block: Configure operation & params
    Block->>Block: Parse JSON values (inputs, outputs, tags, etc)
    
    alt Create Single Run
        Block->>Tool: langsmith_create_run(params)
        Tool->>Utils: normalizeLangsmithRunPayload(params)
        Utils->>Utils: Generate runId (or use provided)
        Utils->>Utils: Generate traceId (or use provided)
        Utils->>Utils: Generate dottedOrder timestamp
        Utils-->>Tool: {runId, payload}
        Tool->>API: POST /runs with normalized payload
        API-->>Tool: Response with run data
        Tool->>Tool: transformResponse (extract runId, message)
        Tool-->>Block: {accepted, runId, message}
    else Create Runs Batch
        Block->>Tool: langsmith_create_runs_batch(params)
        Tool->>Tool: Map post array through normalizeLangsmithRunPayload
        Tool->>Tool: Map patch array through normalizeLangsmithRunPayload
        Tool->>API: POST /runs/batch with normalized payloads
        API-->>Tool: Response with batch data
        Tool->>Tool: transformResponse (collect runIds, messages)
        Tool-->>Block: {accepted, runIds, message, messages}
    end
    
    Block-->>User: Display result outputs

Loading

[greptile-apps]

_{19 files reviewed, 5 comments}

_{Edit Code Review Agent Settings | Greptile}