Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.development
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,5 @@ SOURCEBOT_TELEMETRY_DISABLED=true # Disables telemetry collection
# CONFIG_MAX_REPOS_NO_TOKEN=
NODE_ENV=development
# SOURCEBOT_TENANCY_MODE=single

DEBUG_WRITE_CHAT_MESSAGES_TO_FILE=true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add a trailing newline at EOF.

dotenv-linter flagged EndingBlankLine on Line 80; please add a final blank line to satisfy the linter.

🧰 Tools
🪛 dotenv-linter (4.0.0)

[warning] 80-80: [EndingBlankLine] No blank line at the end of the file

(EndingBlankLine)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.env.development at line 80, The file ends without a trailing newline which
triggers dotenv-linter's EndingBlankLine; open the .env.development file and add
a final blank line (newline character) after the last entry
DEBUG_WRITE_CHAT_MESSAGES_TO_FILE=true so the file ends with a newline
character.

8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Added `find_symbol_definitions`, and `find_symbol_references` tools to the MCP server. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
- Added `list_tree` tool to the ask agent. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
- Added input & output token breakdown in ask details card. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)

### Fixed
- Fixed issue where ask responses would sometimes appear in the details panel while generating. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)

## [4.15.9] - 2026-03-17

### Added
Expand Down
9 changes: 4 additions & 5 deletions packages/shared/src/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,11 @@ const datadogFormat = format((info) => {
return info;
});

const humanReadableFormat = printf(({ level, message, timestamp, stack, label: _label }) => {
const humanReadableFormat = printf(({ level, message, timestamp, stack, label: _label, ...rest }) => {
const label = `[${_label}] `;
if (stack) {
return `${timestamp} ${level}: ${label}${message}\n${stack}`;
}
return `${timestamp} ${level}: ${label}${message}`;
const extras = Object.keys(rest).length > 0 ? ` ${JSON.stringify(rest)}` : '';
const base = `${timestamp} ${level}: ${label}${message}${extras}`;
return stack ? `${base}\n${stack}` : base;
});

const createLogger = (label: string) => {
Expand Down
9 changes: 8 additions & 1 deletion packages/web/next.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,14 @@ const nextConfig = {
]
},

turbopack: {},
turbopack: {
rules: {
'*.txt': {
loaders: ['raw-loader'],
as: '*.js',
},
},
},
Comment on lines +66 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if webpack config for raw-loader exists elsewhere in the codebase
rg -n "raw-loader" --type=js --type=ts -g '!node_modules'

# Check for any existing webpack config modifications
ast-grep --pattern 'webpack: ($config) => { $$$ }'

Repository: sourcebot-dev/sourcebot

Length of output: 49


🏁 Script executed:

# Examine the next.config.mjs file
cat -n packages/web/next.config.mjs

Repository: sourcebot-dev/sourcebot

Length of output: 4920


🏁 Script executed:

# Search for .txt file imports
rg "import\s+.*\.txt" --type=js --type=ts --type=jsx --type=tsx

# Also check for require statements with .txt
rg "require\(.*\.txt" --type=js --type=ts

# Check if raw-loader is in package.json dependencies
cat packages/web/package.json | grep -A 5 -B 5 "raw-loader"

Repository: sourcebot-dev/sourcebot

Length of output: 427


🏁 Script executed:

# Search for any .txt imports without type restrictions
rg "\.txt['\"]" 

# Also list any .txt files in the repo
fd "\.txt$" packages/web --type f

Repository: sourcebot-dev/sourcebot

Length of output: 2295


🏁 Script executed:

# Check if there's any webpack modification in the entire next.config.mjs
grep -n "webpack" packages/web/next.config.mjs

# Check if there are any other config files that might handle webpack
fd "webpack" packages/web --type f

# Check if there's anything in tsconfig or other config files
cat packages/web/tsconfig.json | head -30

Repository: sourcebot-dev/sourcebot

Length of output: 616


🏁 Script executed:

# Check if there's a build or test script that might have caught this
cat packages/web/package.json | grep -A 10 '"scripts"'

# Also verify if there are any error suppression configs that might hide this
rg "suppress\|ignore.*raw-loader\|webpack.*rule" packages/web

Repository: sourcebot-dev/sourcebot

Length of output: 834


Add webpack configuration for raw-loader to fix production builds.

The turbopack config at lines 66–73 only applies to next dev --turbopack. Production builds use webpack via next build, which has no rule for .txt files and will fail when importing them. This affects 7 files that import .txt files in production code.

🔧 Proposed fix
     turbopack: {
         rules: {
             '*.txt': {
                 loaders: ['raw-loader'],
                 as: '*.js',
             },
         },
     },
+
+    webpack: (config) => {
+        config.module.rules.push({
+            test: /\.txt$/,
+            use: 'raw-loader',
+        });
+        return config;
+    },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
turbopack: {
rules: {
'*.txt': {
loaders: ['raw-loader'],
as: '*.js',
},
},
},
turbopack: {
rules: {
'*.txt': {
loaders: ['raw-loader'],
as: '*.js',
},
},
},
webpack: (config) => {
config.module.rules.push({
test: /\.txt$/,
use: 'raw-loader',
});
return config;
},
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/next.config.mjs` around lines 66 - 73, The turbopack rule for
'*.txt' only covers dev; add an equivalent webpack rule in the Next.js config's
webpack property so production builds (next build) use raw-loader for .txt
imports: update the exported config to include a webpack function that pushes a
module.rules entry matching /\.txt$/ and uses 'raw-loader' (mirroring the
turbopack loaders setting), and ensure the loader is installed; locate the
existing turbopack block (the turbopack.rules '*.txt' entry) and add the
corresponding webpack.module.rules rule in the same next.config.mjs config
object.


// @see: https://github.com/vercel/next.js/issues/58019#issuecomment-1910531929
...(process.env.NODE_ENV === 'development' ? {
Expand Down
3 changes: 3 additions & 0 deletions packages/web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@
"escape-string-regexp": "^5.0.0",
"fast-deep-equal": "^3.1.3",
"fuse.js": "^7.0.0",
"glob-to-regexp": "^0.4.1",
"google-auth-library": "^10.1.0",
"graphql": "^16.9.0",
"http-status-codes": "^2.3.0",
Expand Down Expand Up @@ -202,6 +203,7 @@
"@tanstack/eslint-plugin-query": "^5.74.7",
"@testing-library/dom": "^10.4.1",
"@testing-library/react": "^16.3.0",
"@types/glob-to-regexp": "^0.4.4",
"@types/micromatch": "^4.0.9",
"@types/node": "^20",
"@types/nodemailer": "^6.4.17",
Expand All @@ -218,6 +220,7 @@
"jsdom": "^25.0.1",
"npm-run-all": "^4.1.5",
"postcss": "^8",
"raw-loader": "^4.0.2",
"react-email": "^5.1.0",
"react-grab": "^0.1.23",
"tailwindcss": "^3.4.1",
Expand Down
63 changes: 35 additions & 28 deletions packages/web/src/features/chat/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,15 @@ import {
} from "ai";
import { randomUUID } from "crypto";
import _dedent from "dedent";
import { ANSWER_TAG, FILE_REFERENCE_PREFIX, toolNames } from "./constants";
import { createCodeSearchTool, findSymbolDefinitionsTool, findSymbolReferencesTool, listCommitsTool, listReposTool, readFilesTool } from "./tools";
import { ANSWER_TAG, FILE_REFERENCE_PREFIX } from "./constants";
import { findSymbolReferencesDefinition } from "@/features/tools/findSymbolReferences";
import { findSymbolDefinitionsDefinition } from "@/features/tools/findSymbolDefinitions";
import { readFileDefinition } from "@/features/tools/readFile";
import { grepDefinition } from "@/features/tools/grep";
import { Source } from "./types";
import { addLineNumbers, fileReferenceToString } from "./utils";
import { tools } from "./tools";
import { listTreeDefinition } from "../tools";

const dedent = _dedent.withOptions({ alignValues: true });

Expand Down Expand Up @@ -198,14 +203,7 @@ const createAgentStream = async ({
providerOptions,
messages: inputMessages,
system: systemPrompt,
tools: {
[toolNames.searchCode]: createCodeSearchTool(selectedRepos),
[toolNames.readFiles]: readFilesTool,
[toolNames.findSymbolReferences]: findSymbolReferencesTool,
[toolNames.findSymbolDefinitions]: findSymbolDefinitionsTool,
[toolNames.listRepos]: listReposTool,
[toolNames.listCommits]: listCommitsTool,
},
tools,
temperature: env.SOURCEBOT_CHAT_MODEL_TEMPERATURE,
stopWhen: [
stepCountIsGTE(env.SOURCEBOT_CHAT_MAX_STEP_COUNT),
Expand All @@ -223,39 +221,46 @@ const createAgentStream = async ({
return;
}

if (toolName === toolNames.readFiles) {
output.forEach((file) => {
if (toolName === readFileDefinition.name) {
onWriteSource({
type: 'file',
repo: output.metadata.repo,
path: output.metadata.path,
revision: output.metadata.revision,
name: output.metadata.path.split('/').pop() ?? output.metadata.path,
});
} else if (toolName === grepDefinition.name) {
output.metadata.files.forEach((file) => {
onWriteSource({
type: 'file',
language: file.language,
repo: file.repository,
repo: file.repo,
path: file.path,
revision: file.revision,
name: file.path.split('/').pop() ?? file.path,
});
});
} else if (toolName === toolNames.searchCode) {
output.files.forEach((file) => {
} else if (toolName === findSymbolDefinitionsDefinition.name || toolName === findSymbolReferencesDefinition.name) {
output.metadata.files.forEach((file) => {
onWriteSource({
type: 'file',
language: file.language,
repo: file.repository,
repo: file.repo,
path: file.fileName,
revision: file.revision,
name: file.fileName.split('/').pop() ?? file.fileName,
});
});
} else if (toolName === toolNames.findSymbolDefinitions || toolName === toolNames.findSymbolReferences) {
output.forEach((file) => {
onWriteSource({
type: 'file',
language: file.language,
repo: file.repository,
path: file.fileName,
revision: file.revision,
name: file.fileName.split('/').pop() ?? file.fileName,
} else if (toolName === listTreeDefinition.name) {
output.metadata.entries
.filter((entry) => entry.type === 'blob')
.forEach((entry) => {
onWriteSource({
type: 'file',
repo: output.metadata.repo,
path: entry.path,
revision: output.metadata.ref,
name: entry.name,
});
});
});
}
});
},
Expand Down Expand Up @@ -312,6 +317,8 @@ const createPrompt = ({
<selected_repositories>
The user has explicitly selected the following repositories for analysis:
${repos.map(repo => `- ${repo}`).join('\n')}

When calling tools that accept a \`repo\` parameter (e.g. \`read_file\`, \`list_commits\`, \`list_tree\`, \`grep\`), use these repository names exactly as listed above, including the full host prefix (e.g. \`github.com/org/repo\`).
</selected_repositories>
` : ''}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import { AnswerCard } from './answerCard';
import { DetailsCard } from './detailsCard';
import { MarkdownRenderer, REFERENCE_PAYLOAD_ATTRIBUTE } from './markdownRenderer';
import { ReferencedSourcesListView } from './referencedSourcesListView';
import { uiVisiblePartTypes } from '../../constants';
import isEqual from "fast-deep-equal/react";

interface ChatThreadListItemProps {
Expand Down Expand Up @@ -102,7 +101,12 @@ const ChatThreadListItemComponent = forwardRef<HTMLDivElement, ChatThreadListIte
return part.text !== answerPart?.text;
})
.filter((part) => {
return uiVisiblePartTypes.includes(part.type);
// Only include text, reasoning, and tool parts
return (
part.type === 'text' ||
part.type === 'reasoning' ||
part.type.startsWith('tool-')
)
})
)
// Then, filter out any steps that are empty
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@ import { Collapsible, CollapsibleContent, CollapsibleTrigger } from "@/component
import { Separator } from '@/components/ui/separator';
import { Skeleton } from '@/components/ui/skeleton';
import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip';
import { cn } from '@/lib/utils';
import { cn, getShortenedNumberDisplayString } from '@/lib/utils';
import { Brain, ChevronDown, ChevronRight, Clock, InfoIcon, Loader2, List, ScanSearchIcon, Zap } from 'lucide-react';
import { memo, useCallback } from 'react';
import useCaptureEvent from '@/hooks/useCaptureEvent';
import { MarkdownRenderer } from './markdownRenderer';
import { FindSymbolDefinitionsToolComponent } from './tools/findSymbolDefinitionsToolComponent';
import { FindSymbolReferencesToolComponent } from './tools/findSymbolReferencesToolComponent';
import { ReadFilesToolComponent } from './tools/readFilesToolComponent';
import { SearchCodeToolComponent } from './tools/searchCodeToolComponent';
import { ReadFileToolComponent } from './tools/readFileToolComponent';
import { GrepToolComponent } from './tools/grepToolComponent';
import { ListReposToolComponent } from './tools/listReposToolComponent';
import { ListCommitsToolComponent } from './tools/listCommitsToolComponent';
import { ListTreeToolComponent } from './tools/listTreeToolComponent';
import { SBChatMessageMetadata, SBChatMessagePart } from '../../types';
import { SearchScopeIcon } from '../searchScopeIcon';
import isEqual from "fast-deep-equal/react";
Expand Down Expand Up @@ -105,15 +106,35 @@ const DetailsCardComponent = ({
</div>
)}
{metadata?.totalTokens && (
<div className="flex items-center text-xs">
<Zap className="w-3 h-3 mr-1 flex-shrink-0" />
{metadata?.totalTokens} tokens
</div>
<Tooltip>
<TooltipTrigger asChild>
<div className="flex items-center text-xs cursor-help">
<Zap className="w-3 h-3 mr-1 flex-shrink-0" />
{getShortenedNumberDisplayString(metadata.totalTokens, 0)} tokens
</div>
</TooltipTrigger>
<TooltipContent side="bottom">
<div className="space-y-1 text-xs">
<div className="flex justify-between gap-4">
<span className="text-muted-foreground">Input</span>
<span>{metadata.totalInputTokens?.toLocaleString() ?? '—'}</span>
</div>
<div className="flex justify-between gap-4">
<span className="text-muted-foreground">Output</span>
<span>{metadata.totalOutputTokens?.toLocaleString() ?? '—'}</span>
</div>
<div className="flex justify-between gap-4 border-t border-border pt-1">
<span className="text-muted-foreground">Total</span>
<span>{metadata.totalTokens.toLocaleString()}</span>
</div>
</div>
</TooltipContent>
</Tooltip>
)}
{metadata?.totalResponseTimeMs && (
<div className="flex items-center text-xs">
<Clock className="w-3 h-3 mr-1 flex-shrink-0" />
{metadata?.totalResponseTimeMs / 1000} seconds
{Math.round(metadata.totalResponseTimeMs / 1000)} seconds
</div>
)}
<div className="flex items-center text-xs">
Expand Down Expand Up @@ -166,49 +187,65 @@ const DetailsCardComponent = ({
className="text-sm"
/>
)
case 'tool-readFiles':
case 'tool-read_file':
return (
<ReadFilesToolComponent
<ReadFileToolComponent
key={index}
part={part}
/>
)
case 'tool-searchCode':
case 'tool-grep':
return (
<SearchCodeToolComponent
<GrepToolComponent
key={index}
part={part}
/>
)
case 'tool-findSymbolDefinitions':
case 'tool-find_symbol_definitions':
return (
<FindSymbolDefinitionsToolComponent
key={index}
part={part}
/>
)
case 'tool-findSymbolReferences':
case 'tool-find_symbol_references':
return (
<FindSymbolReferencesToolComponent
key={index}
part={part}
/>
)
case 'tool-listRepos':
case 'tool-list_repos':
return (
<ListReposToolComponent
key={index}
part={part}
/>
)
case 'tool-listCommits':
case 'tool-list_commits':
return (
<ListCommitsToolComponent
key={index}
part={part}
/>
)
case 'tool-list_tree':
return (
<ListTreeToolComponent
key={index}
part={part}
/>
)
case 'data-source':
case 'dynamic-tool':
case 'file':
case 'source-document':
case 'source-url':
case 'step-start':
return null;
default:
// Guarantees this switch-case to be exhaustive
part satisfies never;
return null;
}
})}
Expand Down
Loading
Loading