Conversation
…king across services - Introduced RequestPerfMiddleware to log request performance metrics, including slow request thresholds. - Updated various services and retrievers to utilize the new performance logging utility for better tracking of execution times. - Enhanced existing methods with detailed performance logs for operations such as embedding, searching, and indexing. - Removed deprecated logging setup in stream_new_chat and replaced it with the new performance logger.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![recurseml[bot]](https://avatars.githubusercontent.com/in/1054195?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review by RecurseML
🔍 Review performed on f1f100b..a43956b
✨ No bugs found, your code is sparkling clean
✅ Files analyzed, no issues (10)
• surfsense_backend/app/agents/new_chat/chat_deepagent.py
• surfsense_backend/app/app.py
• surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
• surfsense_backend/app/retriever/chunks_hybrid_search.py
• surfsense_backend/app/retriever/documents_hybrid_search.py
• surfsense_backend/app/services/connector_service.py
• surfsense_backend/app/services/llm_router_service.py
• surfsense_backend/app/tasks/chat/stream_new_chat.py
• surfsense_backend/app/utils/perf.py
• surfsense_web/components/assistant-ui/connector-popup/constants/connector-constants.ts
1 participant
Description
Motivation and Context
FIX #
Screenshots
API Changes
Change Type
Testing Performed
Checklist
High-level PR Summary
This PR introduces a centralized performance monitoring system for the SurfSense backend by creating a new
perf.pyutility module and instrumenting critical paths with timing logs. A newRequestPerfMiddlewaretracks all HTTP request durations and logs slow requests exceeding a configurable threshold (default 2000ms) with system snapshots. Performance logging has been added throughout the application including knowledge base search, LLM router operations, indexing pipeline, retriever services, and connector service operations. Additionally, the PR comments out managed Google connectors (Drive, Gmail, Calendar) from the web frontend connector popup, making them unavailable for quick OAuth connection.⏱️ Estimated Review Time: 15-30 minutes
💡 Review Order Suggestion
surfsense_backend/app/utils/perf.pysurfsense_backend/app/app.pysurfsense_backend/app/tasks/chat/stream_new_chat.pysurfsense_backend/app/agents/new_chat/chat_deepagent.pysurfsense_backend/app/agents/new_chat/tools/knowledge_base.pysurfsense_backend/app/services/llm_router_service.pysurfsense_backend/app/services/connector_service.pysurfsense_backend/app/indexing_pipeline/indexing_pipeline_service.pysurfsense_backend/app/retriever/chunks_hybrid_search.pysurfsense_backend/app/retriever/documents_hybrid_search.pysurfsense_web/components/assistant-ui/connector-popup/constants/connector-constants.tssurfsense_web/components/assistant-ui/connector-popup/constants/connector-constants.ts