Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Quick reference to all MCP tools:
|------|------------|---------|---------|-------|
| `markdownify` | `website_url` | Convert webpage to markdown | 2 | No |
| `smartscraper` | `user_prompt`, `website_url`, `number_of_scrolls?`, `markdown_only?` | AI-powered data extraction | 10+ | No |
| `searchscraper` | `user_prompt`, `num_results?`, `number_of_scrolls?` | AI-powered web search | Variable | No |
| `searchscraper` | `user_prompt`, `num_results?`, `number_of_scrolls?`, `time_range?` | AI-powered web search | Variable | No |
| `smartcrawler_initiate` | `url`, `prompt?`, `extraction_mode`, `depth?`, `max_pages?`, `same_domain_only?` | Start multi-page crawl | 100+ | Yes (returns request_id) |
| `smartcrawler_fetch_results` | `request_id` | Get crawl results | N/A | No (polls status) |

Expand Down Expand Up @@ -376,6 +376,11 @@ npx @modelcontextprotocol/inspector scrapegraph-mcp

## 📅 Changelog

### January 2026
- ✅ Added `time_range` parameter to SearchScraper for filtering results by recency
- ✅ Supported time ranges: `past_hour`, `past_24_hours`, `past_week`, `past_month`, `past_year`
- ✅ Documentation updated to reflect SDK changes (scrapegraph-py#77, scrapegraph-js#2)

### October 2025
- ✅ Initial comprehensive documentation created
- ✅ Project architecture fully documented
Expand Down
8 changes: 5 additions & 3 deletions .agent/system/project_architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ScrapeGraph MCP Server - Project Architecture

**Last Updated:** October 2025
**Last Updated:** January 2026
**Version:** 1.0.0

## Table of Contents
Expand Down Expand Up @@ -255,14 +255,15 @@ The server exposes 5 tools to AI assistants:

---

### 3. `searchscraper(user_prompt: str, num_results: int = None, number_of_scrolls: int = None)`
### 3. `searchscraper(user_prompt: str, num_results: int = None, number_of_scrolls: int = None, time_range: str = None)`

**Purpose:** Perform AI-powered web searches with structured results

**Parameters:**
- `user_prompt` (str) - Search query or instructions
- `num_results` (int, optional) - Number of websites to search (default: 3 = 30 credits)
- `number_of_scrolls` (int, optional) - Number of infinite scrolls per website
- `time_range` (str, optional) - Filter results by recency. Valid values: `past_hour`, `past_24_hours`, `past_week`, `past_month`, `past_year`

**Returns:**
```json
Expand All @@ -282,7 +283,8 @@ The server exposes 5 tools to AI assistants:
"Research the latest AI developments in 2025"
→ AI calls: searchscraper(
user_prompt="Latest AI developments in 2025",
num_results=5
num_results=5,
time_range="past_week"
)
```

Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,13 @@ Execute AI-powered web searches with structured, actionable results.
searchscraper(
user_prompt: str,
num_results: int = None,
number_of_scrolls: int = None
number_of_scrolls: int = None,
time_range: str = None # Filter by: past_hour, past_24_hours, past_week, past_month, past_year
)
```
- **Credits**: Variable (3-20 websites × 10 credits)
- **Use case**: Multi-source research and data aggregation
- **Time filtering**: Use `time_range` to filter results by recency (e.g., `"past_week"` for recent results)

### Advanced Scraping Tools

Expand Down Expand Up @@ -518,6 +520,7 @@ The server enables sophisticated queries across various scraping scenarios:
- **SearchScraper**: "Research and summarize recent developments in AI-powered web scraping"
- **SearchScraper**: "Search for the top 5 articles about machine learning frameworks and extract key insights"
- **SearchScraper**: "Find recent news about GPT-4 and provide a structured summary"
- **SearchScraper with time_range**: "Search for AI news from the past week only" (uses `time_range="past_week"`)

### Website Analysis
- **Sitemap**: "Extract the complete sitemap structure from the ScrapeGraph website"
Expand Down
Loading
Loading