Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/md_v2/api/arun_many.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ async def arun_many(
config: Optional[Union[CrawlerRunConfig, List[CrawlerRunConfig]]] = None,
dispatcher: Optional[BaseDispatcher] = None,
...
) -> Union[List[CrawlResult], AsyncGenerator[CrawlResult, None]]:
) -> RunManyReturn:
"""
Crawl multiple URLs concurrently or in batches.

Expand All @@ -20,16 +20,16 @@ async def arun_many(
- A list of `CrawlerRunConfig` objects with url_matcher patterns
:param dispatcher: (Optional) A concurrency controller (e.g.β€€MemoryAdaptiveDispatcher).
...
:return: Either a list of `CrawlResult` objects, or an async generator if streaming is enabled.
:return: RunManyReturn containing either a list of `CrawlResult` objects or an async generator if streaming is enabled.
"""
```

## Differences from `arun()`

1.β€€**Multiple URLs**:
- Instead of crawling a single URL, you pass a list of them (strings or tasks).β€€
- The function returns either a **list** of `CrawlResult` or an **async generator** if streaming is enabled.
1.β€€**Multiple URLs**:

- Instead of crawling a single URL, you pass a list of them (strings or tasks).
- The function returns `RunManyReturn` which contains either a **list** of `CrawlResult` or an **async generator** if streaming is enabled.

2.β€€**Concurrency & Dispatchers**:

Expand Down Expand Up @@ -164,7 +164,7 @@ results = await crawler.arun_many(

### Return Value

Either a **list** of [`CrawlResult`](./crawl-result.md) objects, or an **async generator** if streaming is enabled.β€€You can iterate to check `result.success` or read each item’s `extracted_content`, `markdown`, or `dispatch_result`.
Returns a **`RunManyReturn`** object which contains either a **list** of [`CrawlResult`](./crawl-result.md) objects, or an **async generator** if streaming is enabled.β€€You can iterate to check `result.success` or read each item's `extracted_content`, `markdown`, or `dispatch_result`.

---

Expand Down
4 changes: 2 additions & 2 deletions docs/md_v2/api/async-webcrawler.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ async def arun(
url: str,
config: Optional[CrawlerRunConfig] = None,
# Legacy parameters for backward compatibility...
) -> CrawlResult:
) -> RunManyReturn:
...
```

Expand Down Expand Up @@ -143,7 +143,7 @@ async def arun_many(
urls: List[str],
config: Optional[CrawlerRunConfig] = None,
# Legacy parameters maintained for backwards compatibility...
) -> List[CrawlResult]:
) -> RunManyReturn:
"""
Process multiple URLs with intelligent rate limiting and resource monitoring.
"""
Expand Down