feat: add AG2 multi-agent document processing example

[faridun-ag2]

Summary

Add an example demonstrating AG2 (formerly AutoGen) multi-agent framework combined with Unstructured for intelligent document processing and analysis.

Also addresses #4320 (confidence scores) by showcasing the existing detection_class_prob metadata — agents can filter low-confidence extractions for better RAG quality.

What's included

• examples/ag2_multiagent_document_processing/run.py — Main example script
• examples/ag2_multiagent_document_processing/test_e2e.py — 17 end-to-end tests
• examples/ag2_multiagent_document_processing/README.md — Setup instructions

How it works

1. Unstructured partitions a document into structured elements (Title, NarrativeText, Table, ListItem, etc.) using Hi-Res strategy for PDFs
2. AG2 Document Agent wraps Unstructured as a registered tool, callable by agents
3. AG2 Analyst Agent receives extracted content and produces comprehensive analysis
4. Agents collaborate via AG2 GroupChat with automatic tool execution

Key features

• 25+ file types: PDF, HTML, Word, PowerPoint, email, images (OCR), and more
• Multi-agent pipeline: Document extraction → Analysis → Summary
• Confidence score filtering ([Feature Request] Add document layout analysis confidence scores #4320): Elements display [conf=X.XX] scores from Hi-Res detection model; agents can filter low-confidence extractions via filter_low_confidence tool
• Tool registration: Unstructured partitioning exposed as AG2 tool via decorator pattern
• End-to-end tests: 16 offline tests + 1 live LLM test
• CLI support: python run.py --file path/to/document.pdf
• Uses existing example-docs/: No new test fixtures needed

Confidence scores demo (addresses #4320)

When processing PDFs with Hi-Res strategy, each element shows its detection confidence:

[1] (Title) [conf=0.48] LayoutParser: A Unified Toolkit...
[2] (NarrativeText) [conf=0.95] Recent advances in document image analysis...
[3] (ListItem) [conf=0.29] 4 Z. Shen et al.

The filter_low_confidence AG2 tool lets agents filter noisy elements:

• Elements below a threshold are removed
• Elements without scores (HTML, non-Hi-Res) are kept by default
• Stats report kept/filtered/no-score counts

Testing

# Offline tests (no API key needed) — 16 tests
pytest examples/ag2_multiagent_document_processing/test_e2e.py -v -k "not live_llm"

# Full pipeline test (requires OPENAI_API_KEY)
pytest examples/ag2_multiagent_document_processing/test_e2e.py -v -k "live_llm"

Why AG2?

AG2 is a major multi-agent framework with 500K+ monthly PyPI downloads and 4,300+ GitHub stars.
Unstructured is the natural document processing layer for AG2 agentic RAG pipelines — this example
shows how they work together.

Checklist

• Code formatted with Black (100 char line)
• Ruff checks pass
• Pre-commit hooks pass (except pre-existing flake8 E501 repo-wide issue)
• 17 e2e tests included and passing
• Example is self-contained and runnable
• Follows conventional commit format
• Demonstrates confidence scores from Hi-Res strategy ([Feature Request] Add document layout analysis confidence scores #4320)

[faridun-ag2]

Hi @cragwolfe @qued !
Would appreciate a review when you get a chance! This adds an AG2 (multi-agent framework) integration example that uses Unstructured for document processing, and also demonstrates the existing detection_class_prob confidence scores relevant to #4320.

Happy to address any feedback. Thanks!