简体中文 | English

⭐ If you like this project, please click the "Star" button in the upper right corner to support us. Your support is our motivation to move forward!

Contents

• Contents
• 📋 Project Overview
• ✨ Core Features
  • Feature Comparison
• 🏗️ System Architecture
• 🚀 Quick Start
  • Environment Checklist
  • One-Click Deployment (Recommended)
  • Data & Volume Layout
  • Usage Guide
    • LLM API Load Testing
    • General API Load Testing
• 🔧 Configuration
  • Database Configuration
  • LDAP/AD Authentication Configuration
  • Resource Configuration
• 🤝 Development Guide
  • Technology Stack
  • Development Environment Setup
• 🗺️ Development Roadmap
  • In Development
  • Planned
• 🗂️ Dataset Reference Notes
• 👥 Contributing
• 📝 Citation
• 📄 Open Source License

📋 Project Overview

LMeterX is a professional large language model performance testing platform that can be applied to model inference services based on large model inference frameworks (such as LiteLLM, vLLM, TensorRT-LLM, LMDeploy, and others), and also supports performance testing for cloud services like Azure OpenAI, AWS Bedrock, Google Vertex AI, and other major cloud providers. Through an intuitive Web interface, users can easily create and manage test tasks, monitor testing processes in real-time, and obtain detailed performance analysis reports, providing reliable data support for model deployment and performance optimization.

✨ Core Features

• Broad Framework Compatibility - Supports mainstream inference frameworks (vLLM, LiteLLM, TRT-LLM) and cloud platforms, ensuring seamless environment migration.
• Full Modality & Scenarios - Supports GPT, Claude, Llama to document parsing models like MinerU and dots.ocr, covering text, multimodal, and streaming.
• Hybrid Protocol Testing   - Supports standard Chat APIs and business HTTP interfaces, enabling full-stack load testing from base models to upper-level services.
• Extreme High-Concurrency - Leverages multi-process architecture to simulate high-load concurrency, accurately detecting system performance limits and stability.
• Built-in Dual-Mode Datasets - Pre-configured with high-quality self-built datasets and ShareGPT standard sets, supporting one-click invocation to lower data preparation barriers.
• Automated Warm-up Mechanism   - Supports automatic model service warm-up to eliminate cold-start effects, ensuring the accuracy of test data.
• Fine-grained Multi-dimensional Metrics - Real-time collection of TTFT, RPS, TPS, and throughput distribution, providing comprehensive performance measurement.
• AI-Driven Data Insights   - AI-powered analysis reports with multi-model comparison, intuitively identifying optimization directions.
• One-stop Web Console - Manage task scheduling, monitoring, and real-time logs through an intuitive interface, reducing operational complexity.
• Enterprise-Grade Security & Scaling   - Supports distributed elastic deployment and LDAP/AD integration for high availability and secure enterprise authentication.

Feature Comparison

Dimension	LMeterX	EvalScope	llmperf
Usage	Web UI for full-lifecycle task creation, monitoring & stop (load-test)	CLI for ModelScope ecosystem (eval & load-test)	CLI, Ray-based (load-test)
Concurrency & Stress	Multi-process / multi-task, enterprise-scale load testing	Command-line concurrency (--parallel, --rate)	Command-line concurrency
Test Report	Multi-model / multi-version comparison, AI analysis, visual dashboard	Basic report + visual charts (requires gradio, plotly, etc.)	Simple report
Model & Data Support	OpenAI-compatible, custom data & model interfaces	OpenAI-compatible by default; extending APIs needs custom code	OpenAI-compatible
Deployment & Scaling	Docker / K8s ready, easy horizontal scaling	pip install or source code	Source code only

🏗️ System Architecture

LMeterX adopts a microservices architecture design, consisting of four core components:

1. Backend Service: FastAPI-based REST API service responsible for task management and result storage
2. Load Testing Engine: Locust-based load testing engine that executes actual performance testing tasks
3. Frontend Interface: Modern Web interface based on React + TypeScript + Ant Design
4. MySQL Database: Stores test tasks, result data, and configuration information

🚀 Quick Start

Environment Checklist

• Docker 20.10.0+ with the daemon running
• Docker Compose 2.0.0+ (docker compose plugin or standalone docker-compose)
• At least 4GB free memory and 5GB disk space

Need more deployment options? See the Complete Deployment Guide for Kubernetes, air-gapped installs, and advanced tuning.

One-Click Deployment (Recommended)

# Download and run the one-click deployment script
curl -fsSL https://raw.githubusercontent.com/MigoXLab/LMeterX/main/quick-start.sh | bash

After the script finishes:

• Check container health: docker compose ps
• Tail logs if needed: docker compose logs -f
• Scale services (if needed): docker compose up -d --scale backend=2 --scale engine=2
• Open the web UI at http://localhost:8080 (see Usage Guide)

Data & Volume Layout

• ./data → mounted to /app/data in the engine service (large datasets are not baked into the image)
• ./logs → shared log output for backend and engine
• ./upload_files → user-supplied payloads and exported reports

For custom data, please refer to the Dataset Usage Guide.

Usage Guide

LLM API Load Testing

1. Access Web Interface: Open http://localhost:8080
2. Create Test Task: Navigate to Test Tasks → Create Task, configure API request information, test data, and request/response field mappings.
   • 2.1 Basic Information: For OpenAI-like and Claude-like APIs, you only need to configure API path, model, and response mode. You can also supplement the complete payload in request parameters.
   • 2.2 Data & load: Select the dataset type, concurrency, load testing time, etc., as needed.
   • 2.3 Field Mapping: For custom APIs, you need to configure the prompt field path in payload, and response data paths for model output fields, usage fields, etc. This field mapping is crucial for updating request parameters with datasets and correctly parsing streaming/non-streaming responses.

   💡 Tip: For custom multimodal dataset load tests, follow the Dataset Guide for data preparation, mounting, and troubleshooting.

3. API Testing: In Test Tasks → Create Task, click the "Test" button in the Basic Information panel to quickly test API connectivity (use a lightweight prompt for faster feedback).
4. Real-time Monitoring: Navigate to Test Tasks → Logs/Monitoring Center to view full-chain test logs and troubleshoot exceptions
5. Result Analysis: Navigate to Test Tasks → Results to view detailed performance results and export reports
6. Result Comparison: Navigate to Pref Insight to select multiple models or versions for multi-dimensional performance comparison
7. AI Analysis: In Test Tasks → Results or Pref Insight, support intelligent performance evaluation for single or multiple tasks

General API Load Testing

1. Access Web Interface: Open http://localhost:8080, switch to the "General API" tab
2. Create Test Task: Navigate to Test Tasks → Create Task
   • Paste your complete curl command and click "One-Click Parse" to automatically parse request method, URL, headers, and request body
   • Verify that the parsed request information is complete and accurate
3. API Testing: Click the "Test" button to verify API connectivity and ensure request information is correct before load testing
4. Dataset Preparation (Optional): If using dataset load testing, prepare a JSONL format file in advance. Each line must be a complete payload JSON object
5. Start Load Testing: Configure concurrent users, test duration, and other parameters, then click "Create" to start the load testing task
6. Real-time Monitoring: During testing, click the "Logs" button to view load testing status and real-time logs
7. Result Analysis: After testing completes, click the "Results" button to view load testing results, including RPS, response time, success rate, and other metrics
8. Copy Template: To test the same API again, click "..." → "Copy Template" in the actions column. Note that the dataset needs to be re-uploaded after copying, and it's recommended to repeat steps 3-7
9. Performance Comparison: To compare performance across different versions or concurrency levels, navigate to the "Performance Comparison" page

🔧 Configuration

Database Configuration

# ================= Database Configuration =================
DB_HOST=mysql           # Database host (container name or IP)
DB_PORT=3306            # Database port
DB_USER=lmeterx         # Database username
DB_PASSWORD=lmeterx_password  # Database password (use secrets management in production)
DB_NAME=lmeterx         # Database name

LDAP/AD Authentication Configuration

# ================= LDAP Authentication Configuration =================
# Enable or disable LDAP authentication (on/off)
LDAP_ENABLED=on

# LDAP server connection
LDAP_SERVER=ldap://ldap.example.com    # LDAP server address
LDAP_PORT=389                          # LDAP server port (389 for LDAP, 636 for LDAPS)
LDAP_USE_SSL=false                     # Use SSL/TLS connection (true for LDAPS)
LDAP_TIMEOUT=5                         # Connection timeout in seconds

# LDAP search configuration
LDAP_SEARCH_BASE=dc=example,dc=com     # Base DN for user search
LDAP_SEARCH_FILTER=(sAMAccountName={username})  # LDAP search filter

# Authentication method 1: Direct bind with DN template (recommended for simple setups)
LDAP_USER_DN_TEMPLATE=cn={username},ou=users,dc=example,dc=com

# Authentication method 2: Bind with service account (recommended for Active Directory)
LDAP_BIND_DN=cn=service,ou=users,dc=example,dc=com    # Service account DN
LDAP_BIND_PASSWORD=service_password                   # Service account password

# JWT configuration (optional)
JWT_SECRET_KEY=your-secret-key-here    # JWT signing key (change in production)
JWT_EXPIRE_MINUTES=480                 # Token expiration time in minutes (default: 8 hours)

Configuration Notes:

• Simple LDAP Setup: Use LDAP_USER_DN_TEMPLATE for direct user binding
• Active Directory: Use LDAP_BIND_DN + LDAP_BIND_PASSWORD for service account binding
• Security: Always use LDAP_USE_SSL=true in production environments
• Frontend: Set VITE_LDAP_ENABLED=on to enable login UI

Resource Configuration

# ================= High-Concurrency Load Testing Deployment Requirements =================
# When concurrent users exceed this threshold, the system will automatically enable multi-process mode (requires multi-core CPU support)
MULTIPROCESS_THRESHOLD=1000

# Minimum number of concurrent users each child process should handle (prevents excessive processes and resource waste)
MIN_USERS_PER_PROCESS=500

# ⚠️ IMPORTANT NOTES:
#   - When concurrency ≥ 1000, enabling multi-process mode is strongly recommended for performance.
#   - Multi-process mode requires multi-core CPU resources — ensure your deployment environment meets these requirements.

# ================= Deployment Resource Limits =================
deploy:
  resources:
    limits:
      cpus: '2.0'       # Recommended minimum: 2 CPU cores (4+ cores recommended for high-concurrency scenarios)
      memory: 2G        # Memory limit — adjust based on actual load (minimum recommended: 2G)

🤝 Development Guide

We welcome all forms of contributions! Please read our Contributing Guide for details.

Technology Stack

LMeterX adopts a modern technology stack to ensure system reliability and maintainability:

• Backend Service: Python + FastAPI + SQLAlchemy + MySQL
• Load Testing Engine: Python + Locust + Custom Extensions
• Frontend Interface: React + TypeScript + Ant Design + Vite
• Deployment & Operations: Docker + Docker Compose + Nginx

Development Environment Setup

1. Fork the Project to your GitHub account
2. Clone Your Fork, create a development branch for development
3. Follow Code Standards, use clear commit messages (follow conventional commit standards)
4. Run Code Checks: Before submitting PR, ensure code checks, formatting, and tests all pass, you can run make all
5. Write Clear Documentation: Write corresponding documentation for new features or changes
6. Actively Participate in Review: Actively respond to feedback during the review process

🗺️ Development Roadmap

In Development

• Support for client resource monitoring

Planned

• CLI command-line tool

🗂️ Dataset Reference Notes

LMeterX builds test samples based on the open-source ShareGPT dataset, strictly adhering to the original license requirements.

• Data Source: Uses the ShareGPT dataset as the original dialogue corpus.

• Adjustment Scope:

• Filtered high-quality dialogue samples, removing low-quality or irrelevant data for the load testing scenario.

• Random sampling was performed to reduce the data size while preserving diverse dialogues.

👥 Contributing

We welcome any contributions from the community! Please refer to our Contributing Guide Thanks to all developers who have contributed to the LMeterX project!

📝 Citation

If you use EvalScope in your research, please cite our work:

@software{LMeterX2025,
  author  = {LMeterX Team},
  title   = {LMeterX: Enterprise-Grade Performance Benchmarking Platform for Large Language Models},
  year    = {2025},
  url     = {https://github.com/MigoXLab/LMeterX},
}

📄 Open Source License

This project is licensed under the Apache 2.0 License.