GPT-Load

English | 中文 | 日本語

Release (https://img.shields.io/github/v/release/tbphp/gpt-load) !Go Version (https://img.shields.io/badge/Go-1.23+-blue.svg) License (https://img.shields.io/badge/license-MIT-green.svg)

A high-performance, enterprise-grade AI API transparent proxy service designed specifically for enterprises and developers who need to integrate multiple AI services. Built with Go, featuring intelligent key management, load balancing, and comprehensive monitoring capabilities, designed for high-concurrency production environments.

For detailed documentation, please visit Official Documentation (https://www.gpt-load.com/docs?lang=en)

Features

• Transparent Proxy: Complete preservation of native API formats, supporting OpenAI, Google Gemini, and Anthropic Claude among other formats
• Intelligent Key Management: High-performance key pool with group-based management, automatic rotation, and failure recovery
• Load Balancing: Weighted load balancing across multiple upstream endpoints to enhance service availability
• Smart Failure Handling: Automatic key blacklist management and recovery mechanisms to ensure service continuity
• Dynamic Configuration: System settings and group configurations support hot-reload without requiring restarts
• Enterprise Architecture: Distributed leader-follower deployment supporting horizontal scaling and high availability
• Modern Management: Vue 3-based web management interface that is intuitive and user-friendly
• Comprehensive Monitoring: Real-time statistics, health checks, and detailed request logging
• High-Performance Design: Zero-copy streaming, connection pool reuse, and atomic operations
• Production Ready: Graceful shutdown, error recovery, and comprehensive security mechanisms
• Dual Authentication: Separate authentication for management and proxy, with proxy authentication supporting global and group-level keys

Supported AI Services

GPT-Load serves as a transparent proxy service, completely preserving the native API formats of various AI service providers:

• OpenAI Format: Official OpenAI API, Azure OpenAI, and other OpenAI-compatible services
• Google Gemini Format: Native APIs for Gemini Pro, Gemini Pro Vision, and other models
• Anthropic Claude Format: Claude series models, supporting high-quality conversations and text generation

Quick Start

System Requirements

• Go 1.23+ (for source builds)
• Docker (for containerized deployment)
• MySQL, PostgreSQL, or SQLite (for database storage)
• Redis (for caching and distributed coordination, optional)

Method 1: Docker Quick Start

docker run -d --name gpt-load \
    -p 3001:3001 \
    -e AUTH_KEY=your-secure-key-here \
    -v "$(pwd)/data":/app/data \
    ghcr.io/tbphp/gpt-load:latest

Please change your-secure-key-here to a strong password (never use the default value), then you can log in to the management interface:

Method 2: Using Docker Compose (Recommended)

Installation Commands: