🍋 Lemonade: Local LLM Serving with GPU and NPU acceleration

Download | Documentation | Discord

Lemonade helps users run local LLMs with the highest performance by configuring state-of-the-art inference engines for their NPUs and GPUs.

Startups such as Styrk AI (https://styrk.ai/styrk-ai-and-amd-guardrails-for-your-on-device-ai-revolution/), research teams like Hazy Research at Stanford (https://www.amd.com/en/developer/resources/technical-articles/2025/minions--on-device-and-cloud-language-model-collaboration-on-ryz.html), and large companies like AMD (https://www.amd.com/en/developer/resources/technical-articles/unlocking-a-wave-of-llm-apps-on-ryzen-ai-through-lemonade-server.html) use Lemonade to run LLMs.

Getting Started

1. Install: Windows (https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) · Ubuntu (https://lemonade-server.ai/install_options.html) · Source (https://lemonade-server.ai/install_options.html)
2. Get Models: Browse and download with the Model Manager
3. Chat: Try models with the built-in chat interface
4. Connect: Use Lemonade with your favorite apps:

Want your app featured here? Discord · GitHub Issue · Email

Using the CLI

To run and chat with Gemma 3:

lemonade-server run Gemma-3-4b-it-GGUF

To install models ahead of time, use the pull command:

lemonade-server pull Gemma-3-4b-it-GGUF

To check all models available, use the list command:

lemonade-server list

Tip: You can use --llamacpp vulkan/rocm to select a backend when running GGUF models.

Model Library

Lemonade supports GGUF, FLM, and ONNX models across CPU, GPU, and NPU (see supported configurations).

Use lemonade-server pull or the built-in Model Manager to download models. You can also import custom GGUF/ONNX models from Hugging Face.

Browse all built-in models → (https://lemonade-server.ai/docs/server/server_models/)

Supported Configurations

Lemonade supports the following configurations, while also making it easy to switch between them at runtime. Find more information about it here.

• See supported AMD ROCm platforms

  Architecture
  Platform Support
  GPU Models
  
  
  
  
  gfx1151 (STX Halo)
  Windows, Ubuntu
  Ryzen AI MAX+ Pro 395
  
  
  gfx120X (RDNA4)
  Windows, Ubuntu
  Radeon AI PRO R9700, RX 9070 XT/GRE/9070, RX 9060 XT
  
  
  gfx110X (RDNA3)
  Windows, Ubuntu
  Radeon PRO W7900/W7800/W7700/V710, RX 7900 XTX/XT/GRE, RX 7800 XT, RX 7700 XT
  

Project Roadmap

Integrate Lemonade Server with Your Application

You can use any OpenAI-compatible client library by configuring it to use http://localhost:8000/api/v1 as the base URL. A table containing official and popular OpenAI clients on different languages is shown below.

Feel free to pick and choose your preferred language.

Python Client Example

from openai import OpenAI

# Initialize the client to use Lemonade Server
client = OpenAI(
    base_url="http://localhost:8000/api/v1",
    api_key="lemonade"  # required but unused
)

# Create a chat completion
completion = client.chat.completions.create(
    model="Llama-3.2-1B-Instruct-Hybrid",  # or any other available model
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

# Print the response
print(completion.choices[0].message.content)

For more detailed integration instructions, see the Integration Guide.

Beyond an LLM Server

The Lemonade Python SDK is also available, which includes the following components:

• 🐍 Lemonade Python API: High-level Python API to directly integrate Lemonade LLMs into Python applications.
• 🖥️ Lemonade CLI: The lemonade CLI lets you mix-and-match LLMs (ONNX, GGUF, SafeTensors) with prompting templates, accuracy testing, performance benchmarking, and memory profiling to characterize your models on your hardware.

FAQ

To read our frequently asked questions, see our FAQ Guide

Contributing

We are actively seeking collaborators from across the industry. If you would like to contribute to this project, please check out our contribution guide.

New contributors can find beginner-friendly issues tagged with "Good First Issue" to get started.

Maintainers

This project is sponsored by AMD. It is maintained by @danielholanda @jeremyfowers @ramkrishna @vgodsoe in equal measure. You can reach us by filing an issue (https://github.com/lemonade-sdk/lemonade/issues), emailing [email protected], or joining our Discord (https://discord.gg/5xXzkMu8Zk).

License and Attribution

This project is:

• Built with C++ (server) and Python (SDK) with ❤️ for the open source community,
• Standing on the shoulders of great tools from:
  • ggml/llama.cpp (https://github.com/ggml-org/llama.cpp)
  • OnnxRuntime GenAI (https://github.com/microsoft/onnxruntime-genai)
  • Hugging Face Hub (https://github.com/huggingface/huggingface_hub)
  • OpenAI API (https://github.com/openai/openai-python)
  • IRON/MLIR-AIE (https://github.com/Xilinx/mlir-aie)
  • and more...
• Accelerated by mentorship from the OCV Catalyst program.
• Licensed under the Apache 2.0 License (https://github.com/lemonade-sdk/lemonade/blob/main/LICENSE).
  • Portions of the project are licensed as described in NOTICE.md.