Self hosted LLM gateway

A self hosted LLM gateway is a proxy that runs on your own machine or server and sits between your agents and the model APIs. RelayPlane is one you can start with two commands, it is free under the MIT license, and it can stop a run once it hits a dollar cap.

Your provider keys stay on your hardware, and every request passes through a process you control.

Run it on your laptop first

Local is the quickest test. Install the package and start the proxy, then point your client at port 4100. The quick start says Node.js 18 or newer is enough, and Claude Code users on a Max plan need no API key.

npm install -g @relayplane/proxy
relayplane start
export ANTHROPIC_BASE_URL=http://localhost:4100

Run relayplane stats to see cost by model, or open http://localhost:4100 for the dashboard. The installation page also covers npx and a build from source.

Move it to a server when a team shares it

A laptop proxy dies when the lid closes. For a shared box, the VPS deployment guide uses Ubuntu 22.04 or Debian 12, Node.js 20 or newer, and a domain name for TLS. It runs the proxy under systemd or PM2 and puts nginx in front. TLS comes from Let's Encrypt.

sudo npm install -g @relayplane/proxy
sudo systemctl daemon-reload
sudo systemctl enable relayplane
sudo systemctl start relayplane

The guide keeps provider keys in /etc/relayplane/.env with mode 600, sets the port with RELAYPLANE_PORT, and tells you to block the proxy port with ufw so only nginx can reach it. It also says to pin an exact version in production instead of using @latest.

Point your tools at it

Clients change one base URL. Nothing else in the agent code moves.

# ~/.claude/settings.json (Claude Code)
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://proxy.yourdomain.com/anthropic"
  }
}

# .env for any app using the OpenAI SDK
OPENAI_BASE_URL=https://proxy.yourdomain.com/openai

Hit the health endpoint at /health to confirm the proxy answers before you switch real traffic over.

Add a spend cap on day one

Owning the gateway means nobody else stops a runaway agent for you. One command sets a daily limit, and a running proxy reloads it within seconds, per the budget cap docs.

relayplane cap set --day 10

The same limit by hand lives in the config file. With onBreach set to block, the client gets a 429 and the request is never forwarded. The other two settings are warn and downgrade, which sends the call to a cheaper model instead.

// ~/.relayplane/config.json
{
  "budget": {
    "enabled": true,
    "dailyUsd": 10,
    "dailyCapUSD": 10,
    "onBreach": "block"
  }
}

To cap a single job instead of the day, see how to cap AI spend per task.

What self hosting costs you

The software costs nothing. The server is yours to pay for, and so is the upkeep. You patch the OS and watch the service logs. Certbot renews the certificate. Upgrades happen when you choose. Model spend is still billed by whichever provider you call.

Telemetry is off by default, so nothing leaves your network unless you turn it on. See the telemetry docs.

When another gateway fits better

If you want a hosted service so you run nothing yourself, a self hosted proxy is the wrong shape. The same goes for teams that need a different set of features than a single proxy offers. The LLM gateway comparison lays out where each tool is stronger, and the open source gateway page compares the self hosted options on cost tracking.

Start the proxy

MIT licensed and free. Two commands to a running gateway.

Open the quick start