Self-host LiteLLM Proxy with Levelrail

One OpenAI-format gateway in front of 100+ model providers, with keys, budgets, and spend tracking.

  • Memory1 GiB
  • Archarm64
  • Services2

How to self-host LiteLLM Proxy

  1. Pull the images

    2 images are pulled on your server.

    Show images
    • postgres:16-alpine
    • ghcr.io/berriai/litellm:main-stable
  2. Create the app

    2 services start in a single Compose app, with 1 named volume mount for your data, on port 4000.

  3. Fill in the secrets

    3 secrets are generated and stored encrypted, 3 settings are preset, you do not have to fill in anything.

  4. Open it

    Without a domain the app gets an sslip.io address; add --domain SERVICE=HOST to use your own.

LiteLLM Proxy requirements

Memory
About 1 GiB
Architectures
amd64 image published, arm64 image published
Ports
4000
Domain
Optional, an sslip.io address is used without one
Settings
none to fill in
License
not yet recorded
  • Listed. It is in the catalog but not verified. Back up the data volume before changing an image tag.
  • Host ports. It publishes port 4000 on the host, so open them in the VPS firewall only if you want that.

Install Levelrail first

On a server with Levelrail running, run the command above, or open the dashboard: Apps, New app, Browse templates, then LiteLLM Proxy. New to Levelrail? Install it on a fresh VPS first.

docker pull ghcr.io/glincker/levelrail:latest

Then levelrail-cli login --device. See Getting started and the LiteLLM Proxy docs page.

Deploy LiteLLM Proxy your way

Every format below deploys the same pinned images. Pick the one that fits how you work.

Deploy LiteLLM Proxy from the CLI

levelrail-cli templates deploy litellm --name my-litellm

Deploy LiteLLM Proxy with the API

Create a token in the dashboard first. This one-click route deploys it as a trial app you can keep or remove.

curl -X POST -H "Authorization: Bearer $TOKEN" \
  https://levelrail.example.com/api/v1/service-templates/litellm/deploy

LiteLLM Proxy configuration

LiteLLM Proxy services

ServiceImagePortsVolumes
db postgres:16-alpine none litellm_db_data -> /var/lib/postgresql/data
litellm ghcr.io/berriai/litellm:main-stable 4000 none

LiteLLM Proxy environment variables

VariableServiceValue
POSTGRES_DB db Preset by the template
POSTGRES_PASSWORD db Generated at deploy
POSTGRES_USER db Preset by the template
DATABASE_URL litellm Generated at deploy
LITELLM_MASTER_KEY litellm Generated at deploy
STORE_MODEL_IN_DB litellm Preset by the template

Ports and volumes

Ports
4000
Volumes
1 named mount: litellm_db_data -> /var/lib/postgresql/data

LiteLLM Proxy FAQ

How much memory does LiteLLM Proxy need?

The catalog lists about 1 GiB of memory for LiteLLM Proxy across its 2 services.

Does LiteLLM Proxy run on arm64?

For LiteLLM Proxy, an arm64 image is published; amd64 is image published.

Does LiteLLM Proxy need configuration before it starts?

No. The template fills in its own defaults and generates secrets at deploy time.

Which database does LiteLLM Proxy bundle?

The template includes Postgres, running as its own service.

Which ports does LiteLLM Proxy use?

LiteLLM Proxy listens on port 4000 inside the stack.

Where does LiteLLM Proxy keep its data?

In a named volume: litellm_db_data mounted at /var/lib/postgresql/data.

Can I deploy LiteLLM Proxy on Dokploy or Dokku?

The services and pinned images on this page can be run on any Docker host. The public Dokploy templates list has an entry with the id litellm for this app, so check that entry as well. Dokku deploys container images with its own commands, so the pinned image postgres:16-alpine, ghcr.io/berriai/litellm:main-stable is what you deploy there.

levelrail-cli templates deploy litellm --name my-litellm