Self-host LiteLLM Proxy with Levelrail
One OpenAI-format gateway in front of 100+ model providers, with keys, budgets, and spend tracking.
- Memory1 GiB
- Archarm64
- Services2
How to self-host LiteLLM Proxy
-
Pull the images
2 images are pulled on your server.
Show images
postgres:16-alpineghcr.io/berriai/litellm:main-stable
-
Create the app
2 services start in a single Compose app, with 1 named volume mount for your data, on port 4000.
-
Fill in the secrets
3 secrets are generated and stored encrypted, 3 settings are preset, you do not have to fill in anything.
-
Open it
Without a domain the app gets an sslip.io address; add --domain SERVICE=HOST to use your own.
LiteLLM Proxy requirements
- Memory
- About 1 GiB
- Architectures
- amd64 image published, arm64 image published
- Ports
- 4000
- Domain
- Optional, an sslip.io address is used without one
- Settings
- none to fill in
- License
- not yet recorded
- Listed. It is in the catalog but not verified. Back up the data volume before changing an image tag.
- Host ports. It publishes port 4000 on the host, so open them in the VPS firewall only if you want that.
Install Levelrail first
On a server with Levelrail running, run the command above, or open the dashboard: Apps, New app, Browse templates, then LiteLLM Proxy. New to Levelrail? Install it on a fresh VPS first.
docker pull ghcr.io/glincker/levelrail:latest Then levelrail-cli login --device. See Getting started and the LiteLLM Proxy docs page.
Deploy LiteLLM Proxy your way
Every format below deploys the same pinned images. Pick the one that fits how you work.
Deploy LiteLLM Proxy from the CLI
levelrail-cli templates deploy litellm --name my-litellm Deploy LiteLLM Proxy with the API
curl -X POST -H "Authorization: Bearer $TOKEN" \
https://levelrail.example.com/api/v1/service-templates/litellm/deploy LiteLLM Proxy configuration
LiteLLM Proxy services
| Service | Image | Ports | Volumes |
|---|---|---|---|
| db | postgres:16-alpine | none | litellm_db_data -> /var/lib/postgresql/data |
| litellm | ghcr.io/berriai/litellm:main-stable | 4000 | none |
LiteLLM Proxy environment variables
| Variable | Service | Value |
|---|---|---|
| POSTGRES_DB | db | Preset by the template |
| POSTGRES_PASSWORD | db | Generated at deploy |
| POSTGRES_USER | db | Preset by the template |
| DATABASE_URL | litellm | Generated at deploy |
| LITELLM_MASTER_KEY | litellm | Generated at deploy |
| STORE_MODEL_IN_DB | litellm | Preset by the template |
Ports and volumes
- Ports
- 4000
- Volumes
- 1 named mount: litellm_db_data -> /var/lib/postgresql/data
LiteLLM Proxy FAQ
How much memory does LiteLLM Proxy need?
The catalog lists about 1 GiB of memory for LiteLLM Proxy across its 2 services.
Does LiteLLM Proxy run on arm64?
For LiteLLM Proxy, an arm64 image is published; amd64 is image published.
Does LiteLLM Proxy need configuration before it starts?
No. The template fills in its own defaults and generates secrets at deploy time.
Which database does LiteLLM Proxy bundle?
The template includes Postgres, running as its own service.
Which ports does LiteLLM Proxy use?
LiteLLM Proxy listens on port 4000 inside the stack.
Where does LiteLLM Proxy keep its data?
In a named volume: litellm_db_data mounted at /var/lib/postgresql/data.
Can I deploy LiteLLM Proxy on Dokploy or Dokku?
The services and pinned images on this page can be run on any Docker host. The public Dokploy templates list has an entry with the id litellm for this app, so check that entry as well. Dokku deploys container images with its own commands, so the pinned image postgres:16-alpine, ghcr.io/berriai/litellm:main-stable is what you deploy there.