Self-host vLLM with Levelrail

A fast, high-throughput inference server that exposes an OpenAI-compatible API for open-weight language models.

  • Authorvllm-project
  • LicenceApache-2.0
  • Memory8 GiB
  • Archarm64
  • Services1

How to self-host vLLM

  1. Pull the images

    One image is pulled on your server.

    Show images
    • vllm/vllm-openai:v0.30.0
  2. Create the app

    One service starts in a single Compose app, with 1 named volume mount for your data, on port 8000.

  3. Fill in the secrets

    1 setting is preset, you do not have to fill in anything.

  4. Open it

    Without a domain the app gets an sslip.io address; add --domain SERVICE=HOST to use your own.

vLLM requirements

Memory
About 8 GiB
Architectures
amd64 image published, arm64 image published
Ports
8000
Domain
Optional, an sslip.io address is used without one
Settings
none to fill in
License
Apache-2.0, permissive license
  • Listed. It is in the catalog but not verified. Back up the data volume before changing an image tag.
  • GPU. It needs an NVIDIA GPU on the node to be practical.
  • Host ports. It publishes port 8000 on the host, so open them in the VPS firewall only if you want that.

Install Levelrail first

On a server with Levelrail running, run the command above, or open the dashboard: Apps, New app, Browse templates, then vLLM. New to Levelrail? Install it on a fresh VPS first.

docker pull ghcr.io/glincker/levelrail:latest

Then levelrail-cli login --device. See Getting started and the vLLM docs page.

Deploy vLLM your way

Every format below deploys the same pinned images. Pick the one that fits how you work.

Deploy vLLM from the CLI

levelrail-cli templates deploy vllm --name my-vllm

Deploy vLLM with the API

Create a token in the dashboard first. This one-click route deploys it as a trial app you can keep or remove.

curl -X POST -H "Authorization: Bearer $TOKEN" \
  https://levelrail.example.com/api/v1/service-templates/vllm/deploy

vLLM configuration

vLLM services

ServiceImagePortsVolumes
vllm vllm/vllm-openai:v0.30.0 8000 vllm_hf_cache -> /root/.cache/huggingface

vLLM environment variables

VariableServiceValue
HF_HOME vllm Preset by the template

Ports and volumes

Ports
8000
Volumes
1 named mount: vllm_hf_cache -> /root/.cache/huggingface

vLLM FAQ

How much memory does vLLM need?

The catalog lists about 8 GiB of memory for vLLM.

Does vLLM run on arm64?

For vLLM, an arm64 image is published; amd64 is image published.

What licence is vLLM under?

vLLM is licensed under Apache-2.0, which is permissive, with few conditions on reuse.

Does vLLM need configuration before it starts?

No. The template fills in its own defaults.

Which ports does vLLM use?

vLLM listens on port 8000 inside the stack.

Where does vLLM keep its data?

In a named volume: vllm_hf_cache mounted at /root/.cache/huggingface.

Can I deploy vLLM on Dokploy or Dokku?

The services and pinned images on this page can be run on any Docker host. The public Dokploy templates list we compared does not have an entry with the same id for vLLM, so on Dokploy you would use a Docker Compose application with the file from this page. Dokku deploys container images with its own commands, so the pinned image vllm/vllm-openai:v0.30.0 is what you deploy there.

levelrail-cli templates deploy vllm --name my-vllm