Self-host Promptfoo with Levelrail

Test and compare prompts, models and agents with a local web UI for browsing evaluation results. Runs the viewer with results stored on a volume.

  • Authorpromptfoo
  • LicenceMIT
  • Memory256 MiB
  • Services1

How to self-host Promptfoo

  1. Pull the images

    One image is pulled on your server.

    Show images
    • ghcr.io/promptfoo/promptfoo:0.125.1
  2. Create the app

    One service starts in a single Compose app, with 1 named volume mount for your data, on port 3000.

  3. Fill in the secrets

    You do not have to fill in anything.

  4. Open it

    Without a domain the app gets an sslip.io address; add --domain SERVICE=HOST to use your own.

After the first start

  • Open the address to browse evals. Run evaluations from the CLI with this server as the share target.

Promptfoo requirements

Memory
About 256 MiB
Architectures
amd64 unknown, arm64 unknown
Ports
3000
Domain
Optional, an sslip.io address is used without one
Settings
none to fill in
License
MIT, permissive license
  • Listed. It is in the catalog but not verified. Back up the data volume before changing an image tag.

Install Levelrail first

On a server with Levelrail running, run the command above, or open the dashboard: Apps, New app, Browse templates, then Promptfoo. New to Levelrail? Install it on a fresh VPS first.

docker pull ghcr.io/glincker/levelrail:latest

Then levelrail-cli login --device. See Getting started and the Promptfoo docs page.

Deploy Promptfoo your way

Every format below deploys the same pinned images. Pick the one that fits how you work.

Deploy Promptfoo from the CLI

levelrail-cli templates deploy promptfoo --name my-promptfoo

Deploy Promptfoo with the API

Create a token in the dashboard first. This one-click route deploys it as a trial app you can keep or remove.

curl -X POST -H "Authorization: Bearer $TOKEN" \
  https://levelrail.example.com/api/v1/service-templates/promptfoo/deploy

Promptfoo configuration

Promptfoo services

ServiceImagePortsVolumes
promptfoo ghcr.io/promptfoo/promptfoo:0.125.1 3000 data -> /home/promptfoo/.promptfoo

Promptfoo environment variables

The catalog lists no environment variables for this template.

Ports and volumes

Ports
3000
Volumes
1 named mount: data -> /home/promptfoo/.promptfoo

Promptfoo FAQ

How much memory does Promptfoo need?

The catalog lists about 256 MiB of memory for Promptfoo.

Does Promptfoo run on arm64?

For Promptfoo, arm64 support is not confirmed; amd64 is unknown.

What licence is Promptfoo under?

Promptfoo is licensed under MIT, which is permissive, with few conditions on reuse.

Does Promptfoo need configuration before it starts?

No. The template fills in its own defaults.

Which ports does Promptfoo use?

Promptfoo listens on port 3000 inside the stack.

Where does Promptfoo keep its data?

In a named volume: data mounted at /home/promptfoo/.promptfoo.

Can I deploy Promptfoo on Dokploy or Dokku?

The services and pinned images on this page can be run on any Docker host. The public Dokploy templates list we compared does not have an entry with the same id for Promptfoo, so on Dokploy you would use a Docker Compose application with the file from this page. Dokku deploys container images with its own commands, so the pinned image ghcr.io/promptfoo/promptfoo:0.125.1 is what you deploy there.

levelrail-cli templates deploy promptfoo --name my-promptfoo