Self-host Text Embeddings Inference with Levelrail

A lean server that turns text into embeddings with popular open models, on CPU with no extra setup.

  • Authorhuggingface
  • LicenceApache-2.0
  • Memory2 GiB
  • Services1

How to self-host Text Embeddings Inference

  1. Pull the images

    One image is pulled on your server.

    Show images
    • ghcr.io/huggingface/text-embeddings-inference:cpu-1.8
  2. Create the app

    One service starts in a single Compose app, with 1 named volume mount for your data, on port 80.

  3. Fill in the secrets

    You do not have to fill in anything.

  4. Open it

    Without a domain the app gets an sslip.io address; add --domain SERVICE=HOST to use your own.

Text Embeddings Inference requirements

Memory
About 2 GiB
Architectures
amd64 image published, arm64 not available
Ports
80
Domain
Optional, an sslip.io address is used without one
Settings
none to fill in
License
Apache-2.0, permissive license
  • Listed. It is in the catalog but not verified. Back up the data volume before changing an image tag.
  • Host ports. It publishes port 80 on the host, so open them in the VPS firewall only if you want that.

Install Levelrail first

On a server with Levelrail running, run the command above, or open the dashboard: Apps, New app, Browse templates, then Text Embeddings Inference. New to Levelrail? Install it on a fresh VPS first.

docker pull ghcr.io/glincker/levelrail:latest

Then levelrail-cli login --device. See Getting started and the Text Embeddings Inference docs page.

Deploy Text Embeddings Inference your way

Every format below deploys the same pinned images. Pick the one that fits how you work.

Deploy Text Embeddings Inference from the CLI

levelrail-cli templates deploy text-embeddings-inference --name my-text-embeddings-inference

Deploy Text Embeddings Inference with the API

Create a token in the dashboard first. This one-click route deploys it as a trial app you can keep or remove.

curl -X POST -H "Authorization: Bearer $TOKEN" \
  https://levelrail.example.com/api/v1/service-templates/text-embeddings-inference/deploy

Text Embeddings Inference configuration

Text Embeddings Inference services

ServiceImagePortsVolumes
tei ghcr.io/huggingface/text-embeddings-inference:cpu-1.8 80 tei_data -> /data

Text Embeddings Inference environment variables

The catalog lists no environment variables for this template.

Ports and volumes

Ports
80
Volumes
1 named mount: tei_data -> /data

Text Embeddings Inference FAQ

How much memory does Text Embeddings Inference need?

The catalog lists about 2 GiB of memory for Text Embeddings Inference.

Does Text Embeddings Inference run on arm64?

For Text Embeddings Inference, amd64 only, no arm64 image; amd64 is image published.

What licence is Text Embeddings Inference under?

Text Embeddings Inference is licensed under Apache-2.0, which is permissive, with few conditions on reuse.

Does Text Embeddings Inference need configuration before it starts?

No. The template fills in its own defaults.

Which ports does Text Embeddings Inference use?

Text Embeddings Inference listens on port 80 inside the stack.

Where does Text Embeddings Inference keep its data?

In a named volume: tei_data mounted at /data.

Can I deploy Text Embeddings Inference on Dokploy or Dokku?

The services and pinned images on this page can be run on any Docker host. The public Dokploy templates list we compared does not have an entry with the same id for Text Embeddings Inference, so on Dokploy you would use a Docker Compose application with the file from this page. Dokku deploys container images with its own commands, so the pinned image ghcr.io/huggingface/text-embeddings-inference:cpu-1.8 is what you deploy there.

levelrail-cli templates deploy text-embeddings-inference --name my-text-embeddings-inference