Serve an open LLM with vLLM on a single Lium GPU

Serve Qwen2.5-7B-Instruct with vLLM on one RTX 4090 on Lium and call your own OpenAI-compatible endpoint from your laptop with curl and the openai Python client. 18 minutes of pod time, about 14 cents.

Lium team15 October 20265 min read

vllm

llm-inference

qwen

openai-api

rtx-4090

You don't need a rack of H100s to run your own LLM endpoint. A 7B model fits on one 24 GB consumer GPU, and vLLM turns it into an OpenAI-compatible API that any existing client can talk to. On Lium that GPU is a single command away and billed by the second.

In this guide we serve Qwen/Qwen2.5-7B-Instruct with vLLM on one RTX 4090, then call it from a laptop with curl and with the openai Python client. The model is Apache-2.0 licensed and needs no Hugging Face token. If you want a frontier-size model on a whole 8-GPU node, see Run DeepSeek V4.1 Flash on your own Lium GPUs; this post is the single-GPU version of the same idea.

Every command and number below comes from our test run on 15 October 2026: a 1x RTX 4090 at $0.47 per hour, ready 27 seconds after we asked for it, serving requests eight minutes after the rent, 18 minutes of pod time and about $0.14 in total.

1. Rent a GPU

You need a Lium account with a little balance and Python 3 on your laptop. Install the CLI and sign in once; lium init opens the browser and saves your API key and SSH key.

bash

curl -fsSL https://lium.io/install.sh | bash   # or: pip install lium.io
lium init

Qwen2.5-7B-Instruct takes about 14.3 GiB of GPU memory in bf16, so any 24 GB card leaves room for the KV cache. --gpu RTX4090 rents the cheapest matching node, --ttl 2h removes the pod two hours later even if you forget it, -y skips the confirmation and --no-ssh returns to your shell.

bash

lium up --gpu RTX4090 --name vllm --ttl 2h -y --no-ssh

Without --image or -t, the pod starts from the default PyTorch (CUDA + DinD) template. A one-line check confirms the GPU is visible:

bash

lium exec vllm 'python3 -c "import torch; print(torch.cuda.get_device_name(0))"'

It printed NVIDIA GeForce RTX 4090.

2. Install vLLM

The PyTorch template keeps pip pinned to the image's own torch build (see /etc/pip/constraints.txt on the pod), which protects the image from accidental upgrades. vLLM ships against its own torch version, so install it into a separate virtual environment with uv, which the template already includes:

bash

lium exec vllm 'uv venv -q /root/vllm-env && uv pip install -q --python /root/vllm-env vllm && /root/vllm-env/bin/vllm --version'

This took 87 seconds and installed vLLM 0.31.0.

3. Start the server

Generate an API key on your laptop and keep this terminal open; you'll use the key again in step 4.

bash

export VLLM_API_KEY=$(openssl rand -hex 16)

Start the server in the background with lium exec -d, which prints the PID and writes the output to a log file. --host 127.0.0.1 keeps the port private to the pod; you'll reach it through SSH. --max-model-len 8192 caps the context so the KV cache fits comfortably in 24 GB.

bash

lium exec vllm -d --log /root/vllm.log -e VLLM_API_KEY=$VLLM_API_KEY \
  '/root/vllm-env/bin/vllm serve Qwen/Qwen2.5-7B-Instruct --host 127.0.0.1 --port 8000 --max-model-len 8192'

Follow the log until it prints Application startup complete:

bash

lium exec vllm 'tail -n 20 /root/vllm.log'

On our pod the server was ready 4 minutes 19 seconds after the start. Most of that was the first download of the 15 GB of weights and vLLM's one-time profiling and CUDA graph warmup (86 seconds); reading the weights onto the GPU took 2.3 seconds. vLLM reported room for 105,552 tokens of KV cache, enough for about 12 requests at the full 8,192-token context at once.

4. Call the endpoint

Open a second terminal for an SSH tunnel. It sends your laptop's 127.0.0.1:8000 to the same address inside the pod without opening that port to the internet. The $(...) part prints the pod's SSH command from lium describe, and the flags after it turn that into a tunnel. Leave it running.

bash

$(lium describe vllm --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["access"]["ssh_cmd"])') \
  -N -L 8000:127.0.0.1:8000

Back in the first terminal, where VLLM_API_KEY is still set, send a chat request:

bash

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $VLLM_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model": "Qwen/Qwen2.5-7B-Instruct",
       "messages": [{"role": "user", "content": "Explain what a GPU does in two sentences."}]}'

The answer came back in 1.3 seconds, 57 completion tokens:

A GPU, or Graphics Processing Unit, is designed to handle the complex calculations required for rendering images, videos, and other visual elements in real-time, particularly in applications like video games and graphics-intensive software. It can also be used for general-purpose computing tasks that involve parallel processing.

A request without the key gets 401, so the endpoint stays yours even inside the tunnel.

Because the server speaks the OpenAI API, the official openai Python package works unchanged; only the base URL and key differ. Install it with pip install openai and save this as chat.py:

python

import os
from openai import OpenAI
 
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key=os.environ["VLLM_API_KEY"])
reply = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "Write a haiku about renting a GPU."}],
)
print(reply.choices[0].message.content)

bash

python3 chat.py

text

Cloud GPUs at hand,
Tasks compute in swift dance,
Renting power flows.

For a speed check we asked for an 800-word essay with max_tokens=1000: the 4090 generated 1,000 tokens in 16.3 seconds, about 61 tokens per second for a single request. vLLM batches concurrent requests, so total throughput with several clients is higher.

Anything that accepts an OpenAI base URL, from LangChain to your own app, can now point at http://127.0.0.1:8000/v1 with the same key.

Clean up

Stop the tunnel with Ctrl+C, then remove the pod. Billing stops the moment it is gone, and the two-hour TTL would have removed it anyway.

bash

lium rm vllm -y

Our pod ran for 18 minutes at $0.47 per hour, about $0.14 for the whole test.

Why Lium for this

A private endpoint for a 7B model is a one-GPU job, and Lium rents exactly one GPU by the second with no minimum, so an afternoon of testing costs cents rather than a monthly instance. The pod was ready in under half a minute, the CLI drives the whole loop from install to teardown without a browser, and --ttl makes sure a forgotten server never runs up a bill. When you outgrow one card, the same commands scale to the 8x H100 setup.

Run it on Lium

See RTX-4090 pricing and availability before you start.

Browse GPUs

Serve an open LLM with vLLM on a single Lium GPU

1. Rent a GPU

2. Install vLLM

3. Start the server

4. Call the endpoint

Clean up

Why Lium for this

Related articles

Run DeepSeek V4.1 Flash on your own Lium GPUs

Fine-tune an open LLM with LoRA on Lium

Best GPU cloud for open-source LLM inference in 2026