Cost per million tokens: self-hosting Qwen3.8 and Nemotron 3.5 on a rented H100 vs paying per token
We measured Qwen3.8-27B and Nemotron 3.5 Lightning on one $1.40/hour H100 on Lium. With the GPU kept busy, it did $12.37 an hour of Qwen3.8-27B API work for $1.40, and served Nemotron at $0.14 per million output tokens.
Lium team6 min read
llm-inference
cost
vllm
qwen
nemotron
h100
pricing
If you keep a GPU busy, serving a current open model yourself on Lium costs a fraction of a per-token API. On 10 November 2026 we served two of the newest open-weight models with vLLM on one H100 rented on Lium at $1.40 an hour: Qwen3.8-27B (listed on OpenRouter on 14 August 2026) and NVIDIA Nemotron 3.5 Lightning (30B total, 3B active, listed on 11 August 2026).
- Qwen3.8-27B: with 64 requests in flight the H100 generated 5.7 million output tokens an hour, $0.25 per million output tokens. The same hour of work costs $12.37 through the Qwen3.8-27B API, so self-hosting was 8.8x cheaper.
- Nemotron 3.5 Lightning: 10.1 million output tokens an hour, $0.14 per million output tokens, 2.4x cheaper than its own API and 5.0x cheaper than the same tokens through Claude Haiku 5.5.
The catch is load. With one request at a time the same GPU cost $8.06 (Qwen3.8-27B) and $1.53 (Nemotron) per million output tokens, more than the APIs. This post shows the measured numbers, the break-even point and how to stay on the cheap side of it.
Why self-hosting on Lium wins once you have steady traffic:
- 8.8x cheaper than the same model's API at full load: $1.40 an hour on Lium vs $12.37 an hour of API calls for the identical Qwen3.8-27B workload.
- Per-second billing with no minimum: you pay for the minutes the GPU runs. Our test of both models, install included, took 23 minutes and $0.54.
- No per-token meter at all: long prompts, retries and big batch jobs cost nothing extra once the GPU is yours.
- Your own fine-tunes: a fine-tuned open model runs as-is on the same GPU, with no per-token price for it at all.
- Built for agents: Claude Code and Codex can rent the GPU, start vLLM and remove the pod with the
liumCLI, on an API key with a server-enforced budget.
The test
Every number below comes from one run on 10 November 2026, 07:35 to 07:59 UTC.
- GPU: 1x H100 80 GB (one GPU of a Spot node in the United States), rented with
lium upat $1.40 per hour, the cheapest 1x H100 ask inlium ls --gpu H100at 07:35 UTC. The card ran at a 700 W power limit. - Models:
Qwen/Qwen3.8-27Bandnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, both in bf16. We picked the newest open-weight models that fit one 80 GB GPU and that an API also sells per token, so the comparison is like for like. - Server: vLLM 0.33.0,
--max-model-len 4096 --gpu-memory-utilization 0.90 --max-num-seqs 64, nothing else tuned. The--max-num-seqs 64cap is needed: with vLLM's default of 1,024 sequences, neither model starts next to its bf16 weights on one 80 GB card. - Load:
vllm bench servewith random prompts of 512 tokens and answers of exactly 256 tokens, at 1, 16 and 64 requests in flight (20, 128 and 512 requests, all completed).
The cost column divides the $1.40 hourly price by the output tokens produced in an hour and counts the prompt tokens as free, because on your own GPU they are. Nemotron only activates 3B of its 30B parameters per token, which is why it generated 1.8x the tokens of the dense Qwen3.8-27B on the same card.
The same work through a per-token API
To compare like with like, we priced the exact token mix of each run (two prompt tokens for every output token) at per-token API prices listed by OpenRouter's models API, read on 10 November 2026 at 08:27 UTC. The first two API lines are the same models we ran; the others are current small closed models a buyer would compare against. We compare cost only, not answer quality.
At 16 requests in flight the Lium GPU still beat each model's own API: $1.40 an hour against $4.95 for the Qwen3.8-27B tokens and $1.50 for the Nemotron tokens.
With one request at a time the APIs are cheaper: that hour of single-request traffic costs $0.38 through the Qwen3.8-27B API and $0.31 through the Nemotron API. A GPU you pay for by the hour only wins when it is busy.
Where self-hosting breaks even
For this workload shape (two prompt tokens per output token), the $1.40 H100 is cheaper than the same model's API once your average traffic is above:
- 179 output tokens per second for Qwen3.8-27B (0.64 million output tokens an hour), 11% of what the GPU sustained at 64 in flight,
- 1,144 output tokens per second for Nemotron 3.5 Lightning (4.1 million an hour), 41% of what the GPU sustained.
The bigger and pricier the model's API, the sooner your own GPU pays for itself. Workloads that clear that bar easily:
- Batch jobs: classifying, extracting or summarising a backlog of documents. Fill the queue, run at full concurrency, then remove the pod.
- Evals and synthetic data: thousands of prompts with long contexts, where prompt tokens dominate a per-token bill and cost nothing on your own GPU.
- Steady product traffic: an internal assistant or agent backend with requests all day.
- Fine-tuned open models that no API serves at all.
How to stay on the cheap side
- Batch your requests. vLLM's Qwen3.8-27B throughput went from 48 to 1,580 output tokens per second, 33x, just by sending 64 requests at once. Send work in parallel, not in a loop.
- Rent by the job, not by the month. On Lium you pay per second, so a batch job that needs two hours costs two hours.
--ttlremoves the pod when the window ends, even if your script crashes:bashlium up --gpu H100 --name batch --ttl 2h -y --no-ssh - Pick the GPU that holds the model. A 27B to 30B model in bf16 needs an 80 GB card such as the H100; a 7B or 8B model fits a 24 GB RTX 4090 for much less an hour. See best GPU cloud for LLM inference for sizing.
- Check live prices before you start. Asks move during the day;
lium ls --gpu H100lists them cheapest first, and /stats shows how many GPUs of each model are listed and rented.
The full setup, from
lium up to calling the endpoint with the openai client, is in Serve an open LLM with vLLM on a single Lium GPU.Let your agent run it
A batch job is a good fit for Claude Code or Codex: give the agent a Lium API key with a daily or lifetime USD budget, and it can rent the GPU, start vLLM, push the queue through at full concurrency and remove the pod. The budget is enforced by the server, not the agent: once a billing tick reaches it, Lium stops the key's pods, so a runaway loop cannot keep spending. See Give Claude Code a GPU and Give Codex a GPU.
Bottom line
A per-token API is a meter that never stops; a Lium GPU is a fixed $1.40 an hour that you can fill. In our test one H100 did $12.37 an hour of Qwen3.8-27B API work for $1.40, and the Nemotron 3.5 Lightning workload would have cost $7.06 an hour through Claude Haiku 5.5. If you have steady traffic or a batch of work to get through, rent a GPU on Lium, batch your requests, and keep the difference.
Run it on Lium
Get new posts by e-mail: Subscribe to the Lium newsletter
