Skip to content
State of GPU Compute 2026

Use case · Inference

Serve fast. Scale to zero.

Put an open or custom model behind a Serverless endpoint or a dedicated 8-GPU server. Workers start when requests queue, stop when traffic drops, and the meter counts seconds.
Get startedTalk to sales

from €3.00 /GPU·hrper-second metermin 0 workers

A Nexpod Serverless endpoint: requests stream into one H200 server while worker replicas scale in and out above it, billed per second.
serverless / llm-eu
live
llm-euServingH200 · EU-FRA-1
Req/s
1.24k
p95
291 ms
Workers
9/12
+6 workers
cost now €36.00/hr9 GPUs · per sec

Teams serving models on Nexpod

Placeholder
  • Customer
  • CUSTOMER
  • customer
  • Customer Co
  • customer.ai
  • CUSTOMER

Customer stories

Endpoints in production.

Chat assistantH200
Customer
PlaceholderServerless

Placeholder quote. A customer explains, in two plain sentences, what changed once their chat assistant workload moved to Nexpod.

Customer nameRole, Company
Read case study
Image generationB200
Customer
PlaceholderServerless

Placeholder quote. A customer explains, in two plain sentences, what changed once their image generation workload moved to Nexpod.

Customer nameRole, Company
Read case study
Speech to textH100
Customer
PlaceholderCloud Servers

Placeholder quote. A customer explains, in two plain sentences, what changed once their speech to text workload moved to Nexpod.

Customer nameRole, Company
Read case study
Search rankingH100
Customer
PlaceholderServerless

Placeholder quote. A customer explains, in two plain sentences, what changed once their search ranking workload moved to Nexpod.

Customer nameRole, Company
Read case study

Bigger models

Fit the whole model on one node.

A B300 server carries eight GPUs with 288 GB HBM3e each. Large open models load onto a single machine, so you skip splitting them across hosts.

See B300 servers
2,304
GB HBM3e per node
8 × 288 GB · NVIDIA B300

01Latency

Fast first token, steady p95.

Weights load from local NVMe and stay resident on the GPU. Set a floor of warm workers and the slow path stops being the common one.

Warm workers
Keep a minimum per endpoint so the first request of the day never waits.
Batching per GPU
Run vLLM, TGI or your own server and batch requests on every device.
Latency · llm-eu
p50p95
p95 291 ms
SLO 300 ms
182 ms

02Cost

Idle costs nothing. Busy costs seconds.

Serverless workers are metered per second of runtime. When the queue is empty, the endpoint drops to zero workers and zero euros.

Per-second metering
€0.00083 per GPU-second on H100. No rounding up to the hour.
Reserved floor
Book a baseline at the reserved rate. Traffic above it runs at the on-demand rate.
Spend · last 60 min
€1.7317
H100 · €0.00083/GPU·s
billed per second
€0.1983 · 238 GPU-s
−60m€0.00idlenow

03Deploy

One command to a live endpoint.

Point nexctl at a container image or a Hub template. You get an HTTPS endpoint with logs, metrics and health checks attached.

Any container
Bring your own image, or start from a vLLM template in Nexpod Hub.
Health checks included
Failing workers are drained and replaced without a redeploy.
  1. 1

    Pick a template or image

    A Hub template brings the server, routes and scaling defaults.

    hub/vllmOpenAI-style routes · batching
    hub/tgitext generation server
    ghcr.io/your-org/llm:1.4your own image
  2. 2

    Deploy with nexctl

    Choose the family and the worker range. The image is pulled once per region.

    $ nexctl endpoints deploy llm-eu \
        --template hub/vllm \
        --family h200 --min 1 --max 24
    → pulling image · 9.4 GB
    → warming 1 worker on 8× H200
    ✓ live in 38s
  3. 3

    Send the first request

    An HTTPS endpoint with logs and metrics, scaling on queue depth.

    llm-eulive in 38 s
    https://llm-eu.eu-fra-1.nexpod.run/v1
    health
    passing
    first req
    182 ms
    workers
    1 → 24

Developer tools

Deploy from a laptop, ship from CI.

A REST API at api.nexpod.ai/v1, Python and TypeScript SDKs, and nexctl for everything else. Promote a revision from your pipeline and roll it back with one command.

Get startedRead the docs
import nexpod
 
client = nexpod.Client()  # reads NEXPOD_API_KEY
 
endpoint = client.endpoints.create(
    name="llm-eu",
    template="hub/vllm",
    family="H200",
    region="eu-fra-1",
    workers={"min": 1, "max": 12},
)
 
reply = endpoint.run({"prompt": "Summarise this ticket in one line."})
print(reply.output)

REST API

Every console action is an API call first, documented and versioned.

$ curl api.nexpod.ai/v1/endpoints

SDKs and nexctl

Python and TypeScript SDKs plus one CLI binary for macOS and Linux.

$ pip install nexpod

Deploy on merge

Add one step to your pipeline and every merge ships a new revision.

$ nexctl endpoints deploy --wait

More use cases

Same servers, other workloads.

ft-mistral-07 · loss0.61

Fine-tuning

Adapt open models on hardware you control

run_8f21c · trace6.9 s

Agents

Bursty, long-running agent workloads

render-batch-11475%

Compute-heavy tasks

Rendering, simulation and batch jobs

Start with inference

Your endpoint, live today. Billed per second.

Deploy a model from a template or your own image, send the first request, and watch the meter count seconds instead of hours.

Get startedTalk to sales
On-demand ratesEUR · excl. VAT
  • B300€6.00/GPU·hr
  • B200€5.00/GPU·hr
  • H200€4.00/GPU·hr
  • H100€3.00/GPU·hr