Your first multi-node run on Instant Clusters

One 8-GPU server takes you a long way. When a run needs more memory or more throughput than a single machine has, an Instant Cluster joins several servers on one fast fabric. This guide takes a working single-node script to four nodes.

Create the cluster

A cluster is made of identical nodes in one region. Four H200 nodes give you 32 GPUs and about 4.5 TB of HBM between them.

Test the fabric first

Before a long run, spend two minutes on a collective test. If one node has a weak link you want to know now, not six hours into training.

Every node should report about the same bus bandwidth. A node that is clearly slower than the rest can be swapped with nexctl clusters replace-node, without rebuilding the cluster.

Launch with torchrun

Your training script stays the same. torchrun starts eight processes on each node and joins them into one process group. The cluster exports the rendezvous address as NEXPOD_MASTER_ADDR.

Checkpoint to shared storage

Write checkpoints to a volume that every node mounts, never to local disk. When a node fails you replace it and resume from the last checkpoint instead of starting again.

• Save every 30 to 60 minutes of training time.
• Keep the last three checkpoints and prune the older ones.
• Test a resume once, early, so you know it works on the day you need it.

Know what the run costs

A four-node H200 cluster runs at four times the server rate: €128.00 per hour. Billing is per second, per node, and stops when you delete the cluster. Here is the bill for an example run of 6 hours and 12 minutes.

Delete the cluster as soon as the last checkpoint is written. A cluster left running over a weekend costs more than most runs do.

Next steps

Once four nodes run cleanly, going wider is a change to one flag. The Instant Clusters page lists the node counts available in each region.