How a 64-GPU cluster comes up in 84 seconds

When we say a cluster is ready in about 90 seconds, people ask what we cut. The honest answer is nothing. We moved the slow parts earlier, so they have already happened by the time you ask.

The timeline

Here is a real 64-GPU Forge 80 cluster in ams-1, taken from our launch logs last week.

$ ferrite cluster create --gpu forge-80 --count 64 --region ams-1
  0.0s  request accepted, 8 nodes reserved
  6.2s  nodes powered, firmware checked
 31.8s  image mounted from hall cache
 58.4s  fabric test passed (64 GPUs, 3.1 TB/s)
 84.1s  cluster "bright-otter" ready

$ ferrite cluster create --gpu forge-80 --count 64 --region ams-1
  0.0s  request accepted, 8 nodes reserved
  6.2s  nodes powered, firmware checked
 31.8s  image mounted from hall cache
 58.4s  fabric test passed (64 GPUs, 3.1 TB/s)
 84.1s  cluster "bright-otter" ready

$ ferrite cluster create --gpu forge-80 --count 64 --region ams-1
  0.0s  request accepted, 8 nodes reserved
  6.2s  nodes powered, firmware checked
 31.8s  image mounted from hall cache
 58.4s  fabric test passed (64 GPUs, 3.1 TB/s)
 84.1s  cluster "bright-otter" ready

Images come from the same hall

Every hall keeps a cache of the most used training images on local flash. A cold pull from a registry takes minutes. A mount from the hall cache takes seconds, and it is the same bytes.

The fabric test is not optional

Before a cluster is handed over we run a short all-to-all test across every GPU. It costs about 25 seconds. We tried skipping it once and won't again: one bad cable can halve a run's speed, and you would only find out hours in.

The fastest cluster is the one you don't have to rebuild at 3am because a link was flapping.

What we still can't speed up

• Powering nodes from cold takes a few seconds and is bound by the hardware

• Large custom images that aren't cached yet still pull from your registry

• Clusters over 512 GPUs span two rows and need a second fabric check

If you launch the same image often, pin it to the hall cache from the console and your next launch skips the pull entirely.