🐝 Swarm Parameter Server

Gradient accumulation over HTTP — a software interconnect in place of NVLink.

Global loss

Swarm contributors

WorkerAcceptedStepsLocal steps StaleCompressionUploadedLast lossIdle

Sample the global model

The model starts from random weights — early samples are noise. Watch them
become English as the swarm trains.