Tesla and xAI have been to scale coherent GPU AI clusters beyond the 33,000 GPU limit by NOT synchronize all nodes simultaneously. Synchronizing all nodes becomes increasingly challenging at scale – ...