Increase 12 times！HKBU and MassGrid Publish New AI Training Algorithms with Low Bandwidth and High Efficiency
In terms of scaling efficiency, we evaluate gTop-k on a cluster with 32 GPU machines which are interconnected with 1 Gbps Ethernet. The experimental results show that our method achieves 2.7−12× higher