Net4AI

Network acceleration for generative AI clusters involves optimizing the communication and data transfer between multiple AI models, often distributed across different compute nodes. In this context, a large number of research challenges need to be tackled. Reducing latency is crucial to support model parallelism and real-time interactions with end-users. Controlling how bandwidth is utilized is key to maximize throughput (large amount of data can be quickly transferred) and minimize job completion times. As clusters grow in size, scalability is also a challenge as the complexity of managing communication increases. Ensuring that the network can handle node failures gracefully is essential for reliability and fault tolerance. Optimized software frameworks using AI frameworks that are optimized for specific hardware accelerators, lead to better performance. Large Language Models (LLM) are so large that a dedicated computing cluster with thousands of GPUs is now needed both at training and inference time. E.g., Meta is designing dedicated clusters with the goal of training LLM in 1 day instead of 1 month. To accommodate ever-larger models, data parallelism is no longer enough and there has been a recent shift towards more sophisticated model parallelism techniques. Large models are divided into smaller, manageable pieces that can be distributed across multiple GPUs or TPUs. As a large number of messages, of small size, together with massive sets of elephants flows are exchanged, efficient communication between devices is essential. Latency builds up very quickly in large clusters, along with the number of hops, and potential congestion. The network fabric is becoming a significant bottleneck: network acceleration techniques will be developed from in Net4AI to remove this limit and deeply evaluated according to acceleration and power efficiency.