Accelerating AI with open standards: NoLimit's expanding vision
In May the foundation released Architecture 1.0. That document is the culmination of thousands of hours spent enhancing Ethernet for AI and HPC workloads. It defines a new network stack, from the application through the libfabric API to the NoLimit Transport (NLT) and the link and PHY optimizations beneath it, and enables a breadth of hardware and software optimizations designed from the ground up for performant next-generation deployments. With the hardware side complete, ongoing work covers everything needed for a robust software ecosystem, from standardizing configuration APIs to coordinating the necessary Linux kernel work.
Beyond 1.0
AI and HPC workloads are evolving quickly, and we continue to standardize the best ideas coming out of the field. The pace is obvious in the traffic itself: congestion patterns change faster than the hardware underneath them. That calls for flexible congestion management. To that end we are standardizing Programmable Congestion Management (PCM), which lets anyone implement a new congestion-control algorithm in a standard language and run it on any NIC that supports it. We are also standardizing Congestion Signaling (CSIG), which lets packets carry high-fidelity information about network congestion so the transport can react more accurately and more quickly.
A second focus is small-message performance. NLT 1.0 was designed for forward-looking scales of a million hosts coordinating on a single job. To get there, a basic NLT packet carries 104 bytes of headers. That is only 2.5 percent overhead on the 4096-byte packets many workloads target, but it is significant for smaller transactions such as 256-byte transfers. We are pursuing optimizations across the stack, including a reduced-size forwarding header, aiming to cut that overhead in half for optimized deployments. Smaller overhead means better efficiency for small-payload workloads, whether HPC or local scale-up fabrics.
To improve scale-up networks specifically, we are also working on an optimized transport layer. Scale-up networks are simpler than scale-out networks but have their own traffic management requirements, so work is in progress on a scale-up-focused transport that leverages the robust NLT defined in 1.0.
Finally, we continue to standardize in-network collectives for Ethernet. By moving the reduction operation common to AI and HPC out of the hosts and into the network, INC can double the performance of all-reduce, and latency-sensitive operations such as barriers and small all-gathers see substantial gains too.
We hope to wrap many of these technologies into an updated architecture release in the first half of next year.
Working with the wider ecosystem
Beyond the technology, we are collaborating with several other standards bodies. We are working with SNIA and NVM Express to improve storage application performance, and coordinating with OCP to standardize our Switch Abstraction Interface and Redfish configuration models, along with technologies such as CSIG that have uses outside AI and HPC networks.
We are excited about what comes next and about continuing to make Ethernet the dominant platform for AI and HPC networks. Members can read the 1.0 architecture through the member portal, and if you are not yet a member we invite you to join, contribute your ideas, and help define the future of networking for AI and HPC.