Project Prometheus is building faster, more efficient datacenter systems by co-designing distributed systems with new network primitives. Prometheus takes advantage of new programmable hardware to accelerate applications. We are working across the entire system stack, from applications and distributed algorithms to network design and device architecture.
In the last decade, the scale of distributed systems in the datacenter has grown dramatically. At the same time, single-node performance has increased substantially, thanks largely to the deployment of new types of accelerators. The result is that communication, coordination, and other classic distributed systems challenges have become the bottleneck in many systems. Our goal is to build a distributed systems accelerator that addresses this bottleneck and gives order-of-magnitude improvements in distributed system performance and efficiency.
The enabling technologies are already here, in the form of programmable network cards and network switches, capable of running custom logic at terabits per second. We are using them in a new way, to support distributed systems rather than traditional network protocols. This requires co-design between distributed algorithms and new network functionality.
Some examples of the benefits gained from this approach include:
- replicated storage systems that scale with the number of replicas
- up to a 35x speedup in transaction processing performance
- a reduction in DNN training time by as much as 67%
- a key-value store that meets strict latency SLOs using 88% fewer machines