In-Network Splitter Function Enabling Highly-Parallel Event Stream Processing

Abstract

Cloud data center workloads are comprised with a substantial volume of data-analytics tasks which rely heavily on data parallel stream processing. Splitting the data streams into independent data partitions and their distribution across multiple operators is a heavy operation than can become a performance bottleneck. Emerging in-network hardware accelerators in cloud infrastructures can enhance the performance of the splitter function and improve the parallelism degree of data-intensive cloud applications. Building on Programming Protocol-independent Packet Processors (P4), we devised a P4-based splitter fully in the data plane to benefit from line-rate performance of P4 packet processors, e.g., Tofino Switch. We present a summary of the splitter switch capabilities and complement our proposed design with new discussion points for future consideration.