Paper Details

Reference:

Jeff Barber, Ximing Yu, Laney Kuenzel Zamore, Jerry Lin, Vahid Jazayeri, Shie Erlich, Tony Savor, and Michael Stumm,
"Bladerunner: Stream processing at scale for a live view of backend data mutations at the edge",
In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (SOSP'21), ACM, October, 2021, pp. 708–723.

Download:

PDF ; Talk

Abstract:

Consider a social media platform with hundreds of millions of online users at any time, utilizing a social graph that has many billions of nodes and edges. The problem this paper addresses is how to provide each user a continuously fresh, up-to-date view of the parts of the social graph they are currently interested in, so as to provide a positive interactive user experience. The problem is challenging because the social graph mutates at a high rate, users change their focus of interest frequently, and some mutations are of interest to many online users.

We describe Bladerunner, a system we use at Facebook to deliver relevant social graph updates to user devices efficiently and quickly. The heart of Bladerunner is a set of back-end stream processors that obtain streams of social graph updates and process them on a per application and per-user basis before pushing selected updates to user devices. Separate stream processors are used for each application to enable application-specific customization, complex filtering, aggregation and other message delivery operations on a per-user basis. This strategy minimizes device processing overhead and last-mile bandwidth usage, which are critical given that users are mostly on mobile devices.

Keywords:

Event-driven stream processing, Publish-subscribe, Mobile cloud infrastructure, Internet-scale data dissemination, Request-stream protocol, Social networks, Scalability

Reference Info:

DOI: 10.1145/3477132.3483572
ISBN: 9781450387095
OCLC: 9301233429

BibTeX:

@inproceedings(Barber-sosp-21,
    author = {Jeff Barber and Ximing Yu and Laney Kuenzel Zamore and Jerry Lin and Vahid Jazayeri and Shie Erlich and Tony Savor and Michael Stumm},
    title = {Bladerunner: {S}tream processing at scale for a live view of backend data mutations at the edge},
    booktitle = {Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (\textbf{SOSP'21})},
    publisher = {ACM},
    month = {October},
    year = {2021},
    pages = {708-723},
    doi = {10.1145/3477132.3483572},
    isbn = {9781450387095},
    keywords = {Event-driven stream processing, Publish-subscribe, Mobile cloud infrastructure, Internet-scale data dissemination, Request-stream protocol, Social networks, Scalability}
)