A scalable and elastic publish/subscribe service

Abstract

The rapid growth of sense-and-respond applications and the emerging cloud computing model present a new challenge: providing publish/subscribe as a scalable and elastic cloud service. This paper presents the Blue Dove attribute based publish/subscribe service that seeks to address such a challenge. Blue Dove uses a gossip-based one-hop overlay to organize servers into a scalable cluster. It proactively exploits skew ness in data distribution to achieve high performance. By assigning each subscription to multiple servers through a multidimensional subscription space partitioning technique, it provides multiple candidate servers for each publication message. A message can be matched on any of its candidate servers with one hop forwarding. The performance-aware forwarding in Blue Dove ensures that the message is sent to the least loaded candidate server for processing, leading to low latency and high throughput. The evaluation shows that Blue Dove has a linear capacity increase as the system scales up, adapts to sudden workload changes within tens of seconds, and achieves multifold higher throughput than the techniques used in the existing enterprise and peer-to-peer pub/sub systems. © 2011 IEEE.