An in-memory object caching framework with adaptive load Balancing

Abstract

The extreme latency and throughput requirements of modern web applications are driving the use of distributed inmemory object caches such as Memcached. While extant caching systems scale-out seamlessly, their use in the cloud - with its unique cost and multi-tenancy dynamics - presents unique opportunities and design challenges. In this paper, we propose MBal, a high-performance inmemory object caching framework with adaptive Multiphase load Balancing, which supports not only horizontal (scale-out) but vertical (scale-up) scalability as well. MBal is able to make efficient use of available resources in the cloud through its fine-grained, partitioned, lockless design. This design also lends itself naturally to provide adaptive load balancing both within a server and across the cache cluster through an event-driven, multi-phased load balancer. While individual load balancing approaches are being leveraged in in-memory caches, MBal goes beyond the extant systems and offers a holistic solution wherein the load balancing model tracks hotspots and applies different strategies based on imbalance severity - key replication, server-local or cross-server coordinated data migration. Performance evaluation on an 8-core commodity server shows that compared to a state-of-the-art approach, MBal scales with number of cores and executes 2.3× and 12× more queries/second for GET and SET operations, respectively.