QORG: Cloud-based queuing system with strong consistency

Abstract

Message queuing enables asynchronous communications between loosely coupled components in large-scale distributed systems. Traditional enterprise queueing systems provide strong queuing consistency, but incur high cost and lack elasticity. Cloud-based queueing services are highly elastic and scalable at the cost of reduced consistency. We introduce QORG, a Queue reORGanization messaging system which brings the best of enterprise and cloud-based queuing techniques together. By maintaining a message caching layer and performing just-in-time prefetching of messages from the persistent queue layer, QORG enables fast retrieval of queued messages in FIFO order, without incurring additional overhead to enqueue and dequeue operations. © 2011 ACM.