Accelerating large-scale data distribution in booming internet: Effectiveness, bottlenecks and practices

Abstract

With the booming of Internet and consumer electronics industry, there are rising amount of content (audio, video, software) that needs to be distributed to increasing number of end consumers in higher speed. Whereas the explosive growth of data-concentrated applications over Internet contributes to the entertainment electronics, the large-scale distribution of huge data to end consumers with guarantee of Quality of Service (QoS), remains an elusive goal. The potential integration of Peer-to-Peer (P2P) paradigm into the Internet content distribution infrastructure provides a disruptive market opportunity to scale the Internet for higher quality data delivery. While the theoretical benefits of P2P in regular Internet has been widely reported, it remains unknown on its performance for large-scale mass data delivery over high-speed networks, highly promising yet controversial. This paper thus provides an experimental and analytical performance study over BitTorrent-like P2P networks for accelerating large-scale content distribution over booming Internet. Our study explores the unique strength of P2P in high speed networks, identifies performance bottlenecks, and quantifies the special requirements in the new scenario, i.e. file piece length and seed capacity. Further, to enable services before full downloading, this paper proposes a Piece-On-Demand (POD) scheme to modify BitTorrent in integration with Filesystem in Userspace (FUSE) to both decrease file distribution time and increase service availability. To demonstrate the feasibility of adopting P2P as a reliable consumer network for content distribution, we further illustrate our industrial exploration on utilizing P2P in deployment for application/services with huge image, providing unprecedented deployment speed and scalability for enabling on-demand services to large-scale users. © 2009 IEEE.