Scalability limitations when running a Java web server on a chip multiprocessor

Abstract

In this paper, we consider scalability problems for a Java-based Web server running on a CMP (chip multiprocessor). We found that the performance of the Web server scaled poorly as the number of threads increased, even though the CPUs were not fully utilized. Searching for the causes of this scalability problem, we found that the normal use of the Web server was a poor fit to the object lifetimes a JVM expects. A relatively small fraction of objects that were long-lived but not permanent resulted in frequent scans of all of the live objects. They increased not only the load on the CMP but also the number of Web responses that failed to meet the response time criteria for the server. In addition, we found that the performance of the GC scaled poorly as the number of parallel GC threads increased in the CMP. The long-lived objects tax the CMP, and this is emerging as a general problem for CMPs running Java-based Web servers. To address the problem, we used object pooling for these classes to recycle the instances rather than cycling the memory through the GC. We achieved a 48% improvement in the scalability with this object pooling. We also improved the throughput by 9% by using object pooling for large short-lived objects, even though object pooling is not recommended for use in modern JVMs. We show which data classes affect the GC overheads based on the detailed analysis of object lifetimes. We also show that thread affinity is important for the object pools of these classes. Copyright 2010 ACM.