Paper Details

Reference:

David Lion, Adrian Chiu, Michael Stumm, and Ding Yuan,
"Investigating managed language runtime performance: Why JavaScript and Python are 8x and 29x slower than C++, yet Java and Go can be faster",
In Proceedings USENIX Annual Technical Conference (USENIX-ATC'22), Carlsbad, CA, USA, Usenix Association, July, 2022, pp. 835–852.

Download:

PDF ; Talk

Abstract:

The most widely used programming languages today are managed languages. They are popular because their vast features improve many aspects of code development, including increased productivity and safety. However, as a product or service scales in usage, performance issues become a problem. Developers are then often faced with complex choices as they must decide whether the desired performance can be squeezed from existing code, or whether their language has reached its performance limits, requiring years of code to be ported to a new more-performant language. To make matters worse, runtime performance is shrouded in mystery as it involves complex interactions of different components, such as interpreter, just-in-time (JIT) compiler, thread library, and Garbage Collection (GC) system.

We present an in-depth performance analysis and comparison of Java, Go, JavaScript, and Python, using C++ as a baseline. We carefully instrumented the different language runtimes, so that developers can precisely measure the number of cycles taken to execute any bytecode instruction, or the overhead of dynamic type checking in JavaScript. This allows us to accurately identify sources of overhead. We further created 6 applications from the ground up to establish the LangBench benchmark; the applications cover a range of complexity, and they cover a variety of application scenarios differing in compute intensity, memory usage, network and disk I/O intensity, and available concurrency. We comprehensively analyze their completion times, resource usage, and scalability.

Overall, we found that V8/Node.js and CPython exhibit excessive overheads, executing applications 8.01x and 29.50x slower on average than their C++ counterparts, respectively. Making matters worse, applications on these two runtimes scale poorly in that they cannot effectively utilize more than one core. In contrast, OpenJDK and Go applications are performance competitive to C++, running only 1.43x and 1.30x slower, respectively, and they can easily scale to multiple cores. There are applications where OpenJDK and Go outperform their C++ counterparts.

Keywords:

Language Runtimes, performance analysis, LangBench, Java, Go, Javascript, Python

Reference Info:

ISBN: 978-1-939133-29-40

BibTeX:

@inproceedings(Lion-UsenixATC-22,
    author = {David Lion and Adrian Chiu and Michael Stumm and Ding Yuan},
    title = {Investigating managed language runtime performance: {W}hy {JavaScript} and {P}ython are 8x and 29x slower than {C}++, yet {J}ava and {G}o can be faster},
    booktitle = {Proceedings USENIX Annual Technical Conference (\textbf{USENIX-ATC'22})},
    location = {Carlsbad, CA, USA},
    publisher = {Usenix Association},
    month = {July},
    year = {2022},
    pages = {835-852},
    isbn = {978-1-939133-29-40},
    keywords = {Language Runtimes, performance analysis, LangBench, Java, Go, Javascript, Python }
)