Over the last year the V8 team has developed a new methodology to measure and understand real-world JavaScript performance. We’ve used the insights that we gleaned from it to change how the V8 team makes JavaScript faster. Our new real-world focus represents a significant shift from our traditional performance focus. We’re confident that as we continue to apply this methodology in 2017, it will significantly improve users’ and developers’ ability to rely on predictable performance from V8 for real-world JavaScript in both Chrome and Node.js.
The old adage “what gets measured gets improved” is particularly true in the world of JavaScript virtual machine (VM) development. Choosing the right metrics to guide performance optimization is one of the most important things a VM team can do over time. The following timeline roughly illustrates how JavaScript benchmarking has evolved since the initial release of V8:
Evolution of JavaScript benchmarks.
Historically, V8 and other JavaScript engines have measured performance using synthetic benchmarks. Initially, VM developers used microbenchmarks like SunSpider and Kraken. As the browser market matured a second benchmarking era began, during which they used larger but nevertheless synthetic test suites such as Octane and JetStream.
Microbenchmarks and static test suites have a few benefits: they’re easy to bootstrap, simple to understand, and able to run in any browser, making comparative analysis easy. But this convenience comes with a number of downsides. Because they include a limited number of test cases, it is difficult to design benchmarks which accurately reflect the characteristics of the web at large. Moreover, benchmarks are usually updated infrequently; thus, they tend to have a hard time keeping up with new trends and patterns of JavaScript development in the wild. Finally, over the years VM authors explored every nook and cranny of the traditional benchmarks, and in the process they discovered and took advantage of opportunities to improve benchmark scores by shuffling around or even skipping externally unobservable work during benchmark execution. This kind of benchmark-score-driven improvement and over-optimizing for benchmarks doesn’t always provide much user- or developer-facing benefit, and history has shown that over the long-term it’s very difficult to make an “ungameable” synthetic benchmark.
Measuring real websites: WebPageReplay & Runtime Call Stats
Given an intuition that we were only seeing one part of the performance story with traditional static benchmarks, the V8 team set out to measure real-world performance by benchmarking the loading of actual websites. We wanted to measure use cases that reflected how end users actually browsed the web, so we decided to derive performance metrics from websites like Twitter, Facebook, and Google Maps. Using a piece of Chrome infrastructure called WebPageReplay we were able to record and replay page loads deterministically.
In tandem, we developed a tool called Runtime Call Stats which allowed us to profile how different JavaScript code stressed different V8 components. For the first time, we had the ability not only to test V8 changes easily against real websites, but to fully understand how and why V8 performed differently under different workloads.
We now monitor changes against a test suite of approximately 25 websites in order to guide V8 optimization. In addition to the aforementioned websites and others from the Alexa Top 100, we selected sites which were implemented using common frameworks (React, Polymer, Angular, Ember, and more), sites from a variety of different geographic locales, and sites or libraries whose development teams have collaborated with us, such as Wikipedia, Reddit, Twitter, and Webpack. We believe these 25 sites are representative of the web at large and that performance improvements to these sites will be directly reflected in similar speedups for sites being written today by JavaScript developers.
Analyzing these new, real-world performance metrics and comparing them to traditional benchmarks with Runtime Call Stats has also given us more insight into how various workloads stress V8 in different ways.
From these measurements, we discovered that Octane performance was actually a poor proxy for performance on the majority of our 25 tested websites. You can see in the chart below: Octane’s color bar distribution is very different than any other workload, especially those for the real-world websites. When running Octane, V8’s bottleneck is often the execution of JavaScript code. However, most real-world websites instead stress V8’s parser and compiler. We realized that optimizations made for Octane often lacked impact on real-world web pages, and in some cases these optimizations made real-world websites slower.
Distribution of time running all of Octane, running the line-items of Speedometer and loading websites from our test suite on Chrome M57.
We also discovered that another benchmark was actually a better proxy for real websites. Speedometer, a WebKit benchmark that includes applications written in React, Angular, Ember, and other frameworks, demonstrated a very similar runtime profile to the 25 sites. Although no benchmark matches the fidelity of real web pages, we believe Speedometer does a better job of approximating the real-world workloads of modern JavaScript on the web than Octane.
Bottom line: A faster V8 for all
Over the course of the past year, the real-world website test suite and our Runtime Call Stats tool has allowed us to deliver V8 performance optimizations that speed up page loads across the board by an average of 10-20%. Given the historical focus on optimizing page load across Chrome, a double-digit improvement to the metric in 2016 is a significant achievement. The same optimizations also improved our score on Speedometer by 20-30%.
These performance improvements should be reflected in other sites written by web developers using modern frameworks and similar patterns of JavaScript. Our improvements to builtins such as Object.create and Function.prototype.bind, optimizations around the object factory pattern, work on V8’s inline caches, and ongoing parser improvements are intended to be generally applicable improvements to underlooked areas of JavaScript used by all developers, not just the representative sites we track.
We plan to expand our usage of real websites to guide V8 performance work. Stay tuned for more insights about benchmarks and script performance.
The history of JavaScript benchmarks is a story of constant evolution. As the web expanded from simple documents to dynamic client-side applications, new JavaScript benchmarks were created to measure workloads that became important for new use cases. This constant change has given individual benchmarks finite lifespans. As web browser and virtual machine (VM) implementations begin to over-optimize for specific test cases, benchmarks themselves cease to become effective proxies for their original use cases. One of the first JavaScript benchmarks, SunSpider, provided early incentives for shipping fast optimizing compilers. However, as VM engineers uncovered the limitations of microbenchmarks and found new ways to optimizearound SunSpider’s limitations, the browser community retired SunSpider as a recommended benchmark.
Designed to mitigate some of the weaknesses of early microbenchmarks, the Octane benchmark suite was first released in 2012. It evolved from an earlier set of simple V8 test cases and became a common benchmark for general web performance. Octane consists of 17 different tests, which were designed to cover a variety of different workloads, ranging from Martin Richards’ kernel simulation test to a version of Microsoft’s TypeScript compiler compiling itself. The contents of Octane represented the prevailing wisdom around measuring JavaScript performance at the time of its creation.
Diminishing returns and over-optimization
In the first few years after its release, Octane provided a unique value to the JavaScript VM ecosystem. It allowed engines, including V8, to optimize their performance for a class of applications that stressed peak performance. These CPU-intensive workloads were initially underserviced by VM implementations. Octane helped engine developers deliver optimizations that allowed computationally-heavy applications to reach speeds that made JavaScript a viable alternative to C++ or Java. In addition, Octane drove improvements in garbage collection which helped web browsers avoid long or unpredictable pauses.
By 2015, however, most JavaScript implementations had implemented the compiler optimizations needed to achieve high scores on Octane. Striving for even higher benchmark scores on Octane translated into increasingly-marginal improvements in the performance of real web pages. Investigations into the execution profile of running Octane versus loading common websites (such as Facebook, Twitter, or Wikipedia) revealed that the benchmark doesn’t exercise V8’s parser or the browser loading stack the way real-world code does. Moreover, the style of Octane’s JavaScript doesn’t match the idioms and patterns employed by most modern frameworks and libraries (not to mention transpiled code or newer ES2015+ language features). This means that using Octane to measure V8 performance didn’t capture important use cases for the modern web, such as loading frameworks quickly, supporting large applications with new patterns of state management, or ensuring that ES2015+ features are as fast as their ES5 equivalents.
In addition, we began to notice that JavaScript optimizations which eked out higher Octane scores often had a detrimental effect on real-world scenarios. Octane encourages aggressive inlining to minimize the overhead of function calls, but inlining strategies that are tailored to Octane have led to regressions from increased compilation costs and higher memory usage in real-world use cases. Even when an optimization may be genuinely useful in the real-world, as is the case with dynamic pretenuring, chasing higher Octane scores can result in developing overly-specific heuristics which have little effect or even degrade performance in more generic cases. We found that Octane-derived pretenuring heuristics led to performance degradations in modern frameworks such as Ember. The `instanceof` operator was another example of an optimization tailored to a narrow set of Octane-specific cases that led to significant regressions in Node.js applications.
Another problem is that over time, small bugs in Octane become a target for optimizations themselves. For example, in the Box2DWeb benchmark, taking advantage of a bug where two objects were compared using the `<` and `>=` operators gave a ~15% performance boost on Octane. Unfortunately, this optimization had no effect in the real world and complicates more general types of comparison optimizations. Octane sometimes even negatively penalizes real-world optimizations: engineers working on other VMs have noticed that Octane seems to penalize lazy parsing, a technique that helps most real websites load faster given the amount of dead code frequently found in the wild.
Beyond Octane and other synthetic benchmarks
These examples are just some of the many optimizations which increased Octane scores to the detriment of running real websites. Unfortunately, similar issues exist in other static or synthetic benchmarks, including Kraken and JetStream. Simply put, such benchmarks are insufficient methods of measuring real-world speed and create incentives for VM engineers to over-optimize narrow use cases and under-optimize generic cases, slowing down JavaScript code in the wild.
Given the plateau in scores across most JS VMs and the increasing conflict between optimizing for specific Octane benchmarks rather than implementing speedups for a broader range of real-world code, we believe that it is time to retire Octane as a recommended benchmark.
Octane enabled the JS ecosystem to make large gains in computationally-expensive JavaScript. The next frontier, however, is improving the performance of real web pages, modern libraries, frameworks, ES2015+ language features, new patterns of state management, immutable object allocation, and modulebundling. Since V8 runs in many environments, including server side in Node.js, we are also investing time in understanding real-world Node applications and measuring server-side JavaScript performance through workloads such as AcmeAir.
Check back here for more posts about improvements in our measurement methodology and new workloads that better represent real-world performance. We are excited to continue pursuing the performance that matters most to users and developers!
Today we are excited to announce the launch of a new JavaScript execution pipeline for V8 5.9 that will reach Chrome Stable in M59. With the new pipeline, we achieve big performance improvements and significant memory savings on real-world JavaScript applications. We’ll discuss the numbers in more detail at the end of this post, but first let’s take a look at the pipeline itself.
The new pipeline is built upon Ignition, V8’s interpreter, and TurboFan, V8’s newest optimizing compiler. These technologies shouldbefamiliar to those of you who have followed the V8 blog over the last few years, but the switch to the new pipeline marks a big new milestone for both.
For the first time, Ignition and TurboFan are used universally and exclusively for JavaScript execution in V8 5.9. Furthermore, starting with 5.9, Full-codegen and Crankshaft, the technologies that served V8 well since 2010, are no longer used in V8 for JavaScript execution, since they no longer are able to keep pace with new JavaScript language features and the optimizations those features require. We plan to remove them completely very soon. That means that V8 will have an overall much simpler and more maintainable architecture going forward.
A Long Journey
The combined Ignition and TurboFan pipeline has been in development for almost 3½ years. It represents the culmination of the collective insight that the V8 team has gleaned from measuring real-world JavaScript performance and carefully considering the shortcomings of Full-codegen and Crankshaft. It is a foundation with which we will be able to continue to optimize the entirety of the JavaScript language for years to come.
The TurboFan project originally started in late 2013 to address the shortcomings of Crankshaft. Crankshaft can only optimize a subset of the JavaScript language. For example, it was not designed to optimize JavaScript code using structured exception handling, i.e. code blocks demarcated by JavaScript’s try, catch, and finally keywords. It is difficult to add support for new language features in Crankshaft, since these features almost always require writing architecture-specific code for nine supported platforms. Furthermore, Crankshaft’s architecture is limited in the extent that it can generate optimal machine code. It can only squeeze so much performance out of JavaScript, despite requiring the V8 team to maintain more than ten thousand lines of code per chip architecture.
TurboFan was designed from the beginning not only to optimize all of the language features found in the JavaScript standard at the time, ES5, but also all the future features planned for ES2015 and beyond. It introduces a layered compiler design that enables a clean separation between high-level and low-level compiler optimizations, making it easy to add new language features without modifying architecture-specific code. TurboFan adds an explicit instruction selection compilation phase that makes it possible to write far less architecture-specific code for each supported platform in the first place. With this new phase, architecture-specific code is written once and it rarely needs to be changed. These and other decisions lead to a more maintainable and extensible optimizing compiler for all of the architectures that V8 supports.
The original motivation behind V8’s Ignition interpreter was to reduce memory consumption on mobile devices. Before Ignition, the code generated by V8’s Full-codegen baseline compiler typically occupied almost one third of the overall JavaScript heap in Chrome. That left less space for a web application’s actual data. When Ignition was enabled for Chrome M53 on Android devices with limited RAM, the memory footprint required for baseline, non-optimized JavaScript code shrank by a factor of nine on ARM64-based mobile devices.
Later the V8 team took advantage of the fact that Ignition’s bytecode can be used to generate optimized machine code with TurboFan directly rather than having to re-compile from source code as Crankshaft did. Ignition’s bytecode provides a cleaner and less error-prone baseline execution model in V8, simplifying the deoptimization mechanism that is a key feature of V8’s adaptive optimization. Finally, since generating bytecode is faster than generating Full-codegen’s baseline compiled code, activating Ignition generally improves script startup times and in turn, web page loads.
By coupling the design of Ignition and TurboFan closely, there are even more benefits to the overall architecture. For example, rather than writing Ignition’s high-performance bytecode handlers in hand-coded assembly, the V8 team instead uses TurboFan’s intermediate representation to express the handlers’ functionality and lets TurboFan do the optimization and final code generation for V8’s numerous supported platforms. This ensures Ignition performs well on all of V8’s supported chip architectures while simultaneously eliminating the burden of maintaining nine separate platform ports.
Running the Numbers
History aside, now let’s take a look at the new pipeline’s real-world performance and memory consumption.
The V8 team continually monitors the performance of real-world use cases using the Telemetry - Catapult framework. Previously in this blog we’ve discussed why it’s so important to use the data from real-world tests to drive our performance optimization work and how we use WebPageReplay together with Telemetry to do so. The switch to Ignition and TurboFan shows performance improvements in those real-world test cases. Specifically, the new pipeline results in significant speed-ups on user interaction story tests for well-known websites:
Reduction in time spent in V8 for user interaction benchmarks
Although Speedometer is a synthetic benchmark, we’ve previously uncovered that it does a better job of approximating the real-world workloads of modern JavaScript than other synthetic benchmarks. The switch to Ignition and TurboFan improves V8’s Speedometer score by 5%-10%, depending on platform and device.
The new pipeline also speeds up server-side JavaScript. AcmeAir, a benchmark for Node.js that simulates the server backend implementation of a fictitious airline, runs more than 10% faster using V8 5.9.
Improvements on Web and Node.js benchmarks
Ignition and TurboFan also reduce V8’s overall memory footprint. In Chrome M59, the new pipeline slims V8’s memory footprint on desktop and high-end mobile devices by 5-10%. This reduction is a result of bringing the Ignition memory savings that have been previously covered in this blog to all devices and platforms supported by V8.
These improvements are just the start. The new Ignition and TurboFan pipeline paves the way for further optimizations that will boost JavaScript performance and shrink V8’s footprint in both Chrome and in Node.js for years to come. We look forward to sharing those improvements with you as we roll them out to developers and users. Stay tuned.
Ever since its initial release of Speedometer 1.0 in 2014, the Blink and V8 teams have been using the benchmark as a proxy for real-world use of popular JavaScript frameworks and we achieved considerable speedups on this benchmark. We verified independently that these improvements translate to real user benefits by measuring against real-world websites and observed that improvements of page load times of popular websites also improved the Speedometer score.
JavaScript has rapidly evolved in the meantime, adding many new language features with ES2015 and later standards. The same is true for the frameworks themselves, and as such Speedometer 1.0 has become outdated over time. Hence using Speedometer 1.0 as an optimization indicator raises the risk of not measuring newer code patterns that are actively used.
The Blink and V8 teams have already completed a first round of improvements, underlying the importance of this benchmark to us and continuing our journey of focusing on real-world performance. Comparing Chrome 60 from July 2017 with the latest Chrome 64 we have achieved about a 21% improvement on the total score (runs per minute) on a mid-2016 Macbook Pro (4 core, 16GB RAM).
Let’s zoom into the individual Speedometer 2.0 line items. We doubled the performance of the React runtime by improving Function.prototype.bind. Vanilla-ES2015, AngularJS, Preact, and VueJS improved by 19%–42% due to speeding up the JSON parsing and various other performance fixes. The jQuery-TodoMVC app's runtime was reduced by improvements to Blink's DOM implementation, including more lightweight form controls and tweaks to our HTML parser. Additional tweaking of V8's inline caches in combination with the optimizing compiler yielded improvements across the board.
A significant change over Speedometer 1.0 is the calculation of the final score. Previously the average of all scores favoured working only on the slowest line items. When looking at the absolute times spent in each line item we see for instance that the EmberJS-Debug version takes roughly 35 times as long as the fastest benchmark. Hence to improve the overall score focusing on EmberJS-Debug has the highest potential.
Speedometer 2.0 uses the geometric mean for the final score, favouring equal investments into each framework. Let us consider our recent 16.5% improvement of Preact from above. It would be rather unfair to forgo the 16.5% improvement just because of its minor contribution to the total time.
We are looking forward to bring further performance improvements to Speedometer 2.0 and through that to the whole web. Stay tuned for more performance high-fives.
This month marks the 10-year anniversary of shipping not just Google Chrome, but also the V8 project. This post gives an overview of major milestones for the V8 project in the past 10 years as well as the years before, when the project was still secret.
A visualization of the V8 code base over time, created using gource.
Before V8 shipped: the early years
Google hired Lars Bak in the autumn of 2006 to build a new JavaScript engine for the Chrome web browser, which at the time was still a secret internal Google project. Lars had recently moved back to Aarhus, Denmark, from Silicon Valley. Since there was no Google office there and Lars wanted to remain in Denmark, Lars and several of the project’s original engineers began working on the project in an outbuilding on his farm. The new JavaScript runtime was christened “V8”, a playful reference to the powerful engine you can find in a classic muscle car. Later, when the V8 team had grown, the developers moved from their modest quarters to a modern office building in Aarhus, but the team took with them their singular drive and focus on building the fastest JavaScript runtime on the planet.
Launching and evolving V8
V8 went open-source the same day Chrome was launched: on September 2nd, 2008. The initial commit dates back to June 30th, 2008. Prior to that date, V8 development happened in a private CVS repository. Initially, V8 supported only the ia32 and ARM instruction sets and used SCons as its build system.
2009 saw the introduction of a brand new regular expression engine named Irregexp, resulting in performance improvements for real-world regular expressions. With the introduction of an x64 port, the number of supported instruction sets increased from two to three. 2009 also marked the first release of the Node.js project, which embeds V8. The possibility for non-browser projects to embed V8 was explicitly mentioned in the original Chrome comic. With Node.js, it actually happened! Node.js grew to be one of the most popular JavaScript ecosystems.
2010 witnessed a big boost in runtime performance as V8 introduced a brand-new optimizing JIT compiler. Crankshaft generated machine code that was twice as fast and 30% smaller than the previous (unnamed) V8 compiler. That same year, V8 added its fourth instruction set: 32-bit MIPS.
2011 came, and garbage collection was vastly improved. A new incremental garbage collector drastically reduced pause times while maintaining great peak performance and low memory usage. V8 introduced the concept of Isolates, which allows embedders to spin up multiple instances of the V8 runtime in a process, paving the way for lighter-weight Web Workers in Chrome. The first of V8’s two build system migrations occurred as we transitioned from SCons to GYP. We implemented support for ES5 strict mode. Meanwhile, development moved from Aarhus to Munich (Germany) under new leadership with lots of cross-pollination from the original team in Aarhus.
2012 was a year of benchmarks for the V8 project. The team did speed sprints to optimize V8’s performance as measured through the Sunspider and Kraken benchmark suites. Later, we developed a new benchmark suite named Octane (with V8 Bench at its core) that brought peak performance competition to the forefront and spurred massive improvements in runtime and JIT technology in all major JS engines. One outcome of these efforts was the switch from randomized sampling to a deterministic, count-based technique for detecting “hot” functions in V8’s runtime profiler. This made it significantly less likely that some page loads (or benchmark runs) would randomly be much slower than others.
2013 witnessed the appearance of a low-level subset of JavaScript named asm.js. Since asm.js is limited to statically-typed arithmetic, function calls, and heap accesses with primitive types only, validated asm.js code could run with predictable performance. We released a new version of Octane, Octane 2.0 with updates to existing benchmarks as well as new benchmarks that target use cases like asm.js. Octane spurred the development of new compiler optimizations like allocation folding and allocation-site-based optimizations for type transitioning and pretenuring that vastly improved peak performance. As part of an effort we internally nicknamed “Handlepocalypse”, the V8 Handle API was completely rewritten to make it easier to use correctly and safely. Also in 2013, Chrome’s implementation of TypedArrays in JavaScript was moved from Blink to V8.
In 2014, V8 moved some of the work of JIT compilation off the main thread with concurrent compilation, reducing jank and significantly improving performance. Later that year, we landed the initial version of a new optimizing compiler named TurboFan. Meanwhile, our partners helped port V8 to three new instruction set architectures: PPC, MIPS64, and ARM64. Following Chromium, V8 transitioned to yet another build system, GN. The V8 testing infrastructure saw significant improvements, with a Tryserver now available to test each patch on various build bots before landing. For source control, V8 migrated from SVN to Git.
2015 was a busy year for V8 on a number of fronts. We implemented code caching and script streaming, significantly speeding up web page load times. Work on our runtime system’s use of allocation mementos was published in ISMM 2015. Later that year, we kicked off the work on a new interpreter named Ignition. We experimented with the idea of subsetting JavaScript with strong mode to achieve stronger guarantees and more predictable performance. We implemented strong mode behind a flag, but later found its benefits did not justify the costs. The addition of a commit queue made big improvements in productivity and stability. V8’s garbage collector also began cooperating with embedders such as Blink to schedule garbage collection work during idle periods. Idle-time garbage collection significantly reduced observable garbage collection jank and memory consumption. In December, the first WebAssembly prototype landed in V8.
In 2016, we shipped the last pieces of the ES2015 (previously known as “ES6”) feature set (including promises, class syntax, lexical scoping, destructuring, and more), as well as some ES2016 features. We also started rolling out the new Ignition and TurboFan pipeline, using it to compile and optimize ES2015 and ES2016 features, and shipping Ignition by default for low-end Android devices. Our successful work on idle-time garbage collection was presented at PLDI 2016. We kicked off the Orinoco project, a new mostly-parallel and concurrent garbage collector for V8 to reduce main thread garbage collection time. In a major refocus, we shifted our performance efforts away from synthetic micro-benchmarks and instead began to seriously measure and optimize real-world performance. For debugging, the V8 inspector was migrated from Chromium to V8, allowing any V8 embedder (and not just Chromium) to use the Chrome DevTools to debug JavaScript running in V8. The WebAssembly prototype graduated from prototype to experimental support, in coordination with other browser vendors experimental support for WebAssembly. V8 received the ACM SIGPLAN Programming Languages Software Award. And another port was added: S390.
In 2017, we finally completed our multi-year overhaul of the engine, enabling the new Ignition and TurboFan pipeline by default. This made it possible to later remove Crankshaft (130,380 deleted lines of code) and Full-codegen from the codebase. We launched Orinoco v1.0, including concurrent marking, concurrent sweeping, parallel scavenging, and parallel compaction. We officially recognized Node.js as a first-class V8 embedder alongside Chromium. Since then, it’s impossible for a V8 patch to land if doing so breaks the Node.js test suite. Our infrastructure gained support for correctness fuzzing, ensuring that any piece of code produces consistent results regardless of the configuration it runs in.
In an industry-wide coordinated launch, V8 shipped WebAssembly on-by-default. We implemented support for JavaScript modules as well as the full ES2017 and ES2018 feature sets (including async functions, shared memory, async iteration, rest/spread properties, and RegExp features). We shipped native support for JavaScript code coverage, and launched the Web Tooling Benchmark to help us measure how V8’s optimizations impact performance for real-world developer tools and the JavaScript output they generate. Wrapper tracing from JavaScript objects to C++ DOM objects and back allowed us to resolve long-standing memory leaks in Chrome and to handle the transitive closure of objects over the JavaScript and Blink heap efficiently. We later used this infrastructure to increase the capabilities of the heap snapshotting developer tool.
2018 saw an industry-wide security event upend what we thought we knew about CPU information security with the public disclosure of the Spectre/Meltdown vulnerabilities. V8 engineers performed extensive offensive research to help understand the threat for managed languages and develop mitigations. V8 shipped mitigations against Spectre and similar side-channel attacks for embedders that run untrusted code.
Chrome’s V8 Bench score over the years shows the performance impact of V8’s changes. (We’re using the V8 Bench because it’s one of the few benchmarks that can still run in the original Chrome beta.)
Our score on this benchmark went up 4× over the last ten years!
However, you might notice two performance dips over the years. Both are interesting because they correspond to significant events in V8’s history. The performance drop in 2015 happened when V8 shipped baseline versions of ES2015 features. These features were cross-cutting in the V8 code base, and we therefore focused on correctness rather than performance for their initial release. We accepted these slight speed regressions to get features to developers as quickly as possible. In early 2018, the Spectre vulnerability was disclosed, and V8 shipped mitigations to protect users against potential exploits, resulting in another regression in performance. Luckily, now that Chrome is shipping Site Isolation, we can disable the mitigations again, bringing performance back on par.
Another take-away from this chart is that it starts to level off around 2013. Does that mean V8 gave up and stopped investing in performance? Quite the opposite! The flattening of the graphs represents the V8 team’s pivot from synthetic micro-benchmarks (such as V8 Bench and Octane) to optimizing for real-world performance. V8 Bench is an old benchmark that doesn’t use any modern JavaScript features, nor does it approximate actual real-world production code. Contrast this with the more recent Speedometer benchmark suite:
Although V8 Bench shows minimal improvements from 2013 to 2018, our Speedometer 1 score went up (another) 4× during this same time period. (We used Speedometer 1 because Speedometer 2 uses modern JavaScript features that weren’t yet supported in 2013.)
Although V8 was originally built for Google Chrome, it has always been a stand-alone project with a separate codebase and an embedding API that allows any program to use its JavaScript execution services. Over the last 10 years, the open nature of the project has helped it become a key technology not only for the Web Platform, but in other contexts, like Node.js. Along the way the project evolved and remain relevant despite many changes and dramatic growth.
Initially, V8 supported only two instruction sets. In the last 10 years the list of supported platforms reached eight: ia32, x64, ARM, ARM64, 32- and 64-bit MIPS, 64-bit PPC, and S390. V8’s build system migrated from SCons to GYP to GN. The project moved from Denmark to Germany, and now has engineers all over the world, including in London, Mountain View, and San Francisco, with contributors outside of Google from many more places. We’ve transformed our entire JavaScript compilation pipeline from unnamed components to Full-codegen (a baseline compiler) and Crankshaft (an feedback-driven optimizing compiler) to Ignition (an interpreter) and TurboFan (a better feedback-driven optimizing compiler). V8 went from being “just” a JavaScript engine to also supporting WebAssembly. The JavaScript language itself evolved from ECMAScript 3 to ES2018; the latest V8 even implements post-ES2018 features.
The story arc of Web is a long and enduring one. Celebrating Chrome and V8’s 10th birthday is a good opportunity to reflect that even though this is a big milestone, the Web Platform’s narrative has lasted for more than 25 years. We have no doubt the Web’s story will continue for at least that long in the future. We’re committed to making sure that V8, JavaScript, and WebAssembly continue to be interesting characters in that narrative. We’re excited to see what the next decade has in store. Stay tuned!
Every six weeks, we create a new branch of V8 as part of our release process. Each version is branched from V8’s git master immediately before a Chrome Beta milestone. Today we’re pleased to announce our newest branch, V8 version 5.6, which will be in beta until it is released in coordination with Chrome 56 Stable in several weeks. V8 5.6 is filled with all sorts of developer-facing goodies, so we’d like to give you a preview of some of the highlights in anticipation of the release.
Ignition and TurboFan pipeline for ES.next (and more) shipped
Starting with 5.6, V8 can optimize the entirety of the JavaScript language. Moreover, many language features are sent through a new optimization pipeline in V8. This pipeline uses V8’s Ignition interpreter as a baseline and optimizes frequently executed methods with V8’s more powerful TurboFan optimizing compiler. The new pipeline activates for new language features (e.g. many of the new features from the ES2015 and ES2016 specifications) or whenever Crankshaft (V8’s “classic” optimizing compiler) cannot optimize a method (e.g. try-catch, with).
Why are we only routing some JavaScript language features through the new pipeline?
The new pipeline is better-suited to optimizing the whole spectrum of the JS language (past and present). It's a healthier, more modern codebase, and it has been designed specifically for real-world use cases including running V8 on low-memory devices.
We've started using the Ignition/TurboFan with the newest ES.next features we've added to V8 (ES.next = JavaScript features as specified in ES2015 and later) and will route more features through it as we continue improving its performance. In the middle term, the V8 team is aiming to switch all JavaScript execution in V8 to the new pipeline. However, as long as there are still real-world use cases where Crankshaft runs JavaScript faster than the new Ignition/TurboFan pipeline, for the short term we'll support both pipelines to ensure that JavaScript code running in V8 is as fast as possible in all situations.
So, why does the new pipeline use both the new Ignition interpreter and the new Turbofan optimizing compiler?
Running JavaScript fast and efficiently requires having multiple mechanisms, or tiers, under the hood in a JavaScript virtual machine to do the low-level busywork of execution. For example, it’s useful to have a first tier that starts executing code quickly, and then a second optimizing tier that spends longer compiling hot functions in order to maximize performance for longer-running code.
Ignition and TurboFan are V8’s two new execution tiers that are most effective when used together. Due to efficiency, simplicity and size considerations, TurboFan is designed to optimize JavaScript methods starting from the bytecode produced by V8's Ignition interpreter. By designing both components to work closely together, there are optimizations that can be made to both because of the presence of the other. As a result, starting with 5.6 all functions which will be optimized by TurboFan first run through the Ignition interpreter. Using this unified Ignition/TurboFan pipeline enables the optimization of features that were not optimizable in the past, since they now can take advantage of TurboFan's optimizations passes. For example, by routing Generators through both Ignition and TurboFan, Generators runtime performance has nearly tripled.
For more information on V8's journey to adopt Ignition and TurboFan please have a look at Benedikt's dedicated blog post.
Performance improvements
V8 5.6 delivers a number of key improvements in memory and performance footprint.
Developers typically start using new language features with the help of transpilers because of two challenges: backwards-compatibility and performance concerns.
V8's goal is to reduce the performance gap between transpilers and V8’s “native” ES.next performance in order to eliminate the latter challenge. We’ve made great progress in bringing the performance of new language features on-par with their transpiled ES5 equivalents. In this release you will find the the performance of ES2015 features is significantly faster than in previous V8 releases, and in some cases ES2015 feature performance is approaching that of transpiled ES5 equivalents.
Particularly the spread operator should now be ready to be used natively. Instead of writing ...
// Like Math.max, but returns 0 instead of -∞ for no arguments.
function specialMax(...args) {
if (args.length === 0) return 0;
return Math.max.apply(Math, args);
}
… you should now be able to write ...
function specialMax(...args) {
if (args.length === 0) return 0;
return Math.max(...args);
}
… and get similar performance results. In particular 5.6 includes speed-ups for the following micro-benchmarks:
Chromium 56 (which includes 5.6) is going to ship the WebAssembly browser preview. Please refer to the dedicated blog post for further information.
V8 API
Please check out our summary of API changes. This document is regularly updated a few weeks after each major release.
Developers with an active V8 checkout can use 'git checkout -b 5.6 -t branch-heads/5.6' to experiment with the new features in V8 5.6. Alternatively you can subscribe to Chrome's Beta channel and try the new features out yourself soon.
BlinkOn is a biannual meeting of Blink, V8, and Chromium contributors. BlinkOn 6 was held in Munich on June 16 and June 17. The V8 team gave a number of presentations on architecture, design, performance initiatives, and language implementation.
Outlines the history of how V8 measures JavaScript performance, the different eras of benchmarking, and a new technique to measure page loads across real-world, popular websites with detailed breakdowns of time per V8 component.
Introduces V8’s new Ignition Interpreter, explaining the architecture of the engine as a whole, and how Ignition affects memory usage and startup performance.
Explains how V8 uses the Response, Animation, Idle, Loading (RAIL) metrics to target low-latency garbage collection and the recent optimizations we’ve made to reduce jank on mobile.
Provides an update on the implementation of new language features in V8, how those features integrate with the web platform, and the standards process which continues to evolve the ECMAScript language.
Tracing Wrappers from V8 to Blink (Lightning Talk)
JavaScript performance has always been important to the V8 team, and in this post we would like to discuss a new JavaScript Web Tooling Benchmark that we have been using recently to identify and fix some performance bottlenecks in V8. You may already be aware of V8’s strong commitment to Node.js and this benchmark extends that commitment by specifically running performance tests based on common developer tools built upon Node.js. The tools in the Web Tooling Benchmark are the same ones used by developers and designers today to build modern web sites and cloud-based applications. In continuation of our ongoing efforts to focus on real-world performance rather than artificial benchmarks, we created the benchmark using actual code that developers run every day.
The Web Tooling Benchmark suite was designed from the beginning to cover important developer tooling use cases for Node.js. Because the V8 team focuses on core JavaScript performance, we built the benchmark in a way that focuses on the JavaScript workloads and excludes measurement of Node.js-specific I/O or external interactions. This makes it possible to run the benchmark in Node.js, in all browsers, and in all major JavaScript engine shells, including ch (ChakraCore), d8 (V8), jsc (JavaScriptCore) and jsshell (SpiderMonkey). Even though the benchmark is not limited to Node.js, we are excited that the Node.js benchmarking working group is considering using the tooling benchmark as a standard for Node performance as well (nodejs/benchmarking#138).
The individual tests in the tooling benchmark cover a variety of tools that developers commonly use to build JavaScript-based applications, for example:
Based on past experience with other benchmarks like Speedometer, where tests quickly become outdated as new versions of frameworks become available, we made sure it is straight-forward to update each of the tools in the benchmarks to more recent versions as they are released. By basing the benchmark suite on npm infrastructure, we can easily update it to ensure that it is always testing the state of the art in JavaScript development tools. Updating a test case is just a matter of bumping the version in the package.json manifest.
We created a tracking bug and a spreadsheet to contain all the relevant information that we have collected about V8’s performance on the new benchmark up to this point. Our investigations have already yielded some interesting results. For example, we discovered that V8 was often hitting the slow path for instanceof (v8:6971), incurring a 3–4× slowdown. We also found and fixed performance bottlenecks in certain cases of property assignments of the form of obj[name] = val where obj was created via Object.create(null). In these cases, V8 would fall off the fast-path despite being able to utilize the fact that obj has a null prototype (v8:6985). These and other discoveries made with the help of this benchmark improve V8, not only in Node.js, but also in Chrome.
We not only looked into making V8 faster, but also fixed and upstreamed performance bugs in the benchmark’s tools and libraries whenever we found them. For example, we discovered a number of performance bugs in Babel where code patterns like
value = items[items.length - 1];
lead to accesses of the property "-1", because the code didn’t check whether items is empty beforehand. This code pattern causes V8 to go through a slow-path due to the "-1" lookup, even though a slightly modified, equivalent version of the JavaScript is much faster. We helped to fix these issues in Babel (babel/babel#6582, babel/babel#6581 and babel/babel#6580). We also discovered and fixed a bug where Babel would access beyond the length of a string (babel/babel#6589), which triggered another slow-path in V8. Additionally we optimized out-of-bounds reads of arrays and strings in V8. We’re looking forward to continue working with the community on improving the performance of this important use case, not only when run on top of V8, but also when run on other JavaScript engines like ChakraCore.
Our strong focus on real-world performance and especially on improving popular Node.js workloads is shown by the constant improvements in V8’s score on the benchmark over the last couple of releases:
Over the last several years, the V8 team has come to recognize that no one JavaScript benchmark — even a well-intentioned, carefully crafted one — should be used as a single proxy for a JavaScript engine’s overall performance. However, we do believe that the new Web Tooling Benchmark highlights areas of JavaScript performance that are worth focusing on. Despite the name and the initial motivation, we have found that the Web Tooling Benchmark suite is not only representative of tooling workloads, but is representative of a large range of more sophisticated JavaScript applications that are not tested well by front end-focused benchmarks like Speedometer. It is by no means a replacement for Speedometer, but rather a complementary set of tests.
The best news of all is that given how the Web Tooling Benchmark is constructed around real workloads, we expect that our recent improvements in benchmark scores will translate directly into improved developer productivity through less time waiting for things to build. Many of these improvements are already available in Node.js: at the time of writing, Node 8 LTS is at V8 6.1 and Node 9 is at V8 6.2.
V8 uses code caching to cache the generated code for frequently-used scripts. Starting with Chrome 66, we are caching more code by generating the cache after top-level execution. This leads to a 20-40% reduction in parse and compilation time during the initial load.
Background
V8 uses two kinds of code caching to cache generated code to be reused later. The first is the in-memory cache that is available within each instance of V8. The code generated after the initial compile is stored into this cache, keyed on the source string. This is available for reuse within the same instance of V8. The other kind of code caching serializes the generated code and stores it on disk for future use. This cache is not specific to a particular instance of V8 and can be used across different instances of V8. This blog post focuses on this second kind of code caching as used in Chrome. (Other embedders also use this kind of code caching; it’s not limited to Chrome. However, this blog post only focuses on the usage in Chrome.)
Chrome stores the serialized generated code onto the disk cache and keys it with the URL of the script resource. When loading a script, Chrome checks the disk cache. If the script is already cached, Chrome passes the serialized data to V8 as a part of compile request. V8 then deserializes this data instead of parsing and compiling the script. There are also additional checks involved to ensure that the code is still usable (for example: a version mismatch makes the cached data unusable).
Real-world data shows that the code cache hit rates (for scripts that could be cached) is high (~86%). Though the cache hit rates are high for these scripts, the amount of code we cache per script is not very high. Our analysis showed that increasing the amount of code that is cached would reduce the time spent in parsing and compiling JavaScript code by around 40%.
Increasing the amount of code that is cached
In the previous approach, code caching was coupled with the requests to compile the script.
Embedders could request that V8 serialize the code it generated during its top-level compilation of a new JavaScript source file. V8 returned the serialized code after compiling the script. When Chrome requests the same script again, V8 fetches the serialized code from the cache and deserializes it. V8 completely avoids recompiling functions that are already in the cache. These scenarios are shown in the following figure:
V8 only compiles the functions that are expected to be immediately executed (IIFEs) during the top-level compile and marks other functions for lazy compilation. This helps improve page load times by avoiding compiling functions that are not required, however it means that the serialized data only contains the code for the functions that are eagerly compiled.
Prior to Chrome 59, we had to generate the code cache before any execution has started. The earlier baseline compiler of V8 (Full-codegen) generates specialized code for the execution context. Full-codegen used code patching to fast-path operations for the specific execution context. Such code cannot be serialized easily by removing the context specific data to be used in other execution contexts.
With the launch of Ignition in Chrome 59, this restriction is no longer necessary. Ignition uses data-driven inline caches to fast-path operations in the current execution context. The context-dependent data is stored in feedback vectors and is separate from the generated code. This has opened the possibility of generating code caches even after the execution of the script. As we execute the script, more functions (that were marked for lazy compile) are compiled, allowing us to cache more code.
V8 exposes a new API, ScriptCompiler::CreateCodeCache, to request code caches independent of the compile requests. Requesting code caches along with compile requests is deprecated and would not work in V8 v6.6 onwards. Since version 66, Chrome uses this API to request the code cache after the top-level execute. The following figure shows the new scenario of requesting the code cache. The code cache is requested after the top level execute and hence contains the code for functions that were compiled later during the execution of the script. In the later runs (shown as hot runs in the following figure), it avoids compilation of functions during top level execute.
Results
The performance of this feature is measured using our internal real-world benchmarks. The following graph shows the reduction in the parse and compile time over the earlier caching scheme. There is a reduction of around 20–40% in both parse and compilation time on most of the pages.
Data from the wild shows similar results with a 20–40% reduction in the time spent in compiling JavaScript code both on desktop and mobile. On Android, this optimization also translates to a 1–2% reduction in the top-level page-load metrics like the time a webpage takes to become interactive. We also monitored the memory and disk usage of Chrome and did not see any noticeable regressions.
Every six weeks, we create a new branch of V8 as part of our release process. Each version is branched from V8’s Git master immediately before a Chrome Beta milestone. Today we’re pleased to announce our newest branch, V8 version 6.6, which is in beta until its release in coordination with Chrome 66 Stable in several weeks. V8 v6.6 is filled with all sorts of developer-facing goodies. This post provides a preview of some of the highlights in anticipation of the release.
JavaScript language features
Function.prototype.toString() now returns exact slices of source code text, including whitespace and comments. Here’s an example comparing the old and the new behavior:
// Note the comment between the `function` keyword// and the function name, as well as the space following// the function name.function /* acomment */ foo () {}
// Previously:
foo.toString();
// → 'function foo() {}'// ^ no comment// ^ no space// Now:
foo.toString();
// → 'function /* comment */ foo () {}'
Line separator (U+2028) and paragraph separator (U+2029) symbols are now allowed in string literals, matching JSON. Previously, these symbols were treated as line terminators within string literals, and so using them resulted in a SyntaxError exception.
The catch clause of try statements can now be used without a parameter. This is useful if you don’t have a need for the exception object in the code that handles the exception.
In addition to String.prototype.trim(), V8 now implements String.prototype.trimStart() and String.prototype.trimEnd(). This functionality was previously available through the non-standard trimLeft() and trimRight() methods, which remain as aliases of the new methods for backward compatibility.
The Array.prototype.values() method gives arrays the same iteration interface as the ES2015 Map and Set collections: all can now be iterated over by keys, values, or entries by calling the same-named method. This change has the potential to be incompatible with existing JavaScript code. If you discover odd or broken behavior on a website, please try to disable this feature via chrome://flags/#enable-array-prototype-values and file an issue.
Code caching after execution
The terms cold and warm load might be well-known for people concerned about loading performance. In V8, there is also the concept of a hot load. Let’s explain the different levels with Chrome embedding V8 as an example:
Cold load: Chrome sees the visited web page for the first time and does not have any data cached at all.
Warm load: Chrome remembers that the web page was already visited and can retrieve certain assets (e.g. images and script source files) from the cache. V8 recognizes that the page shipped the same script file already, and therefore caches the compiled code along with the script file in the disk cache.
Hot load: The third time Chrome visits the web page, when serving script file from the disk cache, it also provides V8 with the code cached during the previous load. V8 can use this cached code to avoid having to parse and compile the script from scratch.
Before V8 v6.6 we cached the generated code immediately after the top-level compile. V8 only compiles the functions that are known to be immediately executed during the top-level compile and marks other functions for lazy compilation. This meant that cached code only included top-level code, while all other functions had to be lazily compiled from scratch on each page load. Beginning with version 6.6, V8 caches the code generated after the script’s top-level execution. As we execute the script, more functions are lazily compiled and can be included in the cache. As a result, these functions don’t need to be compiled on future page loads, reducing compile and parse time in hot load scenarios by between 20–60%. The visible user change is a less congested main thread, thus a smoother and faster loading experience.
Look out for a detailed blog post on this topic soon.
Background compilation
For some time V8 has been able to parse JavaScript code on a background thread. With V8’s new Ignition bytecode interpreter that shipped last year, we were able to extend this support to also enable compilation of the JavaScript source to bytecode on a background thread. This enables embedders to perform more work off the main thread, freeing it up to execute more JavaScript and reduce jank. We enabled this feature in Chrome 66, where we see between 5% to 20% reduction on main-thread compilation time on typical websites. For more details, please see the recent blog post on this feature.
Removal of AST numbering
We have continued to reap benefits from simplifying our compilation pipeline after the Ignition and TurboFan launch last year. Our previous pipeline required a post-parsing stage called "AST Numbering", where nodes in the generated abstract syntax tree were numbered so that the various compilers using it would have a common point of reference.
Over time this post-processing pass had ballooned to include other functionality: numbering suspend point for generators and async functions, collecting inner functions for eager compilation, initializing literals or detecting unoptimizable code patterns.
With the new pipeline, the Ignition bytecode became the common point of reference, and the numbering itself was no longer required — but, the remaining functionality was still needed, and the AST numbering pass remained.
In V8 v6.6, we finally managed to move out or deprecate this remaining functionality into other passes, allowing us to remove this tree walk. This resulted in a 3-5% improvement in real-world compile time.
Asynchronous performance improvements
We managed to squeeze out some nice performance improvements for promises and async functions, and especially managed to close the gap between async functions and desugared promise chains.
In addition, the performance of async generators and async iteration was improved significantly, making them a viable option for the upcoming Node 10 LTS, which is scheduled to include V8 v6.6. As an example, consider the following Fibonacci sequence implementation:
asyncfunction* fibonacciSequence() {
for (let a = 0, b = 1;;) {
yield a;
const c = a + b;
a = b;
b = c;
}
}
asyncfunctionfibonacci(id, n) {
forawait (const value of fibonacciSequence()) {
if (n-- === 0) return value;
}
}
We’ve measured the following improvements for this pattern, before and after Babel transpilation:
Finally, bytecode improvements to “suspendable functions” such as generators, async functions, and modules, have improved the performance of these functions while running in the interpreter, and decreased their compiled size. We’re planning on improving the performance of async functions and async generators even further with upcoming releases, so stay tuned.
This is the first V8 version that officially ships without GYP files. If your product needs the deleted GYP files, you need to copy them into your own source repository.
Memory profiling
Chrome’s DevTools can now trace and snapshot C++ DOM objects and display all reachable DOM objects from JavaScript with their references. This feature is one of the benefits of the new C++ tracing mechanism of the V8 garbage collector. For more information please have a look at the dedicated blog post.
V8 API
Please use git log branch-heads/6.5..branch-heads/6.6 include/v8.h to get a list of the API changes.
Developers with an active V8 checkout can use git checkout -b 6.6 -t branch-heads/6.6 to experiment with the new features in V8 v6.6. Alternatively you can subscribe to Chrome’s Beta channel and try the new features out yourself soon.
