Monday, July 27, 2015

Code caching

V8 uses just-in-time compilation (JIT) to execute Javascript code. This means that immediately prior to running a script, it has to be parsed and compiled - which can cause considerable overhead. As we announced recently, code caching is a technique that lessens this overhead. When a script is compiled for the first time, cache data is produced and stored. The next time V8 needs to compile the same script, even in a different V8 instance, it can use the cache data to recreate the compilation result instead of compiling from scratch. As a result the script is executed much sooner.

Code caching has been available since V8 version 4.2 and not limited to Chrome alone. It is exposed through V8’s API, so that every V8 embedder can take advantage of it. The test case used to exercise this feature serves as an example of how to use this API.

When a script is compiled by V8, cache data can be produced to speed up later compilations by passing v8::ScriptCompiler::kProduceCodeCache as an option. If the compilation succeeds, the cache data is attached to the source object and can be retrieved via v8::ScriptCompiler::Source::GetCachedData. It can then be persisted for later, for example by writing it to disk.

During later compilations, the previously produced cache data can be attached to the source object and passed v8::ScriptCompiler::kConsumeCodeCache as an option. This time, code will be produced much faster, as V8 bypasses compiling the code and deserializes it from the provided cache data.

Producing cache data comes at a certain computational and memory cost. For this reason, Chrome only produces cache data if the same script is seen at least twice within a couple of days. This way Chrome is able to turn script files into executable code twice as fast on average, saving users valuable time on each subsequent page load.

Posted by Yang Guo, Software Engineer

21 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. It is always great to see more optimizations! Thank you!

    Is Chrome (Chrome, Chromium, or V8) storing the cached code on disk? If so, is it verifying the integrity of the file?
    If not, this seems somewhat dangerous.

    ReplyDelete
    Replies
    1. We have a quick integrity check to avoid accidental data corruptions (those happen quite often across the hundreds of millions of users). But the integrity check does not use a cryptographic hash. We do not rely on this check to prevent malicious data.

      The reason is that if an attacker can write to disk, he has already won. In that case, there are many easier ways to cause harm than to abuse Chrome's code cache.

      V8 itself does not deal with storing or loading the cache data (as apparent from the API). Chrome does that by allocating a secondary stream for the cache data alongside the script source in the HTTP cache.

      Delete
    2. This is Totally New to me,I'm quite Com.Illiterate,this is Amazing!!! Thank You All for Your Service!!

      Delete
  3. Also, does it apply to content scripts, background scripts and other extension system scripting notions?

    ReplyDelete
    Replies
    1. Currently Chrome only caches scripts that have an HTTP cache entry. However, this is an implementation detail in Chrome. The V8 feature is not inherently limited to those scripts.

      Delete
  4. Thank you for the answers. Do you also cache optimizations?

    ReplyDelete
    Replies
    1. Currently, optimized code compiled in V8 are context-specific. They can therefore not be ported from one V8 instance to another. This may change in the future though.

      Delete
    2. Is it under development? I was wondering when the feature would be released. Could you explain a little more about what prevents caching optimized code the future plan? Thanks!

      Delete
    3. Not under development unfortunately. Optimized code are context-dependent, but the code cache must not be. Besides, optimized code is only generated for hot code, not on initial compile. Caching optimized code would not improve startup performance at all.

      Delete
    4. Not under development unfortunately. Optimized code are context-dependent, but the code cache must not be. Besides, optimized code is only generated for hot code, not on initial compile. Caching optimized code would not improve startup performance at all.

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Does this mean that tab runtime memory equals its disk cache? Will this allow OS VM manager to "page out" site's memory without using the pagefile but just "paging it out" using browser's disk cache?

    ReplyDelete
    Replies
    1. Unfortunately not. A browser tab consists of a lot more than the script's compiled code.

      Delete
  7. It is mentioned above that "it produces cache data if the same script is seen at least twice within a couple of days". Is it possible to force producing code cache on the 1st time loading of the javascript? Is there any configuration in Source/bindings/core/v8?

    Also, for file:// uris, it is not seen that compileAndProduceCache() / compileAndConsumeCache() etc are not happening, but subsequent loading (3rd time onwards) of scripts are faster just like in http:// requests. Hows it done for file:// uri? Any code paths to look for?

    ReplyDelete
    Replies
    1. You can run Chrome with --v8-cache-options=code to cache on first compile.

      Delete
    2. You can run Chrome with --v8-cache-options=code to cache on first compile.

      Delete
    3. I tried to run the Chromium with --v8-cache-options=code
      But this option can't work with 'file://' uris.
      How can we enable this feature with local file?

      Delete
  8. After some investigation I see that caching functionality is supported for http(s). (On Resource creation, CacheHandler is created only for http family). About producing code cache, if the resource is seen 2nd time with in the so called "hotHours" (72 hours in current implementation), then it produces cache for future use.

    ReplyDelete
  9. How dose chrome identify the same script for code caching? Two same urls may have different content, and two different urls may have same content.

    ReplyDelete