Image Decoding in Blink

image-decoding.md

Image Decoding in Blink / Chrome (true as of M62)

Blink decodes off the main thread for image elements and for CSS styles (an image as an element's background style, for example). Moving the decode to the compositor thread (or the compositor worker thread pool?) does free-up the main thread to work on other tasks. We call this deferred decoding. With deferred decoding, the decode work remains on the critical path for presenting a frame to the display, so it can still cause animation jank.

The HTMLImageElement.decode() API should help with the jank problem. Also deferred decoding does not work with SVG image resources. There are still cases where decoding images happens synchronously on the main thread: 2D canvas drawImage() and createPattern(), and WebGL texture uploads.

For canvas-related usecases, there is a recently shipped API called createImageBitmap that can be used for async decoding. The API is async, but the implementation does not yet fully multi-thread all the work, this is still a work in progress. Currently png and jpeg decoding are done in a background thread.

Chrome on Android has the same behavior as desktop Chrome.

A few things which could use an update here.

1. The decode is indeed on a raster worker pool managed by the compositor, not on the compositor thread.
2. I'm not sure what you mean by deferred decoding not working for SVG images. We do raster these SVGs, like other content, on raster worker threads.
3. Any draw call on a canvas context which uses an image element instead of a ImageBitmap will get synchronous decoding.
4. For ImageBitmap, it does look like we have a code-path which schedules decode on a worker thread pool and is not limited to any particular codes (png or jpeg). IIUC, this doesn't work if you create an ImageBitmap using an image element, filed crbug.com/1008034 to clarify that.