Webcam content often require denoising and is often intelligible even when downscaled or with high quantization levels. Screencast content of presentations or webpages with a lot of text content is completely unintelligible if the quantization levels are too high or if the content is downscaled or otherwise blurry.
Video encoding libraries (like libvpx, OpenH264) provide a way for API users to inform the encoder about
type of the encoded content.
In some cases the web application can make a more-educated guess or take user input,
but currently there is no way to inform VideoEncoder what kind of content is being encoded.
Let's add a contentHint field to VideoEncoderConfig. It will take video content hint values that are already used for MediaStreamTrack: "motion", "text", "detail".
This gives web developers a way to communicate to VideoEncoder the expected type of the content they intent to encode.
This setting is not intended to replace encoder-level settings completely but rather complement them
with a simpler hint that does not require broad knowledge of video encoders.
const encoder_config = {
codec: 'vp8',
width: 1280,
height: 720,
bitrate: 3000000,
bitrateMode: 'constant',
framerate: 30,
latencyMode: 'realtime',
contentHint: 'detail' // We're going to encode things captured from a <canvas>
};
const init = {
output: (chunk, config) => {
console.log(chunk.byteLength);
},
error: (e) => {
console.log(e.message);
}
};
const support = await VideoEncoder.isConfigSupported(encoder_config);
if (!support.supported) {
return;
}
let encoder = new VideoEncoder(init);
encoder.configure(encoder_config);
// Capture a video frame from <canvas>
const frame = new VideoFrame(canvas, { timestamp: performance.now() * 1000 });
// Encode it
encoder.encode(frame);
frame.close();- MediaStreamTrack Content Hints: https://www.w3.org/TR/mst-content-hint/#video-content-hints
- WebCodecs video content hint: https://www.w3.org/TR/webcodecs/#dom-videoencoderconfig-contenthint
- Chromium implementation CL: https://chromium-review.googlesource.com/c/chromium/src/+/4953613
- w3c Media Working Group discussion: https://www.w3.org/2023/06/13-mediawg-minutes.html#t01