Currently there are two rate control modes for video encoding in WebCodecs:
- Constant bitrate mode
- Variable bitrate mode
The bitrate is set via VideoEncoderConfig and can be periodically changed. But there is demand for controlling the quality and the size of each of the encoded frames for following reasons:
- A website might know extra information about the video frames. Some frames contain a lot of motion and new details and require more quality. Others are static and have mostly one color.
- Rapidly changing network conditions. A website might want to quickly adjust the quality of the picture reacting to sudden packet loss increase. This can be done faster than regular encoder bitrate change.
- A website might want to implement a rate control algorithm. It is not computationally heavy and reduces reliance on implementations by OS, browser, and GPU driver vendors. External rate control algorithms are available for different video encoders and even shipped as libraries (https://github.com/intel/libmebo/blob/master/explainer.md )
People working on video recording and video conferencing express their needs:
Usually perf-frame quality and bitrate control is done via controlling the quantizer. Quantizer (QP) is a parameter that helps determine the portion of details discarded in lossy compression. A smaller quantization leads to a higher quality frame, but uses more resources, and a larger quantization does the opposite. The concept of QP exists in most popular codecs, although its ranges are codec specific.
We add
- A new rate control mode: quantizer ( https://www.w3.org/TR/webcodecs/#dom-videoencoderbitratemode-quantizer )
- A set of codec specific extensions to VideoEncoderEncodeOptions. Each extension adds a quantizer field and defines a range of valid values for it.
When VideoEncoder is configured with bitrateMode = “quantizer”, it reads these fields and passes them to the underlying platform encoder or encoding library. Unlike target bitrate, QP range and the meaning of the QP value are parts of the codec specification. It means that different underlying encoders will interpret it the same way and provide very similar results.
These knobs will allow website authors to control compression quality and the size of each encoded frame.
const encoder_config = {
codec: 'vp09.00.10.08',
width: 800,
height: 600,
bitrateMode: 'quantizer',
framerate: 30,
latencyMode: "realtime",
};
const init = {
output: (chunk, config) => {
console.log(chunk.byteLength);
},
error: (e) => {
console.log(e.message);
}
};
const support = await VideoEncoder.isConfigSupported(encoder_config);
if (!support.supported) {
return;
}
let encoder = new VideoEncoder(init);
encoder.configure(encoder_config);
//.........
const frame = new VideoFrame(canvas, { timestamp: performance.now() * 1000 });
let encode_options = { keyFrame: insert_keyframe };
encode_options.vp9 = { quantizer : 50 };
encoder.encode(frame, encode_options);
frame.close();