Skip to content

Instantly share code, notes, and snippets.

@CoolOppo
Last active November 7, 2023 15:24
Show Gist options
  • Star 30 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save CoolOppo/f54d86abe726bb7827e6 to your computer and use it in GitHub Desktop.
Save CoolOppo/f54d86abe726bb7827e6 to your computer and use it in GitHub Desktop.
Converting videos to GIFs optimally using ImageMagick

ImageMagick Video to GIF Optimization Summary

A software developer who uses IM to create Movie GIFs, Benoit Rouleau, in discussion with me, gave me a AVI video of a plane flying over, to help us mutually explore IM video conversion techniques.

However while the AVI itself is quite small, the uncompressed video is a massive 6201996 bytes in size, and involves 32448 colors, over 107 frames.

IM however has no real trouble converting this video into a GIF animation. However be warned that you will probably get some unsupported 'AVI chunk' errors, which can be ignored by using a "-quiet" control setting.
  convert -quiet -delay 1 plane.avi plane.gif

This used ImageMagick's the default Color Quantization and Dithering methods, to produce a very reasonable conversion of the video. Very few color problems exist, because the video uses very few colors to start with. This is not always the case, especially as GIF has 256 colors per frame limit.

However the animation file is 1077859 bytes in size, which while only 1/5th the size, due to color reduction and GIF pixel data compression, it is still rather large.

Also if you study the resulting animation further you will find that of the 107 frames in the image, 106 frames had their own own separate local color table added. That is each and every frame in the GIF animation required there own color index table. That is while each frame has less that 256 colors (due to the GIF format limitations), the whole animation is using a total of 7525 colors.

Unfortunately the GIF format does not compress color tables, so all those extra color tables could be using up to:   256 colors * 3 byte per color * 106 frames;   or 81,408 bytes of file space. Not a lot for a 1Gbyte video but still an appreciable amount of space, especially as we optimize the video further.

Added to this is that the animation will not GIF frame optimize very well. Not only because the background is moving (due to the camera panning upward), but also because IM used a Error Correction Dither (Hilbert Curve Dither), which produces a pseudo-random pattern of colors that is different from frame to frame. A later example will make this 'dither noise' much more visible.

Common Global Color Table

Here I Generate a Single Global Color Table for all the frames of the video.
  convert -quiet -delay 1 plane.avi  +map   plane_cgc.gif
This naturally results in 0 local color tables, and a file size of 1060471 bytes.

As you can see the resulting animation has no extra local colortables. Instead IM generated a single global color table of 256 of the 'best' colors based on all the frames in the animation.

Unfortunately this also resulted in the pixel data not compressing as well as it did before, as a stronger dither was required. The result is a slightly worse looking animation, that is roughly the same size as the previous.

For this specific video of limited colors, I could even reduce the number of colors used even further say to only 64 colors without too many problems, producing an even smaller animation file size. This however is very dependent on the video sequence used, and may not look very good.

Your own video may have a better result or worse result, especially when dealing with a video that uses a lot more colors and possibly multiple scenes.

Universal Global Color Table

The better way to generate a 'smaller' GIF animation is to just supply a general universal range of colors rather than generate the 'best' global color table for an animation. Use one that should work well regardless of what colors are present in the original video.

Another reason for doing this is that you can make you video longer without serious detrimental effects on the color selection, or resorting local color tables for each frame. Each frame is dithered to the same color map, completely independently of what other frames are in the animation.

Here I use a '332' color map which is usually regarded as being a very good standard colormap when no transparency is needed. I have often seen this colormap (or a 219 color 'web-safe' colormap) used often in various video formats.
  convert -quiet -delay 1 plane.avi -map colormap_332.png plane_ugc.gif

This animation has 0 local color tables, and as a result the animation is smaller or 704091 bytes in size.

The problem however is that you will often see a obvious and annoying 'noise' in areas of constant color. This noise was also present in ALL the previous video animations. It is only now visible due to the use of a more universal, and thus more widely spread out color mapping.

The noise is actually caused by the dithering of the reduced color set when regenerating the image. However, this produces a pseudo-random pattern of colors that changes from frame to frame, resulting in the appearance of background noise in the image. See Problems with E-Dithers for more detail as to why this happens.

We could just turn off the color dithering to remove the 'dither noise'...
  convert -quiet -delay 1 plane.avi \
          +dither -map colormap_332.png plane_ugc_nd.gif
Which has 0 local color tables, and is 110866 bytes in size.

The resulting animation is a very small 1/60th the size of the original animation, generally because of the large expanses of solid color producing extremely good pixel compression. But while it fixes the dither noise, and make for a very small file size, you get color banding instead, which is generally regarded as a very bad trade-off.

Ordered Dithered Video

The real solution is to use a different color dithering technique, which does not produce a different pattern from one frame to the next.

For example here I used a Ordered Dither using Posterized Color Levels to dither the same universal '332' colormap.
  convert -quiet -delay 1 plane.avi \
          -ordered-dither o8x8,8,8,4 +map plane_od.gif
Which has 0 local color tables, and is 303037 bytes in size.

The above also used the +map operator, to ensure that all the images use the exact same global color map (which the ordered dither already reduced to a maximum of 256 colors). As the number of colors is already optimal, the +map operator does no dithering, or color reduction.

The resulting dither pattern is not random, and does not change greatly from one frame to the next. Thus the 'dither noise' has been remove from the animation resulting in a fixed color pattern from from to frame.

The pattern is also very repetitive allowing much better compression.

And finally as the color map is fixed, it should work reasonably well regardless of what video is used.

Higher Quality Ordered Dithered Video

This specific video however only uses a small range of colors, mostly various shades of blue, so it doesn't actually use a lot of the colors provided by a general uniform colormap.

In fact only 31 colors were used in the last video animation!

This is extremely low, and as such also quite visible. But it also means that this particular animation can benefit from using a large number of 'color levels' in the ordered dither operation, so as improve the overall quality.

First however we need to determine how many color levels the animation can handle before it reaches the 256 color limit imposed by both the GIF file format and the global colormap re-mapping.

The tricky part however is that you must determine these BEFORE you save the animation to the limited GIF format. And here is the command I use...

    convert -quiet plane.avi -ordered-dither o8x8,23 -append -format %k info:
235

Basically I increased and decreased the number of color levels to use, until I had a figure that was just within the required 256 color limit.

I can then apply the discovered 'color level' choice to the plane animation.
  convert -quiet -delay 1 plane.avi \
          -ordered-dither o8x8,23 +map plane_od2.gif
Which has 0 local color tables, is 660614 bytes in size, and 235 colors.

As you can see a very high quality, ordered dithered video was generated, which is on a par with the 'best colormap' global color map version we generated earlier, but also 1/3 smaller in size, while the 'dither noise' is now much harder to see.

Of course as the quality is so much higher, it does require a larger file size, as it doesn't compress as well as the low quality version.

On the other hand you now actually have a good control over the quality vs file size trade-off in the form of the number of 'color levels' used.

Just remember this technique is a special case, for an animation that does not use too many colors. And making the video longer by adding more frames will also add more colors, and thus require a reduction in the 'color level' quality control.

This is about the best method of color optimization I have yet seen for general GIF animations. It removes 'dither noise', provides some quality control, and retains the ability to use other GIF animation optimization methods, such as Frame Optimization.

Compression (Transparency) Optimization

Because this video uses a panning camera, the background of the video changes from frame to frame. This means the GIF animation will not Frame Optimize very well.

However we can still use a simple Transparency Optimization to further reduce the final size of the GIF animation.
  convert plane_od2.gif  -layers OptimizeTransparency +map plane_opt.gif
Which has 0 local color tables, is 531981 bytes in size, and 236 colors.

That is one extra color, a transparent color index, was added to the image, and any pixel that does not change the currently displayed color was made transparent. This in turn generates large segments of transparent areas in the original animation, as well as repeats of similar pixel sequences, which generates a improved LZW compression in the final GIF image.

Not bad, the animation is now half that of the direct conversion to GIF, and still a reasonably high quality.

If you like to add to the above, discuss the techniques to further improve them, please contact me, or the IM forum. I am more than happy to hear about your views, techniques and discussions, or look at a specific video/animation problem you may have.

One such discussion is Finding the "right levels" for quantization with anim GIF.

@suprematis
Copy link

Outstanding write-up. I wish some people developing GIFs conversion apps would apply some of these principles, specially when it comes to size and color optimization. Keep up the good work!!!

@jnelken
Copy link

jnelken commented May 29, 2019

TIL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment