preview_cardでGIFを引っ張ってくるときのメモリ消費がつらい

gif_preview_card_consumes_memory.md

400x400で、フレーム数が多いGIFだとかなりやばいっぽい。

サンプルは400x400, 198フレームのGIFファイル。

普通にapt-getで入る版

--------------------------------------------------------------------------------
Command:            /usr/bin/convert sample3.gif -coalesce -auto-orient -resize 400x400> -layers optimize -quality 80 -strip out.gif
Massif arguments:   (none)
ms_print arguments: massif.out.21536
--------------------------------------------------------------------------------


    MB
430.8^                   #
     |                  @#
     |                 :@#
     |               @::@#
     |              :@::@#
     |            :::@::@#
     |           ::::@::@#
     |         :@::::@::@#
     |       :::@::::@::@#
     |   @  ::::@::::@::@#
     |   @::::::@::::@::@#:::::::::::@:::::::::::@::::@
     |   @::::::@::::@::@#:::::::::::@:::::::::::@::::@
     |   @::::::@::::@::@#:::::::::::@:::::::::::@::::@
     |  @@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     |  @@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     |  @@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     |  @@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     | @@@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     | @@@::::::@::::@::@#:::::::::::@:::::::::::@::::@    @:::::@:::::::@::::
     |@@@@::::::@::::@::@#:::::::::::@:::::::::::@::::@::::@:::::@:::::::@::::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   18.79

CACHE_SHIFT=3でビルドした版

--------------------------------------------------------------------------------
Command:            convert sample3.gif -coalesce -auto-orient -resize 400x400> -layers optimize -quality 80 -strip out.gif
Massif arguments:   (none)
ms_print arguments: massif.out.19103
--------------------------------------------------------------------------------


    MB
182.8^          #
     |          #
     |          #
     |        @@#
     |        @ #
     |       @@ #
     |       @@ #
     |     @@@@ #
     |   @ @ @@ #                                        ::
     |   @:@ @@ #                                     :::::
     |   @:@ @@ #::::::::::@@::@::::::::::::::::::@:::: :::
     |   @:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |   @:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |   @:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |  :@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |  :@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |  :@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::
     |  :@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::      ::::@:::::::@
     | ::@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::@@::::: ::@:::::::@
     | ::@:@ @@ #::: :: :::@ ::@: : :: :::::::: ::@: :: :::@ ::: : ::@:::::::@
   0 +----------------------------------------------------------------------->Gi
     0                                                                   25.77

メモリ消費がやべーのはどこか？

99.48% (190,690,805B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->87.04% (166,839,552B) 0x4F3D863: AcquireAlignedMemory (memory.c:262)
| ->87.02% (166,814,208B) 0x4E9F2BA: OpenPixelCache (cache.c:3537)
| | ->86.58% (165,971,712B) 0x4E886A0: GetImagePixelCache (cache.c:1634)
| | | ->86.58% (165,971,712B) 0x4EA1879: SyncImagePixelCache (cache.c:5260)
| | |   ->86.58% (165,971,712B) 0x4EB741F: CompositeImage (composite.c:588)
| | |     ->86.58% (165,971,712B) 0x4F30FEB: CoalesceImages (layer.c:331)
| | |     | ->43.29% (82,985,856B) 0x5496A57: MogrifyImageList (mogrify.c:7839)
| | |     | | ->43.29% (82,985,856B) 0x5497674: MogrifyImages (mogrify.c:8952)
| | |     | |   ->43.29% (82,985,856B) 0x543E923: ConvertImageCommand (convert.c:3267)
| | |     | |     ->43.29% (82,985,856B) 0x548C18A: MagickCommandGenesis (mogrify.c:183)
| | |     | |       ->43.29% (82,985,856B) 0x4011F9: MagickMain (magick.c:149)
| | |     | |         ->43.29% (82,985,856B) 0x573282E: (below main) (libc-start.c:291)
| | |     | |
| | |     | ->43.29% (82,985,856B) 0x5496B1F: MogrifyImageList (mogrify.c:8390)
| | |     |   ->43.29% (82,985,856B) 0x5497674: MogrifyImages (mogrify.c:8952)
| | |     |     ->43.29% (82,985,856B) 0x543E923: ConvertImageCommand (convert.c:3267)
| | |     |       ->43.29% (82,985,856B) 0x548C18A: MagickCommandGenesis (mogrify.c:183)
| | |     |         ->43.29% (82,985,856B) 0x4011F9: MagickMain (magick.c:149)
| | |     |           ->43.29% (82,985,856B) 0x573282E: (below main) (libc-start.c:291)
| | |     |
| | |     ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.44% (842,496B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.01% (25,344B) in 1+ places, all below ms_print's threshold (01.00%)
|

どうやらGetImagePixelCacheで派手にメモリを食っているっぽい。

対処

preview_cardでGIFが飛んできたときには最初の一枚だけにする

(Cache_Shift=3 でビルドした版）

--------------------------------------------------------------------------------
Command:            convert sample3.gif[0] -coalesce -auto-orient -resize 400x400> -quality 80 -strip out.png
Massif arguments:   (none)
ms_print arguments: massif.out.23540
--------------------------------------------------------------------------------


    MB
2.248^       #
     |       #
     |       #
     |       #
     |      @#:
     |  :@: @#:                           :::::::::::::::::::::::::::::::::  :
     |  :@: @#:                           :                                  :
     |  :@: @#:                           :                                  :
     |  :@::@#:                           :                                  :
     |  :@::@#:                           :                                :::
     |  :@::@#:::::::::::::::::::::::::::::                                : :
     |  :@::@#:                         : :                                : :
     |  @@::@#:                         : :                                : :
     |::@@::@#:                         : :                                : :
     |: @@::@#@                         : :                                : :
     |: @@::@#@                         : :                                : :
     |: @@::@#@                         : :                                : :
     |: @@::@#@                         : :                                : :
     |@ @@::@#@                         : :                                : :
     |@ @@::@#@                         : :                                : @
   0 +----------------------------------------------------------------------->Mi
     0                                                                   535.9

いい感じぽい

diff --git a/app/models/preview_card.rb b/app/models/preview_card.rb
index 716b8224..c04143ae 100644
--- a/app/models/preview_card.rb
+++ b/app/models/preview_card.rb
@@ -33,7 +33,7 @@ class PreviewCard < ApplicationRecord

   has_and_belongs_to_many :statuses

-  has_attached_file :image, styles: { original: '400x400>' }, convert_options: { all: '-quality 80 -strip' }
+  has_attached_file :image, styles: -> f{ gif2png f }, convert_options: { all: '-quality 80 -strip' }

   include Attachmentable
   include Remotable
@@ -51,6 +51,17 @@ class PreviewCard < ApplicationRecord
     save!
   end

+  class << self
+    private
+    def gif2png(f)
+      if f.content_type == 'image/gif'
+        {original: ['400x400>', :png]}
+      else
+        {original: '400x400>'}
+      end
+    end
+  end
+
   private

   def extract_dimensions

こんな感じのコードに落とし込みました。

なんかちょっとバグってる感ある

同じURLに対しては同じPreviewCardになるようになってたのね。なるほどね。