Skip to content

Instantly share code, notes, and snippets.

@buzztaiki

buzztaiki/fix_docker_mysql_memory_problem.md

Created May 24, 2023 09:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save buzztaiki/18cf2159cc074aed7251653a9577d496 to your computer and use it in GitHub Desktop.
Save buzztaiki/18cf2159cc074aed7251653a9577d496 to your computer and use it in GitHub Desktop.
Download ZIP
docker で mysql を動かしたらメモリ食いすぎで起動しないのを何とかする
Raw
fix_docker_mysql_memory_problem.md

docker で mysql を動かしたらメモリ食いすぎで起動しないのを何とかする

以下のあたり

こんな状況になって

top - 16:11:06 up  7:32,  0 users,  load average: 0.84, 2.05, 1.42
Tasks:   2 total,   1 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.4 us,  3.0 sy,  0.2 ni, 87.1 id,  3.3 wa,  0.9 hi,  0.2 si,  0.0 st
KiB Mem:  16262332 total, 16100992 used,   161340 free,    10252 buffers
KiB Swap:  8388604 total,  6197880 used,  2190724 free.   443820 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    1 mysql     20   0 17.005g 0.013t      0 S   0.0 85.0   0:05.59 mysqld
  172 root      20   0   24152   2336   2000 R   0.0  0.0   0:00.15 top

docker run で起動しなかったりする。

原因

詳細な原因は分からないけどファイル記述子の上限が高すぎるとメモリを食うらしい。

❯❯ cat /proc/$(pgrep dockerd)/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        unlimited            unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             unlimited            unlimited            processes
Max open files            1073741816           1073741816           files
Max locked memory         67108864             67108864             bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       63327                63327                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

❯❯ docker run --rm busybox sh -c 'ulimit -a'
core file size (blocks)         (-c) unlimited
data seg size (kb)              (-d) unlimited
scheduling priority             (-e) 0
file size (blocks)              (-f) unlimited
pending signals                 (-i) 63327
max locked memory (kb)          (-l) 65536
max memory size (kb)            (-m) unlimited
open files                      (-n) 1073741816
POSIX message queues (bytes)    (-q) 819200
real-time priority              (-r) 0
stack size (kb)                 (-s) 8192
cpu time (seconds)              (-t) unlimited
max user processes              (-u) unlimited
virtual memory (kb)             (-v) unlimited
file locks                      (-x) unlimited

解決

コンテナのデフォルトのファイル記述子の上限を制限するか、コンテナ単位で制限する。

以下はデフォルトを設定する方法。

% sudo vim /etc/docker/daemon.json

❯❯ cat /etc/docker/daemon.json
{
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  }
}

% sudo systemctl restart docker.service

❯❯ cat /proc/$(pgrep dockerd)/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        unlimited            unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             unlimited            unlimited            processes
Max open files            1073741816           1073741816           files
Max locked memory         67108864             67108864             bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       63327                63327                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

❯❯ docker run --rm busybox sh -c 'ulimit -a'
core file size (blocks)         (-c) unlimited
data seg size (kb)              (-d) unlimited
scheduling priority             (-e) 0
file size (blocks)              (-f) unlimited
pending signals                 (-i) 63327
max locked memory (kb)          (-l) 65536
max memory size (kb)            (-m) unlimited
open files                      (-n) 64000
POSIX message queues (bytes)    (-q) 819200
real-time priority              (-r) 0
stack size (kb)                 (-s) 8192
cpu time (seconds)              (-t) unlimited
max user processes              (-u) unlimited
virtual memory (kb)             (-v) unlimited
file locks                      (-x) unlimited
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment