Skip to content

Instantly share code, notes, and snippets.

@kaiix
Forked from kkc/EC2.md
Last active February 3, 2016 04:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kaiix/f8bb04be375a45671ccc to your computer and use it in GitHub Desktop.
Save kaiix/f8bb04be375a45671ccc to your computer and use it in GitHub Desktop.
EC2 settings

EC2 相關設定

記憶體

  1. 設定swap(預設為60, 當系統使用到超過40% memory, 就會嘗試使用swap)

    sysctl -w vm.swappiness=0   # from 60 -> 0
    
  2. Huge Page (especially THP)

    for mongodb: https://jira.mongodb.org/browse/DOCS-2131

    echo never > /sys/kernel/mm/transparent_hugepage/enabled
    

File System

  1. File System flush issues

     # vm.dirty_ratio = 80                  # from 40
     # vm.dirty_background_ratio = 5        # from 10
     # vm.dirty_expire_centisecs = 12000    # from 3000
    
  2. For Mongodb Disk Mounting

    modify /etc/fstab

    /dev/xvdf /mongodb_data ext4 defaults,auto,noatime,noexec 0 0
    /dev/xvdg /journal ext4 defaults,auto,noatime,noexec 0 0
    /dev/xvdh /log ext4 defaults,auto,noatime,noexec 0 0
    
  3. 設定readahead

    sudo blockdev --setra 32 /dev/xvdf
    

    可以利用sudo blockdev --report觀看是否設定成功

    每次開完機自動設定

    echo 'ACTION=="add", KERNEL=="xvdf", ATTR{bdi/read_ahead_kb}="16"' | sudo tee -a /etc/udev/rules.d/85-ebs.rules
    

    ref: http://docs.mongodb.org/ecosystem/platforms/amazon-ec2/

  4. ulimit

    $ sudo nano /etc/security/limits.conf
    * soft nofile 64000
    * hard nofile 64000
    * soft nproc 32000
    * hard nproc 32000
    
    $ sudo nano /etc/security/limits.d/90-nproc.conf
    * soft nproc 32000
    * hard nproc 32000
    

Mongodb EC2 Related Settings

Virtual Memory & File System

sudo vim /etc/sysctl.conf
vm.swappiness = 0                       # from 60
vm.dirty_ratio = 80                     # from 40
vm.dirty_background_ratio = 5           # from 10
vm.dirty_expire_centisecs = 12000       # from 3000
  1. 設定swap(預設為60, 當系統使用到超過40% memory, 就會嘗試使用swap)
  2. vm.dirty_ratio 是如果 dirty page 真的超過多少就強制回寫
  3. 這個參數主要是降低 Linux 寫檔案(dirty page)的 buffer 大小,如果是寫入的量很大的系統建議降低此值
  4. Ref: http://parseflo.at/post/linux-tuning-for-write-heavy-system/
sudo sysctl -p /etc/sysctl.conf         # reload settings

Huge Pages

Huge Page (especially THP) for mongodb: https://jira.mongodb.org/browse/DOCS-2131

echo never > /sys/kernel/mm/transparent_hugepage/enabled

Storage I/O

change xvdf

echo noop | sudo tee /sys/block/xvdf/queue/scheduler
/sys/block/*/queue/rq_affinity  2
/sys/block/*/queue/scheduler        noop
/sys/block/*/queue/nr_requests  256
/sys/block/*/queue/read_ahead_kb    256
mdadm –chunk=64 ...

modify /etc/fstab (using discard to activate TRIM function)

/dev/xvdf /mongodb_data ext4 defaults,auto,discard,noatime,noexec 0 0
/dev/xvdg /journal ext4 defaults,auto,discard,noatime,noexec 0 0
/dev/xvdh /log ext4 defaults,auto,discard,noatime,noexec 0 0
sudo vim /etc/security/limits.conf
* soft nofile 64000
* hard nofile 64000
* soft nproc 32000
* hard nproc 32000

Networking

sudo vim /etc/sysctl.conf

net.core.somaxconn = 1000
net.core.netdev_max_backlog = 5000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 12582912 16777216
net.ipv4.tcp_rmem = 4096 12582912 16777216
net.ipv4.tcp_max_syn_backlog = 8096
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_abort_on_overflow = 1    # maybe

Reference

  1. http://www.brendangregg.com/blog/2015-03-03/performance-tuning-linux-instances-on-ec2.html
  2. http://parseflo.at/post/linux-tuning-for-write-heavy-system/
  3. http://docs.basho.com/riak/latest/ops/tuning/linux/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment