Skip to content

Instantly share code, notes, and snippets.

@ktheory
Last active November 10, 2023 23:41
  • Star 67 You must be signed in to star a gist
  • Fork 8 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save ktheory/3c3616fca42a3716346b to your computer and use it in GitHub Desktop.
EC2 EBS-SSD vs instance-store performance on an EBS-optimized m3.2xlarge
# /tmp/test = EBS-SSD
# /mnt/test = instance-store
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/tmp/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 3.26957 s, 82.1 MB/s
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/tmp/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 3.61336 s, 74.3 MB/s
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/tmp/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 3.31484 s, 81.0 MB/s
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/mnt/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.291084 s, 922 MB/s
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/mnt/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.29238 s, 918 MB/s
root@ip-10-0-2-6:~# dd bs=1M count=256 if=/dev/zero of=/mnt/test
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.291242 s, 922 MB/s
## BENCHMARK
# /dev/xvda1 = EBS-SSD
# /dev/xvdb = instance-store SSD
ubuntu@ip-10-0-2-6:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 20G 777M 18G 5% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 15G 8.0K 15G 1% /dev
tmpfs 3.0G 332K 3.0G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 15G 0 15G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/xvdb 74G 52M 70G 1% /mnt
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvda1
/dev/xvda1:
Timing buffered disk reads: 366 MB in 3.01 seconds = 121.66 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvda1
/dev/xvda1:
Timing buffered disk reads: 406 MB in 3.01 seconds = 134.77 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvda1
/dev/xvda1:
Timing buffered disk reads: 406 MB in 3.01 seconds = 134.74 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvda1
/dev/xvda1:
Timing cached reads: 20350 MB in 1.99 seconds = 10237.13 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvda1
/dev/xvda1:
Timing cached reads: 20306 MB in 1.99 seconds = 10214.82 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvda1
/dev/xvda1:
Timing cached reads: 20324 MB in 1.99 seconds = 10223.70 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvdb
/dev/xvdb:
Timing buffered disk reads: 1920 MB in 3.00 seconds = 639.98 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvdb
/dev/xvdb:
Timing buffered disk reads: 2230 MB in 3.00 seconds = 742.96 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -t /dev/xvdb
/dev/xvdb:
Timing buffered disk reads: 2212 MB in 3.00 seconds = 736.89 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvdb
/dev/xvdb:
Timing cached reads: 20326 MB in 1.99 seconds = 10224.69 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvdb
/dev/xvdb:
Timing cached reads: 20538 MB in 1.99 seconds = 10331.94 MB/sec
ubuntu@ip-10-0-2-6:~$ sudo hdparm -T /dev/xvdb
/dev/xvdb:
Timing cached reads: 20350 MB in 1.99 seconds = 10237.58 MB/sec

Device reads (no cache)

hdparm -t /dev/xvd*

Trial EBS-SSD (mb/s) instance-store (mb/s)
1 121.66 639.98
2 134.77 742.96
3 134.74 736.89

Cache reads

hdparm -T /dev/xvd*

Trial EBS-SSD (mb/s) instance-store (mb/s)
1 10,237.13 10,224.69
2 10,214.82 10,331.94
3 10,223.70 10,237.58

Writes

dd bs=1M count=256 if=/dev/zero of=/{tmp,mnt}/test

Trial EBS-SSD (mb/s) instance-store (mb/s)
1 82.1 922
2 74.3 918
3 81.0 922

Conclusions:

Instance-store is over 5x faster than EBS-SSD for uncached reads.

Instance-store and EBS-SSD are equalivent for cached reads.

Instance-store is over 10x faster than EBS-SSD for writes.

@SOODANS
Copy link

SOODANS commented Aug 7, 2018

Thanks...for valuable input.

There is some more information in the below url
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html

@rpietzsch
Copy link

Thanks super helpful data. Can you share some details on what kind of ESB storage type (GP2/GP3/IO2/...) you used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment