s3の転送速度計測rubyスクリプト

Readme.md

計測環境

• 鳥取某所

• ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-darwin11.2.0]

• aws-sdk (1.7.0)

• 鳥取某所2

• ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-darwin10.8.0]

• aws-sdk (1.6.6)

• EC2

• t1.micro

• Amazon Linux 64bit

• ruby 1.8.7 (2012-10-12 patchlevel 371) [x86_64-linux]

• aws-sdk (1.6.9)

計測結果

鳥取某所より実施

• 同時刻でのBNRスピードテストでの計測ではアップ15Mbps、ダウン：65Mbbpsくらいだった
• GMT +9:00

2012/11/04 20:14:21 US Standard up:5.65Mbps(722.70KB/s) down:4.29Mbps(549.06KB/s)
2012/11/04 20:20:07 Northern California up:4.04Mbps(517.71KB/s) down:4.17Mbps(533.49KB/s)
2012/11/04 20:27:00 Oregon up:5.67Mbps(725.35KB/s) down:5.02Mbps(643.08KB/s)
2012/11/04 20:32:19 Sao Paulo up:2.62Mbps(335.68KB/s) down:1.90Mbps(243.77KB/s)
2012/11/04 20:45:06 Ireland up:2.75Mbps(351.51KB/s) down:2.06Mbps(263.57KB/s)
2012/11/04 20:57:01 Singapore up:6.49Mbps(830.85KB/s) down:3.62Mbps(463.64KB/s)
2012/11/04 21:03:02 Tokyo up:20.04Mbps(2565.07KB/s) down:20.46Mbps(2619.22KB/s)

鳥取某所より実施2

• 同時刻でのBNRスピードテストでの計測ではアップ、ダウン：600Kbbpsくらいだった
• GMT+9:00
• 10回の平均

2012/11/06 09:43:24 US Standard up:0.49Mbps(62.70KB/s) down:1.20Mbps(153.40KB/s)
2012/11/06 09:47:44 Northern California up:0.29Mbps(36.83KB/s) down:0.91Mbps(115.98KB/s)
2012/11/06 09:54:15 Oregon up:0.37Mbps(47.90KB/s) down:1.36Mbps(173.51KB/s)
2012/11/06 10:00:41 Sao Paulo up:0.28Mbps(35.70KB/s) down:0.83Mbps(106.18KB/s)
2012/11/06 10:07:38 Ireland up:0.51Mbps(64.99KB/s) down:1.18Mbps(150.45KB/s)
2012/11/06 10:12:10 Singapore up:0.58Mbps(74.84KB/s) down:1.45Mbps(185.59KB/s)
2012/11/06 10:15:54 Tokyo up:0.47Mbps(59.89KB/s) down:2.12Mbps(270.77KB/s)

EC2(ap-northeast-1a Tokyo)より実施

• GMT

2012/11/05 00:22:50 Tokyo up:64.58Mbps(8265.77KB/s) down:69.15Mbps(8851.62KB/s)
2012/11/05 00:23:23 Sao Paulo up:2.12Mbps(270.87KB/s) down:1.94Mbps(248.37KB/s)
2012/11/05 00:37:11 Singapore up:5.74Mbps(734.19KB/s) down:7.45Mbps(953.29KB/s)
2012/11/05 00:41:33 Ireland up:1.95Mbps(249.74KB/s) down:2.35Mbps(300.88KB/s)
2012/11/05 00:54:37 Oregon up:5.41Mbps(693.05KB/s) down:6.22Mbps(795.53KB/s)
2012/11/05 00:59:31 Northern California up:3.96Mbps(506.82KB/s) down:4.24Mbps(542.70KB/s)
2012/11/05 01:06:30 US Standard up:3.97Mbps(508.46KB/s) down:4.32Mbps(552.54KB/s)

EC2(us-east-1b N.Virginia)

• GMT

2012/11/05 01:07:53 Tokyo up:4.32Mbps(553.31KB/s) down:4.94Mbps(631.79KB/s)
2012/11/05 01:14:08 Singapore up:1.99Mbps(254.31KB/s) down:2.26Mbps(289.35KB/s)
2012/11/05 01:27:16 US Standard up:43.42Mbps(5558.12KB/s) down:48.44Mbps(6200.17KB/s)
2012/11/05 01:27:59 Sao Paulo up:3.53Mbps(451.49KB/s) down:3.74Mbps(479.16KB/s)
2012/11/05 01:35:42 Ireland up:5.66Mbps(724.44KB/s) down:7.03Mbps(900.06KB/s)
2012/11/05 01:40:10 Oregon up:7.00Mbps(896.31KB/s) down:7.99Mbps(1022.88KB/s)
2012/11/05 01:43:59 Northern California up:24.49Mbps(3134.23KB/s) down:22.67Mbps(2901.71KB/s)

EC2(us-west-2b Oregon)

• GMT

2012/11/05 01:28:25 Oregon up:35.88Mbps(4593.13KB/s) down:59.58Mbps(7625.91KB/s)
2012/11/05 01:29:04 US Standard up:6.52Mbps(834.71KB/s) down:7.98Mbps(1021.82KB/s)
2012/11/05 01:33:10 Singapore up:2.81Mbps(359.85KB/s) down:2.89Mbps(369.99KB/s)
2012/11/05 01:42:57 Tokyo up:6.95Mbps(889.83KB/s) down:6.83Mbps(874.65KB/s)
2012/11/05 01:47:07 Sao Paulo up:2.27Mbps(290.37KB/s) down:2.44Mbps(312.49KB/s)
2012/11/05 01:58:59 Northern California up:7.62Mbps(975.50KB/s) down:7.90Mbps(1011.69KB/s)
2012/11/05 02:02:44 Ireland up:2.92Mbps(374.36KB/s) down:3.64Mbps(465.35KB/s)

EC2(us-west-1c N.California)

• GMT

2012/11/05 01:34:16 Sao Paulo up:2.48Mbps(317.38KB/s) down:2.44Mbps(312.11KB/s)
2012/11/05 01:45:42 Oregon up:42.30Mbps(5414.63KB/s) down:29.71Mbps(3802.84KB/s)
2012/11/05 01:46:33 Northern California up:7.36Mbps(941.51KB/s) down:8.15Mbps(1042.65KB/s)
2012/11/05 01:50:17 US Standard up:7.77Mbps(994.63KB/s) down:9.16Mbps(1173.06KB/s)
2012/11/05 01:53:47 Tokyo up:5.19Mbps(664.00KB/s) down:4.81Mbps(615.90KB/s)
2012/11/05 01:59:25 Singapore up:2.54Mbps(325.04KB/s) down:2.84Mbps(363.42KB/s)
2012/11/05 02:09:53 Ireland up:3.01Mbps(385.65KB/s) down:3.11Mbps(398.61KB/s)

EC2(eu-west-1a Ireland)

• GMT

2012/11/05 01:41:17 Northern California up:7.56Mbps(968.05KB/s) down:7.86Mbps(1006.71KB/s)
2012/11/05 01:45:03 Singapore up:2.20Mbps(281.99KB/s) down:2.17Mbps(277.73KB/s)
2012/11/05 01:57:50 Sao Paulo up:2.83Mbps(362.28KB/s) down:2.24Mbps(286.64KB/s)
2012/11/05 02:09:04 Oregon up:4.35Mbps(557.12KB/s) down:4.56Mbps(584.19KB/s)
2012/11/05 02:15:29 Tokyo up:3.57Mbps(456.79KB/s) down:3.80Mbps(486.25KB/s)
2012/11/05 02:23:13 Ireland up:23.59Mbps(3019.09KB/s) down:16.39Mbps(2097.63KB/s)
2012/11/05 02:24:40 US Standard up:7.99Mbps(1023.18KB/s) down:8.47Mbps(1083.74KB/s)

EC2(ap-southeast-1b Singapore)

• GMT

2012/11/05 01:48:35 Northern California up:2.64Mbps(337.60KB/s) down:2.99Mbps(383.23KB/s)
2012/11/05 01:58:45 Ireland up:1.81Mbps(231.86KB/s) down:1.88Mbps(240.46KB/s)
2012/11/05 02:14:00 Sao Paulo up:1.60Mbps(205.19KB/s) down:1.68Mbps(215.47KB/s)
2012/11/05 02:31:11 US Standard up:2.75Mbps(351.42KB/s) down:3.21Mbps(411.27KB/s)
2012/11/05 02:40:47 Singapore up:32.14Mbps(4114.29KB/s) down:32.61Mbps(4174.21KB/s)
2012/11/05 02:41:41 Oregon up:3.30Mbps(421.88KB/s) down:3.86Mbps(494.23KB/s)
2012/11/05 02:49:37 Tokyo up:8.80Mbps(1125.92KB/s) down:9.33Mbps(1194.55KB/s)

EC2(sa-east-1a Sao Paulo)

• GMT

2012/11/05 02:05:21 Ireland up:1.80Mbps(229.93KB/s) down:2.91Mbps(372.56KB/s)
2012/11/05 02:17:49 Northern California up:4.68Mbps(599.23KB/s) down:4.99Mbps(638.72KB/s)
2012/11/05 02:23:56 Tokyo up:0.96Mbps(123.07KB/s) down:4.54Mbps(580.83KB/s)
2012/11/05 02:41:17 Sao Paulo up:33.88Mbps(4336.75KB/s) down:36.86Mbps(4717.78KB/s)
2012/11/05 02:42:08 Oregon up:3.25Mbps(416.17KB/s) down:3.78Mbps(483.90KB/s)
2012/11/05 02:50:18 Singapore up:0.96Mbps(122.95KB/s) down:1.67Mbps(213.68KB/s)
2012/11/05 03:12:58 US Standard up:5.44Mbps(696.48KB/s) down:5.85Mbps(748.51KB/s)

s3_updown.rb

require 'rubygems'
gem 'aws-sdk'
require 'aws'
require 'optparse'

test_count = 100
data_size = 1024 * 1024
access_key_id = nil
secret_access_key = nil
buckets = {
  'mechamogera-us'        => 'US Standard',
  'mechamogera-nc'        => 'Northern California',
  'mechamogera-oregon'    => 'Oregon',
  'mechamogera-saopaulo'  => 'Sao Paulo',
  'mechamogera-ireland'   => 'Ireland',
  'mechamogera-singapore' => 'Singapore',
  'mechamogeratest'       => 'Tokyo',
}

opt = OptionParser.new
opt.on('-a', '--access-key-id=VAL') { |v| access_key_id = v }
opt.on('-s', '--secret-access-key=VAL') { |v| secret_access_key = v }
opt.parse!(ARGV)

AWS.config(:access_key_id => access_key_id, :secret_access_key => secret_access_key) if access_key_id && secret_access_key
updown_data = "a" * data_size

times = Hash.new
buckets.each do |bucket_name, region|
  s3 = AWS::S3.new

  bucket = s3.buckets[bucket_name]
  up_obj = bucket.objects["down_test#{Time.now.to_i}"]
  up_obj.write(updown_data,
               :content_type => "text/plain",
               :acl => :private,
               :storage_class => :reduced_redundancy)
  times[bucket_name] = Array.new

  start_time = Time.now
  test_count.times do |idx|
    object = bucket.objects["updown_test#{Time.now.to_i}"]
    times[bucket_name] << Hash.new(0)
    before_time = Time.now
    object.write(updown_data,
                 :content_type => "text/plain",
                 :acl => :private,
                 :storage_class => :reduced_redundancy
                )
    times[bucket_name].last[:up] = Time.now - before_time

    before_time = Time.now
    up_obj.read
    times[bucket_name].last[:down] = Time.now - before_time

    object.delete
  end
  up_obj.delete

  up_total = times[bucket_name].inject(0.0) do |t, x| t += x[:up]; t end
  up_avg = up_total / times[bucket_name].size

  down_total = times[bucket_name].inject(0.0) do |t, x| t += x[:down]; t end
  down_avg = down_total / times[bucket_name].size

  printf("#{start_time.strftime("%Y/%m/%d %H:%M:%S")} #{region} " +
         "up:%0.2fMbps(%0.2fKB/s) " +
         "down:%0.2fMbps(%0.2fKB/s)\n",
         (data_size * 8/ 1000000) / up_avg,
         1024.0 / up_avg,
         (data_size * 8/ 1000000) / down_avg,
         1024.0 / down_avg)
  STDOUT.flush
end