Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hartator/e6fe39c93f29b36c8eb80de70f94da53 to your computer and use it in GitHub Desktop.
Save hartator/e6fe39c93f29b36c8eb80de70f94da53 to your computer and use it in GitHub Desktop.
Raw results for Benchmark: Deep directory structure vs. flat directory structure to store millions of files on ext4 article (https://medium.com/@hartator/benchmark-deep-directory-structure-vs-flat-directory-structure-to-store-millions-of-files-on-ext4-cac1000ca28)
1] pry(main)>
[2] pry(main)>
[3] pry(main)>
[4] pry(main)> require 'digest'
=> true
[5] pry(main)> require 'benchmark'
=> true
[6] pry(main)> require 'open-uri'
=> true
[7] pry(main)>
[8] pry(main)> keys = []
=> []
[9] pry(main)>
[10] pry(main)> 10_000_000.times do
[10] pry(main)* keys << Digest::MD5.hexdigest(rand.to_s)
[10] pry(main)* end
=> 10000000
[11] pry(main)>
[12] pry(main)> body = open("https://serpapi.com/samples/3.json").read; nil;
[13] pry(main)>
[14] pry(main)> `rm -rf /root/files/*`
=> ""
[15] pry(main)>
[16] pry(main)> FileUtils.mkdir_p "dir_flat"
=> ["dir_flat"]
[17] pry(main)>
[18] pry(main)> sleep 100
=> 100
[19] pry(main)> puts Benchmark.measure {
[19] pry(main)* keys.each do |key|
[19] pry(main)* File.write "./dir_flat/#{key}", body
[19] pry(main)* end
[19] pry(main)* }
89.495782 478.911081 568.406863 (932.061097)
=> nil
[20] pry(main)>
[21] pry(main)> sleep 100
=> 100
[22] pry(main)> puts Benchmark.measure {
[22] pry(main)* keys.each do |key|
[22] pry(main)* File.read "./dir_flat/#{key}"
[22] pry(main)* end
[22] pry(main)* }
649.349185 361.809478 1011.158663 (2796.507495)
=> nil
[23] pry(main)>
[24] pry(main)> `rm -rf /root/files/*`
=> ""
[25] pry(main)>
[26] pry(main)> FileUtils.mkdir_p "dir_deep"
=> ["dir_deep"]
[27] pry(main)>
[28] pry(main)> sleep 100
=> 100
[29] pry(main)> puts Benchmark.measure {
[29] pry(main)* keys.each do |key|
[29] pry(main)* dir_path = "./dir_deep/#{key[0..1]}/#{key[2..3]}/"
[29] pry(main)* FileUtils.mkdir_p dir_path
[29] pry(main)* end
[29] pry(main)* }
102.970207 29.968887 132.939094 (134.057899)
=> nil
[30] pry(main)>
[31] pry(main)> sleep 100
=> 100
[32] pry(main)> puts Benchmark.measure {
[32] pry(main)* keys.each do |key|
[32] pry(main)* dir_path = "./dir_deep/#{key[0..1]}/#{key[2..3]}/"
[32] pry(main)* File.write dir_path + key, body
[32] pry(main)* end
[32] pry(main)* }
120.121817 501.101713 621.223530 (1120.039820)
=> nil
[33] pry(main)>
[34] pry(main)> sleep 100
=> 100
[35] pry(main)> puts Benchmark.measure {
[35] pry(main)* keys.each do |key|
[35] pry(main)* dir_path = "./dir_deep/#{key[0..1]}/#{key[2..3]}/"
[35] pry(main)* File.read dir_path + key
[35] pry(main)* end
[35] pry(main)* }
828.138352 571.513574 1399.651926 (5753.102668)
=> nil
[36] pry(m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment