Skip to content

Instantly share code, notes, and snippets.

@rylev
Created March 1, 2019 15:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rylev/8cbe987a06d56853fee287f113773ff2 to your computer and use it in GitHub Desktop.
Save rylev/8cbe987a06d56853fee287f113773ff2 to your computer and use it in GitHub Desktop.
Script for downloading the most popular crates and finding how many times they use the unsafe keyword.
require "open-uri"
require "json"
PAGE_SIZE=100
NUM_PAGES=2
crates = []
(1..NUM_PAGES).each do |page|
res = JSON.parse(URI.parse("https://crates.io/api/v1/crates?page=#{page}&per_page=#{PAGE_SIZE}").read)
puts res
crates += res["crates"].map {|c| {name: c["name"], repo: c["repository"]} }
end
if !File.exists?('crates')
Dir.mkdir 'crates'
end
csv = File.open("stats.csv", "w")
csv.puts "name,rank,unsafe occurences,total lines"
crates.each_with_index do |c, i|
repo = c[:repo]
name = c[:name]
if Dir.exist?("crates/#{name}")
puts "#{name} already cloned"
else
puts "cloning #{name}..."
`git clone #{repo} crates/#{name}`
end
if !Dir.exists?("crates/#{name}")
puts "#{name} doesn't exist. Skipping."
next
end
files = Dir["crates/#{name}/**/*.rs"]
sum = files.map do |f|
file = File.open(f)
total = file.each_line.count {|l| l = l.strip; !l.start_with?('//') && !l.empty? }
file.rewind
[file.grep(/unsafe\s*(fn|\{|extern)/).length, total]
end.inject {|sum, nxt| [sum[0] + nxt[0], sum[1] + nxt[1]] }
puts "#{name} has #{sum[0]} unsafe instances and #{sum[1]} lines of code"
csv.puts "#{name},#{i + 1},#{sum[0]},#{sum[1]}"
end
name rank unsafe occurences total lines
rand 1 52 11344
syn 2 17 26701
rand_core 3 52 11344
quote 4 0 1133
libc 5 47 53581
lazy_static 6 3 385
bitflags 7 0 888
log 8 6 1226
proc-macro2 9 0 3448
serde 10 4 25838
unicode-xid 11 0 514
regex-syntax 12 157 37183
memchr 13 41 3933
regex 14 157 37183
crossbeam-utils 15 297 25259
cfg-if 16 0 104
itoa 17 5 352
num-traits 18 6 3195
byteorder 19 52 2472
serde_derive 20 4 25838
serde_json 21 5 10994
crossbeam-epoch 22 297 25259
cc 23 50 3499
aho-corasick 24 6 1933
utf8-ranges 25 2 424
num_cpus 26 13 289
thread_local 27 30 616
smallvec 28 48 1886
time 29 32 2954
crossbeam-deque 30 297 25259
semver 31 2 1794
env_logger 32 0 1900
winapi 33 101 176952
ucd-util 34 3 36150
arrayvec 35 45 2227
parking_lot 36 162 7138
rustc_version 37 0 285
parking_lot_core 38 162 7138
backtrace 39 44 2075
base64 40 0 3674
rustc-demangle 41 0 355
semver-parser 42 0 1426
void 43 0 77
nodrop 44 45 2227
atty 45 6 151
version_check 46 0 137
ryu 47 39 2515
scopeguard 48 3 264
num-integer 49 0 1448
unreachable 50 9 46
url 51 7 21996
owning_ref 52 25 1277
unicode-normalization 53 3 144146
unicode-width 54 0 419
slab 55 4 527
chrono 56 0 8032
rand_chacha 57 52 11344
rand_xorshift 58 52 11344
pkg-config 59 0 657
matches 60 2 451
strsim 61 0 688
backtrace-sys 62 44 2075
lazycell 63 11 423
unicase 64 0 2064
memoffset 65 3 230
rand_pcg 66 52 11344
unicode-bidi 67 0 2423
rand_isaac 68 52 11344
idna 69 7 21996
clap 70 3 22169
rand_hc 71 52 11344
percent-encoding 72 7 21996
lock_api 73 162 7138
textwrap 74 0 929
stable_deref_trait 75 0 145
error-chain 77 1 1643
quick-error 78 0 970
hyper 79 12 17342
vec_map 80 0 989
bytes 81 73 3788
failure 82 13 1835
toml 83 0 8324
autocfg 84 0 337
futures 86 176 17126
failure_derive 87 0 337
fnv 88 0 257
termcolor 89 0 1576
either 90 0 473
crossbeam-channel 91 297 25259
synstructure 92 0 1032
iovec 93 20 273
tokio-timer 94 200 33383
openssl-sys 95 787 21900
openssl 96 787 21900
uuid 97 2 2485
tokio-io 98 200 33383
mio 99 154 12008
net2 100 48 1570
tokio-reactor 101 200 33383
safemem 102 3 79
mime 103 4 2425
tokio-threadpool 104 200 33383
itertools 105 10 7654
tokio-executor 106 200 33383
remove_dir_all 107 3 262
tokio 108 200 33383
tempfile 109 4 1514
tokio-tcp 110 200 33383
httparse 111 36 4743
dtoa 112 9 805
walkdir 113 0 1748
foreign-types 114 16 121
foreign-types-shared 115 16 121
same-file 116 3 459
rand_os 117 52 11344
tokio-current-thread 118 200 33383
tokio-fs 119 200 33383
tokio-udp 120 200 33383
phf_shared 121 0 2050
rustc-serialize 122 3 4728
flate2 123 19 3930
phf 124 0 2050
siphasher 125 17 1041
linked-hash-map 126 57 1338
tokio-uds 127 200 33383
phf_generator 128 0 2050
phf_codegen 129 0 2050
tokio-codec 130 200 33383
generic-array 131 48 1724
term 132 9 2275
hex 133 0 238
futures-cpupool 134 176 17126
adler32 135 0 230
rayon 136 135 21909
typenum 137 53 4033
winapi-build 138 101 176952
glob 139 0 1276
crossbeam 140 297 25259
kernel32-sys 141 101 176952
crc32fast 142 8 1220
miniz-sys 143 19 3930
indexmap 144 1 3968
mio-uds 145 16 791
miow 146 90 1630
unicode-segmentation 147 0 4458
http 148 47 8656
rayon-core 149 135 21909
filetime 150 7 577
h2 151 5 23195
openssl-probe 152 0 81
num-iter 153 0 479
native-tls 154 2 1791
string 155 5 151
redox_syscall 156 175 1930
synom 157 17 26701
want 158 0 352
try-lock 159 3 105
nom 160 1 16440
socket2 161 229 3278
winapi-x86_64-pc-windows-gnu 162 101 176952
digest 163 0 953
byte-tools 164 24 1294
winapi-i686-pc-windows-gnu 165 101 176952
gcc 166 50 3499
scoped-tls 167 1 134
yaml-rust 168 0 3948
pulldown-cmark 169 0 19419
quickcheck 170 0 1905
winapi-util 171 20 326
block-buffer 172 24 1294
mime_guess 173 1 1604
tempdir 174 0 264
nix 175 499 15947
serde_urlencoded 176 2 1227
num 177 0 43
libz-sys 178 4 385
language-tags 179 0 1006
heck 180 0 308
num-rational 181 0 1712
getopts 182 0 1784
rand_jitter 183 52 11344
fake-simd 184 24 1294
reqwest 186 5 7062
hyper-tls 187 1 305
bincode 188 1 2376
crc 189 0 718
encoding_rs 190 286 131513
fuchsia-zircon 191 499 121935
fuchsia-zircon-sys 192 499 121935
libflate 193 16 4262
memmap 194 47 907
build_const 195 0 213
sha2 196 21 6625
untrusted 197 0 235
num-complex 198 0 1930
cloudabi 199 76 1196
sha1 200 2 727
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment