Skip to content

Instantly share code, notes, and snippets.

@seg-lol
Last active October 10, 2020 00:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save seg-lol/0d22cf5002f890305cfd094f9ed1094a to your computer and use it in GitHub Desktop.
Save seg-lol/0d22cf5002f890305cfd094f9ed1094a to your computer and use it in GitHub Desktop.

There is an article on HN about curl getting a portion of it ported over to Rust [1] [2] . I learned a couple things

[1] https://news.ycombinator.com/item?id=24729218

[2] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyper/

git clone https://github.com/hyperium/hyper
cd hyper

cargo install cargo-vendor
# for counting lines of source
cargo install tokei
cargo vendor --offline ../hyper-vend


# ...
   Vendoring winapi-util v0.1.5 (/Users/seg_lol/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-util-0.1.5) to ../hyper-vend/winapi-util
   Vendoring winapi-x86_64-pc-windows-gnu v0.4.0 (/Users/seg_lol/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-x86_64-pc-windows-gnu-0.4.0) to ../hyper-vend/winapi-x86_64-pc-windows-gnu
   Vendoring ws2_32-sys v0.2.1 (/Users/seg_lol/.cargo/registry/src/github.com-1ecc6299db9ec823/ws2_32-sys-0.2.1) to ../hyper-vend/ws2_32-sys
To use vendored sources, add this to your .cargo/config for this project:

[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "../hyper-vend"

Hyper is like 23kloc, but the dependencies sum to over 700kloc.

(py38.env) zircon:hyper$ tokei . 
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 HTML                    1           10           10            0            0
 Markdown                4         2801         2801            0            0
 Rust                   78        26641        19408         3834         3399
 Plain Text              1          139          139            0            0
 TOML                    1          217          174            4           39
-------------------------------------------------------------------------------
 Total                  85        29808        22532         3838         3438
-------------------------------------------------------------------------------
(py38.env) zircon:hyper$ tokei ../hyper-vend/
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Autoconf                1           45           29           11            5
 Dockerfile              1            9            5            0            4
 HTML                    2            6            6            0            0
 JSON                    3         7801         7801            0            0
 Markdown              104        12179        12179            0            0
 Python                  3          757          540           79          138
 ReStructuredText        2          422          422            0            0
 Rust                 2322       822476       666449       105001        51026
 Shell                  10          218           93           72           53
 Plain Text              6        18229        18229            0            0
 TOML                   94         3962         2709          868          385
 YAML                    8          225          184            1           40
-------------------------------------------------------------------------------
 Total                2556       866329       708646       106032        51651
-------------------------------------------------------------------------------
$ cargo-supply-chain crates
The following crates will be ignored because they come from a local directory:
- hyper
Fetching publisher info from crates.io
This will take roughly 2 seconds per crate due to API rate limits
Fetching data for "aho-corasick" (0/80)
Fetching data for "atty" (1/80)
Fetching data for "autocfg" (2/80)
Fetching data for "bitflags" (3/80)
Fetching data for "bytes" (4/80)
Fetching data for "cfg-if" (5/80)
Fetching data for "env_logger" (6/80)
Fetching data for "fnv" (7/80)
Fetching data for "fuchsia-zircon" (8/80)
Fetching data for "fuchsia-zircon-sys" (9/80)
Fetching data for "futures-channel" (10/80)
Fetching data for "futures-core" (11/80)
Fetching data for "futures-sink" (12/80)
Fetching data for "futures-task" (13/80)
Fetching data for "futures-util" (14/80)
Fetching data for "h2" (15/80)
Fetching data for "hashbrown" (16/80)
Fetching data for "hermit-abi" (17/80)
Fetching data for "http" (18/80)
Fetching data for "http-body" (19/80)
Fetching data for "httparse" (20/80)
Fetching data for "httpdate" (21/80)
Fetching data for "humantime" (22/80)
Fetching data for "idna" (23/80)
Fetching data for "indexmap" (24/80)
Fetching data for "iovec" (25/80)
Fetching data for "itoa" (26/80)
Fetching data for "kernel32-sys" (27/80)
Fetching data for "lazy_static" (28/80)
Fetching data for "libc" (29/80)
Fetching data for "log" (30/80)
Fetching data for "matches" (31/80)
Fetching data for "memchr" (32/80)
Fetching data for "mio" (33/80)
Fetching data for "miow" (34/80)
Fetching data for "net2" (35/80)
Fetching data for "num_cpus" (36/80)
Fetching data for "percent-encoding" (37/80)
Fetching data for "pin-project" (38/80)
Fetching data for "pin-project-internal" (39/80)
Fetching data for "pin-project-lite" (40/80)
Fetching data for "pin-utils" (41/80)
Fetching data for "pretty_env_logger" (42/80)
Fetching data for "proc-macro2" (43/80)
Fetching data for "quick-error" (44/80)
Fetching data for "quote" (45/80)
Fetching data for "redox_syscall" (46/80)
Fetching data for "regex" (47/80)
Fetching data for "regex-syntax" (48/80)
Fetching data for "ryu" (49/80)
Fetching data for "serde" (50/80)
Fetching data for "serde_derive" (51/80)
Fetching data for "serde_json" (52/80)
Fetching data for "slab" (53/80)
Fetching data for "socket2" (54/80)
Fetching data for "spmc" (55/80)
Fetching data for "syn" (56/80)
Fetching data for "termcolor" (57/80)
Fetching data for "thread_local" (58/80)
Fetching data for "tinyvec" (59/80)
Fetching data for "tokio" (60/80)
Fetching data for "tokio-macros" (61/80)
Fetching data for "tokio-test" (62/80)
Fetching data for "tokio-util" (63/80)
Fetching data for "tower-service" (64/80)
Fetching data for "tower-util" (65/80)
Fetching data for "tracing" (66/80)
Fetching data for "tracing-core" (67/80)
Fetching data for "try-lock" (68/80)
Fetching data for "unicode-bidi" (69/80)
Fetching data for "unicode-normalization" (70/80)
Fetching data for "unicode-xid" (71/80)
Fetching data for "url" (72/80)
Fetching data for "want" (73/80)
Fetching data for "winapi" (74/80)
Fetching data for "winapi-build" (75/80)
Fetching data for "winapi-i686-pc-windows-gnu" (76/80)
Fetching data for "winapi-util" (77/80)
Fetching data for "winapi-x86_64-pc-windows-gnu" (78/80)
Fetching data for "ws2_32-sys" (79/80)
Dependency crates with the people and teams that can publish them to crates.io:
1. env_logger: team "github:env-logger-rs:publishers", team "github:rust-lang-nursery:libs", team "github:rust-lang-nursery:log-owners", alexcrichton, huonw, sebasmagri, sfackler
2. libc: team "github:rust-lang:libs", JohnTitor, alexcrichton, gnzlbg, huonw, joshtriplett, rust-lang-owner
3. unicode-bidi: team "github:servo:cargo-publish", SimonSapin, behnam, larsbergstrom, mbrubeck, metajack
4. log: team "github:rust-lang-nursery:libs", team "github:rust-lang-nursery:log-owners", alexcrichton, huonw, sfackler
5. net2: team "github:deprecrated:crates", team "github:rust-lang-nursery:libs", team "github:rust-lang:libs", alexcrichton, pfmooney
6. url: team "github:servo:cargo-publish", team "github:servo:rust-url", Hoverbear, SimonSapin, seanmonstar
7. idna: team "github:servo:cargo-publish", team "github:servo:rust-url", SimonSapin, djc
8. regex: team "github:rust-lang-nursery:regex-owners", team "github:rust-lang:libs", BurntSushi, alexcrichton
9. regex-syntax: team "github:rust-lang-nursery:regex-owners", team "github:rust-lang:libs", BurntSushi, alexcrichton
10. bitflags: team "github:rust-lang-nursery:libs", alexcrichton, huonw
11. http-body: team "github:hyperium:http-util", carllerche, seanmonstar
12. percent-encoding: team "github:servo:cargo-publish", team "github:servo:rust-url", SimonSapin
13. serde: team "github:serde-rs:owners", dtolnay, erickt
14. serde_json: team "github:serde-rs:owners", dtolnay, erickt
15. tower-service: team "github:tower-rs:publish", carllerche, seanmonstar
16. bytes: team "github:tokio-rs:core", carllerche
17. cfg-if: team "github:rust-lang:libs", alexcrichton
18. fnv: team "github:servo:cargo-publish", SimonSapin
19. lazy_static: team "github:rust-lang-nursery:libs", Kimundi
20. mio: team "github:tokio-rs:mio-core", carllerche
21. pin-utils: team "github:rust-lang-nursery:futures", MajorBreakfast
22. serde_derive: team "github:serde-rs:owners", dtolnay
23. tokio: team "github:tokio-rs:core", carllerche
24. tokio-macros: team "github:tokio-rs:core", carllerche
25. tokio-test: team "github:tokio-rs:core", carllerche
26. tokio-util: team "github:tokio-rs:core", carllerche
27. tower-util: team "github:tower-rs:publish", carllerche
28. tracing: team "github:tokio-rs:publish-tracing", carllerche
29. tracing-core: team "github:tokio-rs:publish-tracing", hawkw
30. fuchsia-zircon: benbrittain, cramertj, erickt, kulakowski, pylaligand, qwandor-google, raggi, raphlinus, rtsuk, tkilbourn, tmandry
31. fuchsia-zircon-sys: benbrittain, cramertj, erickt, kulakowski, pylaligand, qwandor-google, raggi, raphlinus, rtsuk, tkilbourn, tmandry
32. unicode-normalization: Manishearth, SimonSapin, alexcrichton, huonw, kwantam, sujayakar
33. unicode-xid: Manishearth, SimonSapin, alexcrichton, huonw, kwantam, sujayakar
34. socket2: Thomasdezeeuw, alexcrichton, sfackler, stjepang
35. h2: carllerche, olix0r, seanmonstar
36. autocfg: cuviper, matklad
37. futures-channel: cramertj, taiki-e
38. futures-core: cramertj, taiki-e
39. futures-sink: cramertj, taiki-e
40. futures-task: cramertj, taiki-e
41. futures-util: cramertj, taiki-e
42. http: carllerche, seanmonstar
43. indexmap: bluss, cuviper
44. miow: alexcrichton, yoshuawuyts
45. proc-macro2: alexcrichton, dtolnay
46. redox_syscall: 4lDO2, jackpot51
47. tinyvec: Lokathor, rust-bus-owner
48. aho-corasick: BurntSushi
49. atty: softprops
50. hashbrown: Amanieu
51. hermit-abi: stlankes
52. httparse: seanmonstar
53. httpdate: pyfisch
54. humantime: tailhook
55. iovec: carllerche
56. itoa: dtolnay
57. kernel32-sys: retep998
58. matches: SimonSapin
59. memchr: BurntSushi
60. num_cpus: seanmonstar
61. pin-project: taiki-e
62. pin-project-internal: taiki-e
63. pin-project-lite: taiki-e
64. pretty_env_logger: seanmonstar
65. quick-error: tailhook
66. quote: dtolnay
67. ryu: dtolnay
68. slab: carllerche
69. spmc: seanmonstar
70. syn: dtolnay
71. termcolor: BurntSushi
72. thread_local: Amanieu
73. try-lock: seanmonstar
74. want: seanmonstar
75. winapi: retep998
76. winapi-build: retep998
77. winapi-i686-pc-windows-gnu: retep998
78. winapi-util: BurntSushi
79. winapi-x86_64-pc-windows-gnu: retep998
80. ws2_32-sys: retep998
$ cargo tree
hyper v0.13.8 (/Users/seg_lol/tt/git-archives/github.com/hyperium/hyper)
├── bytes v0.5.6
├── futures-channel v0.3.6
│ └── futures-core v0.3.6
├── futures-core v0.3.6
├── futures-util v0.3.6
│ ├── futures-core v0.3.6
│ ├── futures-task v0.3.6
│ ├── pin-project v0.4.26
│ │ └── pin-project-internal v0.4.26
│ │ ├── proc-macro2 v1.0.24
│ │ │ └── unicode-xid v0.2.1
│ │ ├── quote v1.0.7
│ │ │ └── proc-macro2 v1.0.24 (*)
│ │ └── syn v1.0.42
│ │ ├── proc-macro2 v1.0.24 (*)
│ │ ├── quote v1.0.7 (*)
│ │ └── unicode-xid v0.2.1
│ └── pin-utils v0.1.0
├── h2 v0.2.6
│ ├── bytes v0.5.6
│ ├── fnv v1.0.7
│ ├── futures-core v0.3.6
│ ├── futures-sink v0.3.6
│ ├── futures-util v0.3.6 (*)
│ ├── http v0.2.1
│ │ ├── bytes v0.5.6
│ │ ├── fnv v1.0.7
│ │ └── itoa v0.4.6
│ ├── indexmap v1.6.0
│ │ └── hashbrown v0.9.1
│ │ [build-dependencies]
│ │ └── autocfg v1.0.1
│ ├── slab v0.4.2
│ ├── tokio v0.2.22
│ │ ├── bytes v0.5.6
│ │ ├── fnv v1.0.7
│ │ ├── futures-core v0.3.6
│ │ ├── iovec v0.1.4
│ │ │ └── libc v0.2.79
│ │ ├── lazy_static v1.4.0
│ │ ├── memchr v2.3.3
│ │ ├── mio v0.6.22
│ │ │ ├── cfg-if v0.1.10
│ │ │ ├── iovec v0.1.4 (*)
│ │ │ ├── libc v0.2.79
│ │ │ ├── log v0.4.11
│ │ │ │ └── cfg-if v0.1.10
│ │ │ ├── net2 v0.2.35
│ │ │ │ ├── cfg-if v0.1.10
│ │ │ │ └── libc v0.2.79
│ │ │ └── slab v0.4.2
│ │ ├── pin-project-lite v0.1.10
│ │ ├── slab v0.4.2
│ │ └── tokio-macros v0.2.5
│ │ ├── proc-macro2 v1.0.24 (*)
│ │ ├── quote v1.0.7 (*)
│ │ └── syn v1.0.42 (*)
│ ├── tokio-util v0.3.1
│ │ ├── bytes v0.5.6
│ │ ├── futures-core v0.3.6
│ │ ├── futures-sink v0.3.6
│ │ ├── log v0.4.11 (*)
│ │ ├── pin-project-lite v0.1.10
│ │ └── tokio v0.2.22 (*)
│ └── tracing v0.1.21
│ ├── cfg-if v0.1.10
│ ├── log v0.4.11 (*)
│ ├── pin-project-lite v0.1.10
│ └── tracing-core v0.1.17
│ └── lazy_static v1.4.0
├── http v0.2.1 (*)
├── http-body v0.3.1
│ ├── bytes v0.5.6
│ └── http v0.2.1 (*)
├── httparse v1.3.4
├── httpdate v0.3.2
├── itoa v0.4.6
├── pin-project v0.4.26 (*)
├── socket2 v0.3.15
│ ├── cfg-if v0.1.10
│ └── libc v0.2.79
├── tokio v0.2.22 (*)
├── tower-service v0.3.0
├── tracing v0.1.21 (*)
└── want v0.3.0
├── log v0.4.11 (*)
└── try-lock v0.2.3
[dev-dependencies]
├── futures-util v0.3.6 (*)
├── matches v0.1.8
├── num_cpus v1.13.0
│ └── libc v0.2.79
├── pretty_env_logger v0.4.0
│ ├── env_logger v0.7.1
│ │ ├── atty v0.2.14
│ │ │ └── libc v0.2.79
│ │ ├── humantime v1.3.0
│ │ │ └── quick-error v1.2.3
│ │ ├── log v0.4.11 (*)
│ │ ├── regex v1.3.9
│ │ │ ├── aho-corasick v0.7.13
│ │ │ │ └── memchr v2.3.3
│ │ │ ├── memchr v2.3.3
│ │ │ ├── regex-syntax v0.6.18
│ │ │ └── thread_local v1.0.1
│ │ │ └── lazy_static v1.4.0
│ │ └── termcolor v1.1.0
│ └── log v0.4.11 (*)
├── serde v1.0.116
├── serde_derive v1.0.116
│ ├── proc-macro2 v1.0.24 (*)
│ ├── quote v1.0.7 (*)
│ └── syn v1.0.42 (*)
├── serde_json v1.0.58
│ ├── itoa v0.4.6
│ ├── ryu v1.0.5
│ └── serde v1.0.116
├── spmc v0.3.0
├── tokio v0.2.22 (*)
├── tokio-test v0.2.1
│ ├── bytes v0.5.6
│ ├── futures-core v0.3.6
│ └── tokio v0.2.22 (*)
├── tokio-util v0.3.1 (*)
├── tower-util v0.3.1
│ ├── futures-core v0.3.6
│ ├── futures-util v0.3.6 (*)
│ ├── pin-project v0.4.26 (*)
│ └── tower-service v0.3.0
└── url v1.7.2
├── idna v0.1.5
│ ├── matches v0.1.8
│ ├── unicode-bidi v0.3.4
│ │ └── matches v0.1.8
│ └── unicode-normalization v0.1.13
│ └── tinyvec v0.3.4
├── matches v0.1.8
└── percent-encoding v1.0.1
__doc__ = """\
git clone https://github.com/hyperium/hyper
cd hyper
"""
import os; import re;
print("uniq crates:" + str(len({ x.split(' ')[0] for x in set(re.findall("─ (.*) ", os.popen("cargo tree -e no-dev").read())) })))
deps = { x.split(' ')[0] for x in set(re.findall("─ (.*) ", os.popen("cargo tree -e no-dev").read())) }
def pad(n, s):
return s + ' ' * (n - len(s))
v = sorted(deps)
ml = max(len(x) for x in v)
v2 = [ pad(ml+2,x) for x in v ]
f = list(zip( v2[0:-1:3]
, v2[1:-1:3]
, v2[2:-1:3]))
print("\n".join(''.join(x) for x in f))
"""
uniq crates:42
autocfg bytes cfg-if
fnv futures-channel futures-core
futures-sink futures-task futures-util
h2 hashbrown http
http-body httparse httpdate
indexmap iovec itoa
lazy_static libc log
memchr mio net2
pin-project pin-project-internal pin-project-lite
pin-utils proc-macro2 quote
slab socket2 syn
tokio tokio-macros tokio-util
tower-service tracing tracing-core
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment