Skip to content

Instantly share code, notes, and snippets.

@jingkang99
Last active July 13, 2022 05:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jingkang99/a91d4f8ca7a2f07014d3c5f0b224eb96 to your computer and use it in GitHub Desktop.
Save jingkang99/a91d4f8ca7a2f07014d3c5f0b224eb96 to your computer and use it in GitHub Desktop.
Analyze Go Project Github modules
export EXCLUDEPKG="cloudradar-monitoring|seknox"
# save github package list to a file - gitpkg.txt
grep -r --include=\*.go '"github.com' | grep -Ewv ${EXCLUDEPKG} | awk -F '"' '{print $2}' | sort | uniq -c | awk '{print $2}' > gitpkg.txt
# query star count for each package
for gh in $( awk -F '/' '{print $2"/"$3}' gitpkg.txt | uniq ) ; do star=`curl -s "https://api.github.com/repos/${gh}" | grep stargazers_count | cut -d : -f 2 | tr -d " " | tr -d "," `; printf "github.com/%-35s %s\n" ${gh} ${star} ; done
# grep built-in system modules
grep -r --include=\*.go -A 12 'import (' | grep -P '\s+"(\w|/)+"$' | grep -Ewv "github.com|=" | awk -F '"' '{print $2}' | sort | uniq
# query star, fork, issue count and module description
for gh in $( awk -F '/' '{print $2"/"$3}' gitpkg.txt | uniq ) ; do info=`curl -s https://api.github.com/repos/${gh} | \
jq -r '(.stargazers_count|tostring) + " " + (.forks_count|tostring) + " " + (.open_issues_count|tostring) + "---" + .description'`; \
printf "github.com/%-36s" ${gh}; echo ${info}; done
# example for rport, as of 7/12/22
github.com/asaskevich/govalidator 5413
github.com/deckarep/golang-set 2552
github.com/denisbrodbeck/machineid 629
github.com/go-ole/go-ole 882
github.com/go-sql-driver/mysql 12408
github.com/golang-jwt/jwt 2913
github.com/golang-migrate/migrate 9116
github.com/google/uuid 3718
github.com/gorilla/handlers 1427
github.com/gorilla/mux 16943
github.com/gorilla/websocket 17552
github.com/gregdel/pushover 122
github.com/hashicorp/go-multierror 1631
github.com/hashicorp/go-version 1214
github.com/jmoiron/sqlx 12247
github.com/jordan-wright/email 2081
github.com/jpillora/backoff 541
github.com/jpillora/requestlog 5
github.com/jpillora/sizestr 6
github.com/kardianos/service 3484
github.com/mattn/go-sqlite3 5819
github.com/mitchellh/mapstructure 5882
github.com/mocktools/go-smtp-mock 48
github.com/patrickmn/go-cache 6287
github.com/pkg/errors 7797
github.com/pkg/sftp 1169
github.com/pquerna/otp 1464
github.com/robfig/cron 9913
github.com/scjalliance/comshim 8
github.com/shirou/gopsutil 7936
github.com/sirupsen/logrus 20867
github.com/spf13/cobra 27475
github.com/spf13/viper 19832
github.com/stretchr/testify 17010
github.com/wwt/guac 69
# example for trasa, as of 7/12/22
github.com/BurntSushi/toml 3916
github.com/andelf/go-curl 428
github.com/cloudflare/cfssl 7110
github.com/denisbrodbeck/machineid 629
github.com/digitalocean/godo 1240
github.com/go-chi/chi 11783
github.com/go-chi/hostrouter 44
github.com/go-playground/validator 10815
github.com/go-redis/redis 14854
github.com/google/goexpect 696
github.com/gorilla/websocket 17552
github.com/hashicorp/vault 24436
github.com/huandu/go-sqlbuilder 802
github.com/jaypipes/ghw 1183
github.com/jinzhu/now 3742
github.com/lib/pq 7445
github.com/manifoldco/promptui 4980
github.com/mholt/archiver 3613
github.com/minio/minio-go 1650
github.com/oschwald/geoip2-golang 1345
github.com/phuslu/iploc 259
github.com/pkg/errors 7797
github.com/rs/cors 2124
github.com/russellhaering/gosaml2 232
github.com/russellhaering/goxmldsig 108
github.com/satori/go.uuid 4607
github.com/shirou/gopsutil 7936
github.com/sirupsen/logrus 20867
github.com/spf13/cobra 27475
github.com/spf13/viper 19832
github.com/stretchr/testify 17010
github.com/test-go/testify 26
github.com/trustelem/zxcvbn 55
github.com/tstranex/u2f 290
github.com/vulcand/oxy 1826
-----------------------------------------------------
# rport
bufio
bytes
compress/gzip
context
crypto/aes
crypto/cipher
crypto/ecdsa
crypto/elliptic
crypto/md5
crypto/rand
crypto/sha256
crypto/sha512
crypto/subtle
crypto/tls
crypto/x509
database/sql
database/sql/driver
embed
encoding/base64
encoding/csv
encoding/gob
encoding/json
encoding/pem
errors
fmt
html/template
image/png
io
io/ioutil
log
math
math/rand
mime/multipart
net
net/http
net/http/httptest
net/http/httputil
net/smtp
net/url
os
os/exec
os/signal
os/user
path
path/filepath
reflect
regexp
runtime
sort
strconv
strings
sync
sync/atomic
syscall
testing
time
unsafe
---------------------
# rport
github.com/asaskevich/govalidator 5414 533 158---[Go] Package of validators and sanitizers for strings, numerics, slices and structs
github.com/deckarep/golang-set 2552 211 11---A simple generic set type for the Go language. Trusted by Docker, 1Password, Ethereum and Hashicorp.
github.com/denisbrodbeck/machineid 629 88 9---Get the unique machine id of any host (without admin privileges)
github.com/go-ole/go-ole 882 161 62---win32 ole implementation for golang
github.com/go-sql-driver/mysql 12407 2130 100---Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package
github.com/golang-jwt/jwt 2915 183 44---Community maintained clone of https://github.com/dgrijalva/jwt-go
github.com/golang-migrate/migrate 9116 968 205---Database migrations. CLI and Golang library.
github.com/google/uuid 3720 314 25---Go package for UUIDs based on RFC 4122 and DCE 1.1: Authentication and Security Services.
github.com/gorilla/handlers 1427 230 5---A collection of useful middleware for Go HTTP services & web applications 🛃
github.com/gorilla/mux 16944 1548 19---A powerful HTTP router and URL matcher for building Go web servers with 🦍
github.com/gorilla/websocket 17557 2930 34---A fast, well-tested and widely used WebSocket implementation for Go.
github.com/gregdel/pushover 122 9 1---Go wrapper for the Pushover API
github.com/hashicorp/go-multierror 1631 104 19---A Go (golang) package for representing a list of errors as a single error.
github.com/hashicorp/go-version 1214 129 24---A Go (golang) library for parsing and verifying versions and version constraints.
github.com/jmoiron/sqlx 12248 940 294---general purpose extensions to golang's database/sql
github.com/jordan-wright/email 2081 279 52---Robust and flexible email library for Go
github.com/jpillora/backoff 541 59 3---Simple backoff algorithm in Go (golang)
github.com/jpillora/requestlog 5 4 1---Simple request logging in Go (golang)
github.com/jpillora/sizestr 6 1 0---Pretty print byte counts in Go
github.com/kardianos/service 3485 561 130---Run go programs as a service on major platforms.
github.com/mattn/go-sqlite3 5819 966 110---sqlite3 driver for go using database/sql
github.com/mitchellh/mapstructure 5882 579 39---Go library for decoding generic map values into native Go structures and vice versa.
github.com/mocktools/go-smtp-mock 48 6 3---SMTP mock server written on Golang. Mimic any 📤 SMTP server behaviour for your test environment with fake SMTP server.
github.com/patrickmn/go-cache 6287 759 65---An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.
github.com/pkg/errors 7797 635 42---Simple error handling primitives
github.com/pkg/sftp 1169 336 27---SFTP support for the go.crypto/ssh package
github.com/pquerna/otp 1464 153 7---TOTP library for Go
github.com/robfig/cron 9913 1398 101---a cron library for go
github.com/scjalliance/comshim 8 2 0---Ensures that at least one thread within a Go process maintains an initialized connection to the component object model runtime in Windows
github.com/shirou/gopsutil 7937 1312 144---psutil for golang
github.com/sirupsen/logrus 20869 2148 73---Structured, pluggable logging for Go.
github.com/spf13/cobra 27477 2390 170---A Commander for modern Go CLI interactions
github.com/spf13/viper 19832 1702 435---Go configuration with fangs
github.com/stretchr/testify 17013 1273 333---A toolkit with common assertions and mocks that plays nicely with the standard library
github.com/wwt/guac 69 29 5---Apache Guacamole client ported to Go
-----
# trasa
github.com/BurntSushi/toml 3918 503 11---TOML parser for Golang with reflection.
github.com/andelf/go-curl 428 122 23---golang curl(libcurl) binding.
github.com/cloudflare/cfssl 7111 988 266---CFSSL: Cloudflare's PKI and TLS toolkit
github.com/denisbrodbeck/machineid 629 88 9---Get the unique machine id of any host (without admin privileges)
github.com/digitalocean/godo 1240 276 16---DigitalOcean Go API client
github.com/go-chi/chi 11785 775 27---lightweight, idiomatic and composable router for building Go HTTP services
github.com/go-chi/hostrouter 44 9 2---Little package to map hosts to a variety of http routers for Go API services
github.com/go-playground/validator 10821 965 204---:100:Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving
github.com/go-redis/redis 14860 1849 160---Type-safe Redis client for Golang
github.com/google/goexpect 696 127 18---Expect for Go
github.com/gorilla/websocket 17563 2930 34---A fast, well-tested and widely used WebSocket implementation for Go.
github.com/hashicorp/vault 24442 3416 1190---A tool for secrets management, encryption as a service, and privileged access management
github.com/huandu/go-sqlbuilder 802 77 1---A flexible and powerful SQL string builder library plus a zero-config ORM.
github.com/jaypipes/ghw 1183 140 37---Golang hardware discovery/inspection library
github.com/jinzhu/now 3742 219 7---Now is a time toolkit for golang
github.com/lib/pq 7447 856 282---Pure Go Postgres driver for database/sql
github.com/manifoldco/promptui 4981 258 77---Interactive prompt for command-line applications
github.com/mholt/archiver 3614 336 4---Easily create & extract archives, and compress & decompress files of various formats
github.com/minio/minio-go 1650 503 7---MinIO Client SDK for Go
github.com/oschwald/geoip2-golang 1345 167 7---Unofficial MaxMind GeoIP2 Reader for Go
github.com/phuslu/iploc 259 21 0---Fastest IP To Country Library
github.com/pkg/errors 7797 635 42---Simple error handling primitives
github.com/rs/cors 2125 199 15---Go net/http configurable handler to handle CORS requests
github.com/russellhaering/gosaml2 232 99 35---Pure Go implementation of SAML 2.0
github.com/russellhaering/goxmldsig 108 89 25---Pure Go implementation of XML Digital Signatures
github.com/satori/go.uuid 4608 586 42---UUID package for Go
github.com/shirou/gopsutil 7938 1313 144---psutil for golang
github.com/sirupsen/logrus 20869 2147 73---Structured, pluggable logging for Go.
github.com/spf13/cobra 27479 2390 170---A Commander for modern Go CLI interactions
github.com/spf13/viper 19832 1703 435---Go configuration with fangs
github.com/stretchr/testify 17015 1273 333---A toolkit with common assertions and mocks that plays nicely with the standard library
github.com/test-go/testify 26 1 8---A toolkit with common assertions and mocks that plays nicely with the standard library
github.com/trustelem/zxcvbn 55 10 4---Go implementation of Dropbox's zxcvbn realistic password strength estimator
github.com/tstranex/u2f 290 33 6---Go FIDO U2F Library. This Go package implements the parts of the FIDO U2F specification required on the server side of an application.
github.com/vulcand/oxy 1826 261 60---Go middlewares for HTTP servers & proxies
----------------------
# velocidex
# as of 7/12/22
github.com/Depado/bfchroma 59 7 0---Integrating Chroma syntax highlighter as a Blackfriday renderer
github.com/Masterminds/sprig 3063 328 95---Useful template functions for Go templates.
github.com/Showmax/go-fqdn 32 12 3---Golang package providing access to Fully Qualified Domain Name of the machine.
github.com/ZachtimusPrime/Go-Splunk-HTTP 43 24 4---A simple and lightweight HTTP Splunk logging package for Go. Instantiates a client to your Splunk server and allows you to submit log events as desired.
github.com/alecthomas/assert 80 2 4---A simple assertion library using Go generics
github.com/alecthomas/chroma 3340 283 68---A general purpose syntax highlighter in pure Go
github.com/alecthomas/participle 2507 151 21---A parser library for Go
github.com/alexmullins/zip 182 113 7---Fork of Go's archive/zip to add reading/writing of password protected zip files.
github.com/araddon/dateparse 1736 135 51---GoLang Parse many date strings without knowing format in advance.
github.com/aws/aws-sdk-go 7714 1893 64---AWS SDK for the Go programming language.
github.com/clbanning/mxj 523 93 1---Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.
github.com/coreos/go-oidc 1228 337 17---A Go OpenID Connect client.
github.com/crewjam/saml 681 315 36---SAML library for go
github.com/davecgh/go-spew 5115 340 60---Implements a deep pretty printer for Go data structures to aid in debugging
github.com/dimchansky/utfbom 82 12 2---Detection of the BOM and removing as necessary
github.com/dustin/go-humanize 3233 210 37---Go Humans! (formatters for units to human friendly sizes)
github.com/elastic/go-libaudit 115 58 18---go-libaudit is a library for communicating with the Linux Audit Framework.
github.com/evanphx/json-patch 743 161 18---A Go library to apply RFC6902 patches and create and apply RFC7386 patches
github.com/go-ole/go-ole 882 161 62---win32 ole implementation for golang
github.com/go-sql-driver/mysql 12408 2130 100---Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package
github.com/golang-jwt/jwt 2915 183 44---Community maintained clone of https://github.com/dgrijalva/jwt-go
github.com/golang/mock 7628 542 59---GoMock is a mocking framework for the Go programming language.
github.com/golang/protobuf 8566 1550 82---Go support for Google's protocol buffers
github.com/google/btree 2836 353 8---BTree provides a simple, ordered, in-memory data structure for Go programs.
github.com/google/rpmpack 82 30 5---rpmpack (tar2rpm) - package rpms in pure golang or cli
github.com/google/shlex 132 39 3---Automatically exported from code.google.com/p/go-shlex
github.com/google/uuid 3720 314 25---Go package for UUIDs based on RFC 4122 and DCE 1.1: Authentication and Security Services.
github.com/gorilla/csrf 807 113 1---gorilla/csrf provides Cross Site Request Forgery (CSRF) prevention middleware for Go web applications & services 🔒
github.com/gorilla/schema 1095 189 8---Package gorilla/schema fills a struct with form values.
github.com/grpc-ecosystem/grpc-gateway 13654 1816 97---gRPC to JSON proxy generator following the gRPC HTTP spec
github.com/hillu/go-ntdll 39 8 0---Go interface to NTDLL functions
github.com/jmoiron/sqlx 12249 940 294---general purpose extensions to golang's database/sql
github.com/juju/ratelimit 2357 275 7---Efficient token-bucket-based rate limiter package.
github.com/lib/pq 7447 856 282---Pure Go Postgres driver for database/sql
github.com/magefile/mage 3058 206 84---a Make/rake-like dev tool using Go
github.com/mattn/go-isatty 626 90 9---
github.com/mattn/go-pointer 104 30 2---
github.com/mattn/go-sqlite3 5819 966 110---sqlite3 driver for go using database/sql
github.com/microcosm-cc/bluemonday 2371 153 14---bluemonday: a fast golang HTML sanitizer (inspired by the OWASP Java HTML Sanitizer) to scrub user generated content of XSS
github.com/mitchellh/panicwrap 387 58 13---panicwrap is a Go library for catching and handling panics in Go applications.
github.com/mxplusb/windows 5 0 3---
github.com/olekukonko/tablewriter 3460 303 80---ASCII table in golang
github.com/oschwald/maxminddb-golang 404 84 4---MaxMind DB Reader for Go
github.com/pkg/errors 7797 635 42---Simple error handling primitives
github.com/pkg/sftp 1169 336 27---SFTP support for the go.crypto/ssh package
github.com/prometheus/client_golang 3981 955 62---Prometheus instrumentation library for Go applications
github.com/prometheus/client_model 55 60 1---Data model artifacts for Prometheus.
github.com/qri-io/starlib 71 21 33---qri's standard library for starlark
github.com/rifflock/lfshook 367 80 5---A local file system hook for golang logrus logger
github.com/robertkrimen/otto 6747 555 132---A JavaScript interpreter in Go (golang)
github.com/russross/blackfriday 4960 588 203---Blackfriday: a markdown processor for Go
github.com/sebdah/goldie 174 13 7---Golden file testing for Go
github.com/sergi/go-diff 1337 168 29---Diff, match and patch text in Go
github.com/shirou/gopsutil 7938 1313 144---psutil for golang
github.com/sirupsen/logrus 20869 2147 73---Structured, pluggable logging for Go.
github.com/stretchr/testify 17015 1273 333---A toolkit with common assertions and mocks that plays nicely with the standard library
github.com/tink-ab/tempfile 2 2 0---A Golang tempfile generator that supports deterministic file name suffixes.
github.com/vjeantet/grok 251 77 5---simple library to use/parse grok patterns with go (100%)
github.com/xor-gate/debpkg 60 7 6---Package creation for Debian written in pure Golang
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment