IPv4-wide Scans @ SHA2017

SHA2017-SCAN.md

IPv4-wide Scans @ SHA2017

What? -- Outline & General Information

We're setting up infrastructure to utilize the available bandwidth at the SHA2017 hacker camp in the Netherlands. Participation is more than welcome! This doesn't mean you have to be at the camp physically -- you may suggest scanning proposals/ideas (see below) & do your research remotely via ssh(1) or have someone implement and run your idea at the camp. There will be a workshop on internet-wide scanning - we'll present our results and teach willing participants what we know, have learned scanning the internet in the past, from others and during the camp. We will also let the participants run their own scans -- if we feel they're valuable, ethical and non-disruptive.

Full attribution (professional, academic & otherwise) will go to the original authors of ideas and implementers of scans run during SHA2017 and developed at the mentioned workshop! We should not even have to mention this fact - as this should be the norm (hey there, academia!).

---

bear in mind: we're only doing ethical research. we won't actively exploit vulnerabilities, so some intrusive scans may be rejected if there's even a remote chance that they interfere with others infrastructure or services! all ideas and code will be peer reviewed by at least one person and undergo a Q/A process on site (as best as we can manage with available human heartbeat-cycles -> which means that ideas may be set aside because we didn't have enough time or people to look at them and provide feedback - unfortunately - we've only got a few days).

proposals/submissions: send to shascan@azet.org and what your goals/ideas are (+ code, if you already know how to).

template for submissions

participation: fork & comment on this gist. write code, search for papers, previous research and send proposals as described above. help with the Q/A process and reviewing code. help setting up the infrastructure on site, administer it, and monitor (if possible 24/7). help out during the workshop on the last evening of SHA (beer, etc. welcome) with your newly aquired knowledge -- so people can learn (faster & more in-depth) from you!

Point of Contact -- Main Workshop & Infrastructure Organization

@a_z_e_t (Twitter) || azet@azet.org (SMTP) || azet@jabber.ccc.de (XMPP) || azet @ freenode, OFTC (IRC)

this contact information is for emergencies and technical questions aside from scanning submissions (see above).

---

TODO / wanted

INFRASTRUCTURE STATUS:
currently there's a single 1u server that will be used
server info and build-up status @ https://gist.github.com/azet/54862407b7af1c2813e590ead83f7553#gistcomment-2161084

• (pre) orga on-site @ SHA: 1) get in contact with server transport + NOC/Nick Farr 2) add info. on the scans to the SHA2017 Wiki 3) physically install server @ NOC

  4. get second 10GE NIC from Aaaaaaaaaaaa Village & add to server
  5. connect 10GE LACP trunks to NOC router and uplink (optional + OOB mgmt?)

• set up server: 1) freebsd/linux? 2) drivers? 3) disks/raid/filesystem performance/kernel/network tuning 4) backups 5) monitoring

  6. virtualization/jails/docker/LXC/..
  7. all the userspace stuff & scanning itself
  8. do we want/need OOB management? (e.g. forkbomb, heavy network IO interferes with SSH/monitoring/scans/etc.)

• find someone to bring:

  1. 8-16GB RAM 2) 2nd 10GE (dual port /w SFP+ tranciever) NIC

  2. 1-2 USB3 SSDs (fast & big) 4) fiber optics (i.e. cabling and trancievers)

• benchmarks:

  1. disk IO
  2. Network IO
  3. NUMA + memory specifics
  4. in-memory FS?

• abuse (mails/management) and how to deal with it

  1. do we want to and if: who answers all those mails?
  2. who is PoC there - scan team or SHA NOC?
  3. set up website on public v6/v4 IP and reverse DNS entry with disclaimer
     • that this is a scan server doing good/ethical stuff
     • who to contact in case of an emergency / network issues / ethical questions
     • provide information on ongoing scans and research, network utilization and AS numbers / IPs scanned
       • important (XXX/TODO): who sets that up and codes all that?

Ideas

• OCSP
• uPNP
• FTP

(azet has more ;))

Link- & reading list

• https://recdnsfp.github.io/
• https://arxiv.org/pdf/1612.02636v1.pdf
• https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_svenda.pdf
• https://factorable.net/weakkeys12.extended.pdf

Kernel & Userspace

Scanning

v4-wide

• masscan
• zmap, "ztools"

general

• https://github.com/emmericp/MoonGen (see also: http://dpdk.org)
• https://github.com/Igalia/pflua

Custom runs by participants

decide (FreeBSD 11 / Debian testing):

• LXC/docker (have participants submit dockerfiles?)
• KVM/qemu
• bhyve
• jails

Scripts / Ports / Data / Analytics

• Scripts: e.g. Python -> AMQP (?) -> fanout LXC/docker, ..

Correlation & anomaly detection

• https://github.com/etsy/skyline
• https://github.com/etsy/oculus

System metrics / performance profiling & monitoring

• http://pcp.io/
• dtrace (in case of FreeBSD - who writes probes in D?)
• do we want kernel tracing? if so - https://lttng.org/

Accounting (oh.. we could make ethics & science reproducible?!)

What do we want and need? Public system logs? (automated) Anonymization of datasets?

reproducible results, science, and builds (if possible and doable in a timely fashion)

How to scan once you've got access to the scanning infrastructure:

azet@scan-sha2017:/data/scans$ cat README_zmap 
# Scanning with `zmap`:

1) create a new scan directory:
   - `cd /data/scans`
   - `sudo ./new_scan NAMEOFSCAN` e.g. `sudo ./new_scan https_banners_2`
   - `cd` into the newly created project directory
 
2) run `zmap` as follows to scan for specific ports or the entire internet:
   - sudo /usr/local/sbin/zmap -i enp1s0f0 -o scan_result.csv -b /data/scans/blacklist.txt -v5 -T4 -B10G -p443` 
     (to scan the entire IPv4 rage on port 443 [https])
   - to grab banners you'll need to combine with `zgrab` and `ztools`:
      - https://github.com/zmap/zgrab
      - https://github.com/zmap/zgrab2
      - https://github.com/zmap/zdns
        (more at https://github.com/zmap)
   - take a look at other `zmap` options, too!


azet

azet@scan-sha2017:/data/scans$ cat README_masscan 
# Scanning with `masscan`:

1) create a new scan directory:
   - `cd /data/scans`
   - `sudo ./new_scan NAMEOFSCAN` e.g. `sudo ./new_scan https_banners_2`
   - `cd` into the newly created project directory
 
2) run `masscan` as follows to scan for specific ports or the entire internet:
   - `sudo masscan -c ../masscan_ip4-wide_all_ports.conf -p443`
     (to scan the entire IPv4 rage on port 443 [https])
   - to grab banners add `--banners`
   - take a look at other `masscan` options, too!


azet

If there are any questions, please send a mail to shascan@azet.org. In case of emergencies get in contact with the information provided over there: https://wiki.sha2017.org/index.php?title=Internet-wide_Scanning#Emergencies

Keeping motd up to date:

Please fill in any changes to the infrastructure, system users, tools and scans you're doing!

currently /etc/motd looks like this:

==============================================
   scan-sha2017.external.meta.southbound.io
==============================================

  * Emergency contact:
   - +43 664 6394757 (Phone, Signal, WhatsApp)
   - azet@azet.org (SMTP)
   - azet@jabber.ccc.de (Jabber/XMPP)
   - @a_z_e_t (Twitter)

  * Status:
   - scanning machine set up & operational 
   - >> IN PRODUCTION <<
   - local users allowed to scan (+ sudo):
      - "azet", "hanno", "fr333k".
   - `/data/scans/` contains all scanning info. & data 
      - take a look at `README_masscan` and `README_zmap`!
      - blacklist: `/data/scans/blacklist.txt` (CIDR notation)

  * Ongoing scans:
   - !DONE! ~~no-limit on 0.0.0.0/0 - all ports [azet]~~
   - https/http/ftp scans (with & without banners/data)


-- EOF (azet)

Public "ntopng" Traffic Dashboard:

http://151.216.93.11:8080

Credentials

user: public
password: guest

@BenBE:

your chargen scan request has just started collecting data.

azet@scan-sha2017:/data/scans/chargen-p19-auth:BenBE_started-Sat,_05_Aug_2017_20:50:45_+0000$ sudo masscan -c ../masscan_ip4-wide_all_ports.conf -p19 --banners
/data/scans/blacklist.txt: excluding 1 ranges from file

Starting masscan 1.0.4 (http://bit.ly/14GZzcT) at 2017-08-05 20:52:15 GMT
 -- forced options: -sS -Pn -n --randomize-hosts -v --send-eth
Initiating SYN Stealth Scan
Scanning 4294967295 hosts [1 port/host]
rate:1184.81-kpps,  0.76% done,   1:08:46 remaining, found=5224