IPv4-wide Scans @ SHA2017
We're setting up infrastructure to utilize the available bandwidth at the SHA2017 hacker camp in the Netherlands. Participation is more than welcome! This doesn't mean you have to be at the camp physically -- you may suggest scanning proposals/ideas (see below) & do your research remotely via ssh(1)
or have someone implement and run your idea at the camp. There will be a workshop on internet-wide scanning - we'll present our results and teach willing participants what we know, have learned scanning the internet in the past, from others and during the camp. We will also let the participants run their own scans -- if we feel they're valuable, ethical and non-disruptive.
Full attribution (professional, academic & otherwise) will go to the original authors of ideas and implementers of scans run during SHA2017 and developed at the mentioned workshop! We should not even have to mention this fact - as this should be the norm (hey there, academia!).
bear in mind: we're only doing ethical research. we won't actively exploit vulnerabilities, so some intrusive scans may be rejected if there's even a remote chance that they interfere with others infrastructure or services! all ideas and code will be peer reviewed by at least one person and undergo a Q/A process on site (as best as we can manage with available human heartbeat-cycles -> which means that ideas may be set aside because we didn't have enough time or people to look at them and provide feedback - unfortunately - we've only got a few days).
proposals/submissions: send to shascan@azet.org
and what your goals/ideas are (+ code, if you already know how to).
participation: fork & comment on this gist. write code, search for papers, previous research and send proposals as described above. help with the Q/A process and reviewing code. help setting up the infrastructure on site, administer it, and monitor (if possible 24/7). help out during the workshop on the last evening of SHA (beer, etc. welcome) with your newly aquired knowledge -- so people can learn (faster & more in-depth) from you!
@a_z_e_t (Twitter) || azet@azet.org
(SMTP) || azet@jabber.ccc.de
(XMPP) || azet
@ freenode, OFTC (IRC)
this contact information is for emergencies and technical questions aside from scanning submissions (see above).
INFRASTRUCTURE STATUS:
currently there's a single 1u server that will be used
server info and build-up status @ https://gist.github.com/azet/54862407b7af1c2813e590ead83f7553#gistcomment-2161084
-
(pre) orga on-site @ SHA:
1) get in contact with server transport + NOC/Nick Farr 2) add info. on the scans to the SHA2017 Wiki 3) physically install server @ NOC- get second 10GE NIC from Aaaaaaaaaaaa Village & add to server
- connect 10GE LACP trunks to NOC router and uplink (optional + OOB mgmt?)
-
set up server:
1) freebsd/linux? 2) drivers? 3) disks/raid/filesystem performance/kernel/network tuning 4) backups 5) monitoring- virtualization/jails/docker/LXC/..
- all the userspace stuff & scanning itself
- do we want/need OOB management? (e.g. forkbomb, heavy network IO interferes with SSH/monitoring/scans/etc.)
-
find someone to bring:
-
8-16GB RAM
2) 2nd 10GE (dual port /w SFP+ tranciever) NIC -
1-2 USB3 SSDs (fast & big)
4) fiber optics (i.e. cabling and trancievers)
-
-
benchmarks:
- disk IO
- Network IO
- NUMA + memory specifics
- in-memory FS?
-
abuse (mails/management) and how to deal with it
- do we want to and if: who answers all those mails?
- who is PoC there - scan team or SHA NOC?
- set up website on public v6/v4 IP and reverse DNS entry with disclaimer
- that this is a scan server doing good/ethical stuff
- who to contact in case of an emergency / network issues / ethical questions
- provide information on ongoing scans and research, network utilization and AS numbers / IPs scanned
- important (XXX/TODO): who sets that up and codes all that?
- OCSP
- uPNP
- FTP
(azet
has more ;))
- https://recdnsfp.github.io/
- https://arxiv.org/pdf/1612.02636v1.pdf
- https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_svenda.pdf
- https://factorable.net/weakkeys12.extended.pdf
- masscan
- zmap, "ztools"
decide (FreeBSD 11 / Debian testing):
- LXC/docker (have participants submit dockerfiles?)
- KVM/qemu
- bhyve
- jails
- Scripts: e.g. Python -> AMQP (?) -> fanout LXC/docker, ..
- http://pcp.io/
- dtrace (in case of FreeBSD - who writes probes in D?)
- do we want kernel tracing? if so - https://lttng.org/
What do we want and need? Public system logs? (automated) Anonymization of datasets?
reproducible results, science, and builds (if possible and doable in a timely fashion)
Here's a small template of what your submission should include (example being a project I proposed):
Subject:
Internet-wide deployment of RFC 864-based services (chargen)
Abstract:
The chargen service is an optional service for both TCP and UDP that
generates a stream of response data for any incoming communication on
the service's port. Hosts choosing to implement such a service provide a
simple aid in debugging throughput or other networking issues. In May
1983 ARPA proposed a standard for such character stream generating
services, which became adopted as RFC 864 and were assigned the TCP and
UDP port numbers 19 by IANA.
This proposal tries to examine the number of such hosts on the internet,
differences in implementations in-the-wild and if significant
differences in QoS can be detected.
Motivation:
Hasn't been done before AFAIK.
Keywords:
RFC864, chargen, QoS
Involved Parties:
Benny "BenBE" Baumann (unaffiliated)
Involved Funding Grants:
none
Prior Research:
None known
Related Research:
Various other scans of privileged ports at e.g. scans.io
Relevant Documentation:
RFC 864, Character Generator Protocol, J. Postel
Data to be collected
TCP scan on port 19
Collect first 64KiB per successful connection,
measuring speed/packet loss of connection
Result: IP,RTT,Speed,Data Stream
UDP scan on port 19
Collect responses and timing for up to 256
datagrams sending minimal request datagrams.
Result: IP,Packet #,RTT,Response Data
Additionally relevant information (e.g. for correlation):
Host Fingerprints (e.g. Host OS, TCP Stack)
Geolocation (Spatial grouping of found hosts)
Host Up/Down Status Information
Possible analysis:
Prior Experience with such scans:
none, heard presentations about them
Required support:
Will need help with implementation and deployment of scan.
Depending on number of found hosts also with analysis.
Security Considerations:
none
Remarks:
Looking forward to SHA2017.