how to scan the tor network for partitions

detect_tor_network_partitions

how to scan the tor network for partitions
------------------------------------------

1. setup a machine running Tor and expose its control port as either a tcp port or unix domain socket
with no authentication

   *edit* /etc/tor/torrc
   blah blah easy rtfm

2. install tor_partition_scanner

   virtualenv virtenv-orscanner
   . ./virtenv-orscanner/bin/activate
   mkdir -p code; cd code
   git clone https://github.com/david415/tor_partition_scanner.git
   cd tor_partition_scanner
   pip install -e .
   
3. get a recent consensus file

   Use consensus files from collector if you want others to be able to reproduce your results.
   here --> https://collector.torproject.org/recent/relay-descriptors/consensuses/
   
   wget https://collector.torproject.org/recent/relay-descriptors/consensuses/2017-09-21-23-00-00-consensus
   
4. choose which relays you want in your scan

   Here I am intentionally NOT scanning 50 million tor circuits using the entire consensus.
   Instead I am using a simple python program written using the Stem library to parse the consensus file
   and give us all the realys with the Stable and Fast flags; among those we choose the top 100 in terms of
   consensus bandwidth.
   
   ./helpers/query_fingerprints_from_consensus_file.py 2017-09-21-23-00-00-consensus > top100.relays
   
5. perform scan of top 100 relays

   detect_partitions.py --tor-control tcp:127.0.0.1:9051 --log-dir ./ --status-log ./status_log \
   --relay-list top100.relays --secret secretTorEmpireOfRelays --partitions 1 --this-partition 0 \
   --build-duration .25 --circuit-timeout 60 --log-chunk-size 1000 --max-concurrency 100
   
   9,900 two hop tor circuits are being built.
   As the scan runs you can tail -f the status_log to make sure its working.
   Only circuit build failures if any will be logged in the json log file.

   When the scan completes the status_log should display something like this:
   
   2017-09-22T00:05:44+0000 [-] $BD4C647508162F59CB44E4DFC1C2B2B8A9387CCA -> $DD808ECE4F2E24F377CBE11E335ECDA196FE3B78
   2017-09-22T00:05:44+0000 [-] $0966A24977A0B0DB62546C6F18F9578D97FE86F0 -> $AD00FB62A133F91009AD5F6503E5F21F594BC4C6
   2017-09-22T00:05:50+0000 [orscanner#info] Finished writing measurement values to ./2017-09-22T00:05:50.492698-scan.json.
   2017-09-22T00:05:50+0000 [-] Main loop terminated.

6. Load circuit build failures into sqlite db file

   ./bin/load.py --dbfile scan1.db -p 2017-09-22T00:03:31.610096-scan.json \
   -p 2017-09-22T00:05:42.886622-scan.json \
   -p 2017-09-22T00:05:50.492698-scan.json

7. Count the results

   echo "select first_hop, second_hop from scan_log;" | sqlite3 scan1.db  | wc -l
   2014

8. Attempt to eliminate false positives by retesting the failed circuits

   mkdir scan1
   mv *.json scan1
   echo "select first_hop, second_hop from scan_log;" | sqlite3 scan1.db > scan2.circuits

   detect_partitions.py --tor-control tcp:127.0.0.1:9051 --log-dir ./ --status-log ./status_log \
   --relay-list relays_for_scan1 --build-duration .25 --circuit-timeout 60 --log-chunk-size 1000 \
   --max-concurrency 100 --circuit-file scan2.circuits

   ./bin/load.py --dbfile scan2.db -p 2017-09-22T00:59:31.017246-scan.json -p 2017-09-22T01:04:35.491908-scan.json

   echo "select first_hop, second_hop from scan_log;" | sqlite3 scan2.db | wc -l
   1947
   
   still 1947 circuit build failures!