On Dec 18, 2012, at 8:07 PM, Philipp Winter identity.function@gmail.com wrote:
Hi there,
Deliverable 6 for sponsor Z says:
- Start a tool that a censored developer can run to discover why their Tor is
failing to connect: brainstorm a list of "things to check", and sort them by how useful they'd be to check / how hard they'd be to build. (#7137)
The deliverable is due on Feb. 28, 2013 so we should get started.
Some background about the deliverable: The reason for this project is that debugging possible censorship events is tedious right now. We often have no access to machines in censoring countries and we are dependent on users creating packet dumps for us. This tool should speed up and automate this process to some extent. Censored users should run it and the tool should then collect data which should then somehow reach us.
I created the following wiki page which should contain all the necessary information: https://censorshipwiki.torproject.org/TorCensorshipAnalyzer
Please add/modify stuff and share your opinion. Since there is quite some overlap with OONI, it would be great if the OONI people could give feedback.
I believe you should be using ooniprobe to build a the tests you are interested in building, or you may at least be interested in looking at our code to see how to do the things you are interested in doing.
The main points where ooniprobe would be of use to you (now) are:
# Standard reporting format
All ooniprobe tests share a common base format depending on the test template your test is based on.
I recommend you look at the Test Writing tutorial to get an idea of how this looks like: https://ooni.torproject.org/docs/writing_tests.html
# Collection of packet captures
When you run an ooniprobe test and you have set your ooniprobe.conf file to "includepcap: true" then you will collect a full pcap of what has happened on the probes network during the test run.
Note: This requires the test to be run as root and will include *all* the network traffic during the testing session (i.e. if the user is looking at their favorite kitten website while running the test, such data will be in the pcap)
# Collection of packet captures specific to the sent and received packets
When you run a ooniprobe test that inherits from the scapy test template (https://ooni.torproject.org/docs/api/ooni.templates.html#module-ooni.templat...) the packets sent and received (i.e. that are answers to the packet(s) sent) will be captured.
When configured to not include the probe IP address, source IP of sent packets and dst IP of received packets is replaced with 127.0.0.1. (warning: if the IP address of the probe is present in some other parts of the packet it will not get stripped, for example if it's present in the ICMP citation)
# Reporting system
Currently we only support collection of YAML formatted reports (that means not .pcap files) and only via Tor Hidden Services.
Extending it to support reporting via HTTP(s) should be trivial and is a feature that we have already received a request for.
Adding support for collecting also .pcaps also probably does not require that much amount of time and is something that will happen in the near future.
# Things to come
ooniprobe will soon expose a HTTP based API that binds to localhost that can then be (optionally) exposed as a Tor Hidden Service. Such API will allow researchers to connect to a probe and run some tests and will allow us to build a JS/HTML5 client interface to allow users to select which tests to run and monitor the status of running tests.
More details here: https://ooni.torproject.org/docs/architecture.html#ooniprobe-api
For a birds-eye view of the project see: https://ooni.torproject.org/docs/architecture.html
Even if you do not end up using ooniprobe for developing your system today, I highly encourage you to use the libraries that we are using so that in the future we can find a way to integrate code from each others projects.
The main libraries that we are using are:
* Twisted http://twistedmatrix.com * Scapy http://www.secdev.org/projects/scapy/ * txtorcon https://github.com/meejah/txtorcon
~ Art.