-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Hi Karsten!
Oh sheesh. I did not see it...I will have to figure out why. That is slightly worrying.
So, I am rushing to meet the final deadline, but I still think it is doable. I have mostly finished up my OONI work for the month, and I planned to spend the remainder of this month working on the bridge test.
I have finished most of the actual Tor connection code, as well as one of the four basic packet level scans (the icmp8 one). Two of the other packet level scans, the TCP SYN and ACK ones, are pretty much copies of the icmp8 one with a couple lines changed, and they shouldn't be a big deal.
There is still the vanilla TLS handshake test/scan/thing, which has not been started yet, and will take a bit more time than the others because Python notoriously has problems with SSL bindings and libraries, so I'll need to do a bit of research on newer ones and updates and see which is the best to use now. I hear that tlslite[1] is the current best choice; if anyone else has any input on this it would be very helpful. :)
There were a couple minor hups:
1) When George asked me to test pluggable transports, this required significantly more refactoring than I previously thought was necessary.
2) Arturo redesigned the OONI testing framework API again to use a completely different structure, which was supposed to be backwards compatible and turned out not to be (though I believe that my recent OONI commits fixed that). However, I have been fighting the framework already, because the main scripts in OONI (/ooni/oonicli.py and /ooni/ooniprobe.py) control the reactor, and also expect static iterations through single test and single control functions for each asset (an asset in this case would be one bridge address). The bridge testing is rather dynamic (I would like it to be able to evaluate an approximate danger level to running the next test) and so the framework is kind of troublesome. Also, because the framework handles calling the reactor (in Twisted, the reactor is a sort of event scheduler), and it also expects a rather linear progression of defer.Deferreds (in Twisted, those are standin objects which execute callbacks when they get results from some previous deferred/callback), it would be nicer if I had full control of these myself without needing to hack around the parent scripts. I think it's wise that OONI deals with these things for the testwriter in most cases, because the testwriter shouldn't be expected to be an expert in using Twisted. However, I also think that, in the long term, OONI shouldn't prohibit people who know what they are doing or are doing odd things from being able to do so. As a result, I've decided (for now), to use bits are parts of the OONI code before the recent refactoring, and later (after the deliverable) I will work on adding flags to OONI to give the test script full control of the reactor and deferreds, as well as evaluating whether or not the bridge test is even compatible with the new API. I do not want to get caught up in dealing with this right now, I just want to have it all working and deployable in a way that I know will work.
3) The indirect scans are becoming quite complicated to automate in any sane fashion. I still would like to continue working on this, as I'm quite enjoying the difficulty, but due to their temporary and volatile nature (they will change frequently depending on the blocking methods of a particular country and the currently available in-country bounces/proxies/whatever-thing-the-indirect-scan-uses), as well as the fact that many of these methods are still undiscovered, I think it is safe to add them as specialty cases after the fact without impacting overall general testing. There is one in particular that I would like to finish before the deadline because I am quite proud of it and am having a lot of fun working on it, but I'm first going to concentrate on wrapping up the active scans.
There are other things which I've marked as helpful things to do, but which are not necessarily part of this deliverable:
1) Having a parser for bridge descriptors to turn them into test inputs, and vice versa.
2) Having some undiscoverable method for setting up lots of IPv6 bridges on one OR (Tor currently only allows up to eight, I believe) and having these be discoverable by bridgedb and no one else. I was thinking of this while talking with Aaron, because he reminded me that people on IPv6 have tons of IPs available, and I was thinking that if we configured some type of one-way hash function, we could say that a bridge descriptor for 2001:db8::1:1 should actually mean multiple descriptors for 2001:db8::fa98:38d2 2001:db8::e099:2188 2001:db8::88aa:3b7 or something, derived from the output of hashing the original descriptor with the OR's key or something else. This would help distribute bridges in the future quite a bit, though it doesn't do much for the current bridge situation.
Anyone wanting to help with the above two things, or with an idea for another indirect scan, or with feedback on anything I'm working on, should feel free to contact me and it will be greatly appreciated. :D
.-- .. ... .... - -- . .-.. ..- -.-. -.- <(A)3 isis agora lovecruft .-.-.
[1] https://github.com/trevp/tlslite
On Fri 12 Oct 2012 at 14:56, thus spake Karsten Loesing:
Hi Isis,
did you see this email?
Sorry for being pushy, but we're less than three weeks away from the deadline, which makes me kinda nervous.
Thanks, Karsten
On 10/8/12 9:14 AM, Karsten Loesing wrote:
Hi Isis,
can you give me a brief status update on the sponsor F year 2 deliverable "Automating Bridge Reachability Testing", ticket #6414?
In particular, how did the two project-internal milestones on September 15 and October 7 go?
Do you mind updating #6414 with the status?
The overall deadline is on November 1, which is in 3.5 weeks. How realistic do you think it is that we'll have automated bridge reachability tests by then?
Thanks, Karsten