Tor ships with some testing facilities (tor/src/test/test), but they are not comprehensive. (I'm aware there's an ongoing effort on improving these tests).
To test anything non trivial it is necessary to run a tor node as a part of a tor network. There is another tool called chutney [1] that can be used to set up a small testing Tor network: it generates plenty of torrc configs and spawns a number of Tor servers locally.
This is very handy, but it can take a while before the network starts working - the Auth servers need to establish the consensus and do all the things I have no clue about.
Here's a simple shell script [2]: - It uses chutney to start a testing tor network. - It waits for it to work by trying to establish a connection to localhost:22. - Finally it tears down the network.
Normally it takes around 5 minutes for the network to converge:
$ time ./go.sh real 4m8.330s user 0m2.064s sys 0m0.392s
Of course, you don't have to set a completely fresh Tor network for every single test, but that's what I often do. I'd be eager to hear how people are reusing chutney networks.
In past I wrote this thing called fluxcapacitor [3], it's a tool that speeds up tests. After a few fixes I was able to run chutney on it:
$ time /tmp/fluxcapacitor/fluxcapacitor ./go.sh
real 0m11.450s user 0m2.340s sys 0m2.120s
Running stuff under fluxcapacitor is not deterministic, sometimes it takes 8 seconds, sometimes 15 but it generally works and should go pretty quick.
That's it. I thought someone may find it useful.
Cheers, Marek
[1] https://gitweb.torproject.org/chutney.git [2] https://github.com/majek/dump/blob/master/tor/go.sh [3] https://github.com/majek/fluxcapacitor.git
On Sun, Sep 8, 2013 at 12:35 PM, Marek Majkowski marek@popcount.org wrote:
Tor ships with some testing facilities (tor/src/test/test), but they are not comprehensive. (I'm aware there's an ongoing effort on improving these tests).
To test anything non trivial it is necessary to run a tor node as a part of a tor network. There is another tool called chutney [1] that can be used to set up a small testing Tor network: it generates plenty of torrc configs and spawns a number of Tor servers locally.
This is very handy, but it can take a while before the network starts working - the Auth servers need to establish the consensus and do all the things I have no clue about.
Here's a simple shell script [2]:
- It uses chutney to start a testing tor network.
- It waits for it to work by trying to establish a connection to localhost:22.
- Finally it tears down the network.
Normally it takes around 5 minutes for the network to converge:
$ time ./go.sh real 4m8.330s user 0m2.064s sys 0m0.392s
Of course, you don't have to set a completely fresh Tor network for every single test, but that's what I often do. I'd be eager to hear how people are reusing chutney networks.
In past I wrote this thing called fluxcapacitor [3], it's a tool that speeds up tests. After a few fixes I was able to run chutney on it:
$ time /tmp/fluxcapacitor/fluxcapacitor ./go.sh
real 0m11.450s user 0m2.340s sys 0m2.120s
Running stuff under fluxcapacitor is not deterministic, sometimes it takes 8 seconds, sometimes 15 but it generally works and should go pretty quick.
That's it. I thought someone may find it useful.
Cheers, Marek
Thanks, Marek! This looks quite useful. Can you talk a bit more about under what circumstances (if any) the fluxcapacitor code there might give incorrect results?
Also, I had thought the quick-and-dirty chutney test we shipped (via ./src/test/test-network.sh) bootstrapped in faster than 5 minutes. Am I wrong there? Does it use any tricks that might be helpful in combination with fluxcapacitor?
best wishes,
On Sun, Sep 8, 2013 at 10:12 PM, Nick Mathewson nickm@alum.mit.edu wrote:
$ time /tmp/fluxcapacitor/fluxcapacitor ./go.sh
real 0m11.450s user 0m2.340s sys 0m2.120s
Running stuff under fluxcapacitor is not deterministic, sometimes it takes 8 seconds, sometimes 15 but it generally works and should go pretty quick.
Thanks, Marek! This looks quite useful. Can you talk a bit more about under what circumstances (if any) the fluxcapacitor code there might give incorrect results?
Depends how you define "incorrect results". In a single-process scenario fluxcapacitor, of course, will always work correctly and predictably. When running in multi-process setup (as with Chutney) the "results" are somewhat dependent on the kernel scheduling order.
In the worst case fluxcapacitor might "speed up" a wrong process and cause a timeout to fire when it shouldn't. This is caused by kernel giving CPU to FC rather than a child process that has more immediate work to do. This is generally a hard problem on busy systems and I'm still trying to find a right balance between FC speed and stability. In normal circumstances this problem shouldn't show up, it occurs only when the host is very busy and there is a heavy deficit of cpu power.
In theory the running time of a script under FC should be a constant - it should be purely CPU bound. In reality it varies due to indeterminism of kernel scheduling (and _plenty_ of context switches and page misses).
Fortunately, as I have shown in the example, it's possible to run exactly the same testing script with and without FC. If there is a suspicion FC is malfunctioning, you can always run the tests without it.
Also, I had thought the quick-and-dirty chutney test we shipped (via ./src/test/test-network.sh) bootstrapped in faster than 5 minutes. Am I wrong there? Does it use any tricks that might be helpful in combination with fluxcapacitor?
I was not aware of this script. I'll take a look.
Marek
On Mon, Sep 9, 2013 at 12:02 AM, Marek Majkowski marek@popcount.org wrote:
I was not aware of this script. I'll take a look.
$ cat chutney/tools/bootstrap-network.sh [...] offset=$(expr ( $(date +%s) + $VOTING_OFFSET ) % 300) CONFOPT="TestingV3AuthVotingStartOffset" for file in net/nodes/*a/torrc; do sed -i.bak -e "s/^${CONFOPT}.*$/${CONFOPT} $offset/1" $file done [...]
That's just cheating! It may work, but it's pretty ugly :)
Without fluxcapacitor, as you suggested bootstrapping takes around 24 seconds, with FC ~10 sec.
$ export CHUTNEY_PATH=$HOME/tor/chutney # Stop network after the script is done $ echo './chutney stop $CHUTNEY_NETWORK' >> src/test/test-network.sh # add tor-gencert to the PATH $ export PATH=$PATH:$PWD/src/tools
$ time ./src/test/test-network.sh real 0m23.592s user 0m2.108s sys 0m0.452s
$ time ~/fluxcapacitor/fluxcapacitor ./src/test/test-network.sh real 0m10.164s user 0m2.216s sys 0m1.676s
With options that make tor less noisy it's even faster: ~6.5 seconds:
$ echo -e "TokenBucketRefillInterval 1 second\nServerDNSDetectHijacking 0\n" >> $CHUTNEY_PATH/torrc_templates/common.i
$ time ~/fluxcapacitor/fluxcapacitor ./src/test/test-network.sh real 0m6.489s user 0m2.464s sys 0m1.296s
Fluxcapacitor does make it go faster, but the test (script) must be self-contained, ie: it must tear down the network after it's done.
Marek
Marek Majkowski marek@popcount.org wrote Sun, 8 Sep 2013 17:35:24 +0100:
| In past I wrote this thing called fluxcapacitor [3], it's a tool that | speeds up tests. After a few fixes I was able to run chutney on it: | | $ time /tmp/fluxcapacitor/fluxcapacitor ./go.sh | | real 0m11.450s | user 0m2.340s | sys 0m2.120s | | Running stuff under fluxcapacitor is not deterministic, sometimes it | takes 8 seconds, sometimes 15 but it generally works and should go | pretty quick.
Interesting approach, thanks for sharing.
Did you try running make test-network under fluxcapacitor?
It uses the bootstrap-network.sh in Chutney and should be able to bootstrap a network in under 30s, without help.
-
Linus Nordberg -
Marek Majkowski -
Nick Mathewson