Hi Asis, glad you want to improve our testing of tor! That's one area that could certainly use some love. ;)

I would like to improve on Tor's codebase by writing Integ tests for it in Stem. I would like to know what kind of tests should be included in this. Also, atagar was talking about smoke tests for Tor as a whole. I would love to know everyone's ideas on the same.

We have two python frameworks for testing tor, chutney and stem.

The aim of chutney is to spawn a dedicated tor network for testing, so we can make assertions based on the behaviour of multiple components. For instance, 'if we make a one-hop circuit through an exit it should be rejected'.

This is the proper solution for tor integration testing. The only trouble is that doing this properly takes some work and chutney hasn't been seeing much (any?) activity for quite some time.

The second test framework is stem. Its integration tests spawn a *single* tor instance and exercise that, presently checking controller functionality we can test while offline. Stem's integration tests are a lot more mature than chutney, but since it spawns only a single tor instance it's rather limited in the tor functionality it can test.

In the short term we can get a lot of mileage by expanding stem's integration tests to exercise basic tor functionality. However, we'll quickly hit a wall in terms of what we can test with stem. As such it might be best for a GSoC project to start with a few stem integ tests to boost its test coverage of tor, then switch over to improving chutney.

One idea is...

1. Integrate gcov into stem's integration testing output.

2. Add some tests under the ONLINE target that cover basic functionality...

* Construct a circuit to an exit which allows port 80 and request a site through it. * Do the same for an exit which rejects port 80. * Construct a one-hop circuit through an exit. * Try to use a circuit that runs through a bridge. * Let tor pick the path of a circuit and assert that the first relay's a guard and the last is an exit. * Attempt to request a page from a hidden service. * Host a hidden service and request something from ourselves. * Act as a relay and try to construct a circuit that includes ourselves. * ... etc...

I suspect those last two are no-gos because we would need to be included within the consensus, but worth looking into.

Something to keep in mind is that the tor network is highly unreliable. Tests should retry through alternate relays when they fail to be sure that the failure is with our client rather than individual relays.

3. Once we have a decently reliable set of tests to check basic tor functionality move on to chutney. Nick would be able to best advise on the work needed there.

Also, I would like to start a new project which can automatically download & compile latest Tor. This would greatly help with the Integ tests for Stem & Tor.

Oops! My bad, I resolved the ticket for this but didn't update the volunteer page (now fixed). We already have a jenkins instance that does this...

https://jenkins.torproject.org/job/stem-tor-ci/

Cheers! -Damian