Hi,
so you were right the databases were corrupt, but they shouldn't have been there in the first place. :-)
I didn't want to include large files in the git repo (~120mb in total), so there is a generation script in the tools directory (now added).
I added some instructions on the readme file too. Of course, this procedure is done only once.
I tried it and it should work now, but let me know if there is still a problem.
There is a plan to optimize the way the pre-computed values are used, and this could potentially shrink the size of the databases by a lot.
Vasilis
On 17 October 2015 at 15:56, str4d str4d@i2pmail.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
str4d wrote:
Vasilios Mavroudis wrote:
Hello,
I would like to introduce our project "Crux", which enables the computation of privacy preserving statistics on sensitive data. The project was developed at University College London (UCL) by me, in close collaboration with George Danezis.
This is fantastic work. Haven't read all of your thesis yet, but it looks similar to the stats collection system that we ran for a while inside I2P, but with the privacy protections that I wanted it to have. Well done!
"Crux" was designed with Tor hidden services in mind (in accordance with this <https://research.torproject.org/techreports/hidden-service-stats-201
5
- -04-28.pdf>
Tor
Tech Report), but it can be applied on various kinds of data (given an appropriate parser).
Excellent. I want to use this in I2P :)
It supports three statistics (i.e. mean, median, variance) and its threat model is compatible Tor's (see technical document below).
I will look into this, but I doubt there will be any insurmountable incompatibilities with I2P's threat model.
Currently, we don't have a parser specific for the data collected by the Tor relays, but we plan to keep adding functionality.
The source code can be found here: https://github.com/mavroudisv/Crux The theoretical background, threat model, security argument and technical details can be found here: link http://mavroudisv.eu/files/thesis.pdf
Ideas, comments, feedback much appreciated!
Firstly: needs more docs. I shouldn't have to read your entire thesis in order to figure out how to run it :P The README is a good start, but the command parameters need clarification/definitions, and it is not at all obvious how the relays work. An XLS file is also mentioned without explanation.
I have now tried running the code, but the "unit test" run by authority.py fails because bsbdb.btopen() returns an empty dict. I don't know if this is because the table files are broken or because of a local issue with my setup (I am using a virtualenv, but I get the same empty dict when I try to open the tables manually outside the virtualenv). I am not going to try and fix the latter on my own, because bsddb is deprecated since Python 2.6, so you shouldn't really be using it :) (But it must be something else, because I tried using bsddb3 instead and that also returns an empty dict).
str4d
Secondly, you could probably simplify configuration by managing hidden service connections yourself. I believe the Stem library can do this? I2P has a python library that can be used to listen on and connect to I2P Destinations, with a socket-like interface.
It's not clear to me from your thesis whether you plan to merge the relay component into the Tor relay code (and have the Tor relays talk directly to the query processors), or have your relays running separately and interfacing with the Tor relays to fetch stats. This will potentially alter how it would be used with other networks. In I2P's case I'd imagine it would be easiest (in terms of usability by those setting up relays) to write an I2P plugin in Java that performs the relay role, but I wouldn't want to be duplicating a lot of logic that subsequently needs to be kept in-sync with upstream :)
str4d
Vasilis
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
-----BEGIN PGP SIGNATURE-----
iQIcBAEBCgAGBQJWItIgAAoJEBO17ljAn7PglsQP/3tOrL9EZfYMb5fxxVI6/wJm qDvA8RpbEJDhoMjOc9sLNyMVgvGGPxEdma8I/R8H37yIPOp2GJg82MoXfUk+AGFe 5ZIIXwf2EzHH0RFLRI5xaQ4DW3Ru1yIzH+E3vhuGclL87YRKnAM0xkMQcueR5xo5 /ETe9TI7oPCdETI4J6iFoM+Ob9qsRiEVeuT5HEFFKwQhBvl0NSpfMHV9ftuxQHuu I2B9jomnBdzF9OXnSQrYXg3ho2rxX4hO1TZfQHg5DgHvkt8TQYQM3cwdGE7/rsqC d4+kBW2RuAz68wq+fLPeeZ2U2ff2QqmDLqVyUJvpP5RqeUcv66J/iYm7md886tTc palwVGodKiTJ6DBM0z1K97x6Sl1jJXEAkZQAr2yQ8jrC/BJqQRPu2vHeZILqbMEQ UU3JLjIk/Jc2RGK6zitvj4kTrLdfACEQMYy3brQUawZhaM42QNQgb3LQq1bJXzUo vISZckX/CVfF+R63oO085LYOO7oTLyY3EhA1PCXiV09Usl/To66Oh4AVYtz/lPLH 6uoWR/ic5X9gFIXt1t+617+i/L/ILMfqkqN7GzoSbQ8QfyBJrjIikuj575M7AH3o RuhRYHUTmaCRL+9niDFG/mSVGBxBqiSfW1eS5pGE3hIAcLQd0eFT/38HRdonkfLP Ug5ihMFSKfKmCIljQ6mm =jDmS -----END PGP SIGNATURE-----
-
Vasilios Mavroudis