Hey everyone,
Although it hasn't been obvious, I actually have been making some serious progress on my Onion Name System (OnioNS) project. The paper was accepted into PoPETS, the design is finally stable, and the software is looking really promising, although not yet ready for use. I've focused on improving the maturity of the software and set up the following:
* PGP-signed commits using my torproject identity * Continuous integration via Travis CI * Use clang-analyzer and cppcheck static analysis scanners * Automated enforcement of code styling via clang-format * Inclusion of a manpage and command-line help menu * Packaging for Debian (Wheezy+) on my own machine * Automated building for Ubuntu (14.04+) on Launchpad * Protection against memory leaks and buffer overflows * Software is split to minimize installation footprint * Use of CMake, which should help the port to Windows * No RPATH hacks * Support for release and debug builds * Support for Clang and GCC; all warnings enabled for debug * No reliance on OpenSSL (CVE-2016-6309, are you serious?!) * Use of well-maintained libraries for networking and parsing * Minimal dependencies: Botan, JsonCpp, libcurl, libmicrohttpd * Non-native dependencies included as git submodules
At this point, I am interested in taking the next step and try to integrate with the Tor Project infrastructure. I am looking for information on the following:
1) How do I upload packages to deb.torproject.org? 2) How do I join Gitian to generate reproducible builds? 3) How do I set up an Onion Name System component in Trac so that I can migrate bug tracking from Github to Trac? 4) How do I set up a repository on gitweb.torproject.org? I'm more comfortable using Github, but there's no need to centralize git.
I have a LDAP account, which should help with some of these. I wasn't able to find much useful documentation on these topics, so I would appreciate some help. Also, if anyone has any experience with RPM packaging, I would like to figure that out. I haven't had too much luck with Alien.
As I approach a proper release, my overall goal is to improve the maturity, trust, usability of the OnioNS software. Please let me know how I can accomplish the above tasks.
Recent commits: https://github.com/Jesse-V/OnioNS-common/tree/json-rpc https://github.com/Jesse-V/OnioNS-server/tree/json-rpc https://github.com/Jesse-V/OnioNS-client/tree/json-rpc
I've also updated https://trac.torproject.org/projects/tor/wiki/doc/OnionServiceNamingSystems
Jesse V kernelcorn@torproject.org writes:
[ text/plain ] Hey everyone,
Although it hasn't been obvious, I actually have been making some serious progress on my Onion Name System (OnioNS) project. The paper was accepted into PoPETS, the design is finally stable, and the software is looking really promising, although not yet ready for use. I've focused on improving the maturity of the software and set up the following:
- PGP-signed commits using my torproject identity
- Continuous integration via Travis CI
- Use clang-analyzer and cppcheck static analysis scanners
- Automated enforcement of code styling via clang-format
- Inclusion of a manpage and command-line help menu
- Packaging for Debian (Wheezy+) on my own machine
- Automated building for Ubuntu (14.04+) on Launchpad
- Protection against memory leaks and buffer overflows
- Software is split to minimize installation footprint
- Use of CMake, which should help the port to Windows
- No RPATH hacks
- Support for release and debug builds
- Support for Clang and GCC; all warnings enabled for debug
- No reliance on OpenSSL (CVE-2016-6309, are you serious?!)
- Use of well-maintained libraries for networking and parsing
- Minimal dependencies: Botan, JsonCpp, libcurl, libmicrohttpd
- Non-native dependencies included as git submodules
At this point, I am interested in taking the next step and try to integrate with the Tor Project infrastructure. I am looking for information on the following:
- How do I upload packages to deb.torproject.org?
- How do I join Gitian to generate reproducible builds?
- How do I set up an Onion Name System component in Trac so that I can
migrate bug tracking from Github to Trac? 4) How do I set up a repository on gitweb.torproject.org? I'm more comfortable using Github, but there's no need to centralize git.
I have a LDAP account, which should help with some of these. I wasn't able to find much useful documentation on these topics, so I would appreciate some help. Also, if anyone has any experience with RPM packaging, I would like to figure that out. I haven't had too much luck with Alien.
As I approach a proper release, my overall goal is to improve the maturity, trust, usability of the OnioNS software. Please let me know how I can accomplish the above tasks.
Hello Jesse,
glad to hear you are still working on the OnioNS project, and happy to hear that the paper got accepted in PoPETS. We have great plans for hidden service naming layers, and it's great to see more people working on this topic. Thanks for updating the wiki as well.
FWIW, I feel that using Github (or any other third-party git) for code hosting and ticket tracking is fine for now. If you still want a personal repository on gitweb, check out these docs: https://help.torproject.org/tsa/
I'm also not sure what's the exact policy on gitian and deb.torproject.org, and whether third-party applications (like OnioNS) can be placed there.
Personally, I'm hesitant of giving the impression that Tor officially supports any of the proposed naming systems so early. I'd feel more confident about them after they get deployed and used by the wider community.
On this topic, what's your deployment strategy? If you really really need Tor project infrastructure to achieve it, let's talk about it.
BTW, monitor this list, there will soon be a proposal specifying a Tor API for naming systems like OnioNS, similar to the one we have for pluggable transports.
Cheers!
On 10/04/2016 03:54 PM, George Kadianakis wrote:
Hello Jesse,
glad to hear you are still working on the OnioNS project, and happy to hear that the paper got accepted in PoPETS. We have great plans for hidden service naming layers, and it's great to see more people working on this topic. Thanks for updating the wiki as well.
My pleasure. I'm looking forward to see what you guys have in store.
FWIW, I feel that using Github (or any other third-party git) for code hosting and ticket tracking is fine for now. If you still want a personal repository on gitweb, check out these docs: https://help.torproject.org/tsa/
Thanks, I'll look into it.
I'm also not sure what's the exact policy on gitian and deb.torproject.org, and whether third-party applications (like OnioNS) can be placed there.
That's fine. It's not a big deal, but I would eventually like to have reproducible builds on the official system. For now I have a few ideas for reproducibility, so it's not a priority at present.
Personally, I'm hesitant of giving the impression that Tor officially supports any of the proposed naming systems so early. I'd feel more confident about them after they get deployed and used by the wider community.
I can understand that. How did other software, such as obfs4, get supported? I'm sure an early version of it was buggy and vulnerable, but at some point it was integrated in to the system. I'm just wondering what the requirements are and what the timeline looks like. I was hoping to get started with the process.
On this topic, what's your deployment strategy? If you really really need Tor project infrastructure to achieve it, let's talk about it.
I don't need Tor's build system, no. I can probably deploy just fine on third-party systems (Github, Launchpad, etc) but it seems very disjointed. I was hoping to get started on the process in moving towards *.torproject.org so that it's easier to deploy. Everyone is already familiar with the process and they have deb.torproject.org in their sources.
At present, I am envisioning a deployment strategy like this: 1) I will finish all the primary components and ensure that they all mesh together nicely. This involves a lot of testing and changes on my end. This is currently where I am. 2) I will deploy an alpha build to the approximately dozen volunteers that have expressed interest so far. I will distribute RPMs, DEBs, and a build of the Tor Browser that includes my libraries. I will help run the servers and they can test out the components, particularly the onion service and client software. 3) Fix bugs and deploy into beta. 4) I'll post on tor-dev or tor-relays asking for volunteers to help test the server end. This is part where the system starts to become decentralized. 5) Fix bugs, grow based on interest, add more documentation, expand the project page, etc. 6) When everything seems stable and feature-complete, announce it so that everyone can test out the stable builds.
BTW, monitor this list, there will soon be a proposal specifying a Tor API for naming systems like OnioNS, similar to the one we have for pluggable transports.
That will be neat. At present the "intercept circuit requests, auto-attach or rewrite based on TLD" approach seems pretty reliable, but I will look forward to seeing what you guys come up with.
Hi,
On Tue, Oct 04, 2016 at 07:11:47PM -0400, Jesse V wrote:
I'm also not sure what's the exact policy on gitian and deb.torproject.org, and whether third-party applications (like OnioNS) can be placed there.
That's fine. It's not a big deal, but I would eventually like to have reproducible builds on the official system. For now I have a few ideas for reproducibility, so it's not a priority at present.
If the software is stable, and reproducibly built, I would aim for placing it into the Debian archives. I maintain ooniprobe on deb.torproject.org, but primarily for Ubuntu users and I would prefer that Debian users are using the version from the Debian archive.
One issue we run into is that in trying to support older distributions, we often then have to take on maintaining backports for various packages and in some cases this is a lot of overhead and careful version number management.
If you wanted to set up your own Debian repository, that would also be an alternative solution for the short-term.
- I will finish all the primary components and ensure that they all
mesh together nicely. This involves a lot of testing and changes on my end. This is currently where I am. 2) I will deploy an alpha build to the approximately dozen volunteers that have expressed interest so far. I will distribute RPMs, DEBs, and a build of the Tor Browser that includes my libraries. I will help run the servers and they can test out the components, particularly the onion service and client software. 3) Fix bugs and deploy into beta. 4) I'll post on tor-dev or tor-relays asking for volunteers to help test the server end. This is part where the system starts to become decentralized. 5) Fix bugs, grow based on interest, add more documentation, expand the project page, etc. 6) When everything seems stable and feature-complete, announce it so that everyone can test out the stable builds.
I cannot see that any of this actually requires the use of Tor Project infrastructure. If you really cannot set up a Debian repository for distributing pre-release packages, I can provide you with a simple repository set up to use.
Thanks, Iain.
On 11/14/2016 07:58 AM, Iain R. Learmonth wrote:
If the software is stable, and reproducibly built, I would aim for placing it into the Debian archives. I maintain ooniprobe on deb.torproject.org, but primarily for Ubuntu users and I would prefer that Debian users are using the version from the Debian archive.
One issue we run into is that in trying to support older distributions, we often then have to take on maintaining backports for various packages and in some cases this is a lot of overhead and careful version number management.
If you wanted to set up your own Debian repository, that would also be an alternative solution for the short-term.
This is a really good point, thank you for your suggestion. I will change my approach and set up a Debian repository that Debian and Ubuntu users can use. I looked also into adding into Debian. How do you suggest that I get started with this? It would be great if I could get it into Debian 8 if possible.
-
George Kadianakis -
Iain R. Learmonth -
Jesse V