tor-dev February 2020

tor-dev@lists.torproject.org

16 participants
12 discussions

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

tor relay process health data for operators (controlport)
by nusenu 16 Nov '20

16 Nov '20

Hi, every now and then I'm in contact with relay operators about the "health" of their relays. Following these 1:1 discussions and the discussion on tor-relays@ I'd like to rise two issues with you (the developers) with the goal to help improve relay operations and end user experience in the long term: 1) DNS (exits only) 2) tor relay health data 1) DNS ------ Current situation: Arthur Edelstein provides public measurements to tor exit relay operators via his page at: https://arthuredelstein.net/exits/ This page is updated once daily. the process to use that data looks like this: - first they watch Arthur's measurement results - if their failure rate is non-zero they try to tweak/improve/change their setup - wait for another 24 hours (next measurement) This is a somewhat suboptimal and slow feedback loop and is probably also less accurate and less valuable data when compared to the data the tor process can provide. Suggestion for improvement: Exposes the following DNS status information via tor's controlport to help debug and detect DNS issues on exit relays: (total numbers since startup) - amount of DNS queries send to the resolver - amount of DNS queries send to the resolver due to a RESOLVE request - DNS queries send to resolver due to a reverse RESOLVE request - amount of queries that did not result in any answer from the resolver - breakdown of number of responses by response code (RCODE) https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… - max amount of DNS queries send per curcuit If this causes a significant performance impact this feature should be disabled by default. 2) general relay health metrics -------------------------------- Compared to other server daemons (webserver, DNS server, ..) tor provides little data for operators to detect operational issues and anomalies. I'd suggest to provide the following stats via the control port: (most of them are already written to logfiles by default but not accessible via the controlport as far as I've seen) - total amount of memory used by the tor process - amount of currently open circuits - circuit handshake stats (TAP / NTor) DoS mitigation stats - amount of circuits killed with too many cells - amount of circuits rejected - marked addresses - amount of connections closed - amount of single hop clients refused - amount of closed/failed circuits broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1402 https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1994 - amount of closed/failed OR connections broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2205 If this causes a significant performance impact this feature should be disabled by default. cell stats - extra info cell stats as defined in: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1072 This data should be useful to answer the following questions: - High level questions: Is the tor relay healthy? - is it hitting any resource limits? - is the tor process under unusual load? - why is tor using more memory? - is it slower than usual at handling circuits? - can the DNS resolver handle the amount of DNS queries tor is sending it? This data could help prevent errors from occurring or provide additional data when trying to narrow down issues. When it comes to the question: **Is it "safe" to make this data accessible via the controlport?** I assume it is safe for all information that current versions of tor writes to logfiles or even publishes as part of its extra info descriptor. Should tor provide this or similar data I'm planing to write scripts for operators to make use of that data (for example a munin plugin that connects to tor's controlport). I'm happy to help write updates for control-spec should these features seem reasonable to you. Looking forward to hearing your feedback. nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

5 7

DNS-over-HTTPS (DOH) in Firefox/Torbrowser
by nusenu 01 Jul '20

01 Jul '20

Hi, since Mozilla did tests [0] on DOH [1] in Firefox I was wondering if Torbrowser developers have put any thought into that as well? Note: I'm _not_ suggesting to use DOH in torbrowser I'm just asking because the answer probably matters for exit documentation in the relay guide if clients do DNS themselves over TCP connections instead of relying on the exit (even if torbrowser is not the only tor client). thanks, nusenu [0] https://www.ghacks.net/2018/03/20/firefox-dns-over-https-and-a-worrying-shi… [1] https://datatracker.ietf.org/doc/draft-hoffman-dns-over-https/ -- https://mastodon.social/@nusenu twitter: @nusenu_

4 4

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Request for onionbalance v3 pre-alpha testing
by George Kadianakis 03 Mar '20

03 Mar '20

Hello list, we've been developing Onionbalance v3 for the past months, and I'm pretty hyped to say that the project has reached a stability point that could benefit from some initial testing by curious and adventurous developers and users. The project is not yet ready for proper use in actual production environments (more on this later), but my hope is that this testing will reveal bugs that I can't catch in my test environment and receive user feedback that will speed up the overall process and allow a faster and more stable initial release. This email features a pretty complicated pseudo-guide that if it gets followed to completion it will hopefully result in a functional onionbalance setup. As I said the guide is complicated and in ugly text format, so I suggest you put on some calming music (perhaps some Bill Evans or some Com Truise or some Tor) and go slowly. Trying to speed run this will result in disappointment and perhaps even partial or total loss of faith in humanity. == So what's the current status of Onionbalance v3? == The project lives as a fork of Donncha's original Onionbalance project in this repository: https://github.com/asn-d6/onionbalance The current status is that during my testing I have onionbalanced a real v3 service with two backend instances successfully for a few days with no obvious reachability issues. It also works on a private tor network (using Chutney) for hours with no issues. All the above happened on Debian stable and testing. Also, here are some features that are currently missing but will be included in the initial launch: - This new onionbalance will support both v2 and v3 onion services, altho currently v2 support is botched and only v3 will work for the purposes of this testing. If you can quickly revive v2 support patches are welcome! - Any sort of documentation about v3 mode (this post is all the documentation there is for now!) - Missing streamlined setup process (installation scripts, packages or dependency tracking) - Code documentation and unittests missing - There is no daemon mode. You will have to run this in screen for now - Fun bugs included! Furthermore, v2 and v3 support will not have feature parity at the time of the initial launch, and we will have to incrementally build these features: - Cool v2 features like DISTINCT_DESCRIPTORS are not yet implemented so v3 cannot load balance as deeply as v3 yet (patches welcome!) - There is no way to transfer your existing v3 onion service to onionbalance (patches welcome!) - There is no torrc generation for instances as in v2 (patches welcome!) - It will be possible for clients and HSDirs to figure out whether a descriptor comes from an onionservice vs a regular v3 onion. Making them indistinguishable is possible but requires more engineering and will be left as a TODO for now (see #31857) Finally, there is no guarantee that configs generated with this pre-alpha version of Onionbalance will be forward compatible when we actually launch the project. This means that if you setup onionbalance v3 right now, you might need to re-set it up in a month or so when the actual release happens. tl;dr This is a call for experimental testing and it's meant to help the developers and accelerate development and not to be used in anything remotely serious or close to production. The actual release for operators and users is coming in a month or so. == Before we get started... == OK so let's assume that you still want to help out! Given that this is a pre-alpha test I'm gonna assume that you fill the following prerequisites otherwise you are gonna have an adventure: [ ] Comfortable with git [ ] Familiar with v2 onionbalance (at least in terms of terminology and how it works) [ ] Familiar with setting up and configuring (v3) onion services [ ] Familiar with Linux (or in the mood to find weird bugs in other platforms) [ ] Familiar with compiling C projects and chasing their dependencies [ ] Familiar with running Python projects and chasing their dependencies [ ] Patience to push through various import error messages that might appear The above are not mandatory but if you don't check out all the boxes you might encounter various errors and roadblocks that will frustrate. Actually... even if you check out all these boxes you might encounter various errors that might frustrate you. == Some theory == In any case, if you want to learn more about the architecture of Onionbalance, see: https://blog.torproject.org/cooking-onions-finding-onionbalance As part of this guide, you might hear me say the term "frontend" which is what we call master onionbalance onion service instance (whose address clients type on their browser), or the term "backend" which is the load balancing instances that do the actual introduction/rendezvous work for the frontend. So for example, in the visual below, Alice fetches the frontend's descriptor but does the introduction/rendezvous with the backends: ================================================================================ + + + Alice fetches frontend +^^^^^^^^^^^^^^^^^^^^^^^^^^^^+ + + +~~~~~~~~~~+ descriptor +~~~~~~+ HSDir hash ring ) + + | Alice +~~~~~~~~~~~~~~~~~~~+ +vvvvvvvvvvvv+vvvvvvvvvvvvvvv+ + + | | | + + |the client| +-----------+ | + + +~~~+~~~~~~+ |backend | | Upload frontend + + | |instance 1 +-------+ | descriptor + + | +-----------+ | | + + | | +-------+-----------+ + + | +-----------+ +----+ | + + +~~~~~~~~~~~~~+backend | | frontend | + + Alice does |instance 2 +------------+ onionbalance | + + introduction +-----------+ +----+ service | + + with backend | +-------------------+ + + +-----------+ | + + |backend +-------+ + + |instance 3 | + + +-----------+ + + + ================================================================================ OK enough with theory. Let's jump to practice: == Installation instructions == For the purposes of this guide I will assume you have a frontend service and a bunch of backend instances that you want to load balance. ==== Setup frontend service ==== Our first goal is to get onionbalance and Tor running on your frontend service. - Installing Tor Let's start: SSH to your frontend service, and get yourself a Tor binary to use as the control port for onionbalance. You will want a recent version of Tor (above 0.4.2) but other than that there is no strict requirement. Feel free to get it off your package manager or git. Setup a minimal torrc with a control port enabled. As an example: """ SocksPort 0 ControlPort 127.0.0.1:6666 DataDirectory /home/user/frontend_data/ """ Feel free to tweak your torrc as you feel (also enable logging), but for the purposes of this guide I assume that your control port is at 127.0.0.1:6666. - Running onionbalance for the first time Now we need to run onionbalance and generate keys and a config file for it. Here the fun starts: Let's start by git cloning 'https://github.com/asn-d6/onionbalance'. To run onionbalance v3 you are gonna need a bunch of fun Python dependencies including (but perhaps not limited to): - Python3 above 3.7 - Python3 packages for: pyyaml, pycrypto, setproctitle, future - stem version 1.8 or up (so that it includes v3 support) Please collect all the above whichever way you prefer: if you don't have them, things are gonna crash at some point. There is no dependency tracking for v3 onionbalance right now (i.e. requirements.txt) so I don't enforce the above in an elegant way. Now that you are good to go with dependencies, feel free to install onionbalance using the setup.py script, or you can run it from the source directory if you prefer that. Now is the time to create keys and a config file for your frontend service. Let's do that by running the configuration wizard: $ python3 onionbalance-config.py It should ask you if you want v3 or v2 (say v3!), and a bunch of other questions and by the end you should have a config file somewhere (by default at ./config/master/config.yaml) and the corresponding frontend key file (again at ./config/master/*key). The onion address of your frontend service can be found in the bottom of your config file. So if it says: key: wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.key your onion address is "wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion" For now, note down the onion address of the frontend service and let's move on to the next step! Make sure you got a v3 onion address out of this process, otherwise something went wrong. If you got to this stage without any major turbulance, you will end up with a functional onionbalance in no time. If you had lots of problems so far, brew up some green tea, because there is more fun coming! ==== Setup backend instances ==== OK now with the frontend onion address noted down, let's move to setting up your backend instances: SSH to one of your backend instances and let's setup Tor in there! The catch is that onionbalance v3 backend instances need a very recent version of Tor because of some issues in the v3 protocol (#32709). As a matter of fact, the patch required has not yet been merged to upstream Tor (it will be merged in mid February because we are currently at a code freeze) so you will need to use my personal repository for now (as indicated by the last comment in #32709). So let's git clone 'https://github.com/asn-d6/tor.git' and checkout branch 'ticket32709_044_02'. Build that branch (using the classic './autogen.sh && ./configure && make' combo) and you should have a Tor binary at ./src/app/tor. If this doesn't happen, you are gonna need to install various C dependencies like libssl-dev, libevent-dev, etc. Now you will need a torrc file for your backend instance. As in onionbalance v2, the torrc file needs to setup an onion service (and in this case a v3 one). So far so good but here comes the twist: 1) In the end of your torrc file, inside the HiddenService block please add the following line: 'HiddenServiceOnionBalanceInstance 1'. 2) Now go in your hidden service directory where the hostname and hs_ed25519_public_key files are living (you will need to startup Tor once with the onion service enabled for the directory to get created) you need to create a new file with the name 'ob_config' that has the following line inside: 'MasterOnionAddress wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion' but substitute the onion address with your frontend's onion address. The points (1) and (2) above are extremely important and if you didn't do them correctly, nothing is gonna work. If you want to ensure that you did things correctly, start up Tor, and check that your *info* log file includes the following line: [info] ob_option_parse(): OnionBalance: MasterOnionAddress wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion registered If you don't see that, then something went wrong. Please try again from the beginning of this section till you make it! This is the hardest part of the guide too, so if you can do that you can do anything (fwiw, we are at 75% of the whole procedure right now). After you get that, also make sure that your instances are directly reachable (e.g. using Tor browser). If they are not reachable, then onionbalance won't be able to see them either and things are not gonna work. OK, you are done! Now do the same for the other backend instances and note down the onion addresses of your backend instances because we are gonna need them for the next and final step. ==== Get it running! ==== OK now let's login back to the frontend server! Go to your onionbalance config and add your instance addresses in the right fields. In the end it should look like this (for a setup with 4 backend instances): """ services: - instances: - address: 4a2zwxetkje5lwvahfv75arzrctn6bznkwmosvfyobmyv2fc3idbpwyd.onion name: node1 - address: i5aqar7fgqcsqwalwjltrb3w3xyj23ppgsnuzhhkzlhbt5337aw2joad.onion name: node2 - address: uutpxl2cfkk5qa7z4xtgdxssnhutmoo2y2rw6igh5ez4hpxaz4dap7ad.onion name: node3 - address: 5xeigil3hjxltcs3ti4skerhpc252ci7ejynrtm73v3bcbxlfuulsjqd.onion name: node4 key: wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.key """ Now let's fire up onionbalance by running the following command: $ python3 onionbalance.py -v info -c config/master/config.yaml -p 6666 If everything went right, onionbalance should start running and after a bit your frontend service should be reachable!!! You are an onionbalance expert! If something did not go right, that's OK too, don't get sad because this was quite complicated. Please check all your logs and make sure you did everything right according to this guide. Keep on hammering at it and you are gonna get it. If nothing seems to work, please get in touch with some details and I can try to help you. == Now what? == Now that you managed to make it work, please monitor your frontend service and make sure that it's reachable all the time. Check your logs for any errors or bugs and let me know if you see any. If you want you can make onionbalance logging calmer by using the '-v warning' switch. (Also if you see this log message in your frontend Tor logs: "Bug: Unrecognized signal 132..." that's normal and it will be fixed with #33104). If you are in the mood for coding feel free to start hacking on onionbalance, but I would suggest you first get in touch over Github because I'm also working at it at the same time and it would be a shame if we do duplicate work. If you find bugs or do any quick bugfixes, please submit them over Github! If you do code review, please let me know through Github of any issues you find. Also, if you are gonna be at FOSDEM this upcoming weekend feel free to get in touch if you have any questions or you want to live test it. If you have any other feedback, I'd be glad to hear it! == Next steps == I'm gonna be working on this for the next month: fixing bugs, writing documentation, writing tests, creating packages, submitting and fixing bug reports in tor and stem etc. Expect more updates on this mailing list, and also a blog post to announce the initial launch of the project. Finally, if there are any errors on this post, I will reply to this thread with an errata! Hope this works for you and you have fun onionbalancing your services! Thanks a lot for the testing and looking forward to your feedback! o/ PS: Major thanks to Donncha for the super clean onionbalance codebase (I did not expect to reuse so much code), the Tor network team for the support, and Damian for all the work on stem v3 support! Also OTF for the $$$ ;)

1 2

interested in working on a ticket
by indigo omega 24 Feb '20

24 Feb '20

Hello, I am a new contributor interested in working on ticket 13184: https://trac.torproject.org/projects/tor/ticket/13184 . The ticket says that it was assigned 8 months ago, but then deleted? I am having trouble understanding if this ticket is under current development or not. Should I start working on it and submit it for review, or should I email someone, perhaps a maintainer, to see if someone is actively working on it. Any advice would be much appreciated. Thank you, Amir Ghorbanian

3 2

tor sending requests with a token
by Pouya Miralaee 21 Feb '20

21 Feb '20

Hello to the community and good evening! I have some websites that i want to hide from public eyes, without going to implement them as a HS. So here is my question: can i implement a certificate authority on my servers, and then instruct tor to use that certificate for requesting the websites? or something like a token generated on the servers and tor use that token as an authenticated client? The big picture of my scenario is that i have a custom tor browser (some browser extensions pre-installed by default) and i want my clients using that particular browser only access the mentioned websites! Cheers! -- *Pouya Miralayi*

1 0

TOR Browser Circuit Selection of Exit Nodes
by Vianney Gomezgil Yaspik 19 Feb '20

19 Feb '20

Good evening, A group of students at Johns Hopkins University and I have been analyzing the circuit selection algorithm for TOR´s browser EXIT nodes. It is an exploratory project trying to discover how do the EXIT nodes are selected every time the circuit changes. So far, we have discovered that the time that the browser is accessed, calendar date, physical location, use of a bridge (or not), and the entry node do not change the pattern of the EXIT nodes. Moreover, than approximately 80% of all exit nodes come from 10 specific countries, even though these 10 countries only account for approximately 50% of all available exit nodes. All of which has led us to conclude that the selection of the EXIT nodes in the TOR browser is not random. We would like to further explore however, what are the factors that determine which relay becomes the exit node every time a circuit is changed. Is there anyone that we could speak to or that could give us further insight as to how the selection of the exit node in TOR´s circuit works? v/r

3 2

Making Tor's CI Faster
by teor 13 Feb '20

13 Feb '20

Hi, Want to help make Tor's CI go faster? As part of the Sponsor 55 IPv6 work, I need to run Chutney IPv6 tests on every Tor commit, as part of Tor's CI. We're currently running these tests with fast_finish, because: * Travis CI only has IPv6 on macOS * macOS Chutney tests take 20-45 minutes in Travis CI But fast_finish requires allow_failure. We can't have allow_failure on code that's we're modifying every day, so I'm going to make the Travis IPv6 Chutney job mandatory. To speed up CI after this change, I'm going to make a new IPv6-only test-network target, and run it in the macOS CI: https://trac.torproject.org/projects/tor/ticket/33280 I've also identified one redundant Travis CI job that we can delete: https://trac.torproject.org/projects/tor/ticket/33195#comment:3 Overall, these changes should reduce total CI runtime by 20-35 minutes. Due to parallelism, the wall clock time should reduce by 10-35 minutes. We can squeeze the most benefit from the parallelism by sorting jobs in speed order (longest first): https://trac.torproject.org/projects/tor/ticket/33194 If you have any other ideas for making Tor's CI go faster, please let me know. Slow CI is a particular issue for backports, because each 0.3.5 backport updates 9 branches: * {maint,release}-{0.3.5,0.4.1,0.4.2,0.4.3} and * master. T -- teor ----------------------------------------------------------------------

2 1

Proposal 313: Relay IPv6 Statistics
by teor 10 Feb '20

10 Feb '20

Hi, Here is an initial draft of Proposal 313: Relay IPv6 Statistics. This proposal includes: * logging the number of IPv6 relays in the consensus, and * relays publishing IPv6 connection and consumed bandwidth statistics. This is the third of 3 proposals: * Proposal 311: Relay IPv6 Reachability * Proposal 312: Automatic Relay IPv6 Addresses * Proposal 313: Relay IPv6 Statistics I revised proposals 311 and 312 last week, and merged them to torspec as drafts. There are still some TODO items in the proposal, about: * safely collecting these new statistics on bridges, and * getting accurate IPv6 connection statistics. If you know about tor's statistics, please give us some feedback! The full text is included below, and it is also available as a GitHub pull request: https://github.com/torproject/torspec/pull/108 The related tickets are #33159 (proposal) and #33051 and #33052 (implementation): https://trac.torproject.org/projects/tor/ticket/33159 https://trac.torproject.org/projects/tor/ticket/33051 https://trac.torproject.org/projects/tor/ticket/33052 Please feel free to reply on this list, or via GitHub pull request comments. Filename: 313-relay-ipv6-stats.txt Title: Relay IPv6 Statistics Author: teor Created: 10-February-2020 Status: Draft Ticket: #33159 0. Abstract We propose that Tor relays (and bridges) should log the number of relays in the consensus that support IPv6 extends, and IPv6 client connections. We also propose that Tor relays (and bridges) should collect statistics on IPv6 connections and consumed bandwidth. Like tor's existing connection and consumed bandwidth statistics, these new IPv6 statistics will be published in each relay's extra-info descriptor. 1. Introduction Tor relays (and bridges) can accept IPv6 client connections via their ORPort. But current versions of tor need to have an explicitly configured IPv6 address (see [Proposal 312: Relay Auto IPv6 Address]), and they don't perform IPv6 reachability self-checks (see [Proposal 311: Relay IPv6 Reachability]). As we implement these new IPv6 features in tor, we want to monitor their impact on the IPv6 connections and bandwidth in the tor network. Tor developers also need to know how many relays support these new IPv6 features, so they can test tor's IPv6 reachability checks. (In particular, see section 4.3.1 in [Proposal 311: Relay IPv6 Reachability]: Refusing to Publish the Descriptor.) 2. Scope This proposal modifies Tor's behaviour as follows: Relays, bridges, and directory authorities log the number of relays that support IPv6 clients, and IPv6 relay reachability checks. They also log the corresponding consensus weight fractions. As an optional change, tor clients may also log this information. Relays, bridges, and directory authorities collect statistics on: * IPv6 connections, and * IPv6 consumed bandwidth. The design of these statistics will be based on tor's existing connection and consumed bandwidth statistics. Tor's existing consumed bandwidth statistics truncate their totals to the nearest kilobyte. The existing connection statistics do not perform any binning. We do not proposed to add any extra noise or binning to these statistics. Instead, we expect to leave these changes until we have a consistent privacy-preserving statistics framwework for tor. As an example of this kind of framework, see [Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]. We avoid: * splitting connection statistics into clients and relays, and * collecting circuit statistics. These statistics are more sensitive, so we want to implement privacy-preserving statistics, before we consider adding them. Throughout this proposal, "relays" includes directory authorities, except where they are specifically excluded. "relays" does not include bridges, except where they are specifically included. (The first mention of "relays" in each section should specifically exclude or include these other roles.) Tor clients do not collect any statistics for public reporting. Therefore, clients are out of scope in this proposal. (Except for some optional changes to client logs, where they log the number of IPv6 relays in the consensus). When this proposal describes Tor's current behaviour, it covers all supported Tor versions (0.3.5.7 to 0.4.2.5), as of January 2020, except where another version is specifically mentioned. 3. Logging IPv6 Relays in the Consensus We propose that relays (and bridges) log: * the number of relays, and * the consensus weight fraction of relays, in the consensus that: * have an IPv6 ORPort, * support IPv6 reachability checks, and * support IPv6 clients. In order to test these changes, and provide easy access to these statistics, we propose implementing a script that: * downloads a consensus, and * calculates and reports these statistics. As well as the statistics listed above, this script should also report the following relay statistic: * support IPv6 reachability checks and IPv6 clients. The following consensus weight fractions should divide by the total consensus weight: * have an IPv6 ORPort (all relays have an IPv4 ORPort), and * support IPv6 reachability checks (all relays support IPv4 reachability). The following consensus weight fractions should divide by the "usable Guard" consensus weight: * support IPv6 clients, and * support IPv6 reachability checks and IPv6 clients. "Usable Guards" have the Guard flag, but do not have the Exit flag. If the Guard also has the BadExit flag, the Exit flag should be ignored. We propose that these logs happen whenever tor: * receives a consensus from a directory server, or * loads a live, valid, cached consensus from disk. As an optional change, tor clients may also log this information. Some of this information is not directly relevant to clients, but these logs may help developers (and users). 4. Collecting IPv6 Consumed Bandwidth Statistics We propose that relays (and bridges) collect IPv6 consumed bandwidth statistics. To minimise development and testing effort, we propose re-using the existing "bw_array" code in rephist.c. In particular, tor currently counts these bandwidth statistics: * read, * write, * dir_read, and * dir_write. We propose adding the following bandwidth statistics: * ipv6_read, and * ipv6_write. (The IPv4 statistics can be calculated by subtracting the IPv6 statistics from the existing total consumed bandwidth statistics.) We propose adding a new BandwidthStatistics torrc option and consensus parameter, which activates reporting of all these statistics. Currently, the existing statistics are controlled by ExtraInfoStatistics, but we propose using the new BandwidthStatistics option for them as well. The default value of this option should be "auto", which checks the consensus parameter. If there is no consensus parameter, the default should be 1. (The existing bandwidth statistics are reported by default.) TODO: Should we collect IPv6 bandwidth statistics on bridges? On bridges, should bandwidth statistics be on or off by default? If we do want to collect bridge statistics, and we think it's safe, modify proposals 311 and 312 to allow bridge statistics. 5. Collecting IPv6 Connection Statistics We propose that relays (and bridges) collect IPv6 connection statistics. To minimise development and testing effort, we propose re-using the existing "bidi" code in rephist.c. (This code may require some refactoring, because the "bidi" totals are globals, rather than a struct.) In particular, tor currently counts these connection statistics: * below threshold, * mostly read, * mostly written, and * both read and written. We propose adding IPv6 variants of all these statistics. (The IPv4 statistics can be calculated by subtracting the IPv6 statistics from the existing total connection statistics.) We propose using the existing ConnDirectionStatistics torrc option, and adding a consensus parameter with the same name. This option will control the new and existing connection statistics. The default value of this option should be "auto", which checks the consensus parameter. If there is no consensus parameter, the default should be 0. (The existing connection direction statistics are reported by default.) TODO: Do enough relays report ConnDirectionStatistics, for accurate IPv6 connection statistics? * at least 25% of relays have IPv6 * at the end of the project, we expect at least 33% of relays to have deployed tor 0.4.4-stable If not, we should turn on ConnDirectionStatistics by default. (Or set the consensus parameter for a few days, to collect these statistics.) 6. Directory Protocol Specification Changes We propose adding IPv6 variants of the consumed bandwidth and connection direction statistics to the tor directory protocol. We propose the following additions to the [Tor Directory Protocol] specification, in section 2.1.2. Each addition should be inserted below the existing consumed bandwidth and connection direction specifications. "ipv6-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] "ipv6-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] Declare how much bandwidth the OR has used recently, on IPv6 connections. See "read-history" and "write-history" for more details. (The migration notes do not apply to IPv6.) "ipv6-conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL [At most once] Number of IPv6 connections, that are used uni-directionally or bi-directionally. See "conn-bi-direct" for more details. We also propose the following replacement, in the same section: "dirreq-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] "dirreq-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] Declare how much bandwidth the OR has spent on answering directory requests. See "read-history" and "write-history" for more details. (The migration notes do not apply to dirreq.) This replacement is optional, but it may avoid the 3 *read-history definitions getting out of sync. 7. Optional Changes We propose some optional changes to help relay operators, tor developers, and tor's network health. We also expect that these changes will drive IPv6 relay adoption. Some of these changes may be more appropriate as future work, or along with other proposed features. 7.1. Log IPv6 Statistics in Tor's Heartbeat Logs We propose this optional change, so relay operators can see their own IPv6 statistics: We propose that tor logs its IPv6 consumed bandwidth and connection statistics in its regular "heartbeat" logs. These heartbeat statistics should be collected over the lifetime of the tor process, rather than using the state file, like the statistics in sections 4 and 5. Tor's existing heartbeat logs already show its consumed bandwidth and connections (in the link protocol counts). We may also want to show IPv6 consumed bandwidth and connections as a propotion of the total consumed bandwidth and connections. These statistics only show a relay's local bandwidth usage, so they can't be used for reporting. 7.2. Show IPv6 Statistics on Consensus Health The [Consensus Health] website displays a wide rage of tor statistics, based on the most recent consensus. We propose this optional change, to: * help relay operators diagnose issues with IPv6 on their relays, and * to drive IPv6 adoption on tor relays. Consensus Health adds an IPv6 section, with the following relay statistics: * have an IPv6 ORPort, * support IPv6 reachability checks, * support IPv6 clients, and * support IPv6 reachability checks and IPv6 clients. The definitions of these statistics are in section 3. These changes can be tested using the script proposed in section 3. 7.3. Add an IPv6 Reachability Pseudo-Flag on Relay Search The [Relay Search] website displays tor relay information, based on the current consensus and relay descriptors. We propose this optional change, to: * help relay operators diagnose issues with IPv6 on their relays, and * drive IPv6 adoption on tor relays. Relay Search adds a pseudo-flag for relay IPv6 reachability support. This pseudo-flag should be given to relays that have: * a reachable IPv6 ORPort (in the consensus), and * support tor subprotocol version "Relay=3" (or later). See [Proposal 311: Relay IPv6 Reachability] for details. TODO: Is this a useful change? Are there better ways of driving IPv6 adoption? 7.4. Add IPv6 Connections and Consumed Bandwidth Graphs to Tor Metrics The [Tor Metrics: Traffic] website displays connection and bandwidth information for the tor network, based on relay extra-info descriptors. We propose these optional changes, to: * help tor developers improve IPv6 support on the tor network, * help diagnose issues with IPv6 on the tor network, and * drive IPv6 adoption on tor relays. Tor Metrics adds the following information to the graphs on the Traffic page: Consumed Bandwidth by IP version * added to the existing [Tor Metrics: Advertised bandwidth by IP version] page * as a stacked graph, like [Tor Metrics: Advertised and consumed bandwidth by relay flags] Fraction of connections used uni-/bidirectionally by IP version * added to the existing [Tor Metrics: Fraction of connections used uni-/bidirectionally] page * as a stacked graph, like [Tor Metrics: Advertised and consumed bandwidth by relay flags] 8. Test Plan We provide a quick summary of our testing plans. 8.1. Testing IPv6 Relay Consensus Count Logging We propose to test these changes using chutney networks. However, chutney creates a limited number of relays, so we also need to test these changes on the public tor network. Therefore, we propose to test these changes on the public network with a small number of relays and bridges. We can use the script and the tor logs to cross-check these calculations. The Tor Metrics team may also independently check these calculations. Once these changes are merged, they will be monitored by tor developers, as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.) 8.2. Testing IPv6 Extra-Info Statistics We propose to test the connection and consumed bandwidth statistics using chutney networks. However, chutney runs for a short amount of time, and creates a limited amount of traffic, so we also need to test these changes on the public tor network. In particular, we have struggled to test statistics using chutney, because tor's hard-coded statistics period is 24 hours. (And most chutney networks run for under 1 minute.) Therefore, we propose to test these changes on the public network with a small number of relays and bridges. During 2020, the Tor Metrics team will analyse these statistics on the public tor network, and provide IPv6 progress reports. We expect that we may discover some bugs during the first analysis. Once these changes are merged, they will be monitored by tor developers, as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.) References: [Consensus Health]: https://consensus-health.torproject.org/ [Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]: https://gitweb.torproject.org/torspec.git/tree/proposals/288-privcount-with… [Proposal 311: Relay IPv6 Reachability]: https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-rea… [Proposal 312: Relay Auto IPv6 Address]: https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv… [Relay Search]: https://metrics.torproject.org/rs.html [Tor Directory Protocol]: (version 3) https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [Tor Manual Page]: https://2019.www.torproject.org/docs/tor-manual.html.en [Tor Metrics: Advertised and consumed bandwidth by relay flags]: https://metrics.torproject.org/bandwidth-flags.html [Tor Metrics: Advertised bandwidth by IP version]: https://metrics.torproject.org/advbw-ipv6.html [Tor Metrics: Fraction of connections used uni-/bidirectionally]: https://metrics.torproject.org/connbidirect.html [Tor Metrics: Traffic]: https://metrics.torproject.org/bandwidth-flags.html [Tor Specification]: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev February 2020