Hi,
tl;dr: ideas to reduce the cw fraction that outdated relays make up of the tor network
This email can be somewhat summarized by these three polls: https://twitter.com/nusenu_/status/909356846900801536 https://twitter.com/nusenu_/status/909359701908971520 https://twitter.com/nusenu_/status/909361044988071936
This email was sparked by a recent thread on twitter.
As of 2017-09-16 more than 27% CW fraction of the tor network (>2500 relays) run a version of tor that is not recommended by tor's directory authorities [2]. More than 8% CW fraction (>1100 relays) even run tor branches that reached end-of-life and are no longer maintained.
So if we agree that 27% CW fraction is to much, we could aim at reducing that fraction within the tor network. Here are a few ideas.
1) Auto-senescence -------------------
Alfie John suggested [1]:
automatic shutdown after its version is too old
as used in zcash (the tor snap from Chad Miller does auto-shutdown already today), Moritz replied:
Tor already has this mechanism in the form of recommended/required versions in consensus
I looked into that [3] but this is about tor _protocols_ (not versions) and is probably better suitable for avoiding compatibility issues within the tor network as multiple protocols coexist but you can not use that to remove specific tor versions because many versions share the same protocol set [4].
[1] https://twitter.com/alfiedotwtf/status/906310842672603136 [2] https://consensus-health.torproject.org/#recommendedversions [3] https://gitweb.torproject.org/torspec.git/tree/proposals/264-subprotocol-ver... [4] https://gist.github.com/nusenu/1302a04b26dac8e2ef838117f5f3fd2b
So back to Alfie's suggestion. If tor should shutdown when 'too old' we have to know what 'too old' is.
To come up with a timespan for 'too old' I looked into 61 currently used tor versions that are no longer recommended by dirauths. I measured the time between release date (as mentioned in the changelog) and the last consensus where the given version was listed as a recommended relay tor version [5], but these graphs show the "recommended" lifespan of releases before the new support policy [6] with shorter release cycles for "regular" releases has been introduced.
[5] https://twitter.com/nusenu_/status/909355193434853376 [6] https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/CoreTorR...
So instead of of using this historical data we could simply follow the support policy.
regular release: Auto-shutdown 18 months (=9+grace period) after the branch became stable.
LTS relase: Auto-shutdown 4 years (=3+grace period) after the branch became stable.
Dates would be hardcoded and do not require relays to contact dirauths. So old relays will not but any load on dir auths and fallbackdirs everytime they start.
Lets simulate these timespans for tor 0.2.4 and 0.2.5 (and lets assume they are LTS releases)
- tor 0.2.4 relays - if they had such a logic - would shutdown on 2017-12-11 (eol date was 2017-08-01) - tor 0.2.5 relays whould shutdown on 2018-10-24 (eol date is 2018-05-01) (not mentioning 0.2.6/7 here) - tor 0.2.9 relays would shutdown on 2020-12-19 (eol date is 2020-01-01)
You could add a torrc option to allow operators (and package maintainers) to opt-out of auto-shutdown.
There is no doubt that any mechanism to disable old relays _automatically_ and by _default_ would shrink the tor network (compared to not having that), so having a good answer for "What is an acceptable CW fraction for outdated tor versions on the network?" would be important, but since the reasons for removing versions from the recommended list are very different the recommended datapoint might not be enough.
2) consensus weight penalty for outdated relays -----------------------------------------------
Instead of (or additionally to) telling relays to not upload descriptors (complete removal) you could _automatically_ reduce/cap their consensus weight on the dirauth side if they run - unsupported branches (harsher penalty) - run not-recommended tor releases over a long time (less harsh penalty)
(The distinction between unsupported branches and not-recommended is currently not available as consensus information.)
This approach makes the recommended version list a more critical consensus item and as you know this is not always managed ideally: - new releases not added to the list in a timely manner -> operators upgrading timely get warnings - old releases not removed timely or before deb.torproject.org provided new releases
3) update tor dir auth code to reject old tor releases (not include them in consensus) -------------------------------------------------------------------------
As soon as a tor directory operator updates to a new release the dirauth would no longer vote for specific tor versions (I guess this is the current mostly manual approach).
Another important aspect is the practical reality in current package repositories, but I simply assume they will follow LTS releases and are fine with the 4 years (3+grace period) lifespan.
regards, nusenu
Hi nusenu,
This looks like it needs a proposal: I think we might have some similar proposals already.
On 17 Sep 2017, at 20:48, nusenu nusenu-lists@riseup.net wrote:
- Auto-senescence
I think automatic timed shutdown can be unhelpful or dangerous: * what if we need it earlier due to a severe bug or mandatory feature? * what if it isn't needed, and the relay version is fine?
- consensus weight penalty for outdated relays
I can't see much point in this: if the relays are insecure, they should be eliminated. If not, they should be used.
- update tor dir auth code to reject old tor releases (not include them
in consensus)
As soon as a tor directory operator updates to a new release the dirauth would no longer vote for specific tor versions (I guess this is the current mostly manual approach).
Another important aspect is the practical reality in current package repositories, but I simply assume they will follow LTS releases and are fine with the 4 years (3+grace period) lifespan.
In the past, we've excluded relay versions when they don't have a required feature. For example:
0.2.4.8-alpha is the first version that supports ntor onion keys, so earlier versions and relays without ntor onion keys are excluded
0.2.4.18-rc doesn't believe enough current directory authorities, so it is excluded (we may have to re-do the minimum version when we get a 9th directory authority, or if authorities change)
0.2.9.0-alpha to 0.2.9.5-alpha deliver expired consensuses to clients, so they are excluded
In the future, when we require network-wide features, we will exclude versions that don't support this feature. And similarly with a number of newer protocol features.
I can't think of any other bugs or features that have this significant an impact. But if they are, we can use this manual process.
We have a ticket to make a plan to kill off old client versions: https://trac.torproject.org/projects/tor/ticket/15940 But there's no equivalent ticket for relay versions.
T -- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n ------------------------------------------------------------------------
- Auto-senescence
I think automatic timed shutdown can be unhelpful or dangerous:
Yes it reduces the number of options once such a feature would be implemented and deployed.
- what if we need it earlier due to a severe bug or mandatory feature?
This should not be an issue since that auto-shutdown only mandates an upper limit but does not stop you from removing a relay before that limit has been reached.
- what if it isn't needed, and the relay version is fine?
Yes this can be an issue, but if you say "every relay that runs versions past its eol date" [1] is "not fine" then the auto-shutdown date can be specified with a very high likelihood to a date that is past the eol date because you have an estimate of how long you plan to support it (3y for LTS).
I'm less concerned about auto-shutdown for tor clients since most tor users might be using TBB with auto updates and it would help you having to do things like prop266.
[1] https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/CoreTorR...
- consensus weight penalty for outdated relays
I can't see much point in this: if the relays are insecure, they should be eliminated. If not, they should be used.
I'm happy with "insecure -> should be removed". With "outdated" I meant "not running a recommended version" I'm not sure if that is the same as 'insecure'.
A CW penalty would be a strong incentive for relay operators to keep their relays up to date (to a recommended version). This would likely reduce the number of relays running not-recommended versions because currently the incentive is inverted (never restart/update your tor instance - uptime!). ..but it would also affect testers running master.
- update tor dir auth code to reject old tor releases (not include them
in consensus)
In the past, we've excluded relay versions when they don't have a required feature.
Does this step (excluding specific versions) require a code change or a dir auth configuration change? (like it does for changing recommended versions list) If it does: Maybe it could be turned into a configurable option for dir auths like recommended version.
(3) will not stop old relays from contacting dir auths.
We have a ticket to make a plan to kill off old client versions: https://trac.torproject.org/projects/tor/ticket/15940 But there's no equivalent ticket for relay versions.
-
nusenu -
teor