RFC of proposal draft for "Migration to ed25519 HS identity keys and privacy-preserving directory documents"
Here is another HS proposal draft.
This one specifies how to migrate from the current HS protocol to the one specified in proposals "Migrate HS identity keys to Ed25519" and "Stop HS address enumeration by HSDirs": https://lists.torproject.org/pipermail/tor-dev/2013-August/005279.html https://lists.torproject.org/pipermail/tor-dev/2013-August/005280.html (I will soon write a proposal that merges these two proposals into one)
This proposal is in serious need for comments. Specifically, see section "1.1. From the PoV of Hidden Services" which I left entirely unspecified. There are also probably multiple migration concerns which I have forgotten or completely ignored.
Inlining:
Filename: xxx-hs-id-keys-migration.txt Title: Migration to ed25519 HS identity keys and privacy-preserving directory documents Author: George Kadianakis Created: 13 September 2013 Target: 0.2.5.x Status: Draft
[More draft than Guinness.]
0. Overview and motivation
Proposal XXX introduces two new HS features: a) Stronger HS identity keys. b) New-style HS directory documents so that HS addresses are not leaked to HSDirs.
This document specifies how different Tor actors must act after proposal XXX is implemented, so that the migration can proceed smoothly.
We aim for the migration to be smooth both from the perspective of Hidden Services (introduce grace period so that HS operators don't wake up one day to find that their HS can't be accessed with that address anymore) and from an implementation perspective (avoid bloating the Tor protocol or introducing sensitive flag days).
1. Migration strategy
After proposal XXX is implemented: a) The HS address format will change (called "new-style HS address" in this document) b) New HSDirs will be introduced (called "HSDirV3" in this document) c) New-style HS descriptors will be introduced (called "HS V3 descriptors" in this document).
The following sections investigate how these changes influence the various Tor actors:
1.1. From the PoV of Hidden Services:
=== XXX DISCUSSION XXX ===
I see (at least) three migration strategies here. I'm not sure which one is better so I'll write all of them and we can then discuss them and pick one.
a) After proposal XXX is implemented, Tor (configured as an HS) will only publish HS V3 descriptors by default. There will be a torrc parameter that the HS operator can use, that turns on publishing of HS V2 descriptors for backwards compatibility with the old HS address (the old identity key must be kept around).
b) After proposal XXX is implemented, Tor (configured as an HS) will publish both V3 and V2 HS descriptors for each Hidden Service. There will be a torrc parameter that turns off publishing of V2 HS descriptors. This parameter will eventually be switched off by default.
c) After proposal XXX is implemented, Tor (configured as an HS) will only publish V3 HS descriptors. The code that publishes V2 descriptors can be disabled or removed. HSes who want to publish V2 descriptors (and keep the same address) will have to maintain two copies of Tor -- one running the latest Tor version, and another one running an older version that supports V2 descriptors.
The engineer inside me tells me to go with c), but I feel that it leaves all the burden to the HS operators.
I haven't checked how much implementation effort doing a) or b) would take us.
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
1.3. From the PoV of HS directories:
Tor relays will advertise themselves as HSDirV3 servers using the "hidden-service-dir" router descriptor line.
For a while, relays will also continue being HSDirV2 servers. We will specify a timeout period of X months (4?), after which relays will stop being HSDirV2 servers by default (using the HidServDirectoryV2 torrc parameter).
1.4. From the PoV of directory authorities:
Authorities will continue voting for HSDirV2 servers. Eventually, when all relays have upgraded and no one is claiming to be HSDirV2, we can disable and remove the relevant code.
XXX Need to specify grace periods. XXX What did I forget?
On Fri, Sep 13, 2013 at 05:39:30PM +0300, George Kadianakis wrote:
Here is another HS proposal draft.
Inlining:
Filename: xxx-hs-id-keys-migration.txt Title: Migration to ed25519 HS identity keys and privacy-preserving directory documents Author: George Kadianakis Created: 13 September 2013 Target: 0.2.5.x Status: Draft
[More draft than Guinness.]
- Overview and motivation
Proposal XXX introduces two new HS features: a) Stronger HS identity keys. b) New-style HS directory documents so that HS addresses are not leaked to HSDirs.
This document specifies how different Tor actors must act after proposal XXX is implemented, so that the migration can proceed smoothly.
I assume XXX will be the number given to the combined prop for ecc keys and preventing enumeration, yes?
We aim for the migration to be smooth both from the perspective of Hidden Services (introduce grace period so that HS operators don't wake up one day to find that their HS can't be accessed with that address anymore) and from an implementation perspective (avoid bloating the Tor protocol or introducing sensitive flag days).
- Migration strategy
After proposal XXX is implemented: a) The HS address format will change (called "new-style HS address" in this document) b) New HSDirs will be introduced (called "HSDirV3" in this document) c) New-style HS descriptors will be introduced (called "HS V3 descriptors" in this document).
The following sections investigate how these changes influence the various Tor actors:
1.1. From the PoV of Hidden Services:
=== XXX DISCUSSION XXX ===
I see (at least) three migration strategies here. I'm not sure which one is better so I'll write all of them and we can then discuss them and pick one.
a) After proposal XXX is implemented, Tor (configured as an HS) will only publish HS V3 descriptors by default. There will be a torrc parameter that the HS operator can use, that turns on publishing of HS V2 descriptors for backwards compatibility with the old HS address (the old identity key must be kept around).
I think I actually like this idea the best - from a usability perspective. To ease the transition, Tor can have this option enabled by default for some 0.2.5 alpha/beta releases. If you're comfortable that this is working correctly, then you can flip the default to disabled for the -rc's. This will allow relay and hidden service operators, who run the latest version, to be early adoptors and begin to stress-test the new system. Alternatively, perhaps a better plan is to leave the option as enabled by default in 0.2.5 and disable it in 0.2.6. This latter option provides operators with many months to transition users to their new address without needing to know that they need to adjust their torrc.
In general, I think a PublishV2HiddenService is a better name than its opposite, for b) below.
b) After proposal XXX is implemented, Tor (configured as an HS) will publish both V3 and V2 HS descriptors for each Hidden Service. There will be a torrc parameter that turns off publishing of V2 HS descriptors. This parameter will eventually be switched off by default.
c) After proposal XXX is implemented, Tor (configured as an HS) will only publish V3 HS descriptors. The code that publishes V2 descriptors can be disabled or removed. HSes who want to publish V2 descriptors (and keep the same address) will have to maintain two copies of Tor -- one running the latest Tor version, and another one running an older version that supports V2 descriptors.
The engineer inside me tells me to go with c), but I feel that it leaves all the burden to the HS operators.
Indeed, although it is definitely the easiest, providing backward compat is the better choice (unless the protocol is completely broken).
I haven't checked how much implementation effort doing a) or b) would take us.
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
Depending on how this flag day is handled, maybe a consensus param will be useful too?
1.3. From the PoV of HS directories:
Tor relays will advertise themselves as HSDirV3 servers using the "hidden-service-dir" router descriptor line.
For a while, relays will also continue being HSDirV2 servers. We will specify a timeout period of X months (4?), after which relays will stop being HSDirV2 servers by default (using the HidServDirectoryV2 torrc parameter).
The hidden-service-dir line currently looks like:
"hidden-service-dir" *(SP VersionNum) NL
and if VersionNum is empty, it defaults to V2. When should this default be changed? Is it better to change this immediately, or modify the default as close to the flag day/release as possible?
1.4. From the PoV of directory authorities:
Authorities will continue voting for HSDirV2 servers. Eventually, when all relays have upgraded and no one is claiming to be HSDirV2, we can disable and remove the relevant code.
It may be better to chisel a day in stone for this. There will likely be a loooong tail on the network completely migrating, and waiting for the very last relay to upgrade could take years.
XXX Need to specify grace periods. XXX What did I forget?
Is there a similar high-profile migration that Tor or another project went through that we can base this and/or improve upon this time around?
I think if we stick to KISS but "give them time" it will result in a happier userbase.
It may also be a good idea to complete to implementation one component at a time, such as HSDirs -> Dir Auths -> HS -> Clients. The obvious disadvantage of this will be that testing is more difficult, until more components are done but it ensures that if any node runs Master then they won't start using the pieces before they're ready to be put together.
On 9/13/13, George Kadianakis desnacked@riseup.net wrote:
This proposal is in serious need for comments.
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
As commented earlier on this list... https://lists.torproject.org/pipermail/tor-dev/2013-August/005286.html
This addressing change will break onioncat (IPv6) within Tor. https://www.onioncat.org/
I suggest a solution to transporting IPv6 within Tor be maintained/deployed concurrently with any change in current onion addressing and or transport mechanics.
On Thu, Sep 19, 2013 at 4:53 AM, grarpamp grarpamp@gmail.com wrote:
On 9/13/13, George Kadianakis desnacked@riseup.net wrote:
This proposal is in serious need for comments.
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
As commented earlier on this list... https://lists.torproject.org/pipermail/tor-dev/2013-August/005286.html
This addressing change will break onioncat (IPv6) within Tor. https://www.onioncat.org/
I suggest a solution to transporting IPv6 within Tor be maintained/deployed concurrently with any change in current onion addressing and or transport mechanics.
I have nothing against onioncat, but let's remember that the fact that you can squeeze a Tor hidden service address into an IPv6 address is completely accidental.
If we want to keep that ability when hidden service addresses become longer than 128 bits, the obvious solution would be to add some kind of translation layer.
yrs,
On 9/19/13, Nick Mathewson nickm@alum.mit.edu wrote:
On Thu, Sep 19, 2013 at 4:53 AM, grarpamp grarpamp@gmail.com wrote:
I suggest a solution to transporting IPv6 within Tor be maintained/deployed concurrently with any change in current onion addressing and or transport mechanics.
I have nothing against onioncat, but let's remember that the fact that you can squeeze a Tor hidden service address into an IPv6 address is completely accidental.
If we want to keep that ability when hidden service addresses become longer than 128 bits, the obvious solution would be to add some kind of translation layer.
Of course... since we have hidden services, there's no reason we shouldn't provide an IPv6 layer for them. Onioncat proved the utility in that. Whether it's n-bits backed, onioncat fronted or some other new form doesn't really matter... so long as there's a working IPv6 layer with the same user facing interface/route semantic as onioncat provides. That's the necessary key to making it useful.
On Fri, Sep 13, 2013 at 10:39 AM, George Kadianakis desnacked@riseup.net wrote:
Here is another HS proposal draft.
[...]
1.1. From the PoV of Hidden Services:
=== XXX DISCUSSION XXX ===
I see (at least) three migration strategies here. I'm not sure which one is better so I'll write all of them and we can then discuss them and pick one.
a) After proposal XXX is implemented, Tor (configured as an HS) will only publish HS V3 descriptors by default. There will be a torrc parameter that the HS operator can use, that turns on publishing of HS V2 descriptors for backwards compatibility with the old HS address (the old identity key must be kept around).
b) After proposal XXX is implemented, Tor (configured as an HS) will publish both V3 and V2 HS descriptors for each Hidden Service. There will be a torrc parameter that turns off publishing of V2 HS descriptors. This parameter will eventually be switched off by default.
c) After proposal XXX is implemented, Tor (configured as an HS) will only publish V3 HS descriptors. The code that publishes V2 descriptors can be disabled or removed. HSes who want to publish V2 descriptors (and keep the same address) will have to maintain two copies of Tor -- one running the latest Tor version, and another one running an older version that supports V2 descriptors.
The engineer inside me tells me to go with c), but I feel that it leaves all the burden to the HS operators.
I haven't checked how much implementation effort doing a) or b) would take us.
Let's also consider d) Exactly like a, except that if we find a hidden service with a RSA1024 hidden service identity key, the default behavior is to publish both V2 and V3. Otherwise, the default is to publish V3 only.
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
Are you sure? Assuming that the addresses are 255 bits long, we can do it in 51 base32 characters. If they're 256 bits long, we can round up to 52 base32 characters, and reserve the leftover 4 bits as a version field or something. How did you get 56 here?
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
I think this warning is likely to be obnoxious in practice, since it won't tell users which address they should use instead.
1.3. From the PoV of HS directories:
Tor relays will advertise themselves as HSDirV3 servers using the "hidden-service-dir" router descriptor line.
Don't they already do that to advertise themselves as v2 hidden service directories, or am I mistaken?
For a while, relays will also continue being HSDirV2 servers. We will specify a timeout period of X months (4?), after which relays will stop being HSDirV2 servers by default (using the HidServDirectoryV2 torrc parameter).
X months starting when? It's probably not the most user-friendly idea to stop supporting the current hidden service design before the new code is stable, for instance.
1.4. From the PoV of directory authorities:
Authorities will continue voting for HSDirV2 servers. Eventually, when all relays have upgraded and no one is claiming to be HSDirV2, we can disable and remove the relevant code.
XXX Need to specify grace periods. XXX What did I forget?
Should there be any means to link new/old identities? In other words, if I have been running OLD = old19jkbba73bap5.onion, and I am now going to be changing my address to NEW = new8787dsfg9gprioj57669h8941bvyzbli81b9ayeotoabervla.onion, should there be any way to tell people "hey, use the new one instead", or for the new one to prove that it is the same as the old one?
One fine option here is "no", since most of the solutions I can think of are insecure and invite enumeration opportunities. But people may want to do this anyway, and I worry that if they do, the solutions they pick might be even worse than the ones I come up with.
peace,
Nick Mathewson nickm@alum.mit.edu writes:
On Fri, Sep 13, 2013 at 10:39 AM, George Kadianakis desnacked@riseup.net wrote:
Here is another HS proposal draft.
[...]
1.1. From the PoV of Hidden Services:
=== XXX DISCUSSION XXX ===
I see (at least) three migration strategies here. I'm not sure which one is better so I'll write all of them and we can then discuss them and pick one.
a) After proposal XXX is implemented, Tor (configured as an HS) will only publish HS V3 descriptors by default. There will be a torrc parameter that the HS operator can use, that turns on publishing of HS V2 descriptors for backwards compatibility with the old HS address (the old identity key must be kept around).
b) After proposal XXX is implemented, Tor (configured as an HS) will publish both V3 and V2 HS descriptors for each Hidden Service. There will be a torrc parameter that turns off publishing of V2 HS descriptors. This parameter will eventually be switched off by default.
c) After proposal XXX is implemented, Tor (configured as an HS) will only publish V3 HS descriptors. The code that publishes V2 descriptors can be disabled or removed. HSes who want to publish V2 descriptors (and keep the same address) will have to maintain two copies of Tor -- one running the latest Tor version, and another one running an older version that supports V2 descriptors.
The engineer inside me tells me to go with c), but I feel that it leaves all the burden to the HS operators.
I haven't checked how much implementation effort doing a) or b) would take us.
Let's also consider d) Exactly like a, except that if we find a hidden service with a RSA1024 hidden service identity key, the default behavior is to publish both V2 and V3. Otherwise, the default is to publish V3 only.
Good idea!
1.2. From the PoV of the HS client:
Tor clients can distinguish new-style HS addresses from old ones by their length. Legacy addresses are 16 base32 characters, while new ones are 56 (XXX) base32 characters.
Are you sure? Assuming that the addresses are 255 bits long, we can do it in 51 base32 characters. If they're 256 bits long, we can round up to 52 base32 characters, and reserve the leftover 4 bits as a version field or something. How did you get 56 here?
Yep, you are right.
(I did 'len(base64.b32encode("a"*32))' in Python, which was a mistake for base32 since 32 is not a multiple of 5.)
If Tor is asked to connect to a legacy address it SHOULD throw a warning and advocate the use of new-style addresses (it should still connect to the HS however). In the future, when old-style HS addresses are close to depletion, we can introduce a torrc parameter that blocks client connections to old-style HS addresses.
I think this warning is likely to be obnoxious in practice, since it won't tell users which address they should use instead.
1.3. From the PoV of HS directories:
Tor relays will advertise themselves as HSDirV3 servers using the "hidden-service-dir" router descriptor line.
Don't they already do that to advertise themselves as v2 hidden service directories, or am I mistaken?
I meant that they will also have to append '3' in their hidden-service-dir line.
I think it currently looks like this: """ opt hidden-service-dir """ (and it defaults to version 2)
and after this proposal it should look like this:
""" opt hidden-service-dir 2 3 """
For a while, relays will also continue being HSDirV2 servers. We will specify a timeout period of X months (4?), after which relays will stop being HSDirV2 servers by default (using the HidServDirectoryV2 torrc parameter).
X months starting when? It's probably not the most user-friendly idea to stop supporting the current hidden service design before the new code is stable, for instance.
Hm, maybe we should change those "X months" grace periods to "When 0.2.6.x. becomes stable" or to "When 0.2.6.x. becomes stable && no earlier than in X months"?
1.4. From the PoV of directory authorities:
Authorities will continue voting for HSDirV2 servers. Eventually, when all relays have upgraded and no one is claiming to be HSDirV2, we can disable and remove the relevant code.
XXX Need to specify grace periods. XXX What did I forget?
Should there be any means to link new/old identities? In other words, if I have been running OLD = old19jkbba73bap5.onion, and I am now going to be changing my address to NEW = new8787dsfg9gprioj57669h8941bvyzbli81b9ayeotoabervla.onion, should there be any way to tell people "hey, use the new one instead", or for the new one to prove that it is the same as the old one?
One fine option here is "no", since most of the solutions I can think of are insecure and invite enumeration opportunities. But people may want to do this anyway, and I worry that if they do, the solutions they pick might be even worse than the ones I come up with.
I also thought that something like this could be useful, but I didn't want to add more implementation pain to this proposal.
A plausible solution here (that doesn't lead to obvious enumeration opportunities) is to have Hidden Services send a "You are using my old HS address. Try this one" relay cell right after the RP circuit has been spliced and before the RELAY_COMMAND_BEGIN command has been sent by the client. This way the the new address is not leaked to HSDirs, IPs or RPs.
Still, I'm not sure if this is worth the implemention effort.
Hi,
I'm posting a new revision of this proposal. Hopefully it addresses questions/comments from Nick and Matthew.
It's still a draft and needs feedback from the community. Specifically, I have two XXXes in the end of the proposal that require discussion.
Enjoy.
-
George Kadianakis -
grarpamp -
Matthew Finkel -
Nick Mathewson