Hi everyone,
We’re working on a peer-to-peer group chat app where peers connect over v3 onion addresses.
One issue are groups where there are many users but only a few are online in a given moment. Onion addresses are forever, and existing peers might know every peer in the network, but it will take a while to try connecting to all of them to find one that is online.
In this case, it seems helpful for one or more peers to share one or more onion addresses that would serve as reliable “trackers", e.g.
1. All members know the keypairs for these addresses. 2. All online members ping these addresses at random intervals to say they’re online. 3. If they can’t connect to an address, they start hosting it themselves.
We’re going to start testing it, but we’re wondering if folks here know the likely outcome of trying to “share” hosting of an onion service in this spontaneous-volunteer sort of way and if there are downsides.
I *think* the most important question is how long it takes for the network to stop routing incoming traffic to an offline client when there’s an online one available. How long will the address likely be unreachable in one of these transition moments, assuming some peer immediately detects that a “tracker” onion address has gone offline and begins hosting it themselves? (And does this question make sense?)
Thanks!
Holmes
On 26 Mar (08:55:54), Holmes Wilson wrote:
Hi everyone,
Greetings,
We’re working on a peer-to-peer group chat app where peers connect over v3 onion addresses.
One issue are groups where there are many users but only a few are online in a given moment. Onion addresses are forever, and existing peers might know every peer in the network, but it will take a while to try connecting to all of them to find one that is online.
In this case, it seems helpful for one or more peers to share one or more onion addresses that would serve as reliable “trackers", e.g.
- All members know the keypairs for these addresses.
- All online members ping these addresses at random intervals to say they’re online.
- If they can’t connect to an address, they start hosting it themselves.
We’re going to start testing it, but we’re wondering if folks here know the likely outcome of trying to “share” hosting of an onion service in this spontaneous-volunteer sort of way and if there are downsides.
I *think* the most important question is how long it takes for the network to stop routing incoming traffic to an offline client when there’s an online one available. How long will the address likely be unreachable in one of these transition moments, assuming some peer immediately detects that a “tracker” onion address has gone offline and begins hosting it themselves? (And does this question make sense?)
Interesting idea!
So sharing onion address key material between peers can be fine until they are used at the same time. What will happen is that the two peers hosting the same onion address (service) will start competing on the onion service directory side where service's upload what we call a "descriptor" which is what client fetch in order to initiate a connection to the service.
With v3, it gets even more complicated actually because of the "revision counter" in the descriptor which v2 didn't have.
It is simply a number that keeps going up in the descriptor so the onion service directory (relay) doesn't accept a previous descriptor (replay). And so, your two peers sharing the onion keys will require to somehow sync that revision counter for your idea to work (located on disk in the state file, "HidServRevCounter").
Else, one will inevitably be higher than the other and thus will always succeed where it will always fail for the other peer.
One approach here, and it is a hack of course, is for the second peer detecting that the onion is down to download the descriptor (offline one) still on the directory, get the revision counter out of it ("revision-counter" field), do a +1 on it and set its state file with it. It is a hack but would work... You will likely have to deal with "descriptor not found anymore on the directories" and so you could bruteforce the revision counter from your last one like adding +1000 to it and that might do the trick. But I would definitely avoid as much as possible brute forcing it by small steps else your service will add undesirable load to the network with a lot of upload attempts.
Now lets say you are able to pull that off, if I understand your question correctly, once an onion address goes offline, if the other peer would become the service then few seconds later, all clients will be able to connect to it. In other words, launching a new service takes few seconds before it can be reached. The new descriptor will be used by the directories immediately upon upload as long as the revision counter is higher than the previous one :).
Hope this help! David
Interesting!
So it seems important to not have two clients hosting the same onion address at the same time, which is doable, and then we have to figure out this revision counter.
What happens if the revision counter is not incremented correctly?
Is it that the now offline host could become the destination for incoming traffic to the onion address?
Thanks so much for this feedback! Holmes
On Fri, Apr 2, 2021 at 1:41 PM David Goulet dgoulet@torproject.org wrote:
On 26 Mar (08:55:54), Holmes Wilson wrote:
Hi everyone,
Greetings,
We’re working on a peer-to-peer group chat app where peers connect over
v3
onion addresses.
One issue are groups where there are many users but only a few are
online in
a given moment. Onion addresses are forever, and existing peers might
know
every peer in the network, but it will take a while to try connecting to
all
of them to find one that is online.
In this case, it seems helpful for one or more peers to share one or more onion addresses that would serve as reliable “trackers", e.g.
- All members know the keypairs for these addresses.
- All online members ping these addresses at random intervals to say they’re online.
- If they can’t connect to an address, they start hosting it themselves.
We’re going to start testing it, but we’re wondering if folks here know
the
likely outcome of trying to “share” hosting of an onion service in this spontaneous-volunteer sort of way and if there are downsides.
I *think* the most important question is how long it takes for the
network
to stop routing incoming traffic to an offline client when there’s an
online
one available. How long will the address likely be unreachable in one of these transition moments, assuming some peer immediately detects that a “tracker” onion address has gone offline and begins hosting it
themselves?
(And does this question make sense?)
Interesting idea!
So sharing onion address key material between peers can be fine until they are used at the same time. What will happen is that the two peers hosting the same onion address (service) will start competing on the onion service directory side where service's upload what we call a "descriptor" which is what client fetch in order to initiate a connection to the service.
With v3, it gets even more complicated actually because of the "revision counter" in the descriptor which v2 didn't have.
It is simply a number that keeps going up in the descriptor so the onion service directory (relay) doesn't accept a previous descriptor (replay). And so, your two peers sharing the onion keys will require to somehow sync that revision counter for your idea to work (located on disk in the state file, "HidServRevCounter").
Else, one will inevitably be higher than the other and thus will always succeed where it will always fail for the other peer.
One approach here, and it is a hack of course, is for the second peer detecting that the onion is down to download the descriptor (offline one) still on the directory, get the revision counter out of it ("revision-counter" field), do a +1 on it and set its state file with it. It is a hack but would work... You will likely have to deal with "descriptor not found anymore on the directories" and so you could bruteforce the revision counter from your last one like adding +1000 to it and that might do the trick. But I would definitely avoid as much as possible brute forcing it by small steps else your service will add undesirable load to the network with a lot of upload attempts.
Now lets say you are able to pull that off, if I understand your question correctly, once an onion address goes offline, if the other peer would become the service then few seconds later, all clients will be able to connect to it. In other words, launching a new service takes few seconds before it can be reached. The new descriptor will be used by the directories immediately upon upload as long as the revision counter is higher than the previous one :).
Hope this help! David _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
David Goulet dgoulet@torproject.org writes:
On 26 Mar (08:55:54), Holmes Wilson wrote:
Hi everyone,
Greetings,
We’re working on a peer-to-peer group chat app where peers connect over v3 onion addresses.
One issue are groups where there are many users but only a few are online in a given moment. Onion addresses are forever, and existing peers might know every peer in the network, but it will take a while to try connecting to all of them to find one that is online.
In this case, it seems helpful for one or more peers to share one or more onion addresses that would serve as reliable “trackers", e.g.
- All members know the keypairs for these addresses.
- All online members ping these addresses at random intervals to say they’re online.
- If they can’t connect to an address, they start hosting it themselves.
We’re going to start testing it, but we’re wondering if folks here know the likely outcome of trying to “share” hosting of an onion service in this spontaneous-volunteer sort of way and if there are downsides.
I *think* the most important question is how long it takes for the network to stop routing incoming traffic to an offline client when there’s an online one available. How long will the address likely be unreachable in one of these transition moments, assuming some peer immediately detects that a “tracker” onion address has gone offline and begins hosting it themselves? (And does this question make sense?)
Interesting idea!
So sharing onion address key material between peers can be fine until they are used at the same time. What will happen is that the two peers hosting the same onion address (service) will start competing on the onion service directory side where service's upload what we call a "descriptor" which is what client fetch in order to initiate a connection to the service.
With v3, it gets even more complicated actually because of the "revision counter" in the descriptor which v2 didn't have.
It is simply a number that keeps going up in the descriptor so the onion service directory (relay) doesn't accept a previous descriptor (replay). And so, your two peers sharing the onion keys will require to somehow sync that revision counter for your idea to work (located on disk in the state file, "HidServRevCounter").
Else, one will inevitably be higher than the other and thus will always succeed where it will always fail for the other peer.
Hello all,
this revision counter sync issue is not a problem anymore since we introduced the Order-Preserving-Encryption revision counter logic: https://gitlab.torproject.org/tpo/core/tor/-/blob/master/src/feature/hs/hs_s... https://gitlab.torproject.org/tpo/core/torspec/-/blob/master/rend-spec-v3.tx... Feel free to try it and let us know if it doesn't work. The solution assumes that all peers have reasonably synchronized clocks.
In other news, the above "all members know all keypairs" approach seems super dangerous in terms of security, especially if not all those members are 100% trusted by each other.
To answer the performance question, if a peer immediately notices the onion service being offline and begins hosting it, it should be pretty-much immediately reachable by new clients.
On Apr 6, 2021, at 6:37 AM, George Kadianakis desnacked@riseup.net wrote:
David Goulet dgoulet@torproject.org writes:
On 26 Mar (08:55:54), Holmes Wilson wrote:
Hi everyone,
Greetings,
We’re working on a peer-to-peer group chat app where peers connect over v3 onion addresses.
One issue are groups where there are many users but only a few are online in a given moment. Onion addresses are forever, and existing peers might know every peer in the network, but it will take a while to try connecting to all of them to find one that is online.
In this case, it seems helpful for one or more peers to share one or more onion addresses that would serve as reliable “trackers", e.g.
- All members know the keypairs for these addresses.
- All online members ping these addresses at random intervals to say
they’re online. 3. If they can’t connect to an address, they start hosting it themselves.
We’re going to start testing it, but we’re wondering if folks here know the likely outcome of trying to “share” hosting of an onion service in this spontaneous-volunteer sort of way and if there are downsides.
I *think* the most important question is how long it takes for the network to stop routing incoming traffic to an offline client when there’s an online one available. How long will the address likely be unreachable in one of these transition moments, assuming some peer immediately detects that a “tracker” onion address has gone offline and begins hosting it themselves? (And does this question make sense?)
Interesting idea!
So sharing onion address key material between peers can be fine until they are used at the same time. What will happen is that the two peers hosting the same onion address (service) will start competing on the onion service directory side where service's upload what we call a "descriptor" which is what client fetch in order to initiate a connection to the service.
With v3, it gets even more complicated actually because of the "revision counter" in the descriptor which v2 didn't have.
It is simply a number that keeps going up in the descriptor so the onion service directory (relay) doesn't accept a previous descriptor (replay). And so, your two peers sharing the onion keys will require to somehow sync that revision counter for your idea to work (located on disk in the state file, "HidServRevCounter").
Else, one will inevitably be higher than the other and thus will always succeed where it will always fail for the other peer.
Hello all,
this revision counter sync issue is not a problem anymore since we introduced the Order-Preserving-Encryption revision counter logic: https://gitlab.torproject.org/tpo/core/tor/-/blob/master/src/feature/hs/hs_s... https://gitlab.torproject.org/tpo/core/torspec/-/blob/master/rend-spec-v3.tx... Feel free to try it and let us know if it doesn't work. The solution assumes that all peers have reasonably synchronized clocks.
Nice, that’s convenient! We’ll try it! What happens if peers don’t have reasonably synchronized clocks, just out of curiosity?
In other news, the above "all members know all keypairs" approach seems super dangerous in terms of security, especially if not all those members are 100% trusted by each other.
Thank you for flagging this. Are there threats beyond denial of service that we should be worried about? It would be extremely helpful to hear about them, if so.
If you’re mostly worried about the fact that any member can now wreak havoc with this shared peer discovery service, I *think* that’s okay for this mechanism. Though I’d hugely appreciate being corrected on this!
Here’s why I *think* it may be okay, though again I welcome being corrected:
First, each peer will already know the full list of member addresses, and the signing key of an admin, so we can build things such that whoever runs this service does not have the power to pollute a user’s list of peers—just the power to offer users useless offline peers, or offer them nothing.
Second, because this “shared” service is only for learning more quickly which ones are online, even if a malicious member uses their knowledge of the keys to disable the service for all users forever, peers would still have other (albeit slower) ways to connect to each other. (Worst case: try all peers. Or we could have 3 or more shared addresses where each is known to subgroup of users, so that an attacker would need to control multiple accounts to break the mechanism.)
I assume there are big things I’m missing and I’m extremely grateful for any feedback!
The alternatives we’ve considered for discovering which peers are online either require a peer that is always online (which is ideal but not practical for all groups) or some global network that metadata would probably leak to, and that an even larger set of malicious peers could probably wreak havoc with.
It would be amazing if the Tor network offered some kind of "meta-onion" address that would return a list of all online onion addresses with the keys for this address. (Since then we wouldn’t have to worry about one malicious peer with the keys taking down the whole discovery mechanism for everyone.) Is there anything like this? It seems like it might be useful in other cases as well, such as load balancing.
Again thanks everyone for the responses and feedback!!!
H
To answer the performance question, if a peer immediately notices the onion service being offline and begins hosting it, it should be pretty-much immediately reachable by new clients. _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
-
David Goulet -
George Kadianakis -
Holmes Wilson