Hi,
we had some discussions during the dev meeting regarding hidden services and why users sometimes report that the access to them fails. The general problem might be that the specific hidden service doesn't exist anymore. However in some cases the HS exists (see URL:http://tor.stackexchange.com/q/1442/88 as an example).
So one theory was that the transistion from TAP to ntor might cause problems. I tried to test it and found no evidence so far.
What did I do? I checked out the current Tor source from git and the tor-0.2.3.25 tagged version. I built both versions and gave them a minimal torrc (SocksPort, Log, DataDir, ControlPort). After the Tor process built a circuit I used a random page from the hidden wiki and tried to access it. If it failed, I tried either the HS from TorProject or DDG. If it succeeded, I built a new circuit and tried another HS.
During the first tests I used the hotel network, which was kind of flaky. With a more stable connection I tested nearly 20 hidden services and found no difference in the behaviour of both Tor versions. Both failed on one random HS and opened all other reasonably fast.
While this is no final proof it is good evidence that there is no connection between failing hidden services and TAP/ntor. So maybe we need another theory to test. ;)
and why users sometimes report that the access to them fails. The
As an anecdote, I seemed to feel a step improvement in HS access sometime between 1-2 years ago, perhaps in the move to later 0.2.3.x. And that it's very good now.
I think a big user problem was the socks5 timeout issue with firefox that has since been fixed.
and found no difference in the behaviour of both Tor versions. Both failed on one random HS and opened all other reasonably fast.
I see a few what I would call random failures to prime (from a service known to be up... reliably fetch the HS desciptor the first time, or build the path the first time). I've never really looked into it other than wanting the below feature to eliminate any browser/socks and that sort of non-tor issue, and to better script these kinds of tests against hundreds of onions. ie: Answer these questions in successive individual order... - can we get a descriptor off the hsdirs, why/not, and print it out in full - can we rendezvous, build and connect to the far end tor, why/not, and print - can we get reply from whatever may be listening on HiddenServicePort
So maybe we need another theory to test. ;)
Would be nice to be able to fetch and print out the HS descriptor from the controller. I posted/ticketed a rough proposed command operation spec for that maybe 6mo ago that we had some mutual review/interest/code on this list. Don't know it's current status... Qingping?
https://trac.torproject.org/projects/tor/ticket/8993 https://trac.torproject.org/projects/tor/ticket/3521
On Thu 20 Feb 2014 05:41:53 PM EST, grarpamp wrote:
Would be nice to be able to fetch and print out the HS descriptor from the controller. I posted/ticketed a rough proposed command operation spec for that maybe 6mo ago that we had some mutual review/interest/code on this list. Don't know it's current status... Qingping?
Printing out the whole descriptor in controller will be something nice to have :)
However, I think to trouble shoot this issue, current events in controller is probably enough. You can at least get following information:
* can we get a descriptor off the hsdirs * if not, likely something bad happened on hsdir * can we establish rendezvous circuit * if not, chosen RP suddenly disappeared from the network? * can we receive introduction ack * if not, try another introduction point * can RP join the rendezvous circuit * can we get any reply from the stream * if not, there is something wrong with HS
This should be able to help us narrow the problem.
If we can reproduce this over our own HS, then it should be easy to trouble shoot the problem since we will have control over RP, IP and HS. But we will never know what goes wrong on hsdir :(
I can maybe help with this a bit since my HS profiling tool already collects all these information.
Is it very easy to reproduce this for any HS?
-- qp
On Thu, Feb 20, 2014 at 9:01 PM, Qingping Hou dave2008713@gmail.com wrote:
On Thu 20 Feb 2014 05:41:53 PM EST, grarpamp wrote:
Would be nice to be able to fetch and print out the HS descriptor from the controller. I posted/ticketed a rough proposed command operation spec for that maybe 6mo ago that we had some mutual review/interest/code on this list. Don't know it's current status... Qingping?
Printing out the whole descriptor in controller will be something nice to have :)
However, I think to trouble shoot this issue, current events in controller is probably enough. You can at least get following information:
- can we get a descriptor off the hsdirs
- if not, likely something bad happened on hsdir
- can we establish rendezvous circuit
- if not, chosen RP suddenly disappeared from the network?
- can we receive introduction ack
- if not, try another introduction point
- can RP join the rendezvous circuit
- can we get any reply from the stream
- if not, there is something wrong with HS
This should be able to help us narrow the problem.
Yes events and other debugging would help find any protocol buggyness.
My goals were different... what are the HS metainfo, and is HS reachable/up?
- can we get a descriptor off the hsdirs, why/not, and print it out in full
- can we rendezvous, build and connect to the far end tor, why/not, and print
- can we get reply from whatever may be listening on HiddenServicePort
Clarifying the above... The first info can only come from querying the controller. The second should have read to just return a 0/1 and why for an end2end circ build to HS. The latter is just a userspace action, not a scope for any controller function.
I should try to write a query spec.
Yes, the 'why/not' parts of the hsdir and circuit queries could perhaps be ideally supplied by your deeper event tracking.
It's easier to answer the goals when asking the controller directly on sync demand, rather than async watching events triggered externally by sending socks5 requests for onions.
Is it very easy to reproduce this for any HS?
Any trouble I've seen with HS has felt like inconsistent network issues, such as packet loss, or just server issues. The deeper events or logging could rule out Tor there. And the query goals I have could rule in a reproducible HS.
-
grarpamp -
Jens Kubieziel -
Qingping Hou