This is the second part of our preliminary analysis of how Tor users interact with onion services [0]. In this part, we analyse the issue of onion service discovery. Onion services are private by default, so it's the operator's responsibility to disseminate their domain if they want it to be public.
Question 3.6 in our survey asked:
How do you discover new onion sites?
The breakdown looks as follows. Respondents could select multiple answers, so the percentages are based on the total number of respondents.
Method Percentage -----------------------------------------------------------------------
From social networking sites such as Reddit or Twitter 50.67
I browse the list of onion site search engines such as ahmia.fi 50.50 I randomly encounter them while browsing the web 47.65 Recommendations from friends and family 19.46 Other (see below for common themes) 18.12 I am not interested in learning about new onion sites 4.19
The data shows that social networking sites, search engines, and "random encounters" are rather popular. Respondents who selected "Other" mostly brought up onion service lists and aggregators.
Following up, question 3.7 then asked:
Are you satisfied with the way you discover new onion sites?
61% selected "Yes" while the remaining 39% selected "No."
Some respondents who selected "Yes" brought up that they have no interest in learning about new onion services; in part because they only use Facebook's (or some other) onion service.
Among the respondents who selected "No," there are a bunch of reoccurring themes, in no particular order:
- The most prominent complaint was about broken links on onion site lists. There is non-trivial churn among onion sites and our respondents were frustrated that existing lists are typically not curated and contain many dead links.
- Many respondents were not aware of search engines such as ahmia.fi. Among those that were, many were not satisfied with both the search results and the number of indexed onion sites. Unsurprisingly, a "Google for onion sites" was a frequent wish.
- Several respondents were unhappy with existing aggregators. In addition to broken links, some distrust lists because they occasionally contain scam and phishing sites. The difficulty of telling apart two given onion domain names exacerbates this issue.
- Some respondents would like aggregators to be more verbose in their description of onion sites. In particular, these respondents were trying to avoid illegal and pornographic content, which is often difficult if the description is vague and the onion domain reveals nothing about its content.
- Many respondents expressed frustration about the difficulty of finding out if site X also provides a corresponding onion service. A common wish was to have site X list its onion service prominently in a footer. Ironically, some respondents were surprised that torproject.org has a corresponding onion site -- they couldn't find it on the web site.
- Two respondents compared the current state of onion services with the web of the 90s: Few sites existed, they linked to each other only sparsely, and search engines were experimental at best.
- Interestingly, some respondents voiced frustration about various usability issues, but mentioned in the same sentence that this is an inherent trade-off of privacy technology, suggesting that there is nothing that can be done about it.
There are two potential solutions that would address some of the above issues:
- Have next-gen onion services opt-in to a broadcast mechanism that automatically propagates them. Naturally, we would like such a mechanism to be censorship-resistant and built in a way that only the owner of an onion service is authorised to broadcast their service.
- Websites could use an HTTP header to announce the existence of a corresponding onion site. This issue was discussed in Feb 2017 over at tor-onions. Someone brought up the Alt-Svc header as a potential solution [1]. In a subsequent survey question we asked if our respondents would appreciate an automatic redirect from a web site to its corresponding onion site. The overall tendency leaned towards "Yes," provided that the implementation is sound and users can override the redirect.
Again, it's important to take these results with a grain of salt. Our data has some survivor bias: Presumably, we mostly heard from people who are Tor users despite usability issues. We likely didn't hear from many people who once experimented with Tor or onion services, decided it's not usable enough, and gave up.
The above was joint work with my colleagues Marshini Chetty, Annie Edmundson, Nick Feamster, and Laura M. Roberts.
[0] https://nymity.ch/onion-services/ [1] https://lists.torproject.org/pipermail/tor-onions/2016-February/000045.html
On Tue, Oct 03, 2017 at 08:25:15PM -0400, Philipp Winter wrote:
- Many respondents were not aware of search engines such as ahmia.fi. Among those that were, many were not satisfied with both the search results and the number of indexed onion sites. Unsurprisingly, a "Google for onion sites" was a frequent wish.
Someone at the Tor dev meeting brought up that Google is indexing tor2web.org including all the onion sites it knows about. That means that we can use Google as a search engine for (a subset of) onion services by searching for "site:onion.to foo".
That sounds terrabad. Can we finally set fire to tor2web? It was never a good idea.
On Sat, Oct 14, 2017 at 10:45:18AM -0400, Philipp Winter wrote:
On Tue, Oct 03, 2017 at 08:25:15PM -0400, Philipp Winter wrote:
- Many respondents were not aware of search engines such as ahmia.fi. Among those that were, many were not satisfied with both the search results and the number of indexed onion sites. Unsurprisingly, a "Google for onion sites" was a frequent wish.
Someone at the Tor dev meeting brought up that Google is indexing tor2web.org including all the onion sites it knows about. That means that we can use Google as a search engine for (a subset of) onion services by searching for "site:onion.to foo". _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Sat, 14 Oct 2017 15:12:05 +0000, dawuud wrote:
That sounds terrabad. Can we finally set fire to tor2web? It was never a good idea.
Why? There is exactly nothing that would stop google from actually indexing .onion domains (it knows about), like it now is with onion.to, even thought that happens a) accidentally and b) requires there being links to .onion.to instead of to .onion on the clearweb.
- Andreas
Plaintext communications intermediaries like tor2web violate the end to end principle and the principle of least authority. If we as the Tor community are committed to human rights then it follows we would abolish terrible things like tor2web or at least frown upon it's use.
I am frowning so hard right now.
Sincerely,
David Stainton
On Sat, Oct 14, 2017 at 06:50:17PM +0200, Andreas Krey wrote:
On Sat, 14 Oct 2017 15:12:05 +0000, dawuud wrote:
That sounds terrabad. Can we finally set fire to tor2web? It was never a good idea.
Why? There is exactly nothing that would stop google from actually indexing .onion domains (it knows about), like it now is with onion.to, even thought that happens a) accidentally and b) requires there being links to .onion.to instead of to .onion on the clearweb.
- Andreas
-- "Totally trivial. Famous last words." From: Linus Torvalds <torvalds@*.org> Date: Fri, 22 Jan 2010 07:29:21 -0800 _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On 14 October 2017 at 19:43, dawuud dawuud@riseup.net wrote:
Plaintext communications intermediaries like tor2web violate the end to end principle and the principle of least authority. If we as the Tor community are committed to human rights then it follows we would abolish terrible things like tor2web or at least frown upon it's use.
I would recommend continuing to enable/support Tor2Web, or at least not moving to make such a solution inoperable.
Dawuud is absolutely right re: violation of E2E* and a bunch of other criticisms also apply; however I have three observations on this topic:
1) Someone invented Tor2web, therefore someone else is likely to want to reimplement it; ideas tend to persist in this way
2) (as observed above) Google *do* crawl onion sites via "onion.to", which is a fun surprise for people who insist that "The Dark Web Is Not Indexed And Is Therefore Spooky"
3) Making such a move to block Tor2web-like sites might engender false trust amongst the people who set up Onion sites: "It's Okay, Google Can't Get At Us"
I would recommend investing more effort in Tor2web/similar, because having a permeable barrier between IP-Space and OnionSpace appears useful.
At very most I might propose that:
a) OnionSites become aware of the X-Tor2web header which (from legit T2W instances, at least) permits the OnionSite operator to block or redirect the user to use a "proper" Onion network connection
b) That TheTorProject consider indexing known Tor2web sites and publish them, perhaps adding a feature to optionally block them from TorBrowser access**, thereby to prevent stupid intra-Tor deanonymisation loops
- a
*although speaking as a geek I believe that re-engineering T2W to support SSL via SNI-Sniffing would address this, it would be a gross and pointless hack, complicated still further by certificate issuance, and all reasonable use cases for which would be better addressed by running a local copy of Tor.
**the hardcore alternative of blocking them from being accessed by exit nodes causing a likely-intolerable argument.
On 15 Oct 2017, at 04:08, Alec Muffett alec.muffett@gmail.com wrote:
On 14 October 2017 at 19:43, dawuud dawuud@riseup.net wrote: Plaintext communications intermediaries like tor2web violate the end to end principle and the principle of least authority. If we as the Tor community are committed to human rights then it follows we would abolish terrible things like tor2web or at least frown upon it's use.
I would recommend continuing to enable/support Tor2Web, or at least not moving to make such a solution inoperable.
v2 onion service Tor2web would be easy for HSDirs to block, due to an implementation bug. We've chosen not to block it. But we haven't spent much time on fixing its bugs, either.
As far as I am aware, no-one is writing Tor2web for v3 onion services.
We have open tickets for protecting relays that handle onion service traffic from knowing both the client and service IP address.
So if anyone does write v3 Tor2web, they will need to write it so it: * uses a 3-hop path for all descriptors, because otherwise that can be used for a selective denial of service; * uses a 3-hop path to connect to intro and rend when a descriptor has the single onion service flag; * retry using a 3-hop path on failure (internal reachability or actual connection failure)
And I'm not sure whether we would merge this feature into core tor, due to the user security issues that David and others have mentioned.
T
-
Alec Muffett -
Andreas Krey -
dawuud -
Philipp Winter -
teor