tor-dev

tor-dev@lists.torproject.org

3580 discussions

Tor-related postdoc position available
by Ian Goldberg 19 Sep '11

19 Sep '11

In case you're interested but hadn't yet heard, I've got a postdoctoral researcher position available "in the field of privacy-enhancing technologies, preferably on the topic of privacy-preserving communications systems". If you (or someone you know) is finishing up a PhD and would like to work on PETs/Tor stuff with us Waterloo folks, drop me a line (noting "Postdoctoral position" in the subject line). Details are here: http://crysp.uwaterloo.ca/prospective/postdoc/ Thanks, - Ian

1 0

Building an automatic censorship-detection system for Tor
by Karsten Loesing 17 Sep '11

17 Sep '11

In the past years we observed sudden drops in Tor usage in certain countries. Some of these events may be attributed to countries specifically wanting to block Tor, as it enables their citizens to circumvent their censorship infrastructure. Other events likely result from country-wide Internet outages that are unrelated to Tor. Two such events happened in January 2011 in Iran [0] and Egypt [1]. The key point is that we can derive from observed Tor usage when something potentially interesting is going on in a country. We should be able to build a system that keeps track of our daily user estimates per country and automatically sends out a warning whenever there are sudden changes. George Danezis wrote "An anomaly-based censorship-detection system for Tor" [2]. George's detection code [3] uses our daily user estimates [4] as input and gives a list of sudden upturns and downturns per country as output. We integrated the results of his script in the metrics website [5] to demonstrate what events the detector recognizes as possible censorship events. For example, the detector would have warned in January 2011 about the events in Iran [6] and Egypt [7], shown as red dots for downturns. However, as one can see, George's script also detects quite a few false positives. Whenever there's a red or blue dot, the script would have issued a warning for a human to check. It would be neat to reduce these false warnings while still catching the really suspicious events. Want to help us make our censorship-detection system better? Any suggestion to improve George's algorithm or to come up with an alternative approach to detect possible censorship events in our data would be much appreciated! Let us know if we can help you get started. [0] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [1] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [2] https://metrics.torproject.org/papers/detector-2011-08-11.pdf [3] https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2718 [4] https://metrics.torproject.org/csv/direct-users.csv [5] https://metrics.torproject.org/users.html#censorship-events [6] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [7] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-…

3 2

Run Tor relay in background on OS X
by Doug Ransom 03 Sep '11

03 Sep '11

Hi, I was thinking of tweaking TOR and/or Vidalia so users can easily (i.e. w/o being a skilled unix sysadmin) to configure TOR to run at boot. Any thoughts or concerns before I poke at the source code? In the past I have had some success running TOR at boot using Launchd, but its hard and doesnt play nice with Vidalia. Doug Ransom doug.ransom(a)alumni.uvic.ca

1 0

Research problem: better guard rotation parameters
by Roger Dingledine 30 Aug '11

30 Aug '11

The next in my series of research blog posts is up: https://blog.torproject.org/blog/research-problem-better-guard-rotation-par… "What algorithm should we use to assign Guard flags such that a) we assign the flag to as many relays as possible, yet b) we minimize the chance that Alice will use the adversary's node as a guard?" --Roger

1 0

Exit Enclaving and the future of?
by Jacob Appelbaum 29 Aug '11

29 Aug '11

Hi, I'll try to sum up Exit Enclaving in no certain terms and hope that Nick or someone else will chime in to correct my assumptions. This should probably be formalized and put into the path-spec at some point. I was surprised to not see it in there to begin with but I probably just missed it. Anyway, without further delay... When Tor builds a circuit it does so with certain constraints in mind - most of these are described in the path-spec. Exit Enclaving is a set of client circuit building constraints that all go together. When a user wishes to connect to a given IP address, Tor will check this IP against all of the Tor nodes in the Tor consensus. If the IP address in question is a Tor node in the consensus, the Tor client will check the selected Tor node's exit policy. If the Tor node's exit policy allows connections to *itself* - that is the IP address of Tor node - we check that the requested port is allowed by that policy. If we find that this set of constraints is matched, we build a *four* hop circuit to the Tor node that has the same IP as our request. The Tor node that matches is the fourth hop and we force exiting to the service offered by that node. The user then exits from the Tor node that met the constraints and they're connected to the service offered by that Tor node. As a result, client security is greatly improved by using Tor as the full path to the services offered on that Tor node. Obviously, with a bit of firewall trickery, it would be possible to offer the services on another physical host but the general point still stands - the service on a given IP address may only be enclaved by an IP address that is a confirmed Tor relay. I think Exit Enclaving is confusing for the uninitiated. It is documented but not in the standard Tor ways and it appears to be one of the lesser understood things in Tor land. Here's the official documentation as far as I've found it : https://trac.torproject.org/projects/tor/wiki/doc/ExitEnclave Here's the important bits from that page: SocksPort 9050 # what port to open for local application connections SocksListenAddress 127.0.0.1 # accept connections only from localhost ORPort 9001 Nickname archivetpo ExitPolicyRejectPrivate 0 ExitPolicy accept 38.229.70.19:443 ExitPolicy reject *:* There are a few references to the term (Exit) Enclave in the torspec.git repo but none of them actually define how it works with absolute authority. At best we've described how to try to become one without actually explaining the essence concretely in a specification. I think we should add a paragraph to the man page as well as the torspec. I'd be happy to write it once we actually hammer out the current and pas client behaviour. There is a problem with the current Exit Enclaving approach and if anything, it's not the worst one to have. If the user attempts to connect with a name, we are unable to try to satisfy the constraints that require an IP address; so we just skip Enclaving entirely for that connection. After the first sucessful connection or the first time Tor resolves the name, we're able to take the above steps in our attempt to opportunistically enclave. Before that - the Tor client only knows the name and thus the IP address is a mystery. Generally this results in the first connection exiting via any other node that allows exiting to the target host. I think this is perhaps an anonymity concern as it shows that the Tor client has likely not connected to the host in question - an evil Exit might take note of this and watch for correlated streams over time, etc. Here's an example of a semi-popular website documenting their enclaving experience: http://www.gabrielweinberg.com/blog/2010/08/duckduckgo-now-operates-a-tor-e… Recently the CCC jabber server became an exit enclave and a hidden service, similar to the DuckDuckGo setup: https://twitter.com/#!/jabbercccde/status/107850926638895104 https://twitter.com/#!/jabbercccde/status/107850540842627072 As I stated previously, I think that the use of Exit Enclaves greatly improves client security. It reduces the chances of a network sniffer getting lucky, it helps to mitigate the severity of services getting crypto stuff wrong, etc. I think it would be nice to make this easier to set-up and configure for server administrators. I'd also like to make this more flexible for Tor clients and if possible, find some methods for making Tor clients automatically enclave on the first connection. There are a few ways that we might see Exit Enclaving happen on the first connection. It is possible to make the first connection enclave if the client is using the TorDNSPort; Tor will have resolved and cached the names used by all clients, connections by names should then Enclave and connections by IP will Enclave as well. Alternatively, having all connections to Tor happen by IP address only would force all clients to Enclave on first connection. I think this is probably dangerous in all cases except a Transparent Tor network. Another possible hack is to set MapAddress for the hosts that matter to you - this doesn't scale very well and it's confusing for most people; it isn't dynamic and it seems that it will likely desynchronise over time. I'd like to advocate for a simple set of options to make this stuff easier to use and more likely to be used by a server admin. Here's a bug where I suggest an option OfferExitEnclave that is either a bool or a port list: https://trac.torproject.org/projects/tor/ticket/800 My thought was that a service provider who wants to offer enhanced service to Tor users could set a single flag and it would create a Tor Exit node that only exits to itself; I imagine that it might make sense to change the advertised bandwidth. It might even be a good idea to set a reasonable bandwidth cap of sorts but I think this is probably not great until we can automatically guess intelligently. I imagine that flipping the hypothetical OfferExitEnclave switch would actually be short hand for the following: ORPort auto Nickname exitenclave MaxAdvertisedBandwidth 20KB ExitPolicyRejectPrivate 0 ExitPolicy accept ORListenConfirmedPublicAddress:* ExitPolicy reject *:* Thus when we want to help someone enclave their entire host - we tell them to flip a single flag like so: OfferExitEnclave 1 If they want to enclave a specific service on their host - we tell them to flip the same flag like so: OfferExitEnclave 1 ExitEnclavePorts 5222 I'm not sold on that set of options but I think offering at least the OfferExitEnclave option as a bool would result in a lot more clarity for a server admin who wants to enhance their services for use with Tor. I think that generally, the nice part about Exit Enclaving is that it works as well as the rest of Tor - it does not really rely on external dependencies at all - no need for DNS or DNSSEC stuff, no need for the authorities to do any extra work on a per node basis, the client can use a different exit if they like, etc. On the client side, I think that would could make some improvements that aren't merely cosmetic but when enabled would improve client security. As I've already stated, Transparent Tor networks with TorDNSPort do this automatically by the nature of how they function. Other than the Torouter and probably TAILS, I'm not sure of other projects that takes advantage of this option. I'm not even certain that it will work in the long term for Torouter because of SRV records and jabber clients. Generally the client situation as it stands today lacks a general switch to force attempts at Exit Enclaving by name. Such an option would force a full round trip. Certainly we lack a general manner that attempts to Exit Enclave for all hosts and I'd like to see that change. I think it would be nice to have a few options and I'm far from sold on any of them. I'll lay them out as if they were already implemented. LearnExitEnclaveNames 0|1 This option could learn what possible names are frequently used and opportunistically resolve them, perhaps even caching some meta-data about exit enclaves that we've used, if it was stable, etc - this is could be something like a mash up TrackHostExits, LongLivedPorts, MapAddress, and something that learns overtime the hosts you use regularly. I think it would need some work to be done in a privacy preserving manner, probably with something like scrypt, if it needed to be cached on disk. This would solve the problem of a host being a known Exit Enclave after the first resovle and then later expiring from Tor's internal DNS cache. I think this is a common problem for IM connections and simply marking that name as something worth the round trip would be a huge gain, even if only for a single Tor run where the results are never cached on disk. My Tor often runs for days at a time and I think that the DNS cache internally expires before my Tor client exits. I'm pretty sure that means that it's up to my IM client or other software to cache the IP or we're back to square zero. AttemptExitEnclaving 0|1 This option could simply force the full round trip for all connections. Simple enough but likely a major performance hit. The default would be zero and keep things as they are today. AttemptExitEnclavingPorts [port list] This option could attempt Enclaving for any connections to any host if the connection is to one of the listed ports. AlwaysAttemptEnclaveNames [list of names] This option could take a list of names where we want to force a full round trip and that we'd like to enclave. Perhaps it would even fail to work if the Enclaving would fail and perhaps it could fail in a helpful manner. I imagine that some people might like this for customised browser bundles or if they're a power user who knows what they want, etc. I know that Nick had some suggestions - perhaps adding known Exit Enclaving names into the consensus or into descriptors or something that the directory authorities handle. I'm not sure that I fully understand his suggestions and I look forward to his reply to this email. :) All the best, Jake

1 0

lround() missing in MSVC
by Gisle Vanem 24 Aug '11

24 Aug '11

lround() is missing in MS Visual-C's <math.h>. Not available anywhere. Here is an easy patch: --- Git-latest\src\common\util.c Sun Aug 14 13:48:49 2011 +++ src\common\util.c Tue Aug 23 21:15:25 2011 @@ -335,7 +335,11 @@ long tor_lround(double d) { +#ifdef _MSC_VER + return (long)(d > 0 ? d + 0.5 : ceil(d - 0.5)); +#else return lround(d); +#endif } /** Returns floor(log2(u64)). If u64 is 0, (incorrectly) returns 0. */ ---------------- --gv

2 1

Tor2web 2.0 is live!
by Arturo Filastò 20 Aug '11

20 Aug '11

Hi all, We are glad to announce the release of the new tor2web software. For those of you who are not aware of what tor2web is let us give you a brief description. The goal of tor2web is that of promoting the use of Tor Hidden Services (https://www.torproject.org/docs/hidden-services.html.en). Hidden Services allow people to run TCP based services without disclosing the identity/location of their server. In the specific they allow people to anonymously publish content to the web. Also, since you are being reached trough the Tor network, you are not required to have a static ip address or purchase a domain. This lowers the entry barrier to content publishing and protect the content publisher from retaliation and Denial of Service attacks. The problem though is that Hidden Services are usually only accessible by installing a Tor client (https://www.torproject.org/projects/torbrowser.html.en). Tor2web creates a transport, by acting as a web proxy, between the internet and the Tor network. This means that anonymous publishers are able to reach a much wider audience. The user visiting a website though tor2web is always advised to install a Tor client as by doing so he will protect his identity and leverage Hidden Services end-to-end encryption. This version of tor2web (called tor2web 2.0) is based on glype PHP web proxy (http://www.glype.com) and it is by no means the definitive solution. We are currently working on a new design that will be able to withstand other "attacks" that are currently possible. What we have implemented is: * A clear disclaimer warning the user that the content is not being served directly from the server, but it comes from the Tor network * Contact forms for abuse complaints and to report broken websites * Transparent rewriting of URLs into the tor2web form (i.e. so4rmjdiwmqjosxz.onion become so4rmjdiwmqjosxz.tor2web.org) * Blocklists to allow a tor2web node maintainer to block particular websites, the blocklists are stored in md5 format so the node maintainer does not need to store potentially illegal site lists. At this current stage we would like the community to stand-up and help us by: * Finding security and functional bugs in the existing implementation * Volounteering to run new tor2web servers: In this first stage we are looking for reliable systems, run or endorsed by trustworthy organizations involved in anonymity and privacy research and development. For the new release the goals that we wish to further pursue are: * Distribute responsibility across multiple actors * Minimize the probability of takedown of a tor2web node If you want further information on the tor2web project visit: Wiki for new developments: http://wiki.tor2web.org/ Tor2web original website: http://www.tor2web.org Github: https://github.com/globaleaks/tor2web-2.0 Mailing List: tor2web-talk(a)lists.tor2web.org on http://bit.ly/pxFwNS . IRC: irc.oftc.net #tor2web Have a nice day, Some Random GlobaLeaks Contributors Please spread across the anonimity communities and mailing lists

1 0

Blocking-Test: GSoC Penultimate Status Report
by Brandon Wiley 12 Aug '11

12 Aug '11

Scoring and reporting for detectors and encoders is pretty much done. The graphs aren't quite ready yet, for aesthetic reasons, but I have some nice tables to report: Length detector: 16% accuracy Timing detector: 89% accuracy Entropy detector: 94% accuracy SSL was correctly identified 25% of the time, 70% of the time other protocols were misidentified as being SSL, and 5% of the time SSL was misidentified as another protocol. obfsproxy was correctly identified 55% of the time, 10% of the time other protocols were misidentified as being obfsproxy, and 35% of the time obfsproxy was misidentified as another protocol. Dust was correctly identified 48% of the time, 4% of the time other protocols were misidentified as being Dust, and 48% of the time Dust was misidentified as another protocol. obfsproxy is distinguishable from SSL 96% of the time. obfsproxy is distinguishable from Dust 98% of the time. Dust is distinguishable from SSL 56% of the time. I'm sure these numbers are confusing as several different sorts of things are being measured, all with similar numbers. It is probably quite unclear how all of these percentages add up. I think this is where the graphs will help. In particular I think it's hard to visualize accuracy when there are both false positives and false negatives to take into account. I think the most interesting result here is the entropy detector. It's the simplest and also the most effective. Not only that, but the entropy detector only looks at the entropy of the first packet, so it's inexpensive. The original reason that I implemented this detector was that there was an argument against Dust that it could be trivially filtered by setting an entropy threshold above which all traffic is filtered. Surprisingly, Dust has lower entropy than SSL, so this attack will not work to specifically target Dust. This is unexpected because Dust uses totally random bytes, whereas the SSL handshake does not. This is a result of a general issue with how Shannon entropy is defined. Shannon entropy is normally defined across a large statistical sampling, in which case all three protocols tested will converge to maximum since they are all encrypted and therefore apparently random. However, the attacker does not in this case get a large statistical sample. Since they don't know a priori which connections are Dust connections, they only get however many packets are in each trace to use as samples. The current version of Dust that I'm testing has most of the advanced obfuscation stripped out as I haven't had time to add it back in after completely reimplementing the protocol to use TCP instead of UDP and all of the modified assumptions that come with that change. So this version of Dust is basically just the ntor protocol with no packet length or timing hiding in use. The first packet is therefore 32 random bytes consisting of the curve25519 ephemeral ECDH key, as compared to the 1k first packet of SSL. This is the reason for the low entropy calculation, because given only 32 bytes to sample almost all possible bytes are going to have a probability of 0 or 1. There is an upper bound on entropy when sample sizes are small. Achieving maximum first-order entropy here would require at least 512 bytes so that each value 0 to 255 could occur exactly twice. There are of course many ways to measure entropy and you could say that I'm just doing it wrong. Some alternative measurements have already been suggested that I might implement as well. I am not suggesting that Dust is in any way immune to entropy-based detection. In fact, Dust is currently detected very well by the current entropy detector (it passed 100% of the packet length and timing tests, so all detection of Dust was done by the entropy detector) precisely because it has strangely low entropy because of it's small first packets. I bring this up only as an interesting example of how intuition can fail when trying to use Shannon entropy to measure individual messages instead of channels. As far as the state of the project goes, I think the project can be considered done. Looking at the project documentation<http://stepthreeprivacy.org/documentation>, everything didn't work out exactly as originally planned, I think it all worked out for the best and the project is where I wanted it to be by the completion of Summer of Code. I have generation of HTTP and HTTPS traffic, both over Tor and plain. I have two encoders: obfs2 and Dust. I have three detectors: length, timing, and entropy. The string detector didn't work out, but the entropy detector worked out better than planned, so on balance I think it was a win. I didn't implement all of the detector modes because upon further reflection it doesn't really make sense to just try all of these different modes. You want the mode that works best. So for packet lengths and timing that is full conversations and for entropy that's the first packet (could be first N packets, but I tried N=1 and it worked great). Random sampling of packets just seems like a way to limit your success at this point when non-random sampling of just the first packet seems to be so effective. All of the utilities have been written, and more. I have the specified utilities for separating the pcap files into streams, and extracting the strings, lengths, and timings. Additionally, I have some utilities for extracting the entropy of streams, tagging the streams with what protocol they are using (by port), and extracting dictionary and corpus models for latent semantic analysis. I also have some utilities for graphing distributions of properties of different protocols which I hope to finish up next week during the wrap-up period. Looking to the future, I have changed my goals somewhat from what I had in mind at the beginning of the project. After having attended the TorDev meetup in Waterloo and the PETS conference, I am much more interested in integrating my work with the Tor project instead of just testing the various protocols that have been proposed in order to see how my own protocol stacks up against the competition. As part of this transition, I have rewritten Dust from scratch, both in terms of implementation and in terms of the protocol, in order to be better suited for use as a pluggable transport for Tor. My goal for Dust is to have it shipped with both Tor clients and bridges so that it can be used in the field to contact bridges despite DPI filtering of Tor connections. My goal now for the blocking-test project is to focus on blocking resistance for Tor. So instead of adding all of the protocols under the sun, I'm going to concentrate on protocols that have been implemented or are under consideration to become pluggable transports. Additionally, the next protocol I'm going to add after GSoC is over is just plain Tor. There is at least one bug filed about a suspicion that Tor can currently be fingerprinted based on some packet characteristics and I'd like to be able to close that bug because we now have the capability to easily generate the necessary information. Finally, I would like to offer some insight into what I think is the future of blocking-resistant transports now that I have done some work trying to break them. Fundamentally, the only thing an attacker can do is limit the bitrate of the connection. Given any sort of channel, we can encode arbitrary information over that channel. However, the smaller the channel the slower the bitrate. The goal of encodings should be to maximize the bitrate given the constraints of the censor. It's easy to develop very slow encodings such as natural language encodings over HTTP. However, in order to maximize the throughput you need to have a good definition of the attack model. All encoding for blocking-resistant purposes lowers the efficiency above just transmitting the normal content (assumed it has already been compressed). Therefore all encoded conversations are going to be longer than unencoded conversations. I think this is the future of writing detectors because it's a fundamental constraint on encoding. I think this should even be able to defeat something like Telex (although let's not get into the details of attacks on Telex specifically, I'm talking about a general idea here). Given a (static, for the sake of argument simplicity) website, I can download every page on that website. Then I can compare the length of intercepted downloads and detect when they don't match up. For dynamic websites you can do the same thing with a statistical model. I think, therefore, that the future of blocking-resisting protocols is to encode single logical connections over an ensemble of multiple actual connections. Ideally these connections would mimic normal traffic in terms of number of connections, duration, directional flow rates, etc.. It's something to think about anyway. It could be an interesting problem.

1 0

Instructions for building the Vidalia-bundle?
by Steve Snyder 09 Aug '11

09 Aug '11

Is there documentation anywhere on how to build the Vidalia-bundle for Windows? If so, where can I find it? Thanks.

3 3

GSoC Update
by Brandon Wiley 05 Aug '11

05 Aug '11

With the TorDev meetup and everything it's been a while since I sent out an update on the state of the project, although I think we more or less caught up on the status at the meetup. The bulk of the implementation work had already been completed, so what I've been working on since then is just cleaning up the code and rewriting some parts. In particular this week I rewrote the detectors to be mostly python. The core Bayesian model is still in BUGS/JAGS and there is still R code to interface with that. However, the output of the models after training is now converted to a CSV file and the actually detection code is a python program which reads the CSV file with the trained model's prediction data and the compares it to the data of a stream to be classified. This means that you only need R for training the models. If we can put together a canonical model for each protocol then you can use it to classify new traffic just using python. I think this is a win because R was the bulkiest dependency. It also segments the process more nicely in that there can be some people gathering traces, other people building models, and then other people running detectors against new traces. In terms of code cleanup, I've been adding comments and refactoring large files (in particular pavement.py) to be more readable. I also have the scoring tasks implemented so that, given the output of the detectors, the various detectors and encoders are scored based on their number of correct guesses, false positives, and false negatives. The only functionality left to implement is some sort of reporting, probably in the form of graphs of the scores so that detectors and encoders can be compared. The rest of the work is just going to be refining everything and getting both the workflow and the code structure to be smooth. Also, I've stopped working on the string detector because I realized that it's irrelevant for the two encoders I actually have support for right now: obfs2 and Dust. They both already use totally randomized packet contents and so will not be vulnerable to this kind of detector. Other encodings will be vulnerable, but the string detector can wait until we support these encodings. So instead I've implemented an entropy detector, which should work very well against both obfs2 and the current implementation of Dust. The entropy detector is actually the easiest model so far as the entropy of a trace is just a single number, so that's convenient.

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev