tor-dev June 2013

tor-dev@lists.torproject.org

54 participants
48 discussions

Torsocks reengineered
by David Goulet 14 Jun '13

14 Jun '13

Hi everyone, About a week ago I've sent an email to the Torsocks maintainers to address some concerns I had about important issues of the current code base. For the reasons found below, I proposed a rewrite that I am willing to do and submit it to the community once I feel ok with it. After some back and forth, the consensus was that I should go ahead and do it. Best case scenario, I come up with a tested library that has a smaller set of problems (hopefully) and could eventually be considered to replace the current version. Worst case, I lose my time :). Now, I'm not looking to do that alone in my corner or anything, what I want is this effort to be as open as possible for everyone that want to help, contribute or/and follow it. Some help would be really appreciated for the compat. part (Windows, OS X, BSD, etc...). Here is the repository I've created and started working on the rewrite. It has been branched from upstream master at commit e0bf65b5d3ca0e28b827c08d80b0fe1841d5a149 https://github.com/dgoulet/torsocks/tree/rewrite There might be some rebasing going on in that branch in the next weeks so please don't consider it right now as "git stable" since I'm in the heavy part of getting everything together before starting a "normally growing" master branch. If you like to contribute or help, let me know and we'll make something work. Big thanks to nickm, ioerror and sysrqb for their help on all this. Cheers! David Torsocks rewrite reasons: --------------- (You can see that as either we put in 2000 lines of code to fix it in a ~3000 lines repository or we start fresh with some key aspect, maintainability and security in mind.) Code and Maintenance Issues: 1) The code needs to be easier to maintain, so that it isn't sporadically maintained. Considering the number of issues (see below), it needs more love on a regular basis right now. 2) It has never been safe from the start so a rewrite will NOT impact the current state of security. There is not much tests right now. 3) Small code base. Fix it clean before it gets too big. 4) One of the biggest technical issues is that it's not thread safe and fixing it right now would add an important impact on performance adding locks to multiple global data structures. A clean rewrite would allow to take into account this *very* important feature from the start without any ugly hacks in the current code base. 5) This is a very, and I can't stress this enough, _VERY_ intrusive library so extra care MUST be put in making it bullet proof in terms of error handling. Right now, there is multiple call sites that can fail on error. 6) There is not even close to enough tests, thus not loosing that stable portion of a project. 7) No compatibility layer is present for multiple OS and architecture. 8) FD passing through Unix socket support. * Should at least be detected and notify the user or even deny execution since the process receiving the socket is not under torsocks control. 9) Symbol name space polluted. (should all be prefixed). 10) Libc hijacked calls of torsocks don't return correct values in error path. * This is bad because for instance the caller expect errno to be set and makes decision upon that value that MUST be right once returned. There is multiple call sites that are problematic. 11) IPv6 support. ----------------

7 11

Tor over Open Garden mesh network
by Paige Peterson 13 Jun '13

13 Jun '13

Hello all, I was recommended to join this group after emailing the general help address with the following message: I am the Community Manager for Open Garden <https://opengarden.com>, a p2p mesh networking application for phones, tablets and laptops. Currently, Tor doesn't work over Open Garden and when talking to my lead developer about it a while ago, he said that Tor would need to support an upstream proxy - which he can't find any documentation on. Some users of Open Garden have requested the ability to use Tor over the Open Garden protocol and as the Community Manager, I am attempting to understand whether or not this is feasible to implement. If there is any light that can be shed from your side as to what would need to happen to support each other, then please enlighten me! If it would be more productive to have a conversation with our lead developer, then I can ask him to join the list as well. Thanks for your time! -- Paige Peterson Open Garden Community Manager forum.opengarden.com

2 2

Building better pluggable transports - GSoC 2013 project
by Chang Lan 11 Jun '13

11 Jun '13

Hello everyone! I am a Tor GSoC student who will be working on the pluggable transports this summer. My mentor is Steven and my co-mentor is George Kadianakis. It is great to be part of the Tor community! Steven already kicked off the discussion about how to build better transports. The original project proposal[1] discussed the possibility of sending data over UDP with extra efforts to guarantee reliable in-order delivery. However, as George mentioned recently[3], ScrambleSuite[2] may already solve the issue of scanning resistance. Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome! I'll be keeping my GSoC log here: http://changlan.github.io/obfsproxy I'm also available #tor-dev@oftc with the handle "clan", feel free to ping me there! [1]: http://changlan.info/gsoc.html [2]: http://www.cs.kau.se/philwint/scramblesuit/ [3]: https://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/changla… Cheers, Chang

5 6

Help Wanted: Evaluate our Tool for (privacy-preserving) Automated Debugging
by Christian Grothoff 11 Jun '13

11 Jun '13

Hi! Collecting good bug reports from users is a challenge, as most users are unable (or unwilling) to inspect problems in depth. Existing tools to automatically report program failures are often problematic as they either produce superficial reports or are prone to leaking private information (i.e. by including a full snapshot of the process address space in the form of a core dump). At TUM, we are developing Monkey[1], a free software tool that automatically generates high-quality privacy-preserving bug reports (for C code) which we hope to use for GNUnet and Tor eventually. (It runs with GNUnet and Tor already, but might need more work prior to real-world use.) We have reproduced and analyzed a set of real-world bugs using the system and would now like to study how well developers can find the bugs using the extracted information. Please participate in our public "survey" (or rather, experiment) where we measure your performance at identifying bugs based on crash reports collected via different methods, including traditional stack traces, core dumps (accessed via a live gdb session) and of course Monkey. So if you can spare the time, please help our efforts by participating in our survey[1] and passing the link to your colleagues and friends. (Who finds more bugs faster?) [0]: https://gnunet.org/monkey/ [1]: https://gnunet.org/monkey/survey/index.php/943594?lang=en Happy hacking! Christian Grothoff & Markus Teich p.s.: We'll be happy to talk with you about Monkey at the Tor developer meeting in Munich; however, Markus's thesis deadline is earlier...

1 0

Re: [tor-dev] grabbing Tor circuit (node) data- Tor stem, torrc and Tor control port
by Damian Johnson 11 Jun '13

11 Jun '13

Hi Sarah. I'm not really sure what you're trying to ask in most of these questions. Assuming that your goal is simply 'I want to connect to port 9051 and dump information about all the relays I'm presently using in my circuits' then the following should do the trick. Take the following with a grain of salt - I just wrote it, I haven't tried running it. Should be close though. ;) from stem.control import Controller with Controller.from_port(port = 9051) as controller: my_relay_fingerprints = [] # fingerprints of all relays in our circuits for circ in controller.get_circuits(): my_relay_fingerprints += [fp for (fp, nickname) in circ.path] for fingerprint in my_relay_fingerprints: desc = controller.get_network_status(fingerprint) country = controller.get_info("ip-to-country/%s" % desc.address, "unknown") print "relay: %s:" % fingerprint print " address: %s:%s" % (desc.address, desc.or_port) print " locale: %s" % country Did you have any other questions? On Mon, Jun 10, 2013 at 9:43 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > Damian, thanks, this is very helpful. > > is there a way to do this in torrc? Else, i suppose i will need to: > > 1) create a socket or connection to my port 9051 ; do i need/can i use > TORRC_CONTROL_SOCKET ? > 2) call get_circuits() ; grab the relay fingerprints > Do I need circuit = controller.get_circuit(circ_id) > 3) return class:`stem.events.CircuitEvent` ; for the given circuit > Not sure whether or where to use the "path" attribute > 4) call controller.get_network_status() to get IP address, nickname, ORPort, > Should I use: > desc_by_fingerprint = controller.get_network_status(test_relay.fingerprint) > > + desc_by_nickname = > controller.get_network_status(test_relay.nickname) > > 5) use Maxmind- i already have the GeoIPLite DB to grab AS and country, and > onion code also from Arturo > > > Any guidance is appreciated > > > https://lists.torproject.org/pipermail/tor-commits/2012-December/051174.html > > get_circuit(self, circuit_id, default = UNDEFINED): > + """ > + Provides a circuit presently available from tor. > + > + :param int circuit_id: circuit to be fetched > + :param object default: response if the query fails > + > + :returns: :class:`stem.events.CircuitEvent` for the given circuit > + > + :raises: > + * :class:`stem.ControllerError` if the call fails > + * ValueError if the circuit doesn't exist > + > + An exception is only raised if we weren't provided a default > response. > + """ > + > + try: > + for circ in self.get_circuits(): > + if circ.id == circuit_id: > + return circ > + > + raise ValueError("Tor presently does not have a circuit with the id > of '%s'" % circuit_id) > + except Exception, exc: > + if default: return default > + else: raise exc > + > def get_circuits(self): > """ > Provides the list of circuits Tor is currently handling. > > > > On Jun 10, 2013, at 10:34 AM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > > They certainly are! If you're interested in tor and development then I > would definitely suggest being on that list. Including it for this > thread. > > I am trying to figure out how to pull in the nodes that are actually used in > my Tor circuits. They are the nodes reflected in the Network map function. > > > You want the get_circuits() method. As you mentioned the 'path' > attribute has the relays in your present circuits... > > https://stem.torproject.org/api/control.html#stem.control.Controller.get_ci… > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. > > > That certainly sounds painful. The circuit paths will provide the > relay fingerprints which you can use for get_network_status() to get > the address, nickname, ORPort, etc... > > https://stem.torproject.org/api/control.html#stem.control.Controller.get_ne… > > As for locales that would be done via get_info('ip-to-country/<address>')... > > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672 > > ... and ultimately to AS and country. > > > AS will require the Maxmind AS database or something else. I know that > Onionoo includes the AS information so the options that come to mind > are either to (a) see how it does it or (b) query Onionoo for this > information. > > https://onionoo.torproject.org/ > > And i have read much of the control-spec, don't know how much is out of date > > > The control-spec should be up to date. If there are any inaccuracies > then please let us know! > > Did you have any other questions? -Damian > > On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > For example, I am trying to figure out how to pull in the nodes that are > actually used in my Tor circuits. They are the nodes reflected in the > Network map function. I have read your fine stem tutorials, it is so > impressive you have dome so much work and documentation on your own > initiative. It is certainly a huge help to me and otter researchers like me. > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. I am analyzing > their country, so I am seeking to map nickname or fingerprint of the node to > IP address and ultimately to AS and country. Ultimately, I am analyzing > jurisdictional arbitrage and the hostility factors of differ countries to > Tor traffic, and whether it is possible to incorporate country or AS in to > the Tor path algorithm. > > I can see the entry guard nicknames and fingerprints in my state file and > pick them up there, if necessary. Wish i could just edit the logs to display > all nodes, then figure out how to map the node fingerprint or nickname to > its IP address them AS and country. I understand there is a Tor atlas > function but have not read it carefully yet. > > I have read documentation on the contents of the consensus file, the > descriptors, geoip and state files, and played around with them a bit to > understand them. I grasp that a fingerprint is a hash of the node's public > key, and one or both of which seem to be the most reliable ID for a node. > > I can see from your documentation that variable path(nickname, fingerprint) > contains the data i want. Just need to be able to get that data easily. Not > sure if editing my torrc file is the way or what. If you have any > suggestions, they are much appreciated. > > And i have read much of the control-spec, don't know how much is out of date > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt > > I have read this on modifying torrc, and played around with modifying torrc, > on an elementary basis: > https://www.torproject.org/docs/faq.html.en#torrc > > and some tickets on modifying torrc > https://trac.torproject.org/projects/tor/ticket/6147 > > and what I believe is your excellent documentation on stem events; > https://stem.torproject.org/api/response.html#stem.response.events.Bandwidt… > > > > > > > > On Jun 8, 2013, at 2:31 PM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi Sarah. Yup, I'd be more than happy to help if you have any stem > questions. Email would definitely be better since we look to keep > missing each other on irc. I'd appreciate it though if we discussed it > on the tor-dev@ email list... > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev > but was unable to reach you. We met at the tor dev meeting which i attended > in March. I have some Qs for you regarding stem. Please let me know if you > are available, thanks > > >

2 1

traversing the source code of tor
by H S 10 Jun '13

10 Jun '13

Hi there, We would like to implement some specific protocol in Tor as an implementation project. Our one of pseudo codes that we have to take into account is the one mentioned on the paper under the link http://www.cypherpunks.ca/~iang/pubs/UC-OR.pdf , page 5, figure 2. I would like to know how can I connect or relate this pseudo code with the Tor source code to help us understand where we should insert our desired modification on the source code of the Tor. For instance in the first of the pseudo code line we have "upon an input (setup):". And know in which file/lines of the Tor source code we have this?

2 1

Re: [tor-dev] stem
by Damian Johnson 10 Jun '13

10 Jun '13

> Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. They certainly are! If you're interested in tor and development then I would definitely suggest being on that list. Including it for this thread. > I am trying to figure out how to pull in the nodes that are actually used in > my Tor circuits. They are the nodes reflected in the Network map function. You want the get_circuits() method. As you mentioned the 'path' attribute has the relays in your present circuits... https://stem.torproject.org/api/control.html#stem.control.Controller.get_ci… > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. That certainly sounds painful. The circuit paths will provide the relay fingerprints which you can use for get_network_status() to get the address, nickname, ORPort, etc... https://stem.torproject.org/api/control.html#stem.control.Controller.get_ne… As for locales that would be done via get_info('ip-to-country/<address>')... https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672 > ... and ultimately to AS and country. AS will require the Maxmind AS database or something else. I know that Onionoo includes the AS information so the options that come to mind are either to (a) see how it does it or (b) query Onionoo for this information. https://onionoo.torproject.org/ > And i have read much of the control-spec, don't know how much is out of date The control-spec should be up to date. If there are any inaccuracies then please let us know! Did you have any other questions? -Damian On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > For example, I am trying to figure out how to pull in the nodes that are > actually used in my Tor circuits. They are the nodes reflected in the > Network map function. I have read your fine stem tutorials, it is so > impressive you have dome so much work and documentation on your own > initiative. It is certainly a huge help to me and otter researchers like me. > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. I am analyzing > their country, so I am seeking to map nickname or fingerprint of the node to > IP address and ultimately to AS and country. Ultimately, I am analyzing > jurisdictional arbitrage and the hostility factors of differ countries to > Tor traffic, and whether it is possible to incorporate country or AS in to > the Tor path algorithm. > > I can see the entry guard nicknames and fingerprints in my state file and > pick them up there, if necessary. Wish i could just edit the logs to display > all nodes, then figure out how to map the node fingerprint or nickname to > its IP address them AS and country. I understand there is a Tor atlas > function but have not read it carefully yet. > > I have read documentation on the contents of the consensus file, the > descriptors, geoip and state files, and played around with them a bit to > understand them. I grasp that a fingerprint is a hash of the node's public > key, and one or both of which seem to be the most reliable ID for a node. > > I can see from your documentation that variable path(nickname, fingerprint) > contains the data i want. Just need to be able to get that data easily. Not > sure if editing my torrc file is the way or what. If you have any > suggestions, they are much appreciated. > > And i have read much of the control-spec, don't know how much is out of date > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt > > I have read this on modifying torrc, and played around with modifying torrc, > on an elementary basis: > https://www.torproject.org/docs/faq.html.en#torrc > > and some tickets on modifying torrc > https://trac.torproject.org/projects/tor/ticket/6147 > > and what I believe is your excellent documentation on stem events; > https://stem.torproject.org/api/response.html#stem.response.events.Bandwidt… > > > > > > > > On Jun 8, 2013, at 2:31 PM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi Sarah. Yup, I'd be more than happy to help if you have any stem > questions. Email would definitely be better since we look to keep > missing each other on irc. I'd appreciate it though if we discussed it > on the tor-dev@ email list... > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev > but was unable to reach you. We met at the tor dev meeting which i attended > in March. I have some Qs for you regarding stem. Please let me know if you > are available, thanks > >

2 1

Gitian-based Deterministic Build System for TBB (Need MacOS Help!)
by Mike Perry 10 Jun '13

10 Jun '13

Over the past couple weeks I've been redoing the TBB build system to use Gitian to produce alpha TBBs using Tor Launcher instead of Vidalia. I have succeeded in producing deterministic, localized builds of TBB for Linux and Windows. This means that independent people all over the world can now easily produce their own bundles for these platforms fresh from sources, and have their bundles exactly match the bundles the Tor Project releases, down to the SHA256 hash. If we leverage this property wisely, it will allow us to defend against targeted attacks against our bundlers and their build machines, and even ultimately ensure the integrity of our bundles in the event of key compromise of the gpg keys used to sign the bundles. My plan for this is for there to be between 2-3 official signers for each bundle, where each person produces their build independently, and signs the (identical) result files. To further protect against targeted attack, in addition to these 2-3 official signers, we need some people to be "secret verifiers". Ideally these people would not be publicly affiliated with the Tor Project, but would still produce their own bundles anyway. If their SHA256 ever fails to match the signed bundles, that person should anonymously open a trac ticket (using the cypherpunks account) and attach the bundle files that differ for analysis. The differing files can be found easily enough with 'diff -r'. To ensure the existence of these "secret verifiers", I believe that the official signers should occasionally conspire to conduct "Fire Drills", where they all agree to alter the bundle in some innocuous way (such as adding whitespace to a config file or a Firefox JS file), and ensure that a verifiers anonymously report the verification failure. In future versions of Tor, we should probably add a consensus field consisting of a url to a file that lists the current recommended bundle hashes and versions, along with the current SHA256 of that file, to anchor the bundle authentication the Tor's current trust root (the 9 dirauth keys). To try out the new build system, please see the README, and let me know where the system could use clarification or improvement to make it easier to use: https://gitweb.torproject.org/builders/tor-browser-bundle.git/blob/HEAD:/gi… The build system has some quirks that are worth mentioning: 1. It requires you run it from either an Ubuntu 12.04 or above host with KVM support, *or* you run it from an Ubuntu 12.04 or above chroot/VM. The bundle scripts try to detect your current situation and suggest that you "export USE_LXC=1" from your shell if you need to, to cause the system to use LXC instead of KVM (so that you can build from an Ubuntu VM or on a machine that does not otherwise support KVM). 2. We currently have no MacOS support. To support MacOS, we need to create cross-compilers for it so that we can produce builds from the Gitian VMs (which again are Ubuntu). A few people have done this. I have sent them mail asking for instructions on how to reproduce their compiler packages: http://www.tarnyko.net/en/?q=node/9 https://launchpad.net/~flosoft/+archive/cross-apple/+packages http://wiki.freepascal.org/Cross_compiling_OSX_on_Linux Unfortunately, at least one of those URLs say that to produce a cross-compiler, you need access to an OSX SDK. Since I do not have a Mac that is currently supported by recent OSX SDKs, and since we *really* want to be sure that the cross-compilers we produce use code from a fresh known-good SDK install, I won't be doing this. Please let me know if you'd like to help tackle this problem. In the meantime, I am going to work on the rest of the "Short Term" TODO items, and produce official alpha bundles for Linux and Windows, so we can test Tor Launcher in an official alpha release. Here's the TODO file: https://gitweb.torproject.org/builders/tor-browser-bundle.git/blob/HEAD:/gi… Happy building! -- Mike Perry

3 6

Academic Project Guidance
by mahesh 09 Jun '13

09 Jun '13

Hi, I am final year undergraduate student at Pune Institute Of Computer Technology, Pune, India. I have to do a academic project in span of 6-months with other three students.I am looking for a mentor who would discuss ideas with us and provide proper guidance. Also I am also interested under working an organization's guidance. Please interested people let me know. Sincerely, Mahesh

1 0

Format-Transforming Encryption Pluggable Transport
by Kevin P Dyer 09 Jun '13

09 Jun '13

Last September I announced initial results [1] towards a framework, which we call Format-Transforming Encryption (FTE), for encoding messages using regular languages. We're excited to announce that we've made progress towards an implementation and would like to invite alpha testers. Our source code is available on github [2]. Tor Bundles are available for OSX/Linux [3] that include our FTE framework and are configured by default to work with FTE+Tor bridges deployed in the United States. Unfortunately we don't, yet, support Windows. The latest version of our paper is available on the Cryptology ePrint Archive [4]. In the paper we describe our framework and provide a comprehensive security evaluation of FTE's success in evading six DPI systems --- including using regular expressions from open-source DPI systems to evade detection by a closed-source black-box commercial DPI system. By "evade" we mean that it's easy for FTE to tunnel arbitrary TCP streams (e.g., Tor) such that they are (mis)classified by DPI systems as a configurable target protocol (e.g., HTTP, SMB, RTSP, etc.) of one's choosing. We release FTE in its alpha stage because we believe it is well positioned to evade the suspected protocol white-listing [5] recently reported in Iran. More generally, we're optimistic FTE has long-term potential as a tool to enable users to control how their traffic is classified by passive DPI systems. As one example, over the last month, we've successfully tunneled Tor through the Great Firewall of China, using FTE to make our traffic "look like" HTTP. We're eager for feedback on this alpha release, so please do not hesitate to contact us with questions. -Kevin P Dyer (and his co-authors) [1] https://lists.torproject.org/pipermail/tor-dev/2012-September/003993.html [2] https://github.com/redjack/FTE [3] https://github.com/redjack/FTE/tree/master/TorBundles [4] http://eprint.iacr.org/2012/494 [5] https://lists.torproject.org/pipermail/tor-dev/2013-May/004787.html

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev June 2013