Hi, all!
I've seen a couple of emails from people looking into new ways to do naming for onion services. That's great! Before anybody gets too far, I'd like to send this quick note to let you know that integrating stuff like this into Tor is actually easier than you think.
Here's how you do it, using a Tor controller. (See control-spec.txt for protocol documentation. Also see one of the several friendly libraries, like steam, that exist to interface with Tor over this protocol.)
First, you set the Tor option "__LeaveStreamsUnattached". This tells Tor that it shouldn't try to handle new client requests immediately, but it should instead let the controller take responsibility.
In the controller, you make sure that you are watching STREAM events so that you find out about new streams.
Whenever there's a new stream, you check its address. If the address is one that you don't want to rewrite, you just call ATTACHSTREAM on it, with a circuit ID of 0. (The 0 means "Tor, you figure out how to attach this one.".
Otherwise, you do whatever magic dance you do in order to find out the real address for the stream.
If the lookup operation is successful, you say "REDIRECTSTREAM (stream ID) (new address". And then you ATTACHSTREAM as above.
If the lookup operation fails, you call "CLOSESTREAM (stream ID) 2". (The 2 means "resolve failed".
And that's it for the Tor integration! All you need to do now is figure out how the name lookup works.
cheers,
On 2 Aug 2016, at 23:37, Nick Mathewson nickm@torproject.org wrote:
Hi, all!
I've seen a couple of emails from people looking into new ways to do naming for onion services. That's great! Before anybody gets too far, I'd like to send this quick note to let you know that integrating stuff like this into Tor is actually easier than you think.
Here's how you do it, using a Tor controller. (See control-spec.txt for protocol documentation. Also see one of the several friendly libraries, like steam, that exist to interface with Tor over this protocol.)
First, you set the Tor option "__LeaveStreamsUnattached". This tells Tor that it shouldn't try to handle new client requests immediately, but it should instead let the controller take responsibility.
In the controller, you make sure that you are watching STREAM events so that you find out about new streams.
Whenever there's a new stream, you check its address. If the address is one that you don't want to rewrite, you just call ATTACHSTREAM on it, with a circuit ID of 0. (The 0 means "Tor, you figure out how to attach this one.".
Otherwise, you do whatever magic dance you do in order to find out the real address for the stream.
If the lookup operation is successful, you say "REDIRECTSTREAM (stream ID) (new address". And then you ATTACHSTREAM as above.
If the lookup operation fails, you call "CLOSESTREAM (stream ID) 2". (The 2 means "resolve failed".
And that's it for the Tor integration! All you need to do now is figure out how the name lookup works.
Has someone put together an example resolver that just does simple extension substitution? It would amuse me to be able to visit 3g2upl4pq6kufc4m.chive, or 3g2upl4pq6kufc4m.allium.
Tim
Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmmp: teor at torproject dot org
On Tue, Aug 2, 2016 at 9:45 AM, teor teor2345@gmail.com wrote: [...]
Has someone put together an example resolver that just does simple extension substitution? It would amuse me to be able to visit 3g2upl4pq6kufc4m.chive, or 3g2upl4pq6kufc4m.allium.
I think I did one of these long ago when I was testing out the controller features long long ago. But I can't find it right now. It was probably very kludgy; we're probably better off that I can't find it. :)
On 08/02/2016 09:45 AM, teor wrote:
Has someone put together an example resolver that just does simple extension substitution? It would amuse me to be able to visit 3g2upl4pq6kufc4m.chive, or 3g2upl4pq6kufc4m.allium.
Tim
Tim Wilson-Brown (teor)
Thanks Nick for the helpful tips. I will certainly keep that in mind for my implementation. I didn't realize that it was so easy to close streams like that if the lookup failed, that's extremely helpful to know.
I've picked up a full-time job now and I haven't posted here for some time. I have been (mostly silently) working on the Onion Name System (OnioNS) project, which as you may remember maps "<meaningful name>.tor" to "<random letters>.onion". It has been undergoing some major and much-needed redesigns thanks to excellent comments from the reviewers of the whitepaper. I've also switched dependencies; instead of raw sockets, my own application protocol, and the massive Boost Asio library, I'm now using libmicrohttpd and RPC-JSON, so the code is very versatile, clean, and has a small footprint. It can still resolve "example.tor" to 3g2upl4pq6kufc4m.onion (DDG) or whatever else the resolver says.
Tim, to answer your question, here is my Python script that does the circuit interception and lookup rewrite: https://github.com/Jesse-V/OnioNS-client/blob/master/src/assets/onions-stem.... The network is down so the code won't fully run, but I'm posting this here so everyone can see what Nick's instructions look like in practice. Like I said, I haven't implemented the failure case properly, but the code is mostly there. Cut out the network code to the localhost resolver and you could easily do trivial resolves of .onion addresses.
Nick Mathewson:
Hi, all!
I've seen a couple of emails from people looking into new ways to do naming for onion services. That's great! Before anybody gets too far, I'd like to send this quick note to let you know that integrating stuff like this into Tor is actually easier than you think.
Here's how you do it, using a Tor controller. (See control-spec.txt for protocol documentation. Also see one of the several friendly libraries, like steam, that exist to interface with Tor over this protocol.)
First, you set the Tor option "__LeaveStreamsUnattached". This tells Tor that it shouldn't try to handle new client requests immediately, but it should instead let the controller take responsibility.
In the controller, you make sure that you are watching STREAM events so that you find out about new streams.
Whenever there's a new stream, you check its address. If the address is one that you don't want to rewrite, you just call ATTACHSTREAM on it, with a circuit ID of 0. (The 0 means "Tor, you figure out how to attach this one.".
Otherwise, you do whatever magic dance you do in order to find out the real address for the stream.
If the lookup operation is successful, you say "REDIRECTSTREAM (stream ID) (new address". And then you ATTACHSTREAM as above.
If the lookup operation fails, you call "CLOSESTREAM (stream ID) 2". (The 2 means "resolve failed".
And that's it for the Tor integration! All you need to do now is figure out how the name lookup works.
cheers,
Hi Nick,
Let's say that the name lookup operation will generate network traffic, and therefore should be subject to stream isolation. (This is the case for a subset of Namecoin-based lookup methods.)
How can a Tor controller obtain the needed information to perform stream isolation on the lookup, prior to issuing ATTACHSTREAM? (Subject to the constraint that if two streams would ordinarily use different Tor circuits, then their respective name lookups should use different Tor circuits.) From briefly looking at control-spec.txt, it looks like this info is only available by examining a circuit rather than a stream, which is problematic if the name lookup needs to be done before attaching a stream to a circuit.
It's likely that I'm just misreading the spec, but any chance you (or anyone else) could provide some pointers here?
Cheers, -Jeremy
On Tue, Aug 2, 2016 at 5:10 PM, Jeremy Rand jeremyrand@airmail.cc wrote:
Nick Mathewson:
Hi, all!
I've seen a couple of emails from people looking into new ways to do naming for onion services. That's great! Before anybody gets too far, I'd like to send this quick note to let you know that integrating stuff like this into Tor is actually easier than you think.
Here's how you do it, using a Tor controller. (See control-spec.txt for protocol documentation. Also see one of the several friendly libraries, like steam, that exist to interface with Tor over this protocol.)
First, you set the Tor option "__LeaveStreamsUnattached". This tells Tor that it shouldn't try to handle new client requests immediately, but it should instead let the controller take responsibility.
In the controller, you make sure that you are watching STREAM events so that you find out about new streams.
Whenever there's a new stream, you check its address. If the address is one that you don't want to rewrite, you just call ATTACHSTREAM on it, with a circuit ID of 0. (The 0 means "Tor, you figure out how to attach this one.".
Otherwise, you do whatever magic dance you do in order to find out the real address for the stream.
If the lookup operation is successful, you say "REDIRECTSTREAM (stream ID) (new address". And then you ATTACHSTREAM as above.
If the lookup operation fails, you call "CLOSESTREAM (stream ID) 2". (The 2 means "resolve failed".
And that's it for the Tor integration! All you need to do now is figure out how the name lookup works.
cheers,
Hi Nick,
Let's say that the name lookup operation will generate network traffic, and therefore should be subject to stream isolation. (This is the case for a subset of Namecoin-based lookup methods.)
How can a Tor controller obtain the needed information to perform stream isolation on the lookup, prior to issuing ATTACHSTREAM?
Conceptually, a controller doesn't isolate streams: the streams are isolated based on their own properties, so that's more of a client thiing.
One of the properties that you can use for isolation here is the SOCKS username and password: two streams with different SOCKS credentials should never be sent over the same circuit. This should make stream isolation happen just fine.
[This is assuming that you use 'ATTACHSTREAM' with a circuit ID of 0, and let Tor make its own isolation decisions.]
Is that about what you wanted to know?
cheers,
Nick Mathewson:
On Tue, Aug 2, 2016 at 5:10 PM, Jeremy Rand jeremyrand@airmail.cc wrote:
Nick Mathewson:
Hi, all!
I've seen a couple of emails from people looking into new ways to do naming for onion services. That's great! Before anybody gets too far, I'd like to send this quick note to let you know that integrating stuff like this into Tor is actually easier than you think.
Here's how you do it, using a Tor controller. (See control-spec.txt for protocol documentation. Also see one of the several friendly libraries, like steam, that exist to interface with Tor over this protocol.)
First, you set the Tor option "__LeaveStreamsUnattached". This tells Tor that it shouldn't try to handle new client requests immediately, but it should instead let the controller take responsibility.
In the controller, you make sure that you are watching STREAM events so that you find out about new streams.
Whenever there's a new stream, you check its address. If the address is one that you don't want to rewrite, you just call ATTACHSTREAM on it, with a circuit ID of 0. (The 0 means "Tor, you figure out how to attach this one.".
Otherwise, you do whatever magic dance you do in order to find out the real address for the stream.
If the lookup operation is successful, you say "REDIRECTSTREAM (stream ID) (new address". And then you ATTACHSTREAM as above.
If the lookup operation fails, you call "CLOSESTREAM (stream ID) 2". (The 2 means "resolve failed".
And that's it for the Tor integration! All you need to do now is figure out how the name lookup works.
cheers,
Hi Nick,
Let's say that the name lookup operation will generate network traffic, and therefore should be subject to stream isolation. (This is the case for a subset of Namecoin-based lookup methods.)
How can a Tor controller obtain the needed information to perform stream isolation on the lookup, prior to issuing ATTACHSTREAM?
Conceptually, a controller doesn't isolate streams: the streams are isolated based on their own properties, so that's more of a client thiing.
One of the properties that you can use for isolation here is the SOCKS username and password: two streams with different SOCKS credentials should never be sent over the same circuit. This should make stream isolation happen just fine.
[This is assuming that you use 'ATTACHSTREAM' with a circuit ID of 0, and let Tor make its own isolation decisions.]
Is that about what you wanted to know?
cheers,
Hi Nick,
I'm looking at the docs for StreamEvent in Stem:
https://stem.torproject.org/api/response.html#stem.response.events.StreamEve...
And I don't see any obvious way to get the SOCKS auth data from that. Using the SOCKS auth data was, indeed, the first thought that occurred to me.
Is there a way to get the SOCKS auth data from a StreamEvent prior to issuing ATTACHSTREAM, so that I can use that data when performing the name lookup?
Cheers, -Jeremy
On Thu, Aug 4, 2016 at 6:37 PM, Jeremy Rand jeremyrand@airmail.cc wrote: [...]
Hi Nick,
I'm looking at the docs for StreamEvent in Stem:
https://stem.torproject.org/api/response.html#stem.response.events.StreamEve...
And I don't see any obvious way to get the SOCKS auth data from that. Using the SOCKS auth data was, indeed, the first thought that occurred to me.
Is there a way to get the SOCKS auth data from a StreamEvent prior to issuing ATTACHSTREAM, so that I can use that data when performing the name lookup?
Oh! You want to *get* the isolation information!
That's not implemented, but it wouldn't be too hard to do. The hard part will be deciding the right interface to expose to controllers. I've started a ticket at https://trac.torproject.org/projects/tor/ticket/19859 to track the problem space.
-
Jeremy Rand -
Jesse V -
Nick Mathewson -
Nick Mathewson -
teor