Also, I agree with Tim. The present wording makes it sound like txtorcon is the only game in town when it comes to custom circuit construction.

I looked at the "to russia with love" examples, but I don't see how you map particular streams to circuits?

The last example on the page it seems like you're just "really hoping" that the next stream to open is the one you want to map onto the new circuit, or am I missing something? That is, what if you had 2 custom circuits and 2 streams to open? Or what if some other client used your Tor at that moment to make a connetion?

To plug the newest txtorcon API (the "Tor" object): this provides really simple support for both "raw" TCP connections and specifically Web connections (via Twisted's built-in Agent or treq, the "requests-like API for Twisted") routed over Tor (on custom circuits or otherwise). This all uses the source-ports of the streams to determine which ones to map to which circuits. So, you can safely have thousands of custom streams in-flight on any number of different custom circuits (and play nicely with other clients using the same tor instance).

For example, "use any Tor circuit and make a Web request":

https://github.com/meejah/txtorcon/blob/master/examples/web_client_treq.py#L...

vs. "build a custom circuit and make a request over that":

https://github.com/meejah/txtorcon/blob/master/examples/web_client_custom_ci...

Cheers,

Damian Johnson atagar@torproject.org writes:

Maybe python3's asyncio (the builtin which seems intended to replace twisted) has better options. I certainly wouldn't be surprised if what Twisted provides is better than the builtins we had on python 2.7.

I don't think asyncio is "intended" to replace Twisted (even if it took a lot of inspiration from it). Twisted will be around for a while yet -- and supports Python3, including the "async def" and "await" syntax sugar. Hawkowl did a talk on this topic last year:

https://ep2016.europython.eu/conference/talks/the-report-of-twisteds-death

There's no built-in SOCKS5 support in Twisted proper, but there's txsocksx which is very good (although it lacks Python3 support). Because of the latter, txtorcon now has its own "sans-io" style SOCKS5 implementation (with the custom Tor opcodes as well) which supports Python3 as well as of course Python2 and PyPy.

So, you can do async DNS (at least A and PTR lookups) via tor + txtorcon nicely:

https://github.com/meejah/txtorcon/blob/master/examples/dns_lookups.py

Hey Damian!

Sorry, in my nearsightedness, having only ever had used Stem for parsing, I had not realised that enough of the control protocol was implemented in Stem to do this without using txtorcon! Please feel free to reword the posting (or suggest a change) that you think would more accurately reflect this.

No problem in the least. :P

For what it's worth Stem is a complete implementation of the control and directory specification. That is to say, everything in there *should* be supported (I keep an eye on all spec commits to ensure we continue to match everything tor has). If Stem's missing something then that's a bug I'd love to know about.

Blog post looks great. I'd probably only suggest a couple small tweaks...

* s/also outsourceable in Python to txtorcon/also outsourceable in Python to txtorcon or stem * Drop the paragraph after it since it doesn't seem to add much.

It's probably not worth including in the blog post but for what it's worth the following list all the controller and descriptor parser libs I'm aware of...

https://stem.torproject.org/faq.html#are-there-any-other-controller-librarie... https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#are-the...

Also, I would be delighted to co-mentor with you (and also meejah, if the intern decides to go the txtorcon route), that sounds great!

Great!