I felt it premature to publicly discuss using my user level network stack as the basis for a tor transport given the number of potential candidates and the issues that I raise below. However, Sebastian felt strongly that it belongs on tor-dev.
Cheers
---------- Forwarded message ---------- From: K. Macy kmacy@freebsd.org Date: Sat, Jan 21, 2012 at 3:05 PM Subject: Re: Tor Transports
On Fri, Jan 20, 2012 at 11:35 PM, Steven Murdoch Steven.Murdoch@cl.cam.ac.uk wrote:
Hi All,
One hop-by-hop transport protocol we will likely be considering for an alternative Tor transport is TCP, and Kip Macy (a FreeBSD developer, Cc'd) has been working on porting the FreeBSD network stack to userspace, with the Tor use-case in mind. Unlike many other attempts though, maintainability has been a primary concern, so we should be able to keep in sync with the FreeBSD stack with manageable effort.
Another advantage of this approach is that we will get IPSec for free, which could be very useful if we decide DTLS is not the way to go.
The FreeBSD network stack is extremely well tested, so we are in a good position from that respect. One downside is that the code size might be large (unless we can find a way to do some clever link-time optimization away of dead code).
His code is here: http://gitorious.org/~kmm/freebsd/kmm-sandbox/commits/work/svn_trunk_libpleb...
It obviously also supports SCTP to whatever extent that FreeBSD does. The author of the RFCs and of the reference SCTP stack on all operating systems except Linux is a FreeBSD committer. Another reason why I chose to take this approach is you get all of the kernel firewall bits "for free." Thus one could envision supporting transparent proxying in the application itself to allow for transparent Tor usage in an OS independent manner.
I have a few additional comments to add to set expectations appropriately.
1) I have a prototype "safe" BPF API whereby an unprivileged application can only receive and send packets on a single or pre-specified set of IP addresses and only advertise its private stack's MAC address. Without this functionality one needs to either layer the stack on top of kernel UDP (perfectly reasonable approach, just requires writing another simple virtual NIC driver) or running as root, whereby plebeian networking becomes a misnomer - a patrician poseur as it were. Having even such a simple kernel module goes against the grain of Tor conventions of not doing anything as root (although configuring things like transparent proxying require a certain amount of futzing as root).
2) The libc shim is written with the idea that it overrides the relevant libc symbols by means of LD_PRELOAD. The library differentiates between "kernel descriptors" for files, unix sockets, or anything outside the scope of the network stack and "user descriptors". For example, when a kernel descriptor is passed to write() libplebnet will call _write() with the arguments and the call should functional as it normally does.
3) At this instant libplebnet does not support localhost communication over TCP/UDP. I believe that the most efficient way of supporting this would be by means of directing those packets to a tap device.
3) Without a (most likely) substantial amount of work to share user level network state across address spaces, the apache style (pre-)forked process with shared accept socket does not work. I have not given it any thought as I don't see a use case for it.
4) Stack configuration was, to my surprise at least, the biggest hurdle. The UNIX APIs for configuring network stack state are anything but user friendly. To enable continued use of the standard tools (ifconfig, route, arp, etc) I wrote a separate library called libplebconf which one would LD_PRELOAD with the pid of the the process to be configured set as an environment variable. At startup libplebnet creates a thread which listens on a unix socket at "/tmp/<pid>" and accepts as messages serialized system calls to the various configuration calls that the aforementioned programs use. To simplify things a bit I've added an additional environment variable with the path to an rc.conf which gets executed at startup to configure the process using libplebnet's ip and route.
Most of this will be moot in the event that you choose to cut out layer 3 and interface directly between your L4 of choice and DTLS. I haven't given any thought to how to eliminate this complexity in the event that you choose to use FreeBSD's IPSEC.
To avoid boring most readers I have kept the discussion rather superficial. Nonetheless, I hope that this missive clarifies matters more than it confuses them.
Cheers, Kip
On Sat, Jan 21, 2012 at 08:09:58PM +0100, K. Macy wrote:
I felt it premature to publicly discuss using my user level network stack as the basis for a tor transport given the number of potential candidates and the issues that I raise below. However, Sebastian felt strongly that it belongs on tor-dev.
Hi Kip,
Thanks for keeping us informed. I look forward to seeing which transport Steven picks first for prototyping and testing, and seeing how it goes. Getting ipsec for 'free' is a big deal, since our other options for link encryption at the datagram packet level are dtls and "oh come on, how hard could that be, let's roll our own." In any case, more options is always good.
--Roger
Thus spake K. Macy (kmacy@freebsd.org):
On Fri, Jan 20, 2012 at 11:35 PM, Steven Murdoch wrote:
One hop-by-hop transport protocol we will likely be considering for an alternative Tor transport is TCP, and Kip Macy (a FreeBSD developer, Cc'd) has been working on porting the FreeBSD network stack to userspace, with the Tor use-case in mind. Unlike many other attempts though, maintainability has been a primary concern, so we should be able to keep in sync with the FreeBSD stack with manageable effort.
I have a few additional comments to add to set expectations appropriately.
- I have a prototype "safe" BPF API whereby an unprivileged
application can only receive and send packets on a single or pre-specified set of IP addresses and only advertise its private stack's MAC address. Without this functionality one needs to either layer the stack on top of kernel UDP (perfectly reasonable approach, just requires writing another simple virtual NIC driver) or running as root, whereby plebeian networking becomes a misnomer - a patrician poseur as it were. Having even such a simple kernel module goes against the grain of Tor conventions of not doing anything as root (although configuring things like transparent proxying require a certain amount of futzing as root).
Can you explain the privilege separation of the safe BPF api? What handles ultimately wrapping up packets for your virtual interfaces on the other side of the BPF API? Does it run as root as a daemon to manage your virtual interface, providing the BPF API to unprivileged apps through the LD_PRELOAD wrappers, or .. ?
Also, how portable is the code that provides this virtual interface support?
On Mon, Jan 30, 2012 at 8:23 AM, Mike Perry mikeperry@torproject.org wrote:
Thus spake K. Macy (kmacy@freebsd.org):
On Fri, Jan 20, 2012 at 11:35 PM, Steven Murdoch wrote:
One hop-by-hop transport protocol we will likely be considering for an alternative Tor transport is TCP, and Kip Macy (a FreeBSD developer, Cc'd) has been working on porting the FreeBSD network stack to userspace, with the Tor use-case in mind. Unlike many other attempts though, maintainability has been a primary concern, so we should be able to keep in sync with the FreeBSD stack with manageable effort.
I have a few additional comments to add to set expectations appropriately.
- I have a prototype "safe" BPF API whereby an unprivileged
application can only receive and send packets on a single or pre-specified set of IP addresses and only advertise its private stack's MAC address. Without this functionality one needs to either layer the stack on top of kernel UDP (perfectly reasonable approach, just requires writing another simple virtual NIC driver) or running as root, whereby plebeian networking becomes a misnomer - a patrician poseur as it were. Having even such a simple kernel module goes against the grain of Tor conventions of not doing anything as root (although configuring things like transparent proxying require a certain amount of futzing as root).
Can you explain the privilege separation of the safe BPF api?
The filter creation / installation is not handled by the unprivileged process. The most sophisticated possibility is my recollection of Robert Watson's suggestion: A dedicated privileged daemon accepts requests for an IP & mac address and returns a descriptor that is a bpf handle with the corresponding filter installed. Thus libplebnet would send a message to the daemon saying "my MAC is 80:ee:73:00:ba:be and my desired IP is 192.168.0.42", the daemon would fill out a filter template with those values, install it and then hand pass the descriptor to libplebnet.
What handles ultimately wrapping up packets for your virtual interfaces on the other side of the BPF API?
The libplebnet process creates the packets, but if attempts to advertise a MAC other than its own or spoof source addresses the packets will be dropped by the egress filter.
Does it run as root as a daemon to manage your virtual interface, providing the BPF API to unprivileged apps through the LD_PRELOAD wrappers, or .. ?
The daemon installing filters would run as root. The actual filtering happens in BPF at L2, see the original McCanne/Van Jacobson paper if that part is unclear.
Also, how portable is the code that provides this virtual interface support?
I think the biggest portability question is BPF itself, which I have not looked in to. My virtual ethernet driver is very simple, and my "safe BPF" extension is a handful of extra checks in BPF.
Thanks, Kip
-
K. Macy -
Mike Perry -
Roger Dingledine