Hello,
I just pushed a change to obfs4proxy master to use `utls` to mask the ClientHello signature (currently Chrome 70.x).
https://gitlab.com/yawning/obfs4/commit/4d453dab2120082b00bf6e63ab4aaeeda6b8...
I understand that this is being worked on for the original meek (see: https://bugs.torproject.org/29077), but I felt inspired and it was relatively easy to get something working.
Caveats: * This is only lightly tested, and may be doing something wrong or distinct. It seems to work well enough to watch videos over it. YMMV. * Azure uses HTTP 2. Not really a problem. * `utls.HelloFirefox_Auto` will fail to handshake with Azure due to an incompatible group being negotiated. * `utls.HelloChrome_Auto` ironically fails to handshake with `google.com` in a standalone test case for me. * `utls.HelloIOS_Auto` seems to work in all cases, so I may switch to that before I tag.
Questions, comments, feedback appreciated,
On Mon, Jan 21, 2019 at 05:12:41AM +0000, Yawning Angel wrote:
I just pushed a change to obfs4proxy master to use `utls` to mask the ClientHello signature (currently Chrome 70.x).
https://gitlab.com/yawning/obfs4/commit/4d453dab2120082b00bf6e63ab4aaeeda6b8...
I understand that this is being worked on for the original meek (see: https://bugs.torproject.org/29077), but I felt inspired and it was relatively easy to get something working.
Thanks, you found a clever approach that I hadn't thought of. I tried just setting DialTLS on the main http.Transport--that doesn't work because net/http doesn't know that utls has negotiated HTTP/2, and starts sending HTTP/1.1 on an HTTP/2 connection. Setting DialTLS on an http2.Transport works, but only with HTTP/2 servers.
If I may interpret, your code builds an http.RoundTripper wrapper around http.Transport and http2.Transport. When the caller makes its first request, the wrapper initiates the utls connection, then inspects what protocol was negotiated with ALPN, and creates its own internal http.Transport or http2.Transport as appropriate. Then, it simply forwards all requests to its internal transport--also setting DialTLS on the internal transport so that future connections will also use utls, but re-using the same transport instead of making a new one each time.
This looks better than what I was trying to do. I will probably start working on doing the meek-client implementation in this style.
As for the TODO, my plan was was to expose a "utls" SOCKS arg to make it configurable per bridge, and just reuse the utls Client Hello ID names: utls=HelloChrome_Auto
(Whoops I sent my last reply directly instead of to the list. It wasn't all that important for the general public, and lists.tp.o has been flaky for me recently anyway.)
On 1/21/19 5:22 PM, David Fifield wrote:
As for the TODO, my plan was was to expose a "utls" SOCKS arg to make it configurable per bridge, and just reuse the utls Client Hello ID names: utls=HelloChrome_Auto
Done.
https://gitlab.com/yawning/obfs4/commit/e4020b18f7aaafe9f4cb345630bfe18a5e44...
As long as there's enough bridge line interoperability between implementations, I'm not particularly bothered if other people actually do use utls.HelloGolang or not, I'm choosing not to.
As a side note: Implementing support for the missing DH groups in utls is likely trivial (assuming you don't care that it's vartime, extremely bad for actual TLS, fine for meek_lite) and would increase compatibility a good amount.
That said HelloChrome_Auto and HelloIOS_Auto both work fine against the Azure bridge, so it might not be worth the effort.
Regards,
On Mon, Jan 21, 2019 at 05:12:41AM +0000, Yawning Angel wrote:
I just pushed a change to obfs4proxy master to use `utls` to mask the ClientHello signature (currently Chrome 70.x).
https://gitlab.com/yawning/obfs4/commit/4d453dab2120082b00bf6e63ab4aaeeda6b8...
// This also assumes that req.URL.Host will remain constant for the // lifetime of the roundTripper, which is a valid assumption for meeklite.
Am I wrong, or is the actual restriction less strict? You can reuse the roundTripper for different hosts--the ServerName is taken from the addr argument to dialTLS--but only if those different hosts negotiate the same ALPN, because the choice of http.Transport or http2.Transport is made only once and persists for the lifetime of the roundTripper.
For the same reason, I don't think we'll be able to use HelloRandomized, only HelloRandomizedALPN or HelloRandomizedNoALPN. Otherwise we may negotiate different ALPN even against the same server during the lifetime of roundTripper. I tried adding a conn.SetReadDeadline(time.Now().Add(10*time.Second)) inside dialTLS to force it to re-dial frequently, and with HelloRandomized it does indeed eventually trip the "horrifically wrong" branch with an error like: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x12\x04\x00\x00\x00\x00\x00\x00\x05\x00\x10\x00\x00\x00\x03\x00\x00\x00\xfa\x00\x06\x00\x10\x01@" Despite the error, the client recovers quickly, redialing until it gets a compatible ALPN.
On 1/24/19 6:47 AM, David Fifield wrote:
// This also assumes that req.URL.Host will remain constant for the // lifetime of the roundTripper, which is a valid assumption for meeklite.
Am I wrong, or is the actual restriction less strict? You can reuse the roundTripper for different hosts--the ServerName is taken from the addr argument to dialTLS--but only if those different hosts negotiate the same ALPN, because the choice of http.Transport or http2.Transport is made only once and persists for the lifetime of the roundTripper.
The lock protecting `roundTripper.initConn` is only held in `dialTLS`, and the `roundTripper.transport` is not protected by a lock at all.
If the target host changes and there is simultaneous access (two threads call into `roundTripper.RoundTrip` right after initialization simultaneously), there is no guarantee that the connection used to create the inner `http.RoundTripper` instance will be passed to the correct thread.
Regards,
On Thu, Jan 24, 2019 at 07:33:39AM +0000, Yawning Angel wrote:
On 1/24/19 6:47 AM, David Fifield wrote:
// This also assumes that req.URL.Host will remain constant for the // lifetime of the roundTripper, which is a valid assumption for meeklite.
Am I wrong, or is the actual restriction less strict? You can reuse the roundTripper for different hosts--the ServerName is taken from the addr argument to dialTLS--but only if those different hosts negotiate the same ALPN, because the choice of http.Transport or http2.Transport is made only once and persists for the lifetime of the roundTripper.
The lock protecting `roundTripper.initConn` is only held in `dialTLS`, and the `roundTripper.transport` is not protected by a lock at all.
If the target host changes and there is simultaneous access (two threads call into `roundTripper.RoundTrip` right after initialization simultaneously), there is no guarantee that the connection used to create the inner `http.RoundTripper` instance will be passed to the correct thread.
I see, you're right. It has to do with the reuse of the initConn.
On 1/24/19 7:38 AM, David Fifield wrote:
I see, you're right. It has to do with the reuse of the initConn.
A proper "general" solution that solves that problem and the ALPN issue is to have a `initConn` and `http.RoundTripper` instance per destination host, and some additional locking.
With more implementation cleverness this could be brought down to two `http.RoundTripper` instances, and a host -> pointer + net.Conn map, and some locking.
But for something like meek_lite where the number of destination hosts is not large, the existing wrapper works fine and I don't see much reason to over engineer it.
Regards,
On Thu, Jan 24, 2019 at 07:44:48AM +0000, Yawning Angel wrote:
On 1/24/19 7:38 AM, David Fifield wrote:
I see, you're right. It has to do with the reuse of the initConn.
A proper "general" solution that solves that problem and the ALPN issue is to have a `initConn` and `http.RoundTripper` instance per destination host, and some additional locking.
With more implementation cleverness this could be brought down to two `http.RoundTripper` instances, and a host -> pointer + net.Conn map, and some locking.
But for something like meek_lite where the number of destination hosts is not large, the existing wrapper works fine and I don't see much reason to over engineer it.
I don't disagree, it's fine for this use case.
-
David Fifield -
Yawning Angel