tor-dev May 2020

tor-dev@lists.torproject.org

28 participants
29 discussions

Start a nNew thread

So Windows is Adding a Package Manager.....
by Keifer Bly 24 May '20

24 May '20

2 1

tor-dev Digest, Vol 112, Issue 21
by Support Team 20 May '20

20 May '20

Thank you for the insights. On Wed, May 20 2020, at 12:00 PM, <tor-dev(a)lists.torproject.org> Send tor-dev mailing list submissions to tor-dev(a)lists.torproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev or, via email, send a message with subject or body 'help' to tor-dev-request(a)lists.torproject.org You can reach the person managing the list at tor-dev-owner(a)lists.torproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of tor-dev digest..." Today's Topics: 1. Re: Proposal 319: RELAY_FRAGMENT cells (Nick Mathewson) 2. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 3. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 4. Re: Proposal 320: Removing TAP usage from v2 onion services (Sebastian Hahn) ---------------------------------------------------------------------- Message: 1 Date: Tue, 19 May 2020 13:29:54 -0400 From: Nick Mathewson To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal 319: RELAY_FRAGMENT cells Message-ID: Content-Type: text/plain; charset="UTF-8" On Thu, May 14, 2020 at 3:15 PM David Goulet wrote: > > On 11 May (16:47:24), Nick Mathewson wrote: [...] > > # Onion service concerns. > > > > We allocate a new extension for use in the ESTABLISH_INTRO by onion services, > > to indicate that they can receive a wide INTRODUCE2 cell. This extension > > contains: > > > > struct wide_intro2_ok { > > u16 max_len; > > } > > > > We allocate a new extension for use in the 'ESTABLISH_RENDEZVOUS' > > cell, to indicate acceptance of wide 'RENDEZVOUS2' cells. This > > extension contains: > > > > struct wide_rend2_ok { > > u16 max_len; > > } > > > > (Note that 'ESTABLISH_RENDEZVOUS' cells do not currently have a an > > extension mechanism. They should be extended to use the same > > extension format as 'ESTABLISH_INTRO' cells, with extensions placed > > after the rendezvous cookie.) > > Why would a client need to announce wide cells in the ESTABLISH phase as > opposed to using protover "Relay=N" ? This is not for announcing support of wide cells -- this is for reporting a setting for how wide fragmented cells should be. > The maximum length of a fragmented cell is capped to 2^16 (u16) so we don't > really need the establish process to inform us of the maximum expected length > but rather use the max_len in the first fragment? This all comes back to an earlier part of the proposal: Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. In other words, I'm imagining that there is a maximum length for each cell type that is much shorter than 65535, even though we're using two bytes for the length field. The extension in the establish_intro cell is to tell the intro point the longest introduce1 cell that it should accept; this extension in the establish_rend cell is to tell the rendezvous point the longest rendezvous1 cell that it should accept. Another way we could do this would be with a set of network parameters to describe the maximum length of each fragmented cell. Do you think that would be simpler? (I can't quite remember why I specified it this way in the first place.) > Furthermore, ESTABLISH_INTRO has extensions (only 1 as of today) so they could > also be fragments themselves and thus I'm not sure I see the point of having > two different ways of "expecting" fragments for the ESTABLISH_* cells and the > INTRO/RENDEZVOUS cells? The difference thing here is that everybody can tell which protocols that a relay supports, but there is no automatic way to tell which protocols an onion service or client supports. Since INTRODUCE2/RENDEZVOUS2 cells are handled by these clients, they need to get opted into by the relays. (I'm not sure I understood the question completely.) > > # Compatibility > > > > This proposal will require the allocation of a new 'Relay' protocol version, > > to indicate understanding of the RELAY_FRAGMENTED command. > > Here is a thought about a DoS vector. Here goes: > > As an upper limit of 65KB total fragment size, it represents ~126 cells in > total so I could basically send *125* cells and then stop which will put in > memory a bit more than 64KB and it will stay there until the last fragment is > received. > > And then I do that on 1000 different circuits bringing the total count in > memory to 64GB. All stuck there, all "waiting" for the last fragment. > > Our OOM would kick in killing circuits but it just seems to me a very easy way > to continously kick the OOM of a _service_ which is pretty bad side channel. A few responses here: First, we shouldn't allow 65535-byte fragmented cells. The actual maximum length should be something more like 1024 or 4096 bytes. Second, we should make sure that when we are reassembling cells, we use the same buf_t buffers that we use for other stuff. Our buffers are timestamped, so we can tell which buffer has had data stalling for the longest, and we should use that to make sure we're killing off the right circuits preferentially. Third, fragments should only be allowed at an onion service for INTRODUCE2, and those should only come one at a time from each introduction point, so the number that it's reassembling at the time will be limited by the number of intro circuits it has open. It'll be the the intro points that have to be keeping a bunch of cells in assembly at once, and be ready to kill off circuits that dawdle too long. Does this make more sense? If so I'll try to clarify it in the proposal.

1 0

Proposal 319: RELAY_FRAGMENT cells
by Nick Mathewson 19 May '20

19 May '20

``` Filename: 319-wide-everything.md Title: RELAY_FRAGMENT cells Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` (This proposal is part of the Walking Onions spec project.) # Introduction Proposal 249 described a system for `CREATE` cells to become wider, in order to accommodate hybrid crypto. And in order to send those cell bodies across circuits, it described a way to split `CREATE` cells into multiple `EXTEND` cells. But there are other cell types that can need to be wider too. For example, `INTRODUCE` and `RENDEZVOUS` cells also contain key material used for a handshake: if handshakes need to grow larger, then so do these cells. This proposal describes an encoding for arbitrary "wide" relay cells, that can be used to send a wide variant of anything. To be clear, although this proposal describes a way that all relay cells can become "wide", I do not propose that wide cells should actually be _allowed_ for all relay cell types. # Proposal We add a new relay cell type: `RELAY_FRAGMENT`. This cell type contains part of another relay cell. A `RELAY_FRAGEMENT` cell can either introduce a new fragmented cell, or can continue one that is already in progress. The format of a RELAY_FRAGMENT body is one of the following: // First body in a series struct fragment_begin { // What relay_command is in use for the underlying cell? u8 relay_command; // What will the total length of the cell be once it is reassembled? u16 total_len; // Bytes for the cell body u8 body[]; } // all other cells. struct fragment_continued { // More bytes for the cell body. u8 body[]; } To send a fragmented cell, first a party sends a RELAY_FRAGMENT cell containing a "fragment_begin" payload. This payload describes the total length of the cell, the relay command Fragmented cells other than the last one in sequence MUST be sent full of as much data as possible. Parties SHOULD close a circuit if they receive a non-full fragmented cell that is not the last fragment in a sequence. Fragmented cells MUST NOT be interleaved with other relay cells on a circuit, other than cells used for flow control. (Currently, this is only SENDME cells.) If any party receives any cell on a circuit, other than a flow control cell or a RELAY_FRAGEMENT cell, before the fragmented cell is complete, than it SHOULD close the circuit. Parties MUST NOT send extra data in fragmented cells beyond the amount given in the first 'total_len' field. Not every relay command may be sent in a fragmented cell. In this proposal, we allow the following cell types to be fragmented: EXTEND2, EXTENDED2, INTRODUCE1, INTRODUCE2, RENDEZVOUS. Any party receiving a command that they believe should not be fragmented should close the circuit. Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. All `RELAY_FRAGMENT` cells for the fragmented cell must have the same Stream ID. (For those cells allowed above, the Stream ID is always zero.) Implementations SHOULD close a circuit if they receive fragments with mismatched Stream ID. # Onion service concerns. We allocate a new extension for use in the ESTABLISH_INTRO by onion services, to indicate that they can receive a wide INTRODUCE2 cell. This extension contains: struct wide_intro2_ok { u16 max_len; } We allocate a new extension for use in the `ESTABLISH_RENDEZVOUS` cell, to indicate acceptance of wide `RENDEZVOUS2` cells. This extension contains: struct wide_rend2_ok { u16 max_len; } (Note that `ESTABLISH_RENDEZVOUS` cells do not currently have a an extension mechanism. They should be extended to use the same extension format as `ESTABLISH_INTRO` cells, with extensions placed after the rendezvous cookie.) # Handling RELAY_EARLY The first fragment of each EXTEND cell should be tagged with `RELAY_EARLY`. The remaining fragments should not. Relays should accept `EXTEND` cells if and only if their _first_ fragment is tagged with `RELAY_EARLY`. > Rationale: We could allow any fragment to be tagged, but that would give > hostile guards an opportunity to move RELAY_EARLY tags around and build a > covert channel. But if we later move to a relay encryption method that > lets us authenticate RELAY_EARLY, we could then require only that _any_ > fragment has RELAY_EARLY set. # Compatibility This proposal will require the allocation of a new 'Relay' protocol version, to indicate understanding of the RELAY_FRAGMENTED command.

4 3

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Fri, 15 May 2020 11:20:54 +1000 teor <teor(a)riseup.net> wrote: > You could try deleting functions, and then running tor's > "make test-network-all". If that test still passes, then the code is probably > unused (in practice). > > Chutney's CI does a few more tests for different tor versions, but I'm pretty > sure they all use the same code. This is what I will end up doing. I planned on it, but humans tend to know the purpose of the code better, and computers just run what is given to them :) so I figured it was best to check here first, with people more experienced with the codebase. Thanks, Caitlin

1 0

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Thu, 14 May 2020 19:58:45 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > isInExpectedDirInfoDocs looks like it might once have done something useful. My impression is to remove it unless we anticipate needing it in the near future. > configure and restart and wait_for_bootstrap are all in use; they are > invoked by name, from the command line, at the end of runConfigFile, > where it says `return getattr(network, verb)()`. Ah, I see now, thank you. I think it would be wise to comment the functions as such to indicate that they are used on the command line. > I think the rest are likely to be unused. I did grep most of these and found no signs of use, so this is my impression as well. For now I will make a mental note of all this while I work on more pertinent improvements to the code, but I do believe it would be nice to deal with truly unused code sooner rather than later. Caitlin

1 0

Deprecating Tor Protocol Versions
by teor 15 May '20

15 May '20

Hi all, Nick and I were talking about how we remove legacy features in tor, and their corresponding subprotocol versions. Here is a list of the current subprotocol versions: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n2049 Here's a recent protocol version proposal, which deals with recommending and requiring new features: https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-remov… But we don't have a similar proposal for removing support for older protocol versions from the tor codebase. For an example of what that proposal could look like, see our proposal for deprecating consensus methods: https://gitweb.torproject.org/torspec.git/tree/proposals/290-deprecate-cons… Here's the original conversation Nick and I had: https://github.com/torproject/tor/pull/1874#discussion_r423713579 But after reading our consensus methods deprecation proposal, I've changed my mind. I think that we should check for "protocol N, and any later version" by default. That's what we do for consensus methods, and it seems to work well. We can drop the earliest consensus methods, and recent tor versions just keep working. If we need an incompatible change, we can make it another protocol version, and recommend then require support for it. So here's an edited version of my notes on that ticket: There are a few instances of ">=" and "=" confusion in protocol versions. We should try to fix them all. It only matters when we remove protocol versions. We haven't really specified, tested, or exercised this functionality in practice. And so our reviewers lack experience. (And when we did discover a need for it with NSS and LinkAuth, it was more complicated than we expected.) I'd like to see a proposal that tells us how to check future protocol versions as they are added. Along with a migration plan for disabling protocol versions. So let's also open a ticket to check for "any future version". We should replace all "=" checks with ">=". Let's make sure we check all the places where we use protocol versions, even if they don't have a summary flag. Overall, I think it would be helpful if future protocol versions were orthogonal. Or if they depend on earlier features, that dependency should be clearly documented. (For example, Relay=3 IPv6 extends depends on Relay=2 EXTEND2 cells. So if we were checking EXTEND2 cell support, it would be Relay=2 or Relay=3.) T -- teor ----------------------------------------------------------------------

2 2

Re: [tor-dev] Chutney code refactoring
by c 15 May '20

15 May '20

On Fri, 17 Apr 2020 18:01:42 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > If you want to work on this it would be helpful to maybe start by > listing (here or elsewhere) some places where you *don't* feel like > you could (or would want to) write documentation: those would be a > good target for devs who _have_ worked on Chutney before. I came across Node.specialize() which does not seem to be called elsewhere, and I cannot guess at its purpose. So, if anyone knows off the top of their head what this is for (not what it does, because it's quite obvious from the code itself) then I'd greatly appreciate and I will document its purpose accordingly. It is one of the functions that is listed as "to be called by the user". Other than that I added docstrings where they were missing in TorNet (useful both for Python's on-line help() and for checking whether code agrees with intent) and made mental note of some areas I would like to improve in the code (changing function names to better align with what they actually do, figure out if some longer functions can be split into more atomical and easier-to-digest parts, ...). Caitlin

3 3

woswos - GSoC 2020
by Barkin Simsek 12 May '20

12 May '20

Hi everyone, I'm Barkin (woswos in IRC), and I study Computer Engineering at New York University Abu Dhabi. This summer, I will be working on the "Cloudflare CAPTCHA Monitoring" project as a part of the GSoC program with my mentors Georg and Roger. I’m very excited about working on this project! The Cloudflare CAPTCHA Monitoring project aims to track how often Cloudflare fronted webpages return CAPTCHAs to Tor clients. The project aims to achieve this by fetching webpages via both Tor and other mainstream web browsers and comparing the results. The tests are repeated periodically to find the patterns over time. Collected metadata, metrics, and results are analyzed and displayed on a dashboard to understand how Cloudflare manipulates internet traffic and affects people's access to the internet. I will be creating a bunch of trac tickets to record my progress and to get feedback from the community. Until then, you can leave your comments and suggestions under the ticket #33010 You can find more detailed information about the project on this wiki page: https://trac.torproject.org/projects/tor/wiki/doc/CAPTCHAMonitor This wiki page will also contain the links to the trac tickets I mentioned. I try to place all sorts of information on the wiki page to make it the "go-to place" to get information about my project. Best, Barkin

1 0

Proposal 318: Limit protover values to 0-63
by Nick Mathewson 12 May '20

12 May '20

``` Filename: 318-limit-protovers.md Title: Limit protover values to 0-63. Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` # Limit protover values to 0-63. I propose that we no longer accept protover values higher than 63, so that they can all fit nicely into 64-bit fields. (This proposal is part of the Walking Onions spec project.) ## Motivation Doing this will simplify our implementations and our protocols. Right now, an efficient protover implementation needs to use ranges to represent possible protocol versions, and needs workarounds to prevent an attacker from constructing a protover line that would consume too much memory. With Walking Onions, we need lists of protocol versions to be represented in an extremely compact format, which also would benefit from a limited set of possible versions. I believe that we will lose nothing by making this change. Currently, after nearly two decades of Tor development and 3.5 years of experiences with protovers in production, we have no protocol version high than 5. Even if we did someday need to implement higher protocol versions, we could simply add a new subprotocol name instead. For example, instead of "HSIntro=64", we could say "HSIntro2=1". ## Migration Immediately, authorities should begin rejecting relays with protocol versions above 63. (There are no such relays in the consensus right now.) Once this change is deployed to a majority of authorities, we can remove support in other Tor environments for protocol versions above 63.

2 1

Walking onions: week 10 update
by Nick Mathewson 12 May '20

12 May '20

Walking Onions: week 10 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html Week 3: Expanding ENDIVEs into SNIPs. https://lists.torproject.org/pipermail/tor-dev/2020-March/014194.html Week 4: Voting (part 1) and extending circuits. https://lists.torproject.org/pipermail/tor-dev/2020-March/014207.html Week 5: Voting (part 2) https://lists.torproject.org/pipermail/tor-dev/2020-April/014218.html Week 6: Editing, voting rules, and client behavior (part 1) https://lists.torproject.org/pipermail/tor-dev/2020-April/014222.html Week 7: Exit policies: How do they work? https://lists.torproject.org/pipermail/tor-dev/2020-April/014232.html Week 8: Onion services, relay honesty, migration and families https://lists.torproject.org/pipermail/tor-dev/2020-April/014255.html Week 9: (There was no week 9) == Now with an introduction I left the Introduction section of the proposal till late in the game; now I have it in place as a new section 1: == Proposals into the pipeline I've also revised all of the "other proposals" that were logically separate from the rest of Walking Onions. Three of them were ready to go to tor-dev, so I've added them as: 318-limit-protovers.md Limit protover values to 0-63. 319-wide-everything.md RELAY_FRAGMENT cells 320-tap-out-again.md Removing TAP usage from v2 onion services == In the home stretch I've got till the end of the month to wrap up this proposal, so let's see where we are. We have a minimal viable proposal, I believe: if we were to sit down and start implementing, we would probably get most of the way based on what we have now. There are a few additional things I'd like to wrap up before I call the proposal done, however: * I've got all the necessary machinery written up for generating and calculating indices, but before I actually move ahead, I should fill in the details in sections 2 through 4. * One of the original goals of Walking Onions was to permit us to move to alternative topologies where not every relay connects to every other relay. That's not something that we can include in this proposal, since the research here is quite preliminary, but I still want to prepare for it by making sure that clients can handle getting SNIPs that say "this SNIP is only valid when given by a relay of type X". * I should take another pass through the CDDL format to see whether there are any opportunities to save a bunch of bytes. I should also fill in a couple of example SNIPs to see how big they are with this proposal. * There are places all over the proposal where I have marked uncertain points with XXXX. I should go through them and resolve them all before I call the proposal done. [INTRO] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/01-intro…

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev May 2020