Hello!
This email is about a new feature that has some implications here for applications that use Tor. I want to make sure we've thought about and prepared for those implications, since we still have time to tweak this feature.
== Overview
In 0.4.0.x, Tor will begin supporting a new "dormant" mode in which it does not initiate network activity, and tries to avoid CPU wakeups. This mode is intended to help mobile devices sleep, and to minimize the impact of unused Tor clients on the network, and on the devices that run them. For the implementation history, see: https://trac.torproject.org/projects/tor/ticket/28335 and https://trac.torproject.org/projects/tor/ticket/28624
By default, Tor will become dormant if it is a pure client (not a relay, not an onion service[*]), and if it has received not client activity for 24 hours. (You can change the interval with DormantClientTimeout.)
Unlike "DisableNetwork 1", Tor in dormant mode will not close any listener ports, and will become active again as soon as it receives a client connection. But like a "DisableNetwork 1" Tor, a dormant Tor will not build predicted circuits, and will not download directory information -- and therefore, will not bootstrap.
Controllers can make Tor become dormant or non-dormant by sending the commands "SIGNAL DORMANT" and "SIGNAL ACTIVE" respectively -- these are now documented in control-spec.txt.
Dormant state is persistent: if Tor has been idle for 20 hours, and you stop tor and start it again, and idle for 4 more hours, Tor will become dormant. If you stop tor and start it again, Tor will start in dormant mode.
== Documentation
Here's the documentation we have so far -- please let me know if you have ideas for clarification:
DormantClientTimeout N minutes|hours|days|weeks If Tor spends this much time without any client activity, enter a dormant state where automatic circuits are not built, and directory information is not fetched. Does not affect servers or onion services. Must be at least 10 minutes. (Default: 24 hours)
DormantTimeoutDisabledByIdleStreams 0|1 If true, then any open client stream (even one not reading or writing) counts as client activity for the purpose of DormantClientTimeout. If false, then only network activity counts. (Default: 1)
DormantOnFirstStartup 0|1 If true, then the first time Tor starts up with a fresh DataDirectory, it starts in dormant mode, and takes no actions until the user has made a request. (This mode is recommended if installing a Tor client for a user who might not actually use it.) If false, Tor bootstraps the first time it is started, whether it sees a user request or not.
After the first time Tor starts, it begins in dormant mode if it was dormant before, and not otherwise. (Default: 0)
== Compatibility issues
I see two issues here: one minor, and one major.
Minor issue: some applications periodically make requests to the tor network on their own -- for example, Tor Browser's update requests. These requests prevent Tor from becoming dormant. If this is undesired, we can add some way around this.
Major issue: some applications expect that Tor will always bootstrap when it starts, and delay presenting their own UI until Tor is ready. But if Tor starts as dormant, then it will not bootstrap until it receives a request from the client or a "SIGNAL ACTIVE" command from the controller. This could lead to breakage as the application waits for Tor to tell it that it's ready, and Tor waits for somebody to tell it that it's needed.
Are all application developers okay with the issues above, and okay with working around them? If not, we may need changes in Tor before 0.4.0.x is released. Let's talk!
[*] I'd like to add support for dormant onion services, but that's harder. See https://trac.torproject.org/projects/tor/ticket/28424 .
peace,
On Thu, Dec 13, 2018 at 03:56:50PM -0500, Nick Mathewson wrote:
== Compatibility issues
I see two issues here: one minor, and one major.
Minor issue: some applications periodically make requests to the tor network on their own -- for example, Tor Browser's update requests. These requests prevent Tor from becoming dormant. If this is undesired, we can add some way around this.
Major issue: some applications expect that Tor will always bootstrap when it starts, and delay presenting their own UI until Tor is ready. But if Tor starts as dormant, then it will not bootstrap until it receives a request from the client or a "SIGNAL ACTIVE" command from the controller. This could lead to breakage as the application waits for Tor to tell it that it's ready, and Tor waits for somebody to tell it that it's needed.
Are all application developers okay with the issues above, and okay with working around them? If not, we may need changes in Tor before 0.4.0.x is released. Let's talk!
I'm thinking about how Dormant Mode will interact with pluggable transports. Our current transports will (I think) become dormant when tor does, because they only send something when tor does. At most, they may chop up and pad some of tor's packets (like obfs4 iat-mode does), but they don't send traffic of their own while tor is quiet. There's nothing requiring that though: a transport is in general free to send dummy traffic whenever it wants, perhaps as a form of traffic flow obfuscation.
If transports need to become dormant too, then there needs to be some way for tor to tell them to be. Now that https://bugs.torproject.org/28179 (Handle output from PT processes with the event loop) is almost finished, perhaps the stdin/stdout channel would work for it. A request to become dormant really does apply to the entire PT process (not just a single transport or connection), so it's a good match for a process-global channel like stdin. The PT process could respond with a "SIGNAL DORMANT" message on its stdout, which would inform tor that the PT has understood the request and will try to become dormant.
Or simpler but more drastic, tor could terminate its PT subprocesses when it goes dormant (cleanly, by closing their stdin).
Hi,
On Thu, Dec 13, 2018 at 04:15:56PM -0700, David Fifield wrote:
If transports need to become dormant too, then there needs to be some way for tor to tell them to be. Now that https://bugs.torproject.org/28179 (Handle output from PT processes with the event loop) is almost finished, perhaps the stdin/stdout channel would work for it. A request to become dormant really does apply to the entire PT process (not just a single transport or connection), so it's a good match for a process-global channel like stdin. The PT process could respond with a "SIGNAL DORMANT" message on its stdout, which would inform tor that the PT has understood the request and will try to become dormant.
I've just opened bug #28849 for us to try to figure out how this should work both for the new process module itself, but especially for the only consumer of the process module right now: the pluggable transports.
One part of this that especially affects PT's running on Windows is that we would like to disable the Process I/O timer on Windows (which currently ticks once a second) when we are in the dormant mode. This would probably mean that once the stdout or stderr pipe's buffer, in the PT process, is full writing to stdout/stderr will block which would effectively be the same result as described in ticket #26360 (which #28179 as a side-effect also happens to fix).
The easy way out here would of course be to "just" terminate the PT's when we enter the dormant mode and re-spawn them when we leave the dormant mode. If we decide to extend the PT protocol to handle `SIGNAL DORMANT` would we also need to have a method to inform the PT that it can start interacting with the rest of the world again?
Would it be bad if `SIGNAL DORMANT` also means that the PT should not write to stdout/stderr until it's been informed that we are no longer dormant? :-)
Cheers, Alex.
On Fri, Dec 14, 2018 at 04:28:26AM +0100, Alexander Færøy wrote:
On Thu, Dec 13, 2018 at 04:15:56PM -0700, David Fifield wrote:
If transports need to become dormant too, then there needs to be some way for tor to tell them to be. Now that https://bugs.torproject.org/28179 (Handle output from PT processes with the event loop) is almost finished, perhaps the stdin/stdout channel would work for it. A request to become dormant really does apply to the entire PT process (not just a single transport or connection), so it's a good match for a process-global channel like stdin. The PT process could respond with a "SIGNAL DORMANT" message on its stdout, which would inform tor that the PT has understood the request and will try to become dormant.
I've just opened bug #28849 for us to try to figure out how this should work both for the new process module itself, but especially for the only consumer of the process module right now: the pluggable transports.
One part of this that especially affects PT's running on Windows is that we would like to disable the Process I/O timer on Windows (which currently ticks once a second) when we are in the dormant mode. This would probably mean that once the stdout or stderr pipe's buffer, in the PT process, is full writing to stdout/stderr will block which would effectively be the same result as described in ticket #26360 (which #28179 as a side-effect also happens to fix).
The easy way out here would of course be to "just" terminate the PT's when we enter the dormant mode and re-spawn them when we leave the dormant mode. If we decide to extend the PT protocol to handle `SIGNAL DORMANT` would we also need to have a method to inform the PT that it can start interacting with the rest of the world again?
As I think about it, I'm thinking that terminating the subprocess is better from a KISS perspective. It's forward-compatible too, in the sense that you can decide to start sending a "SIGNAL DORMANT" stdin message in the future. And yes, if there's a "SIGNAL DORMANT" there should also be a "SIGNAL ACTIVE" or something.
Would it be bad if `SIGNAL DORMANT` also means that the PT should not write to stdout/stderr until it's been informed that we are no longer dormant? :-)
That could be tricky. It may not be worth it.
On 12/13/18 3:56 PM, Nick Mathewson wrote:
... In 0.4.0.x, Tor will begin supporting a new "dormant" mode in which it does not initiate network activity, and tries to avoid CPU wakeups. ...
== Compatibility issues
I see two issues here: one minor, and one major.
Minor issue: some applications periodically make requests to the tor network on their own -- for example, Tor Browser's update requests. These requests prevent Tor from becoming dormant. If this is undesired, we can add some way around this.
Tor Browser's update requests happen twice per day (and only for desktop browsers). I am not 100% sure, but I don't think any other timer-based requests occur more frequently in Tor Browser, so I don't think this will cause too much activity.
Major issue: some applications expect that Tor will always bootstrap when it starts, and delay presenting their own UI until Tor is ready. But if Tor starts as dormant, then it will not bootstrap until it receives a request from the client or a "SIGNAL ACTIVE" command from the controller. This could lead to breakage as the application waits for Tor to tell it that it's ready, and Tor waits for somebody to tell it that it's needed.
Since most users will use their browser very soon after they start it, for Tor Browser we will probably need to do "SIGNAL ACTIVE" each time during startup. For us it would be convenient if there was a way to do that on the command line or equivalently via a torrc option.
Are all application developers okay with the issues above, and okay with working around them? If not, we may need changes in Tor before 0.4.0.x is released. Let's talk!
I am sure I can find out somewhere, but what is the approximate schedule for 0.4.0.x?
Thanks!
On Thu, Dec 20, 2018 at 9:51 AM Mark Smith mcs@pearlcrescent.com wrote:
On 12/13/18 3:56 PM, Nick Mathewson wrote:
... In 0.4.0.x, Tor will begin supporting a new "dormant" mode in which it does not initiate network activity, and tries to avoid CPU wakeups. ...
== Compatibility issues
I see two issues here: one minor, and one major.
Minor issue: some applications periodically make requests to the tor network on their own -- for example, Tor Browser's update requests. These requests prevent Tor from becoming dormant. If this is undesired, we can add some way around this.
Tor Browser's update requests happen twice per day (and only for desktop browsers). I am not 100% sure, but I don't think any other timer-based requests occur more frequently in Tor Browser, so I don't think this will cause too much activity.
This is enough activity to make sure that under the (current) default schedule, Tor will never become dormant. We should think of a workaround if this is not the intended behavior.
Major issue: some applications expect that Tor will always bootstrap when it starts, and delay presenting their own UI until Tor is ready. But if Tor starts as dormant, then it will not bootstrap until it receives a request from the client or a "SIGNAL ACTIVE" command from the controller. This could lead to breakage as the application waits for Tor to tell it that it's ready, and Tor waits for somebody to tell it that it's needed.
Since most users will use their browser very soon after they start it, for Tor Browser we will probably need to do "SIGNAL ACTIVE" each time during startup. For us it would be convenient if there was a way to do that on the command line or equivalently via a torrc option.
We could add an option like that, sure.
Are all application developers okay with the issues above, and okay with working around them? If not, we may need changes in Tor before 0.4.0.x is released. Let's talk!
I am sure I can find out somewhere, but what is the approximate schedule for 0.4.0.x?
We're planning to put the first alpha out in mid-January, and aim for a stable in mid-April. Our alpha plans usually run on schedule; our stables seem to be happening late.
cheers
Hi Nick,
Is the guard connection closed when becoming dormant?
On 13/12/2018 20:56, Nick Mathewson wrote:
DormantTimeoutDisabledByIdleStreams 0|1 If true, then any open client stream (even one not reading or writing) counts as client activity for the purpose of DormantClientTimeout. If false, then only network activity counts. (Default: 1)
When this option's set to 0 and Tor becomes dormant, will it close any idle client connections that are still open?
Will it close client connections on receiving SIGNAL DORMANT?
If Tor doesn't close client connections when becoming dormant, will it become active again (or send an event that the controller can use to trigger SIGNAL ACTIVE) if there's activity on an open client stream?
Cheers, Michael
Regarding mobile, dormant, battery, etc...
There may still be a ticket for phone, mobile, laptop, wifi users... if an interface change happens, typically interface down event or IP change, then tor should tear down all internet associated state, to not expose traveling user to identifying IP traffic retries sequence nums id nums same tor node sets, etc. Controller should accept pre and post interface change signals too.
On Fri, Dec 21, 2018 at 6:34 AM Michael Rogers michael@briarproject.org wrote:
Hi Nick,
Is the guard connection closed when becoming dormant?
No; it times out independently.
On 13/12/2018 20:56, Nick Mathewson wrote:
DormantTimeoutDisabledByIdleStreams 0|1 If true, then any open client stream (even one not reading or writing) counts as client activity for the purpose of DormantClientTimeout. If false, then only network activity counts. (Default: 1)When this option's set to 0 and Tor becomes dormant, will it close any idle client connections that are still open?
No. By default, it won't go dormant if there are any idle client connections. See DormantTimeoutDisabledByIdleStreams for the option that overrides that behavior.
Will it close client connections on receiving SIGNAL DORMANT?
No.
If Tor doesn't close client connections when becoming dormant, will it become active again (or send an event that the controller can use to trigger SIGNAL ACTIVE) if there's activity on an open client stream?
No, but that might be a good idea if DormantTimeoutDisabledByIdleStreams is set to 0.
Hi Nick,
Thanks very much for the reply. Follow-ups inline below.
On 02/01/2019 21:00, Nick Mathewson wrote:
On Fri, Dec 21, 2018 at 6:34 AM Michael Rogers michael@briarproject.org wrote:
Hi Nick,
Is the guard connection closed when becoming dormant?
No; it times out independently.
That's good news from my point of view, but in that case I think the idea of terminating the pluggable transport process when becoming dormant might need to be reconsidered?
On 13/12/2018 20:56, Nick Mathewson wrote:
DormantTimeoutDisabledByIdleStreams 0|1 If true, then any open client stream (even one not reading or writing) counts as client activity for the purpose of DormantClientTimeout. If false, then only network activity counts. (Default: 1)When this option's set to 0 and Tor becomes dormant, will it close any idle client connections that are still open?
No. By default, it won't go dormant if there are any idle client connections. See DormantTimeoutDisabledByIdleStreams for the option that overrides that behavior.
When DormantTimeoutDisabledByIdleStreams is set to 0, what happens to idle client connections when Tor goes dormant?
Will it close client connections on receiving SIGNAL DORMANT?
No.
If Tor doesn't close client connections when becoming dormant, will it become active again (or send an event that the controller can use to trigger SIGNAL ACTIVE) if there's activity on an open client stream?
No, but that might be a good idea if DormantTimeoutDisabledByIdleStreams is set to 0.
Sorry, do you mean it might be a good idea for Tor to become active again/send an event in that case? Should I open a ticket for this if it looks like we'll need it?
Cheers, Michael
Sorry, I have one more follow-up question.
On 02/01/2019 21:00, Nick Mathewson wrote:
On Fri, Dec 21, 2018 at 6:34 AM Michael Rogers michael@briarproject.org wrote:
Hi Nick,
Is the guard connection closed when becoming dormant?
No; it times out independently.
In that case I assume keepalives from the guard don't prevent Tor from becoming dormant, or wake it from dormant mode. On Android, a keepalive will briefly wake the CPU. When that happens, will Tor consume the keepalive from the network buffer while remaining dormant, or does the controller need to wake Tor from dormant mode periodically to ensure it reads from the guard connection?
Thanks again, Michael
-
Alexander Færøy -
David Fifield -
grarpamp -
Mark Smith -
Michael Rogers -
Nick Mathewson -
Nick Mathewson