Hi,
I see a number of warning log messages on a dedicated server: [WARN] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [27615 similar message(s) suppressed in last 60 seconds]
The relay is running on a dedicated hardware with the following specifications:
CPU: Intel(R) Xeon(TM) CPU 3.00GHz RAM: 6G Kernel: Linux 3.16.0-5-amd64 Tor version: 0.3.2.9 flags: Fast, Guard, HSDir, Running, Stable, V2Dir, Valid exit policy: reject *:*
Setting the NumCPUs option to the actual number of CPUs (2) didn't help. Is this hardware really too old/slow to run a relay on one ethernet Gigabit link?
Cheers, ~Vasilis
On Wed, Feb 21, 2018 at 01:13:00PM +0000, Vasilis wrote:
I see a number of warning log messages on a dedicated server: [WARN] Your computer is too slow to handle this many circuit creation requests!
You get that warning message when there are too many create cells coming in, and your relay ends up sending back preemptively destroy cells for some of them. That is, it tries to estimate internally how long it will take to handle the current queue of create cells, and if the queue gets so big that the one that just arrived will take several seconds before it can be processed, Tor just sends back a destroy cell instead, and gives you this warn.
The flood of circuits created by the ddos storm will be causing this sort of warning sometimes. For example, my FreeBogatov relay gets 30-70 million create requests per 6 hours, and when that number goes over about 100 million, there are times where it can't keep up.
(Careful though because the heartbeat message about number of circuits does not count circuits that come from client connections. That is, the circuits in the heartbeat count are only circuits that come via other relays. So non-Guards are giving you a reasonably accurate count, and Guards are leaving out an unknown number of circuits from their count, and that unknown number could be quite large.)
Ultimately, the fix needs to be that more and more relays upgrade to a version of Tor tht includes the DDoS mitigation. One of the main goals of the mitigation is not to help *your* relay in particular, since hey maybe your relay is huge and it can keep up, but rather to slow down the mass of circuits heading towards *other* relays after yours.
That is, you need *other* relays to deploy the mitigation in order to help you. https://en.wikipedia.org/wiki/Herd_immunity
Setting the NumCPUs option to the actual number of CPUs (2) didn't help.
Are you sure you only have 2 cores? These days each cpu has many cores, so a system with 2 cpus could easily have 8 cores.
Is this hardware really too old/slow to run a relay on one ethernet Gigabit link?
Well, there are times where it isn't able to keep up. But if you turn off the relay or turn down its capacity, then it will just increase the load on the other relays. So I think we shouldn't worry too much about these warnings during this period of overload.
Oh, I guess I should ask: are you using 0.3.3.2-alpha or a version with the ddos mitigation? If not, that's a clear next step.
--Roger
Roger Dingledine:
On Wed, Feb 21, 2018 at 01:13:00PM +0000, Vasilis wrote:
I see a number of warning log messages on a dedicated server: [WARN] Your computer is too slow to handle this many circuit creation requests!
You get that warning message when there are too many create cells coming in, and your relay ends up sending back preemptively destroy cells for some of them. That is, it tries to estimate internally how long it will take to handle the current queue of create cells, and if the queue gets so big that the one that just arrived will take several seconds before it can be processed, Tor just sends back a destroy cell instead, and gives you this warn.
The flood of circuits created by the ddos storm will be causing this sort of warning sometimes. For example, my FreeBogatov relay gets 30-70 million create requests per 6 hours, and when that number goes over about 100 million, there are times where it can't keep up.
(Careful though because the heartbeat message about number of circuits does not count circuits that come from client connections. That is, the circuits in the heartbeat count are only circuits that come via other relays. So non-Guards are giving you a reasonably accurate count, and Guards are leaving out an unknown number of circuits from their count, and that unknown number could be quite large.)
Ultimately, the fix needs to be that more and more relays upgrade to a version of Tor tht includes the DDoS mitigation. One of the main goals of the mitigation is not to help *your* relay in particular, since hey maybe your relay is huge and it can keep up, but rather to slow down the mass of circuits heading towards *other* relays after yours.
That is, you need *other* relays to deploy the mitigation in order to help you. https://en.wikipedia.org/wiki/Herd_immunity
Makes sense great explanation, thank you! Wasn't planning to stop running/administering any of the relays.
Setting the NumCPUs option to the actual number of CPUs (2) didn't help.
Are you sure you only have 2 cores? These days each cpu has many cores, so a system with 2 cpus could easily have 8 cores.
It's an old processor with 2 CPU and 1 core per CPU.
Is this hardware really too old/slow to run a relay on one ethernet Gigabit link?
Well, there are times where it isn't able to keep up. But if you turn off the relay or turn down its capacity, then it will just increase the load on the other relays. So I think we shouldn't worry too much about these warnings during this period of overload.
Oh, I guess I should ask: are you using 0.3.3.2-alpha or a version with the ddos mitigation? If not, that's a clear next step.
I 'll upgrade to the alpha version and closely monitor its activity.
Thanks, ~Vasilis
Hi,
Running for more than a week the alpha version 0.3.3.2 (git-7b1d356bdb76607d) the issue seems to be resolved.
Heartbeat: Tor's uptime is 7 days 11:59 hours, with 19157 circuits open. I've sent 2372.16 GB and received 2372.27 GB.
Cheers, ~Vasilis
Hi,
*UPDATE** I'm still seeing these warning messages but in a lower frequency: Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [1077 similar message(s) suppressed in last 60 seconds]
The defenses seems to be working (?): DoS mitigation since startup: 45482775 circuits rejected, 157 marked addresses. 2187600 connections closed. 993 single hop clients refused.
~Vasilis
tor-relays@lists.torproject.org
-
Roger Dingledine -
Vasilis