tor-relays July 2022

tor-relays@lists.torproject.org

20 participants
13 discussions

How to reduce tor CPU load on a single bridge?
by David Fifield 19 Dec '22

19 Dec '22

The main Snowflake bridge (https://metrics.torproject.org/rs.html#details/5481936581E23D2D178105D44DB6…) is starting to become overloaded, because of a recent substantial increase in users. I think the host has sufficient CPU and memory headroom, and pluggable transport process (that receies WebSocket connections and forwards them to tor) is scaling across multiple cores. But the tor process is constantly using 100% of one CPU core, and I suspect that the tor process has become a bottleneck. Here are issues about a recent CPU upgrade on the bridge, and observations about the proportion of CPU used by different processes: https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… I have the impression that tor cannot use more than one CPU core—is that correct? If so, what can be done to permit a bridge to scale beyond 1×100% CPU? We can fairly easily scale the Snowflake-specific components around the tor process, but ultimately, a tor client process expects to connect to a bridge having a certain fingerprint, and that is the part I don't know how to easily scale. * Surely it's not possible to run multiple instances of tor with the same fingerprint? Or is it? Does the answer change if all instances are on the same IP address? If the OR ports are never used? * OnionBalance does not help with this, correct? * Are there configuration options we could set to increase parallelism? * Is migrating to a host with better single-core performance the only immediate option for scaling the tor process? Separate from the topic of scaling a single bridge, here is a past issue with thoughts on scaling beyond one bridge. it looks as though there are not ways to do it that do not require changes to the way tor handles its Bridge lines. https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… * Using multiple snowflake Bridge lines does not work well, despite that we could arrange to have the Snowflake proxy connect the user to the expected bridge, because tor will try to connect to all of them, not choose one at random. * Removing the fingerprint from the snowflake Bridge line in Tor Browser would permit the Snowflake proxies to round-robin clients over several bridges, but then the first hop would be unauthenticated (at the Tor layer). It would be nice if it were possible to specify a small set of permitted bridge fingerprints.

7 57

relayor v22.0.0-rc is released
by nusenu 03 Dec '22

03 Dec '22

Hi, relayor v22.0.0-rc is released. relayor is an ansible role that helps you with running tor relays with minimal effort (automate everything). https://github.com/nusenu/ansible-relayor#main-benefits-for-a-tor-relay-ope… Changes since relayor v21.2.0-rc: * MetricsPort support improvements: * generate nginx reverse server config for remote prometheus scraping on the relay (we do not install nginx) * generate htpasswd file for HTTP basic auth of MetricsPort on the relay * debian/ubuntu: support new alpha repos (branch name is no longer included in the repo name). * FreeBSD 12.3 is supported kind regards, nusenu -- https://nusenu.github.io

2 4

[New Initiative] Tor Weather: Improving the Tor Weather
by Sarthik Gupta 18 Aug '22

18 Aug '22

Hello everyone, As we all know, nodes are the building blocks of the tor-network. More often than not these nodes require some maintenance, which could be in the form of software upgrades, troubleshooting issues, rebooting the relays in case of failures, etc. As of today, if any relay disappears from the tor-network, no one will know. Tor-Weather aims at solving this by creating a notification service which relay operators can subscribe to in order to get various types of updates for their relays. The tor-weather service will offer a plethora of notifications options for the relays. These include, the node being down, running on EOL/Outdated version, losing a flag, ranking in top 20/50/100, etc. These notifications can be subscribed & customized by the relay operators to fit their needs using a web-frontend. Folks interested in the project can refer this <https://lists.torproject.org/pipermail/tor-dev/2022-May/014734.html> thread in the tor-dev mailing list for regular updates. Suggestions are always welcomed! Please reach out to us in irc (#tor-dev) for any ideas, questions, or suggestions you might have. Thanks, Sarthik Gupta

4 6

Setting a bridge to automatically change IP adresses
by Keifer Bly 10 Aug '22

10 Aug '22

3 3

When will relays disappear form tor metrics
by Anders Burmeister 01 Aug '22

01 Aug '22

Hi heroes Ended and deleted my two relays Aerodynamik02 and Aerodynamik03. When can I expect they will disappear from tor metrics?https://metrics.torproject.org/rs.html#details/EFEACD781604EB80FBC0… Best regards Anders

2 1

Impact on running a tor relay on other internet services?
by Thoughts 28 Jul '22

28 Jul '22

Hi all - I've been running a TOR non-exit relay for several months now. Its rare, but I'm seeing what I believe is the occasional connection attack, with my relay complaining about the number of connections and suggesting I reduce capacity. Those are rare, and most of the time my server is running at about 20% CPU. During attacks, which seem unrelated to my Tor Upload/Download rate, CPU jumps to well over 100% (quad core, so 400% is max). I'd normally just ignore this, but it seems to be impacting other aspects of my network experience: Messenger Rooms will unexpected close, NetFlix gets "unable to stream this title", family complains about slow and dropped connections, etc. Just had it happen a few minutes ago with a Messenger Room and sure enough, CPU is at 130%, even though I'm only pumping about 15MB/Sec (37.5MB/S limit, 56.2 burst, 40.3 observered) over my gigabit ISP connection. Speedtest shows the performing within acceptable parameters. So contemplating what I can do, since this is bothersome. I've come up with a few alternatives, and curious about your thoughts: 1) Do some type of connection limiting at my PFSense Plus firewall. Perhaps limiting things to, say, 30 connections per IP address? Not even sure that is possible, but figure it might lighten the load on the TOR server. 2) Drop being a TOR non-exit relay and convert to a bridge. Not sure how long, if ever, it would take for my IP address, which is now public, to fade off of block lists... Not ideal, but at least as a bridge I'd still be servicing the environment. 3) Try connection limiting via iptables on the TOR host. Just seems like doing that at the firewall would be better. Thoughts? Kevin

1 1

relay memory leak?
by Fran 26 Jul '22

26 Jul '22

Hey, new non-exit relay, Debian 11, tor 0.4.7.8-1~d11.bullseye+1, ~ 1 week old (-> no guard) KVM VM with atm 4 cores, host passthrough AMD EPYC (-> AES HW accel.). As can be seen at the attached screenshots memory consumption is irritating as well as the quite high CPU load. All was fine when it had ~100 Mbit/s but then onion skins exploded (110 per second -> up to 4k per second) as well as CPU and memory. Tor complains: > Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. And from time to time memory killer takes action torrc is pretty basic: Nickname 123 ContactInfo 123 RunAsDaemon 1 Log notice syslog RelayBandwidthRate 2X MBytes RelayBandwidthBurst 2X MBytes SocksPort 0 ControlSocket 0 CookieAuthentication 0 AvoidDiskWrites 1 Address xxxx OutboundBindAddress yyyy ORPort xxxx:yyy Address [zzzz] OutboundBindAddress [zzz] ORPort [zzz]:xxx MetricsPort hhhh:sss MetricsPortPolicy accept fffffff DirPort yy Sandbox 1 NoExec 1 CellStatistics 1 ExtraInfoStatistics 1 ConnDirectionStatistics 1 EntryStatistics 1 ExitPortStatistics 1 HiddenServiceStatistics 1 Ideas/suggestions (apart from limiting BW) to fix this? Thanks fran

3 5

Next Tor Relay operator meetup + MCH activities
by gus 26 Jul '22

26 Jul '22

Hi everyone, This month we won't have our monthly online check-in as many of us are traveling or on vacation mode. But, don't feel sad, we will have two activities this weekend! 1. (livestream): Modernizing the Tor Ecosystem for the Future, 2022-07-24, 15:00-15:50 with Alexander Færøy (ahf), Network Team Lead. Read more: https://program.mch2022.org/mch2021-2020/talk/MUP7MX/ 2. Relay Operator Meetup @ May Contain Hackers (MCH2022) ahf, Artikel10 and many others Tor contributors are at MCH and they are organizing a relay operator meetup! If you are at MCH2022, please get in touch with them!! https://blog.torproject.org/event/mch2022/ The next meetup will be at the end of August. I'll send the invitation at the beginning of August. :) cheers, Gus -- The Tor Project Community Team Lead

3 3

Tor bridges limit
by Me 25 Jul '22

25 Jul '22

Good day all. I am currently running two middle relays on my home network and was wondering if I can add any bridges to my existing set up. I am aware of the limit of two middle relays per ip address but do not see anything stating whether there was any such limit for bridges. I am also not clear on whether the bridges are recognized by the tor network in the same manner as Middle relays and whether the two unit limit applies to the total of devices connected. Thank you in advance for any guidance and always remember.... They are watching

2 1

Relay Overloaded and Dropping Onionskins
by bidulock_ringrose＠aleeas.com 21 Jul '22

21 Jul '22

Greetings! I hope this is the right list to be asking this, if it is not please forgive me. I am purposefully omitting some identifying information for privacy sake. I run 2 non-exit relays both with an advertised bandwidth of around 8 MiB/s each. I have noticed that they have been overloaded a lot lately. These relays have been bottlenecked at the 3-4 MiB/s mark ever since I put them online. Upon further investigation, when I curled the MetricsPort according to https://support.torproject.org/relay-operators/relay-bridge-overloaded/, the following metrics stood out to me. Both relays run on the same machine with the same IP address. I hope the obfuscation makes sense. Side note: I am using Toralf's ddos-inbound script, which has not dropped any connections at all for me when using the -b then -s switch. CPU utilization is high (80%) on one core but low on the rest (5-30%) In the syslog, I also get spammed with "Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [28xxx similar message(s) suppressed in last 34200 seconds]" Relay 1: tor_relay_load_onionskins_total{type="ntor_v3",action="processed"} 750xxxx tor_relay_load_onionskins_total{type="ntor_v3",action="dropped"} 17 tor_relay_load_global_rate_limit_reached_total{side="read"} 6xxxx tor_relay_load_global_rate_limit_reached_total{side="write"} 17xxxx Relay 2: tor_relay_load_onionskins_total{type="ntor_v3",action="processed"} 10xxxxxx tor_relay_load_onionskins_total{type="ntor_v3",action="dropped"} 28xxxx tor_relay_load_global_rate_limit_reached_total{side="read"} 20xxxxx tor_relay_load_global_rate_limit_reached_total{side="write"} 19xxxx All other metrics are normal according to the article on overloaded relays. This runs in a Debian Proxmox VM using the host cpu, so no CPU virtualization. 4 cores, 8GB memory, and AES is supported. It's 2x Xeon 2628v3s with NUMA enabled in the VM (2 sockets, 2 cores per socket). Enabling NUMA and de-virtualizing the CPU has helped increase my top bandwidth by around .7 to .9 mbytes/s, but still not great. Thank you in advance.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-relays July 2022