tor-relays December 2017

tor-relays@lists.torproject.org

90 participants
63 discussions

Removed x bytes by killing y circuits
by Felix 14 Dec '17

14 Dec '17

Hi everybody Can someone explain the following tor log entry: Removed 528 bytes by killing 385780 circuits; 0 circuits remain alive. (it's the nicest one, see below) Memory stays at 3 - 4 GB before and after. Only tor restart gets rid of the memory. We love to remove 528 bytes. Tor log becomes 370 MB within 18 hours. ---- Tor 0.3.2.6-alpha (git-87012d076ef58bb9) MaxMemInQueues 2 GB ---- We receive some of these: Dec 14 00:11:00.000 [notice] Heartbeat: Tor's uptime is 8 days 3:00 hours, with 508694 circuits open. Dec 14 00:30:08.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:09.000 [notice] Removed 13344 bytes by killing 594572 circuits; 0 circuits remain alive. Also killed 3 non-linked directory connections. Dec 14 01:11:00.000 [notice] Heartbeat: Tor's uptime is 8 days 4:00 hours, with 67530 circuits open. Dec 14 02:07:16.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 02:07:17.000 [notice] Removed 206448 bytes by killing 484352 circuits; 0 circuits remain alive. Also killed 2 non-linked directory connections. Dec 14 03:11:00.000 [notice] Heartbeat: Tor's uptime is 8 days 6:00 hours, with 379182 circuits open. Dec 14 03:13:17.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 03:13:17.000 [notice] Removed 528 bytes by killing 385780 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Dec 14 05:11:00.000 [notice] Heartbeat: Tor's uptime is 8 days 8:00 hours, with 303958 circuits open. Dec 14 05:15:17.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 05:15:18.000 [notice] Removed 528 bytes by killing 317854 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. and lots of: Dec 14 00:30:22.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:22.000 [notice] Removed 528 bytes by killing 221 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Dec 14 00:30:22.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:22.000 [notice] Removed 528 bytes by killing 297 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Dec 14 00:30:22.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:22.000 [notice] Removed 528 bytes by killing 395 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Dec 14 00:30:22.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:22.000 [notice] Removed 528 bytes by killing 468 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Dec 14 00:30:22.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Dec 14 00:30:22.000 [notice] Removed 528 bytes by killing 509 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. -- Thanks and cheers, Felix

2 2

Re: [tor-relays] Issues with faravahar?
by Sina Rabbani 12 Dec '17

12 Dec '17

Dear Tor Relay Operators, Faravahar had some hardware issues last night, while I was on an airplane without internet access. It took me a few hours to respond and I apologize for the delay. Please note that redundancy is built into the network, and I am committed to operating this system with very high availability. A maintenance window to replace the hardware will be announced soon, this system has been running since 2012 and it's time to upgrade. All the best, Sina "Be the change that you wish to see in the world." - Mahatma Gandhi ----- On Dec 12, 2017, at 5:38 PM, teor teor2345(a)gmail.com wrote: > On 13 Dec 2017, at 08:17, tor <tor(a)anondroid.com> wrote: > >>> this time though I ran Wireshark while started tor, and it appears tor is >>> transmitting encrypted info over port 80. >> >>> Surely tor shouldn't be transmitting unencrypted messages? >> >> I'm not an expert on the protocol but you may just be looking at responses from >> the proxy in front of Faravahar? > > Tor's directory documents are public, and signed, and they only contain > public keys. So it's entirely safe to transmit them unencrypted. > > T > _______________________________________________ > tor-relays mailing list > tor-relays(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

1 0

Issues with faravahar?
by r1610091651 12 Dec '17

12 Dec '17

Hi I'm seeing regular issues with faravahar in logs lately. Is somebody working on this? Logs: Dec 12 10:32:56.000 [warn] HTTP status 502 ("Bad Gateway") was unexpected while uploading descriptor to server '154.35.175.225:80'. Possibly the server is misconfigured? Dec 12 10:33:56.000 [warn] Received http status code 502 ("Bad Gateway") from server '154.35.175.225:80' while fetching "/tor/server/d/706E3C29265BD073DF77DC457A3CD8B1BC16C6E6+E223A1B036E3F7315DCADE32F6A4428F15148987.z". I'll try again soon. Thanks

8 8

DoS attacks on multiple relays
by null 12 Dec '17

12 Dec '17

Hi, We're experiencing what looks like a DoS attack on multiple relays in our family: https://atlas.torproject.org/#search/family:CBEAE10CBBB86C51059246B2EF92EB2… The relays are currently running Tor 0.3.1.9 on Linux kernel 4.4.0 (although when the problem started the relays were running Tor 0.3.1.8). The attack knocked 3 of 6 relays offline overnight. By the time we looked at logs, the Tor service had stopped and this was the last line in the log: "Tor[xyz]: Failing because we have 16351 connections already. Please read doc/TUNING for guidance." The attack is still ongoing. When it's happening, the number of connections rises very rapidly, until the attack succeeds in stopping the service. $ ss -s Total: 15855 (kernel 0) TCP: 24520 (estab 23969, closed 305, orphaned 31, synrecv 0, timewait 261/0), ports 0 Transport Total IP IPv6 * 0 - - RAW 0 0 0 UDP 8 4 4 TCP 24215 24213 2 INET 24223 24217 6 FRAG 0 0 0 ... and only a few seconds later: $ ss -s Total: 12120 (kernel 0) TCP: 27389 (estab 20026, closed 1906, orphaned 45, synrecv 0, timewait 1587/0), ports 0 Transport Total IP IPv6 * 0 - - RAW 0 0 0 UDP 8 4 4 TCP 25483 25481 2 INET 25491 25485 6 FRAG 0 0 0 That's obviously much larger than the normal number of connections, more than we've ever seen, and seems like more connections than would be needed for a relay. We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor: elif [ "$system_max" -gt "40000" ] ; then MAX_FILEDESCRIPTORS=16384 Surely that is high enough for normal service? We haven't started looking into where the traffic is coming from or other characteristics. We are wondering if: 1) this is a known attack, 2) if other operators are experiencing it, 3) if there are any ideas for mitigating it, and 4) if any additional information would be helpful. Thanks.

7 11

Action relays on 188.214.30.0/24 subnet
by x9p 12 Dec '17

12 Dec '17

Shouldn't an action be taken against such relays? https://atlas.torproject.org/#search/188.214.30 They are clearly being run by the same entity/person, probably in an automated fashion, don't have family set, and are using the same datacenter. cheers. x9p

2 3

UbuntuCore stats update
by Chad MILLER 11 Dec '17

11 Dec '17

Hi all. I generate* the packages that make up those UbuntuCore relays and bridges you hear about some time in here. I intended it to be a low-friction way normal joes can help Tor. There have been a good number of volunteers. The automatic-update system of Snap means the security update of a few days ago gives some population info through download stats. About 2200+ machines updated to last week's release. Almost all are amd64, though a few percent are i386 or armhf. I don't know of any arm64 yet. They're mostly desktops and servers. I see several new downloads every day. Judging from the new Atlas, about 800 are have checked in to try to join the consensus, and a little more than 100 are active at any time. Some working details: The package has a kill-switch so that it no longer starts after a few months of staleness (if I'm ever hit by a bus). At first launch, Tor creates a key and the last two bits of the key determines the role of the instance, with a 1/4 chance of becoming a obfs4 bridge. The default bandwidth limit is a modest 4 megabits per second. Also by default, it tries to punch holes in NAT to make itself available for incoming connections, but I don't have a lot of confidence that is often successful. I remain on this list and am always happy to answer questions or suggestions. * http://bazaar.launchpad.net/~privacy-squad/+junk/tor-middle-relay-snap/files -- Chad Miller chad.org gpg:a806deac30420066

4 5

Weird issue caused downtime of my relay
by Aneesh Dogra 11 Dec '17

11 Dec '17

Hello Everyone, I run a tor exit relay named "rippedlion". I was out for vacation past few days and saw I had a 2 day downtime. I had a look at the logs and apparently /var/lib/tor got detected to be owned as <unknown>? How do I prevent this in future? Snippet from notices.log: Dec 08 06:54:57.000 [warn] Tried to establish rendezvous on non-OR circuit with purpose Acting as rendevous (pending) Dec 08 06:56:25.000 [warn] Tried to establish rendezvous on non-OR circuit with purpose Acting as rendevous (pending) Dec 08 07:00:21.000 [warn] /var/lib/tor is not owned by this user (root, 0) but by <unknown> (112). Perhaps you are running Tor as the wrong user? Dec 08 07:00:21.000 [err] Can't create/check datadirectory /var/lib/tor Dec 08 07:00:21.000 [err] Unable to update Ed25519 keys! Exiting. PS: I am now running tor; as a low privledged user that only has access to /var/lib/tor and /var/log/tor. Thanks! -- Regardless, I hope you're well and happy - Aneesh

2 2

Re: [tor-relays] DoS attacks on multiple relays
by null 11 Dec '17

11 Dec '17

@ x9p: > # netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk > -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort > | egrep -v ' 1 | 2 | 3 ' > > with this information in hand, double the max of it (mine was 10 > connections from 188.214.30.0/24): > > 10 188.214.30 > > iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20 > --connlimit-mask 24 -j REJECT --reject-with tcp-reset Thank you! This was extremely helpful. In our case we found a handful of IPs that had *thousands* of concurrent connections on several of our relays. The offending IPs were not in the consensus. After restarting the Tor service, these suspect connections come back rapidly, again across several of our relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation. @ Scott Bennett: > What you are seeing is most likely the same phenomenon brought up on > this list repeatedly over at least the last decade or so. That phenomenon > is providing HSDir service, or perhaps a rendez-vous point, for a popular > hidden service. > If you don't like it, you can set > > HidServDirectoryV2 0 Thanks for your reply. The data suggests this was not the case (this time). Some of these relays have been up almost a year with the same configuration, often with the HSDir flag. The recent issues just started occurring, and happened across several relays during the same timeframe. Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log: [WARN] Skipping obsolete configuration option 'HidServDirectoryV2' It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this instead: MinUptimeHidServDirectoryV2 52 weeks Anyone have any info on why HidServDirectoryV2 is consider obsolete? Is using MinUptimeHidServDirectoryV2 instead going to achieve the same effect? >> We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks >> like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor: >> >> elif [ "$system_max" -gt "40000" ] ; then >> MAX_FILEDESCRIPTORS=16384 >> >> Surely that is high enough for normal service? > > If by normal you mean "low traffic", then yes, it's probably enough. > However, that's really not very high in a general sense. Why does the default /etc/init.d/tor (from the Debian/Ubuntu pkg) cap MAX_FILEDESCRIPTORS at 16384 then? We have it set to 64000 otherwise. Obviously we could comment out these lines, but it seems odd the default config tries to cap it at 16384 if that's too low for a high traffic relay. Thanks everyone for your replies!

5 15

ISP is aking me to send a selfie holding my identity card
by Tanous 10 Dec '17

10 Dec '17

Hi, Im running an exit relay for 116 days. Today i received an email from my ISP saying that my account has been locked for security reasons. They asked me to send a copy of my identity card and a selfie holding it. I found that very odd and i feel uncomfortable sending that data to them. Should i give up to running my exit relay and find another ISP? By the way, i had received an abuse complaint a day before, due to Brute force attempts. Best Regards, Tanous

11 26

Re: [tor-relays] DoS attacks on multiple relays
by teor 09 Dec '17

09 Dec '17

On 9 Dec 2017, at 13:24, x9p <tor(a)x9p.org> wrote: >> By "private guards" do you mean "bridges"? >> That would be a very bad idea: it would make the bridge and its onion >> services stand out within minutes or hours on the network, because >> each circuit gets a different middle node, and the nodes would not >> be evenly distributed. > > Sorry, I meant EntryNodes > >> If you block a guards on an onion service, it will look different, but >> that >> might be unnoticeable for a few months. (More precisely, it's safe in >> proportion the guard rotation period, divided by the number of related >> onion services blocking those guards, divided by the consensus weight >> fraction of blocked guards. We don't expect that people will do this >> calculation themselves, which is why we say "don't do that".) > > Would it be a better approach than firewall blocking, setting > "ExcludeNodes + StrictNodes" with the offending/suspicious fingerprints? No, this is much worse: it blocks these nodes for guard, middle, intro, and rend points. That's even more detectable than blocking middle nodes after a bridge. If you must block, only block a few guards, and only short-term. This is a hard area to get right - reducing the threat of node subsets needs more research. T

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-relays December 2017