teor teor2345@gmail.com wrote:
On 8 Jun 2017, at 18:44, Scott Bennett bennett@sdf.org wrote:
Roger Dingledine arma@mit.edu wrote:
Or at least partially solved, at any rate. The problem is solved,but the mystery remains.
On Sun, Jun 04, 2017 at 07:14:06PM -0500, Scott Bennett wrote:
Which versions are the Running votes coming from versus the non-Running?You can see the votes at https://www.seul.org/~arma/moria1-v3-status-votes
I have a few commands in a crontab entry that extract relay IP addressesfrom the most recently received consensus, sort them, and load them into a table in pf. They run every 15 minutes. Anything coming from the addresses in the table is immediately passed. Anything not passed gets checked against a much larger table of probers, attempted intruders, etc. and is blocked if it matches a table entry.
Well heck. Sounds like we have a winner here.
Turn off your weird censorship infrastructure, confirm that things start working after a while, and now you have something to debug. :)
The misuse of the word "censorship" aside, I tried disabling pf andrestarting tor. To my surprise, the authorities connected to my relay successfully and distributed its information in subsequent consensuses! I do not know why, which is the remaining mystery. However, having to turn off one's firewall in order to run tor is not a solution to the problem.
If your firewall is blocking connections that tor needs to make, turning off that rule is precisely the solution to the problem.
As noted more than once previously, the pf rules *pass* all traffic from relay addresses *first*, so that traffic has already gone on to tor before the block list is applied. As also noted previously, the rules had not (and still have not) been changed in a long time. They were working just fine for years, and then suddenly they were not. Now they are working just fine again since tor got rebuilt, so they are not the problem.
What I think I know is that the mismatched openssl stub and library versions somehow interact with pf and pre-0.3.0.7 tor authorities in some mysterious manner that causes connections from those authorities to my relay to fail, but connections from 0.3.0.7 authorities to work just fine. Having the stub version and the library version be identical apparently avoids the erroneous interaction(s).
It's possible. But to confirm, you'd need to provide the sections of the firewall log that show what happens when a directory authority tries to check the reachability of your relay.
That stuff isn't logged because it would soon exhaust the available disk space with garbage.
(As an aside, I'd encourage you not to keep logs at this level on an ongoing basis, they would contain client IP addresses and connection times.)
That's one reason; the other is the space and kernel overhead requirements.
Or perhaps your firewall collected enough state to want to block tor while it was running, and lost that state when you turned it off and on again.
I don't understand how an application could directly interfere with the packet filter's operation. I've previously posted the rules. As I described, it appears to be a problem of openssl misbehaving, apparently due to mismatched stub and library versions, though I don't understand how openssl trouble could occur with pf enabled and not occur with pf disabled. In any case, rebuilding tor so that the stub and library versions match has eliminated the problem. tor now plays nicely with the authorities, regardless of whether pf is enabled. Now, having said that, I should mention that tor does mess around with /dev/bpf, at least during startup, which is one reason why tor has to have root privileges during startup. I've never looked into exactly what it does with /dev/bpf, however, so I have no idea whether that could be involved and caused to malfunction somehow by openssl problems.
You have a complex enough system that there are probably more possibilities neither of us have thought of.
Complex? Compared to what? You've already seen the handful of rules involved, which includes some near duplication of ORPort rules to provide the same protection from miscreants to the DirPort that is given to the ORPort. In a nutshell, the rules look at incoming SYN packets to see if they come from tor relay addresses and immediately lets pass those that do. (By doing this step first, relay traffic is protected from the next step, even if the relay's address has gotten onto the block list, e.g., an exit that has been abused by some cracker or wannabe-cracker.) If they do not, then they are checked against a very large block list and, if found in the block list, are dropped with no RST. (Yes, I know dropping without RST violates the TCP standards, but crackers deserve no courtesies. Slowing them down is good. They can just wait until they time out.) Any SYN packets destined to the ORPort or DirPort that survive at this point are either from clients or from relays that have just started up and therefore do not yet appear in the relays list used in the first step. If a relay is new or has been down long enough to have been omitted from at least one consensus *and* is in the block list, then its SYNs will be dropped until up to 15 minutes after the next consensus that lists it has been downloaded. That is the only, and very temporary, situation in which the rules block traffic from legitimate source addresses. Authorities will rarely, and only briefly (i.e., not much over a couple of hours at most, certainly not *days*), ever be in this situation. There should *never* be a time when a majority of authorities is in this situation simultaneously. It's worth pointing out that, without pf's very efficient protection (or that of some other packet filter), tor itself is stuck with the load of handling useless inbound connection attempts. pf takes that load off of tor when it is caused by IPv4 addresses that are known sources of trouble.
I'd keep an eye on things over the next few weeks.
I think I will be keeping an eye on openssl updates to make sure that tor gets rebuilt each time the openssl library gets replaced, now that I know that openssl updates can subvert one of the advantages of dynamically linked libraries. :-(
Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************