-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Dear Tor BW Authories,
we'd like to bring the following issue to your attention. Since the last two weeks the amount of unmeasured relays is steadily rising [1].
unmeasured relays on 2015-05-01: 387 unmeasured relays today: 1087 (and growing)
Graph showing the last 5 months: https://trac.torproject.org/projects/tor/attachment/ticket/13450/unmeasured_...
Data: https://trac.torproject.org/projects/tor/attachment/ticket/13450/2015-01-01_...
Would be great if you had some time to look into this: https://trac.torproject.org/projects/tor/ticket/13450 (follow ups should probably go to trac)
..so bw doesn't go wasted. A contributed but unused relay doesn't make relay ops happy.
thanks!
[1] https://lists.torproject.org/pipermail/tor-relays/2015-May/006946.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Graph showing the last 5 months: https://trac.torproject.org/projects/tor/attachment/ticket/13450/unmeasured_...
you
can also see that the adv. bw. is going down since the beginning of the month: https://metrics.torproject.org/bandwidth.html
while the relay count didn't change that much: https://metrics.torproject.org/networksize.html
On Thu, May 14, 2015 at 08:01:19PM +0000, nusenu wrote:
Dear Tor BW Authories,
we'd like to bring the following issue to your attention. Since the last two weeks the amount of unmeasured relays is steadily rising [1].
Hello nusenu,
Thanks for the kick.
For other statistics, each hour I copy over the votes that moria1 sees: http://freehaven.net/~arma/moria1-v3-status-votes
And there's also https://consensus-health.torproject.org/#bwauthstatus
It is clear that the bwauth scripts are buggy and need a rewrite. And not just buggy -- I think their design is probably flawed too.
I know of several research groups who have been working on better bandwidth estimation schemes, with the main goal of making them more robust to cheating, but with the secondary goal of still being adequately accurate.
I had been hoping that one of those would be ready by now, but the research group I've been paying the most attention to has quite a bit of work left to do before they have a cheating-resistance estimation scheme that doesn't open up too many other problems.
Another hope we have is Aaron Gibson's upcoming OTF fellowship, which focuses on improving the bwauth system. I think we'll have to find the right balance there between a redesign, which would take a while but could be better for us in the long run, and bugfixes on the current system, which would help things in the short term but take time away from the redesign.
I'll call Aaron's attention to this thread too and see if he can provide some insights.
--Roger
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
now even DocTor starts to complain
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005772....
On Mon, May 18, 2015 at 09:02:27PM +0000, nusenu wrote:
now even DocTor starts to complain
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005772....
Yep. moria1's bwauth seems to be working fine, so I'm the only one paying attention to the thread here, and also the only one who doesn't have anything to fix. :(
I haven't heard back from Aaron Gibson yet.
In the mean time, I've asked some other developers to get one or two new bwauths going, which will hopefully lessen the magnitude of the problem in the short term, until somebody can fix the underlying bwauth flaws.
--Roger
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Roger Dingledine:
Yep. moria1's bwauth seems to be working fine, so I'm the only one paying attention to the thread here
sounds alarming if people operating tor's key infrastructure do not care (or have not enough time to care)
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005774....:
WARNING: The following directory authorities are not reporting bandwidth scanner results: tor26
If one more bw auth goes down, all relays will get the unmeasured flag, because the threshold of 3 votes is no longer reached, is that understanding of the spec correct?
current stats: 2015-05-19 19:00: 1367 unmeasured ("lost" ~1000 relays and 10GBit/s bw since 2015-05-01)
On Tue, May 19, 2015 at 08:47:56PM +0000, nusenu wrote:
sounds alarming if people operating tor's key infrastructure do not care (or have not enough time to care)
Well, we are working on a variety of approaches now for fixing it. Some in the short term (get more bwauths, fix bugs), some in the long term (make and deploy better design for computing weights).
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005774....:
WARNING: The following directory authorities are not reporting bandwidth scanner results: tor26
If one more bw auth goes down, all relays will get the unmeasured flag, because the threshold of 3 votes is no longer reached, is that understanding of the spec correct?
No, I believe if there are only two dir auths who express weight opinions in their votes, then the group collectively backs off to using self-advertised weights.
current stats: 2015-05-19 19:00: 1367 unmeasured ("lost" ~1000 relays and 10GBit/s bw since 2015-05-01)
We also took some steps in the past few weeks to cut out extra tiny relays. It would be good for somebody to track a list of what those events were.
I don't mean to say that we lost zero relays due to the above issue; but I think an unknown number of them are better explained by other events in the network.
--Roger
Hi Roger,
On 19.05.2015, at 23:01, Roger Dingledine arma@mit.edu wrote:
Well, we are working on a variety of approaches now for fixing it. Some in the short term (get more bwauths, fix bugs), some in the long term (make and deploy better design for computing weights).
What does it take to run a bwauth and/or dir auth? I would be happy to help, if I can.
Jannis
To be a bwauth you have to be a dirauth, if the bwauth draft spec I read was correct. But how do you become a dirauth? The addresses are hardcoded into Tor, so it's not like I could just spin up a dirauth in an evening and let the network do the rest. There's got to be more to it.
I was interested in hearing the response as well, so...
Bump.
Jannis Wiese:
What does it take to run a bwauth and/or dir auth? I would be happy to help, if I can.
Jannis
Matt Speak Freely
On Wednesday, 20 May 2015, Speak Freely when2plus2is5@riseup.net wrote:
To be a bwauth you have to be a dirauth, if the bwauth draft spec I read was correct. But how do you become a dirauth? The addresses are hardcoded into Tor, so it's not like I could just spin up a dirauth in an evening and let the network do the rest. There's got to be more to it.
You don't need to *be* a DirAuth, but you do need to work closely with a DirAuth to have them use your data to vote. Being a DirAuth requires an incredible amount of trust by the community, established operational skills, and usually a solid public reputation.
Right now Tor has two new BWAuths spinning up, one of which will hopefully be included in the vote this week or weekend. One of the things we're finding through the process is that the torflow scripts are a little buggy. So far they've been buggy in obvious ways, but the worry is that they're buggy in non-obvious way - that some change in the dependencies since they've been written are causing wrong results. We're not sure if that's the case yet. If it is, we will hopefully be getting an indication soon, and will want to work on debugging it, possible with the affected relay operators.
The code is at https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority
While reviewing and testing the code may be useful, I would caution against everyone spinning up a BWAuth though. By it's very nature, it uses a lot of bandwidth. I don't know if it would put strain on the network if suddenly 16 BWAuths were running instead of 6. (Maybe someone better at the math part of the network bandwidth would have an idea.)
-tom
Hi all,
Wanted to provide an update (even if it's not as good news as I hoped to give) because I know this is a very frustrating issue for everyone.
At a high level, the bwauth scripts segment the network into four segments ranked by relay speed, and measure each of these segments. They are 0-10, 10-40[0], 30-60, 60-100 (so the top 10% of relays by speed, and so on.) After 26 hours[1], the first scanner we added is currently at:
Segment 1: Completed, looped a couple times Segment 2: 34.2 of the 10-40 Segment 3: 46.5 of the 30-60 Segment 4: 77.4 out of the 60-100
This isn't very encouraging - it's slow going. Two of things that were just brainstormed in #tor-dev were:
1) Increasing the number of scanners tended to overwhelm the tor instance that supported these scanners, so we want to try doubling the scanners *and* the tors. Hopefully this will let us work our way through the list of relays faster. 2) We want to look at the possibility of relays moving around in the percentiles in each consensus, getting unlucky, and not being measured; potentially fixable by pinning a relay to a percentile, and then when we get all the way through a segment, unpin it, get that segment from the latest consensus, and restart. This may result in a relay being pinned into scanner#2, scanner#1 completes, measures the relay, then scanner#2 measures it... that wouldn't be so bad, double measuring is better than not measuring. When torflow was written over four years ago, there may have been a good reason it didn't work this way, and we need to see if we can re-reason out what that was.
So we are working on identifying the subtler issues and seeing if we can fix them, in addition to just adding more of the same.
-tom [2]
[0] The 10-40 overlap is a bug we just found. [1] Well, more like 50 hours, but the first 24 were lost because of a breaking change in a python point release [2] (Most of this is aagbsn's knowledge, I'm just transcribing it.)
I thank you, Tom.
I lack all of the cited qualifications - now I know. :)
Matt Speak Freely
On Wed, May 20, 2015 at 04:13:57PM -0500, Tom Ritter wrote:
Right now Tor has two new BWAuths spinning up, one of which will hopefully be included in the vote this week or weekend.
And, it looks like Tom's bwauth data is now being used by maatuska. So I expect this will be good news for many, though not all, of the relay operators who have been part of this thread.
You can investigate the individual votes about your relay by looking at the copies of the votes that moria1 exports each hour: http://freehaven.net/~arma/moria1-v3-status-votes (Be aware that while the file is used to be small, it's now up to 28 megabytes. The network sure has been growing.)
--Roger
GIGGITY GIGGITY GOO!!!
It appears as of this morning, 3 of my relays are now reporting something better than 20, in the upper hundreds.
My other one that started working the week before is now chomping along it's merry way.
So I will turn the 3 back on to exit-relays.
Matt Speak Freely
Hey,
thank’s a lot my relay is now also able to support Tor. After over 2 weeks consensus weight is raising! Thank’s to all who helped to fix this bug…
Best regards, Marcus
-- Mein öffentlicher Schlüssel (S/MIME) ist hier verfügbar: https://www.dropbox.com/sh/1c6mccutvv7osq8/AAD66cjh4kmgE5d5xLgQjM_0a?dl=1
Am 25.05.2015 um 15:18 schrieb Speak Freely when2plus2is5@riseup.net:
GIGGITY GIGGITY GOO!!!
It appears as of this morning, 3 of my relays are now reporting something better than 20, in the upper hundreds.
My other one that started working the week before is now chomping along it's merry way.
So I will turn the 3 back on to exit-relays.
Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
indeed, thanks for you promptly interverntion. it feels good to see the mb/s rising! also tor websurfing already responds faster subjectively to me GIGGITY ((:
Am Montag, 25. Mai 2015 16:32 schrieb Marcus marcus.freifunk@gmx.de:
Hey,
thank’s a lot my relay is now also able to support Tor. After over 2 weeks consensus weight is raising! Thank’s to all who helped to fix this bug…
Best regards, Marcus
-- Mein öffentlicher Schlüssel (S/MIME) ist hier verfügbar: https://www.dropbox.com/sh/1c6mccutvv7osq8/AAD66cjh4kmgE5d5xLgQjM_0a?dl=1
Am 25.05.2015 um 15:18 schrieb Speak Freely when2plus2is5@riseup.net:
GIGGITY GIGGITY GOO!!!
It appears as of this morning, 3 of my relays are now reporting something better than 20, in the upper hundreds.
My other one that started working the week before is now chomping along it's merry way.
So I will turn the 3 back on to exit-relays.
Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi operators,
I believe especially the DirAuth/bwauth operators are working on this, but I would like to understand the ongoing issues.
On my relay (8827944C4BDCBDAC9079803F47823403C11A9B7A), I see a decline in consensus weight fraction basically since May 14 (that’s where the first drop in weight fraction occurred) and the same in traffic of course. The connections went down by more than 50% from 1500-1600 to now about 700 (total values). I did notice the strange peak in traffic yesterday, but it’s now back to low levels. My relay is of course working nowhere near capacity (I have set up a RelayBandwithRate of 2MB and a Burst of 5MB) and I hope you understand that's a bit frustrating for me.
At the moment I just see urras missing from the consensus and the measured entries of longclaw are quite a bit off the chart (I’ve subscribed to the consensus-health mailing list) - is that all what you need to have the tor traffic not distributed consistently (at least from my point of view) any more? Most importantly: Can I do something to help?
Cheers, Jannis
Hrm. So this gets into the inner workings of the bwauth system which is... complicated.[0] Honestly, I'm not actually sure how the individual data from the different bwauths is combined into a single value for the consensus.
I'm not sure what the answer is for your problem, but I'm beginning to wonder if the general approach to this problem is "There should be a bwauth debugging mechanism similar to https://consensus-health.torproject.org/ or (the unimplemented) proposal 164." I don't know if said mechanism would be a component of atlas/globe or an entirely separate site, but if the bwauths exported their hourly files, and this hypothetical tool aggregated that data, it may help figure these things out. It might give an answer like "moria spazzed out and undervoted me, let me wait until it scans me again."
-tom
[0] https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R... [1] https://gitweb.torproject.org/torspec.git/tree/proposals/164-reporting-serve...
On 31 May 2015 at 17:31, Jannis Wiese mail@janniswiese.com wrote:
Hi operators,
I believe especially the DirAuth/bwauth operators are working on this, but I would like to understand the ongoing issues.
On my relay (8827944C4BDCBDAC9079803F47823403C11A9B7A), I see a decline in consensus weight fraction basically since May 14 (that's where the first drop in weight fraction occurred) and the same in traffic of course. The connections went down by more than 50% from 1500-1600 to now about 700 (total values). I did notice the strange peak in traffic yesterday, but it's now back to low levels. My relay is of course working nowhere near capacity (I have set up a RelayBandwithRate of 2MB and a Burst of 5MB) and I hope you understand that's a bit frustrating for me.
At the moment I just see urras missing from the consensus and the measured entries of longclaw are quite a bit off the chart (I've subscribed to the consensus-health mailing list) - is that all what you need to have the tor traffic not distributed consistently (at least from my point of view) any more? Most importantly: Can I do something to help?
Cheers, Jannis
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I set my bridge up using the guide at https://www.sky-ip.org/tutorials.html and I have a steady flow of connections definitely visible via tor-arm.
On June 1, 2015 11:02:53 AM CDT, Tom Ritter tom@ritter.vg wrote:
Hrm. So this gets into the inner workings of the bwauth system which is... complicated.[0] Honestly, I'm not actually sure how the individual data from the different bwauths is combined into a single value for the consensus.
I'm not sure what the answer is for your problem, but I'm beginning to wonder if the general approach to this problem is "There should be a bwauth debugging mechanism similar to https://consensus-health.torproject.org/ or (the unimplemented) proposal 164." I don't know if said mechanism would be a component of atlas/globe or an entirely separate site, but if the bwauths exported their hourly files, and this hypothetical tool aggregated that data, it may help figure these things out. It might give an answer like "moria spazzed out and undervoted me, let me wait until it scans me again."
-tom
[0] https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R... [1] https://gitweb.torproject.org/torspec.git/tree/proposals/164-reporting-serve...
On 31 May 2015 at 17:31, Jannis Wiese mail@janniswiese.com wrote:
Hi operators,
I believe especially the DirAuth/bwauth operators are working on
this, but I would like to understand the ongoing issues.
On my relay (8827944C4BDCBDAC9079803F47823403C11A9B7A), I see a
decline in consensus weight fraction basically since May 14 (that's where the first drop in weight fraction occurred) and the same in traffic of course. The connections went down by more than 50% from 1500-1600 to now about 700 (total values). I did notice the strange peak in traffic yesterday, but it's now back to low levels. My relay is of course working nowhere near capacity (I have set up a RelayBandwithRate of 2MB and a Burst of 5MB) and I hope you understand that's a bit frustrating for me.
At the moment I just see urras missing from the consensus and the
measured entries of longclaw are quite a bit off the chart (I've subscribed to the consensus-health mailing list) - is that all what you need to have the tor traffic not distributed consistently (at least from my point of view) any more?
Most importantly: Can I do something to help?
Cheers, Jannis
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi all,
thanks for your answer and the point to the BwAuth specs, Tom - I’ll definitely get into that, or at least try to.
The proposal sounds interesting and I think it would be a good thing to have the whole voting mechanism a bit more transparent. But I guess that might not happen in the near future as it seems to require some bit of work at the back-end…
On 01.06.2015, at 18:56, torelay torelay@ruggedinbox.com wrote:
I set my bridge up using the guide at https://www.sky-ip.org/tutorials.html https://www.sky-ip.org/tutorials.html and I have a steady flow of connections definitely visible via tor-arm.
Thanks for the suggestion! I checked my setup with the tutorial and while I didn’t know that one, I did everything shown there when I was originally setting up the relay. So unfortunately, that doesn’t solve my problem :(
Meanwhile, my consensus weight increased again and I’m now at about the level I was two weeks ago. So maybe Nick was right when he said
Right now, if you want to know the reason why your server was listed a certain way in the Tor directory, the following steps are recommended:
- Wait a while, see if things get better.
:)
Best regards, Jannis
On Mon, Jun 01, 2015 at 11:02:53AM -0500, Tom Ritter wrote:
Hrm. So this gets into the inner workings of the bwauth system which is... complicated.[0] Honestly, I'm not actually sure how the individual data from the different bwauths is combined into a single value for the consensus.
The weight in the consensus is the median of the bwauth votes.
(That design aims to best resist manipulation of the weight by a small number of bwauths.)
I'm not sure what the answer is for your problem, but I'm beginning to wonder if the general approach to this problem is "There should be a bwauth debugging mechanism similar to https://consensus-health.torproject.org/ or (the unimplemented) proposal 164." I don't know if said mechanism would be a component of atlas/globe or an entirely separate site, but if the bwauths exported their hourly files, and this hypothetical tool aggregated that data, it may help figure these things out. It might give an answer like "moria spazzed out and undervoted me, let me wait until it scans me again."
Yes, I'd love to have something like this. Once upon a time I asked for us to export more of the bwauth internals: https://trac.torproject.org/projects/tor/ticket/2532 But at the time Mike decided that the intermediate bwauth numbers weren't useful enough to export.
Another ticket you might enjoy is: https://trac.torproject.org/projects/tor/ticket/2394
I think ultimately we need somebody to simplify the bwauth design as much as possible (but no more), and then we can start assessing its output to see how it compares to reality.
In parallel, or maybe instead, we might want more work on bandwidth estimation algorithms that are resilient to cheating. Right now the bwauth results can be gamed, and making them harder to game is an open research area. Some research groups have been working on it, and I look forward to some of their papers going public. But none of the papers that I've seen so far are perfect designs, so don't set your expectations too high. :)
But all of that said, just visualizing the various Measured= lines by each of the bwauths might be a great start, and this can be done with the published votes as-is: http://freehaven.net/~arma/moria1-v3-status-votes
--Roger
Roger Dingledine arma@mit.edu writes:
On Mon, May 18, 2015 at 09:02:27PM +0000, nusenu wrote:
now even DocTor starts to complain
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005772....
From what I've seen, there were only two messages from DocTor about
this (Mon May 18 08:06:49 UTC 2015[0], and Tue May 19 10:06:13 UTC 2015[1).
Yep. moria1's bwauth seems to be working fine, so I'm the only one paying attention to the thread here, and also the only one who doesn't have anything to fix. :(
I'm seeing this thread as well, but its not apparently obvious that there is something to fix.
nusenu nusenu@openmailbox.org writes:
sounds alarming if people operating tor's key infrastructure do not care (or have not enough time to care)
I don't think that is a fair assessment, I dont think anyone running a bwauth is failing to care, or doesn't prioritize the proper running of the service. There were only two complaints from DocTor, and it has stopped. Two isolated warning usually isn't a cause for alarm, or suggestions that bwauths are not caring.
There are two possibilities here:
1. there is an actual problem on the existing bwauths that needs to be fixed: if that is true, it is not obvious what the problem is. The bandwidth for longclaw seems to be working fine, and the bwauths processes seem to be doing what they are supposed to be doing. If there is something amiss with longclaw's bwauths that I am missing, and can do something to resolve, please let me know.
2. none of the bwauths are down, but rather we've reached a tipping point and the bwauths cannot measure the entire network in the measurement run. if this is true, then this is indicative of a trend and we will be seeing more of these alerts from DocTor in the future, and this is not a case of bwauths failing to care for critical tor infrastructure
micah
0. https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005772.... 1. https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/005773....
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi nusenu,
Again thanks for keeping an eye on things. At least partially, the balance of measured/unmeasured should be fixed in the new few days.
To relay operators, in the mean time, please bare for few more days and sorry for the inconvenience, I can imagine what it feels like to have a high capacity unmeasured relay. Developers already know that a new bandwidth measurement system is badly needed, just need to find workarounds for the short term until such a system will be ready.
On 5/19/2015 12:02 AM, nusenu wrote:
now even DocTor starts to complain
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/0
05772.html
_______________________________________________
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Tue, May 19, 2015 at 12:27:12AM +0300, s7r wrote:
Hi nusenu,
Again thanks for keeping an eye on things. At least partially, the balance of measured/unmeasured should be fixed in the new few days.
To relay operators, in the mean time, please bare for few more days and sorry for the inconvenience, I can imagine what it feels like to have a high capacity unmeasured relay. Developers already know that a new bandwidth measurement system is badly needed, just need to find workarounds for the short term until such a system will be ready.
I've even been pondering crazy hacks at this point like exporting moria1's bwauth file to a second dir auth, so it gets to vote twice and we're more likely to have the required minimum of three opinions for each relay.
(This is a terrible idea, but we're in the realm of terrible ideas at this point.)
Another terrible idea, while we're at it, is to disable the active measurements and go back to self-advertised weights by the relays.
--Roger
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Is splitting the bwauth data into a seperate consensus of some description a possibility and have a few people run them without needing to be a dirauth? That would mean lesser responsibility is placed on bwauth ops and perhaps the load or task could be spread to people known by Tor Project but not in the current dirauth pool.
T
On 18/05/2015 22:45, Roger Dingledine wrote:
On Tue, May 19, 2015 at 12:27:12AM +0300, s7r wrote:
Hi nusenu,
Again thanks for keeping an eye on things. At least partially, the balance of measured/unmeasured should be fixed in the new few days.
To relay operators, in the mean time, please bare for few more days and sorry for the inconvenience, I can imagine what it feels like to have a high capacity unmeasured relay. Developers already know that a new bandwidth measurement system is badly needed, just need to find workarounds for the short term until such a system will be ready.
I've even been pondering crazy hacks at this point like exporting moria1's bwauth file to a second dir auth, so it gets to vote twice and we're more likely to have the required minimum of three opinions for each relay.
(This is a terrible idea, but we're in the realm of terrible ideas at this point.)
Another terrible idea, while we're at it, is to disable the active measurements and go back to self-advertised weights by the relays.
--Roger
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
- -- Activist, anarchist and a bit of a dreamer. Keybase: https://keybase.io/thomaswhite
PGP Keys: https://www.thecthulhu.com/pgp-keys/ Current Fingerprint: BA81 407C BD61 CD15 E5D9 ADA9 5FA2 426F F34E 0FD4 Master Fingerprint: DDEF AB9B 1962 5D09 4264 2558 1F23 39B7 EF10 09F0
Twitter: @CthulhuSec XMPP: thecthulhu at jabber.ccc.de XMPP-OTR: 77E6C8C6 95FDE863 1172A1E1 8C114C01 691398AC
On Mon, 18 May 2015, Roger Dingledine wrote:
I've even been pondering crazy hacks at this point like exporting moria1's bwauth file to a second dir auth, so it gets to vote twice and we're more likely to have the required minimum of three opinions for each relay.
Are the existing bwauths down, or are they just falling behind? I think I read something last year suggesting that they were taking more than 18 hours to get through a measurement run, and the data was only allowed to be 24 hours old.
If they are just falling behind, could the consensus be based on the 3 most recent measurements instead, however old those are?
-- Aaron
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
At least partially, the balance of measured/unmeasured should be fixed in the new few days.
Thanks for the pre-info, appreciated.
After changing my exit node to a regular node it jumped from 20 to 7000 rating. I changed it back to exit and it jumped to 9000 and stayed on that value. This was a few days ago.
Right now I've seen an increase to 17000. I assume this is due to the new authorities? I'm glad that my node is being used again.
On 18.05.2015 11:27 PM, s7r wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi nusenu,
Again thanks for keeping an eye on things. At least partially, the balance of measured/unmeasured should be fixed in the new few days.
To relay operators, in the mean time, please bare for few more days and sorry for the inconvenience, I can imagine what it feels like to have a high capacity unmeasured relay. Developers already know that a new bandwidth measurement system is badly needed, just need to find workarounds for the short term until such a system will be ready.
On 5/19/2015 12:02 AM, nusenu wrote:
now even DocTor starts to complain
https://lists.torproject.org/pipermail/tor-consensus-health/2015-May/0
05772.html
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBCAAGBQJVWlkwAAoJEIN/pSyBJlsRPUEH/3x9638bl0mSoq2oEpC3MazV GgCEh/MG+6dWDKdASG0qdrpdFPt4VrihuwhIOB5l5G9x6RUhf1LEZvGjdqYiyz0Y JRB1m4Mek8x0iHKeVQeY8VnWX3WQBEpgXc4EVUHgkTGXZByLSAueyYnviD0hGeiy ftfaBhN6SE5nzxAVBmzHBiC5rmFl5cam3o9YxguGOkugaWeLHgHDECbQ+yjacy1N 6VjudnuuFeAu2myfo6g2W7tCHswVaDwqyUhmhuab9OUguImv0HdHoGgeJM/tn3TZ EQuRSKYV4jhfCkzpcLvzcsueJKuLfchGyJ8JxiD/vbLuXe9swLJsUcwORg4BMQQ= =w3+J -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hey NOC,
Nah I had the same experience with *1* relay. Following the same pattern, none of the other relays have left 20. I'm now at 16200 and averaging ~30mb/s.
4 still in limbo. What's also interesting is I setup a non-exit and it also will not get past 20.
What does arm say your measured speed is? Mine was 45Kb/s for the longest time, but very recently went to 126.5Kb/s, but that's silly.
Matt Speak Freely
Matt,
135 Kb/s measured, I'm currently pushing data at 40Mbit in/out (limit is 80 Mbit with burst to 120 Mbit).
On 21.05.2015 11:14 PM, Speak Freely wrote:
Hey NOC,
Nah I had the same experience with *1* relay. Following the same pattern, none of the other relays have left 20. I'm now at 16200 and averaging ~30mb/s.
4 still in limbo. What's also interesting is I setup a non-exit and it also will not get past 20.
What does arm say your measured speed is? Mine was 45Kb/s for the longest time, but very recently went to 126.5Kb/s, but that's silly.
Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hey,
I still have the problem, that my relay (non-exit) is "not measured". I tried to set up a new ID but this doesn’t work. The relay ist still not measured since one week. (12FD624EE73CEF37137C90D38B2406A66F68FAA2) I don’t know much about the DA, but I checked that only two of nine have measured my relay. Only gabelmoo and moria1 did measure the relay. An other smaller relay I own (FFB78E1F3E29091212C23A24A9779072800E1D96) is listed as „measured“ in the consensus with only three DA which measured the bandwidth. (gabelmoo, moria1 and long claw)
Based on this information I have some questions: 1) Is three DA which measured the bandwidth the „necessary“ number to be not „unmeasured“? 2) Is it only by chance that the DA are almost the same, which measured my both relays?! 3) Is there any idea, how I can maybe force a DA to measure the relay?!
I am a bit frustrated that I rented a Server to support TOR and now the relay is useless… :(
Thanks and Best regards, Marcus
Am 22.05.2015 um 01:24 schrieb Network Operations Center noc@schokomil.ch:
Matt,
135 Kb/s measured, I'm currently pushing data at 40Mbit in/out (limit is 80 Mbit with burst to 120 Mbit).
On 21.05.2015 11:14 PM, Speak Freely wrote:
Hey NOC, Nah I had the same experience with *1* relay. Following the same pattern, none of the other relays have left 20. I'm now at 16200 and averaging ~30mb/s. 4 still in limbo. What's also interesting is I setup a non-exit and it also will not get past 20. What does arm say your measured speed is? Mine was 45Kb/s for the longest time, but very recently went to 126.5Kb/s, but that's silly. Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I have a non-exit relay that has been up and running with 100% uptime on a 15mbps circuit for going on 20 days that is still unmeasured.
I also have an exit node on a 100mbps un-metered link that I brought online 7 days ago and purchased specifically to donate to Tor that is still unmeasured by the bwauths.
Perhaps it's time to go back to self-advertised bandwidth since this system is clearly broken?
Furthermore, I understand the necessity of having the bwauths run by trusted/known members of the development community, but what is the criteria for this trust? Does running a broken directory server for going on three weeks without taking the steps to fix it still fall within this "trust" definition? ---
Julian Plamann
julian (at) amity.be GPG: 0x96881D83
On 2015-05-22 10:32 am, Marcus wrote:
Hey,
I still have the problem, that my relay (non-exit) is "not measured". I tried to set up a new ID but this doesn't work. The relay ist still not measured since one week. (12FD624EE73CEF37137C90D38B2406A66F68FAA2) I don't know much about the DA, but I checked that only two of nine have measured my relay. Only gabelmoo and moria1 did measure the relay. An other smaller relay I own (FFB78E1F3E29091212C23A24A9779072800E1D96) is listed as „measured" in the consensus with only three DA which measured the bandwidth. (gabelmoo, moria1 and long claw)
Based on this information I have some questions:
- Is three DA which measured the bandwidth the „necessary" number to be not „unmeasured"?
- Is it only by chance that the DA are almost the same, which measured my both relays?!
- Is there any idea, how I can maybe force a DA to measure the relay?!
I am a bit frustrated that I rented a Server to support TOR and now the relay is useless... :(
Thanks and Best regards, Marcus
Am 22.05.2015 um 01:24 schrieb Network Operations Center noc@schokomil.ch:
Matt,
135 Kb/s measured, I'm currently pushing data at 40Mbit in/out (limit is 80 Mbit with burst to 120 Mbit).
On 21.05.2015 11:14 PM, Speak Freely wrote: Hey NOC, Nah I had the same experience with *1* relay. Following the same pattern, none of the other relays have left 20. I'm now at 16200 and averaging ~30mb/s. 4 still in limbo. What's also interesting is I setup a non-exit and it also will not get past 20. What does arm say your measured speed is? Mine was 45Kb/s for the longest time, but very recently went to 126.5Kb/s, but that's silly. Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays [1] _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays [1]
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays [1]
Links: ------ [1] https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hello Julian
Thanks for your contribution to Tor! Please keep them running, as much as you can. We already struggle to (at least partially) solve the issue for the short term, by launching more bandwidth authorities. On the long term, some research groups are working on a better bandwidth measurement system. We are all anxiously waiting for it, but unfortunately there is no ETA for it at this moment.
To answer your question, it's not exactly true that _current system is clearly broken_, or totally broken. It works, but does not work 100% (e.g. we are missing some bandwidth). It's better than going back to self advertising bandwidth. By default (if not explicitly set a MaxAdvertisedBandwidth in torrc), Tor will self advertise the port speed which it sees on the server where running, which is very very often (I could say almost) of a higher value than that server can really handle. For example, take a VPS which has a network port of 1 gbps seen by the operating system, but it's running on a cluster with 100 another virtual machines and each is individually capped at hypervisor lever to 10mbit. Going back to self advertised bandwidth would assign unfair weights to some relays and cause Tor clients to choose paths in a buggy way, we could overload the less capable relays and cause circuit failure for clients, decreasing the Tor performance / user experience overall.
The bandwidth authorities are not so sensitive as directory authorities, but a bandwidth authority is useless unless it talks to a directory authority which is allowed to vote in the consensus. That is why bandwidth authorities are run by the same trusted people who also run directory authorities.
The analogy you are describing does not reflect the reality in this situation. For the issue we are facing, no directory authority operator has a solution, nor anyone. It's not like someone can do something about it, but does not do it. Still, everyone is making efforts to fix it, in the measure in which it is technically possible. The system where we have a known number of directory authorities, hardcoded and run by trusted people is very well researched and studied, and it eliminates a lot of real and practicable attacks. I am all for decentralized and distributed systems, but at this moment I really think we are doing the best we can. Everyone would prefer Tor to be decentralized and trustless, but this is not that simple and cannot be done immediately. A lot of research is needed.
P.S. having the semi-centralized system we currently use (with few Directory authorities run by trusted people) does not mean you are 100% depending on the directory authorities or that they can compromise your anonymity. You are trusting them for consensus data (info about all the relays in the Tor network) which your client uses to build circuits. If something happens to the majority (50% + 1) of the directory authorities, this would prevent the Tor network to function at all, for anyone, but the previous activity within the Tor network of an user won't be compromised. This is a fair trust limit I could say, when you take in consideration the fact that total decentralization will open the door to many other attacks, far worse than this.
(fortunately) there is no way to 'force' a bandwidth authority or directory authority to do anything.
Please bare little more and keep the relays running if you can. After all, we are a community comprised in volunteers involved in research and experiments, sometimes things like this happen. Just keep calm, run relays and use Tor. It'll be fixed.
On 5/23/2015 8:58 AM, Julian Plamann wrote:
I have a non-exit relay that has been up and running with 100% uptime on a 15mbps circuit for going on 20 days that is still unmeasured.
I also have an exit node on a 100mbps un-metered link that I brought online 7 days ago and purchased specifically to donate to Tor that is still unmeasured by the bwauths.
Perhaps it's time to go back to self-advertised bandwidth since this system is clearly broken?
Furthermore, I understand the necessity of having the bwauths run by trusted/known members of the development community, but what is the criteria for this trust? Does running a broken directory server for going on three weeks without taking the steps to fix it still fall within this "trust" definition?
--- Julian Plamann
julian (at) amity.be GPG: 0x96881D83
On 2015-05-22 10:32 am, Marcus wrote:
Hey,
I still have the problem, that my relay (non-exit) is "not measured". I tried to set up a new ID but this doesn't work. The relay ist still not measured since one week. (12FD624EE73CEF37137C90D38B2406A66F68FAA2) I don't know much about the DA, but I checked that only two of nine have measured my relay. Only gabelmoo and moria1 did measure the relay. An other smaller relay I own (FFB78E1F3E29091212C23A24A9779072800E1D96) is listed as „measured" in the consensus with only three DA which measured the bandwidth. (gabelmoo, moria1 and long claw)
Based on this information I have some questions: 1) Is three DA which measured the bandwidth the „necessary" number to be not „unmeasured"? 2) Is it only by chance that the DA are almost the same, which measured my both relays?! 3) Is there any idea, how I can maybe force a DA to measure the relay?!
I am a bit frustrated that I rented a Server to support TOR and now the relay is useless... :(
Thanks and Best regards, Marcus
Am 22.05.2015 um 01:24 schrieb Network Operations Center <noc@schokomil.ch mailto:noc@schokomil.ch>:
Matt,
135 Kb/s measured, I'm currently pushing data at 40Mbit in/out (limit is 80 Mbit with burst to 120 Mbit).
On 21.05.2015 11:14 PM, Speak Freely wrote:
Hey NOC, Nah I had the same experience with *1* relay. Following the same pattern, none of the other relays have left 20. I'm now at 16200 and averaging ~30mb/s. 4 still in limbo. What's also interesting is I setup a non-exit and it also will not get past 20. What does arm say your measured speed is? Mine was 45Kb/s for the longest time, but very recently went to 126.5Kb/s, but that's silly. Matt Speak Freely _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org mailto:tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________
tor-relays mailing list tor-relays@lists.torproject.org mailto:tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________
tor-relays mailing list tor-relays@lists.torproject.org mailto:tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Thanks for the eloquent response. Many good points made and apologies if my comments came across as being critical of the Tor development community. I have no plans to take down the relays I'm running any time soon -- bwauth hiccups or not. Regardless of the current issues, I have to say the network at large feels significantly faster and more functional than it did a few years ago.
- --- Julian Plamann
julian (at) amity.be GPG: 0x96881D83
On 2015-05-23 10:56 am, s7r wrote:
- -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hello Julian
Thanks for your contribution to Tor! Please keep them running, as much as you can. We already struggle to (at least partially) solve the issue for the short term, by launching more bandwidth authorities. On the long term, some research groups are working on a better bandwidth measurement system. We are all anxiously waiting for it, but unfortunately there is no ETA for it at this moment.
To answer your question, it's not exactly true that _current system is clearly broken_, or totally broken. It works, but does not work 100% (e.g. we are missing some bandwidth). It's better than going back to self advertising bandwidth. By default (if not explicitly set a MaxAdvertisedBandwidth in torrc), Tor will self advertise the port speed which it sees on the server where running, which is very very often (I could say almost) of a higher value than that server can really handle. For example, take a VPS which has a network port of 1 gbps seen by the operating system, but it's running on a cluster with 100 another virtual machines and each is individually capped at hypervisor lever to 10mbit. Going back to self advertised bandwidth would assign unfair weights to some relays and cause Tor clients to choose paths in a buggy way, we could overload the less capable relays and cause circuit failure for clients, decreasing the Tor performance / user experience overall.
The bandwidth authorities are not so sensitive as directory authorities, but a bandwidth authority is useless unless it talks to a directory authority which is allowed to vote in the consensus. That is why bandwidth authorities are run by the same trusted people who also run directory authorities.
The analogy you are describing does not reflect the reality in this situation. For the issue we are facing, no directory authority operator has a solution, nor anyone. It's not like someone can do something about it, but does not do it. Still, everyone is making efforts to fix it, in the measure in which it is technically possible. The system where we have a known number of directory authorities, hardcoded and run by trusted people is very well researched and studied, and it eliminates a lot of real and practicable attacks. I am all for decentralized and distributed systems, but at this moment I really think we are doing the best we can. Everyone would prefer Tor to be decentralized and trustless, but this is not that simple and cannot be done immediately. A lot of research is needed.
P.S. having the semi-centralized system we currently use (with few Directory authorities run by trusted people) does not mean you are 100% depending on the directory authorities or that they can compromise your anonymity. You are trusting them for consensus data (info about all the relays in the Tor network) which your client uses to build circuits. If something happens to the majority (50% + 1) of the directory authorities, this would prevent the Tor network to function at all, for anyone, but the previous activity within the Tor network of an user won't be compromised. This is a fair trust limit I could say, when you take in consideration the fact that total decentralization will open the door to many other attacks, far worse than this.
(fortunately) there is no way to 'force' a bandwidth authority or directory authority to do anything.
Please bare little more and keep the relays running if you can. After all, we are a community comprised in volunteers involved in research and experiments, sometimes things like this happen. Just keep calm, run relays and use Tor. It'll be fixed.
On 5/23/2015 8:58 AM, Julian Plamann wrote: I have a non-exit relay that has been up and running with 100% uptime on a 15mbps circuit for going on 20 days that is still unmeasured.
I also have an exit node on a 100mbps un-metered link that I brought online 7 days ago and purchased specifically to donate to Tor that is still unmeasured by the bwauths.
Perhaps it's time to go back to self-advertised bandwidth since this system is clearly broken?
Furthermore, I understand the necessity of having the bwauths run by trusted/known members of the development community, but what is the criteria for this trust? Does running a broken directory server for going on three weeks without taking the steps to fix it still fall within this "trust" definition?
- --- Julian Plamann
julian (at) amity.be GPG: 0x96881D83
On 2015-05-22 10:32 am, Marcus wrote:
Hey,
I still have the problem, that my relay (non-exit) is "not measured". I tried to set up a new ID but this doesn't work. The relay ist still not measured since one week. (12FD624EE73CEF37137C90D38B2406A66F68FAA2) I don't know much about the DA, but I checked that only two of nine have measured my relay. Only gabelmoo and moria1 did measure the relay. An other smaller relay I own (FFB78E1F3E29091212C23A24A9779072800E1D96) is listed as „measured" in the consensus with only three DA which measured the bandwidth. (gabelmoo, moria1 and long claw)
Based on this information I have some questions: 1) Is three DA which measured the bandwidth the „necessary" number to be not „unmeasured"? 2) Is it only by chance that the DA are almost the same, which measured my both relays?! 3) Is there any idea, how I can maybe force a DA to measure the relay?!
I am a bit frustrated that I rented a Server to support TOR and now the relay is useless... :(
Thanks and Best regards, Marcus
Am 22.05.2015 um 01:24 schrieb Network Operations Center
tor-relays@lists.torproject.org
-
Aaron Hopkins -
Jannis Wiese -
Julian Plamann -
Marcus -
micah anderson -
Network Operations Center -
nusenu -
Roger Dingledine -
s7r -
Speak Freely -
Thomas White -
Tom Ritter -
tor-server-creator＠use.startmail.com -
torelay