I made some graphs that show the count and total bandwidth of all bridges, broken down by transport. https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwid... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.p... The top part of the graph is non-default bridges, and the bottom is default bridges (the ones that ship with Tor Browser). Note the varying vertical scales. Source code and data are here: https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11.tar.gz https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwid...
I have a question about getting bandwidth numbers. What I want is something like available bandwidth capacity; i.e., how much headroom transports have given their level of use. I think that the bandwidth numbers I'm using are partially conflated with usage. Notice how parts of pt-count.png are flat while pt-bandwidth.png has a weekly pattern.
The bandwidth number I'm using is the "w Bandwidth=" line from a bridge-network-status document. It looks like this: r starman qgM+62FgGytzEtibYqqiPcPtijQ mdOOBxVOTpw8loBezhSDZxLIcXs 2015-07-03 21:39:31 10.174.163.60 9002 0 s Fast Guard Running Stable Valid w Bandwidth=2646 p reject 1-65535 dir-spec.txt says: An estimate of the bandwidth of this relay, in an arbitrary unit (currently kilobytes per second). Used to weight router selection. See section 3.4.2 for details on how the value of Bandwidth is determined in a consensus." Section 3.4.2 says: When we speak of a router's bandwidth in this section, we mean either its measured bandwidth, or its advertised bandwidth.... The bandwidth in a "w" line should be taken as the best estimate of the router's actual capacity that the authority has. For now, this should be the lesser of the observed bandwidth and bandwidth rate limit from the server descriptor. It is given in kilobytes per second, and capped at some arbitrary value (currently 10 MB/s). I don't know where observed bandwidth comes from. Is there some kind of external test that measures it, or does it come from measuring user traffic?
Another option for getting bandwidth is the "bandwidth" line in a bridge-server-descriptor document. It looks like this: bandwidth 1073741824 1073741824 2646895 The three fields are bandwidth-avg bandwidth-burst bandwidth-observed. dir-spec.txt says: Estimated bandwidth for this router, in bytes per second. The "average" bandwidth is the volume per second that the OR is willing to sustain over long periods; the "burst" bandwidth is the volume that the OR is willing to sustain in very short intervals. The "observed" value is an estimate of the capacity this relay can handle. The relay remembers the max bandwidth sustained output over any ten second period in the past day, and another sustained input. The "observed" value is the lesser of these two numbers. So it seems that bandwidth-observed is not what we want, because it's derived from current usage. bandwidth-avg and bandwidth-burst aren't useful because they are operator-controlled and often left at the default of 1 GB/s.
Yet another option is the speed of serving consensuses, the "dirreq-v3-tunneled-dl" line in a bridge-extra-info document: dirreq-v3-tunneled-dl complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627 You can take the "md=" field, for example, to get median B/s for serving consensuses.
I don't know where observed bandwidth comes from. Is there some kind of external test that measures it, or does it come from measuring user traffic?
From bandwidth auths running torflow, no?
On Sat, Jul 11, 2015 at 12:18:07PM -0700, David Fifield wrote:
I made some graphs that show the count and total bandwidth of all bridges, broken down by transport. https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwid... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.p... The top part of the graph is non-default bridges, and the bottom is default bridges (the ones that ship with Tor Browser).
Neat! Though alas you've done the thing that people do where they graph twelve things at once, and all of them are basically variations on the same color. So without some insight into the underlying data, I fear these graphs are of limited use. (One fix is the one Karsten ended up with on the metrics page, where you can choose which transports to graph, providing an easy way to learn which line is which.)
I have a question about getting bandwidth numbers. What I want is something like available bandwidth capacity; i.e., how much headroom transports have given their level of use.
Yeah, that isn't something that we explicitly provide. Nothing measures it.
I think that the bandwidth numbers I'm using are partially conflated with usage.
Correct.
The bandwidth number I'm using is the "w Bandwidth=" line from a bridge-network-status document. It looks like this: w Bandwidth=2646
[snip]
Another option for getting bandwidth is the "bandwidth" line in a bridge-server-descriptor document. It looks like this: bandwidth 1073741824 1073741824 2646895
The consensus weight for bridges is the min of these three bandwidth values. Tonga does no bandwidth testing of its own currently.
So it seems that bandwidth-observed is not what we want, because it's derived from current usage.
Well, ok. But it might be the best you've got. :)
Yet another option is the speed of serving consensuses, the "dirreq-v3-tunneled-dl" line in a bridge-extra-info document: dirreq-v3-tunneled-dl complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627 You can take the "md=" field, for example, to get median B/s for serving consensuses.
Yes, this is an interesting option. It would be useful if all of the users are distributed equally (with respect to user speed) over the bridges. We originally put that feature in to measure *client* speeds, so it's up for grabs whether it would work to turn it around and get something useful about bridge speed.
--Roger
On Sat, Jul 11, 2015 at 12:18:07PM -0700, David Fifield wrote:
I made some graphs that show the count and total bandwidth of all bridges, broken down by transport. https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwid... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.p... The top part of the graph is non-default bridges, and the bottom is default bridges (the ones that ship with Tor Browser). Note the varying vertical scales. Source code and data are here: https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11.tar.gz https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwid...
I have a question about getting bandwidth numbers. What I want is something like available bandwidth capacity; i.e., how much headroom transports have given their level of use. I think that the bandwidth numbers I'm using are partially conflated with usage. Notice how parts of pt-count.png are flat while pt-bandwidth.png has a weekly pattern.
Yet another option is the speed of serving consensuses, the "dirreq-v3-tunneled-dl" line in a bridge-extra-info document: dirreq-v3-tunneled-dl complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627 You can take the "md=" field, for example, to get median B/s for serving consensuses.
I tried extracting extracting the rate of serving consensuses as a proxy for bridge bandwidth. The "bandwidth" one is the one you saw before, summing observed bandwidth of bridges. "dl.md" is the median download rate of consensuses, and "dl.max" is the maximum.
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-count.p... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-bandwid... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-dl.md.p... https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-dl.max....
Source code: https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20.tar.gz
These plots omit all bridges that did not have a "dirreq-v3-tunneled-dl" line.
The new graphs have a different shape than the observed-bandwidth one, at least. They still seem somewhat tied to usage. It looks like there is a weekly cycle in the dl.max one--it could be simply that with more users connecting during the week, the bridge has a higher chance of seeing a fast one that pushes up the maximum.
-
Arlo Breault -
David Fifield -
Roger Dingledine