Mike Perry:
grarpamp:
The questions were of a general "intro to netflow" nature, thus the links, they and other resource describe all the data fields, formation of records, timeouts, aggregation, IPFIX extensibility, etc. Others and I on these lists know what "360 gigs" of netflow looks like.
Well, right, then. Let's get to the meat of it.
*What* specific info are you looking for beyond that?
I am looking to understand what "360 gigs" aka "(3.2 billion records)" of netflow over 3 months looks like, and also if we can expect this to be standard practice, somewhat outside the norm, or indicative of someone who has specifically tuned their netflow config to attack Tor (should the opportunity arise).
Assuming the boingboing comment is accurate, and it's just one exit IP, then we're probably looking at two exits worth of data (either UtahStateExit0+UtahStateExit1, or UtahStateExit2+UtahStateExit3).
Each of these exit pairs appears to have averaged a little over 10Mbit/sec sustained over the most recent 3 month period according to https://globe.torproject.org. The exits are running some version of the Reduced Exit Policy, so there should be no bittorrent traffic. Likely mostly web traffic by connection count, and probably even byte count.
In three months, there are 7,776,000 seconds. So we're looking at 441 records per second in this dataset.
For 10Mbit/sec worth of sustained web traffic, that sounds about connection-level resolution to me. Do you agree?
(Yay! Thinking once and posting two posts at once to three different lists. I'm like some kind of Internet champion! ;)
I think I needed to do one more division. This is roughly one record per 3KB of traffic (which I think you alluded to earlier). Rather high if we expect this to be web traffic, even if there was only 1 web request per connection.
So then, what is the most likely configuration that would generate this many records? Is it indeed likely to be some BOFH scenario, or might there be some common (if half-insane) policy that ends up producing this many records?
Here's Globe for UtahStatExit2 and 3 for easy access: https://globe.torproject.org/#/relay/B4E641BC42DDB6FD2526CFF80504AB5221B0EB8... https://globe.torproject.org/#/relay/7E4E1CC167300932F05AC70ECD2B9A298732C6E...
The bandwidth histories have no current data, but you can click on the 3 month tab to get the numbers I used.