On Mar 17, 2016, at 3:45 AM, Linus Nordberg linus@torproject.org wrote:
Hi,
I'm trying to figure out why a list from [TorBulkExitList.py] is so much larger than what is seen in [exit-addresses].
Let's certainly not rule out the possibility of a bug.
Point in case: Earlier today the list from TorBulkExitList.py contained 58% more addresses than exit-addresses:
--8<---------------cut here---------------start------------->8--- $ curl-tor -q 'https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=8.8.8.8' | egrep -v ^# | wc -l 1554 $ curl-tor -q https://check.torproject.org/exit-addresses | egrep ^ExitNode | wc -l 985
Shouldn't this grep `ExitAddress`? (which only adds 3 right now)
--8<---------------cut here---------------end--------------->8---
Counting relays with the Exit flag in a consensus of roughly the same time gives 912:
--8<---------------cut here---------------start------------->8--- $ curl-tor -q http://171.25.193.9:443/tor/status-vote/current/consensus | egrep '^s.*Exit' | wc -l 912 --8<---------------cut here---------------end--------------->8---
If I read the code [check.py][exitips.py] correctly, we include the following IP addresses in what is served from [TorBulkExitList.py]:
- all routers in the current consensus
- with exit addresses from TorDNSEL file(s) added
- for which stem's exit_policy.is_exiting_allowed() returns true
It's more like,
- all routers in the past 16 consensuses - for which stem's exit_policy.is_exiting_allowed() returns true
but it also includes all ips it has ever known for a relay, https://gitweb.torproject.org/check.git/commit/?id=026e43b08656d78398b15742d... which seems a little suspect.
I haven't read any TorDNSEL code and don't know exactly what above mentioned TorDNSEL file(s) are but I think they include results from active testing of what address exit relays really use for exiting.
Yup, that's it.
[exit-addresses] seems to be from TorDNSEL too but holding more of a snapshot while the files mentioned above supposedly cover some period of time.
Yes.
Before I dig further, perhaps someone here already know why the numbers differ so? I wouldn't expect the churn of addresses used for exiting to be high enough to explain the difference.
If the numbers don't add up, can you file a bug?