-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 05/05/2015 16:19, balooni@espiv.net wrote:
Hi Clodo,
Thank you for your interest in OONI.
Clodo wrote:
I'm the creator of the no-profit service http://www.neumon.org . It's a project similar to OONI, but focused only on DNS and HTTP.
Do you have the code published somewhere?
We release the source of the probe here: https://github.com/AirVPN/neumon-probe Written in C#/Mono. I run it from RaspBian on Raspberry PI. But it's not a great piece of software. Simply it fetch from our backend the list of domains to try to resolve/fetch, do it, and resend the results. All detection are server-side based.
The backend it's written in php, sources never released. Contain basically a lot of mysql queries to detect stuffs and generate report.
NeuMon browse collected DNS servers, and check if can be queried (open and recursive). This because most of DNS ISP are recursive only from it's customers subnet. We maintain a huge list of domains to check (mix of known blocked website, top alexa, etc). Every DNS it's queried for each domains, we collect results, compare against a known good value and discover custom injection (generally that point to blocking page, i published some example here: http://tinyurl.com/pl8znb4 ).
So, i have:
- a huge list of DNS servers, with country geolocation.
- lists of domains blocked, country-based. Not exaustive.
- i know many IP address that are destination of DNS redirection,
typically IP of servers that show html blocking pages. And DNS servers of ISP that redirect to these addresses.
It was quite difficult to find out and interpret the results from [1] could you maybe provide some pointers?
Mainly because i don't publish all results, generally aggregated stats.
I have lists of domains blocked, but isn't available on neumon.org for reason explained below in this mail.
http://www.neumon.org/blacklist.html These IP are destination IP of DNS injection. We manage it manually. They hosts the blocking pages. They have virtual-hosts in webserver for domains, so you maybe cannot view the blocking page by viewing the IP directly. The lists contain also private services (like OpenDNS), not related to censorship. The lists may contain blocking page of private services (like adult-filter services), not related to ISP censorship. The recent tweet of Mikko Hypponen: https://twitter.com/mikko/status/595681341334773760 are screenshots of websites of the above IP list.
http://www.neumon.org/?view=dns_list&country=it This is an example country list of DNS servers open and recursive we detect in Italy. Note that maybe include a customer of an ISP that have it's own DNS server.
Generally, i have a lot of data, catched automatically, that require manual works to obtain nice and clean report, i'm in stall on this kind of works.
How did you find out about these domains and why do you think that they contain CP?
I don't know. Some of them (with domain name with keyword like teen/sex/...) seem like typical porn website, a collection of video and screenshot. Of course i can't know if are real CP. I'm italian, and i know very well the italian situation: here ISP block CP, gambling, proxy, file-sharing, file-hosting, webcam, pharma, escort, drugs, steroid, etc. Sometime, if a CP it's hosted on a public image-hosting, the entire file-hosting services are blocked. ImageShack was DNS blocked for years in Italy for a single CP image. I obtain with my system lists of blocked domain, all of category listed above together, but actually i don't want to publish it (see https://youtu.be/RkmcupFx3FQ?t=1m13s ) because i can't detect CP versus other categories.
Anyway, in my system sometime i have the information of what is classified CP. For example, major ISP in Switzerland redirect CP domain to a server hosted by stopp-kinderpornografie.ch .
In any case it would be very interesting to see these results or of the ones that can be made public.
We also build a probe software, to allow other activists connected to the ISP directly to launch it and detect censorship not based on DNS.
It will be very interesting to instruct the probe software submit results to an ooni backend [2]. In any case the probe software can maybe even written as an ooniprobe test [3].
I understand you already have some DNS tests on ooniprobe. I will study them. But actually i don't understand what are the lists of domain tested by OONI, how you detect spoof, and where/if you results are published.
My mysql data it's around 25 gb. I think maybe better (for maintenance and independency) not to create OONI tests linked to neumon.org project. I think maybe better if i create some webservices in neumon.org to expose my data, where OONI backend can fetch interesting data for your research. For example, i can provide a list of DNS servers we detect (open to query and with recursion enabled). Or i can provide a list of "open/recursive DNS Server IP -> query domain "xxx" -> the result "ip address" it's probably a blocking page.
But nobody want to run a software that fetch also child pornography domains, so nobody want to run our probe.
I don't think that is all about CP only. Right now there are so many blacklists and censored websites worldwide and as far as I know people are interested in finding out of what resources are being blocked. Many of these started blocking gambling related websites and later added a bunch of other websites hence opening the door for censorship and blocking of other websites at will [4], [5].
A particular example: i know a very important ISP that redirect blocked domain to a fixed IP. Interesting and unbelivable, they specify the reverse-lookup info on that IP. So, a reverse lookup on that IP show thousand of domains, updated frequently. I fetch periodically this list to populate my domains tests list.
But, that list contain mixed CP, proxy, and in general all category of blocked domains. Detect what are the domains that may attract mainstream interest for censorship reason, require filter that list by skipping CP, gambling etc, and it's the kind of work that i don't know how to manage with an automatic system.
I hope you can understand my poor english.
Ciao Fabrizio - Clodo