Hello Oonitarians,
During yesterdays we had a very interesting conversation about the ethics of measurements, informed consent and methodologies for achieving it. These are very important discussions to have and we agreed that we should continue it in this thread.
For those that were not on IRC at that time I will explain briefly what it was that sparked the debate. If you are interested in reading the full transcript of the meeting private message me and I will send it to you.
A problem that we have with OONI and I think is common to most network measurements projects is that of acquiring reliable vantage points in non western countries. By reliable I mean vantage points where the tool in question is not just run once and forgotten, but is periodically run, say once every day. One way of acquiring vantage points is to rent VPS' and setting up the tool on such VPS'. The problem with this approach, though, is that what you are measuring is not the network that a real user in that country would be using.
To overcome this issue I have come up with a scheme where by I get in contact with people from countries that interest us and give them some money to buy a raspberry pi and setup ooniprobe on it. As an incentive to keep the probe running and gathering data with a daily resolution I then pay them a small monthly fee to cover bandwidth and power costs. It turns out that not only is this cheaper than renting a VPS in that country, but it also gives us more accurate results, since the measurements are done from the users DSL home connection.
The problem with this approach is that we need to make it absolutely clear that there is some risk involved in running the software and the amount of risk varies greatly from country to country. So far I have limited this to a very small set of people (3 in total, 2 paid and 1 not paid) that I have personally vetted and made sure that they have read and understood what is written here: https://github.com/thetorproject/ooni-probe#read-this-before-running-oonipro....
Some people are of the opinion that still this is not enough and that by paying them the risk is increased. It is not yet fully clear to me why that would be the case, nor what can be done to make the situation better.
Some have suggested we consult some lawyers that have background in international law to tell us how we can make this situation better. I believe this is probably a good idea.
It was also mentioned that Stony Brook university may also have valuable feedback in this area and we should also reach out to them.
I invite all the people present during yesterdays meeting to integrate their feedback into this thread and forward this email to people that can further advise.
~ Arturo
Hi Arturo and Oonitarians,
Thank you for starting this thread, I think this might be a more constructive of a medium than the frenetic exchange that occurred over IRC. My response could probably drag on, but the more that I have written, the more that I recall that I have narrow concerns and that my objection comes from the plan dovetailing into another thread we have had about the current state of data sanitization and risk disclosure for OONI on M-Lab.
There were a number of directions in that IRC conversation, so for time I do not want to reenact it or attempt a line for line critique of those logs. Moreover, there were some parallels that were made to current and historical programs that I do not think are constructive, and previous efforts at network measurement collection often differ in risks than those under OONI.
My argument was not that by paying individuals the risk faced by them increases. Instead, between the deployment strategy and the collection of data from new sources, I want to point out that:
The risk disclosures and documentation of data policies for OONI are currently inadequate for actual informed consent, in both the description of risks and lack of localization. In fact, I could not find a privacy or data retention policy anywhere on ooni.nu that would inform me what others do with my data (including banal matters like what logging is done by bouncers and helpers). Paid participation in OONI data collection has the possibility of distorting personal assessments of risk in a manner that may encourage improper choices. Renumeration is not in itself bad, but instead represents an area that OONI has not previously engaged in, and it might not be appropriate to launch into this immediately.
I suspect there is actually enough standing research on the second point, since similar issues are present in pharmacological studies, etc. There is the concurrent work on ethnics in censorship measurement that has developed over the past year or so, and I was surprised that OONI does not seem connected to it, hence the Stony Brook comment. However, my point here was that this is an issue that should be acknowledged, that OONI cannot fully account for these influences, that others in our community could probably inform us how to do this correctly, and that, until then, OONI should not engage in such a practice. We can continue this argument if others are interested in it, but I would be surprised if you could not avoid this issue for now by finding volunteers that do not require money to participate.
The first matter is more straightforward and solvable within a shorter window of time. Thus far, installation of OONI has required a meaningful amount of work that has created high barriers to entry, which has been a proxy for informed consent. The more that OONI offers a pre-packaged product (so to speak), the more that it needs to reassess the state of these aspects of the project; so it might not even be about those three people (but you do also want a process for selecting candidates that is vetted by others by the way).
The documentation on risks is currently inadequate and in some cases nonexistent. I noted three mentions of potential risk within the course of downloading and running OONI to conduct the HTTP Field Manipulation test.
Upon runtime:
WARNING: running ooniprobe involves some risk that varies greatly from country to country. You should be aware of this when running the tool. Read more about this in the manpage or README.
From the README:
Running ooniprobe is a potentially risky activity. This greatly depends on the jurisdiction in which you are in and which test you are running. It is technically possible for a person observing your internet connection to be aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted.
Futhermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk.
From ts-006-header-field-manipulation.md:
If the user is behind a transparent HTTP proxy that sets the X-Forwarded-For header their IP address will end up being part of the final report.
In summary, I am told that a network intermediary could determine that I am running a test and that a device search could reveal that I have installed the tool.
Between these three notices, I am not told how OONI uses my data or who has access to it, and what assurances are offered about the confidentiality of my reports. After all, OONI does attempt to conceal my identity through reporting AS by default and submitting over Tor, but is there any further data sanitization – will the maintainers seek to clean obvious PII from reports? Moreover, the header-field-manipulation.md notice is only on the bottom of the spec (not displayed at runtime) and is inadequate in addressing the potential risks of de-anonymization – we all know that this is a more expansive problem (e.g. the tracking codes inserted into headers).
Physical devices likely necessitate even further documentation, off the cuff, 1.) will OONI push down new tests to the probes that could change the risks to the users, and if so, what process will be provided for opting out of those tests? 2.) What is the default configuration for reporting and what tests are run against what targets?
—
I want to push aggressively back on the notion that lawyers will help in anything other than advising OONI on its own liabilities. Within the countries where collaboration with international (or “illegal”) non-governmental organizations is deemed a crime, the retention of lawyers to support participants is likely to be of little help and one should not assume that as a fallback strategy.
My counterproposal was simply that OONI should instead focus within this pilot to countries that qualify under any arbitrary metric as less prone to risk, this might use Freedom House or Transparency International style rankings. Under that condition, I would be less concerned about payments. The pilot should attempt to operate as a larger initiative would, and have a survey that would measure the understanding of the participant on their risks, as well as factors like usability and needs. During that time, OONI should reach out to colleagues who are also deploying rPi-based tests in order to understand how they have approached informed consent, to shore up its literature on risk as I have noted above, and translate disclosure material if necessary.
I think this has been a decent start and given people enough to push back on, so I’ll leave it there. Thanks again for starting this Arturo and I hope the 31c3 presentation went well.
Cordially, Collin
On Dec 23, 2014, at 2:15 AM, Arturo Filastò art@torproject.org wrote:
Hello Oonitarians,
During yesterdays we had a very interesting conversation about the ethics of measurements, informed consent and methodologies for achieving it. These are very important discussions to have and we agreed that we should continue it in this thread.
For those that were not on IRC at that time I will explain briefly what it was that sparked the debate. If you are interested in reading the full transcript of the meeting private message me and I will send it to you.
A problem that we have with OONI and I think is common to most network measurements projects is that of acquiring reliable vantage points in non western countries. By reliable I mean vantage points where the tool in question is not just run once and forgotten, but is periodically run, say once every day. One way of acquiring vantage points is to rent VPS' and setting up the tool on such VPS'. The problem with this approach, though, is that what you are measuring is not the network that a real user in that country would be using.
To overcome this issue I have come up with a scheme where by I get in contact with people from countries that interest us and give them some money to buy a raspberry pi and setup ooniprobe on it. As an incentive to keep the probe running and gathering data with a daily resolution I then pay them a small monthly fee to cover bandwidth and power costs. It turns out that not only is this cheaper than renting a VPS in that country, but it also gives us more accurate results, since the measurements are done from the users DSL home connection.
The problem with this approach is that we need to make it absolutely clear that there is some risk involved in running the software and the amount of risk varies greatly from country to country. So far I have limited this to a very small set of people (3 in total, 2 paid and 1 not paid) that I have personally vetted and made sure that they have read and understood what is written here: https://github.com/thetorproject/ooni-probe#read-this-before-running-oonipro....
Some people are of the opinion that still this is not enough and that by paying them the risk is increased. It is not yet fully clear to me why that would be the case, nor what can be done to make the situation better.
Some have suggested we consult some lawyers that have background in international law to tell us how we can make this situation better. I believe this is probably a good idea.
It was also mentioned that Stony Brook university may also have valuable feedback in this area and we should also reach out to them.
I invite all the people present during yesterdays meeting to integrate their feedback into this thread and forward this email to people that can further advise.
~ Arturo
Hi Arturo and all,
When it comes to ethics of soliciting measurements and informed consent, I have a different take which has been my research topic over the past years. There are many reasons why I think that directly measuring censorship is scary. First of all, you need to acquire reliable vantage points to run your measurements. Volunteering one’s machine to foreign researchers, or operating a device on their behalf, might be viewed by the government as espionage. Besides, many regions, especially places where we don’t have good infrastructure, have a limited number of companies/volunteers (if any) that allow foreigners to rent computers inside the country. All the current direct approaches, such as RIPE Atlas [1] or other distributed platforms or volunteers running Raspberry Pis are often easy to spot and data collected from them may not be reliable. For example, regarding China, we showed [2] that censorship is different in CERNET (China Education and Research Network) compared to other ISPs.
When it comes to measuring connectivity, I believe that it is better to involve the whole country in doing the measurements rather than volunteers whose safety is at stake. Therefore, I have developed effective methods for remotely measuring Internet censorship around the world, without requiring access to any of the machines whose connectivity is tested to or from. These techniques are based on novel network inference channels, a.k.a idle scans. That is, given two arbitrary IP addresses on the Internet that meet some simple requirements such as global IPID behaviour, our proposed technique can discover packet drops (e.g., due to censorship) between the two remote machines, as well as infer in which direction the packet drops are occurring. Here are more references to read [3,4]. Basically, for one of the idle scans (hybrid idle scan), we only create unsolicited packets (a bunch of SYNACK and RST segments) between two remote IPs, and look at the changes in the global IPID variable to infer whether censorship is happening and if so, in which direction packets are dropped.
Back to my main point, why I am trying so hard to convince you that we also need to use side channels and how this relates to ethics, well, here is the story: The discussion you brought up has been discussed heavily in academia in the past six months after two papers got rejected from the IMC conference because of ethics. One of them was my paper [2] after having received good reviews on the technical contribution. Here is the link to the reviews:
https://imc2014.cs.wisc.edu/hotcrp/paper/243?cap=0243a2kWYrwVqbv0
I personally just got an email with above link from IMC, and because of having had a single-entry visa, I couldn’t attend IMC or the Citizen Lab workshop where a lot of the discussions about ethics were taking place. The ethical issues that usually come up are two: First, using idle scans, no consent from users is collected. Second, censors could mistakenly assume that two machines measured by us are deliberately communicating with each other. This could have negative consequences if a censor believes that a user is communicating with a sensitive or forbidden IP address.
In response to the latter argument, it is unlikely that a censor would come to such a conclusion as only RST segments are created from a client inside a country to a server and only SYN/ACK segments are sent from a server to a client inside the censoring country. An adversary would not witness a full TCP handshake, let alone any actual data transfer.
One mitigation technique that I have been focusing on is to use routers instead of end points for the side channel measurements.
If you or anyone else is interested in using these techniques, I am more than happy to help.
Roya
[2] http://arxiv.org/abs/1410.0735
[3]http://arxiv.org/pdf/1312.5739v1.pdf
[4]http://www.usenix.org/event/sec10/tech/full_papers/Ensafi.pdf
Hi folks,
Firstly, hello! Having met Arturo and Vasilis at the 31C3, I'm keen to get involved more.
On consent: at an absolute minimum, I agree we need better warnings for users.
Below is some proposed text to inform users about the risks of OONI. I'm willing to take on the job of refining this based on feedback.
IMO we need two versions. One short and simple, that users should have to read (and agree to?) before first running ooniprobe. The second comprehensive, to be put on the website and in the docs.
A) THE SHORT VERSION
WARNING: Running OONI may be illegal in your country, or forbidden by your ISP. By running OONI you will connect to web services which may be banned, and use web censorship circumvention methods such as Tor. The OONI project will publish data submitted by probes, possibly including your IP address or other identifying information. In addition, your use of OONI will be clear to anybody who has access to your computer, and to anybody who can monitor your internet connection (such as your employer, ISP or government).
[link to long version]
B) THE LONG VERSION
LEGALITY
OONI does several things which may be illegal in your country, and/or banned by your ISP.
OONI's http test will download data from controversial websites, specifically targeting those which may be censored in your country. These may include, for example, sites containing pornography or hate speech. You can find a list of sites checked at https://github.com/citizenlab/test-lists
Even where these sites are not blocked, it may be illegal to access them. It may also be illegal to bypass censorship, as OONI attempts by using Tor.
In the most extreme case, any form of network monitoring could be illegal or banned, or even considered a form of espionage.
[Include link to some resource on relevant laws globally. Someone like the EFF must have one of these; does anybody have a link?]
PRIVACY
OONI IS NOT DESIGNED TO PROTECT YOUR PRIVACY. It will reveal information about your internet connection to the whole world. Particular groups, such as your ISP and web services used by the ooni tests, will be able to discover even more detailed information about you.
THE PUBLIC will be able to see the information collected by OONIprobe. This will definitely include your approximate location, the network (ASN) you are connecting from, and when you ran ooniprobe. Other identifying information, such as your IP address, is not deliberately collected, but may be included in HTTP headers or other metadata. The full page content downloaded by OONI could potentially include further information, for example if a website includes tracking codes or custom content based on your network location.
You can see what information OONI releases to the public at https://ooni.torproject.org/reports/. You should expect this information to remain online PERMANENTLY. [include details of retention policy, once we have one]
THE OONI PROJECT will also be able to see your IP address [What other info do we get?]
ORGANIZATIONS MONITORING YOUR INTERNET CONNECTION will be able to see all web traffic generated by OONI, including your IP address, and will likely be able to link it to you personally. These organizations might include your government, your ISP, and your employer.
ANYBODY WITH ACCESS TO YOUR COMPUTER, now or in the future, may be able to detect that you have installed or run ooni
SERVICES CONNECTED TO BY OONI will be able to see your IP address, and may be able to detect that you are using OONI
On 28/12/14 06:50, royaen wrote:
Hi Arturo and all,
When it comes to ethics of soliciting measurements and informed consent, I have a different take which has been my research topic over the past years. There are many reasons why I think that directly measuring censorship is scary. First of all, you need to acquire reliable vantage points to run your measurements. Volunteering one’s machine to foreign researchers, or operating a device on their behalf, might be viewed by the government as espionage. Besides, many regions, especially places where we don’t have good infrastructure, have a limited number of companies/volunteers (if any) that allow foreigners to rent computers inside the country. All the current direct approaches, such as RIPE Atlas [1] or other distributed platforms or volunteers running Raspberry Pis are often easy to spot and data collected from them may not be reliable. For example, regarding China, we showed [2] that censorship is different in CERNET (China Education and Research Network) compared to other ISPs.
When it comes to measuring connectivity, I believe that it is better to involve the whole country in doing the measurements rather than volunteers whose safety is at stake. Therefore, I have developed effective methods for remotely measuring Internet censorship around the world, without requiring access to any of the machines whose connectivity is tested to or from. These techniques are based on novel network inference channels, a.k.a idle scans. That is, given two arbitrary IP addresses on the Internet that meet some simple requirements such as global IPID behaviour, our proposed technique can discover packet drops (e.g., due to censorship) between the two remote machines, as well as infer in which direction the packet drops are occurring. Here are more references to read [3,4]. Basically, for one of the idle scans (hybrid idle scan), we only create unsolicited packets (a bunch of SYNACK and RST segments) between two remote IPs, and look at the changes in the global IPID variable to infer whether censorship is happening and if so, in which direction packets are dropped.
Back to my main point, why I am trying so hard to convince you that we also need to use side channels and how this relates to ethics, well, here is the story: The discussion you brought up has been discussed heavily in academia in the past six months after two papers got rejected from the IMC conference because of ethics. One of them was my paper [2] after having received good reviews on the technical contribution. Here is the link to the reviews:
https://imc2014.cs.wisc.edu/hotcrp/paper/243?cap=0243a2kWYrwVqbv0
I personally just got an email with above link from IMC, and because of having had a single-entry visa, I couldn’t attend IMC or the Citizen Lab workshop where a lot of the discussions about ethics were taking place. The ethical issues that usually come up are two: First, using idle scans, no consent from users is collected. Second, censors could mistakenly assume that two machines measured by us are deliberately communicating with each other. This could have negative consequences if a censor believes that a user is communicating with a sensitive or forbidden IP address.
In response to the latter argument, it is unlikely that a censor would come to such a conclusion as only RST segments are created from a client inside a country to a server and only SYN/ACK segments are sent from a server to a client inside the censoring country. An adversary would not witness a full TCP handshake, let alone any actual data transfer.
One mitigation technique that I have been focusing on is to use routers instead of end points for the side channel measurements.
If you or anyone else is interested in using these techniques, I am more than happy to help.
Roya
[2] http://arxiv.org/abs/1410.0735
[3]http://arxiv.org/pdf/1312.5739v1.pdf
[4]http://www.usenix.org/event/sec10/tech/full_papers/Ensafi.pdf
ooni-dev mailing list ooni-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/ooni-dev
Some further thoughts:
ACTIVE CONSENT. Users should probably be shown the legal/privacy warning when first using ooni, and have to signal their agreement/understanding.
FILTERING BY CONSENT. Once we have a appropriate warnings in place, we should encourage researchers to use only that data which comes with consent. We could include a flag in the data reported. But it might be enough just to track (and document) the version number after which we get adequate consent.
ACADEMIC ETHICS REQUIREMENTS. Whatever we come up with for ensuring consent, we should check it will be sufficient for academics to use our data. AIUI ethics boards react to specific research proposals, so we can't get a general all-clear for ooni. The best we can do is for somebody to propose an ooni-based project, and share with us the feedback from their university ethics committee.
PUBLISHING CONTENT. ooni users aren't just *downloading* objectionable content, but *publishing* it in the form of reports. It'd take a bloody-minded lawyer to make that argument, but there's no shortage of them in the world...
MONITORING BY THE MONITORED. All sites in our the http_requests have a complete list of ooni users, don't they? [maybe not identified as such, but you just need to control 2 and take the intersection of their log files]. And once your user list is shared by hackforums.net and 911lies.org, you may as well just consider it public ;)
best, Dan
On 02/01/15 18:30, Dan O'Huiginn wrote:
Hi folks,
Firstly, hello! Having met Arturo and Vasilis at the 31C3, I'm keen to get involved more.
On consent: at an absolute minimum, I agree we need better warnings for users.
Below is some proposed text to inform users about the risks of OONI. I'm willing to take on the job of refining this based on feedback.
IMO we need two versions. One short and simple, that users should have to read (and agree to?) before first running ooniprobe. The second comprehensive, to be put on the website and in the docs.
A) THE SHORT VERSION
WARNING: Running OONI may be illegal in your country, or forbidden by your ISP. By running OONI you will connect to web services which may be banned, and use web censorship circumvention methods such as Tor. The OONI project will publish data submitted by probes, possibly including your IP address or other identifying information. In addition, your use of OONI will be clear to anybody who has access to your computer, and to anybody who can monitor your internet connection (such as your employer, ISP or government).
[link to long version]
B) THE LONG VERSION
LEGALITY
OONI does several things which may be illegal in your country, and/or banned by your ISP.
OONI's http test will download data from controversial websites, specifically targeting those which may be censored in your country. These may include, for example, sites containing pornography or hate speech. You can find a list of sites checked at https://github.com/citizenlab/test-lists
Even where these sites are not blocked, it may be illegal to access them. It may also be illegal to bypass censorship, as OONI attempts by using Tor.
In the most extreme case, any form of network monitoring could be illegal or banned, or even considered a form of espionage.
[Include link to some resource on relevant laws globally. Someone like the EFF must have one of these; does anybody have a link?]
PRIVACY
OONI IS NOT DESIGNED TO PROTECT YOUR PRIVACY. It will reveal information about your internet connection to the whole world. Particular groups, such as your ISP and web services used by the ooni tests, will be able to discover even more detailed information about you.
THE PUBLIC will be able to see the information collected by OONIprobe. This will definitely include your approximate location, the network (ASN) you are connecting from, and when you ran ooniprobe. Other identifying information, such as your IP address, is not deliberately collected, but may be included in HTTP headers or other metadata. The full page content downloaded by OONI could potentially include further information, for example if a website includes tracking codes or custom content based on your network location.
You can see what information OONI releases to the public at https://ooni.torproject.org/reports/. You should expect this information to remain online PERMANENTLY. [include details of retention policy, once we have one]
THE OONI PROJECT will also be able to see your IP address [What other info do we get?]
ORGANIZATIONS MONITORING YOUR INTERNET CONNECTION will be able to see all web traffic generated by OONI, including your IP address, and will likely be able to link it to you personally. These organizations might include your government, your ISP, and your employer.
ANYBODY WITH ACCESS TO YOUR COMPUTER, now or in the future, may be able to detect that you have installed or run ooni
SERVICES CONNECTED TO BY OONI will be able to see your IP address, and may be able to detect that you are using OONI
On 28/12/14 06:50, royaen wrote:
Hi Arturo and all,
When it comes to ethics of soliciting measurements and informed consent, I have a different take which has been my research topic over the past years. There are many reasons why I think that directly measuring censorship is scary. First of all, you need to acquire reliable vantage points to run your measurements. Volunteering one’s machine to foreign researchers, or operating a device on their behalf, might be viewed by the government as espionage. Besides, many regions, especially places where we don’t have good infrastructure, have a limited number of companies/volunteers (if any) that allow foreigners to rent computers inside the country. All the current direct approaches, such as RIPE Atlas [1] or other distributed platforms or volunteers running Raspberry Pis are often easy to spot and data collected from them may not be reliable. For example, regarding China, we showed [2] that censorship is different in CERNET (China Education and Research Network) compared to other ISPs.
When it comes to measuring connectivity, I believe that it is better to involve the whole country in doing the measurements rather than volunteers whose safety is at stake. Therefore, I have developed effective methods for remotely measuring Internet censorship around the world, without requiring access to any of the machines whose connectivity is tested to or from. These techniques are based on novel network inference channels, a.k.a idle scans. That is, given two arbitrary IP addresses on the Internet that meet some simple requirements such as global IPID behaviour, our proposed technique can discover packet drops (e.g., due to censorship) between the two remote machines, as well as infer in which direction the packet drops are occurring. Here are more references to read [3,4]. Basically, for one of the idle scans (hybrid idle scan), we only create unsolicited packets (a bunch of SYNACK and RST segments) between two remote IPs, and look at the changes in the global IPID variable to infer whether censorship is happening and if so, in which direction packets are dropped.
Back to my main point, why I am trying so hard to convince you that we also need to use side channels and how this relates to ethics, well, here is the story: The discussion you brought up has been discussed heavily in academia in the past six months after two papers got rejected from the IMC conference because of ethics. One of them was my paper [2] after having received good reviews on the technical contribution. Here is the link to the reviews:
https://imc2014.cs.wisc.edu/hotcrp/paper/243?cap=0243a2kWYrwVqbv0
I personally just got an email with above link from IMC, and because of having had a single-entry visa, I couldn’t attend IMC or the Citizen Lab workshop where a lot of the discussions about ethics were taking place. The ethical issues that usually come up are two: First, using idle scans, no consent from users is collected. Second, censors could mistakenly assume that two machines measured by us are deliberately communicating with each other. This could have negative consequences if a censor believes that a user is communicating with a sensitive or forbidden IP address.
In response to the latter argument, it is unlikely that a censor would come to such a conclusion as only RST segments are created from a client inside a country to a server and only SYN/ACK segments are sent from a server to a client inside the censoring country. An adversary would not witness a full TCP handshake, let alone any actual data transfer.
One mitigation technique that I have been focusing on is to use routers instead of end points for the side channel measurements.
If you or anyone else is interested in using these techniques, I am more than happy to help.
Roya
[2] http://arxiv.org/abs/1410.0735
[3]http://arxiv.org/pdf/1312.5739v1.pdf
[4]http://www.usenix.org/event/sec10/tech/full_papers/Ensafi.pdf
ooni-dev mailing list ooni-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/ooni-dev
On 1/2/15 6:30 PM, Dan O'Huiginn wrote:
Hi folks,
Firstly, hello! Having met Arturo and Vasilis at the 31C3, I'm keen to get involved more.
On consent: at an absolute minimum, I agree we need better warnings for users.
I fully agree!
Below is some proposed text to inform users about the risks of OONI. I'm willing to take on the job of refining this based on feedback.
IMO we need two versions. One short and simple, that users should have to read (and agree to?) before first running ooniprobe. The second comprehensive, to be put on the website and in the docs.
A) THE SHORT VERSION
WARNING: Running OONI may be illegal in your country, or forbidden by your ISP. By running OONI you will connect to web services which may be banned, and use web censorship circumvention methods such as Tor. The OONI project will publish data submitted by probes, possibly including your IP address or other identifying information. In addition, your use of OONI will be clear to anybody who has access to your computer, and to anybody who can monitor your internet connection (such as your employer, ISP or government).
[link to long version]
This is music to my ears!
Really good job!
I think this is perfect. I would just replace the OONI with ooniprobe, since you are running a specific tool part of OONI.
I would also add a note such as: "Please read more about the involved with running ooniprobe here: " before the link to the long version.
B) THE LONG VERSION
LEGALITY
OONI does several things which may be illegal in your country, and/or banned by your ISP.
OONI's http test will download data from controversial websites, specifically targeting those which may be censored in your country. These may include, for example, sites containing pornography or hate speech. You can find a list of sites checked at https://github.com/citizenlab/test-lists
We should create a specific repository for test lists and make that have the ones we depend on as submodules.
It should point to this: https://github.com/TheTorProject/ooni-test-lists
Even where these sites are not blocked, it may be illegal to access them. It may also be illegal to bypass censorship, as OONI attempts by using Tor.
In the most extreme case, any form of network monitoring could be illegal or banned, or even considered a form of espionage.
[Include link to some resource on relevant laws globally. Someone like the EFF must have one of these; does anybody have a link?]
PRIVACY
OONI IS NOT DESIGNED TO PROTECT YOUR PRIVACY. It will reveal information about your internet connection to the whole world. Particular groups, such as your ISP and web services used by the ooni tests, will be able to discover even more detailed information about you.
What do you mean by this last statement? Are there things in particular ooniprobe gives away about the user that would not have been given away already?
THE PUBLIC will be able to see the information collected by OONIprobe. This will definitely include your approximate location, the network (ASN) you are connecting from, and when you ran ooniprobe. Other identifying information, such as your IP address, is not deliberately collected, but may be included in HTTP headers or other metadata. The full page content downloaded by OONI could potentially include further information, for example if a website includes tracking codes or custom content based on your network location.
You can see what information OONI releases to the public at https://ooni.torproject.org/reports/. You should expect this information to remain online PERMANENTLY. [include details of retention policy, once we have one]
THE OONI PROJECT will also be able to see your IP address [What other info do we get?]
ORGANIZATIONS MONITORING YOUR INTERNET CONNECTION will be able to see all web traffic generated by OONI, including your IP address, and will likely be able to link it to you personally. These organizations might include your government, your ISP, and your employer.
ANYBODY WITH ACCESS TO YOUR COMPUTER, now or in the future, may be able to detect that you have installed or run ooni
SERVICES CONNECTED TO BY OONI will be able to see your IP address, and may be able to detect that you are using OONI
I would say you create a new directory inside of ooni-probe/docs/source/ called "information" or something similar that we can use to put in there also the data retention policy and other related information.
I would create two files called "risks-short.rst" and "risks-long.rst" and put the content of what you have so far.
Then you can submit a pull request and I will merge it. Then I will make it so they are included as part of the ooniprobe software and displayed when needed.
~ Arturo
A) THE SHORT VERSION
WARNING: Running OONI may be illegal in your country, or forbidden by your ISP. By running OONI you will connect to web services which may be banned, and use web censorship circumvention methods such as Tor. The OONI project will publish data submitted by probes, possibly including your IP address or other identifying information. In addition, your use of OONI will be clear to anybody who has access to your computer, and to anybody who can monitor your internet connection (such as your employer, ISP or government).
I would strongly advise against this kind of wording. It reads like hackers trying to CYA and also to settle some kind of long standing argument, without considering real world impact of *those* statements. Advocating for somethign that may be illegal is at best, a CYA but at worst, actually setting people up for massive problems.
All the best, Jacob
-----BEGIN PGP MESSAGE----- Charset: windows-1252 Version: GnuPG v1.4.15 (Darwin) Comment: GPGTools - http://gpgtools.org
hQIMA5qq1v9FvppnARAAnQNWTBtgzOxC4pO5bxCbpnwAkyNHqlt8QdcwhvCOTHi6 NYDuQ39XrNKjPGBOex518C6HLCIKZKInhh2AuTh6bc4A53DMg72RijNdLNDdEkd9 a1PhR+K+0aIaIsPay6I6srGzYv2f32zZs+MRFTFlTucE+KTczW0MDUtasghAhTJc NFi5DljvKzROC52puhzFO0UPUKNlDHSGSfMii4vhfNyjtbBvGPl/qY0vdOi4Nar8 2h40SzvPAgdStO0R2/0ElEbPDjmGYnPFxpVrJrQ4p4C4/NgcgR0jmG6Ue1DOnNq1 GNzJh19wpa9gmiNbfUOH1+s9I57iB54+Ie0zA9H81cquT/gC0xcJQ3fyW7o/gsRF 2CNQpOY0l5L0hc5dEBDTY/i5AttC4NVnruOzNjrNgeOW+uaRyoGdp6C5M02U5tEF jeKZORh+Mib0yHoE3hCs13eG9mQdFTrjW/b78sLxGcS3+H/7vUS9dm66u39DmPdQ ZvT1BTWgeANG/SRXtPqdOP1oGmGn1loi/bVFLGVULhz3qEIjpXVIM6BK0ssAgy9D Usl8QzhoIBO+plM32jzHvlmAdlwcxiLJPQSdNXZ+SptJD4bl5k5TPfeNpVmG09FD In3rNXJXrZOFVFiO82HRP45Zgz2FsUoFCq2OTbhimqJDZsrE2Wf8LdqKPICFs0eF AgwDB5zYufQjsnwBD/9adxQAY3lG3qxJi5LTOwTKgWIk90VxiZVu9KXmWTL3E+5X 2v8Le4b8Ep2o/TrhmlniclIS7EpCmC4GcVMTnEKyWpxoEcacPetIAdD1U3wI2xbd C1Zxd3jLJFHkCN3eKVlN1d14+PQ2SsUIqHVWE08oq9J/fI0FbTeRsO827voyMNt1 pNs0EwWa1otoHgd0rDXOjMlYuhzS5y6tVZ5VU070WUeRKa+Ft2kbhMxUblSc486f SS3UMNxlZZqSNmpnfQYmlk3bykyveQJfOCeUvO2XWYalZ7WKcl+nvkWWDxTppNVI bTB6UtyTwqa+8zgzdzLJe7sZNcVhT5f1EhQwmkV0+kS/1p+e/eLowELRJ+Aop/ku zMgrc7LQFTFk9q7NS+JOZgEWaJw9g59CDt8hHJePRs4eIdBtYRdzXAMYMtQr4V3m pyPtYqZmb/oFaD99Yfvm4/BTyv4mMFcoO4yAZWMto6XhutSHoX7Va4p0MnkBmxTB 3SnTbmJjSDiK42Ub7WnXglJ7UVcgNwGcBtAAVdLgtVV8gp+bbEYsAlXW6OHnFvuT fKycjtxRLwQymKRnWA78p3fy6m1si+ZKK/1mpF/mE7HUFdpAQKkwuUug1D3UiOkD kIvmgGfl0DGt9Ofrim8NEvgnQepyCoFL0RcvarbfxVoNlK/BsjS+9SDshyMU5NLr AVij7ax66lJEk+Y+oFtYYfzas9j5ORu8RQg16Hfr8Agvf5fMiXFBj/mjrGk7X15h 3eO6pjU0zNiQcVV0uDx+/QnhzIUiWHdPFdosbb89DdrX16ea+pmH+bXuNie6p2OS 48zzoWFaqnn+ryKQj7uX1n0YT+n8ml0kUZUYOaPlm6/MtpT5y7rhhXg6MbsJ7/IB Hd/HCXLJeiNgJRlkJVJ3qwUmKIOmY0Z4UeFQ2wDRjnkD7oiAtQnSLZ/nyeS/FR9K L0hyQNaqnwhYcuWyCXyfUNqquNMmqRhtI2uY5paNnF2sXIROjEDzso+Z+QyhUFdG xYZ/2rDSttaxEZnhh5WCZUvQaWO6859dGplnvBPLS190Rn6b4F5hfa7G8IaMAYWX 0B1M3INVfGMQRR49b0xcZH2wlcQW60Wpt7G4+p70q01VBjmNq5GIu78Hz3qyjAC7 3YIzyInZ3hgwMMUs527mnzRP7dUfUvziSpZZ3iz/yE4rpV7CEV1lJdeM0T1r5YP7 CYAMBIRgAyVmZtOA36lqmQpbqseCCwkNXWY5cFF7+0z6JLmz6ap070gTUQzKPPOY 0uWL1rljrRECHn5XUtS0g8MCo08sVknJfIL6yFHEszsB7qBLgtQIga4PBwUrEqov Z6KnTiivfQWha4BfnvkPdsJ7wSOamHkYi00Y6DRp2crTYhhgz7cAiD3b/ARqF8jr d9vv+5VG/tng0ukk/q7VkyCsVdxzxoaYNfi8mB0XtdpPaXhZS7MCsuc+za63CNvg aXlDlR7tjDZ6efv9JSM/df5K7JiWQHIIwEEAsdgCwWtds9kwgNm8Bi4dpsr6S24I wci8ZBijPhlWuEa7SSOgqQVKvWvYbnhJalrGARCneGvsHeyfYWLpdJ3ab1YIbhrk 89O0jgX0S7iTuPEPSRYJskfUxW5QW77PfhutJgX4P7S/5j4qp+0ZSbmGzXmk4RQd h7T1o4PUETP/SBCp6ZvcOO8MupKQ4wC9RJ8RW4HEBNd/Au/+S7b4Ounr6uRDzk7V m3af/DPg0gRhI9hqJBO1xpKE0z3i7nc6ssBF0IwDHQDmxcyamLqjt8pCsQOEqw5A uAIKY5N2GxC1sFcjovM/n/J14ILraFxsGQShFiWIpkCNepUsYJFyo88TGBIcHjio qoaTXeCZxaEbJeEh7ya6aOnZ1+TblKgSKdrXipMSjYgDsgrhmg3I3mep5vbuBRC5 hiK65HBxLAdmS+znXt0a3/Gox45r7ocsibMJsjzqX9tiKQlgpHsn4E8jiJ2idaio 4jtmq0SyC2qpGIT8ZWAUJPgb1EuJogHvox3+eCaGbmn4ycEtl+Zufu173AL0Xckp vZHPzFM00AV6vIP+HmB6V0TjmFhUDvv43sYn65ZI8Som674vKw8kHTroIQ/uFK+o UrWYg+i3b3eN8Ve13OsgUbbtHuk/BIWQnYLeQFQd72b6OF4CfyGOrlPrMtMoL7Ns bJiZPcsQRyLnkxKqc6NqCCd+eDV5KCxEduoHOkO9vpHT6+YXxwilRhf6lVCpFNpw KgkjRBKVkieWbNkGGI3LTFsCa88F4kff0GbENZUnaIw+XIDW/xQZZYOCrY5c7KwJ xVptfbv6m28Mo+XOjY5QrV0XuhIacJ+wgWnUU9J3om2qKnPin4bYnQ2bACIP8nq/ ldVs4SMGYPf9U5AHvM0FuUjFxXHDVSYmJ3TbIF7AcKMallZUDKphgvgdHKM7HzKP dQumY0/zN4dE+/HT1A7AMYTTZTPBsQEAJ5BsZZ33ULT23UhBdsnklKsLcK9oPp7O WZcIjTBcfsuGADHm+K0l/doyqkpAAOupEktaHOG0OB16HLz0AA5nXMX3yzn0ReET tKhd5DW07n3w/8qhGeB2tYRHRUdnk81cKwC1De1kr88BZVZyy9Jga7QJ+ATcULOm k4aHQRs7O9JLdVgzHW5U74jiMA+FlKcgKlLsWBVDy4GMnQFaP0fWyau5eBDaVsX2 oF1huSUaNCEd+FT+Q/gzjDP34EIKDHErK7Ry6BpZjt7nqHDwijz9MI1C/Bo4l9Mi WW4gzJTmbcr8dCSuzhXf1l+O05rAMkqLT9sKdUbiSR0UwivpxPR9uSBQ6sO7HL0D 4eUeAq70OAzYU9Omau6SfLNB0ubDlcBRE052xX2pbfG7qZZTJ08ELLiPBj0WBOuB 1S1T35UFXWwb05pZl9Bu9435KDdYLArcxbT8zn9OCQjW6VDjHLdsfgFe8Ug87IZ6 e2Bt62/w52hpz2eSGLLI/qBbnn6hxEw14+xz4SOTcb6EPT4vQjygm3eLnIV2dbh5 vQf2NCgzKiRmoFFh5FBR2aHbOgR83JSiO5AU2uhx+Cjts7WpLaSexCdOKnsI7bKm sYlXnBgFDnGORe5TtXgVznjA8ZQk6JZHo+btyl2YIwTzmRgLshm8oYRrdLd+He96 EiFY09uDg/1oHed4fZ3EShUOJx42suLyiyaAaihHLG1J5qkY52oSxU5u8kWREbyH uhxsWHE9Xo25Ak0A7ntdxtnY3MmcdFz4npuQHkTAuue0gz9ubs9dWjb2Ti/D7D0X Xn3CXrPx5hEjMisooYwjNR5MtI57zChix5lt38m5XoqcTFp0HNgWCIhYCA1j53xw rX8QqtvW/BdXjO5SlPbvsSpNedemypmjkgeA3zKq9ErWrSEaAWiC9U1kcZloiohU y5ulkfVrKrfMhpEuqbBfE0fRkDhh89LYrx6jxBWQtqXqMDCKE/gn+dqlexZGZOdr GRgsRetuQLoyA6pgyRoYpHWwqIr5PFv0S5U3gAFbcppxIcO/3fOYrDso+bdiQBeL gNKYeeFCiSZepA+/YggjqIemxbfVGStwPT3tkFLHdVe6f74rMUPue5diHCpGKURF puky9+2mrUemVlER4bnM67tMatBcf87d6N7KTa7AlH2+Iy4+ztoh/XJ2T+RMHI3P 8tV4Fz+vhAlGpJqc6VW2HuQPo7FtVt+89vNug4IJQex2ZWL+h+h3PIw6QMWRFB5H xjlp9RGKkAW3FGUMz1jhsUSr/oC8w48wIlYlGa+buhBFyHgczX3Ar+vwH6WLQp5w I6tlq+ed0Ubk8HdpUCKm+KuzQbz8HlEUYPAKjKOtkmjsTPdn0EOioLR5Vfy4lu3Y X4IVZLBACmmCA/KPCbQHX9JJCBe1UJgQ9YMfpzP+0LIrbIxhXi6mfv1khTrJsEnE 6iYNJ1EmSmrDHZDdQUP9ep9AuykuI/oz6SZ5o0JKdzD+jxOujNgBZf/dYU5LKF1m WuTyHpcSzJptBtadw0RephOyHS8U7E4KiKIOgisklcf21+lvIK0aX9sfU915amC5 MLxxrjonwXkvCZzGCrYdRcO/Kt0t3tdmcKHUFILHzKS3npvK4vrFrKm/kqaucCt4 ReM6u8eeVqn+T+6DT+ryuq/zaxGsZ4KHhxoXbAnIdqACom4qtgCjogjLwLW9EOmI PMcNOfLe/inhlSEGHQF4QoK7x7C6vnC2qk30MbMu8lj00dvcAUH5qbDjAgWZLPkT HnyKZIrTh+2j/U8i2soXj9DGXNPncp+5Bmo12vPXC3ko93BF6gHrw75CJsB40Ut6 B7+XnMivyqtpcd8NJUKPJPKnx2HuP0R4gnjG839fjUeNgJGJL+xWR0eOiuXcrTow dn38SWaZYEJCsOCbyi5IuQzP6vHOHL4wYzPBwu3dfwU2yZxPYqhP2xESELPzfTns 4YKTMMMoD4k/dk1LsnX5hQObsTr/XJPMA+HMAQWiKiITMqIyLMAw9+qPkBpz96WG awee09mLC7C8WVcdQnIS3QiRCsqTUH0l0P4FoBrjPqWd/mWPzx+VJkwy94fujC33 E5sfvDSj9aTR0ASYZCtMWZmeWXHDeiYpkQfXA88Og3T3V9aoEb94cQznW1qOWxw6 thQp4drx4ANAIC3TTZFj9ejYg8AtlWTN7faa+rtKf114JXCZho73oOKNxMkdxoqc +UYjV3RT4zAFot/CX8g+7AV6Y25kwAp2WMu64DOCI57wlozuPxHQE2XtO/0z5N6n 8gyDQEsLRk+etGnw9IfwJrNUXqXnv08n6RpHZd8B79lpzpAGt6sRpfC2Zyce7brT 58BXpR4lI95HIHyQeX/pk8e6MxdZtkMDgkrWcw4xEKS4nXD7BIxLhKLkTsRiJfYr DUn/XnDk5qArqEzUZWWOwPNJXdBWcMbEuknSx2h3Rp5TPQiipET/UfdZVsaexhGK +X6ZbGJtZZYFmaJxxfh1zfKjtOmKwm+b+4gCrQz0zE2YRuNkvmrR0mHIeRvaR2Ik cuzpSXp/ZalvalhRlccYrJzScazFDhEUJZiiJA7+EfEQWTmJDsGjg0hkx2nT0apT 9RszIURqwVEJM8UVVArAEDsx4z4fMlj2QfAnGoHNDj3i0pbr30SC7fDM/qUhBcHa WyZRxgZinTBrZ0aMuRNkBEXalIK59ArgOEsR0AsbjaoN9Dguqf8ajn/T =FSLH -----END PGP MESSAGE-----
I had created a PGP key for the ooni-dev mailing list and always forget to disable encryption when sending to the list.
Should create a rule for that...
Anyways here it goes:
On 12/28/14 6:50 AM, royaen wrote:
Hi Arturo and all,
Hi Roya,
Thanks for the taking the time to chime into this discussion.
When it comes to ethics of soliciting measurements and informed consent, I have a different take which has been my research topic over the past years. There are many reasons why I think that directly measuring censorship is scary. First of all, you need to acquire reliable vantage points to run your measurements. Volunteering one’s machine to foreign researchers, or operating a device on their behalf, might be viewed by the government as espionage. Besides, many regions, especially places where we don’t have good infrastructure, have a limited number of companies/volunteers (if any) that allow foreigners to rent computers inside the country. All the current direct approaches, such as RIPE Atlas [1] or other distributed platforms or volunteers running Raspberry Pis are often easy to spot and data collected from them may not be reliable. For example, regarding China, we showed [2] that censorship is different in CERNET (China Education and Research Network) compared to other ISPs.
The reason why all these projects rely on vantage points from the network point of view is that this is the way to have the most accurate measurements and in a lot of cases it is the only way to measure that particular kind of censorship.
Being from the vantage point of the censored user allows you to fully emulate what a user would be doing when accessing the censored site.
When it comes to measuring connectivity, I believe that it is better to involve the whole country in doing the measurements rather than volunteers whose safety is at stake. Therefore, I have developed effective methods for remotely measuring Internet censorship around the world, without requiring access to any of the machines whose connectivity is tested to or from. These techniques are based on novel network inference channels, a.k.a idle scans. That is, given two arbitrary IP addresses on the Internet that meet some simple requirements such as global IPID behaviour, our proposed technique can discover packet drops (e.g., due to censorship) between the two remote machines, as well as infer in which direction the packet drops are occurring. Here are more references to read [3,4]. Basically, for one of the idle scans (hybrid idle scan), we only create unsolicited packets (a bunch of SYNACK and RST segments) between two remote IPs, and look at the changes in the global IPID variable to infer whether censorship is happening and if so, in which direction packets are dropped.
Your research is very interesting and I believe very important for getting more data when it would just be too risky to have network vantage. I do think, though, that we can't rely only on these sorts of measurements. They complement and extend what we measure from the network vantage point of the user, but may not work as reliably in all censorship systems and only give you a subset of the information we are interested in acquiring.
For example things that we are interested in gathering with ooniprobe are also fingerprints for the censorship equipment being used. This is something that I don't think will be as accurately measured with indirect scans.
Back to my main point, why I am trying so hard to convince you that we also need to use side channels and how this relates to ethics, well, here is the story: The discussion you brought up has been discussed heavily in academia in the past six months after two papers got rejected from the IMC conference because of ethics. One of them was my paper [2] after having received good reviews on the technical contribution. Here is the link to the reviews:
https://imc2014.cs.wisc.edu/hotcrp/paper/243?cap=0243a2kWYrwVqbv0
Yes I perfect agree with the fact that we should also be collecting measurements gathers using these sorts of technique using ooniprobe. It would be epic if you or somebody else were to implement ooniprobe measurements for them.
I would however like to make the point that with the OONI project our main goal is not that of publishing academic papers. If that comes as part of the process then it's great, but our top priority is finding the right balance between the safety of users and impact we can reach by exposing facts about internet censorship. This is something very tough, but I think that by not being directly affiliated with a university (hence not having to jump through the various hoops you folks have to before doing your research), we have a slight advantage. We don't have to get approval from IRBs or have to publish a certain amount of papers per year. The only people we are accountable to are our users.
I personally just got an email with above link from IMC, and because of having had a single-entry visa, I couldn’t attend IMC or the Citizen Lab workshop where a lot of the discussions about ethics were taking place. The ethical issues that usually come up are two: First, using idle scans, no consent from users is collected. Second, censors could mistakenly assume that two machines measured by us are deliberately communicating with each other. This could have negative consequences if a censor believes that a user is communicating with a sensitive or forbidden IP address.
In response to the latter argument, it is unlikely that a censor would come to such a conclusion as only RST segments are created from a client inside a country to a server and only SYN/ACK segments are sent from a server to a client inside the censoring country. An adversary would not witness a full TCP handshake, let alone any actual data transfer.
One mitigation technique that I have been focusing on is to use routers instead of end points for the side channel measurements.
I think that the censor would have a pretty hard job proving in a just court of law that such user was engaging in censorship measurements (assuming they consider censorship measurements to be an illegal thing). Unfortunately in some countries were we measure the courts of law are not just and we have to make all sorts of crazy assumptions on how they will interpret what we are doing. Using routers instead of real users when doing the scans could be a safer move if it does not affect your measurement.
If you or anyone else is interested in using these techniques, I am more than happy to help.
I will keep your experiments in mind if somebody comes wanting to hack on something interesting and point them to you.
I think the best thing to do would be to create a ticket(s) for implementing your tests on the Ooni component on: https://trac.torproject.org/
~ Arturo
Roya
[2] http://arxiv.org/abs/1410.0735
[3]http://arxiv.org/pdf/1312.5739v1.pdf
[4]http://www.usenix.org/event/sec10/tech/full_papers/Ensafi.pdf
Hi Arturo, Roya,
It's great to see this discussion here. I wanted to chime in as well to end my long-time lurking around here, and largely to support what Roya's been saying. I've published a bit of research in this area, some specifically on ethics and some on getting data with a relatively light touch, and I'm involved with a few other people who are working on a set of ethical principles associated with network measurement. (That work coming out of the IMC papers that Roya mentioned.) I think OONI is a key testbed for this kind of thinking.
To clarify: in the comments below, I'm not attacking anything you've said because I appreciate that OONI even considers these issues. I did want to pick them apart a bit, though. :)
On Mon, Jan 05, 2015 at 02:27:47PM +0100, Arturo Filastò wrote:
The reason why all these projects rely on vantage points from the network point of view is that this is the way to have the most accurate measurements and in a lot of cases it is the only way to measure that particular kind of censorship.
Being from the vantage point of the censored user allows you to fully emulate what a user would be doing when accessing the censored site.
I know that you are consciously making a balance between user safety and effectiveness, but it's worth explicitly stating that sometimes there are measurements you just can't have without compromising user safety, and sometimes you have to accept a loss in accuracy.
Your research is very interesting and I believe very important for getting more data when it would just be too risky to have network vantage. I do think, though, that we can't rely only on these sorts of measurements. They complement and extend what we measure from the network vantage point of the user, but may not work as reliably in all censorship systems and only give you a subset of the information we are interested in acquiring.
I think it might be useful to try and draw a line at which the risk to users overwhelms the need for accuracy. At the moment, even though ethics are being discussed, they seem to be subordinate to functionality.
For example things that we are interested in gathering with ooniprobe are also fingerprints for the censorship equipment being used. This is something that I don't think will be as accurately measured with indirect scans.
Of course the balance swings in the other direction, and you shouldn't abandon key data just because of a vague and unspecified risk. Have you considered alternative approaches to fingerprinting censorship equipment in detail, though? It strikes me that there are possibly several approaches that wouldn't rely on end user vantage points. In a sense, the danger of OONI is that you /can/ answer your questions with end user installs, so you have less incentive to find alternative less OONI-like approaches.
Another factor here is that, as you say below, you are trying to balance user safety against impact of getting data, but your impact is equally fuzzy. How do you measure your impact? How do you judge whether having detailed information about a filtering device is worth the possible risk to an end user?
Yes I perfect agree with the fact that we should also be collecting measurements gathers using these sorts of technique using ooniprobe. It would be epic if you or somebody else were to implement ooniprobe measurements for them.
This relates back to the OONI-centric idea -- shouldn't side channel techniques like these be an alternative data source, rather than worked into the end-user installed OONI model?
I would however like to make the point that with the OONI project our main goal is not that of publishing academic papers. If that comes as part of the process then it's great, but our top priority is finding the right balance between the safety of users and impact we can reach by exposing facts about internet censorship.
This is key, and again -- what is your measure of impact? How do you weigh it against potentially unknown user risks?
This is something very tough, but I think that by not being directly affiliated with a university (hence not having to jump through the various hoops you folks have to before doing your research), we have a slight advantage. We don't have to get approval from IRBs or have to publish a certain amount of papers per year. The only people we are accountable to are our users.
I know what you mean here, and I'm sure you didn't mean this how it reads, but this is the thing that convinced me to reply to this email! The fact that you don't have to 'jump through the hoops' of IRB approval is deeply worrying to me, because you have no independent oversight to balance your wishes against the risks to other people. IRB isn't (meant to be) an adversarial process where people try to stop you doing things, it's a second set of thinking about the appropriate balance between risks and benefits of research.
I believe that you (we!) are doing this for the right reasons, but I find it a bad policy to trust anyone who thinks they're doing good things for good reasons. That way lies Jack Bauer. :)
I think that the censor would have a pretty hard job proving in a just court of law that such user was engaging in censorship measurements (assuming they consider censorship measurements to be an illegal thing). Unfortunately in some countries were we measure the courts of law are not just and we have to make all sorts of crazy assumptions on how they will interpret what we are doing. Using routers instead of real users when doing the scans could be a safer move if it does not affect your measurement.
I know that you qualify it in the second sentence, but 'proving something in a just court of law' isn't even worth mentioning in the specific field we're talking about.
I definitely think that using routers is a great idea if it can be managed, as is using alternative services where possible and trying to locate probes in organizations' networks rather than personal users' ones. The niggle I have is 'if it does not affect your measurement'. I really think that it should be 'if the balance is right between the effect on the measurement, and the risk to the user'.
I'm really happy to see these discussions happening here, and I hope that nothing above came across as an attack -- I think you're fighting the good fight. Roya has been doing some amazing work in this field, and I think there's huge potential for combining ooniprobe-ish data sources with others to maximize the 'impact' of what comes out of all these filtering measurement projects.
My wider point is that impact requires analysis as well as data, but we can have that discussion later. :)
All the best,
Joss
Hi list,
It is more fair to consider OONI as an end-user network measurements tool running voluntarily and it should not be compared directly with various network measurements projects. Some relying on closed source software, non documented/hidden methodologies or could potential disclose user information to law enforcement.
There is nothing really illegal or improper (except ISP limitations/rules) that disallows a user to conduct network related measurements on it's own Internet connection. The user can decide the risks associated while running OONI, pretty much as in any other application for instance nmap and other security/port scanners.
Hi all,
It's great to see a lively discussion around ethics.
I would suggest reaching out to or CCIng the IRC Ethics mailing list ("icr-ethics-l: ethics in information control research" ICR-ETHICS-L@listserv.utoronto.ca) .
This is an effort led out of Oxford Internet Institute and Citizen Lab, looking at the various ethical challenges raised by networked systems, measurement particularly. They've been doing good, careful work analyzing complex issues such as these, and I'm sure they would be happy to consider this issue.
Cheers, Meredith
On Mon, Jan 12, 2015 at 9:37 PM, jony-port@bitmessage.ch wrote:
Hi list,
It is more fair to consider OONI as an end-user network measurements tool running voluntarily and it should not be compared directly with various network measurements projects. Some relying on closed source software, non documented/hidden methodologies or could potential disclose user information to law enforcement.
There is nothing really illegal or improper (except ISP limitations/rules) that disallows a user to conduct network related measurements on it's own Internet connection. The user can decide the risks associated while running OONI, pretty much as in any other application for instance nmap and other security/port scanners.
ooni-dev mailing list ooni-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/ooni-dev
On Mon, Jan 05, 2015 at 02:27:47PM +0100, Arturo Filastò wrote:
Your research is very interesting and I believe very important for getting more data when it would just be too risky to have network vantage. I do think, though, that we can't rely only on these sorts of measurements. They complement and extend what we measure from the network vantage point of the user, but may not work as reliably in all censorship systems and only give you a subset of the information we are interested in acquiring.
I think that Roya was advocating, like you suggest, for idle scans to complement OONI's tests. While idle scans are limited to the network and transport layer, many of OONI's tests are on the application layer.
As for integrating the idle scan code into OONI: I worked a little bit with Roya on this project and have some understanding of the code. Integrating it would involve quite a bit of work but is not impossible. Idle scans also require an IP spoofing-capable machine, which can be difficult to organise since most ISPs do egress filtering.
Cheers, Philipp
-
Arturo Filastò -
Collin Anderson -
Dan O'Huiginn -
Jacob Appelbaum -
jony-port＠bitmessage.ch -
Joss Wright -
Meredith Whittaker -
Philipp Winter -
royaen