Hello,
OONI's May 2022 status report is shared below.
*# OONI Monthly Report: May 2022*
Throughout May 2022, the OONI team worked on the following sprints:
* Sprint 64 (1st - 8th May 2022) * Sprint 65 (9th - 22nd May 2022) * Sprint 66 (23rd - 31st May 2022)
Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni
Highlights are shared in this report below.
*## Job opening for OONI Community Coordinator*
In May 2022, we published a job opening for hiring a Community Coordinator for OONI: https://ooni.org/post/2022-job-opening-ooni-community-coordinator/
In an effort to help ensure that our job opening reached interested candidates around the world, we worked on quite extensive outreach efforts. In particular, we posted the job opening on: * All OONI social media channels (Twitter, Mastodon, Facebook, Instagram, Linkedin); * All OONI mailing lists; * Community mailing lists (such as those of the #KeepItOn campaign, Global Voices Advocacy, and NetRights); * OONI Slack channel (and other relevant Slack channels); * Digital Rights Job Board (https://www.digitalrights.community/job-board); * Hasgeek Job Board (https://hasjob.co/openobservatory.org/12jm2); * FOSS Job Board ( https://www.fossjobs.net/job/10936/ooni-community-coordinator-at-open-observ... ).
We also reached out to our partners and many other community members, requesting that they help share this opportunity with their networks.
Following the publication of this job opening, we worked on tracking and reviewing incoming applications, and following-up with applicants.
*## OONI Probe Mobile*
Throughout May 2022, we worked on various improvements to the OONI Probe mobile app in preparation for the next release.
Specifically, we worked on: * Including the Vanilla Tor experiment as part of the Experimental card of OONI Probe (https://github.com/ooni/probe/issues/2103); * Ensuring that the Vanilla Tor and Tor Snowflake experiments only run as part of automated testing (https://github.com/ooni/probe/issues/2102); * Providing a working sample for Flutter on OONI Probe apps with probe-cli integration (https://github.com/aanorbel/probe-shared); * Enabling users to see all logs (https://github.com/ooni/probe/issues/1777 , https://github.com/ooni/probe-android/pull/516); * Fixing an iOS bug affecting the icons displayed when all tests are run ( https://github.com/ooni/probe/issues/2055).
*## OONI Run*
We implemented a fix related to the copy-pasting of URLs into the OONI Run web interface (https://github.com/ooni/run/pull/105). Based on community feedback, we explored how we can add support to OONI Run for saving custom test lists (https://github.com/ooni/probe/issues/867), and we had team meetings to discuss possible implementation approaches. We also wrote an internal design document on implementation approaches for addressing the most pressing OONI Run community needs (as identified through a previous OONI Run usability study and ongoing community interactions).
*## OONI Probe Desktop*
We made some improvement to the description of the Experimental card in OONI Probe Desktop (https://github.com/ooni/probe-desktop/pull/294) and we worked towards ensuring that the autorun directory is deleted when a hard reset is performed (https://github.com/ooni/probe-desktop/pull/295).
*## OONI Probe Web*
We added support to OONI Probe Web for using the check-in API to retrieve information about the probe IP address country code and ASN ( https://github.com/ooni/probe-web/commit/bc680323df0c7404b14651acea11a281c73...). We also worked on improving the UI of OONI Probe Web.
We circulated an internal testing beta of OONI Probe Web ( https://ooni.github.io/probe-web/) to collect community feedback that can help inform further development and improvements.
*## Expanding censorship measurement methodologies### Website measurement*
After some initial experimentation in April 2022, we sketched out a plan for integrating some methodologies from the “Understanding the Practices of Global Censorship through Accurate, End-to-End Measurements” ( https://shhaos.github.io/papers/sigmetrics22.pdf) study into OONI Probe ( https://github.com/ooni/probe/issues/2086).
We started implementing the plan for merging the Websteps prototype ( https://github.com/bassosimone/websteps-illustrated/) into OONI Probe. To this end, we merged several improvements for netxlite, the ooniprobe package that provides basic networking functionality ( https://github.com/ooni/probe/issues/2096).
We also drafted and implemented a plan for observing duplicate DNS replies ( https://github.com/ooni/probe/issues/2099). This plan differs from the Websteps implementation because community feedback (in particular from Jigsaw’s Vinicius Fortuna and Sarah Laplante) convinced us that the implementation of this functionality that we had sketched out in the Websteps prototype could be further simplified and streamlined.
*### New experiments*
We implemented several new exploratory experiments: * dnsping: https://github.com/ooni/spec/blob/master/nettests/ts-035-dnsping.md * simplequicping: https://github.com/ooni/spec/blob/master/nettests/ts-034-simplequicping.md * tcpping: https://github.com/ooni/spec/blob/master/nettests/ts-032-tcpping.md * tlsping: https://github.com/ooni/spec/blob/master/nettests/ts-033-tlsping.md
These are “exploratory” in the sense that we expect users to manually run them from the command line when they are actively researching censorship. We introduced these four new experiments based on community feedback and requests.
*### Vanilla Tor experiment*
We implemented the Vanilla Tor experiment ( https://github.com/ooni/probe-cli/pull/718), which used to be part of the legacy version of OONI Probe. We reintroduced this experiment following discussions with Tor developers, who mentioned that it would be quite useful to them.
*### Data quality improvements*
We investigated and fixed a rather surprising and complex data quality issue. It turns out that the Android implementation of `getaddrinfo` behaves differently from any other existing `getaddrinfo` implementation and only returns one of two return codes: 0 in case of success, and `EAI_NODATA` in case any other possible error occurs, including cases where a domain name does not exists, cases where the server is temporarily unavailable, cases where the server is misconfigured, and cases of censorship.
In practical terms, this means that the result of `getaddrinfo` on Android is unreliable and we need to improve our heuristics on the backend and our algorithms on the probe to work around this. The pull request where we fixed this bug refers to our investigation, as well as to our follow up plans to further improve DNS measurements on Android ( https://github.com/ooni/probe-cli/pull/764). The reference issue describing this problem (https://github.com/ooni/probe/issues/2029) is still open because we have not included the fix into the 3.15.x release cycle (which was being finalized while we were still investigating the root cause of this surprising bug).
*## OONI Explorer*
We fixed a couple of OONI Explorer bugs. Specifically, we added support for searching URLs with a longer TLD (https://github.com/ooni/explorer/pull/757) and we updated the copyright year in the footer ( https://github.com/ooni/explorer/pull/756).
*## Building a web platform for updating test lists*
Based on community feedback on the private beta version of our new test list web platform (https://test-lists.ooni.org/), we added backend support ( https://github.com/ooni/api/pull/286) to enable users to make changes to open pull requests (once a pull request has been opened through a test list web platform submission).
In anticipation of a potentially increased volume of test list pull requests following the upcoming public launch of the test list web platform, we (along with the Citizen Lab) invited Censored Planet and Netalitica team members to get added to the Citizen Lab test list repository as reviewers of pull requests.
*## OONI backend*
*### Migration to the Clickhouse database*
We continued to work on the ongoing migration to the new OLAP database. In May 2022, we migrated 12 API entry points to the new backend host.
*### Reprocessing legacy measurements*
We continued to work on the reprocessing of measurements collected before 2021 to improve data quality and ensure all measurements are reachable from OONI Explorer.
*### OONI API*
We reviewed and deployed a change to support the detection of the ASN and country code of probes based on their IP address on the API side. This will allow the removal of bulky GeoIP database files from the probes. We also improved the account_metadata API call to display login status on web UIs ( https://github.com/ooni/api/pull/292).
*### Other operations*
We added blackbox exporter-based testing and support for the alarming of various API entry points. We finished moving the test helper rotation script to the new backend host (https://github.com/ooni/pipeline/pull/386).
*## Test list updates*
In May 2022, we contributed a few updates to the Citizen Lab test lists. In particular, we updated the US test list ( https://github.com/citizenlab/test-lists/pull/984) and we removed dead URLs from the Global test list (https://github.com/citizenlab/test-lists/pull/995).
*## Notable community use of OONI Probe and OONI data### Sinar Project report*
Malaysia’s Sinar Project published a report on the blocking of a NFT fundraising campaign website (of graphics designer and political activist, Fahmi Reza): https://sinarproject.org/digital-rights/measuring-and-detecting-network-inte...
Their report makes use of OONI Probe and OONI data.
*## Community activities### OONI workshop for civil society groups in Latin America*
On 19th May 2022, OONI’s Maria facilitated a 2-hour OONI workshop for civil society groups in Latin America. This workshop (organized by Derechos Digitales) was facilitated in collaboration with Venezuela Inteligente.
*### OONI presentation at Tactical Tech*
On 24th May 2022, OONI’s Maria presented OONI at Tactical Tech ( https://tacticaltech.org/) in Berlin. This presentation also involved a live demo of using the Measurement Aggregation Toolkit (MAT) to investigate various forms of internet censorship around the world.
*### OONI presentation for journalists*
On 25th May 2022, OONI’s Maria presented OONI’s tools, methods, research, and dataset to a group of journalists from Russia and Belarus.
*### OONI Community Meeting*
On 31st May 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:
1) Job Opening: OONI Community Coordinator ( https://ooni.org/post/2022-job-opening-ooni-community-coordinator/)
2) Detecting regional censorship, which is increasingly becoming an urgent issue (https://github.com/ooni/probe/issues/1581#issuecomment-1125221383)
In particular, we discussed the following questions with the community:
* Should there be an option to also include the region or should we include the region by default? * Should we invest some legal research time to figure out if we could use MaxMind? * Should we try to have location permission in the app and use that? Should we have two apps, one with location and the other without? * Should we allow users to manually annotate/correct the geolocation results if they are inaccurate?
3) Copyright law for enforcing internet censorship
*## Userbase*
In May 2022, 41,882,898 OONI Probe measurements were collected from 2,905 AS networks in 171 countries around the world.
This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/
~ OONI team.
ooni-talk@lists.torproject.org