On Mon, Jan 22, 2024 at 04:33:53PM +0100, ERIK NORDBERG wrote:
> This is a great report.
> I would like to post it as it is as ’2023 year in review' on https://
> opencollective.com/censorship-circumvention/projects/snowflake-daily-operat…
> /updates.
> No need to strip it to cover only snowflake-01, as I see it.
> OK?
Yes, that's OK. Here's the slightly edited text I posted.
---
This is a summary of usage of the [Snowflake](https://snowflake.torproject.org/) pluggable transport in 2023. For the previous year's summary, see the [Snowflake 2022 year in review](https://opencollective.com/censorship-circumvention/projects/snowfl….
The primary Snowflake bridge, called snowflake-01, is supported by donations. In 2023, the [Snowflake Daily Operations team](https://opencollective.com/censorship-circumvention/projects/snowflak… paid about [€4,820](https://opencollective.com/censorship-circumvention/projects/snowflake-daily-operations/expenses?collectiveType=projects&parentCollectiveSlug=censorship-circumvention&collectiveSlug=snowflake-daily-operations&period=2023-01-01T00%3A00%3A00.000Z%E2%86%922023-12-31T23%3A59%3A59.999Z%7EUTC) in hosting expenses to keep the bridge running—with most of that going for bandwidth. In 2024, in addition to paying for bandwidth, we would like to buy new hardware for the bridge, which will enable it to support more users at the busiest times of day. You can help by [donating to support the bridge](https://opencollective.com/censorship-circumvention/projects/snowfl….
Let's start by looking at the daily users and bandwidth. Tor user graphs show an estimate of the [average number of simultaneously connected users](https://metrics.torproject.org/reproducible-metrics.html#users) (not unique IP addresses). For instance, if the line passes through 40,000 on a particular day, it means there were, on average, 40,000 users connected to Snowflake at any given time during that day. The bandwidth graph shows the total amount of Tor traffic transferred per day. As of January 1, 2024, Snowflake supported an average of about 40,000 simultaneous users and transferred about 35 TB of circumvention traffic per day. While the number of users is lower than it was at the beginning of the year, bandwidth is up by more than 50%.
[snowflake-userstats-bridge-transport-2023.png]
[snowflake-bandwidth-2023.png]
These are some of the events that affected Snowflake usage in 2023:
* Between 2023-01-16 and 2023-01-24, and intermittently thereafter, domain fronting rendezvous was [blocked in parts of Iran](https://bugs.torproject.org/tpo/anti-censorship/team/115). As more Snowflake users are in Iran than in any other country, this resulted in an observable decrease in the number of users.
* On 2023-03-13, the Tor anti-censorship team [fixed a security bug](https://forum.torproject.org/t/security-advisory-cross-user-tls-traffi… that had also negatively affected performance since September 2022. This greatly improved bandwidth for all users.
* The big drop in users on 2023-09-20 was not caused by a censor. Rather, it was an [unexpected change](https://forum.torproject.org/t/problems-with-snowflake-since-2023-0… in the infrastructure used by Snowflake for rendezvous. Mitigating the problem required new software releases.
Before 2023, we had used just one, centralized Snowflake bridge. Last year we started using [a second bridge](https://bugs.torproject.org/tpo/anti-censorship/pluggable-transport… in order to scale to more users. Here are the same user and bandwidth graphs, but with the snowflake-01 and snowflake-02 bridges shown separately. Strangely, the domain front change of 2023-09-20 [affected the two bridges differently](https://lists.torproject.org/pipermail/anti-censorship-team/20….
[snowflake-userstats-bridge-transport-multi-2023.png]
[snowflake-bandwidth-multi-2023.png]
We can also break down users by country. The largest contingent of Snowflake users are in Iran, which has been the case since [the Mahsa Amini protests in 2022](https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-prote…. The graph shows also a large number of users apparently from the United States, but we believe that may be partly the result of geolocation errors, and many of them are [actually from Iran](https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/…. After Iran, the countries with the most Snowflake users are Russia and China. On 2023-04-10 the team [fixed a bug](https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/s… in the browser proxy that caused some users to be wrongly attributed to the unknown "??" country code. The cause of the per-country user count estimates becoming less precise between October and December was a [user-counting bug](https://bugs.torproject.org/tpo/core/tor/40871) in Tor that existed from version 0.4.8.4 until 0.4.8.10.
[snowflake-top-countries-2023.png]
Hi,
I started asking questions in #tor-anticensorship about how traffic can
be directed towards certain snowflake bridges. Moving my questions here
to reach more/other people.
--8<---------------cut here---------------start------------->8---
Hey all. In trying to make a 2024 budget for the [Snowflake Operations][]
project operating snowflake-01.tpn I need a better understanding of how
we direct traffic to the running bridges. Both what potential challenges
there are to do it and what the policy for it looks like. The background
is that snowflake-01 is close to going full due to CPU consumption. I
haven't spotted any flat lines yet but have seen momentary CPU
utilisation of 98% a couple of times.
Here are two of the questions I'm looking for an answer to.
1. If we get another server, similar to snowflake-01 wrt performance,
will it be useful to the network? Ie will it offload snowflake-01?
2. If we do **not** get another server and snowflake-01 goes full, will
users have a bad network experience as a result of this? Can traffic be
moved to snowflake-02?
[Snowflake Operations]: https://opencollective.com/censorship-circumvention/projects/snowflake-dail…
--8<---------------cut here---------------end--------------->8---
I've drafted a year-in-review post for Snowflake users in 2023 that
doubles as a request for donations for running the bridge. I'm planning
to post it to the Tor forum.
Is there anything to add?
Erik: is the quoted hosting costs of €4,800 correct?
The post discusses both existing bridges. We could also do one
specifically for snowflake-01 to post to the Open Collective.
---
This is a summary of usage of the
[Snowflake](https://snowflake.torproject.org/) pluggable transport in
2023. For the previous year's summary, see the
[Snowflake 2022 year in review](https://opencollective.com/censorship-circumvention/projects/snowfl….
The primary Snowflake bridge, called snowflake-01, is supported by
donations. In 2023, the
[Snowflake Daily Operations team](https://opencollective.com/censorship-circumvention/projects/snowflak…
spent about
[€4,800](https://opencollective.com/censorship-circumvention/projects/snowflake-daily-operations/expenses?collectiveType=projects&parentCollectiveSlug=censorship-circumvention&collectiveSlug=snowflake-daily-operations&period=2023-01-01T00%3A00%3A00.000Z%E2%86%922023-12-31T23%3A59%3A59.999Z%7EUTC)
to keep the bridge running—with most of that going to pay for bandwidth.
In 2024, we would like to buy new hardware for the bridge, which will
enable it to support more users at the busiest times of day. You can
help by
[donating to support the bridge](https://opencollective.com/censorship-circumvention/projects/snowfl….
[Donate funds for the snowflake-01 bridge](https://opencollective.com/censorship-circumvention/projects/snowfl…
Let's start by looking at the daily users and bandwidth. Tor user graphs
show an estimate of the
[average number of simultaneously connected users](https://metrics.torproject.org/reproducible-metrics.html#users).
For instance, if the line passes through 50,000, it means there were, on
average, 50,000 users using Snowflake at the same time on that day. The
bandwidth graph shows the total amount of Tor traffic transferred per
day.
While there were fewer users at the end than at the beginning of the
year, Snowflake users are using greater total bandwith, now about 35 TB
per day.
[snowflake-userstats-bridge-transport-2023.png]
[snowflake-bandwidth-2023.png]
These are some of the events that affected Snowflake usage in 2023:
* Between 2023-01-16 and 2023-01-24, and intermittently thereafter,
domain fronting rendezvous was
[blocked in parts of Iran](https://bugs.torproject.org/tpo/anti-censorship/team/115).
As more Snowflake users are in Iran than in any other country, this
resulted in an observable decrease in the number of users.
* On 2023-03-13, the Tor anti-censorship team
[fixed a security bug](https://forum.torproject.org/t/security-advisory-cross-user-tls-traffi…
that had also negatively affected performance since September 2022.
This greatly improved bandwidth for all users.
* The big drop in users on 2023-09-20 was not caused by a censor.
Rather, it was an
[unexpected change](https://forum.torproject.org/t/problems-with-snowflake-since-2023-0… in the infrastructure
used by Snowflake for rendezvous. Mitigating the problem required new
software releases.
Before 2023, we had used just one, centralized Snowflake bridge. Last
year we started using
[a second bridge](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transpo…
in order to scale to more users. Here are the same user and bandwidth
graphs, but with the snowflake-01 and snowflake-02 bridges shown
separately:
[snowflake-userstats-bridge-transport-multi-2023.png]
[snowflake-bandwidth-multi-2023.png]
We can also break down users by country. The largest contingent of
Snowflake users are in Iran, which has been the case since
[the Mahsa Amini protests in 2022](https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-prote….
The graph shows also a large number of users apparently from the United
States, but we believe that may be partly the result of geolocation
errors, and many of them are
[actually from Iran](https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/….
After Iran, the countries with the most Snowflake users are Russia and
China. On 2023-04-10 the team
[fixed a bug](https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/s…
in the browser proxy that caused some users to be wrongly attributed to
the unknown `??` country code. The reason for the per-country estimates
becoming less precise from October to December was a
[user-counting bug](https://bugs.torproject.org/tpo/core/tor/40871)
in Tor that existed from version 0.4.8.4 until 0.4.8.10.
[snowflake-top-countries-2023.png]