As an experiment to wring more performance from the snowflake bridge, I started a rough port of the extor-static-cookie shim to Rust. It's not in what I would call a production-ready state with regard to documentation and tests—I wanted to hack something together quickly in order to decide whether it's worth spending more time on. I am also pretty new to Rust async programming. As a side consideration, I wanted to get an idea of what a pluggable transports library for Rust might look like.
https://gitlab.torproject.org/dcf/extor-static-cookie/-/tree/rust Currently commit 559fb5e292b45d82464bdaf978c6e1783c796e3d
I have the Rust implementation running now (since 2022-02-27 03:00) under the flakey4 instance. The other three instances are running the Go implementation. It's too early to say whether it reduces CPU usage; an hour after being restarted, the flakey4 instance is still catching up to the traffic of the other instances. But it looks like it's stable and functional, at least.
After about a day, the instance I restarted has caught up and is processing about as much traffic as the others.
1. The Rust implementation of extor-static-cookie is a little bit more CPU-efficient, but only a little. 2. The Rust implementation uses about 50% more memory, which is still negligible relative to other processes on the server.
I didn't run any controlled benchmarks, just watched the numbers in htop for a while. The three instances of Go implementation of extor-static-cookie hover between 10–12% CPU time and 30 MB of resident memory. The Rust implementation uses about 8–10% CPU time and 48 MB of resident memory.
That there's not a big different is not much of a surprise, given that both programs are essentially copy loops. I didn't post the results when I did it, but when I profiled the Go implementation 3 weeks ago, it spent most of its time in the splice system call, which "moves data between two file descriptors without copying between kernel address space and user address space."
My conclusion is that a Rust version of extor-static-cookie does not have a strong enough benefit to be worth spending much time on.
On Sat, Feb 26, 2022 at 09:04:32PM -0700, David Fifield wrote:
As an experiment to wring more performance from the snowflake bridge, I started a rough port of the extor-static-cookie shim to Rust. It's not in what I would call a production-ready state with regard to documentation and tests—I wanted to hack something together quickly in order to decide whether it's worth spending more time on. I am also pretty new to Rust async programming. As a side consideration, I wanted to get an idea of what a pluggable transports library for Rust might look like.
https://gitlab.torproject.org/dcf/extor-static-cookie/-/tree/rust Currently commit 559fb5e292b45d82464bdaf978c6e1783c796e3d
I have the Rust implementation running now (since 2022-02-27 03:00) under the flakey4 instance. The other three instances are running the Go implementation. It's too early to say whether it reduces CPU usage; an hour after being restarted, the flakey4 instance is still catching up to the traffic of the other instances. But it looks like it's stable and functional, at least.
Quoting David Fifield (2022-02-28 03:58:38)
After about a day, the instance I restarted has caught up and is processing about as much traffic as the others.
- The Rust implementation of extor-static-cookie is a little bit more CPU-efficient, but only a little.
- The Rust implementation uses about 50% more memory, which is still negligible relative to other processes on the server.
I didn't run any controlled benchmarks, just watched the numbers in htop for a while. The three instances of Go implementation of extor-static-cookie hover between 10–12% CPU time and 30 MB of resident memory. The Rust implementation uses about 8–10% CPU time and 48 MB of resident memory.
That there's not a big different is not much of a surprise, given that both programs are essentially copy loops. I didn't post the results when I did it, but when I profiled the Go implementation 3 weeks ago, it spent most of its time in the splice system call, which "moves data between two file descriptors without copying between kernel address space and user address space."
My conclusion is that a Rust version of extor-static-cookie does not have a strong enough benefit to be worth spending much time on.
Nice experiment anyway, will be nice base to take ideas from if we get to write rust PTs in the future (wich might make sense to integrate them into arti).
anti-censorship-team@lists.torproject.org
-
David Fifield -
meskio