(Moving this discussion back to tor-dev@ with Charlie's permission.)
On 8/6/13 4:10 AM, Charlie Belmer wrote:
Hey Karsten,
I've been reading up on Shadow & Scallion to do some performance testing on N23. Before I go too far down that road, I wanted to check in with you.
Are there existing blueprints or best practices I should be following?
Thanks for looking into this!
So, I wouldn't call the following notes blueprints or best practices, but they are a description of what I would do to run Shadow simulations of your branch.
First of all, be sure to read the Shadow wiki which has contains lots of interesting stuff: https://github.com/shadow/shadow/wiki
The next step would be to get a vanilla Tor master (that your branch is based on) running in a minimal or tiny Shadow network. It's probably easiest to use a local Ubuntu VM for this. Or you can start an m1.large EC2 instance for the tiny network, or an even smaller instance for the minimal network.
Once that simulation succeeds, you'll want to run another simulation using your branch, extract performance metrics using Shadow's analyze script, produce a PDF with graphs also using the analyze script, and then wonder if things are faster than before.
The minimal or tiny networks are mostly there to learn that things are working as expected. You'll probably want larger networks for real performance results, but starting small to get everything working is probably easier.
Sounds doable, I guess. However, here's where things start to get tricky: in order to simulate Tor master (or a branch based on it) in Shadow, you'll have to perform quite a few extra steps like compiling your own CMake and Clang/LLVM. Not really hard, but you need to get all pieces together before Shadow does what you want.
I'm pasting detailed instructions that I used to simulate Steven's uTP branch in Shadow below as an example. Not everything applies to your branch, but it should give you the general idea.
If something remains unclear, please feel free to ask!
Depending how it goes, I'd also look at re-trying adaptive 23 or tuning the existing implementation.
Neat! We should probably include Mashael in that discussion once you have a working and simulated n23-5 branch. Maybe she has ideas what parts need tweaking. I'm cc'ing her in this mail so she knows what you're working on.
In any case, having a working rebased n23-5 branch is already a big step forward.
Thanks! Karsten
=== START OF INSTRUCTIONS ===
Launch new EC2 instance running Ubuntu Server 13.04 64 bit
Set ulimit -n to 25000, see Shadow wiki for details https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs-...
$ sudo apt-get update $ sudo apt-get upgrade
Clone and test-compile modified libutp
$ sudo apt-get install build-essential git automake libssl-dev libevent-dev $ mkdir src $ cd src/ $ git clone https://github.com/sjmurdoch/libutp $ cd libutp/ $ make $ cd ../
Clone and test-compile Tor branch utp
$ git clone https://git.torproject.org/tor.git $ cd tor/ $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git $ git fetch sjm217 $ git checkout -b utp sjm217/utp $ ./autogen.sh $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp" LIBS="-lutp -lrt" ./configure --disable-asciidoc $ make $ make distclean $ cd ../
Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any link)
diff --git a/src/or/channeltls.c b/src/or/channeltls.c index 0551b73..b7b36e1 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t port, /* Create a uTP connection */ tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin)); tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0); - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { log_info(LD_CHANNEL, "Trying uTP connection to %s", addr_str); tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct sockaddr*)&sin,
Install CMake 2.8.10
$ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz $ tar xf cmake-2.8.10.2.tar.gz $ cd cmake-2.8.10.2/ $ ./configure $ make $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH $ cd ../ $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake
Build Clang and LLVM from source
$ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz $ tar xaf llvm-3.2.src.tar.gz $ tar xaf clang-3.2.src.tar.gz $ cp -R clang-3.2.src llvm-3.2.src/tools/clang $ cd llvm-3.2.src/ $ mkdir build $ cd build/ $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. $ make $ make install $ export PATH=~/.local/bin/:$PATH $ cd ../../ $ which clang # result: /home/ubuntu/.local/bin//clang
Install Shadow
$ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 python-matplotlib python-numpy python-scipy $ git clone https://github.com/shadow/shadow.git $ cd shadow $ ./setup dependencies $ ./setup build -g -i /home/ubuntu/src/libutp -i /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp --tor-prefix /home/ubuntu/src/tor --tor-lib utp $ ./setup install $ export PATH=~/.shadow/bin/:$PATH
Simulate tiny network in Shadow
$ cd resource/examples/scallion/ $ tar xf tiny-4GB-m1.large.tar.xz $ cd tiny-4GB-m1.large/ $ scallion
Graph simulation results
Add one line to beginning of analyze.py:
import matplotlib; matplotlib.use('Agg')
For additional traffic simulations:
$ grep -v "[traffic-" scallion.log > scallion-notraffic.log
$ cd ~/src/ $ mkdir results $ cd results/ $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output vanilla ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for none of the links"
=== END OF INSTRUCTIONS ===
Hi Charlie,
did you have the chance to simulate your rebased n23-5 branch using Shadow?
If not, maybe I can help by simulating it once and writing down more specific instructions for you?
Thanks! Karsten
On 8/8/13 10:36 AM, Karsten Loesing wrote:
(Moving this discussion back to tor-dev@ with Charlie's permission.)
On 8/6/13 4:10 AM, Charlie Belmer wrote:
Hey Karsten,
I've been reading up on Shadow & Scallion to do some performance testing on N23. Before I go too far down that road, I wanted to check in with you.
Are there existing blueprints or best practices I should be following?
Thanks for looking into this!
So, I wouldn't call the following notes blueprints or best practices, but they are a description of what I would do to run Shadow simulations of your branch.
First of all, be sure to read the Shadow wiki which has contains lots of interesting stuff: https://github.com/shadow/shadow/wiki
The next step would be to get a vanilla Tor master (that your branch is based on) running in a minimal or tiny Shadow network. It's probably easiest to use a local Ubuntu VM for this. Or you can start an m1.large EC2 instance for the tiny network, or an even smaller instance for the minimal network.
Once that simulation succeeds, you'll want to run another simulation using your branch, extract performance metrics using Shadow's analyze script, produce a PDF with graphs also using the analyze script, and then wonder if things are faster than before.
The minimal or tiny networks are mostly there to learn that things are working as expected. You'll probably want larger networks for real performance results, but starting small to get everything working is probably easier.
Sounds doable, I guess. However, here's where things start to get tricky: in order to simulate Tor master (or a branch based on it) in Shadow, you'll have to perform quite a few extra steps like compiling your own CMake and Clang/LLVM. Not really hard, but you need to get all pieces together before Shadow does what you want.
I'm pasting detailed instructions that I used to simulate Steven's uTP branch in Shadow below as an example. Not everything applies to your branch, but it should give you the general idea.
If something remains unclear, please feel free to ask!
Depending how it goes, I'd also look at re-trying adaptive 23 or tuning the existing implementation.
Neat! We should probably include Mashael in that discussion once you have a working and simulated n23-5 branch. Maybe she has ideas what parts need tweaking. I'm cc'ing her in this mail so she knows what you're working on.
In any case, having a working rebased n23-5 branch is already a big step forward.
Thanks! Karsten
=== START OF INSTRUCTIONS ===
Launch new EC2 instance running Ubuntu Server 13.04 64 bit
Set ulimit -n to 25000, see Shadow wiki for details https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs-...
$ sudo apt-get update $ sudo apt-get upgrade
Clone and test-compile modified libutp
$ sudo apt-get install build-essential git automake libssl-dev libevent-dev $ mkdir src $ cd src/ $ git clone https://github.com/sjmurdoch/libutp $ cd libutp/ $ make $ cd ../
Clone and test-compile Tor branch utp
$ git clone https://git.torproject.org/tor.git $ cd tor/ $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git $ git fetch sjm217 $ git checkout -b utp sjm217/utp $ ./autogen.sh $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp" LIBS="-lutp -lrt" ./configure --disable-asciidoc $ make $ make distclean $ cd ../
Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any link)
diff --git a/src/or/channeltls.c b/src/or/channeltls.c index 0551b73..b7b36e1 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t port, /* Create a uTP connection */ tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin)); tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0);
- if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) {
- if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { log_info(LD_CHANNEL, "Trying uTP connection to %s", addr_str); tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct
sockaddr*)&sin,
Install CMake 2.8.10
$ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz $ tar xf cmake-2.8.10.2.tar.gz $ cd cmake-2.8.10.2/ $ ./configure $ make $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH $ cd ../ $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake
Build Clang and LLVM from source
$ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz $ tar xaf llvm-3.2.src.tar.gz $ tar xaf clang-3.2.src.tar.gz $ cp -R clang-3.2.src llvm-3.2.src/tools/clang $ cd llvm-3.2.src/ $ mkdir build $ cd build/ $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. $ make $ make install $ export PATH=~/.local/bin/:$PATH $ cd ../../ $ which clang # result: /home/ubuntu/.local/bin//clang
Install Shadow
$ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 python-matplotlib python-numpy python-scipy $ git clone https://github.com/shadow/shadow.git $ cd shadow $ ./setup dependencies $ ./setup build -g -i /home/ubuntu/src/libutp -i /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp --tor-prefix /home/ubuntu/src/tor --tor-lib utp $ ./setup install $ export PATH=~/.shadow/bin/:$PATH
Simulate tiny network in Shadow
$ cd resource/examples/scallion/ $ tar xf tiny-4GB-m1.large.tar.xz $ cd tiny-4GB-m1.large/ $ scallion
Graph simulation results
Add one line to beginning of analyze.py:
import matplotlib; matplotlib.use('Agg')
For additional traffic simulations:
$ grep -v "[traffic-" scallion.log > scallion-notraffic.log
$ cd ~/src/ $ mkdir results $ cd results/ $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output vanilla ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for none of the links"
=== END OF INSTRUCTIONS ===
-
Karsten Loesing