On 9/3/13 4:32 PM, Charlie Belmer wrote:
> Hi Karsten,
>
> I have run several tests, but I haven't been able to satisfy myself yet. I
> attached the results from my latest round of tests for your review. I am
> comparing the n23 branch (UseN23 1 set in each torrc) with the initial head
> I merged n23 with. It seems that for some reason, the n23 branch completes
> far fewer downloads, though I haven't yet found the cause, or found
> anything interesting in the logs.
>
> $ grep "fg-download-complete" m1.large-tor/data/scallion.log | wc -l
> 8827
> $ grep "fg-download-complete" m1.large-n23/scallion.log | wc -l
> 308
>
> I would definitely appreciate any thoughts, insights, or advice!

Interesting!  Can you re-run the n23 simulation on info or even debug
log level?  That might reveal some more information.  You'll do that by
running "scallion --log-level=INFO".  A tiny network should be
sufficient for that.  Watch out for filling disks.

If you want me to look at the scallion.log file, feel free to post it
somewhere, and I'll see if something looks unusual to me.

Thanks,
Karsten

> On Tue, Sep 3, 2013 at 8:58 AM, Karsten Loesing <karsten@torproject.org>wrote:
>
>> Hi Charlie,
>>
>> did you have the chance to simulate your rebased n23-5 branch using Shadow?
>>
>> If not, maybe I can help by simulating it once and writing down more
>> specific instructions for you?
>>
>> Thanks!
>> Karsten
>>
>>
>> On 8/8/13 10:36 AM, Karsten Loesing wrote:
>>> (Moving this discussion back to tor-dev@ with Charlie's permission.)
>>>
>>> On 8/6/13 4:10 AM, Charlie Belmer wrote:
>>>> Hey Karsten,
>>>>
>>>> I've been reading up on Shadow & Scallion to do some performance
>> testing on
>>>> N23. Before I go too far down that road, I wanted to check in with you.
>>>>
>>>> Are there existing blueprints or best practices I should be following?
>>>
>>> Thanks for looking into this!
>>>
>>> So, I wouldn't call the following notes blueprints or best practices,
>>> but they are a description of what I would do to run Shadow simulations
>>> of your branch.
>>>
>>> First of all, be sure to read the Shadow wiki which has contains lots of
>>> interesting stuff: https://github.com/shadow/shadow/wiki
>>>
>>> The next step would be to get a vanilla Tor master (that your branch is
>>> based on) running in a minimal or tiny Shadow network.  It's probably
>>> easiest to use a local Ubuntu VM for this.  Or you can start an m1.large
>>> EC2 instance for the tiny network, or an even smaller instance for the
>>> minimal network.
>>>
>>> Once that simulation succeeds, you'll want to run another simulation
>>> using your branch, extract performance metrics using Shadow's analyze
>>> script, produce a PDF with graphs also using the analyze script, and
>>> then wonder if things are faster than before.
>>>
>>> The minimal or tiny networks are mostly there to learn that things are
>>> working as expected.  You'll probably want larger networks for real
>>> performance results, but starting small to get everything working is
>>> probably easier.
>>>
>>> Sounds doable, I guess.  However, here's where things start to get
>>> tricky: in order to simulate Tor master (or a branch based on it) in
>>> Shadow, you'll have to perform quite a few extra steps like compiling
>>> your own CMake and Clang/LLVM.  Not really hard, but you need to get all
>>> pieces together before Shadow does what you want.
>>>
>>> I'm pasting detailed instructions that I used to simulate Steven's uTP
>>> branch in Shadow below as an example.  Not everything applies to your
>>> branch, but it should give you the general idea.
>>>
>>> If something remains unclear, please feel free to ask!
>>>
>>>> Depending how it goes, I'd also look at re-trying adaptive 23 or tuning
>> the
>>>> existing implementation.
>>>
>>> Neat!  We should probably include Mashael in that discussion once you
>>> have a working and simulated n23-5 branch.  Maybe she has ideas what
>>> parts need tweaking.  I'm cc'ing her in this mail so she knows what
>>> you're working on.
>>>
>>> In any case, having a working rebased n23-5 branch is already a big step
>>> forward.
>>>
>>> Thanks!
>>> Karsten
>>>
>>>
>>> === START OF INSTRUCTIONS ===
>>>
>>> Launch new EC2 instance running Ubuntu Server 13.04 64 bit
>>>
>>> Set ulimit -n to 25000, see Shadow wiki for details
>>>
>> https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs-and-limits
>>>
>>> $ sudo apt-get update
>>> $ sudo apt-get upgrade
>>>
>>> Clone and test-compile modified libutp
>>>
>>> $ sudo apt-get install build-essential git automake libssl-dev
>> libevent-dev
>>> $ mkdir src
>>> $ cd src/
>>> $ git clone https://github.com/sjmurdoch/libutp
>>> $ cd libutp/
>>> $ make
>>> $ cd ../
>>>
>>> Clone and test-compile Tor branch utp
>>>
>>> $ git clone https://git.torproject.org/tor.git
>>> $ cd tor/
>>> $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git
>>> $ git fetch sjm217
>>> $ git checkout -b utp sjm217/utp
>>> $ ./autogen.sh
>>> $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp"
>>> LIBS="-lutp -lrt" ./configure --disable-asciidoc
>>> $ make
>>> $ make distclean
>>> $ cd ../
>>>
>>> Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any
>>> link)
>>>
>>> diff --git a/src/or/channeltls.c b/src/or/channeltls.c
>>> index 0551b73..b7b36e1 100644
>>> --- a/src/or/channeltls.c
>>> +++ b/src/or/channeltls.c
>>> @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t
>>> port,
>>>    /* Create a uTP connection */
>>>    tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin));
>>>    tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0);
>>> -  if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) {
>>> +  if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) {
>>>      log_info(LD_CHANNEL,
>>>               "Trying uTP connection to %s", addr_str);
>>>      tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct
>>> sockaddr*)&sin,
>>>
>>> Install CMake 2.8.10
>>>
>>> $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz
>>> $ tar xf cmake-2.8.10.2.tar.gz
>>> $ cd cmake-2.8.10.2/
>>> $ ./configure
>>> $ make
>>> $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH
>>> $ cd ../
>>> $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake
>>>
>>> Build Clang and LLVM from source
>>>
>>> $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz
>>> $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz
>>> $ tar xaf llvm-3.2.src.tar.gz
>>> $ tar xaf clang-3.2.src.tar.gz
>>> $ cp -R clang-3.2.src llvm-3.2.src/tools/clang
>>> $ cd llvm-3.2.src/
>>> $ mkdir build
>>> $ cd build/
>>> $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../.
>>> $ make
>>> $ make install
>>> $ export PATH=~/.local/bin/:$PATH
>>> $ cd ../../
>>> $ which clang # result: /home/ubuntu/.local/bin//clang
>>>
>>> Install Shadow
>>>
>>> $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy
>>> libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7
>>> python-matplotlib python-numpy python-scipy
>>> $ git clone https://github.com/shadow/shadow.git
>>> $ cd shadow
>>> $ ./setup dependencies
>>> $ ./setup build -g -i /home/ubuntu/src/libutp -i
>>> /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp
>>> --tor-prefix /home/ubuntu/src/tor --tor-lib utp
>>> $ ./setup install
>>> $ export PATH=~/.shadow/bin/:$PATH
>>>
>>> Simulate tiny network in Shadow
>>>
>>> $ cd resource/examples/scallion/
>>> $ tar xf tiny-4GB-m1.large.tar.xz
>>> $ cd tiny-4GB-m1.large/
>>> $ scallion
>>>
>>> Graph simulation results
>>>
>>> Add one line to beginning of analyze.py:
>>>
>>> import matplotlib; matplotlib.use('Agg')
>>>
>>> For additional traffic simulations:
>>>
>>> $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log
>>>
>>> $ cd ~/src/
>>> $ mkdir results
>>> $ cd results/
>>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1
>>>
>> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log
>>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0
>>>
>> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log
>>> $ python ~/src/shadow/contrib/analyze.py parse --output vanilla
>>>
>> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log
>>> $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp
>>> branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla
>>> 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for
>>> none of the links"
>>>
>>> === END OF INSTRUCTIONS ===
>>>
>>>
>>
>>
>