I did a build of tor-browser-build build and it took even longer than I remember it taking -- from Friday until Tuesday, roughly. This is on a VPS with six 2 GHz CPUs and 16 GB RAM. Some projects I know take a long time to build, like rust and firefox, but it seemed like most of the time was being spent in the "rbm" process, not in building the projects.
I don't know much about profiling Perl, but I tried a tool called NYTProf that produces HTML output: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/
The above NYTProf output is not from a full build. It's from modifying projects/snowflake/config only and then building snowflake for only one target; still it took almost an hour. perl -d:NYTProf rbm/rbm build snowflake --target testbuild --target torbrowser-linux-x86_64 nytprofhtml
To my untrained eye it looks like there is some kind of exponential recursion happening around the RBM::config_p function. See here, where in building one project the function was called 89 million times: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/#sub... Clicking through to the per-line view, we see that 45 million of those calls were from RBM::get_target, and most of the rest were from RBM::config: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/root...
You know how, if you write a Fibonacci number function and don't memoize the recursive calls, it ends up re-computing the same subtrees over and over and doing exponential work? def fib(n): if n == 0 or n == 1: return 1 return fib(n - 1) + fib(n - 2) It feels like something like that is happening here. There's a pattern where RBM::process_template calls a chain of functions that reaches RBM::config_p, which eventually calls RBM::process_template again. It's not an infinite loop, but it seems to be doing the same work many times. I looked at it for a bit but my Perl isn't strong enough to understand specifically what's happening.
Notice how there are three loops of RBM::process_template before doing an RBM::config_p that calls back into further RBM::process_template.
RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:10] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:9] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:18] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::config RBM::config_p RBM::DefaultConfig::__ANON__[rbm/lib/RBM/DefaultConfig.pm:606] RBM::input_files RBM::project_step_config RBM::project_config RBM::process_template Template::process ... et cetera ...
Some more evidence is that while I was waiting for rbm to do something, I was watching the modification times of the git_clones: ls -ltrhd git_clones/*/.git | tail -n 15 It was constantly touching all different projects, but re-touching the same ones over and over.
On Tue, Feb 11, 2020 at 11:45:26PM -0700, David Fifield wrote:
I did a build of tor-browser-build build and it took even longer than I remember it taking -- from Friday until Tuesday, roughly. This is on a VPS with six 2 GHz CPUs and 16 GB RAM. Some projects I know take a long time to build, like rust and firefox, but it seemed like most of the time was being spent in the "rbm" process, not in building the projects.
Hi David,
I've seen this too (although builds taking 2+ days is significantly worse than I've seen). I was going to talk with Nicolas about this after our current releases this week. I did some manual debugging on this and came to roughly the same conclusion, but I still don't know nearly enough about the internals of the system. I found (what seems like) rbm searching for the correct build script is very expense.
There is some duplicated work, as well, when dependencies are shared across projects (and platforms) - but that shouldn't account for much time overall.
- Matt
On Tue, 11 Feb 2020, David Fifield wrote:
I did a build of tor-browser-build build and it took even longer than I remember it taking -- from Friday until Tuesday, roughly. This is on a VPS with six 2 GHz CPUs and 16 GB RAM. Some projects I know take a long time to build, like rust and firefox, but it seemed like most of the time was being spent in the "rbm" process, not in building the projects.
I don't know much about profiling Perl, but I tried a tool called NYTProf that produces HTML output: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/
Thanks, I think this profile can help find a few things to improve.
The above NYTProf output is not from a full build. It's from modifying projects/snowflake/config only and then building snowflake for only one target; still it took almost an hour. perl -d:NYTProf rbm/rbm build snowflake --target testbuild --target torbrowser-linux-x86_64 nytprofhtml
Was it a new build of snowflake where some of the dependencies had to be built, or a build where only snowflake had to be built?
In the first case, I think part of the reason why it takes time is that snowflake has many small dependencies, and each one is built in a separate container, so some time is spent extracting/removing chroots.
If only snowflake and no other dependencies was built, then I don't know why it took almost one hour. In my case, adding a blank line to projects/snowflake/build and rebuilding it takes around 8 minutes.
To my untrained eye it looks like there is some kind of exponential recursion happening around the RBM::config_p function. See here, where in building one project the function was called 89 million times: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/#sub... Clicking through to the per-line view, we see that 45 million of those calls were from RBM::get_target, and most of the rest were from RBM::config: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/root...
You know how, if you write a Fibonacci number function and don't memoize the recursive calls, it ends up re-computing the same subtrees over and over and doing exponential work? def fib(n): if n == 0 or n == 1: return 1 return fib(n - 1) + fib(n - 2) It feels like something like that is happening here. There's a pattern where RBM::process_template calls a chain of functions that reaches RBM::config_p, which eventually calls RBM::process_template again. It's not an infinite loop, but it seems to be doing the same work many times. I looked at it for a bit but my Perl isn't strong enough to understand specifically what's happening.
Notice how there are three loops of RBM::process_template before doing an RBM::config_p that calls back into further RBM::process_template.
Yes, there is a lot of recursive calls.
For example the filename in projects/tor-browser/config is defined like this:
filename: 'tor-browser-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %]'
computing this filename will require computing var/build_id, which is defined like this:
build_id: '[% sha256(c("var/build_id_txt", { buildconf => { num_procs => 4 } })).substr(0, 6) %]' build_id_txt: | [% c("version") %] [% IF c("git_hash") || c("hg_hash"); GET c("abbrev"); END; %] [% IF c("var/container/use_container") && ! c("var/container/global_disable") -%] [% c("var/container/suite") %] [% c("var/container/arch") %] [% END -%] input_files: [% c("input_files_id") %] build: [% c("build", { filename => 'f', output_dir => '/out' }) %]
so this require recursively computing filenames of all other projects, which requires computing sha256sum of all input files.
Maybe there is some caching we can add to avoid computing some things multiple times. However in some cases we cannot avoid computing almost the same thing twice because we cannot know if an option can change the result, for example with the following two lines:
filename: 'name-[% c("version", { option_1 => 1 }) %]' filename: 'name-[% c("version", { option_1 => 2 }) %]'
Some more evidence is that while I was waiting for rbm to do something, I was watching the modification times of the git_clones: ls -ltrhd git_clones/*/.git | tail -n 15 It was constantly touching all different projects, but re-touching the same ones over and over.
I opened a ticket for this: https://trac.torproject.org/projects/tor/ticket/33283
I think this patch should avoid doing multiple `git checkout` and `git log` on the same projects: https://gitweb.torproject.org/user/boklm/rbm.git/commit/?h=bug_33283&id=...
When running this:
rbm/rbm showconf tor-browser filename --target torbrowser-linux-x86_64 --target alpha
in my case the patch improves the time to run this from 3m25 to 2m49.
An other possible improvement is in the function input_file_need_dl where we can avoid computing multiple times the sha256sum of the same files.
Nicolas
On Wed, Feb 12, 2020 at 02:40:38PM +0100, Nicolas Vigier wrote:
The above NYTProf output is not from a full build. It's from modifying projects/snowflake/config only and then building snowflake for only one target; still it took almost an hour. perl -d:NYTProf rbm/rbm build snowflake --target testbuild --target torbrowser-linux-x86_64 nytprofhtml
Was it a new build of snowflake where some of the dependencies had to be built, or a build where only snowflake had to be built?
In the first case, I think part of the reason why it takes time is that snowflake has many small dependencies, and each one is built in a separate container, so some time is spent extracting/removing chroots.
If only snowflake and no other dependencies was built, then I don't know why it took almost one hour. In my case, adding a blank line to projects/snowflake/build and rebuilding it takes around 8 minutes.
It was a build of snowflake only, none of the dependencies. In other words, I had already done a complete "make testbuild" (this is the one that took all weekend), then I modified projects/snowflake/config, then I did "rbm build snowflake" as shown above.
Overnight I did a "make testbuild" to build snowflake for the rest of the targets, and it took 9 hours: real 544m23.414s user 482m3.219s sys 73m29.899s I'll put the top-level stdout at the end of this message; I'm pretty sure it shows that only snowflake and tor-browser neede to be rebuilt, and the android targets didn't need to be rebuilt at all because they don't have snowflake as a dependency.
Notice how there are three loops of RBM::process_template before doing an RBM::config_p that calls back into further RBM::process_template.
Yes, there is a lot of recursive calls.
For example the filename in projects/tor-browser/config is defined like this:
filename: 'tor-browser-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %]'
computing this filename will require computing var/build_id, which is defined like this:
so this require recursively computing filenames of all other projects, which requires computing sha256sum of all input files.
Thank you for the explanation. It makes a lot of sense. I like this feature of rbm a lot, that I can feel confident the right things will be rebuilt when I change something.
I suppose Go projects are especially troublesome. In my build over the weekend I was testing a change that added about 10 new transitive dependencies to snowflake: https://lists.torproject.org/pipermail/anti-censorship-team/2020-February/00... https://gitweb.torproject.org/user/dcf/tor-browser-build.git/commit/?h=snowf... In my build last night (the one whose stdout is below), I removed all those dependencies again and added only 1 dependency: https://gitweb.torproject.org/user/dcf/tor-browser-build.git/commit/?h=snowf...
I opened a ticket for this: https://trac.torproject.org/projects/tor/ticket/33283
I think this patch should avoid doing multiple `git checkout` and `git log` on the same projects: https://gitweb.torproject.org/user/boklm/rbm.git/commit/?h=bug_33283&id=...
Thanks, I'll try the patch after I finish packaging the build from last night.
----
Output of last night's "time make testbuild" with only snowflake needing to be rebuilt. I had already manually built snowflake for linux-x86_64. It was commit 5eff0f3ba406eb7d8cd90711edab7de7d7bda4fa in https://git.torproject.org/user/dcf/snowflake.git.
``` # time make testbuild git submodule update --init ./rbm/rbm build release --target testbuild --target torbrowser-all Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-android-armv7-143b27 Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-android-x86-b005b1 Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-android-x86_64-5512f4 Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-android-aarch64-0dec52 Building project tor-browser - tor-browser-9.5a4-linux-x86_64-7ef7ab Using file /root/tor-browser-build/out/container-image/container-image_wheezy-amd64-a32817165f00.tar.gz Using file /root/tor-browser-build/projects/tor-browser/run_scripts Using directory /root/tor-browser-build/out/firefox/firefox-78e8d897cd2e-linux-x86_64-46c98c Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-linux-x86_64-5408dc Using file /root/tor-browser-build/out/https-everywhere/https-everywhere-2019.11.7-b894fe.xpi Using file /root/tor-browser-build/out/fonts/fonts-720e34851382-074f0d.tar.gz Using file /root/tor-browser-build/out/obfs4/obfs4-0.0.11-linux-x86_64-62b5ad.tar.gz Using file /root/tor-browser-build/out/snowflake/snowflake-5eff0f3ba406-linux-x86_64-fa4d5c.tar.gz Using directory /root/tor-browser-build/projects/tor-browser/Bundle-Data Using file /root/tor-browser-build/out/tor-browser/noscript_security_suite-11.0.11-an+fx.xpi Using file /root/tor-browser-build/projects/tor-browser/RelativeLink/start-tor-browser.desktop Using file /root/tor-browser-build/projects/tor-browser/RelativeLink/execdesktop Using file /root/tor-browser-build/projects/tor-browser/gtk3-settings.ini Build log: /root/tor-browser-build/logs/tor-browser-linux-x86_64.log Finished build of project tor-browser - tor-browser-9.5a4-linux-x86_64-7ef7ab Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-linux-x86_64-7ef7ab Building project tor-browser - tor-browser-9.5a4-linux-i686-56bb5f Using file /root/tor-browser-build/out/container-image/container-image_wheezy-amd64-ae74865f5ad7.tar.gz Using file /root/tor-browser-build/projects/tor-browser/run_scripts Using directory /root/tor-browser-build/out/firefox/firefox-78e8d897cd2e-linux-i686-78f104 Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-linux-i686-8cabbd Using file /root/tor-browser-build/out/https-everywhere/https-everywhere-2019.11.7-b894fe.xpi Using file /root/tor-browser-build/out/fonts/fonts-720e34851382-074f0d.tar.gz Using file /root/tor-browser-build/out/obfs4/obfs4-0.0.11-linux-i686-0d2a0b.tar.gz Building project snowflake - snowflake-5eff0f3ba406-linux-i686-172b69.tar.gz Created /root/tor-browser-build/tmp/rbm-7RbRC/snowflake-5eff0f3ba406.tar.gz Using file /root/tor-browser-build/out/container-image/container-image_wheezy-amd64-f2bcf2cde274.tar.gz Using file /root/tor-browser-build/out/go/go-1.12.13-6f6dc9.tar.gz Using file /root/tor-browser-build/out/uniuri/uniuri-8902c56451e9-linux-i686-095a9f.tar.gz Using file /root/tor-browser-build/out/goptlib/goptlib-v1.1.0-linux-i686-43ea9b.tar.gz Using file /root/tor-browser-build/out/pion-webrtc/pion-webrtc-77c6e3b827e4-linux-i686-02e5a3.tar.gz Using file /root/tor-browser-build/out/quic-go/quic-go-907071221cf9-linux-i686-4b046f.tar.gz Build log: /root/tor-browser-build/logs/snowflake-linux-i686.log Finished build of project snowflake - snowflake-5eff0f3ba406-linux-i686-172b69.tar.gz Using file /root/tor-browser-build/out/snowflake/snowflake-5eff0f3ba406-linux-i686-172b69.tar.gz Using directory /root/tor-browser-build/projects/tor-browser/Bundle-Data Using file /root/tor-browser-build/out/tor-browser/noscript_security_suite-11.0.11-an+fx.xpi Using file /root/tor-browser-build/projects/tor-browser/RelativeLink/start-tor-browser.desktop Using file /root/tor-browser-build/projects/tor-browser/RelativeLink/execdesktop Using file /root/tor-browser-build/projects/tor-browser/gtk3-settings.ini Build log: /root/tor-browser-build/logs/tor-browser-linux-i686.log Finished build of project tor-browser - tor-browser-9.5a4-linux-i686-56bb5f Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-linux-i686-56bb5f Building project tor-browser - tor-browser-9.5a4-windows-i686-465936 Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-860c78a36523.tar.gz Using file /root/tor-browser-build/projects/tor-browser/run_scripts Using directory /root/tor-browser-build/out/firefox/firefox-78e8d897cd2e-windows-i686-89cecd Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-windows-i686-b41491 Using file /root/tor-browser-build/out/https-everywhere/https-everywhere-2019.11.7-46dd96.xpi Using file /root/tor-browser-build/out/fonts/fonts-720e34851382-f954ac.tar.gz Using file /root/tor-browser-build/out/obfs4/obfs4-0.0.11-windows-i686-72bc65.tar.gz Building project snowflake - snowflake-5eff0f3ba406-windows-i686-dcf129.tar.gz Created /root/tor-browser-build/tmp/rbm-oAH9V/snowflake-5eff0f3ba406.tar.gz Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-18fd98cf5daf.tar.gz Using file /root/tor-browser-build/out/go/go-1.12.13-4c7c84.tar.gz Using file /root/tor-browser-build/out/uniuri/uniuri-8902c56451e9-windows-i686-1a6f4a.tar.gz Using file /root/tor-browser-build/out/goptlib/goptlib-v1.1.0-windows-i686-852b18.tar.gz Using file /root/tor-browser-build/out/pion-webrtc/pion-webrtc-77c6e3b827e4-windows-i686-dacf47.tar.gz Using file /root/tor-browser-build/out/quic-go/quic-go-907071221cf9-windows-i686-01524c.tar.gz Build log: /root/tor-browser-build/logs/snowflake-windows-i686.log Finished build of project snowflake - snowflake-5eff0f3ba406-windows-i686-dcf129.tar.gz Using file /root/tor-browser-build/out/snowflake/snowflake-5eff0f3ba406-windows-i686-dcf129.tar.gz Using directory /root/tor-browser-build/projects/tor-browser/Bundle-Data Using file /root/tor-browser-build/out/tor-browser/noscript_security_suite-11.0.11-an+fx.xpi Using file /root/tor-browser-build/out/nsis/nsis-3.04-windows-i686-6d8d58.tar.gz Using file /root/tor-browser-build/out/tbb-windows-installer/tbb-windows-installer-src-0.5-dbf177.tar.gz Using file /root/tor-browser-build/projects/tor-browser/pe_checksum_fix.py Using file /root/tor-browser-build/out/tor-browser/pefile-2017.11.5.tar.gz Build log: /root/tor-browser-build/logs/tor-browser-windows-i686.log Finished build of project tor-browser - tor-browser-9.5a4-windows-i686-465936 Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-windows-i686-465936 Building project tor-browser - tor-browser-9.5a4-windows-x86_64-9f74bd Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-860c78a36523.tar.gz Using file /root/tor-browser-build/projects/tor-browser/run_scripts Using directory /root/tor-browser-build/out/firefox/firefox-78e8d897cd2e-windows-x86_64-ffc30f Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-windows-x86_64-79026c Using file /root/tor-browser-build/out/https-everywhere/https-everywhere-2019.11.7-46dd96.xpi Using file /root/tor-browser-build/out/fonts/fonts-720e34851382-f954ac.tar.gz Using file /root/tor-browser-build/out/obfs4/obfs4-0.0.11-windows-x86_64-bb2f23.tar.gz Building project snowflake - snowflake-5eff0f3ba406-windows-x86_64-57b6ff.tar.gz Created /root/tor-browser-build/tmp/rbm-eOk0z/snowflake-5eff0f3ba406.tar.gz Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-18fd98cf5daf.tar.gz Using file /root/tor-browser-build/out/go/go-1.12.13-846325.tar.gz Using file /root/tor-browser-build/out/uniuri/uniuri-8902c56451e9-windows-x86_64-739ef5.tar.gz Using file /root/tor-browser-build/out/goptlib/goptlib-v1.1.0-windows-x86_64-8eaaca.tar.gz Using file /root/tor-browser-build/out/pion-webrtc/pion-webrtc-77c6e3b827e4-windows-x86_64-629396.tar.gz Using file /root/tor-browser-build/out/quic-go/quic-go-907071221cf9-windows-x86_64-ae697f.tar.gz Build log: /root/tor-browser-build/logs/snowflake-windows-x86_64.log Finished build of project snowflake - snowflake-5eff0f3ba406-windows-x86_64-57b6ff.tar.gz Using file /root/tor-browser-build/out/snowflake/snowflake-5eff0f3ba406-windows-x86_64-57b6ff.tar.gz Using directory /root/tor-browser-build/projects/tor-browser/Bundle-Data Using file /root/tor-browser-build/out/tor-browser/noscript_security_suite-11.0.11-an+fx.xpi Using file /root/tor-browser-build/out/nsis/nsis-3.04-windows-x86_64-91a7bd.tar.gz Using file /root/tor-browser-build/out/tbb-windows-installer/tbb-windows-installer-src-0.5-dbf177.tar.gz Using file /root/tor-browser-build/projects/tor-browser/pe_checksum_fix.py Using file /root/tor-browser-build/out/tor-browser/pefile-2017.11.5.tar.gz Build log: /root/tor-browser-build/logs/tor-browser-windows-x86_64.log Finished build of project tor-browser - tor-browser-9.5a4-windows-x86_64-9f74bd Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-windows-x86_64-9f74bd Building project tor-browser - tor-browser-9.5a4-osx-x86_64-db1da8 Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-1d614657ea1b.tar.gz Using file /root/tor-browser-build/projects/tor-browser/run_scripts Using directory /root/tor-browser-build/out/firefox/firefox-78e8d897cd2e-osx-x86_64-56070b Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-osx-x86_64-cd810e Using file /root/tor-browser-build/out/https-everywhere/https-everywhere-2019.11.7-46dd96.xpi Using file /root/tor-browser-build/out/fonts/fonts-720e34851382-5830a6.tar.gz Using file /root/tor-browser-build/out/obfs4/obfs4-0.0.11-osx-x86_64-9c01d4.tar.gz Building project snowflake - snowflake-5eff0f3ba406-osx-x86_64-f9fc60.tar.gz Created /root/tor-browser-build/tmp/rbm-hxgYM/snowflake-5eff0f3ba406.tar.gz Using file /root/tor-browser-build/out/container-image/container-image_stretch-amd64-c1192d34d787.tar.gz Using file /root/tor-browser-build/out/go/go-1.12.13-bbe0e3.tar.gz Using file /root/tor-browser-build/out/uniuri/uniuri-8902c56451e9-osx-x86_64-6d72b5.tar.gz Using file /root/tor-browser-build/out/goptlib/goptlib-v1.1.0-osx-x86_64-ab0a8a.tar.gz Using file /root/tor-browser-build/out/pion-webrtc/pion-webrtc-77c6e3b827e4-osx-x86_64-e18e7a.tar.gz Using file /root/tor-browser-build/out/quic-go/quic-go-907071221cf9-osx-x86_64-a6f2f4.tar.gz Build log: /root/tor-browser-build/logs/snowflake-osx-x86_64.log Finished build of project snowflake - snowflake-5eff0f3ba406-osx-x86_64-f9fc60.tar.gz Using file /root/tor-browser-build/out/snowflake/snowflake-5eff0f3ba406-osx-x86_64-f9fc60.tar.gz Using directory /root/tor-browser-build/projects/tor-browser/Bundle-Data Using file /root/tor-browser-build/out/tor-browser/noscript_security_suite-11.0.11-an+fx.xpi Using file /root/tor-browser-build/out/libdmg-hfsplus/libdmg-hfsplus-dfd5e5cc3dc1-62fa3e.tar.gz Build log: /root/tor-browser-build/logs/tor-browser-osx-x86_64.log Finished build of project tor-browser - tor-browser-9.5a4-osx-x86_64-db1da8 Using directory /root/tor-browser-build/out/tor-browser/tor-browser-9.5a4-osx-x86_64-db1da8 Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-windows-i686-d97207 Using directory /root/tor-browser-build/out/tor/tor-0.4.3.1-alpha-windows-x86_64-18debd Build log: /root/tor-browser-build/logs/release-linux-x86_64.log
real 544m23.414s user 482m3.219s sys 73m29.899s ```
On Wed, Feb 12, 2020 at 11:30:13AM -0700, David Fifield wrote:
Output of last night's "time make testbuild" with only snowflake needing to be rebuilt. I had already manually built snowflake for linux-x86_64. It was commit 5eff0f3ba406eb7d8cd90711edab7de7d7bda4fa in https://git.torproject.org/user/dcf/snowflake.git.
Sorry, I meant commit b46fd762e0866fe01707e469eb997040f37c16a2 in https://git.torproject.org/user/dcf/tor-browser-build.git.
On Tue, 11 Feb 2020, David Fifield wrote:
To my untrained eye it looks like there is some kind of exponential recursion happening around the RBM::config_p function. See here, where in building one project the function was called 89 million times: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/#sub... Clicking through to the per-line view, we see that 45 million of those calls were from RBM::get_target, and most of the rest were from RBM::config: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/root...
After looking at this more closely, I am expecting RBM::config_p to be called a lot of times (although maybe not 89 millions times), but I would not expect that more than half of this would be from RBM::get_target.
So I opened an other ticket about that: https://trac.torproject.org/projects/tor/ticket/33289
-
David Fifield -
Matthew Finkel -
Nicolas Vigier