On Sun, Jun 9, 2013 at 5:03 PM, Mike Perry mikeperry@torproject.org wrote:
AK:
On Wed, May 22, 2013 at 6:40 PM, Mike Perry mikeperry@torproject.org wrote:
Mike Perry:
Thus spake Jeroen Massar (jeroen@massar.ch):
On 2013-05-15 08:49 , Mike Perry wrote:
Unfortunately, at least one of those URLs say that to produce a cross-compiler, you need access to an OSX SDK. Since I do not have a Mac that is currently supported by recent OSX SDKs, and since we *really* want to be sure that the cross-compilers we produce use code from a fresh known-good SDK install, I won't be doing this. Please let me know if you'd like to help tackle this problem.
The tutorials you provide all use OpenDarwin's cctools, which is a modified/"improved" version of Apple's opensource cctools. Most of that stuff is pretty outdated though, as OSX is at 10.8, not 10.5 from about 6 years ago. It seems though most people (reading what is being done on the web ;) are using the flosoft stuff which is at least 10.6 and embeds the OSX SDK.
Ok, I guess I will just go the flosoft route then. I will work on writing gitian descriptors using these binary packages. However, it looks like right now this means I have to use "Maverick" (11.04) as the Gitian VM's base install. Since "Maverick" is unsupported for security updates, this might be unwise for production..
Is anyone able to work on rebuilding flosofts's .deb packages for either lucid (10.04 LTS) or precise (12.04 LTS) while I work on getting initial Gitian descriptors using Maverick for now? https://launchpad.net/~flosoft/+archive/cross-apple/+packages
Good news, everyone!
I ended up hitting an Internal Compiler Error in flosoft's GCC while trying to build Firefox with them. After a long and painful saga of trying to rebuild a newer version of GCC, I eventually discovered the excellent "toolchain4" cross-compilation project maintained by Ray Donnelly: https://github.com/mingwandroid/toolchain4 https://mingw-and-ndk.googlecode.com/files/multiarch-darwin11-cctools127.2-g...
Despite using the same base GCC version as flosoft (4.2.1), his compiler does not hit the ICE with Firefox.
Moreover, with Ray's help, I was able to use his binaries and the flosoft OSX 10.6 SDK to successfully build a working TBB for Mac from Gitian. These descriptors are now in the repository, and the Makefile should now build bundles for all three platforms by default.
There still are some branding and localization bugs, and I have not yet redone the whole build to verify determinism for MacOS yet, but I am optimistic!
Would this work on an Ubuntu that's built from source, or are we dependent on their binaries?
Unclear. Unfortunately, I've run into a series of differences that arise between different *hardware*, even though the underlying Ubuntu versions and VMs are the same.. Most of those are file reordering differences inside zip files that probably arise from filesystem/blockdevice-level differences, but there is some really spooky shit happening inside of Firefox's libxul.so too (no joke).
I am now working on eliminating those differences, so that at least two different computers can produce the same binaries. I have to deal with that setback first before we could expect a full source rebuild to work, I think. :/
However, I think the better way to achieve what you're getting at is to recompile the Ubuntu toolchain (gcc/g++/binutils) itself from source as a Gitian descriptor. Once we get that piece, we *should* be able to run all the descriptors on different Linux distribution images and still get the same binaries, allowing us to require compromise of *all* Linux distributions that reproduce our builds (including say Gentoo).
This idea is documented in the "Long-Term" section of the TODO file, but it would happen much faster if someone helped to make a simple Gitian descriptor to recompile the Ubuntu toolchain (which we consider known-good because it is what Mozilla uses). https://gitweb.torproject.org/builders/tor-browser-bundle.git/blob/HEAD:/git...
We'd need to do the same with the mingw-w64 toolchain and Ray's 'toolchain4', too.
Note: The toolchains themselves probably don't have to be compiled determistically (except for the statically linked bits like libgcc and the crt stuff).
-- Mike Perry
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Thanks for the reply. Its good to know that these issues are being looked at. The build process was one of the main problems I had with the TBB, and this definitely seems like an improvement.