tbb-commits June 2016

tbb-commits@lists.torproject.org

2 participants
55 discussions

[tor-browser-bundle/master] Bug 19348: Windows %BUILD_TARGET% changed in Tor Browser 6.0
by gk＠torproject.org 17 Jun '16

17 Jun '16

commit 7124bb8dfb187108f0f2a4d885ef8f32afea1af7 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Jun 13 18:15:54 2016 +0200 Bug 19348: Windows %BUILD_TARGET% changed in Tor Browser 6.0 The values in build_targets can now be a list of values rather than a single value and are all included in the generated .htaccess file. They are also all tested in check_update_responses_deployement. Additionally, a trailing '/' in the last RewriteRule in the .htaccess has been added. This missing '/' was the reason why the requests using the new BUILD_TARGET values were returning the en-US complete mar file rather than an error. --- tools/update-responses/config.yml | 5 ++++- tools/update-responses/update_responses | 29 ++++++++++++++++++----------- 2 files changed, 22 insertions(+), 12 deletions(-) diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index adb1067..0425afc 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -5,7 +5,10 @@ download: build_targets: linux32: Linux_x86-gcc3 linux64: Linux_x86_64-gcc3 - win32: WINNT_x86-gcc3 + win32: + - WINNT_x86-gcc3 + - WINNT_x86-gcc3-x86 + - WINNT_x86-gcc3-x64 osx32: Darwin_x86-gcc3 osx64: Darwin_x86_64-gcc3 channels: diff --git a/tools/update-responses/update_responses b/tools/update-responses/update_responses index 72a19d7..1a7cb14 100755 --- a/tools/update-responses/update_responses +++ b/tools/update-responses/update_responses @@ -39,8 +39,10 @@ sub exit_error { exit (exists $_[1] ? $_[1] : 1); } -sub build_target_by_os { - $config->{build_targets}{$_[0]} ? $config->{build_targets}{$_[0]} : $_[0]; +sub build_targets_by_os { + return ($_[0]) unless $config->{build_targets}{$_[0]}; + my $r = $config->{build_targets}{$_[0]}; + return ref $r eq 'ARRAY' ? @$r : ($r); } sub get_nbprocs { @@ -313,16 +315,17 @@ sub write_htaccess { my $files = $config->{versions}{$version}{files}; $htaccess .= "RewriteRule ^[^\/]+/$version/ no-update.xml $flags\n"; foreach my $os (sort keys %$files) { - my $bt = build_target_by_os($os); - foreach my $lang (sort keys %{$files->{$os}}) { - foreach my $from_version (sort keys %{$files->{$os}{$lang}{partial}}) { - $htaccess .= "RewriteRule ^$bt/$from_version/$lang " - . "$from_version-$version-$os-$lang.xml $flags\n"; + foreach my $bt (build_targets_by_os($os)) { + foreach my $lang (sort keys %{$files->{$os}}) { + foreach my $from_version (sort keys %{$files->{$os}{$lang}{partial}}) { + $htaccess .= "RewriteRule ^$bt/$from_version/$lang " + . "$from_version-$version-$os-$lang.xml $flags\n"; + } + $htaccess .= "RewriteRule ^$bt/[^\/]+/$lang " + . "$version-$os-$lang.xml $flags\n"; } - $htaccess .= "RewriteRule ^$bt/[^\/]+/$lang " - . "$version-$os-$lang.xml $flags\n"; + $htaccess .= "RewriteRule ^$bt/ $version-$os-en-US.xml $flags\n"; } - $htaccess .= "RewriteRule ^$bt $version-$os-en-US.xml $flags\n"; } write_htdocs($channel, '.htaccess', $htaccess); } @@ -426,10 +429,14 @@ sub check_has_incremental { return undef; } +sub build_targets_list { + map { ref $_ eq 'ARRAY' ? @$_ : $_ } values %{$config->{build_targets}}; +} + sub check_update_responses_channel { my ($config, $base_url, $channel) = @_; my $channel_version = $config->{channels}{$channel}; - foreach my $build_target (values %{$config->{build_targets}}) { + foreach my $build_target (build_targets_list()) { foreach my $lang (qw(en-US de)) { my $url = "$base_url/$channel/$build_target/1.0/$lang"; my $dom = get_remote_xml($url);

1 0

[tor-browser-bundle/master] Merge hardened changelog
by gk＠torproject.org 17 Jun '16

17 Jun '16

commit 20ab1173a22dda041c22e606042949d54a821091 Author: Georg Koppen <gk(a)torproject.org> Date: Fri Jun 17 07:51:53 2016 +0000 Merge hardened changelog --- Bundle-Data/Docs/ChangeLog.txt | 54 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index ffd2c2f..83963d1 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -1,3 +1,57 @@ +Tor Browser 6.5a1-hardened -- June 8 + * All Platforms + * Update Firefox to 45.2.0esr + * Update Tor to 0.2.8.3-alpha + * Update Torbutton to 1.9.6 + * Bug 18743: Pref to hide 'Sign in to Sync' button in hamburger menu + * Bug 18905: Hide unusable items from help menu + * Bug 17599: Provide shortcuts for New Identity and New Circuit + * Bug 18980: Remove obsolete toolbar button code + * Bug 18238: Remove unused Torbutton code and strings + * Translation updates + * Code clean-up + * Update Tor Launcher to 0.2.8.5 + * Bug 18947: Tor Browser is not starting on OS X if put into /Applications + * Update HTTPS-Everywhere to 5.1.9 + * Update meek to 0.22 (tag 0.22-18371-3) + * Bug 19121: The update.xml hash should get checked during update + * Bug 12523: Mark JIT pages as non-writable + * Bug 19193: Reduce timing precision for AudioContext, HTMLMediaElement, and MediaStream + * Bug 19164: Remove support for SHA-1 HPKP pins + * Bug 19186: KeyboardEvents are only rounding to 100ms + * Bug 18884: Don't build the loop extension + * Bug 19187: Backport fix for crash related to popup menus + * Bug 19212: Fix crash related to network panel in developer tools + * Bug 18703: Fix circuit isolation issues on Page Info dialog + * bug 19115: Tor Browser should not fall back to Bing as its search engine + * Bug 18915+19065: Use our search plugins in localized builds + * Bug 19176: Zip our language packs deterministically + * Bug 18811: Fix first-party isolation for blobs URLs in Workers + * Bug 18950: Disable or audit Reader View + * Bug 18886: Remove Pocket + * Bug 18619: Tor Browser reports "InvalidStateError" in browser console + * Bug 18945: Disable monitoring the connected state of Tor Browser users + * Bug 18855: Don't show error after add-on directory clean-up + * Bug 18885: Disable the option of logging TLS/SSL key material + * Bug 18770: SVGs should not show up on Page Info dialog when disabled + * Bug 18958: Spoof screen.orientation values + * Bug 19047: Disable Heartbeat prompts + * Bug 18914: Use English-only label in <isindex/> tags + * Bug 18996: Investigate server logging in esr45-based Tor Browser + * Bug 17790: Add unit tests for keyboard fingerprinting defenses + * Bug 18995: Regression test to ensure CacheStorage is disabled + * Bug 18912: Add automated tests for updater cert pinning + * Bug 16728: Add test cases for favicon isolation + * Bug 18976: Remove some FTE bridges + * Linux + * Bug 19189: Backport for working around a linker (gold) bug + * Build System + * All PLatforms + * Bug 18333: Upgrade Go to 1.6.2 + * Bug 18919: Remove unused keys and unused dependencies + * Bug 18291: Remove some uses of libfaketime + * Bug 18845: Make zip and tar helpers generate reproducible archives + Tor Browser 6.5a1 -- June 8 * All Platforms * Update Firefox to 45.2.0esr

1 0

[tor-browser/tor-browser-45.2.0esr-6.5-1] fixup! Bug #15502. Isolate blob, mediasource & mediastream URLs to first party
by gk＠torproject.org 13 Jun '16

13 Jun '16

commit 74a5f3138fe10081b8384419f96c4043f1f1d23f Author: Georg Koppen <gk(a)torproject.org> Date: Mon Jun 13 07:49:59 2016 +0000 fixup! Bug #15502. Isolate blob, mediasource & mediastream URLs to first party This fixes bug 19401 which is caused by a rebase error we overlooked. --- browser/extensions/pdfjs/content/PdfStreamConverter.jsm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/browser/extensions/pdfjs/content/PdfStreamConverter.jsm b/browser/extensions/pdfjs/content/PdfStreamConverter.jsm index d68c940..e6ef8e1 100644 --- a/browser/extensions/pdfjs/content/PdfStreamConverter.jsm +++ b/browser/extensions/pdfjs/content/PdfStreamConverter.jsm @@ -264,9 +264,9 @@ ChromeActions.prototype = { var docIsPrivate = this.isInPrivateBrowsing(); var netChannel; try { - netChannel = NetUtil.createNewChannel(blobUri, frontWindow.document, null); + netChannel = createNewChannel(blobUri, frontWindow.document, null); } catch (e) { - netChannel = NetUtil.createNewChannel(originalUri, frontWindow.document, null); + netChannel = createNewChannel(originalUri, frontWindow.document, null); } if ('nsIPrivateBrowsingChannel' in Ci && netChannel instanceof Ci.nsIPrivateBrowsingChannel) {

1 0

[tor-browser/tor-browser-45.2.0esr-6.0-1] fixup! Bug #15502. Isolate blob, mediasource & mediastream URLs to first party
by gk＠torproject.org 13 Jun '16

13 Jun '16

commit abff9655e7e5fc87c78fb6dd2a99dab494971cee Author: Georg Koppen <gk(a)torproject.org> Date: Mon Jun 13 07:49:59 2016 +0000 fixup! Bug #15502. Isolate blob, mediasource & mediastream URLs to first party This fixes bug 19401 which is caused by a rebase error we overlooked. --- browser/extensions/pdfjs/content/PdfStreamConverter.jsm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/browser/extensions/pdfjs/content/PdfStreamConverter.jsm b/browser/extensions/pdfjs/content/PdfStreamConverter.jsm index d68c940..e6ef8e1 100644 --- a/browser/extensions/pdfjs/content/PdfStreamConverter.jsm +++ b/browser/extensions/pdfjs/content/PdfStreamConverter.jsm @@ -264,9 +264,9 @@ ChromeActions.prototype = { var docIsPrivate = this.isInPrivateBrowsing(); var netChannel; try { - netChannel = NetUtil.createNewChannel(blobUri, frontWindow.document, null); + netChannel = createNewChannel(blobUri, frontWindow.document, null); } catch (e) { - netChannel = NetUtil.createNewChannel(originalUri, frontWindow.document, null); + netChannel = createNewChannel(originalUri, frontWindow.document, null); } if ('nsIPrivateBrowsingChannel' in Ci && netChannel instanceof Ci.nsIPrivateBrowsingChannel) {

1 0

[torbutton/master] Bug 19206: Include a 128 bit random tag as part of the domain isolator nonce.
by gk＠torproject.org 10 Jun '16

10 Jun '16

commit 0fbd44f0305a01a14bc7439f5ad66b59d9278a2d Author: Yawning Angel <yawning(a)schwanenlied.me> Date: Tue Jun 7 19:01:41 2016 +0000 Bug 19206: Include a 128 bit random tag as part of the domain isolator nonce. When creating a domain isolation nonce, 128 bits of entropy is drawn from a cryptographic source and saved on a per-domain basis. The new circuit behavior is changed to regenerate the nonce, instead of incrementing a counter. This allows the "right thing" to happen when the same tor instance is used across multiple Tor Browser sessions, for example when using a system wide tor, or a magic anonymity box. --- src/components/domain-isolator.js | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/src/components/domain-isolator.js b/src/components/domain-isolator.js index 6cfd21e..769c47d 100644 --- a/src/components/domain-isolator.js +++ b/src/components/domain-isolator.js @@ -16,6 +16,9 @@ const Cc = Components.classes, Ci = Components.interfaces, Cu = Components.utils let logger = Cc["@torproject.org/torbutton-logger;1"] .getService(Components.interfaces.nsISupports).wrappedJSObject; +// Import crypto object (FF 37+). +Cu.importGlobalProperties(["crypto"]); + // ## mozilla namespace. // Useful functionality for interacting with Mozilla services. let mozilla = {}; @@ -69,7 +72,7 @@ tor.socksProxyCredentials = function (originalProxy, domain) { // Check if we already have a nonce. If not, create // one for this domain. if (!tor.noncesForDomains.hasOwnProperty(domain)) { - tor.noncesForDomains[domain] = 0; + tor.noncesForDomains[domain] = tor.nonce(); } let proxy = originalProxy.QueryInterface(Ci.nsIProxyInfo); return mozilla.protocolProxyService @@ -77,21 +80,34 @@ tor.socksProxyCredentials = function (originalProxy, domain) { proxy.host, proxy.port, domain, // username - tor.noncesForDomains[domain].toString(), // password + tor.noncesForDomains[domain], // password proxy.flags, proxy.failoverTimeout, proxy.failoverProxy); }; -tor.newCircuitForDomain = function(domain) { - // Check if we already have a nonce. If not, create - // one for this domain. - if (!tor.noncesForDomains.hasOwnProperty(domain)) { - tor.noncesForDomains[domain] = 0; - } else { - tor.noncesForDomains[domain] += 1; +tor.nonce = function() { + // Generate a new 128 bit random tag. Strictly speaking both using a + // cryptographic entropy source and using 128 bits of entropy for the + // tag are likely overkill, as correct behavior only depends on how + // unlikely it is for there to be a collision. + let tag = new Uint8Array(16); + crypto.getRandomValues(tag); + + // Convert the tag to a hex string. + let tagStr = ""; + for (var i = 0; i < tag.length; i++) { + tagStr += (tag[i] >>> 4).toString(16); + tagStr += (tag[i] & 0x0F).toString(16); } - logger.eclog(3, "New domain isolation count " +tor.noncesForDomains[domain] + " for " + domain); + + return tagStr; +} + +tor.newCircuitForDomain = function(domain) { + // Re-generate the nonce for the domain. + tor.noncesForDomains[domain] = tor.nonce(); + logger.eclog(3, "New domain isolation for " + domain + ": " + tor.noncesForDomains[domain]); } // __tor.isolateCircuitsByDomain()__.

1 0

[torbutton/master] Bug 19206: Clear out the domain isolator state on `New Identity`.
by gk＠torproject.org 10 Jun '16

10 Jun '16

commit 36d849291ec0b20a58cccc2cd846fcd2540c9bbe Author: Yawning Angel <yawning(a)schwanenlied.me> Date: Tue Jun 7 19:03:31 2016 +0000 Bug 19206: Clear out the domain isolator state on `New Identity`. Additionally clear out the domain isolator state on `New Identity`. In theory this removes the need to explicitly issue a `NEWNYM` as new circuits will be used for all subsequent requests, including those made via the catch-all circuit. --- src/chrome/content/torbutton.js | 7 +++++++ src/components/domain-isolator.js | 16 ++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/src/chrome/content/torbutton.js b/src/chrome/content/torbutton.js index a293c44..e0e541b 100644 --- a/src/chrome/content/torbutton.js +++ b/src/chrome/content/torbutton.js @@ -1872,6 +1872,13 @@ function torbutton_do_new_identity() { getService(Ci.nsIPermissionManager); pm.removeAll(); + // Clear the domain isolation state. + torbutton_log(3, "New Identity: Clearing domain isolator"); + + let domainIsolator = Cc["@torproject.org/domain-isolator;1"] + .getService(Ci.nsISupports).wrappedJSObject; + domainIsolator.clearIsolation(); + torbutton_log(3, "New Identity: Sending NEWNYM"); // We only support TBB for newnym. diff --git a/src/components/domain-isolator.js b/src/components/domain-isolator.js index 769c47d..dbdca1b 100644 --- a/src/components/domain-isolator.js +++ b/src/components/domain-isolator.js @@ -110,6 +110,18 @@ tor.newCircuitForDomain = function(domain) { logger.eclog(3, "New domain isolation for " + domain + ": " + tor.noncesForDomains[domain]); } +// __tor.clearIsolation()_. +// Clear the isolation state cache, forcing new circuits to be used for all +// subsequent requests. +tor.clearIsolation = function () { + // Per-domain nonces are stored in a map, so simply re-initialize the map. + tor.noncesForDomains = {}; + + // Force a rotation on the next catch-all circuit use by setting the creation + // time to the epoch. + tor.unknownDirtySince = 0; +} + // __tor.isolateCircuitsByDomain()__. // For every HTTPChannel, replaces the default SOCKS proxy with one that authenticates // to the SOCKS server (the tor client process) with a username (the first party domain) @@ -190,6 +202,10 @@ DomainIsolator.prototype = { tor.isolationEnabled = false; }, + clearIsolation: function() { + tor.clearIsolation(); + }, + wrappedJSObject: null };

1 0

[tor-browser-bundle/hardened-builds] Update config.yml
by gk＠torproject.org 05 Jun '16

05 Jun '16

commit 2622d3c575a8e190e774d34b4deef75c3bebcbef Author: Georg Koppen <gk(a)torproject.org> Date: Sun Jun 5 21:10:14 2016 +0000 Update config.yml --- tools/update-responses/config.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index 7299bb8..5e4d463 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -9,8 +9,8 @@ build_targets: osx32: Darwin_x86-gcc3 osx64: Darwin_x86_64-gcc3 channels: - hardened: 6.0a5-hardened - release: 5.0 + hardened: 6.5a1-hardened + release: 6.0.1 versions: 6.0.1: platformVersion: 45.2.0 @@ -23,12 +23,12 @@ versions: osx32: minSupportedOSVersion: 10.8 detailsURL: https://blog.torproject.org/blog/end-life-plan-tor-browser-32-bit-macs#upda… - 6.0a5-hardened: - platformVersion: 45.1.0 - detailsURL: https://blog.torproject.org/blog/tor-browser-60a5-released - download_url: https://cdn.torproject.org/aus1/torbrowser/6.0a5 + 6.5a1-hardened: + platformVersion: 45.2.0 + detailsURL: https://blog.torproject.org/blog/tor-browser-65a1-hardened-released + download_url: https://cdn.torproject.org/aus1/torbrowser/6.5a1-hardened incremental_from: - - 6.0a4-hardened + - 6.0a5-hardened migrate_archs: osx32: osx64 osx32:

1 0

[tor-browser-bundle/hardened-builds] Release preparations for 6.5a1-hardened
by gk＠torproject.org 05 Jun '16

05 Jun '16

commit 1470eeb9c650229df3129596ae3364e86bbc7dee Author: Georg Koppen <gk(a)torproject.org> Date: Sun Jun 5 21:07:03 2016 +0000 Release preparations for 6.5a1-hardened --- Bundle-Data/Docs/ChangeLog.txt | 54 ++++++++++++++++++++++++++++++++++++++++++ gitian/versions.alpha | 16 ++++++------- gitian/versions.nightly | 8 +++---- 3 files changed, 66 insertions(+), 12 deletions(-) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index 35d0e9d..40a36c9 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -1,3 +1,57 @@ +Tor Browser 6.5a1-hardened -- June 8 + * All Platforms + * Update Firefox to 45.2.0esr + * Update Tor to 0.2.8.3-alpha + * Update Torbutton to 1.9.6 + * Bug 18743: Pref to hide 'Sign in to Sync' button in hamburger menu + * Bug 18905: Hide unusable items from help menu + * Bug 17599: Provide shortcuts for New Identity and New Circuit + * Bug 18980: Remove obsolete toolbar button code + * Bug 18238: Remove unused Torbutton code and strings + * Translation updates + * Code clean-up + * Update Tor Launcher to 0.2.8.5 + * Bug 18947: Tor Browser is not starting on OS X if put into /Applications + * Update HTTPS-Everywhere to 5.1.9 + * Update meek to 0.22 (tag 0.22-18371-3) + * Bug 19121: The update.xml hash should get checked during update + * Bug 12523: Mark JIT pages as non-writable + * Bug 19193: Reduce timing precision for AudioContext, HTMLMediaElement, and MediaStream + * Bug 19164: Remove support for SHA-1 HPKP pins + * Bug 19186: KeyboardEvents are only rounding to 100ms + * Bug 18884: Don't build the loop extension + * Bug 19187: Backport fix for crash related to popup menus + * Bug 19212: Fix crash related to network panel in developer tools + * Bug 18703: Fix circuit isolation issues on Page Info dialog + * bug 19115: Tor Browser should not fall back to Bing as its search engine + * Bug 18915+19065: Use our search plugins in localized builds + * Bug 19176: Zip our language packs deterministically + * Bug 18811: Fix first-party isolation for blobs URLs in Workers + * Bug 18950: Disable or audit Reader View + * Bug 18886: Remove Pocket + * Bug 18619: Tor Browser reports "InvalidStateError" in browser console + * Bug 18945: Disable monitoring the connected state of Tor Browser users + * Bug 18855: Don't show error after add-on directory clean-up + * Bug 18885: Disable the option of logging TLS/SSL key material + * Bug 18770: SVGs should not show up on Page Info dialog when disabled + * Bug 18958: Spoof screen.orientation values + * Bug 19047: Disable Heartbeat prompts + * Bug 18914: Use English-only label in <isindex/> tags + * Bug 18996: Investigate server logging in esr45-based Tor Browser + * Bug 17790: Add unit tests for keyboard fingerprinting defenses + * Bug 18995: Regression test to ensure CacheStorage is disabled + * Bug 18912: Add automated tests for updater cert pinning + * Bug 16728: Add test cases for favicon isolation + * Bug 18976: Remove some FTE bridges + * Linux + * Bug 19189: Backport for working around a linker (gold) bug + * Build System + * All PLatforms + * Bug 18333: Upgrade Go to 1.6.2 + * Bug 18919: Remove unused keys and unused dependencies + * Bug 18291: Remove some uses of libfaketime + * Bug 18845: Make zip and tar helpers generate reproducible archives + Tor Browser 6.0a5-hardened -- April 28 2016 * All Platforms * Update Firefox to 45.1.0esr diff --git a/gitian/versions.alpha b/gitian/versions.alpha index c6b1862..1d671ff 100755 --- a/gitian/versions.alpha +++ b/gitian/versions.alpha @@ -14,15 +14,15 @@ DATA_OUTSIDE_APP_DIR=1 VERIFY_TAGS=1 -FIREFOX_VERSION=45.1.0esr +FIREFOX_VERSION=45.2.0esr TORBROWSER_UPDATE_CHANNEL=hardened -TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-6.0-1-build3 -TOR_TAG=tor-0.2.8.2-alpha -TORLAUNCHER_TAG=0.2.8.4 -TORBUTTON_TAG=1.9.5.3 -HTTPSE_TAG=5.1.6 +TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-6.5-1-build1 +TOR_TAG=tor-0.2.8.3-alpha +TORLAUNCHER_TAG=0.2.8.5 +TORBUTTON_TAG=1.9.6 +HTTPSE_TAG=5.1.9 NSIS_TAG=v0.3.1 ZLIB_TAG=v1.2.8 LIBEVENT_TAG=release-2.0.22-stable @@ -37,7 +37,7 @@ FTEPROXY_TAG=597f8378f6f4f3de570b8e1064c2e4cb8d67fbd0 # tag 0.2.19 LIBDMG_TAG=dfd5e5cc3dc1191e37d3c3a6118975afdd1d7014 TXSOCKSX_TAG=216eb0894a1755872f4789f9458aa6cf543b8433 # unsigned habnabit/1.13.0.2 GOPTLIB_TAG=0.5 -MEEK_TAG=0.22-18371-2 +MEEK_TAG=0.22-18371-3 FAKETIME_TAG=70aa6b394d9341522dffe8a5a5cf5929e82cc6b9 # unsigned v0.9.6 GOED25519_TAG=c4161f4c7483313562781c61b9a20aba73daf9de GOSIPHASH_TAG=42ba037e748c9062a75e0924705c43b893edefcd @@ -51,7 +51,7 @@ GITIAN_TAG=tor-browser-builder-4-1 OPENSSL_VER=1.0.1t GMP_VER=5.1.3 FIREFOX_LANG_VER=$FIREFOX_VERSION -FIREFOX_LANG_BUILD=build1 +FIREFOX_LANG_BUILD=build2 BINUTILS_VER=2.24 GCC_VER=5.2.0 CLANG_VER=r247539 diff --git a/gitian/versions.nightly b/gitian/versions.nightly index 577c40c..60d5f34 100755 --- a/gitian/versions.nightly +++ b/gitian/versions.nightly @@ -17,11 +17,11 @@ MULTI_LINGUAL=1 VERIFY_TAGS=0 -FIREFOX_VERSION=45.1.0esr +FIREFOX_VERSION=45.2.0esr TORBROWSER_UPDATE_CHANNEL=default -TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-6.0-1 +TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-6.5-1 TOR_TAG=master TORLAUNCHER_TAG=master TORBUTTON_TAG=master @@ -40,7 +40,7 @@ FTEPROXY_TAG=master LIBDMG_TAG=dfd5e5cc3dc1191e37d3c3a6118975afdd1d7014 TXSOCKSX_TAG=216eb0894a1755872f4789f9458aa6cf543b8433 # unsigned habnabit/1.13.0.2 GOPTLIB_TAG=master -MEEK_TAG=0.22-18371-2 #master +MEEK_TAG=0.22-18371-3 #master FAKETIME_TAG=70aa6b394d9341522dffe8a5a5cf5929e82cc6b9 # unsigned v0.9.6 GOED25519_TAG=c4161f4c7483313562781c61b9a20aba73daf9de GOSIPHASH_TAG=42ba037e748c9062a75e0924705c43b893edefcd @@ -55,7 +55,7 @@ GITIAN_TAG=tor-browser-builder-4 OPENSSL_VER=1.0.1t GMP_VER=5.1.3 FIREFOX_LANG_VER=$FIREFOX_VERSION -FIREFOX_LANG_BUILD=build1 +FIREFOX_LANG_BUILD=build2 BINUTILS_VER=2.24 GCC_VER=5.2.0 CLANG_VER=r247539

1 0

[tor-browser-bundle/hardened-builds] Bug 18845: generate reproducible archives without relying on libfaketime
by gk＠torproject.org 05 Jun '16

05 Jun '16

commit 5eacf7266cd60d8b3f6bca63ca427c326a916c16 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon May 16 19:26:26 2016 +0200 Bug 18845: generate reproducible archives without relying on libfaketime --- gitian/build-helpers/ddmg.sh | 3 +++ gitian/build-helpers/dtar.sh | 3 +++ gitian/build-helpers/dzip.sh | 3 +++ gitian/build-helpers/re-dzip.sh | 2 ++ 4 files changed, 11 insertions(+) diff --git a/gitian/build-helpers/ddmg.sh b/gitian/build-helpers/ddmg.sh index c37a1fd..f956aa6 100755 --- a/gitian/build-helpers/ddmg.sh +++ b/gitian/build-helpers/ddmg.sh @@ -11,6 +11,9 @@ shift find $@ -executable -exec chmod 750 {} \; find $@ ! -executable -exec chmod 640 {} \; +[ -n "$REFERENCE_DATETIME" ] && \ + find $@ -exec touch --date="$REFERENCE_DATETIME" {} \; + cd $@ find . -type f | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" > ~/build/filelist.txt find . -type l | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" >> ~/build/filelist.txt diff --git a/gitian/build-helpers/dtar.sh b/gitian/build-helpers/dtar.sh index e8ec340..3aa6f0c 100755 --- a/gitian/build-helpers/dtar.sh +++ b/gitian/build-helpers/dtar.sh @@ -5,6 +5,9 @@ export LC_ALL=C TARFILE=$1 shift +[ -n "$REFERENCE_DATETIME" ] && \ + find $@ -exec touch --date="$REFERENCE_DATETIME" {} \; + # No need to execute chmod on (possibly) dangling symlinks. find $@ ! -type l -executable -exec chmod 700 {} \; find $@ ! -type l ! -executable -exec chmod 600 {} \; diff --git a/gitian/build-helpers/dzip.sh b/gitian/build-helpers/dzip.sh index 2f7dc54..5772c8b 100755 --- a/gitian/build-helpers/dzip.sh +++ b/gitian/build-helpers/dzip.sh @@ -5,6 +5,9 @@ export LC_ALL=C ZIPFILE=$1 shift +[ -n "$REFERENCE_DATETIME" ] && \ + find $@ -exec touch --date="$REFERENCE_DATETIME" {} \; + find $@ -executable -exec chmod 700 {} \; find $@ ! -executable -exec chmod 600 {} \; diff --git a/gitian/build-helpers/re-dzip.sh b/gitian/build-helpers/re-dzip.sh index 656e225..27828e9 100755 --- a/gitian/build-helpers/re-dzip.sh +++ b/gitian/build-helpers/re-dzip.sh @@ -7,6 +7,8 @@ ZIPFILE=`basename $1` mkdir tmp_dzip cd tmp_dzip unzip ../$1 +[ -n "$REFERENCE_DATETIME" ] && \ + find . -exec touch --date="$REFERENCE_DATETIME" {} \; find . -executable -exec chmod 700 {} \; find . ! -executable -exec chmod 600 {} \; find . | sort | zip $ZIPOPTS -X -@ $ZIPFILE

1 0

[tor-browser-bundle/hardened-builds] Bug 18291: Remove some uses of libfaketime
by gk＠torproject.org 05 Jun '16

05 Jun '16

commit 917084783c20e216ef7bfd397152e05b6b2a8ef2 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon May 23 19:58:44 2016 +0200 Bug 18291: Remove some uses of libfaketime Completely stop using libfaketime for Linux builds. For Windows builds we have to keep using it for the tor, pluggable-transports and bundle steps. For OSX builds, we keep it in the bundle steps for the timestamp added when creating the dmg file with genisoimage. Note: the OS X descriptors are broken (which is okay as we only need the Linux ones) --- gitian/descriptors/linux/gitian-bundle.yml | 16 ++++------- gitian/descriptors/linux/gitian-firefox.yml | 32 +++------------------- .../linux/gitian-pluggable-transports.yml | 4 +-- gitian/descriptors/linux/gitian-tor.yml | 4 +-- gitian/descriptors/linux/gitian-utils.yml | 17 +++++------- gitian/descriptors/mac/gitian-firefox.yml | 17 ++---------- .../mac/gitian-pluggable-transports.yml | 4 +-- gitian/descriptors/mac/gitian-utils.yml | 32 +++++++++++++++------- gitian/descriptors/windows/gitian-firefox.yml | 23 ++-------------- gitian/descriptors/windows/gitian-utils.yml | 15 +++++----- .../patches/openssl-Make-build-reproducible.patch | 28 +++++++++++++++++++ 11 files changed, 80 insertions(+), 112 deletions(-) diff --git a/gitian/descriptors/linux/gitian-bundle.yml b/gitian/descriptors/linux/gitian-bundle.yml index e25d40e..541904d 100644 --- a/gitian/descriptors/linux/gitian-bundle.yml +++ b/gitian/descriptors/linux/gitian-bundle.yml @@ -27,8 +27,6 @@ remotes: "dir": "meek" - "url": "https://github.com/googlei18n/noto-fonts.git" "dir": "noto-fonts" -- "url": "https://github.com/wolfcw/libfaketime" - "dir": "faketime" files: # TODO: Can we use an env for this file+version?? - "tor-browser-linux64-gbuilt.zip" @@ -57,21 +55,13 @@ files: script: | INSTDIR="$HOME/install" source versions + export REFERENCE_DATETIME export LIBRARY_PATH="$INSTDIR/lib" export TZ=UTC export LC_ALL=C export TORBROWSER_VERSION=`cat bare-version` umask 0022 - # Building libfaketime - cd faketime - make - DESTDIR="$INSTDIR/faketime" make install - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME=$REFERENCE_DATETIME - export FAKETIME_SKIP_CMDS="rsync" - cd .. - mkdir -p $OUTDIR/ # When we build with MULTI_LINGUAL=1, the browser will be packaged inside a # directory named tor-browser (instead of tor-browser_en-US). Therefore we @@ -192,6 +182,7 @@ script: | cp defaults/preferences/000-tor-browser.js ~/build/ # Set the locale of the bundle. echo "pref(\"general.useragent.locale\", \"en-US\");" >> defaults/preferences/000-tor-browser.js + touch --date="$REFERENCE_DATETIME" defaults/preferences/000-tor-browser.js zip -Xm omni.ja defaults/preferences/000-tor-browser.js rm -rf defaults popd @@ -227,6 +218,7 @@ script: | # app.update.url). pushd ${PKG_DIR}/Browser/ echo ${PKG_LOCALE} > update.locale + touch --date="$REFERENCE_DATETIME" update.locale zip -Xm omni.ja update.locale popd fi @@ -259,6 +251,7 @@ script: | cp ~/build/000-tor-browser.js defaults/preferences/ # Set the locale of the bundle. echo "pref(\"general.useragent.locale\", \"$LANG\");" >> defaults/preferences/000-tor-browser.js + touch --date="$REFERENCE_DATETIME" defaults/preferences/000-tor-browser.js zip -Xm omni.ja defaults/preferences/000-tor-browser.js rm -rf defaults popd @@ -268,6 +261,7 @@ script: | # recreate precomplete file (needs to be accurate for full MAR updates). pushd tor-browser_$LANG/Browser/ echo "$LANG" > update.locale + touch --date="$REFERENCE_DATETIME" update.locale zip -Xm omni.ja update.locale rm -rf dictionaries rm -f precomplete diff --git a/gitian/descriptors/linux/gitian-firefox.yml b/gitian/descriptors/linux/gitian-firefox.yml index 22686b7..f32ee40 100644 --- a/gitian/descriptors/linux/gitian-firefox.yml +++ b/gitian/descriptors/linux/gitian-firefox.yml @@ -27,8 +27,6 @@ reference_datetime: "2000-01-01 00:00:00" remotes: - "url": "https://git.torproject.org/tor-browser.git" "dir": "tor-browser" -- "url": "https://github.com/wolfcw/libfaketime" - "dir": "faketime" - "url": "https://github.com/immunant/selfrando.git" "dir": "selfrando" files: @@ -41,6 +39,9 @@ files: script: | source versions INSTDIR="$HOME/install" + export REFERENCE_DATETIME + export CFLAGS="-frandom-seed=tor" + export CXXFLAGS="-frandom-seed=tor" export TZ=UTC export LC_ALL=C umask 0022 @@ -81,14 +82,6 @@ script: | mkdir -p $INSTDIR/Debug/Browser/components mkdir -p $INSTDIR/Debug/Browser/browser/components - # Building libfaketime - cd faketime - make - DESTDIR="$INSTDIR/faketime" make install - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME=$REFERENCE_DATETIME - cd .. - cd tor-browser mv .mozconfig-asan .mozconfig # .git and the src takes up a lot of useless space, and we need the space to build @@ -96,33 +89,16 @@ script: | find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" rm -f configure rm -f js/src/configure - # |configure| can't cope with nano seconds faked. And even if we would revert - # that feature it would hang sometimes for unknown but to libfaketime related - # reasons. - export LD_PRELOAD="" # Self-Rando wrapper export PATH="$HOME/build/selfrando/Tools/TorBrowser/tc-wrapper/:$PATH" export SELFRANDO_skip_shuffle= + export MOZ_BUILD_DATE=$(date -d "$REFERENCE_DATETIME" +%Y%m%d%H%M%S) make -f client.mk configure CONFIGURE_ARGS="--with-tor-browser-version=${TORBROWSER_VERSION} --enable-update-channel=${TORBROWSER_UPDATE_CHANNEL} --enable-bundled-fonts" - # We need libfaketime for all the timestamps e.g. written into the libraries. - # BUT we need to exclude |make build| from it. Otherwise the build fails close - # to the end, see #12461 comment 8 and later. Additionally, we need to avoid - # breaking the ICU compilation. Exlcuding |bash| helps here. See #12461 - # comment 13. Finally, we need to exclude |python2.7| as not doing so would - # stall the build right at the beginning. See #13877. - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME_SKIP_CMDS="python2.7,bash,make" find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" # Without disabling LSan our build is blowing up: # https://bugs.torproject.org/10599#comment:52 ASAN_OPTIONS="detect_leaks=0" make $MAKEOPTS -f client.mk build - # Packaging is broken with libfaketime enabled, thus we disable it again. See - # #12461 comments 11 and 12 for details. - export LD_PRELOAD="" make -C obj-* package INNER_MAKE_PACKAGE=true - # Without libfaketime enabled we would get different omni.ja and *debug.zip - # files. - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" cp -a obj-*/dist/firefox/* $INSTDIR/Browser/ # Remove firefox-bin (we don't use it, see ticket #10126) rm -f $INSTDIR/Browser/firefox-bin diff --git a/gitian/descriptors/linux/gitian-pluggable-transports.yml b/gitian/descriptors/linux/gitian-pluggable-transports.yml index a3c994d..2793a76 100644 --- a/gitian/descriptors/linux/gitian-pluggable-transports.yml +++ b/gitian/descriptors/linux/gitian-pluggable-transports.yml @@ -6,7 +6,6 @@ suites: architectures: - "amd64" packages: -- "faketime" - "unzip" - "python-setuptools" - "python-dev" @@ -56,7 +55,7 @@ script: | INSTDIR="$HOME/install" PTDIR="$INSTDIR/Tor/PluggableTransports" mkdir -p $PTDIR - export FAKETIME=$REFERENCE_DATETIME + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C export PYTHON=python2 @@ -261,7 +260,6 @@ script: | cd ../.. # Grabbing the results and making sure timestamps don't spoil them - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 cd $INSTDIR ~/build/dzip.sh pluggable-transports-linux$GBUILD_BITS-gbuilt.zip Tor/ Docs/ cp pluggable-transports-linux$GBUILD_BITS-gbuilt.zip $OUTDIR/ diff --git a/gitian/descriptors/linux/gitian-tor.yml b/gitian/descriptors/linux/gitian-tor.yml index f8f83e9..fa0aead 100644 --- a/gitian/descriptors/linux/gitian-tor.yml +++ b/gitian/descriptors/linux/gitian-tor.yml @@ -6,7 +6,6 @@ suites: architectures: - "amd64" packages: -- "faketime" - "unzip" - "automake" - "libtool" @@ -28,7 +27,7 @@ files: script: | INSTDIR="$HOME/install" source versions - export FAKETIME=$REFERENCE_DATETIME + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 @@ -91,7 +90,6 @@ script: | done # Grabbing the results and making sure timestamps don't spoil them - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 ~/build/dzip.sh tor-linux$GBUILD_BITS-gbuilt.zip Data/ Tor/ ~/build/dzip.sh tor-linux$GBUILD_BITS-debug.zip Debug/ cp tor-linux$GBUILD_BITS-gbuilt.zip $OUTDIR/ diff --git a/gitian/descriptors/linux/gitian-utils.yml b/gitian/descriptors/linux/gitian-utils.yml index 15b26f4..2db8739 100644 --- a/gitian/descriptors/linux/gitian-utils.yml +++ b/gitian/descriptors/linux/gitian-utils.yml @@ -6,7 +6,6 @@ suites: architectures: - "amd64" packages: -- "faketime" - "automake" - "libtool" - "zip" @@ -36,6 +35,7 @@ files: - "binutils.tar.bz2" - "gcc.tar.bz2" - "openssl.tar.gz" +- "openssl-Make-build-reproducible.patch" - "gmp.tar.bz2" - "versions" - "dzip.sh" @@ -46,7 +46,7 @@ script: | source versions export TZ=UTC export LC_ALL=C - export FAKETIME=$REFERENCE_DATETIME + export REFERENCE_DATETIME umask 0022 # Config options for hardening-wrapper @@ -78,10 +78,6 @@ script: | # anymore. It seems it got audited for those problems already: # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48817. export DEB_BUILD_HARDENING_FORMAT=0 - # libfaketime gets into our way when building GCC 4.9.x. See: - # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61314 for details. Thus, we - # avoid it for the toolchain and cross our fingers. - # TODO: Test a newer libfaketime than 0.8. # Building GCC tar xjf gcc.tar.bz2 cd gcc-* @@ -127,9 +123,9 @@ script: | cd .. # Building OpenSSL - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 tar xzf openssl.tar.gz cd openssl-* + patch -p1 < ../openssl-Make-build-reproducible.patch find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" ./config -shared --prefix=$INSTDIR/openssl enable-ec_nistp_64_gcc_128 # Using $MAKEOPTS breaks the build. Might be the issue mentioned on @@ -138,8 +134,6 @@ script: | make install cd .. - export LD_PRELOAD="" - # Building GMP tar xjf gmp.tar.bz2 cd gmp-* @@ -154,7 +148,10 @@ script: | cd .. # Grabbing the remaining results and making sure timestamps don't spoil them - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 + # Since we stopped using libfaketime, the binutils, gcc, openssl, + # libevent archives are no longer reproducible. The main reason + # is that they include some .a archives which include timestamps. + # Those files are however not part of the files we ship. cd $INSTDIR ~/build/dzip.sh binutils-$BINUTILS_VER-linux$GBUILD_BITS-utils.zip binutils ~/build/dzip.sh gcc-$GCC_VER-linux$GBUILD_BITS-utils.zip gcc diff --git a/gitian/descriptors/mac/gitian-firefox.yml b/gitian/descriptors/mac/gitian-firefox.yml index 17c830f..302f2a4 100644 --- a/gitian/descriptors/mac/gitian-firefox.yml +++ b/gitian/descriptors/mac/gitian-firefox.yml @@ -13,8 +13,6 @@ reference_datetime: "2000-01-01 00:00:00" remotes: - "url": "https://git.torproject.org/tor-browser.git" "dir": "tor-browser" -- "url": "https://github.com/wolfcw/libfaketime" - "dir": "faketime" files: - "gcc-linux64-precise-utils.zip" - "MacOSX10.7.sdk.tar.gz" @@ -22,11 +20,11 @@ files: - "re-dzip.sh" - "dzip.sh" - "fix-info-plist.py" -- "libfaketime.patch" - "versions" script: | INSTDIR="$HOME/install/" source versions + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 @@ -67,20 +65,9 @@ script: | find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" rm -f configure rm -f js/src/configure - # |configure| can't cope with nano seconds faked. And even if we would revert - # that feature it would hang sometimes for unknown but to libfaketime related - # reasons. - export LD_PRELOAD="" + export MOZ_BUILD_DATE=$(date -d "$REFERENCE_DATETIME" +%Y%m%d%H%M%S) make -f client.mk configure CONFIGURE_ARGS="--with-tor-browser-version=${TORBROWSER_VERSION} --enable-update-channel=${TORBROWSER_UPDATE_CHANNEL} --enable-bundled-fonts" find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - # We need libfaketime for all the timestamps e.g. written into the libraries. - # BUT we need to exclude |make build| from it. Otherwise the build fails close - # to the end, see #12812 comment 6 and #12461 comment 8 and later. - # Additionally, we need to exclude |rsync| due to #10153 which is reproducible - # reliably with the new libfaketime and |python2.7| as well as the build would - # stall otherwise right at the beginning. See #13877 for details. - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME_SKIP_CMDS="python2.7,rsync,make" make $MAKEOPTS -f client.mk build # make -C obj-macos package INNER_MAKE_PACKAGE=true diff --git a/gitian/descriptors/mac/gitian-pluggable-transports.yml b/gitian/descriptors/mac/gitian-pluggable-transports.yml index 5c7ecac..d6a71f2 100644 --- a/gitian/descriptors/mac/gitian-pluggable-transports.yml +++ b/gitian/descriptors/mac/gitian-pluggable-transports.yml @@ -5,7 +5,6 @@ suites: architectures: - "i386" packages: -- "faketime" - "unzip" - "python-setuptools" - "python-dev" @@ -57,8 +56,7 @@ script: | TBDIR="$INSTDIR/TorBrowserBundle.app/TorBrowser" PTDIR="$TBDIR/Tor/PluggableTransports" export LIBRARY_PATH="$INSTDIR/lib" - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 - export FAKETIME=$REFERENCE_DATETIME + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 diff --git a/gitian/descriptors/mac/gitian-utils.yml b/gitian/descriptors/mac/gitian-utils.yml index d457fef..10cc0f3 100644 --- a/gitian/descriptors/mac/gitian-utils.yml +++ b/gitian/descriptors/mac/gitian-utils.yml @@ -6,7 +6,6 @@ architectures: - "i386" - "amd64" packages: -- "faketime" - "automake" - "libtool" - "zip" @@ -16,20 +15,25 @@ reference_datetime: "2000-01-01 00:00:00" remotes: - "url": "https://github.com/libevent/libevent.git" "dir": "libevent" -- "url": "https://github.com/wolfcw/libfaketime" - "dir": "faketime" +- "url": "https://github.com/llvm-mirror/llvm" + "dir": "llvm" +- "url": "https://github.com/llvm-mirror/clang" + "dir": "clang" +- "url": "https://cmake.org/cmake.git" + "dir": "cmake" files: - "apple-uni-sdk-10.6_20110407-0.flosoft1_i386.deb" - "multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120724.tar.xz" - "gcc.tar.bz2" - "openssl.tar.gz" +- "openssl-Make-build-reproducible.patch" - "gmp.tar.bz2" - "versions" - "dzip.sh" -- "libfaketime.patch" script: | INSTDIR="$HOME/install" source versions + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 @@ -59,6 +63,15 @@ script: | export FAKETIME=$REFERENCE_DATETIME cd .. + # Since we stopped using libfaketime, the clang archive is no longer + # reproducible. The reason is that it includes some .a archives and + # other files which include timestamps. + # Those files are however not part of the files we ship. + ~/build/dzip.sh clang-$CLANG_VER-linux64-wheezy-utils.zip clang + cp *utils.zip $OUTDIR/ + else + # dpkg requires sbin directories in the PATH + export PATH="/usr/sbin:/sbin:$PATH" sudo dpkg -i *.deb tar xaf multiarch-darwin*tar.xz export PATH="$PATH:$HOME/build/apple-osx/bin/" @@ -77,7 +90,6 @@ script: | ./autogen.sh find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" ./configure --disable-static --host=i686-apple-darwin11 --prefix=$INSTDIR/libevent - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" make $MAKEOPTS make install cd .. @@ -85,6 +97,7 @@ script: | # Building OpenSSL tar xzf openssl.tar.gz cd openssl-* + patch -p1 < ../openssl-Make-build-reproducible.patch find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" # TODO: Add enable-ec_nistp_64_gcc_128 for 64bit OS X. ./Configure --cross-compile-prefix=i686-apple-darwin11- $CFLAGS darwin64-x86_64-cc --prefix=$INSTDIR/openssl enable-ec_nistp_64_gcc_128 @@ -97,20 +110,19 @@ script: | # Building GMP tar xjf gmp.tar.bz2 cd gmp-* - # |configure| can't cope with nano seconds faked. And even if we would revert - # that feature it would hang sometimes for unknown but to libfaketime related - # reasons. - export LD_PRELOAD="" find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" # Even if we are not shipping libgmpxx anymore we still need --enable-xcc # during compile time. ./configure --host=x86_64-apple-darwin11 --prefix=$INSTDIR/gmp --disable-static --enable-shared --enable-cxx - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" make make install cd .. # Grabbing the results + # Since we stopped using libfaketime, the openssl archive is no + # longer reproducible. The main reason is that it includes some .a + # archives which include timestamps. + # Those files are however not part of the files we ship. cd $INSTDIR ~/build/dzip.sh openssl-$OPENSSL_VER-mac64-utils.zip openssl ~/build/dzip.sh libevent-${LIBEVENT_TAG#release-}-mac64-utils.zip libevent diff --git a/gitian/descriptors/windows/gitian-firefox.yml b/gitian/descriptors/windows/gitian-firefox.yml index 2686c5f..3261199 100644 --- a/gitian/descriptors/windows/gitian-firefox.yml +++ b/gitian/descriptors/windows/gitian-firefox.yml @@ -12,8 +12,6 @@ reference_datetime: "2000-01-01 00:00:00" remotes: - "url": "https://git.torproject.org/tor-browser.git" "dir": "tor-browser" -- "url": "https://github.com/wolfcw/libfaketime" - "dir": "faketime" files: - "gcc-linux32-precise-utils.zip" - "mingw-w64-win32-utils.zip" @@ -28,6 +26,7 @@ files: script: | INSTDIR="$HOME/install" source versions + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 @@ -44,14 +43,6 @@ script: | export LD_LIBRARY_PATH=$INSTDIR/gcc/lib export PATH=$INSTDIR/mingw-w64/bin:$INSTDIR/gcc/bin:$PATH - # Building libfaketime - cd faketime - make - DESTDIR="$INSTDIR/faketime" make install - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME=$REFERENCE_DATETIME - cd .. - # We don't want to link against msvcrt.dll due to bug 9084. i686-w64-mingw32-g++ -dumpspecs > msvcr100.spec sed 's/msvcrt/msvcr100/' -i msvcr100.spec @@ -82,10 +73,7 @@ script: | find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" rm -f configure rm -f js/src/configure - # |configure| can't cope with nano seconds faked. And even if we would revert - # that feature it would hang sometimes for unknown but to libfaketime related - # reasons. - export LD_PRELOAD="" + export MOZ_BUILD_DATE=$(date -d "$REFERENCE_DATETIME" +%Y%m%d%H%M%S) make -f client.mk configure CONFIGURE_ARGS="--with-tor-browser-version=${TORBROWSER_VERSION} --enable-update-channel=${TORBROWSER_UPDATE_CHANNEL} --enable-bundled-fonts" find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" # @@ -93,13 +81,6 @@ script: | cp ~/build/i686* ~/build/bin/ export PATH=~/build/bin:$PATH # - # We need libfaketime for all the timestamps e.g. written into the libraries. - # BUT we need to exclude |make build| from it. Otherwise the build fails close - # to the end, see #12811 comment 14 and #12461 comment 8 and later. - # Additionally, we need to exclude |python2.7| as well as the build would - # stall otherwise right at the beginning. See #13877 for details. - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - export FAKETIME_SKIP_CMDS="python2.7,make" make $MAKEOPTS -f client.mk build # make -C obj-* package INNER_MAKE_PACKAGE=true diff --git a/gitian/descriptors/windows/gitian-utils.yml b/gitian/descriptors/windows/gitian-utils.yml index ae938af..9296724 100644 --- a/gitian/descriptors/windows/gitian-utils.yml +++ b/gitian/descriptors/windows/gitian-utils.yml @@ -5,7 +5,6 @@ suites: architectures: - "i386" packages: -- "faketime" - "automake" - "libtool" - "zip" @@ -25,6 +24,7 @@ files: - "binutils.tar.bz2" - "gcc.tar.bz2" - "openssl.tar.gz" +- "openssl-Make-build-reproducible.patch" - "gmp.tar.bz2" - "enable-reloc-section-ld.patch" - "peXXigen.patch" @@ -33,6 +33,7 @@ files: script: | INSTDIR="$HOME/install" source versions + export REFERENCE_DATETIME export TZ=UTC export LC_ALL=C umask 0022 @@ -109,13 +110,6 @@ script: | cp i686-w64-mingw32/libgcc/shlib/libgcc_s_sjlj-1.dll $INSTDIR/gcclibs cd .. - # XXX: Build the libraries we include into the bundles deterministically. As - # libfaketime breaks the mingw-w64 build (probably due to bug 11459) we omit - # the compiler and linker from it. It seems we get away with this strategy - # and the libgcc* and libss* which we ship, too, are still built in a - # reproducible fashion. - export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 - export FAKETIME=$REFERENCE_DATETIME # Building zlib export CFLAGS="-mwindows -fstack-protector-all -Wstack-protector --param ssp-buffer-size=4 -fno-strict-overflow -Wno-missing-field-initializers -Wformat -Wformat-security" export LDFLAGS="-mwindows -Wl,--dynamicbase -Wl,--nxcompat -Wl,--enable-reloc-section -lssp -L$INSTDIR/gcclibs/" @@ -137,6 +131,7 @@ script: | # Building OpenSSL tar xzf openssl.tar.gz cd openssl-* + patch -p1 < ../openssl-Make-build-reproducible.patch find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" # TODO: Add enable-ec_nistp_64_gcc_128 for 64bit Windows. ./Configure -shared --cross-compile-prefix=i686-w64-mingw32- mingw "-fstack-protector-all -Wstack-protector --param ssp-buffer-size=4 -fno-strict-overflow -Wno-missing-field-initializers -Wformat -Wformat-security -Wl,--dynamicbase -Wl,--nxcompat -Wl,--enable-reloc-section -lssp -L$INSTDIR/gcclibs/" --prefix=$INSTDIR/openssl @@ -158,6 +153,10 @@ script: | cd .. # Grabbing the remaining results + # Since we stopped using libfaketime, the gcc, gmp, zlib, openssl, + # libevent, mingw-w64 archives are no longer reproducible. The main + # reason is that they include some .a archives which include timestamps. + # Those files are however not part of the files we ship. cd $INSTDIR # We might want to bump binutils independent of bumping mingw-w64. touch binutils-$BINUTILS_VER-win32-utils.zip diff --git a/gitian/patches/openssl-Make-build-reproducible.patch b/gitian/patches/openssl-Make-build-reproducible.patch new file mode 100644 index 0000000..d86f5ba --- /dev/null +++ b/gitian/patches/openssl-Make-build-reproducible.patch @@ -0,0 +1,28 @@ +From b88c021b5a7c539f821b7b7c47c72138cc3c3271 Mon Sep 17 00:00:00 2001 +From: Kurt Roeckx <kurt(a)roeckx.be> +Date: Fri, 2 Jan 2015 12:27:57 +0100 +Subject: [PATCH] Make build reproducible + +It contained a date on when it was build. + +Reviewed-by: Rich Salz <rsalz(a)openssl.org> +--- + crypto/cversion.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/crypto/cversion.c b/crypto/cversion.c +index 9e6f50d78182..c417d1d1121f 100644 +--- a/crypto/cversion.c ++++ b/crypto/cversion.c +@@ -68,7 +68,11 @@ const char *SSLeay_version(int t) + return OPENSSL_VERSION_TEXT; + if (t == SSLEAY_BUILT_ON) { + #ifdef DATE ++# ifdef OPENSSL_USE_BUILD_DATE + return (DATE); ++# else ++ return("built on: reproducible build, date unspecified"); ++# endif + #else + return ("built on: date not available"); + #endif

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

tbb-commits June 2016