tbb-commits May 2023

tbb-commits@lists.torproject.org

1 participants
66 discussions

[Git][tpo/applications/tor-browser][tor-browser-102.11.0esr-12.5-1] fixup! Firefox preference overrides.
by richard (＠richard) 18 May '23

18 May '23

richard pushed to branch tor-browser-102.11.0esr-12.5-1 at The Tor Project / Applications / Tor Browser Commits: 2970a44d by hackademix at 2023-05-18T19:30:57+00:00 fixup! Firefox preference overrides. - - - - - 1 changed file: - browser/app/profile/001-base-profile.js Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -48,6 +48,9 @@ pref("security.nocertdb", true); pref("browser.download.useDownloadDir", false); pref("browser.download.manager.addToRecentDocs", false); +// Prevent download stuffing / DOS (tor-browser#41764) +pref("browser.download.enable_spam_prevention", true); + // Misc privacy: Disk pref("signon.rememberSignons", false); pref("browser.formfill.enable", false); View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/2970a44… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/2970a44… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-102.11.0esr-13.0-1] 12 commits: fixup! Bug 40933: Add tor-launcher functionality
by Pier Angelo Vendrame (＠pierov) 17 May '23

17 May '23

Pier Angelo Vendrame pushed to branch tor-browser-102.11.0esr-13.0-1 at The Tor Project / Applications / Tor Browser Commits: 98e75e48 by Pier Angelo Vendrame at 2023-05-17T10:25:44+02:00 fixup! Bug 40933: Add tor-launcher functionality Added a newnym function - - - - - 74e39196 by Pier Angelo Vendrame at 2023-05-17T10:25:52+02:00 fixup! Bug 10760: Integrate TorButton to TorBrowser core Bug 40938: Moving the domain isolator out of torbutton - - - - - 0f9ea290 by Arthur Edelstein at 2023-05-17T10:25:53+02:00 Bug 3455: Add DomainIsolator, for isolating circuit by domain. Add an XPCOM component that registers a ProtocolProxyChannelFilter which sets the username/password for each web request according to url bar domain. Bug 9442: Add New Circuit button Bug 13766: Set a 10 minute circuit dirty timeout for the catch-all circ. Bug 19206: Include a 128 bit random tag as part of the domain isolator nonce. Bug 19206: Clear out the domain isolator state on `New Identity`. Bug 21201.2: Isolate by firstPartyDomain from OriginAttributes Bug 21745: Fix handling of catch-all circuit Bug 41741: Refactor the domain isolator and new circuit - - - - - 14058280 by Pier Angelo Vendrame at 2023-05-17T10:25:53+02:00 fixup! Bug 3455: Add DomainIsolator, for isolating circuit by domain. Refactors to the old JS code. - - - - - c94c9662 by Pier Angelo Vendrame at 2023-05-17T10:25:53+02:00 fixup! Bug 3455: Add DomainIsolator, for isolating circuit by domain. Manage NEWNYM here. - - - - - 994e4ce2 by Pier Angelo Vendrame at 2023-05-17T10:25:54+02:00 fixup! Bug 3455: Add DomainIsolator, for isolating circuit by domain. Removed the XPCOM definition of the domain isolator. - - - - - bcabee6b by Pier Angelo Vendrame at 2023-05-17T10:25:54+02:00 fixup! Bug 10760: Integrate TorButton to TorBrowser core Extract the new identity button from torbutton - - - - - 16cf26ff by Pier Angelo Vendrame at 2023-05-17T10:36:36+02:00 fixup! Bug 3455: Add DomainIsolator, for isolating circuit by domain. Actually added the new circuit button. - - - - - 39b5273c by Pier Angelo Vendrame at 2023-05-17T10:37:31+02:00 fixup! Bug 41600: Add a tor circuit display panel. Use the new domain isolator interface. - - - - - 9cd44b23 by Pier Angelo Vendrame at 2023-05-17T10:38:45+02:00 fixup! Bug 40209: Implement Basic Crypto Safety Use the new domain isolator interface - - - - - b27d2320 by Pier Angelo Vendrame at 2023-05-17T10:38:48+02:00 fixup! Bug 10760: Integrate TorButton to TorBrowser core Remove string changes from Torbutton. We will add them back in the TorStrings commit. - - - - - 5b547f81 by Pier Angelo Vendrame at 2023-05-17T10:38:49+02:00 fixup! Add TorStrings module for localization Add our DTDs where needed. These changes were originally in the torbutton commit, but I think they are better fit here, with all the strings files. - - - - - 15 changed files: - browser/actors/CryptoSafetyParent.jsm - browser/base/content/appmenu-viewcache.inc.xhtml - browser/base/content/browser-menubar.inc - browser/base/content/browser-sets.inc - browser/base/content/browser.js - browser/base/content/navigator-toolbox.inc.xhtml - browser/components/torcircuit/content/torCircuitPanel.js - + toolkit/components/tor-launcher/TorDomainIsolator.jsm - toolkit/components/tor-launcher/TorProtocolService.jsm - toolkit/components/tor-launcher/TorStartupService.jsm - toolkit/components/tor-launcher/moz.build - toolkit/torbutton/chrome/content/torbutton.js - − toolkit/torbutton/components/domain-isolator.js - toolkit/torbutton/jar.mn - toolkit/torbutton/modules/utils.js Changes: ===================================== browser/actors/CryptoSafetyParent.jsm ===================================== @@ -12,6 +12,12 @@ const { XPCOMUtils } = ChromeUtils.import( "resource://gre/modules/XPCOMUtils.jsm" ); +ChromeUtils.defineModuleGetter( + this, + "TorDomainIsolator", + "resource://gre/modules/TorDomainIsolator.jsm" +); + XPCOMUtils.defineLazyGetter(this, "cryptoSafetyBundle", () => { return Services.strings.createBundle( "chrome://browser/locale/cryptoSafetyPrompt.properties" @@ -75,7 +81,11 @@ class CryptoSafetyParent extends JSWindowActorParent { ); if (buttonPressed === 0) { - this.browsingContext.topChromeWindow.torbutton_new_circuit(); + const { browsingContext } = this.manager; + const browser = browsingContext.embedderElement; + if (browser) { + TorDomainIsolator.newCircuitForBrowser(browser.ownerGlobal.gBrowser); + } } } } ===================================== browser/base/content/appmenu-viewcache.inc.xhtml ===================================== @@ -63,9 +63,9 @@ key="new-identity-key"/> <toolbarbutton id="appMenuNewCircuit" class="subviewbutton" - key="torbutton-new-circuit-key" + key="new-circuit-key" label="&torbutton.context_menu.new_circuit_sentence_case;" - oncommand="torbutton_new_circuit();"/> + oncommand="TorDomainIsolator.newCircuitForBrowser(gBrowser);"/> <toolbarseparator/> <toolbarbutton id="appMenu-bookmarks-button" class="subviewbutton subviewbutton-nav" ===================================== browser/base/content/browser-menubar.inc ===================================== @@ -33,9 +33,9 @@ key="new-identity-key"/> <menuitem id="menu_newCircuit" accesskey="&torbutton.context_menu.new_circuit_key;" - key="torbutton-new-circuit-key" + key="new-circuit-key" label="&torbutton.context_menu.new_circuit;" - oncommand="torbutton_new_circuit();"/> + oncommand="TorDomainIsolator.newCircuitForBrowser(gBrowser);"/> <menuseparator/> <menuitem id="menu_openLocation" hidden="true" ===================================== browser/base/content/browser-sets.inc ===================================== @@ -389,5 +389,5 @@ internal="true"/> #endif <key id="new-identity-key" modifiers="accel shift" key="U" oncommand="NewIdentityButton.onCommand(event)"/> - <key id="torbutton-new-circuit-key" modifiers="accel shift" key="L" oncommand="torbutton_new_circuit()"/> + <key id="new-circuit-key" modifiers="accel shift" key="L" oncommand="TorDomainIsolator.newCircuitForBrowser(gBrowser)"/> </keyset> ===================================== browser/base/content/browser.js ===================================== @@ -80,6 +80,7 @@ XPCOMUtils.defineLazyModuleGetters(this, { TabCrashHandler: "resource:///modules/ContentCrashHandlers.jsm", TelemetryEnvironment: "resource://gre/modules/TelemetryEnvironment.jsm", TorConnect: "resource:///modules/TorConnect.jsm", + TorDomainIsolator: "resource://gre/modules/TorDomainIsolator.jsm", Translation: "resource:///modules/translation/TranslationParent.jsm", UITour: "resource:///modules/UITour.jsm", UpdateUtils: "resource://gre/modules/UpdateUtils.jsm", ===================================== browser/base/content/navigator-toolbox.inc.xhtml ===================================== @@ -557,7 +557,7 @@ <toolbarbutton id="new-circuit-button" class="toolbarbutton-1 chromeclass-toolbar-additional" label="&torbutton.context_menu.new_circuit;" - oncommand="torbutton_new_circuit();" + oncommand="TorDomainIsolator.newCircuitForBrowser(gBrowser);" tooltiptext="&torbutton.context_menu.new_circuit;"/> <toolbarbutton id="fullscreen-button" class="toolbarbutton-1 chromeclass-toolbar-additional" ===================================== browser/components/torcircuit/content/torCircuitPanel.js ===================================== @@ -193,7 +193,7 @@ var gTorCircuitPanel = { document .getElementById("tor-circuit-new-circuit") .addEventListener("command", () => { - torbutton_new_circuit(); + TorDomainIsolator.newCircuitForBrowser(gBrowser); // And hide. // NOTE: focus should return to the toolbar button, which we expect to // remain visible during reload. @@ -415,20 +415,14 @@ var gTorCircuitPanel = { */ _updateCurrentBrowser(matchingCredentials = null) { const browser = gBrowser.selectedBrowser; - const { getDomainForBrowser } = ChromeUtils.import( - "resource://torbutton/modules/utils.js" - ); - const domain = getDomainForBrowser(browser); + const domain = TorDomainIsolator.getDomainForBrowser(browser); // We choose the currentURI, which matches what is shown in the URL bar and // will match up with the domain. // In contrast, documentURI corresponds to the shown page. E.g. it could // point to "about:certerror". const scheme = browser.currentURI?.scheme; - const domainIsolator = Cc["@torproject.org/domain-isolator;1"].getService( - Ci.nsISupports - ).wrappedJSObject; - let credentials = domainIsolator.getSocksProxyCredentials( + let credentials = TorDomainIsolator.getSocksProxyCredentials( domain, browser.contentPrincipal.originAttributes.userContextId ); ===================================== toolkit/components/tor-launcher/TorDomainIsolator.jsm ===================================== @@ -0,0 +1,362 @@ +// A component for Tor Browser that puts requests from different +// first party domains on separate Tor circuits. + +var EXPORTED_SYMBOLS = ["TorDomainIsolator"]; + +const { Services } = ChromeUtils.import("resource://gre/modules/Services.jsm"); +const { XPCOMUtils } = ChromeUtils.import( + "resource://gre/modules/XPCOMUtils.jsm" +); +const { ConsoleAPI } = ChromeUtils.import("resource://gre/modules/Console.jsm"); + +Cu.importGlobalProperties(["crypto"]); + +XPCOMUtils.defineLazyServiceGetters(this, { + ProtocolProxyService: [ + "@mozilla.org/network/protocol-proxy-service;1", + "nsIProtocolProxyService", + ], +}); + +ChromeUtils.defineModuleGetter( + this, + "TorProtocolService", + "resource://gre/modules/TorProtocolService.jsm" +); + +const logger = new ConsoleAPI({ + prefix: "TorDomainIsolator", + maxLogLevel: "warn", + maxLogLevelPref: "browser.tordomainisolator.loglevel", +}); + +// The string to use instead of the domain when it is not known. +const CATCHALL_DOMAIN = "--unknown--"; + +// The preference to observe, to know whether isolation should be enabled or +// disabled. +const NON_TOR_PROXY_PREF = "extensions.torbutton.use_nontor_proxy"; + +// The topic of new identity, to observe to cleanup all the nonces. +const NEW_IDENTITY_TOPIC = "new-identity-requested"; + +class TorDomainIsolatorImpl { + // A mutable map that records what nonce we are using for each domain. + #noncesForDomains = new Map(); + + // A mutable map that records what nonce we are using for each tab container. + #noncesForUserContextId = new Map(); + + // A bool that controls if we use SOCKS auth for isolation or not. + #isolationEnabled = true; + + // Specifies when the current catch-all circuit was first used + #catchallDirtySince = Date.now(); + + /** + * Initialize the domain isolator. + * This function will setup the proxy filter that injects the credentials and + * register some observers. + */ + init() { + logger.info("Setup circuit isolation by domain and user context"); + + if (Services.prefs.getBoolPref(NON_TOR_PROXY_PREF)) { + this.#isolationEnabled = false; + } + this.#setupProxyFilter(); + + Services.prefs.addObserver(NON_TOR_PROXY_PREF, this); + Services.obs.addObserver(this, NEW_IDENTITY_TOPIC); + } + + /** + * Removes the observers added in the initialization. + */ + uninit() { + Services.prefs.removeObserver(NON_TOR_PROXY_PREF, this); + Services.obs.removeObserver(this, NEW_IDENTITY_TOPIC); + } + + enable() { + logger.trace("Domain isolation enabled"); + this.#isolationEnabled = true; + } + + disable() { + logger.trace("Domain isolation disabled"); + this.#isolationEnabled = false; + } + + /** + * Return the credentials to use as username and password for the SOCKS proxy, + * given a certain domain and userContextId. Optionally, create them. + * + * @param firstPartyDomain The first party domain associated to the requests + * @param userContextId The context ID associated to the request + * @param create Whether to create the nonce, if it is not available + * @return Either the credential, or null if we do not have them and create is + * false. + */ + getSocksProxyCredentials(firstPartyDomain, userContextId, create = false) { + if (!this.#noncesForDomains.has(firstPartyDomain)) { + if (!create) { + return null; + } + const nonce = this.#nonce(); + logger.info(`New nonce for first party ${firstPartyDomain}: ${nonce}`); + this.#noncesForDomains.set(firstPartyDomain, nonce); + } + if (!this.#noncesForUserContextId.has(userContextId)) { + if (!create) { + return null; + } + const nonce = this.#nonce(); + logger.info(`New nonce for userContextId ${userContextId}: ${nonce}`); + this.#noncesForUserContextId.set(userContextId, nonce); + } + return { + username: this.#makeUsername(firstPartyDomain, userContextId), + password: + this.#noncesForDomains.get(firstPartyDomain) + + this.#noncesForUserContextId.get(userContextId), + }; + } + + /** + * Create a new nonce for the FP domain of the selected browser and reload the + * tab with a new circuit. + * + * @param browser Should be the gBrowser from the context of the caller + */ + newCircuitForBrowser(browser) { + const firstPartyDomain = getDomainForBrowser(browser.selectedBrowser); + this.#newCircuitForDomain(firstPartyDomain); + // TODO: How to properly handle the user context? Should we use + // (domain, userContextId) pairs, instead of concatenating nonces? + browser.reloadWithFlags(Ci.nsIWebNavigation.LOAD_FLAGS_BYPASS_CACHE); + } + + /** + * Clear the isolation state cache, forcing new circuits to be used for all + * subsequent requests. + */ + clearIsolation() { + logger.trace("Clearing isolation nonces."); + + // Per-domain and per contextId nonces are stored in maps, so simply clear + // them. + this.#noncesForDomains.clear(); + this.#noncesForUserContextId.clear(); + + // Force a rotation on the next catch-all circuit use by setting the + // creation time to the epoch. + this.#catchallDirtySince = 0; + } + + async observe(subject, topic, data) { + if (topic === "nsPref:changed" && data === NON_TOR_PROXY_PREF) { + if (Services.prefs.getBoolPref(NON_TOR_PROXY_PREF)) { + this.disable(); + } else { + this.enable(); + } + } else if (topic === NEW_IDENTITY_TOPIC) { + logger.info( + "New identity has been requested, clearing isolation tokens." + ); + this.clearIsolation(); + try { + await TorProtocolService.newnym(); + } catch (e) { + logger.error("Could not send the newnym command", e); + // TODO: What UX to use here? See tor-browser#41708 + } + } + } + + /** + * Setup a filter that for every HTTPChannel, replaces the default SOCKS proxy + * with one that authenticates to the SOCKS server (the tor client process) + * with a username (the first party domain and userContextId) and a nonce + * password. + * Tor provides a separate circuit for each username+password combination. + */ + #setupProxyFilter() { + const filterFunction = (aChannel, aProxy) => { + if (!this.#isolationEnabled) { + return aProxy; + } + try { + const channel = aChannel.QueryInterface(Ci.nsIChannel); + let firstPartyDomain = + channel.loadInfo.originAttributes.firstPartyDomain; + const userContextId = channel.loadInfo.originAttributes.userContextId; + if (firstPartyDomain === "") { + firstPartyDomain = CATCHALL_DOMAIN; + if (Date.now() - this.#catchallDirtySince > 1000 * 10 * 60) { + logger.info( + "tor catchall circuit has been dirty for over 10 minutes. Rotating." + ); + this.#newCircuitForDomain(CATCHALL_DOMAIN); + this.#catchallDirtySince = Date.now(); + } + } + const replacementProxy = this.#applySocksProxyCredentials( + aProxy, + firstPartyDomain, + userContextId + ); + logger.debug( + `Requested ${channel.URI.spec} via ${replacementProxy.username}:${replacementProxy.password}` + ); + return replacementProxy; + } catch (e) { + logger.error("Error while setting a new proxy", e); + return null; + } + }; + + ProtocolProxyService.registerChannelFilter( + { + applyFilter(aChannel, aProxy, aCallback) { + aCallback.onProxyFilterResult(filterFunction(aChannel, aProxy)); + }, + }, + 0 + ); + } + + /** + * Takes a proxyInfo object (originalProxy) and returns a new proxyInfo + * object with the same properties, except the username is set to the + * the domain and userContextId, and the password is a nonce. + */ + #applySocksProxyCredentials(originalProxy, domain, userContextId) { + const proxy = originalProxy.QueryInterface(Ci.nsIProxyInfo); + const { username, password } = this.getSocksProxyCredentials( + domain, + userContextId, + true + ); + return ProtocolProxyService.newProxyInfoWithAuth( + "socks", + proxy.host, + proxy.port, + username, + password, + "", // aProxyAuthorizationHeader + "", // aConnectionIsolationKey + proxy.flags, + proxy.failoverTimeout, + proxy.failoverProxy + ); + } + + /** + * Combine the needed data into a username for the proxy. + */ + #makeUsername(domain, userContextId) { + if (!domain) { + domain = CATCHALL_DOMAIN; + } + return `${domain}:${userContextId}`; + } + + /** + * Generate a new 128 bit random tag. + * + * Strictly speaking both using a cryptographic entropy source and using 128 + * bits of entropy for the tag are likely overkill, as correct behavior only + * depends on how unlikely it is for there to be a collision. + */ + #nonce() { + return Array.from(crypto.getRandomValues(new Uint8Array(16)), byte => + byte.toString(16).padStart(2, "0") + ).join(""); + } + + /** + * Re-generate the nonce for a certain domain. + */ + #newCircuitForDomain(domain) { + if (!domain) { + domain = CATCHALL_DOMAIN; + } + this.#noncesForDomains.set(domain, this.#nonce()); + logger.info( + `New domain isolation for ${domain}: ${this.#noncesForDomains.get( + domain + )}` + ); + } + + /** + * Re-generate the nonce for a userContextId. + * + * Currently, this function is not hooked to anything. + */ + #newCircuitForUserContextId(userContextId) { + this.#noncesForUserContextId.set(userContextId, this.#nonce()); + logger.info( + `New container isolation for ${userContextId}: ${this.#noncesForUserContextId.get( + userContextId + )}` + ); + } +} + +/** + * Get the first party domain for a certain browser. + * + * @param browser The browser to get the FP-domain for. + * + * Please notice that it should be gBrowser.selectedBrowser, because + * browser.documentURI is the actual shown page, and might be an error page. + * In this case, we rely on currentURI, which for gBrowser is an alias of + * gBrowser.selectedBrowser.currentURI. + * See browser/base/content/tabbrowser.js and tor-browser#31562. + */ +function getDomainForBrowser(browser) { + let fpd = browser.contentPrincipal.originAttributes.firstPartyDomain; + + // Bug 31562: For neterror or certerror, get the original URL from + // browser.currentURI and use it to calculate the firstPartyDomain. + const knownErrors = [ + "about:neterror", + "about:certerror", + "about:httpsonlyerror", + ]; + const { documentURI } = browser; + if ( + documentURI && + documentURI.schemeIs("about") && + knownErrors.some(x => documentURI.spec.startsWith(x)) + ) { + const knownSchemes = ["http", "https"]; + const currentURI = browser.currentURI; + if (currentURI && knownSchemes.some(x => currentURI.schemeIs(x))) { + try { + fpd = Services.eTLD.getBaseDomainFromHost(currentURI.host); + } catch (e) { + if ( + e.result === Cr.NS_ERROR_HOST_IS_IP_ADDRESS || + e.result === Cr.NS_ERROR_INSUFFICIENT_DOMAIN_LEVELS + ) { + fpd = currentURI.host; + } else { + logger.error( + `Failed to get first party domain for host ${currentURI.host}`, + e + ); + } + } + } + } + + return fpd; +} + +const TorDomainIsolator = new TorDomainIsolatorImpl(); +// Reduce global vars pollution +TorDomainIsolator.getDomainForBrowser = getDomainForBrowser; ===================================== toolkit/components/tor-launcher/TorProtocolService.jsm ===================================== @@ -4,6 +4,7 @@ var EXPORTED_SYMBOLS = ["TorProtocolService"]; +const { ConsoleAPI } = ChromeUtils.import("resource://gre/modules/Console.jsm"); const { Services } = ChromeUtils.import("resource://gre/modules/Services.jsm"); const { setTimeout } = ChromeUtils.import("resource://gre/modules/Timer.jsm"); ChromeUtils.defineModuleGetter( @@ -11,9 +12,6 @@ ChromeUtils.defineModuleGetter( "FileUtils", "resource://gre/modules/FileUtils.jsm" ); -const { XPCOMUtils } = ChromeUtils.import( - "resource://gre/modules/XPCOMUtils.jsm" -); Cu.importGlobalProperties(["crypto"]); @@ -45,18 +43,9 @@ const TorTopics = Object.freeze({ ProcessRestarted: "TorProcessRestarted", }); -// Logger adapted from CustomizableUI.jsm -XPCOMUtils.defineLazyGetter(this, "logger", () => { - const { ConsoleAPI } = ChromeUtils.import( - "resource://gre/modules/Console.jsm" - ); - // TODO: Use a preference to set the log level. - const consoleOptions = { - // maxLogLevel: "warn", - maxLogLevel: "all", - prefix: "TorProtocolService", - }; - return new ConsoleAPI(consoleOptions); +const logger = new ConsoleAPI({ + maxLogLevel: "warn", + prefix: "TorProtocolService", }); // Manage the connection to tor's control port, to update its settings and query @@ -194,6 +183,10 @@ const TorProtocolService = { TorMonitorService.retrieveBootstrapStatus(); }, + async newnym() { + return this.sendCommand("SIGNAL NEWNYM"); + }, + // TODO: transform the following 4 functions in getters. At the moment they // are also used in torbutton. ===================================== toolkit/components/tor-launcher/TorStartupService.jsm ===================================== @@ -33,6 +33,12 @@ ChromeUtils.defineModuleGetter( "resource:///modules/TorSettings.jsm" ); +ChromeUtils.defineModuleGetter( + this, + "TorDomainIsolator", + "resource://gre/modules/TorDomainIsolator.jsm" +); + /* Browser observer topis */ const BrowserTopics = Object.freeze({ ProfileAfterChange: "profile-after-change", @@ -67,12 +73,16 @@ class TorStartupService { TorSettings.init(); TorConnect.init(); + TorDomainIsolator.init(); + gInited = true; } _uninit() { Services.obs.removeObserver(this, BrowserTopics.QuitApplicationGranted); + TorDomainIsolator.uninit(); + // Close any helper connection first... TorProtocolService.uninit(); // ... and only then closes the event monitor connection, which will cause ===================================== toolkit/components/tor-launcher/moz.build ===================================== @@ -1,5 +1,6 @@ EXTRA_JS_MODULES += [ "TorBootstrapRequest.jsm", + "TorDomainIsolator.jsm", "TorLauncherUtil.jsm", "TorMonitorService.jsm", "TorParsers.jsm", ===================================== toolkit/torbutton/chrome/content/torbutton.js ===================================== @@ -1,6 +1,5 @@ // window globals var torbutton_init; -var torbutton_new_circuit; (() => { // Bug 1506 P1-P5: This is the main Torbutton overlay file. Much needs to be @@ -16,9 +15,7 @@ var torbutton_new_circuit; let { unescapeTorString, - getDomainForBrowser, torbutton_log, - torbutton_get_property_string, } = ChromeUtils.import("resource://torbutton/modules/utils.js"); let { configureControlPortModule, wait_for_controller } = ChromeUtils.import( "resource://torbutton/modules/tor-control-port.js" @@ -46,32 +43,22 @@ var torbutton_new_circuit; // in a component, not the XUL overlay. var torbutton_unique_pref_observer = { register() { - this.forced_ua = false; - m_tb_prefs.addObserver("extensions.torbutton", this); - m_tb_prefs.addObserver("browser.privatebrowsing.autostart", this); - m_tb_prefs.addObserver("javascript", this); + Services.prefs.addObserver("browser.privatebrowsing.autostart", this); }, unregister() { - m_tb_prefs.removeObserver("extensions.torbutton", this); - m_tb_prefs.removeObserver("browser.privatebrowsing.autostart", this); - m_tb_prefs.removeObserver("javascript", this); + Services.prefs.removeObserver("browser.privatebrowsing.autostart", this); }, // topic: what event occurred // subject: what nsIPrefBranch we're observing // data: which pref has been changed (relative to subject) observe(subject, topic, data) { - if (topic !== "nsPref:changed") { - return; - } - switch (data) { - case "browser.privatebrowsing.autostart": - torbutton_update_disk_prefs(); - break; - case "extensions.torbutton.use_nontor_proxy": - torbutton_use_nontor_proxy(); - break; + if ( + topic === "nsPref:changed" && + data === "browser.privatebrowsing.autostart" + ) { + torbutton_update_disk_prefs(); } }, }; @@ -113,62 +100,6 @@ var torbutton_new_circuit; }, }; - var torbutton_new_identity_observers = { - register() { - Services.obs.addObserver(this, "new-identity-requested"); - }, - - observe(aSubject, aTopic, aData) { - if (aTopic !== "new-identity-requested") { - return; - } - - // Clear the domain isolation state. - torbutton_log(3, "Clearing domain isolator"); - const domainIsolator = Cc["@torproject.org/domain-isolator;1"].getService( - Ci.nsISupports - ).wrappedJSObject; - domainIsolator.clearIsolation(); - - torbutton_log(3, "New Identity: Sending NEWNYM"); - // We only support TBB for newnym. - if ( - !m_tb_control_pass || - (!m_tb_control_ipc_file && !m_tb_control_port) - ) { - const warning = torbutton_get_property_string( - "torbutton.popup.no_newnym" - ); - torbutton_log( - 5, - "Torbutton cannot safely newnym. It does not have access to the Tor Control Port." - ); - window.alert(warning); - } else { - const warning = torbutton_get_property_string( - "torbutton.popup.no_newnym" - ); - torbutton_send_ctrl_cmd("SIGNAL NEWNYM") - .then(res => { - if (!res) { - torbutton_log( - 5, - "Torbutton was unable to request a new circuit from Tor" - ); - window.alert(warning); - } - }) - .catch(e => { - torbutton_log( - 5, - "Torbutton was unable to request a new circuit from Tor " + e - ); - window.alert(warning); - }); - } - }, - }; - // Bug 1506 P2-P4: This code sets some version variables that are irrelevant. // It does read out some important environment variables, though. It is // called once per browser window.. This might belong in a component. @@ -258,8 +189,6 @@ var torbutton_new_circuit; true ); - torbutton_new_identity_observers.register(); - torbutton_log(3, "init completed"); }; @@ -374,36 +303,6 @@ var torbutton_new_circuit; return response; } - // Bug 1506 P4: Needed for New IP Address - torbutton_new_circuit = function() { - let firstPartyDomain = getDomainForBrowser(gBrowser.selectedBrowser); - - let domainIsolator = Cc["@torproject.org/domain-isolator;1"].getService( - Ci.nsISupports - ).wrappedJSObject; - - domainIsolator.newCircuitForDomain(firstPartyDomain); - - gBrowser.reloadWithFlags(Ci.nsIWebNavigation.LOAD_FLAGS_BYPASS_CACHE); - }; - - /* Called when we switch the use_nontor_proxy pref in either direction. - * - * Enables/disables domain isolation and then does new identity - */ - function torbutton_use_nontor_proxy() { - let domainIsolator = Cc["@torproject.org/domain-isolator;1"].getService( - Ci.nsISupports - ).wrappedJSObject; - - if (m_tb_prefs.getBoolPref("extensions.torbutton.use_nontor_proxy")) { - // Disable domain isolation - domainIsolator.disableIsolation(); - } else { - domainIsolator.enableIsolation(); - } - } - async function torbutton_do_tor_check() { let checkSvc = Cc["@torproject.org/torbutton-torCheckService;1"].getService( Ci.nsISupports ===================================== toolkit/torbutton/components/domain-isolator.js deleted ===================================== @@ -1,312 +0,0 @@ -// # domain-isolator.js -// A component for TorBrowser that puts requests from different -// first party domains on separate tor circuits. - -// This file is written in call stack order (later functions -// call earlier functions). The code file can be processed -// with docco.js to provide clear documentation. - -// ### Abbreviations - -const { Services } = ChromeUtils.import("resource://gre/modules/Services.jsm"); -const { XPCOMUtils } = ChromeUtils.import( - "resource://gre/modules/XPCOMUtils.jsm" -); - -XPCOMUtils.defineLazyModuleGetters(this, { - ComponentUtils: "resource://gre/modules/ComponentUtils.jsm", -}); - -// Make the logger available. -let logger = Cc["@torproject.org/torbutton-logger;1"].getService(Ci.nsISupports) - .wrappedJSObject; - -// Import crypto object (FF 37+). -Cu.importGlobalProperties(["crypto"]); - -// ## mozilla namespace. -// Useful functionality for interacting with Mozilla services. -let mozilla = {}; - -// __mozilla.protocolProxyService__. -// Mozilla's protocol proxy service, useful for managing proxy connections made -// by the browser. -mozilla.protocolProxyService = Cc[ - "@mozilla.org/network/protocol-proxy-service;1" -].getService(Ci.nsIProtocolProxyService); - -// __mozilla.registerProxyChannelFilter(filterFunction, positionIndex)__. -// Registers a proxy channel filter with the Mozilla Protocol Proxy Service, -// which will help to decide the proxy to be used for a given channel. -// The filterFunction should expect two arguments, (aChannel, aProxy), -// where aProxy is the proxy or list of proxies that would be used by default -// for the given channel, and should return a new Proxy or list of Proxies. -mozilla.registerProxyChannelFilter = function(filterFunction, positionIndex) { - let proxyFilter = { - applyFilter(aChannel, aProxy, aCallback) { - aCallback.onProxyFilterResult(filterFunction(aChannel, aProxy)); - }, - }; - mozilla.protocolProxyService.registerChannelFilter( - proxyFilter, - positionIndex - ); -}; - -// ## tor functionality. -let tor = {}; - -// __tor.noncesForDomains__. -// A mutable map that records what nonce we are using for each domain. -tor.noncesForDomains = new Map(); - -// __tor.noncesForUserContextId__. -// A mutable map that records what nonce we are using for each tab container. -tor.noncesForUserContextId = new Map(); - -// __tor.isolationEabled__. -// A bool that controls if we use SOCKS auth for isolation or not. -tor.isolationEnabled = true; - -// __tor.unknownDirtySince__. -// Specifies when the current catch-all circuit was first used -tor.unknownDirtySince = Date.now(); - -tor.passwordForDomainAndUserContextId = function( - domain, - userContextId, - create -) { - // Check if we already have a nonce. If not, possibly create one for this - // domain and userContextId. - if (!tor.noncesForDomains.has(domain)) { - if (!create) { - return null; - } - tor.noncesForDomains.set(domain, tor.nonce()); - } - if (!tor.noncesForUserContextId.has(userContextId)) { - if (!create) { - return null; - } - tor.noncesForUserContextId.set(userContextId, tor.nonce()); - } - return ( - tor.noncesForDomains.get(domain) + - tor.noncesForUserContextId.get(userContextId) - ); -}; - -tor.usernameForDomainAndUserContextId = function(domain, userContextId) { - return `${domain}:${userContextId}`; -}; - -// __tor.socksProxyCredentials(originalProxy, domain, userContextId)__. -// Takes a proxyInfo object (originalProxy) and returns a new proxyInfo -// object with the same properties, except the username is set to the -// the domain and userContextId, and the password is a nonce. -tor.socksProxyCredentials = function(originalProxy, domain, userContextId) { - let proxy = originalProxy.QueryInterface(Ci.nsIProxyInfo); - let proxyUsername = tor.usernameForDomainAndUserContextId( - domain, - userContextId - ); - let proxyPassword = tor.passwordForDomainAndUserContextId( - domain, - userContextId, - true - ); - return mozilla.protocolProxyService.newProxyInfoWithAuth( - "socks", - proxy.host, - proxy.port, - proxyUsername, - proxyPassword, - "", // aProxyAuthorizationHeader - "", // aConnectionIsolationKey - proxy.flags, - proxy.failoverTimeout, - proxy.failoverProxy - ); -}; - -tor.nonce = function() { - // Generate a new 128 bit random tag. Strictly speaking both using a - // cryptographic entropy source and using 128 bits of entropy for the - // tag are likely overkill, as correct behavior only depends on how - // unlikely it is for there to be a collision. - let tag = new Uint8Array(16); - crypto.getRandomValues(tag); - - // Convert the tag to a hex string. - let tagStr = ""; - for (let i = 0; i < tag.length; i++) { - tagStr += (tag[i] >>> 4).toString(16); - tagStr += (tag[i] & 0x0f).toString(16); - } - - return tagStr; -}; - -tor.newCircuitForDomain = function(domain) { - // Re-generate the nonce for the domain. - if (domain === "") { - domain = "--unknown--"; - } - tor.noncesForDomains.set(domain, tor.nonce()); - logger.eclog( - 3, - `New domain isolation for ${domain}: ${tor.noncesForDomains.get(domain)}` - ); -}; - -tor.newCircuitForUserContextId = function(userContextId) { - // Re-generate the nonce for the context. - tor.noncesForUserContextId.set(userContextId, tor.nonce()); - logger.eclog( - 3, - `New container isolation for ${userContextId}: ${tor.noncesForUserContextId.get( - userContextId - )}` - ); -}; - -// __tor.clearIsolation()_. -// Clear the isolation state cache, forcing new circuits to be used for all -// subsequent requests. -tor.clearIsolation = function() { - // Per-domain and per contextId nonces are stored in maps, so simply clear them. - tor.noncesForDomains.clear(); - tor.noncesForUserContextId.clear(); - - // Force a rotation on the next catch-all circuit use by setting the creation - // time to the epoch. - tor.unknownDirtySince = 0; -}; - -// __tor.isolateCircuitsByDomain()__. -// For every HTTPChannel, replaces the default SOCKS proxy with one that authenticates -// to the SOCKS server (the tor client process) with a username (the first party domain -// and userContextId) and a nonce password. Tor provides a separate circuit for each -// username+password combination. -tor.isolateCircuitsByDomain = function() { - mozilla.registerProxyChannelFilter(function(aChannel, aProxy) { - if (!tor.isolationEnabled) { - return aProxy; - } - try { - let channel = aChannel.QueryInterface(Ci.nsIChannel), - firstPartyDomain = channel.loadInfo.originAttributes.firstPartyDomain, - userContextId = channel.loadInfo.originAttributes.userContextId; - if (firstPartyDomain === "") { - firstPartyDomain = "--unknown--"; - if (Date.now() - tor.unknownDirtySince > 1000 * 10 * 60) { - logger.eclog( - 3, - "tor catchall circuit has been dirty for over 10 minutes. Rotating." - ); - tor.newCircuitForDomain("--unknown--"); - tor.unknownDirtySince = Date.now(); - } - } - let replacementProxy = tor.socksProxyCredentials( - aProxy, - firstPartyDomain, - userContextId - ); - logger.eclog( - 3, - `tor SOCKS: ${channel.URI.spec} via - ${replacementProxy.username}:${replacementProxy.password}` - ); - return replacementProxy; - } catch (e) { - logger.eclog(4, `tor domain isolator error: ${e.message}`); - return null; - } - }, 0); -}; - -// ## XPCOM component construction. -// Module specific constants -const kMODULE_NAME = "TorBrowser Domain Isolator"; -const kMODULE_CONTRACTID = "@torproject.org/domain-isolator;1"; -const kMODULE_CID = Components.ID("e33fd6d4-270f-475f-a96f-ff3140279f68"); - -// DomainIsolator object. -function DomainIsolator() { - this.wrappedJSObject = this; -} - -// Firefox component requirements -DomainIsolator.prototype = { - QueryInterface: ChromeUtils.generateQI([Ci.nsIObserver]), - classDescription: kMODULE_NAME, - classID: kMODULE_CID, - contractID: kMODULE_CONTRACTID, - observe(subject, topic, data) { - if (topic === "profile-after-change") { - logger.eclog(3, "domain isolator: set up isolating circuits by domain"); - - if (Services.prefs.getBoolPref("extensions.torbutton.use_nontor_proxy")) { - tor.isolationEnabled = false; - } - tor.isolateCircuitsByDomain(); - } - }, - - newCircuitForDomain(domain) { - tor.newCircuitForDomain(domain); - }, - - /** - * Return the stored SOCKS proxy username and password for the given domain - * and user context ID. - * - * @param {string} firstPartyDomain - The domain to lookup credentials for. - * @param {integer} userContextId - The ID for the user context. - * - * @return {{ username: string, password: string }?} - The SOCKS credentials, - * or null if none are found. - */ - getSocksProxyCredentials(firstPartyDomain, userContextId) { - if (firstPartyDomain == "") { - firstPartyDomain = "--unknown--"; - } - let proxyPassword = tor.passwordForDomainAndUserContextId( - firstPartyDomain, - userContextId, - // Do not create a new entry if it does not exist. - false - ); - if (!proxyPassword) { - return null; - } - return { - username: tor.usernameForDomainAndUserContextId( - firstPartyDomain, - userContextId - ), - password: proxyPassword, - }; - }, - - enableIsolation() { - tor.isolationEnabled = true; - }, - - disableIsolation() { - tor.isolationEnabled = false; - }, - - clearIsolation() { - tor.clearIsolation(); - }, - - wrappedJSObject: null, -}; - -// Assign factory to global object. -const NSGetFactory = XPCOMUtils.generateNSGetFactory - ? XPCOMUtils.generateNSGetFactory([DomainIsolator]) - : ComponentUtils.generateNSGetFactory([DomainIsolator]); ===================================== toolkit/torbutton/jar.mn ===================================== @@ -43,9 +43,5 @@ torbutton.jar: % component {f36d72c9-9718-4134-b550-e109638331d7} %components/torbutton-logger.js % contract @torproject.org/torbutton-logger;1 {f36d72c9-9718-4134-b550-e109638331d7} -% component {e33fd6d4-270f-475f-a96f-ff3140279f68} %components/domain-isolator.js -% contract @torproject.org/domain-isolator;1 {e33fd6d4-270f-475f-a96f-ff3140279f68} - % category profile-after-change StartupObserver @torproject.org/startup-observer;1 -% category profile-after-change DomainIsolator @torproject.org/domain-isolator;1 % category profile-after-change DragDropFilter @torproject.org/torbutton-dragDropFilter;1 ===================================== toolkit/torbutton/modules/utils.js ===================================== @@ -213,45 +213,6 @@ var unescapeTorString = function(str) { return _torControl._strUnescape(str); }; -var getFPDFromHost = hostname => { - try { - return Services.eTLD.getBaseDomainFromHost(hostname); - } catch (e) { - if ( - e.result == Cr.NS_ERROR_HOST_IS_IP_ADDRESS || - e.result == Cr.NS_ERROR_INSUFFICIENT_DOMAIN_LEVELS - ) { - return hostname; - } - } - return null; -}; - -// Assuming this is called with gBrowser.selectedBrowser -var getDomainForBrowser = browser => { - let fpd = browser.contentPrincipal.originAttributes.firstPartyDomain; - // Bug 31562: For neterror or certerror, get the original URL from - // browser.currentURI and use it to calculate the firstPartyDomain. - let knownErrors = [ - "about:neterror", - "about:certerror", - "about:httpsonlyerror", - ]; - let documentURI = browser.documentURI; - if ( - documentURI && - documentURI.schemeIs("about") && - knownErrors.some(x => documentURI.spec.startsWith(x)) - ) { - let knownSchemes = ["http", "https", "ftp"]; - let currentURI = browser.currentURI; - if (currentURI && knownSchemes.some(x => currentURI.schemeIs(x))) { - fpd = getFPDFromHost(currentURI.host) || fpd; - } - } - return fpd; -}; - var m_tb_torlog = Cc["@torproject.org/torbutton-logger;1"].getService( Ci.nsISupports ).wrappedJSObject; @@ -310,7 +271,6 @@ let EXPORTED_SYMBOLS = [ "bindPrefAndInit", "getEnv", "getLocale", - "getDomainForBrowser", "getPrefValue", "observe", "showDialog", View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/ff98f1… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/ff98f1… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser] Pushed new branch tor-browser-102.11.0esr-13.0-1
by Pier Angelo Vendrame (＠pierov) 17 May '23

17 May '23

Pier Angelo Vendrame pushed new branch tor-browser-102.11.0esr-13.0-1 at The Tor Project / Applications / Tor Browser -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/tree/tor-brows… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][base-browser-102.11.0esr-12.5-1] fixup! Bug 4234: Use the Firefox Update Process for Base Browser.
by Pier Angelo Vendrame (＠pierov) 17 May '23

17 May '23

Pier Angelo Vendrame pushed to branch base-browser-102.11.0esr-12.5-1 at The Tor Project / Applications / Tor Browser Commits: cdb91f71 by Pier Angelo Vendrame at 2023-05-17T08:31:37+02:00 fixup! Bug 4234: Use the Firefox Update Process for Base Browser. Bug 41775: Avoid re-defining some macros in nsUpdateDriver.cpp. They are already defined in toolkit/xre/nsUpdateDriver.h. - - - - - 1 changed file: - toolkit/xre/nsUpdateDriver.cpp Changes: ===================================== toolkit/xre/nsUpdateDriver.cpp ===================================== @@ -64,16 +64,6 @@ static LazyLogModule sUpdateLog("updatedriver"); #endif #define LOG(args) MOZ_LOG(sUpdateLog, mozilla::LogLevel::Debug, args) -#ifdef XP_WIN -# define UPDATER_BIN "updater.exe" -# define MAINTENANCE_SVC_NAME L"MozillaMaintenance" -#elif XP_MACOSX -# define UPDATER_APP "updater.app" -# define UPDATER_BIN "org.mozilla.updater" -#else -# define UPDATER_BIN "updater" -#endif - #ifdef XP_MACOSX static void UpdateDriverSetupMacCommandLine(int& argc, char**& argv, bool restart) { View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/cdb91f7… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/cdb91f7… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-102.11.0esr-12.5-1] fixup! Bug 4234: Use the Firefox Update Process for Base Browser.
by Pier Angelo Vendrame (＠pierov) 17 May '23

17 May '23

Pier Angelo Vendrame pushed to branch tor-browser-102.11.0esr-12.5-1 at The Tor Project / Applications / Tor Browser Commits: ff98f1b1 by Pier Angelo Vendrame at 2023-05-11T11:15:17+02:00 fixup! Bug 4234: Use the Firefox Update Process for Base Browser. Bug 41775: Avoid re-defining some macros in nsUpdateDriver.cpp. They are already defined in toolkit/xre/nsUpdateDriver.h. - - - - - 1 changed file: - toolkit/xre/nsUpdateDriver.cpp Changes: ===================================== toolkit/xre/nsUpdateDriver.cpp ===================================== @@ -64,16 +64,6 @@ static LazyLogModule sUpdateLog("updatedriver"); #endif #define LOG(args) MOZ_LOG(sUpdateLog, mozilla::LogLevel::Debug, args) -#ifdef XP_WIN -# define UPDATER_BIN "updater.exe" -# define MAINTENANCE_SVC_NAME L"MozillaMaintenance" -#elif XP_MACOSX -# define UPDATER_APP "updater.app" -# define UPDATER_BIN "org.mozilla.updater" -#else -# define UPDATER_BIN "updater" -#endif - #ifdef XP_MACOSX static void UpdateDriverSetupMacCommandLine(int& argc, char**& argv, bool restart) { View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/ff98f1b… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/ff98f1b… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] 2 commits: Update rbm for rbm#40018, rbm#40051 and rbm#40052
by boklm (＠boklm) 17 May '23

17 May '23

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 9ce60dc8 by Nicolas Vigier at 2023-05-17T08:10:13+02:00 Update rbm for rbm#40018, rbm#40051 and rbm#40052 - - - - - c9d42e1a by Nicolas Vigier at 2023-05-17T08:10:18+02:00 Bug 40849: Use `go mod vendor` to fetch go dependencies Update conjure, webtunnel, obfs4 and snowflake to use `go mod vendor` to fetch dependencies. We still have some go module projects, for dependencies of projects/ncprop279/config, which doesn't have a `go.sum` file yet. - - - - - 30 changed files: - Makefile - + doc/how-to-update-go-dependencies.txt - − projects/agouti/config - − projects/andybalholm-brotli/config - projects/browser/config - − projects/bsbuffer/config - projects/conjure/build - projects/conjure/config - − projects/edwards25519-extra/config - − projects/edwards25519/config - − projects/go-cmp/config - − projects/go-difflib/config - − projects/go-spew/config - projects/go/config - − projects/gocheck/config - − projects/goerrors/config - − projects/gomock/config - − projects/goprotobuf/config - − projects/goptlib/config - − projects/gotapdance/config - − projects/goxtext/config - − projects/goxxerrors/config - − projects/goyaml/config - − projects/kcp-go/config - − projects/klauspost-compress/config - − projects/logrus/config - projects/ncdns/build - projects/ncprop279/build - − projects/obfs4-lib/config - projects/obfs4/build The diff was not included because it is too large. View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/rbm][main] 3 commits: Bug 40051: Fix test after change of default value for compress_tar
by boklm (＠boklm) 17 May '23

17 May '23

boklm pushed to branch main at The Tor Project / Applications / RBM Commits: 27a89ca5 by Nicolas Vigier at 2023-05-08T18:06:36+02:00 Bug 40051: Fix test after change of default value for compress_tar Default value for compress_tar was changed in a91d4653cb5f66d86fd4d306c564161acfd6fa79. - - - - - 60c5aff5 by Nicolas Vigier at 2023-05-17T08:03:40+02:00 Bug 40018: Add target_replace option in input_files - - - - - 37c204c3 by Nicolas Vigier at 2023-05-17T08:03:43+02:00 Bug 40052: Allow setting sha256sum as norec - - - - - 6 changed files: - doc/rbm_input_files.asc - lib/RBM.pm - test.pl - + test/projects/change-targets/config - test/projects/mozmill-automation/config - test/rbm.conf Changes: ===================================== doc/rbm_input_files.asc ===================================== @@ -92,6 +92,12 @@ target_prepend:: The same as +target+, but instead of replacing the current targets, the new targets are prepended. +target_replace:: + A hash table containing targets to replace. The key is a regular + expression, and the value the replacement. See +perlre+ manual + page for details about the syntax. Note that referencing capture + groups in the replacement is currently not supported. + enable:: The files are enabled by default. If this option is set to 0, then the file is ignored. ===================================== lib/RBM.pm ===================================== @@ -779,7 +779,7 @@ sub input_file_need_dl { my ($input_file, $t, $fname, $action) = @_; return undef if $action eq 'getfpaths'; if ($fname - && $input_file->{sha256sum} + && ($input_file->{sha256sum} || $input_file->{norec}{sha256sum}) && $t->('sha256sum') ne sha256file($fname)) { sha256file($fname, { remove_cache => 1 }); $fname = undef; @@ -787,6 +787,7 @@ sub input_file_need_dl { if ($action eq 'input_files_id') { return undef if $input_file->{input_file_id}; return undef if $input_file->{sha256sum}; + return undef if $input_file->{norec}{sha256sum}; return undef if $input_file->{exec}; return undef if $fname; return 1 if $input_file->{URL}; @@ -810,7 +811,9 @@ sub input_file_id { my ($input_file, $t, $fname, $filename) = @_; return $t->('input_file_id') if $input_file->{input_file_id}; return $input_file->{project} . ':' . $filename if $input_file->{project}; - return $filename . ':' . $t->('sha256sum') if $input_file->{sha256sum}; + if ($input_file->{sha256sum} || $input_file->{norec}{sha256sum}) { + return $filename . ':' . $t->('sha256sum'); + } my $opts = { norec => { output_dir => '/out', getting_id => 1, }}; return $filename . ':' . sha256_hex($t->('exec', $opts)) if $input_file->{exec}; @@ -898,13 +901,18 @@ sub input_files { next; } if ($input_file->{target} || $input_file->{target_append} - || $input_file->{target_prepend}) { + || $input_file->{target_prepend} + || $input_file->{target_replace}) { $input_file = { %$input_file }; foreach my $t (qw/target target_append target_prepend/) { if ($input_file->{$t} && ref $input_file->{$t} ne 'ARRAY') { exit_error("$t should be an ARRAY:\n" . pp($input_file)); } } + if ($input_file->{target_replace} && + ref $input_file->{target_replace} ne 'HASH') { + exit_error("target_replace should be a HASH\n" . pp($input_file)); + } if ($input_file->{target}) { $input_file->{target} = process_template_opt($project, $input_file->{target}, $options); @@ -923,6 +931,14 @@ sub input_files { $input_file->{target_append}, $options) } ]; } + if ($input_file->{target_replace}) { + foreach my $pattern (keys %{$input_file->{target_replace}}) { + my $subst = $input_file->{target_replace}{$pattern}; + $input_file->{target} = [ + map { s/$pattern/$subst/r } @{$input_file->{target}} + ]; + } + } } if ($action eq 'getfnames') { my $getfnames_name; ===================================== test.pl ===================================== @@ -1,7 +1,7 @@ #!/usr/bin/perl -w use strict; use Path::Tiny; -use Test::More tests => 40; +use Test::More tests => 41; use lib 'lib/'; sub set_target { @@ -220,6 +220,14 @@ my @tests = ( 'out/r3' => "1 - build\n2 - build\n3 - build\n", }, }, + { + name => 'multi-steps build with changing targets', + target => [ 'target_a' ], + build => [ 'change-targets', 'build', { pkg_type => 'build' } ], + files => { + 'out/change-targets.txt' => "no\nz\ntta\n", + }, + }, { name => 'build project in a module', target => [], ===================================== test/projects/change-targets/config ===================================== @@ -0,0 +1,49 @@ +# vim: filetype=yaml sw=2 + +targets: + tt_a: + option_a: 'tta' + + +steps: + + build: + filename: change-targets.txt + build: | + #!/bin/sh + cat preptarget.txt replacetarget-1.txt replacetarget-2.txt > [% dest_dir %]/[% c('filename') %] + input_files: + - name: preptarget + refresh_input: 1 + project: change-targets + pkg_type: preptarget + target_prepend: + - target_b + - name: replacetarget + r: 1 + refresh_input: 1 + project: change-targets + pkg_type: replacetarget + target_replace: + '^target_a$': target_z + - name: replacetarget + r: 2 + refresh_input: 1 + project: change-targets + pkg_type: replacetarget + target_replace: + '^target_.*$': 'tt_a' + + preptarget: + filename: preptarget.txt + preptarget: | + #!/bin/sh + echo [% c('option_a') %] > [% dest_dir %]/[% c('filename') %] + input_files: [] + + replacetarget: + filename: 'replacetarget-[% c("r") %].txt' + replacetarget: | + #!/bin/sh + echo [% c('option_a') %] > [% dest_dir %]/[% c('filename') %] + input_files: [] ===================================== test/projects/mozmill-automation/config ===================================== @@ -1,6 +1,7 @@ version: '[% c("abbrev") %]' hg_url: https://hg.mozilla.org/qa/mozmill-automation/ hg_hash: bbad7215c713 +compress_tar: '' t: '[% sha256(exec("cat testrun_remote.py")) %]' build: | #!/bin/sh ===================================== test/rbm.conf ===================================== @@ -15,6 +15,8 @@ targets: - target_c - target_a - target_b + target_z: + option_a: z steps: rpm: option_rpm: 1 View it on GitLab: https://gitlab.torproject.org/tpo/applications/rbm/-/compare/bf35e085111a6f… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/rbm/-/compare/bf35e085111a6f… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser] Pushed new branch base-browser-115.0a1-13.0-1
by Pier Angelo Vendrame (＠pierov) 16 May '23

16 May '23

Pier Angelo Vendrame pushed new branch base-browser-115.0a1-13.0-1 at The Tor Project / Applications / Tor Browser -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/tree/base-brow… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] 4 commits: Bug 32355: binutils: Add linux-cross target
by boklm (＠boklm) 15 May '23

15 May '23

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 1f881995 by JeremyRand at 2023-05-15T11:34:26+00:00 Bug 32355: binutils: Add linux-cross target - - - - - eb83bb70 by Jeremy Rand at 2023-05-15T11:34:26+00:00 Bug 32355: gcc: Add linux-arm target - - - - - 25d02d5d by JeremyRand at 2023-05-15T11:34:26+00:00 Bug 32355: openssl: Add linux-arm target - - - - - 225f00eb by Jeremy Rand at 2023-05-15T11:34:26+00:00 Bug 32355: gcc: Add osname to output filename Makes it easier to tell which outputs are for linux-cross. - - - - - 5 changed files: - projects/binutils/build - projects/binutils/config - projects/gcc/build - projects/gcc/config - projects/openssl/config Changes: ===================================== projects/binutils/build ===================================== @@ -23,7 +23,11 @@ cd [% project %]-[% c("version") %] make -j[% c("num_procs") %] MAKEINFO=true make install MAKEINFO=true -[% IF c("var/linux") %] +# gold is disabled for linux-cross, because of +# https://sourceware.org/bugzilla/show_bug.cgi?id=14995 +# Once we upgrade to glibc 2.26, we might be able to enable gold for +# linux-cross. +[% IF c("var/linux") && ! c("var/linux-cross") %] # Make sure gold is used with the hardening wrapper for full RELRO, see #13031. cd $distdir/bin rm ld ===================================== projects/binutils/config ===================================== @@ -10,6 +10,11 @@ targets: windows: var: configure_opt: '--target=[% c("arch") %]-w64-mingw32 --disable-multilib --enable-deterministic-archives' + linux-cross: + var: + # gold is disabled on cross-compiles until we upgrade to glibc 2.26 and + # binutils 2.28 + configure_opt: '--target=[% c("var/crosstarget") %] --disable-multilib --enable-deterministic-archives --enable-plugins' input_files: - URL: https://ftp.gnu.org/gnu/binutils/binutils-[% c("version") %].tar.xz ===================================== projects/gcc/build ===================================== @@ -16,11 +16,85 @@ # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48817. export DEB_BUILD_HARDENING_FORMAT=0 [% END -%] -distdir=/var/tmp/dist/[% project %] +distdir=/var/tmp/dist/[% c("var/distdir") %] mkdir /var/tmp/build -tar -C /var/tmp/build -xf $rootdir/[% c('input_files_by_name/gcc') %] -cd /var/tmp/build/[% project %]-[% c("version") %] -./configure --prefix=$distdir [% c("var/configure_opt") %] + +[% IF c("var/linux-cross") -%] + + # Install binutils (needed for cross-compiling) + mkdir /var/tmp/dist + cd /var/tmp/dist + tar xf $rootdir/[% c('input_files_by_name/binutils') %] + mv binutils $distdir + export PATH="$distdir/bin:$PATH" + + # Install Linux headers, see Step 2 of + # https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ + # Doing this before gcc configure is intended to solve a limits.h issue + cd /var/tmp/build + mkdir linux + cd linux + tar -xJf $rootdir/linux-[% c("var/linux_version") %].tar.xz + cd linux-[% c("var/linux_version") %] + make ARCH=[% c("arch") %] INSTALL_HDR_PATH=$distdir/[% c("var/crosstarget") %] headers_install + + cd /var/tmp/build + mkdir gcc + cd gcc + tar -xJf $rootdir/[% c('input_files_by_name/gcc') %] + # --with-headers is intended to solve a limits.h issue + [% project %]-[% c("version") %]/configure --prefix=$distdir --with-headers=$distdir/[% c("var/crosstarget") %]/include/linux [% c("var/configure_opt") %] + + # For cross-compiling to work, we need to partially build GCC, then build + # glibc, then come back to finish GCC. + + # Build only the components of GCC that don't need glibc, see Step 3 of + # https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ + cd /var/tmp/build/gcc + make -j[% c("num_procs") %] all-gcc + make install-gcc + # Removing sys-include is intended to solve a limits.h issue + rm --recursive --force $distdir/[% c("var/crosstarget") %]/sys-include + + # Build glibc headers and startup files, see Step 4 of + # https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ + cd /var/tmp/build + mkdir glibc + cd glibc + tar -xJf $rootdir/glibc-[% c("var/glibc_version") %].tar.xz + # TODO: Remove --disable-werror once glibc is upgraded to a version that's + # designed to work with the GCC version we're using. + glibc-[% c("var/glibc_version") %]/configure --prefix=$distdir/[% c("var/crosstarget") %] --build=$MACHTYPE --host=[% c("var/crosstarget") %] --target=[% c("var/crosstarget") %] --with-headers=$distdir/[% c("var/crosstarget") %]/include --disable-multilib --disable-werror libc_cv_forced_unwind=yes + make install-bootstrap-headers=yes install-headers + make -j[% c("num_procs") %] csu/subdir_lib + install csu/crt1.o csu/crti.o csu/crtn.o $distdir/[% c("var/crosstarget") %]/lib + [% c("var/crosstarget") %]-gcc -nostdlib -nostartfiles -shared -x c /dev/null -o $distdir/[% c("var/crosstarget") %]/lib/libc.so + # stdio_lim.h is intended to solve a limits.h issue + touch $distdir/[% c("var/crosstarget") %]/include/gnu/stubs.h $distdir/[% c("var/crosstarget") %]/include/bits/stdio_lim.h + + # Build compiler support library, see Step 5 of + # https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ + cd /var/tmp/build/gcc + make -j[% c("num_procs") %] all-target-libgcc + make install-target-libgcc + + # finish building glibc, see Step 6 of + # https://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ + cd /var/tmp/build/glibc + make -j[% c("num_procs") %] + make install + + # We're done with glibc, we can now finish building gcc... + cd /var/tmp/build/gcc + +[% ELSE -%] + + tar -C /var/tmp/build -xf $rootdir/[% c('input_files_by_name/gcc') %] + cd /var/tmp/build/[% project %]-[% c("version") %] + ./configure --prefix=$distdir [% c("var/configure_opt") %] + +[% END -%] + make -j[% c("num_procs") %] make install # tor-browser-build#31321: we need a link to our GCC, to prevent some projects @@ -28,6 +102,6 @@ make install ln -s gcc $distdir/bin/cc cd /var/tmp/dist [% c('tar', { - tar_src => [ project ], + tar_src => [ c('var/distdir') ], tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), }) %] ===================================== projects/gcc/config ===================================== @@ -1,5 +1,5 @@ # vim: filetype=yaml sw=2 -filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' +filename: '[% project %]-[% c("version") %]-[% IF c("var/linux-cross") %][% c("var/osname") %][% ELSE %]x86[% END %]-[% c("var/build_id") %].tar.gz' # Note: When updating the gcc version, if this includes a libstdc++ # ABI change we should also update projects/firefox/abicheck.cc to # require the new version. @@ -7,14 +7,17 @@ version: '[% pc("gcc-source", "version") %]' container: use_container: 1 var: + distdir: gcc deps: - build-essential - libmpc-dev setup: | mkdir -p /var/tmp/dist tar -C /var/tmp/dist -xf $rootdir/[% c("compiler_tarfile") %] - export PATH="/var/tmp/dist/gcc/bin:$PATH" - export LD_LIBRARY_PATH=/var/tmp/dist/gcc/lib64:/var/tmp/dist/gcc/lib32 + export PATH="/var/tmp/dist/[% c("var/distdir") %]/bin:$PATH" + [% IF ! c("var/linux-cross") -%] + export LD_LIBRARY_PATH=/var/tmp/dist/[% c("var/distdir") %]/lib64:/var/tmp/dist/[% c("var/distdir") %]/lib32 + [% END -%] [% IF c("hardened_gcc") -%] # Config options for hardening-wrapper @@ -25,14 +28,14 @@ var: export DEB_BUILD_HARDENING_PIE=1 # Make sure we use the hardening wrapper - pushd /var/tmp/dist/gcc/bin + pushd /var/tmp/dist/[% c("var/distdir") %]/bin cp /usr/bin/hardened-cc ./ - mv gcc gcc.real - mv c++ c++.real - mv g++ g++.real - ln -sf hardened-cc gcc - ln -sf hardened-cc c++ - ln -sf hardened-cc g++ + mv [% c("var/target_prefix") %]gcc [% c("var/target_prefix") %]gcc.real + mv [% c("var/target_prefix") %]c++ [% c("var/target_prefix") %]c++.real + mv [% c("var/target_prefix") %]g++ [% c("var/target_prefix") %]g++.real + ln -sf hardened-cc [% c("var/target_prefix") %]gcc + ln -sf hardened-cc [% c("var/target_prefix") %]c++ + ln -sf hardened-cc [% c("var/target_prefix") %]g++ popd [% END -%] @@ -50,8 +53,34 @@ targets: arch_deps: - hardening-wrapper - libc6-dev-i386 + linux-cross: + var: + target_prefix: '[% c("var/crosstarget") %]-' + distdir: gcc-cross + # TODO: Consider upgrading to a glibc that works out of the box with the + # GCC version we use. However, removing our glibc version workarounds may + # not be desirable since we want to be able to easily bump the GCC + # version without worrying about linux-cross breakage. + glibc_version: 2.26 + linux_version: 4.10.1 + arch_deps: + - hardening-wrapper + - libc6-dev-i386 + - gawk + linux-arm: + var: + configure_opt: --disable-multilib --enable-languages=c,c++ --target=arm-linux-gnueabihf --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb input_files: - project: container-image - project: gcc-source name: gcc + - name: binutils + project: binutils + enable: '[% c("var/linux-cross") -%]' + - URL: 'https://ftp.gnu.org/gnu/glibc/glibc-[% c("var/glibc_version") %].tar.xz' + sha256sum: e54e0a934cd2bc94429be79da5e9385898d2306b9eaf3c92d5a77af96190f6bd + enable: '[% c("var/linux-cross") -%]' + - URL: 'https://www.kernel.org/pub/linux/kernel/v4.x/linux-[% c("var/linux_version") %].tar.xz' + sha256sum: 6ca06bb5faf5f83600d7388bb623dae41df2a257de85ad5d1792e03302bc3543 + enable: '[% c("var/linux-cross") -%]' ===================================== projects/openssl/config ===================================== @@ -14,6 +14,9 @@ targets: linux-i686: var: configure_opts: -shared linux-x86 + linux-arm: + var: + configure_opts: -shared --cross-compile-prefix=[% c("var/crosstarget") %]- linux-armv4 windows: var: flag_mwindows: '' View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] 2 commits: Bug 40856: Temporarily disable namecoin, until #40845 is fixed
by boklm (＠boklm) 13 May '23

13 May '23

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 664c0d8a by Nicolas Vigier at 2023-05-13T11:31:26+02:00 Bug 40856: Temporarily disable namecoin, until #40845 is fixed - - - - - 076ffcf1 by Nicolas Vigier at 2023-05-13T12:15:07+02:00 Bug 40856: Temporarily stop using obfs4 main branch in nightly Until #40850 is fixed. - - - - - 2 changed files: - projects/obfs4/config - rbm.conf Changes: ===================================== projects/obfs4/config ===================================== @@ -9,11 +9,12 @@ filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/buil container: use_container: 1 -targets: - nightly: - git_hash: main - version: '[% c("abbrev") %]' - tag_gpg_id: 0 +# Stop using `main` on nightly, until tor-browser-build#40850 is fixed +#targets: +# nightly: +# git_hash: main +# version: '[% c("abbrev") %]' +# tag_gpg_id: 0 input_files: - project: container-image ===================================== rbm.conf ===================================== @@ -465,7 +465,8 @@ targets: compiler: gcc configure_opt: '[% c("var/configure_opt_project") %]' # Only build Namecoin for linux on nightly - namecoin: '[% c("var/nightly") && c("var/tor-browser") %]' + # Temporarily disabled until we have a fix for tor-browser-build#40845 + #namecoin: '[% c("var/nightly") && c("var/tor-browser") %]' container: suite: jessie arch: amd64 View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/… You're receiving this email because of your account on gitlab.torproject.org.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

tbb-commits May 2023