Hi everyone,
(tldr; read the last paragraph)
In October [0] we were very excited about the prospect of all Tor Browser platforms following the Firefox Rapid Release schedule. However, in April (now), Android is still the only platform following the rapid release train and Windows, macOS and Linux remain on the extended support release (ESR).
As we move closer to the next ESR transition, this year it is beginning at Firefox 91 in May [1], I am wondering whether we should reverse course and slow down. At this point, we cannot safely transition all platforms onto the rapid release train before October (when 78esr reaches its EOL), so the only option is moving all desktop platforms onto FF91esr and then evaluate migrating onto the rapid release train after that.
Unfortunately, we are still a very small team, and the current situation is already pushing our team to its limits. Yesterday I spoke with Georg about another subject, and he briefly mentioned the idea of keeping the desktop platforms on ESR instead of moving onto the rapid releases. This alone wouldn't make much difference, and I previously discarded this idea, because we're already fighting all of the code churn for Android (although some of the code, like the automatic updater, is not used on Android). The important piece of this plan is reducing Fenix's burden, too.
My current thought is that we move desktop platforms from 78esr to 91esr, and we begin rebasing our Fenix branches (and dependencies) only every 2-3 months. The only exception is geckoview where we continue rebasing our geckoview branches on the existing schedule. In addition, we drop desktop patches from the geckoview branches, and Android patches from the ESR branches. We could've (and should've) done the latter simplification earlier, but the former change makes future ESR transitions a little more complicated. I think we can live with that.
- Matt
[0] https://lists.torproject.org/pipermail/tbb-dev/2020-October/001154.html [1] https://wiki.mozilla.org/Release_Management/Calendar
Does this mean that on Android, you're tracking GeckoView tightly, and Fenix less tightly? So you'd need to essentially uplift Fenix sec fixes until you rebase?
-tom
On Wed, 7 Apr 2021 at 14:49, Matthew Finkel sysrqb@torproject.org wrote:
Hi everyone,
(tldr; read the last paragraph)
In October [0] we were very excited about the prospect of all Tor Browser platforms following the Firefox Rapid Release schedule. However, in April (now), Android is still the only platform following the rapid release train and Windows, macOS and Linux remain on the extended support release (ESR).
As we move closer to the next ESR transition, this year it is beginning at Firefox 91 in May [1], I am wondering whether we should reverse course and slow down. At this point, we cannot safely transition all platforms onto the rapid release train before October (when 78esr reaches its EOL), so the only option is moving all desktop platforms onto FF91esr and then evaluate migrating onto the rapid release train after that.
Unfortunately, we are still a very small team, and the current situation is already pushing our team to its limits. Yesterday I spoke with Georg about another subject, and he briefly mentioned the idea of keeping the desktop platforms on ESR instead of moving onto the rapid releases. This alone wouldn't make much difference, and I previously discarded this idea, because we're already fighting all of the code churn for Android (although some of the code, like the automatic updater, is not used on Android). The important piece of this plan is reducing Fenix's burden, too.
My current thought is that we move desktop platforms from 78esr to 91esr, and we begin rebasing our Fenix branches (and dependencies) only every 2-3 months. The only exception is geckoview where we continue rebasing our geckoview branches on the existing schedule. In addition, we drop desktop patches from the geckoview branches, and Android patches from the ESR branches. We could've (and should've) done the latter simplification earlier, but the former change makes future ESR transitions a little more complicated. I think we can live with that.
- Matt
[0] https://lists.torproject.org/pipermail/tbb-dev/2020-October/001154.html [1] https://wiki.mozilla.org/Release_Management/Calendar _______________________________________________ tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
Yes, that's correct. I haven't seen many Fenix-specific sec patches over the last few months, so I feel comfortable with this trade-off right now. I don't expect we'll fall too far behind Fenix and its dependencies, so uplifting shouldn't be too difficult, and I am hoping we can just track Fenix's nightly train and ride that to release every two months.
On Wed, Apr 07, 2021 at 07:01:42PM +0000, Tom Ritter wrote:
Does this mean that on Android, you're tracking GeckoView tightly, and Fenix less tightly? So you'd need to essentially uplift Fenix sec fixes until you rebase?
-tom
On Wed, 7 Apr 2021 at 14:49, Matthew Finkel sysrqb@torproject.org wrote:
Hi everyone,
(tldr; read the last paragraph)
In October [0] we were very excited about the prospect of all Tor Browser platforms following the Firefox Rapid Release schedule. However, in April (now), Android is still the only platform following the rapid release train and Windows, macOS and Linux remain on the extended support release (ESR).
As we move closer to the next ESR transition, this year it is beginning at Firefox 91 in May [1], I am wondering whether we should reverse course and slow down. At this point, we cannot safely transition all platforms onto the rapid release train before October (when 78esr reaches its EOL), so the only option is moving all desktop platforms onto FF91esr and then evaluate migrating onto the rapid release train after that.
Unfortunately, we are still a very small team, and the current situation is already pushing our team to its limits. Yesterday I spoke with Georg about another subject, and he briefly mentioned the idea of keeping the desktop platforms on ESR instead of moving onto the rapid releases. This alone wouldn't make much difference, and I previously discarded this idea, because we're already fighting all of the code churn for Android (although some of the code, like the automatic updater, is not used on Android). The important piece of this plan is reducing Fenix's burden, too.
My current thought is that we move desktop platforms from 78esr to 91esr, and we begin rebasing our Fenix branches (and dependencies) only every 2-3 months. The only exception is geckoview where we continue rebasing our geckoview branches on the existing schedule. In addition, we drop desktop patches from the geckoview branches, and Android patches from the ESR branches. We could've (and should've) done the latter simplification earlier, but the former change makes future ESR transitions a little more complicated. I think we can live with that.
- Matt
[0] https://lists.torproject.org/pipermail/tbb-dev/2020-October/001154.html [1] https://wiki.mozilla.org/Release_Management/Calendar _______________________________________________ tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
Matthew Finkel:
Hi everyone,
(tldr; read the last paragraph)
In October [0] we were very excited about the prospect of all Tor Browser platforms following the Firefox Rapid Release schedule. However, in April (now), Android is still the only platform following the rapid release train and Windows, macOS and Linux remain on the extended support release (ESR).
As we move closer to the next ESR transition, this year it is beginning at Firefox 91 in May [1], I am wondering whether we should reverse course and slow down. At this point, we cannot safely transition all platforms onto the rapid release train before October (when 78esr reaches its EOL), so the only option is moving all desktop platforms onto FF91esr and then evaluate migrating onto the rapid release train after that.
Unfortunately, we are still a very small team, and the current situation is already pushing our team to its limits. Yesterday I spoke with Georg about another subject, and he briefly mentioned the idea of keeping the desktop platforms on ESR instead of moving onto the rapid releases. This alone wouldn't make much difference, and I previously discarded this idea, because we're already fighting all of the code churn for Android (although some of the code, like the automatic updater, is not used on Android). The important piece of this plan is reducing Fenix's burden, too.
My current thought is that we move desktop platforms from 78esr to 91esr, and we begin rebasing our Fenix branches (and dependencies) only every 2-3 months. The only exception is geckoview where we continue
So, for the timeline for that plan: this change is going to happen with the ESR91 transition? Or now? Or after it?
rebasing our geckoview branches on the existing schedule. In addition, we drop desktop patches from the geckoview branches, and Android patches from the ESR branches. We could've (and should've) done the latter simplification earlier, but the former change makes future ESR transitions a little more complicated. I think we can live with that.
I have three concerns with that plan:
1) Getting rid of mobile patches on desktop and vice versa sounds like a good idea on first glance. However, that would mean extra work for our test suite part as right now we require Linux builds (and thus desktop patches, too) to be sure non of our (GecKoView) patches requires (further) changes. One could argue we could drop that part as those tests are, to a large extent, desktop related tests anyway. I am not sure whether you wanted to make that point, though (and there *are* a bunch of tests that test functionality for mobile as well)
2) We have seen in the past while working on toolchain updates that the GeckoView, android-components, and Fenix parts are quite tightly coupled. You see that at the Gradle dependencies they require and that things broke already during build when the wrong GeckoView version got mixed with android-components/Fenix due to API incompatibilities. Just focusing on continuous GeckoView rebasing while ignoring the other parts will likely hit that problem again causing us to have extra work fixing that up. (Even without *that* extra work there will be additional work needed for de-coupling the those above mentioned tightly coupled projects on the build side, even though we might feel this could be acceptable)
3) Somewhat related to 2) I am worried about instability we introduce by de-coupling GeckoView, android-components, and Fenix. It's often not seen (as it is hardly visible on desktop either) but we greatly benefit from all the testing Mozilla is running on those three components. But that testing is *only* valid if we follow the GeckoView + android-components + Fenix pipeline as close as we can. What happens with your plan is that me move straightly into untested territory and I doubt we can compensate for that given the resource issues you cite.
Georg
- Matt
[0] https://lists.torproject.org/pipermail/tbb-dev/2020-October/001154.html [1] https://wiki.mozilla.org/Release_Management/Calendar _______________________________________________ tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
On Thu, Apr 08, 2021 at 05:53:16AM +0000, Georg Koppen wrote:
Matthew Finkel:
Hi everyone,
(tldr; read the last paragraph)
In October [0] we were very excited about the prospect of all Tor Browser platforms following the Firefox Rapid Release schedule. However, in April (now), Android is still the only platform following the rapid release train and Windows, macOS and Linux remain on the extended support release (ESR).
As we move closer to the next ESR transition, this year it is beginning at Firefox 91 in May [1], I am wondering whether we should reverse course and slow down. At this point, we cannot safely transition all platforms onto the rapid release train before October (when 78esr reaches its EOL), so the only option is moving all desktop platforms onto FF91esr and then evaluate migrating onto the rapid release train after that.
Unfortunately, we are still a very small team, and the current situation is already pushing our team to its limits. Yesterday I spoke with Georg about another subject, and he briefly mentioned the idea of keeping the desktop platforms on ESR instead of moving onto the rapid releases. This alone wouldn't make much difference, and I previously discarded this idea, because we're already fighting all of the code churn for Android (although some of the code, like the automatic updater, is not used on Android). The important piece of this plan is reducing Fenix's burden, too.
My current thought is that we move desktop platforms from 78esr to 91esr, and we begin rebasing our Fenix branches (and dependencies) only every 2-3 months. The only exception is geckoview where we continue
So, for the timeline for that plan: this change is going to happen with the ESR91 transition? Or now? Or after it?
I would begin this next month, when we start the 91esr transition. We could begin slowing our rebasing of Fenix this month, but we already started that effort, so we might as well finish it.
rebasing our geckoview branches on the existing schedule. In addition, we drop desktop patches from the geckoview branches, and Android patches from the ESR branches. We could've (and should've) done the latter simplification earlier, but the former change makes future ESR transitions a little more complicated. I think we can live with that.
I have three concerns with that plan:
- Getting rid of mobile patches on desktop and vice versa sounds like a
good idea on first glance. However, that would mean extra work for our test suite part as right now we require Linux builds (and thus desktop patches, too) to be sure non of our (GecKoView) patches requires (further) changes. One could argue we could drop that part as those tests are, to a large extent, desktop related tests anyway. I am not sure whether you wanted to make that point, though (and there *are* a bunch of tests that test functionality for mobile as well)
I agree this is a current problem we should solve. I'm curious how many desktop-only patches are needed for successfully running the testsuite's tests that cover shared functionality between linux and geckoview. I suspect not many. In particular, if we can drop half of our current geckoview patches, then that is a significant win.
- We have seen in the past while working on toolchain updates that the
GeckoView, android-components, and Fenix parts are quite tightly coupled. You see that at the Gradle dependencies they require and that things broke already during build when the wrong GeckoView version got mixed with android-components/Fenix due to API incompatibilities. Just focusing on continuous GeckoView rebasing while ignoring the other parts will likely hit that problem again causing us to have extra work fixing that up. (Even without *that* extra work there will be additional work needed for de-coupling the those above mentioned tightly coupled projects on the build side, even though we might feel this could be acceptable)
I partially agree with this. GeckoView provides a deprecation period for its (breaking) API changes. That period is usually 3 versions. As for the Fenix and MozAC incompatibility, those issues came from newer Fenix/MozAC branches and an older geckoview branch, if I remember correctly. I believe the reverse situation should be safer.
- Somewhat related to 2) I am worried about instability we introduce by
de-coupling GeckoView, android-components, and Fenix. It's often not seen (as it is hardly visible on desktop either) but we greatly benefit from all the testing Mozilla is running on those three components. But that testing is *only* valid if we follow the GeckoView + android-components + Fenix pipeline as close as we can. What happens with your plan is that me move straightly into untested territory and I doubt we can compensate for that given the resource issues you cite.
This may be the strongest argument for not following the new plan. I suspect that running more tests (on vanilla Fenix and MozAC) should be sufficient and will be less time consuming than our current situation, but we can run an experiment before committing to this process change.
Georg
- Matt
[0] https://lists.torproject.org/pipermail/tbb-dev/2020-October/001154.html [1] https://wiki.mozilla.org/Release_Management/Calendar _______________________________________________ tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
-
Georg Koppen -
Matthew Finkel -
Tom Ritter