Quick recap from #tor-dev IRC convo with Nick, Roger, and Andrew:
We need to get a usable tor 0.2.3.x-rc bundle out RSN so we can declare it "stable", but there are concerns that using Firefox 14 with this will continue to cause unexpected problems and otherwise scare people away from testing tor 0.2.3.x enough.
However, I still need to have a place to commit TBB-alpha patches, and have three bugfixes (including a fix for a FF14 crash bug that was discovered by tor-qa) that I'd like to get into the alpha series. Also, if we don't provide Firefox Rapid Release with regular alpha testing, we're going to be really, really sad when everything breaks at once in November with the next Firefox ESR.
One option is to create a temporary "rc" branch of torbrowser.git's maint-2.2 to build tor 0.2.3.x but with the rest of a "TBB stable" bundle with Vidalia 0.2.x and Firefox 10.x ESR, and leave maint-2.3 as "TBB alpha".
A second option is to create a more permanent "TBB beta" series, with whatever software smells like it is getting close to stable at a given point in time.
A third option is to just keep doing English-only maint-2.3 builds back-to-back until tor-qa stops reporting crash bugs or strange issues.
However, we need input from Erinn to decide the best approach. If the "rc" fork (or a permanent beta branch) messes up the build process or introduces issues with build automation work, perhaps it is not the right way to go, and we should just keep doing english-only maint-2.3 releases back-to-back until FF14 is more stable (Note: it works OK for me now in my test builds with the crash fix applied).
So Erinn, which one is least painful for you?
Also, does anyone else have any other input or suggestions on how to approach this problem? We're probably going to run into this issue periodically due to our different components stabilizing at different points...
* Mike Perry mikeperry@torproject.org [2012:07:31 14:56 -0700]:
Quick recap from #tor-dev IRC convo with Nick, Roger, and Andrew:
We need to get a usable tor 0.2.3.x-rc bundle out RSN so we can declare it "stable", but there are concerns that using Firefox 14 with this will continue to cause unexpected problems and otherwise scare people away from testing tor 0.2.3.x enough.
How unstable is Firefox 14 though? It is Firefox's stable release, after all. Are the recently-released bundles unusable?
However, I still need to have a place to commit TBB-alpha patches, and have three bugfixes (including a fix for a FF14 crash bug that was discovered by tor-qa) that I'd like to get into the alpha series. Also, if we don't provide Firefox Rapid Release with regular alpha testing, we're going to be really, really sad when everything breaks at once in November with the next Firefox ESR.
One option is to create a temporary "rc" branch of torbrowser.git's maint-2.2 to build tor 0.2.3.x but with the rest of a "TBB stable" bundle with Vidalia 0.2.x and Firefox 10.x ESR, and leave maint-2.3 as "TBB alpha".
A second option is to create a more permanent "TBB beta" series, with whatever software smells like it is getting close to stable at a given point in time.
Technically, this is what maint-2.3 should be.
A third option is to just keep doing English-only maint-2.3 builds back-to-back until tor-qa stops reporting crash bugs or strange issues.
I think there's a fourth option, which is urge people on the blog, via twitter, etc, to test the alpha/rc bundles, with me making a commitment to do frequent releases for any issues that pop up.
However, we need input from Erinn to decide the best approach. If the "rc" fork (or a permanent beta branch) messes up the build process or introduces issues with build automation work, perhaps it is not the right way to go, and we should just keep doing english-only maint-2.3 releases back-to-back until FF14 is more stable (Note: it works OK for me now in my test builds with the crash fix applied).
I don't think more bundles is the right answer, in most cases. But let's be clear about the kind of testing we need first. I think Nick's going to say "ALL the testing", but 0.2.3.x server-side is pretty well-tested on Linux by now, fairly poorly tested on Windows, and more or less irrelevant on OSX, with barely any testing for clients on all three platforms. Sound right?
So depending on the timeline people have in mind, my suggestion is to do a big push on the aforementioned social networks and shake out as many bugs in the next week as we can, then iterate. But my opinion depends a bit on how severe you think the Firefox 14 problem is.
That's the short-term solution for getting 0.2.3.x stable though. Longer term, I just can't let the alpha bundles lapse again -- it's my fault we're in this position and I'm really sorry. I think we'll move to a true-experimental bundle situation as outlined in the HACKING document in torbrowser.git where we can keep things held up if they aren't going RC in the next 9 months (or at all) . Then the "middle" branch of TBB can be used strictly for stabilizing.
Happy to hear more input.
On Wed, Aug 01, 2012 at 07:55:12AM +0100, Erinn Clark wrote:
We need to get a usable tor 0.2.3.x-rc bundle out RSN so we can declare it "stable", but there are concerns that using Firefox 14 with this will continue to cause unexpected problems and otherwise scare people away from testing tor 0.2.3.x enough.
How unstable is Firefox 14 though? It is Firefox's stable release, after all. Are the recently-released bundles unusable?
Alas, they do crash a lot, due to one of Mike's Firefox patches: https://trac.torproject.org/projects/tor/ticket/6492
So, good that we found a problem, but bad that we don't have a TBB suitable for testing the Tor 0.2.3 release candidate anymore. :)
Thanks, --Roger
On Wed, Aug 1, 2012 at 2:55 AM, Erinn Clark erinn@torproject.org wrote:
IMO this comes down to the question of: Can we make the alpha TBB branch stable fast enough to have it be useful for testing? The crash bugs that Roger mentions make it less than useful anybody trying to get good testing info for Tor. So if we think that one can get fixed in the next release, great, let's carry on.
But otherwise, I don't think it's a good idea to make Tor 0.2.3.x wait until an unknown number of bugs are fixed on an unknown timeframe.
The reason had I suggested doing Tor 0.2.3.x releases with the stable TBB for a while (not permanently) is that I personally don't have much sense of how far the TBB alpha is from stability. If, once #6492 etc are solved, it's stable and pleasant to use, great -- but if the browser remains crashy enough that folks can't actually use it, maybe we should stick with pairing Tor 0.2.3.x with the current TBB stable, and letting Tor 0.2.4.x and the alpha/experimental TBB stabilize together.
This is of course all said without full knowledge of everything that's going on; there are likely to be important facts here I don't know.
peace,
Thus spake Erinn Clark (erinn@torproject.org):
- Mike Perry mikeperry@torproject.org [2012:07:31 14:56 -0700]:
Quick recap from #tor-dev IRC convo with Nick, Roger, and Andrew:
We need to get a usable tor 0.2.3.x-rc bundle out RSN so we can declare it "stable", but there are concerns that using Firefox 14 with this will continue to cause unexpected problems and otherwise scare people away from testing tor 0.2.3.x enough.
How unstable is Firefox 14 though? It is Firefox's stable release, after all. Are the recently-released bundles unusable?
Unclear, but probably OK now? I had to rewrite my pipelining patch at the last minute due to massive changes in the Firefox 14 pipelining code. That rewrite introduced the crash bug. Once fixed, my test Firefox build behaved fine for a while. I will switch to using it as my main web browser as well.
A third option is to just keep doing English-only maint-2.3 builds back-to-back until tor-qa stops reporting crash bugs or strange issues.
I think there's a fourth option, which is urge people on the blog, via twitter, etc, to test the alpha/rc bundles, with me making a commitment to do frequent releases for any issues that pop up.
Yes, this sounds great. I also commit to fixing reproducible crash bugs immediately.
Roger also agrees that English-only builds are just fine for the maint-2.3 bundles, at least until we get them to be more stable.
So depending on the timeline people have in mind, my suggestion is to do a big push on the aforementioned social networks and shake out as many bugs in the next week as we can, then iterate. But my opinion depends a bit on how severe you think the Firefox 14 problem is.
I can fix reproducible crash bugs quickly, especially if a particular website reliably (or even semi-reliably) triggers the crash.
If we have random instability that is hard to reproduce, that might be a different story and may require dropping random suspicious patches. I think we can just cross that bridge when it starts burning, though.
Therefore, as long as you're good building these English TBB alphas at a pretty quick rate, I think we now have a plan.
I will commit my new FF14 patches, and let's start the builds!
-
Erinn Clark -
Mike Perry -
Nick Mathewson -
Roger Dingledine