Hello,
We now have some automated tests run on TBB nightly builds, and also on new releases: https://people.torproject.org/~boklm/tbbtests/
For the new releases, the script needs to know the version number of that release to be able to download it. Currently, I am updating the version number in this file when there is a new one to be tested: https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/blob/HE...
However, I think that maybe we should have a file somewhere on a git repository, where we write the version number of a new release that needs to be tested. This versions file could be also used by an other script to trigger new builds. So in addition to version numbers it can contain a commit hash, and build type (the makefile rule we need to use: normal build, beta or alpha).
Later we can think about using it for other things too: upload of new versions to mirrors, removal of old versions, website update, update of recommended version file, etc ...
I am thinking about a yaml file that look like this:
---- tbbversions: 3.5.4: type: release build_commit: ada623c4cbc145521980e6bce08611cff7e81cad 3.6-beta-2: type: beta build_commit: ada623c4cbc145521980e6bce08611cff7e81cad ----
We can put this file in a git repository where all commits are signed (with git commit -S), and the scripts that use it check the signatures when pulling (with git pull --verify-signatures). We can use something like this to do that: https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/blob/HE...
Then it would work like this:
- a script check this version file, and if there is a new version defined in the file that has not been built yet, starts a new build, and upload it to people.torproject.org if the build is successful.
- the script that run automated tests check this version file, and when a version defined in this file is available from at least two builders on people.torproject.org, starts running the tests.
What do you think about this ?
Hi,
Nicolas Vigier:
I am thinking about a yaml file that look like this:
tbbversions: 3.5.4: type: release build_commit: ada623c4cbc145521980e6bce08611cff7e81cad 3.6-beta-2: type: beta build_commit: ada623c4cbc145521980e6bce08611cff7e81cad
We can put this file in a git repository where all commits are signed (with git commit -S), and the scripts that use it check the signatures when pulling (with git pull --verify-signatures). We can use something like this to do that: https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/blob/HE...
Then it would work like this:
a script check this version file, and if there is a new version defined in the file that has not been built yet, starts a new build, and upload it to people.torproject.org if the build is successful.
the script that run automated tests check this version file, and when a version defined in this file is available from at least two builders on people.torproject.org, starts running the tests.
What do you think about this ?
hmmm... how do we handle cases like the one with the current beta where we have to rebuild/rebundle things multiple times and it is urgent to get the new bundles out? Even if we get better at it this might still happen in the future. One idea would be to look at the commit in addition to the version. While that might be better it is probably not helping in cases we just want to rebundle things and run the test suite... Might get even more complex if we, say, only need to rebuild the pluggable transport part.
It is actually not necessary to have all bundles available twice on people.tpo (I even doubt it happens very often). Having one and matching sha256sums from a different build should be sufficient.
Apart from that the general idea sounds good to me. It would especially be useful in sending only a mail to tor-qa after the build passes all of our automation tests.
Georg
Hi,
On Fri, 11 Apr 2014, Georg Koppen wrote:
Hi,
hmmm... how do we handle cases like the one with the current beta where we have to rebuild/rebundle things multiple times and it is urgent to get the new bundles out? Even if we get better at it this might still happen in the future. One idea would be to look at the commit in addition to the version. While that might be better it is probably not helping in cases we just want to rebundle things and run the test suite... Might get even more complex if we, say, only need to rebuild the pluggable transport part.
Ah, I have been thinking about that just after sending the mail. I think we can add a build number in the versions file, in addition to the version number. And when either the version or build number changes, we run the tests again.
So the versions file with build number would be something like this: ---- tbbversions: 3.5.4: type: release build_commit: ada623c4cbc145521980e6bce08611cff7e81cad buildnum: 1 3.6-beta-2: type: beta build_commit: ada623c4cbc145521980e6bce08611cff7e81cad buildnum: 3 ----
It is actually not necessary to have all bundles available twice on people.tpo (I even doubt it happens very often). Having one and matching sha256sums from a different build should be sufficient.
Indeed. So the build script should check whether someone already uploaded a matching version and upload only the sha256sums in that case.
Apart from that the general idea sounds good to me. It would especially be useful in sending only a mail to tor-qa after the build passes all of our automation tests.
Ok, I can make it send an email to tor-qa with the results.
Nicolas Vigier:
Hi,
On Fri, 11 Apr 2014, Georg Koppen wrote:
Hi,
hmmm... how do we handle cases like the one with the current beta where we have to rebuild/rebundle things multiple times and it is urgent to get the new bundles out? Even if we get better at it this might still happen in the future. One idea would be to look at the commit in addition to the version. While that might be better it is probably not helping in cases we just want to rebundle things and run the test suite... Might get even more complex if we, say, only need to rebuild the pluggable transport part.
Ah, I have been thinking about that just after sending the mail. I think we can add a build number in the versions file, in addition to the version number. And when either the version or build number changes, we run the tests again.
That is good for (re-)running the test suites. But I am still wondering about re-running the builds as we often only need to rebundle things. But that should not necessarily trigger a whole rebuild. Especially not if we need to get new bundles out urgently and would need to wait on having this rebuild finished before.
Or is this (re-)running builds feature just an orthogonal feature unrelated to actual releases? So, maybe I did not filly understand what you had in mind with "This versions file could be also used by an other script to trigger new builds." then.
Georg
-
Georg Koppen -
Nicolas Vigier