I have a damn silly question. At the point at which a release is created
is it possible to create an identical "roll back release".
So X.5 is created and at the same time X.6.1 - the pointer to production
release points to X.5. When X.6 is released the production release points
to X.6 and if there is a problem it is then re-pointed to X.6.1.
X.6 is clearly released with X.7.1 and so on.
I have a feeling I must be missing something, and if this is a silly idea
I apologise for wasting people's time
Paul Kyberd
-----Original Message-----
From: Testbed Support for GridPP member institutes on behalf of Coles, J (Jeremy)
Sent: Thu 1/29/2009 12:38
To: [log in to unmask]
Subject: Re: Glite Middleware Rollback proposal.
Dear All
Thank you for responses posted to TB-SUPPORT so far. As mentioned at our
UKI ops meeting this morning, I have got extra information from Markus
on the background to the proposal after putting a few questions back to
him. I am posting this below and would be grateful if you could consider
the content and post your suggestions if this alters what you have
already suggested. Noting that Markus needs concrete proposals from
sites on how things might be done, please keep your responses
constructive so that the UK reaction I will compose (for Markus, Oliver
and the GDB) is not just moaning (I wouldn't start from here is not
going to help). I will make a point that the build systems could/should
be improved first... but we need to work from where we are. How could
something best be implemented bearing in mind the constraints on the
build side?
I'll compile the responses early next week, so responses by Monday would
be appreciated.
Thanks,
Jeremy
From Markus:
" there are a some problems for EGEE to follow the common Linux
distribution approach with rollback by re-tagging and rebuilding the
older code base.
There is no uniform way the gLite components manage their versions
a) Some use the "classic" approach via CVS
b) Some depend on the configuration management in the ETICS package
manager
There is no uniform way the information for the creating the RPM spec
files is managed.
This is critical, because to create from the old version an RPM with a
higher version number this information has to be changed.
While ETICS stores a lot of traces about what has been created how and
this can be used to create a replica of the produced RPMs, it is
technically difficult to create from this starting point an identical
RPM with a newer version number. The two above mentioned extra degrees
of freedom for gLite developers doesn't help here.
As a result in most cases neither the integration nor the rollout team
is technically in a position to handle the "old code higher version"
build without
interaction with the developers. Which as a result creates significant
delays, not only for the rollback release, but in addition for the work
on the bug fixes.
These delays can be substantial, because in several teams the release
build with ETICS is done by the same person and that person's
availability can't be guaranteed. In addition developers have been quite
reluctant to invest in recreating old material while believing that the
"real" fix is just
5 lines of code away.
The third alternative approach to just stop the rollout and rush for the
real fix has been demonstrated to not working. The VOMS experience where
we had to iterate with the developers for 6+ months until we had a
version that had no obvious bugs is a very good example. It has to be
noted that during this period we had to ad extra manpower to test new
VOMS releases as quickly as possible to make progress at all.
In addition the goal of a rollback is to contain a situation until a
proper fix is available. This means that the reaction time should be as
low as possible.
We certainly don't suggest to roll back for trivial reasons.
With these constraints in mind, a strategy based on the already existing
previous RPMs was tempting, but we are open to suggestions on how to do
this properly, but please give us advice that is a bit more practical
and concrete than "Do it the RedHat, Debian or Ubuntu"-way, I would
appreciate suggestions that don't require to move the developers to a
different build system.....
markus
ps:
There is another problem with higher versioned old RPMS.
It has the disadvantage in rollout that in most cases problems are
spotted after less than 20% of the infrastructure has moved to the new
(bad) version.
If we could follow the standard Linux approach at the moment any real
update happens to the repository 80% of the sites would roll forward to
the same version at which they already run. Especially when the
additional change requires a rerun of YAIM this can create some problems
when the site uses some modification of the standard setup."
-----Original Message-----
From: Testbed Support for GridPP member institutes
[mailto:[log in to unmask]] On Behalf Of Peter Gronbech
Sent: 26 January 2009 11:28
To: [log in to unmask]
Subject: Glite Middleware Rollback proposal.
At the January GDB meeting Markus Schultz and Andreas Unterkircher
outlined a new proposed method for handling emergency rollback of a
release once a bug has been found.
In particular on page 6 of Andreas talk they suggest having a production
repository and a previous version repository for the rpms.
In the event of a problem a link would be switched back to point at the
previous release thus stopping any more downloads of a faulty release,
but this does leave the institutes that have already upgraded with an
awkward manual reversal procedure.
My suggestion of just releasing an higher numbered rpm with the bug
removed was not favoured.
We would like to ask you your opinions of this scheme so a UK feedback
response can be made.
Thanks Pete
GDB meeting http://indico.cern.ch/conferenceDisplay.py?confId=45461
Andreas Unterkircher's talk
http://indico.cern.ch/getFile.py/access?sessionId=5&resId=1&materialId=1
&confId=45461
--
----------------------------------------------------------------------
Peter Gronbech Senior Systems Manager and Tel No. : 01865 273389
SouthGrid Technical Co-ordinator Fax No. : 01865 273418
Department of Particle Physics,
University of Oxford,
Keble Road, Oxford OX1 3RH, UK E-mail : [log in to unmask]
----------------------------------------------------------------------
--
Scanned by iCritical.
