Hi,
As many of you are aware the RAL CVMFS 'Stratum 1' server failed yesterday
(Wednesday) with problems becoming apparent around the middle of the day.
This failure was caused by a problem with the storage array behind the
server. Work has been going on to investigate and fix this. We are hopeful
that the service will be restored tomorrow (Friday). The reverse proxy
squids in front of this service have also been disabled - and will stay down
until the server is back.
Our experience at the RAL Tier1 was that the CVMS software didn't fail over
as expected. A restart of the CVMFS client should cause the software to use
an available server. We (I believe along with a number of other sites) have
modified our CVMFS configuration to point to a different stratum 1 in
preference (although for the LHC VOs this should not be necessary). At the
RAL Tier1 we have been running a number of different versions of the CVMFS
client and have had to check out the behaviour of each to verify the
software is updating correctly after the configuration change.
For those who support the non-LHC VOs refer to the message Catalin sent out
yesterday. Software (CVMFS) for MICE, NA62 and H1 is available from the CERN
Stratum 1. However, there is no failover for the other two VOs supported on
the RAL Server (enmr.eu and phys-vo-ibergrid.eu).
Gareth Smith
|