---------------------------------------------------------------------------------------------------------------
EGI BROADCAST TOOL : https://operations-portal.egi.eu/broadcast
---------------------------------------------------------------------------------------------------------------
Publication from : john gordon <[log in to unmask]>
Targets : Generic mailing-list/LCG Rollout Mailing List
<[log in to unmask]>
----------------------------------------------------------------------------------------------------------------
Yesterday (Tuesday 20th November) around lunchtime there was a power incident
that affected systems connected to the UPS (Uninterruptible Power Supply) at
RAL including the APEL servers. These systems suffered an over-voltage and
some have been damaged. Most (but not all) of this equipment is located in a
single room within the computer building.
During yesterday afternoon power was restored to the appropriate circuits and
in the evening damaged cooling units were repaired. Currently we are
continuing to assess damage and powering up systems. However, many of the
power distribution units along with many network switches are broken. These
are being replaced and re-establishing networking to appropriate end systems
is the current focus - ahead of being able to check out the systems and
servers.
We are unable to give a time estimate for a return of services until we have
established the extent of the damage caused. We will send out a further
update in around 4 hours when we expect to have a better idea of the damage
and possible timescales for recovery. Sites will see error messages in their
APEL logs but the publication will catch up the next time you publish after
the server is back up.
----------------------------------------------------------------------------------------------------------------
link to this broadcast :
https://operations-portal.egi.eu/broadcast/archive/id/819
----------------------------------------------------------------------------------------------------------------
|