> -----Original Message-----
> From: Testbed Support for GridPP member institutes [mailto:TB-
> [log in to unmask]] On Behalf Of Steve Traylen
> Sent: 27 March 2012 17:26
>
> On Mar 27, 2012, at 4:28 PM, Ewan MacMahon wrote:
>
> > - Sometimes the WN caches seem to go a bit over their configured size
> > limit.
>
> Not much I hope, when the cache goes over it reduces occupancy to 50% (?)
> but not until the current write of files has been done.
>
Here's one I've just caught in the act:
[root@t2wn24 ~]# cat /etc/cvmfs/default.local
#CVMFS_DEBUGLOG=/tmp/cvmfs.log
CVMFS_REPOSITORIES=atlas,atlas-condb,lhcb
CVMFS_HTTP_PROXY="http://t2squid01.physics.ox.ac.uk:3128"
CVMFS_QUOTA_LIMIT=20000
[root@t2wn24 ~]# du -hs /var/cache/cvmfs2/
26G /var/cache/cvmfs2/
I'd say that's a fair way over.
> > - Ditto in the case of a dead squid.
>
> Use another squid at a second site to avoid the problem.
>
I'm slightly fuzzy on how cvmfs behaves in the case of
two squids - is it possible to make it use one as a primary
and only fail over to the other? I don't really want to
have half the requests going to an offsite squid in normal
running (and yes, clearly the other option is to have a
pair of squids at each site).
> What actually happens? It goes into offline mode I guess but you don't
> really know?
>
Mostly it breaks, and logs a lot of stuff like this:
Mar 19 06:25:57 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb/data/f6/3a5c974360f3ba28def8ada545fd7fcb30b72a
Mar 19 06:25:57 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cvmfs-stratum-one.cern.ch/opt/lhcb/data/f6/3a5c974360f3ba28def8ada545fd7fcb30b72a
Mar 19 06:25:57 t2wn48 cvmfs2: (lhcb.cern.ch) failed to fetch /lib/html/PARAM_TMVAWeights_v1r0.html (SHA1: f63a5c974360f3ba28def8ada545fd7fcb30b72a)
Mar 19 06:25:57 t2wn48 cvmfs2: (lhcb.cern.ch) failed to open /lib/html/PARAM_TMVAWeights_v1r0.html, CAS key f63a5c974360f3ba28def8ada545fd7fcb30b72a, error code 115
Mar 19 06:31:45 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb/.cvmfspublished
Mar 19 06:31:45 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cvmfs-stratum-one.cern.ch/opt/lhcb/.cvmfspublished
Mar 19 06:31:45 t2wn48 cvmfs2: (lhcb.cern.ch) unable to load checksum from /.cvmfspublished (7), going to offline mode
Mar 19 06:31:45 t2wn48 cvmfs2: (lhcb.cern.ch) catalog load failure while try to retrieve catalog from http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb
Mar 19 06:35:46 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb/.cvmfspublished
Mar 19 06:35:46 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cvmfs-stratum-one.cern.ch/opt/lhcb/.cvmfspublished
Mar 19 06:35:46 t2wn48 cvmfs2: (lhcb.cern.ch) unable to load checksum from /.cvmfspublished (7), going to offline mode
Mar 19 06:35:46 t2wn48 cvmfs2: (lhcb.cern.ch) catalog load failure while try to retrieve catalog from http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb
Mar 19 06:40:02 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cernvmfs.gridpp.rl.ac.uk/opt/lhcb/.cvmfspublished
Mar 19 06:40:02 t2wn48 cvmfs2: (lhcb.cern.ch) switch proxy / retry on http://cvmfs-stratum-one.cern.ch/opt/lhcb/.cvmfspublished
I haven't put much effort into closer investigation though, I
generally just sort the squid out (and by 'generally', I mean
'both times this happened').
Ewan
|