 |
 |
I submitted a ticket as suggested but I didn't find the suggest support unit to assign it to, so if anyone can help it on its way please do. https://ggus.eu/index.php?mode=ticket_info&ticket_id=150179&come_from=submit ________________________________ From: Testbed Support for GridPP member institutes <[log in to unmask]> on behalf of George, Simon <[log in to unmask]> Sent: 06 January 2021 15:12 To: [log in to unmask] <[log in to unmask]> Subject: Re: [EXT] December 2020 WLCG A/R reports Re. RHUL/ATLAS: The site was working fine for ATLAS analysis and production all month. The low A/R is due to the "unknown" status of the test "org.atlas.WN-swspace-/atlas/Role=lcgadmin" for the first half of December: https://monit-grafana.cern.ch/d/m7XtZsEZk4/wlcg-sitemon-historical-tests?orgId=20&from=1606780800000&to=1609459199999&var-vo=atlas&var-dst_tier=2&var-dst_country=UK&var-dst_federation=UK-London-Tier2&var-dst_experiment_site=UKI-LT2-RHUL&var-service_flavour=All&var-dst_hostname=All&var-metric=org.atlas.WN-swspace-%2Fatlas%2FRole%3Dlcgadmin&var-status=All<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-grafana.cern.ch%2Fd%2Fm7XtZsEZk4%2Fwlcg-sitemon-historical-tests%3ForgId%3D20%26from%3D1606780800000%26to%3D1609459199999%26var-vo%3Datlas%26var-dst_tier%3D2%26var-dst_country%3DUK%26var-dst_federation%3DUK-London-Tier2%26var-dst_experiment_site%3DUKI-LT2-RHUL%26var-service_flavour%3DAll%26var-dst_hostname%3DAll%26var-metric%3Dorg.atlas.WN-swspace-%252Fatlas%252FRole%253Dlcgadmin%26var-status%3DAll&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867546706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5DfVQRtSOgi1DlyRGPqHvZsxJxLtpp2DpDVoZrTkyAc%3D&reserved=0> This test does quite a lot of exploring of the local filesystems (e.g. df) and is not protected against timeouts. Unfortunately our WNs had a hanging network mount due to a misconfiguration of a test network filesystem on our WNs. This filesystem was nothing to do with ATLAS jobs which were going on regardless. However the test got stuck doing a 'df' because of this mount. Consequently the whole test was timing out resulting in status "unknown". I'm not sure if this means the numbers are wrong, but they certainly don't represent the ATLAS experience of the site which was much closer to 100%. ATLAS VO experts please feel free to contradict me! Clearly we got a bit unlucky and the test failure was still useful to flag this problem to us (eventually after a lot of head scratching). Should we be asking for a correction? I would be grateful for feedback. Cheers, Simon ________________________________ From: Testbed Support for GridPP member institutes <[log in to unmask]> on behalf of Doidge, Matt <[log in to unmask]> Sent: 05 January 2021 17:30 To: [log in to unmask] <[log in to unmask]> Subject: [EXT] December 2020 WLCG A/R reports Hello all, The provisional WLCG A/R reports are in, scanning over them I see the following below 90% Atlas [1] QMUL 45%/68% - this is reasonably well understood I believe RHUL 56%/57% SHEFFIELD 53%/53% GLASGOW 82%/82% CMS [2] All good! LHCB [3] QMUL 0%/23% (do these seem okay or a bit low?) UCL 0%/0% (understood) Manchester 43%/43% (I believe this is a problem with the tests and not the site as was discussed in Ops, and thus should be protested). Sheffield 51%/53% Durham 73%/73% Glasgow 0%/0% (again it is just the tests that are failing, and again as discussed in Ops this is understood but needs fixing) Birmingham 0%/0% (same situation as with Glasgow) Bristol 86%/86% (I feel a bit mean singling you guys out for just a few % under) Cambridge 0%/0% (another VAC problem, or just not running anymore?) If anyone thinks these numbers are wrong please let us know, or assign a GGUS ticket to "Grid Monitoring Support Unit - 3rd level experts". Cheers all, Matt [1] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_ATLAS_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=owd%2FjEKgdc%2FrvF1ybvMdsfgyUSC7Cz%2BCKqCimYRYxwk%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_ATLAS_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867556659%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=T%2BgiRcSi1906S30cyk%2Bl%2FicDB3n6JyXxR0FD7mc2tyA%3D&reserved=0> [2] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_CMS_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hSo7IkR7TfC1%2BwFXXybuIFlHd83lgEdg7R2HIiR7u2E%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_CMS_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867556659%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BI6aB5R6PbOHZ%2BPJS5iDAJqVjgyXx0%2F2MuVV927LYDI%3D&reserved=0> [3] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_LHCB_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=usJI5O7XJxu6HMdljk7tf%2F5MYrM8NORSex2mHJQ%2FCfk%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmonit-wlcg-sitemon.web.cern.ch%2Fmonit-wlcg-sitemon%2Freports%2F2020%2F202012%2Fwlcg%2FWLCG_All_Sites_LHCB_Dec2020.pdf&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867566617%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2FAi2ylTAE2cy66efvz2vR2oPk9oLpMFF0p11HOgO%2Fr0%3D&reserved=0> ######################################################################## To unsubscribe from the TB-SUPPORT list, click the following link: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DTB-SUPPORT%26A%3D1&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=U261A8Oyg8zKyGsl3I5G6%2FpbVevp94sj6R9xv3s%2BzQ8%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DTB-SUPPORT%26A%3D1&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867566617%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eq%2BLR2G9UWy7rYVgu75O5FDklYnUvwYpZEt%2Bfw4GS8o%3D&reserved=0> This message was issued to members of https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2FTB-SUPPORT&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jc1oDiAtxX0xsRxNCQVzW4gHMtuZFmMrBJ%2BTuPwD3Hc%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2FTB-SUPPORT&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867576576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZMQR%2BNjcYKtD0am0Dfew5brf93jX2D2X6tj0RoszfuA%3D&reserved=0>, a mailing list hosted by https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2F&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AoFzZb8VY3f%2BvLt8kgMlV7Q9mzH6YNGuuUfQRs3hfeA%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2F&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867576576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j%2FAe2ji5cOnk6PRZ6d9CjkRMt2xhj7HS6HMozXzyhMk%3D&reserved=0>, terms & conditions are available at https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=04%7C01%7CS.George%40RHUL.AC.UK%7C89f345d13b6d4642861808d8b19fa5cb%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637454646941913529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0li25hSL%2FFD0Rw3R0tpuFmDdfl0KrAfawZ1T4mmwbXo%3D&reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867586540%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ol392%2B6YQ618wVdfQQXBspXqEJ7Y7NhXhHib7A3KwPY%3D&reserved=0> This email, its contents and any attachments are intended solely for the addressee and may contain confidential information. In certain circumstances, it may also be subject to legal privilege. Any unauthorised use, disclosure, or copying is not permitted. If you have received this email in error, please notify us and immediately and permanently delete it. Any views or opinions expressed in personal emails are solely those of the author and do not necessarily represent those of Royal Holloway, University of London. It is your responsibility to ensure that this email and any attachments are virus free. ________________________________ To unsubscribe from the TB-SUPPORT list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=TB-SUPPORT&A=1<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DTB-SUPPORT%26A%3D1&data=04%7C01%7CS.George%40RHUL.AC.UK%7C3f60f04b87d0405cbb6b08d8b255767a%7C2efd699a19224e69b601108008d28a2e%7C0%7C0%7C637455427867586540%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xlaiphlNIdQ2jAORcV3VWhHtAOTKRKH8T9WnuVqduBg%3D&reserved=0> ######################################################################## To unsubscribe from the TB-SUPPORT list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=TB-SUPPORT&A=1 This message was issued to members of www.jiscmail.ac.uk/TB-SUPPORT, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/