OK, David, thnaks.
I'll do some more research once everything is running.
I've rolled back to previous version, 1.87-1, and the messages have
dried up, some clusters are working OK, about 80% back to capacity here.
Our storage seems to be online.
(BTW: BouncyCastle always makes me shudder. Whenever it is mentioned, I
suffer the horrors of Java versionitis.)
Cheers,
Ste
On 28/11/17 15:04, David Groep wrote:
> Hi Stephen,
>
> On 2017-11-28 15:58, Stephen Jones wrote:
>> Did just ARGUS break, or diod other things? Daniela says her UI was broken.
>> WHat was the software and the version? What else is broken besides ARGUS,
>> UIs... I thing our DPM is failing too.
> It seems it affects Java/BouncyCastle on EL6, maybe with specific versions
> of BouncyCastle that are relatively old, and any software derived from it.
> This includes ARGUS and VOMS, and of course then affects services
> that talk to such services. If you have DPM linked to ARGUS, it will so
> fail.
> One would have to check if the version of Argus is up to date and
> supports SHA-2, and for the VOMS issue if the server does not
> inadvertently sends an old version.
>
> # rpm -qa | grep -i argus
> argus-pap-1.6.2-1.el6.noarch
> argus-pepcli-2.2.0-1.el6.x86_64
> emi-argus-1.6.0-1.el6.noarch
> argus-pdp-1.6.0-1.el6.noarch
> yaim-argus_server-1.6.0-1.el6.noarch
> argus-pep-api-c-2.2.0-1.el6.x86_64
> argus-pep-server-1.6.1-1.el6.noarch
> argus-pdp-pep-common-1.4.0-2.el6.noarch
> argus-pep-common-2.3.0-1.el6.noarch
>
> I cannot test form non-UK locations for other combinations :((
> If we cannot resolve this, Jens as the CA manager can trigger the next
> step (provided EGI ops is ready as well - I'm checking for emergency
> 'roll-forward' back-to-the-future right now with the old SHA-1 ICA)
>
> DavidG.
>
> PS:
> To get better resolution, please keep everyone in CC that can help. This
> thread is fragmenting uickly...
>
>
>
>> Cheers,
>>
>> Ste
>>
>>
>>
>> On 28/11/17 14:26, Matt Doidge wrote:
>>> Just an FYI that I've had great success rolling back thanks to the link
>>> Steve shared.
>>>
>>> Just in case it's useful, the dummy yum repo snippet I used was:
>>>
>>> [egi-igtf-187]
>>> name=egi-igtf-187
>>> baseurl=https://egi-igtf.ndpf.info/distribution/egi-1.87-1/ca-policy-egi-core-1.87-1/
>>>
>>> enabled=0
>>> gpgcheck=1
>>> gpgkey=https://egi-igtf.ndpf.info/distribution/egi-1.87-1/GPG-KEY-EUGridPMA-RPM-3
>>>
>>>
>>> Cheers,
>>> Matt
>>>
>>> On 28/11/17 13:50, John Kewley wrote:
>>>> Just to let you know that I'm aware of the issue; I wasn't involved in this
>>>> release so wasn't involved in any testing, but I'll see if I can work out
>>>> the issue.
>>>>
>>>> My understanding is that Jens is out of the office, but I'm hoping he'll be
>>>> online at some point this afternoon.
>>>>
>>>> FYI, I haven't yet updated the CA repository, so the "old" 2B certificate
>>>> should still be downloadable from there:
>>>> http://www.ngs.ac.uk/ukca/certificates/cacerts
>>>>
>>>> Cheers
>>>>
>>>> JK
>>>>
>>>>> -----Original Message-----
>>>>> From: Testbed Support for GridPP member institutes [mailto:TB-
>>>>> [log in to unmask]] On Behalf Of Robert Frank
>>>>> Sent: 28 November 2017 13:45
>>>>> To: [log in to unmask]
>>>>> Subject: Re: We think trust anchors 1.88-1 breaks on SL6
>>>>>
>>>>> Have a look here:
>>>>>
>>>>> http://mirror.tier2.hep.manchester.ac.uk/Repositories/EMI/CA/
>>>>>
>>>>> Robert
>>>>>
>>>>> On 28/11/17 13:36, Stephen Jones wrote:
>>>>>> On 28/11/17 13:32, Daniela Bauer wrote:
>>>>>>> How did you roll back to 1.87 ?
>>>>>>
>>>>>> They've taken it away.
>>>>>>
>>>>>> (note to self: always download and KEEP the last good CAs)
>>>>>>
>>>>>> Ste
>>>>>>
>>>>>>
>>>>>>> Cheers,
>>>>>>> Daniela
>>>>>>>
>>>>>>> On 28 November 2017 at 13:30, Robert Frank
>>>>> <[log in to unmask] <mailto:[log in to unmask]>>
>>>>> wrote:
>>>>>>> I've seen it as well in Manchester when I tried to update this
>>>>>>> morning. I've rolled everything back to 1.87 for now.
>>>>>>> I got the impression that it works when both, the server and the
>>>>>>> client use the same version, but more testing is needed to confirm
>>>>>>> this.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Robert
>>>>>>>
>>>>>>> On 28/11/17 13:21, Stephen Jones wrote:
>>>>>>>
>>>>>>> Don't update to 1.88-1
>>>>>>>
>>>>>>> We have same problems too!
>>>>>>>
>>>>>>> Working on it; site is down because ARGUS (SL6) is clobbered
>>>>>>> by this...
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>>
>>>>>>> Ste
>>>>>>>
>>>>>>>
>>>>>>> On 28/11/17 13:17, Daniela Bauer wrote:
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> the latest trust anchor release contains this chage:
>>>>>>>
>>>>>>> * updated UKeScience 2B ICA based on a SHA-2 family digest
>>>>>>> (UK)
>>>>>>>
>>>>>>> When I try and run the cvmfs UI on SL6 I get the following
>>>>>>> error:
>>>>>>>
>>>>>>> lx01:~ > voms-proxy-init --voms gridpp
>>>>>>> Enter GRID pass phrase for this identity:
>>>>>>> Contacting voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>
>>>>>>> <http://voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>>
>>>>>>> [/C=UK/O=eScience/OU=Imperial/L=Physics/CN=voms03.gridpp.ac.u
>>>>> k
>>>>>>> <http://voms03.gridpp.ac.uk> <http://voms03.gridpp.ac.uk>]
>>>>>>> "gridpp"...
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>
>>>>>>> <http://voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>> for VO gridpp:
>>>>>>> java.security.cert.CertificateException: The peer's
>>>>>>> certificate with subject's DN CN=voms03.gridpp.ac.uk
>>>>>>> <http://voms03.gridpp.ac.uk>
>>>>>>> <http://voms03.gridpp.ac.uk>,L=Physics,OU=Imperial,O=eScience,C=
>>>>> UK
>>>>>>> was rejected. The peer's certificate status is: FAILED The
>>>>>>> following validation errors were found:
>>>>>>> error at position 0 in chain, problematic certificate
>>>>>>> subject: CN=voms03.gridpp.ac.uk
>>>>>>> <http://voms03.gridpp.ac.uk>
>>>>>>> <http://voms03.gridpp.ac.uk>,L=Physics,OU=Imperial,O=eScience,C=
>>>>> UK
>>>>>>> (category: CRL): Can not verify the CRL as its issuer's
>>>>>>> public key is unknown or can not be validated Cause:
>>>>>>> Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>
>>>>>>> <http://voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>> for VO gridpp: peer
>>>>>>> not authenticated
>>>>>>> Error contacting voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>
>>>>>>> <http://voms03.gridpp.ac.uk:15000
>>>>>>> <http://voms03.gridpp.ac.uk:15000>> for VO gridpp: REST
>>>>>>> and legacy VOMS endpoints failed.
>>>>>>> Contacting voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>
>>>>>>> <http://voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>>
>>>>>>> [/C=UK/O=eScience/OU=Oxford/L=OeSC/CN=voms02.gridpp.ac.uk
>>>>>>> <http://voms02.gridpp.ac.uk> <http://voms02.gridpp.ac.uk>]
>>>>>>> "gridpp"...
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>
>>>>>>> <http://voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>> for VO gridpp:
>>>>>>> java.security.cert.CertificateException: The peer's
>>>>>>> certificate with subject's DN CN=voms02.gridpp.ac.uk
>>>>>>> <http://voms02.gridpp.ac.uk>
>>>>>>> <http://voms02.gridpp.ac.uk>,L=OeSC,OU=Oxford,O=eScience,C=UK
>>>>>>> was rejected. The peer's certificate status is: FAILED The
>>>>>>> following validation errors were found:
>>>>>>> error at position 0 in chain, problematic certificate
>>>>>>> subject: CN=voms02.gridpp.ac.uk
>>>>>>> <http://voms02.gridpp.ac.uk>
>>>>>>> <http://voms02.gridpp.ac.uk>,L=OeSC,OU=Oxford,O=eScience,C=UK
>>>>>>> (category: CRL): Can not verify the CRL as its issuer's
>>>>>>> public key is unknown or can not be validated Cause:
>>>>>>> Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>
>>>>>>> <http://voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>> for VO gridpp: peer
>>>>>>> not authenticated
>>>>>>> Error contacting voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>
>>>>>>> <http://voms02.gridpp.ac.uk:15000
>>>>>>> <http://voms02.gridpp.ac.uk:15000>> for VO gridpp: REST
>>>>>>> and legacy VOMS endpoints failed.
>>>>>>> Contacting voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>
>>>>>>> <http://voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>>
>>>>>>> [/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk
>>>>>>> <http://voms.gridpp.ac.uk> <http://voms.gridpp.ac.uk>]
>>>>>>> "gridpp"...
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>
>>>>>>> <http://voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>> for VO gridpp:
>>>>>>> java.security.cert.CertificateException: The peer's
>>>>>>> certificate with subject's DN CN=voms.gridpp.ac.uk
>>>>>>> <http://voms.gridpp.ac.uk>
>>>>>>> <http://voms.gridpp.ac.uk>,L=HEP,OU=Manchester,O=eScience,C=U
>>>>> K
>>>>>>> was rejected. The peer's certificate status is: FAILED The
>>>>>>> following validation errors were found:
>>>>>>> error at position 0 in chain, problematic certificate
>>>>>>> subject: CN=voms.gridpp.ac.uk <http://voms.gridpp.ac.uk>
>>>>>>> <http://voms.gridpp.ac.uk>,L=HEP,OU=Manchester,O=eScience,C=U
>>>>> K
>>>>>>> (category: CRL): Can not verify the CRL as its issuer's
>>>>>>> public key is unknown or can not be validated Cause:
>>>>>>> Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Certificate validation error: Can not verify the CRL as
>>>>>>> its issuer's public key is unknown or can not be validated
>>>>>>> Cause: Certification path could not be validated. Cause:
>>>>>>> NullPointerException
>>>>>>> Error contacting voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>
>>>>>>> <http://voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>> for VO gridpp: peer not
>>>>>>> authenticated
>>>>>>> Error contacting voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>
>>>>>>> <http://voms.gridpp.ac.uk:15000
>>>>>>> <http://voms.gridpp.ac.uk:15000>> for VO gridpp: REST and
>>>>>>> legacy VOMS endpoints failed.
>>>>>>> None of the contacted servers for gridpp were capable of
>>>>>>> returning a valid AC for the user.
>>>>>>> User's request for VOMS attributes could not be fulfilled.
>>>>>>>
>>>>>>>
>>>>>>> It works on SL7.
>>>>>>>
>>>>>>> This error is fairly deadly for a lot of stuff we are
>>>>>>> doing here.
>>>>>>>
>>>>>>> Any ideas ?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Daniela
>>>>>>>
>>>>>>>
>>>>>>> -- Sent from the pit of despair
>>>>>>>
>>>>>>> -----------------------------------------------------------
>>>>>>> [log in to unmask]
>>>>>>> <mailto:[log in to unmask]>
>>>>>>> <mailto:[log in to unmask]
>>>>>>> <mailto:[log in to unmask]>>
>>>>>>> HEP Group/Physics Dep
>>>>>>> Imperial College
>>>>>>> London, SW7 2BW
>>>>>>> Tel: +44-(0)20-75947810 <tel:%2B44-%280%2920-75947810>
>>>>>>> http://www.hep.ph.ic.ac.uk/~dbauer/
>>>>>>> <http://www.hep.ph.ic.ac.uk/%7Edbauer/>
>>>>>>> <http://www.hep.ph.ic.ac.uk/%7Edbauer/
>>>>>>> <http://www.hep.ph.ic.ac.uk/%7Edbauer/>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sent from the pit of despair
>>>>>>>
>>>>>>> -----------------------------------------------------------
>>>>>>> [log in to unmask] <mailto:[log in to unmask]>
>>>>>>> HEP Group/Physics Dep
>>>>>>> Imperial College
>>>>>>> London, SW7 2BW
>>>>>>> Tel: +44-(0)20-75947810
>>>>>>> http://www.hep.ph.ic.ac.uk/~dbauer/
>>>>> <http://www.hep.ph.ic.ac.uk/%7Edbauer/>
>>>>>>
>
--
Steve Jones [log in to unmask]
Grid System Administrator office: 220
High Energy Physics Division tel (int): 43396
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 3396
University of Liverpool http://www.liv.ac.uk/physics/hep/
|