Hi Dan,
Sorry for this long reply. I've seen this before, but I don't know when etc.
I'm CC'ing it to some individuals and LCG-ROLLOUT.
--PART1-----------
Anyway, it seems that some Java object is reading a certificate file
called fdf90b95.0
in the /etc/grid-security/certificates directory. This file is really a
link
to DZeScience.pem, which is itself an authority (CA) certificate,
which is needed to establish trust. So far so good.
It seems that this Java object (called a TrustStorage, with a method
called loadAnchors()) expects each CA certificate to have an associated file
which would have the same name, but with the .r0 extension, instead of
the .0 extension. That *.r0 file contains revocations, i.e. a list of
certificates that are no good any-more. OK, that's part one.
--PART2-----------
I have found that each certificate authority delivers its certificates
in an RPM, e.g.
[root@imageserver RPMS]# rpm -qlp ./ca_DZeScience-1.52-1.noarch.rpm
/etc/grid-security/certificates/DZeScience.pem
/etc/grid-security/certificates/fdf90b95.0
/etc/grid-security/certificates/0a49430a.0
...
You can see that this RPM contains the CA cert, 0a49430a.0 and fdf90b95.0.
The latter both link to DZeScience.pem; you'd have to unpack it to see
that. The name of the 0a49430a.0 link was determined using a
hash command, i.e.
[root@imageserver tmp]# openssl x509 -hash -in DZeScience.pem -noout
0a49430a
Sadly, I have no idea how the name of the second link, fdf90b95.0, is
found. It must be some
other kind of hash function. That's part two.
--PART3-----------
Now here's the round up: a cron job, fetch-crl, is responsible on each
worker node
to bring down new revocation files for each CA certificate. It names
those files
by taking hash function of the file (see above) yielding 0a49430a.r0.
In short,
it does not make fdf90b95.r0.
Hence, TrustStorage.loadAnchors() barfs out a message when it tries to
find the CRL (i.e. .r0 version) of fdf90b95.0. That's part 3.
--PART4-----------
Finally, there may (probably is) some logical explanation for all this
jiggery
pokery. I just don't know what it is. I'm throwing open the floor.
Hopefully,
someone has seen this before and knows what to do. If not, too bad.
I hope this helps to progress this issue. It would be good to know if it
happens at other sites.
Cheers,
Steve
On 04/23/2013 11:47 AM, Dan Protopopescu wrote:
> Dear Stephen, Janusz,
>
> Could we somehow get rid of these errors that clog the stdout of the NA62 grid jobs ?
>
> No files found matching /etc/grid-security/certificates/fdf90b95.r0
> Error while reading certificates or CRLs: No files found matching /etc/grid-security/certificates/fdf90b95.r0
> CRL loading for CA /etc/grid-security/certificates/fdf90b95.0 failed, depending on configuration the certificates from the CA might be refused.
> java.security.cert.CertificateException: Error while reading certificates or CRLs: No files found matching /etc/grid-security/certificates/fdf90b95.r0
> at org.glite.security.util.FileCertReader.readFiles(FileCertReader.java:247)
> at org.glite.security.util.FileCertReader.readCRLs(FileCertReader.java:190)
> at org.glite.security.util.FullTrustAnchor.loadCRL(FullTrustAnchor.java:216)
> at org.glite.security.util.FullTrustAnchor.initCAInfo(FullTrustAnchor.java:170)
> at org.glite.security.util.FullTrustAnchor.<init>(FullTrustAnchor.java:136)
> at org.glite.security.util.TrustStorage.loadAnchors(TrustStorage.java:101)
> at org.glite.security.util.TrustStorage.<init>(TrustStorage.java:63)
> at org.glite.security.trustmanager.OpensslCertPathValidator.<init>(OpensslCertPathValidator.java:86)
> at org.glite.security.trustmanager.OpensslTrustmanager.<init>(OpensslTrustmanager.java:51)
> at org.glite.security.trustmanager.ContextWrapper.init(ContextWrapper.java:451)
> at org.glite.security.trustmanager.ContextWrapper.<init>(ContextWrapper.java:266)
> at org.glite.security.trustmanager.axis.AXISSocketFactory.create(AXISSocketFactory.java:83)
> at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)
> at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)
> at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)
> at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
> at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
> at org.apache.axis.client.Call.invoke(Call.java:2767)
> at org.apache.axis.client.Call.invoke(Call.java:2443)
> at org.apache.axis.client.Call.invoke(Call.java:2366)
> at org.apache.axis.client.Call.invoke(Call.java:1812)
> at org.na62.grid.controller.GliteTransferSoapBindingStub.submit(GliteTransferSoapBindingStub.java:107)
> at org.na62.grid.controller.Client.main(Client.java:83)
>
> http://na62.gla.ac.uk/outputs/na62run17002.out
>
> Best regards,
> Dan
>
> --
> Dan PROTOPOPESCU, University of Glasgow, UK
> W:+44(0)141-330-4197 M:+44(0)794-046-3355
> http://ppewww.physics.gla.ac.uk/~protopop/
>
>
>
--
Steve Jones [log in to unmask]
System Administrator office: 220
High Energy Physics Division tel (int): 42334
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
University of Liverpool http://www.liv.ac.uk/physics/hep/
|