none
CACert revocation server offline RRS feed

  • Question

  • I'm using CACert for certificate verification and in Outlook none of the client certificates can be verified as the server is offline. The root is in Trusted for both HCCE_LOCAL_MACHINE and HCCE_CURRENT_USER.

    The most bizarre thing is: all servers are online and I can download the CRLs. They appear to be correct and I can't understand why there's an error (because verification is OK!). I thought maybe because the root CRL against the class 1 root is so large, there could be timeouts, but using the "-t 30" doesn't change behaviour. You can see that everything is verified! But it still shows up with an error status (Class 3 intermediate) with 1000040.

    If I use "certutil -f -verify -urlfetch <mycer>", then there is no error, but "failed" is present in the output. I don't know what the difference is as -f is not documented on MSDN (references http://blogs.technet.com/b/pki/archive/2006/11/30/basic-crl-checking-with-certutil.aspx and http://technet.microsoft.com/en-us/library/cc732443.aspx#BKMK_verify).

    Note, this problem occurs on Windows 7 as well as Windows 8.1.

    I have the support of the CA authority to also further investigate.

    X:\>certutil -t 30 -verify -urlfetch "20130103 011232 jason@onmicrosoft.cer"
    Issuer:
        CN=CAcert Class 3 Root
        OU=http://www.CAcert.org
        O=CAcert Inc.
    Subject:
        E=jason@thecurls.onmicrosoft.com
        CN=Jason Curl
    Cert Serial Number: 011232

    dwFlags = CA_VERIFY_FLAGS_CONSOLE_TRACE (0x20000000)
    dwFlags = CA_VERIFY_FLAGS_DUMP_CHAIN (0x40000000)
    ChainFlags = CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT (0x40000000)
    HCCE_LOCAL_MACHINE
    CERT_CHAIN_POLICY_BASE
    -------- CERT_CHAIN_CONTEXT --------
    ChainContext.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
    ChainContext.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40)
    ChainContext.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000)

    SimpleChain.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
    SimpleChain.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40)
    SimpleChain.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000)

    CertContext[0][0]: dwInfoStatus=104 dwErrorStatus=0
      Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc.
      NotBefore: 03/01/2013 14:17
      NotAfter: 03/01/2015 14:17
      Subject: E=jason@thecurls.onmicrosoft.com, CN=Jason Curl
      Serial: 011232
      SubjectAltName: RFC822 Name=jason@thecurls.onmicrosoft.com
      f9 3c a2 39 9b 27 0d 84 26 29 7f 9b 23 83 2c 68 56 93 d5 1e
      Element.dwInfoStatus = CERT_TRUST_HAS_NAME_MATCH_ISSUER (0x4)
      Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
      ----------------  Certificate AIA  ----------------
      No URLs "None" Time: 0
      ----------------  Certificate CDP  ----------------
      Verified "Base CRL" Time: 0
        [0.0] http://crl.cacert.org/class3-revoke.crl

      ----------------  Base CRL CDP  ----------------
      No URLs "None" Time: 0
      ----------------  Certificate OCSP  ----------------
      Verified "OCSP" Time: 0
        [0.0] http://ocsp.cacert.org

      --------------------------------
        CRL (null):
        Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc.
        aa ce 2f e9 20 5f 91 1d ce 91 47 51 8f ce b6 55 aa 9b 0b 98
      Application[0] = 1.3.6.1.5.5.7.3.4 Secure Email
      Application[1] = 1.3.6.1.5.5.7.3.2 Client Authentication
      Application[2] = 1.3.6.1.4.1.311.10.3.4 Encrypting File System
      Application[3] = 1.3.6.1.4.1.311.10.3.3
      Application[4] = 2.16.840.1.113730.4.1

    CertContext[0][1]: dwInfoStatus=101 dwErrorStatus=1000040
      Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA
      NotBefore: 23/05/2011 19:48
      NotAfter: 20/05/2021 19:48
      Subject: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc.
      Serial: 0a418a
      ad 7c 3f 64 fc 44 39 fe f4 e9 0b e8 f4 7c 6c fa 8a ad fd ce
      Element.dwInfoStatus = CERT_TRUST_HAS_EXACT_MATCH_ISSUER (0x1)
      Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
      Element.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40)
      Element.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000)
      ----------------  Certificate AIA  ----------------
      Verified "Certificate (0)" Time: 0
        [0.0] http://www.CAcert.org/ca.crt

      ----------------  Certificate CDP  ----------------
      No URLs "None" Time: 0
      ----------------  Certificate OCSP  ----------------
      Verified "OCSP" Time: 0
        [0.0] http://ocsp.CAcert.org/

      --------------------------------
      Issuance[0] = 1.3.6.1.4.1.18506

    CertContext[0][2]: dwInfoStatus=109 dwErrorStatus=0
      Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA
      NotBefore: 30/03/2003 14:29
      NotAfter: 29/03/2033 14:29
      Subject: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA
      Serial: 00
      13 5c ec 36 f4 9c b8 e9 3b 1a b2 70 cd 80 88 46 76 ce 8f 33
      Element.dwInfoStatus = CERT_TRUST_HAS_EXACT_MATCH_ISSUER (0x1)
      Element.dwInfoStatus = CERT_TRUST_IS_SELF_SIGNED (0x8)
      Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100)
      ----------------  Certificate AIA  ----------------
      No URLs "None" Time: 0
      ----------------  Certificate CDP  ----------------
      Verified "Base CRL" Time: 12
        [0.0] https://www.cacert.org/revoke.crl

      ----------------  Certificate OCSP  ----------------
      No URLs "None" Time: 0
      --------------------------------

    Exclude leaf cert:
      b3 cf c2 7a ca 14 01 93 ea dc 46 c0 c0 6e b6 d1 b3 7f 39 b3
    Full chain:
      f3 71 fa 99 64 60 c4 01 75 62 d9 f8 94 15 bc 11 2f 1b c2 bd
      Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc.
      NotBefore: 03/01/2013 14:17
      NotAfter: 03/01/2015 14:17
      Subject: E=jason@thecurls.onmicrosoft.com, CN=Jason Curl
      Serial: 011232
      SubjectAltName: RFC822 Name=jason@thecurls.onmicrosoft.com
      f9 3c a2 39 9b 27 0d 84 26 29 7f 9b 23 83 2c 68 56 93 d5 1e
    The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613)
    ------------------------------------
    Revocation check skipped -- server offline
    Cert is an End Entity certificate
    Leaf certificate revocation check passed
    CertUtil: -verify command completed successfully.

    Friday, September 12, 2014 5:58 AM

All replies

  • Hi Jason Curl,

    As the issue here is mostly related with CA, I will help to move this thread to Windows Server/Security forum.

    The reason why we doing this is that in the more suitable forum, you will get more professional help to deal with the related issue.

    Thank you for your understanding.

    Best regards


    Michael Shao
    TechNet Community Support

    Monday, September 15, 2014 3:01 AM
    Moderator
  • Thankyou Michael,

    Until now, I haven't solved the problem. I thought this might be related to timeouts (with the download time being 15s and 20s) as per TechNet, but increasing this timeout to 60s and 90s which is ample time to download the CRL is also not working with certutil and outlook 2010/2013.

    I would definitely appreciate any tips for further debugging (best on the client side, server side checks would take longer).

    Monday, September 15, 2014 5:55 AM
  • I was able to download the CRLs so they are indeed there. What I would do to narrow this down is use the same certutil command but do it on each certificate in the chain. This would narrow down which server specifically is having the problem as its not clear in the output. The odd use of different AIA, OCSP and HTTP makes the architecture a bit odd - but I know its a community driven provider.

    I also noticed that your user certificate doesnt have an AIA defined, so it may be difficult to properly define the AIA chain. Does your OS have the certificate for the issuer in your Trusted root or intermediate store? It's the intermediate CA called "CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc." which issued your user certificate . Its an odd name as its really not the root. This may be the issue itself.


    Mark B. Cooper, President and Founder of PKI Solutions Inc., former Microsoft Senior Engineer and subject matter expert for Microsoft Active Directory Certificate Services (ADCS). Known as “The PKI Guy” at Microsoft for 10 years.

    Tuesday, September 16, 2014 5:35 AM
  • There are two things that I've determined:

    1) The URL originally had a redirect from https://www.cacert.org => https://crl.cacert.org and this resulted in an error due to the HTTP 301 redirect.

    2) For one week, they removed the redirect (it was reactivated yesterday). During this time, the CRL is about 5MB. If I used certutil -url https://www.cacert.org, modified the GUI for the timeout, waited about 60s for the download, the CRL was cached and certificates in Outlook could be verified.

    What's odd though, if I modified the group policy for my computer with a timeout of 120s (default 15s), Outlook would still time out.

    Tuesday, September 16, 2014 5:43 AM
  • Hi Jason,

    After you modified the group policy with a timeout of 120 seconds, are you able to download CRL successfully using certutil command?

    In addition, would you please tell us how are DNS settings configured of this machine?

    Best Regards,
    Amy

    Friday, September 19, 2014 2:50 AM
    Moderator
  • Hello Amy,

    Setting the GPO to 120s did not resolve the issue above. "certutil -verify -urlfetch <mycer.cer>" was still unsuccessful. I diff'ed the outputs and they didn't differ in any significant way.

    On all machines, they connect to a Win2k8R2 (Win7 x64) AD server, hence that's the DNS server. The AD has a GPO to add the Class1 from CACert and the Class 3 from CACert in the trusted root for COMPUTER. Using MMC on the local computer, the 2 roots are visible in User and LocalComputer in the Trusted Root Certification Authorities; Intermediate Certificate Authorities (4 locations).

    I did some more experimentation and after downloading the CRL and double clicking on it, installing it in the store MY\Intermediate Certification Authorities (the default), then programs such as Outlook do see the CR and the chain of trust is then verified.

    Is there any way to get more diagnostics out of the tool as to why certutil fails, with -t 60, or with the modified GPO (Computer Configuration -> Policies -> Windows Settings -> Public Key Policies -> Certificate Path Validation Settings, tab Network Retrieval, URL timeout 60, Cumulative timeout 90).

    Note, in case of testing, now that the redirect has been activated, a different error occurs that the CRL can't be downloaded due to HTTP error 301 (and it's much slower).

    I'm going to see if there's a way to download the CRL using say, wget, and then push this out to all machines. I have no idea how this could be done for now.

    Friday, September 19, 2014 7:08 PM
  • Hello Jason,

    Thanks for your post.

    I have tried to run the following commands to retrieve the crl,  only use the HTTP path can successed retrieve the crl if we set the timeout value to 180. I think because I do not have the root ca cert, so i am failed by using https path to retrieve the crl.

    certutil -url https://crl.cacert.org/class3-revoke.crl
    certutil -url http://crl.cacert.org/class3-revoke.crl

    In this case, I think the issue should be caused by the CRL size is too large and clients needs time to download it from the server. I think you can enable the OCSP on a server to workaround this issue.

    In order for applications to determine if a certificate has been revoked, the application examines the CRL Distribution Point (CDP) extension in the certificate. This extension will have information on locations where the CRL can be obtained. These locations are normally either HTTP or LDAP locations.

    The application then can go to those locations to download the CRL. There are, however, some potential issues with this scenario. CRLs over time can get rather large depending on the number of certificates issued and revoked. If CRLs grow to a large size, and many clients have to download CRLs, this can have a negative impact on network performance. More importantly, by default Windows clients will timeout after 15 seconds while trying to download a CRL. Additionally, CRLs have information about every currently valid certificate that has been revoked, which is an excessive amount of data given the fact that an application may only need the revocation status for a few certificates. So, aside from downloading the CRL, the application or the OS has to parse the CRL and find a match for the serial number of the certificate that has been revoked.

    With the above limitations, which mostly revolve around scalability, it is clear that there are some drawbacks to using CRLs. Hence, the introduction of Online Certificate Status Protocol (OCSP). OCSP reduces the overhead associated with CRLs. There are server/client components to OCSP: The OCSP responder, which is the server component, and the OCSP Client. The OCSP Responder accepts status requests from OCSP Clients. When the OCSP Responder receives the request from the client it then needs to determine the status of the certificate using the serial number presented by the client. First the OCSP Responder determines if it has any cached responses for the same request. If it does, it can then send that response to the client. If there is no cached response, the OCSP Responder then checks to see if it has the CRL issued by the CA cached locally on the OCSP. If it does, it can check the revocation status locally, and send a response to the client stating whether the certificate is valid or revoked. The response is signed by the OCSP Signing Certificate that is selected during installation. If the OCSP does not have the CRL cached locally, the OCSP Responder can retrieve the CRL from the CDP locations listed in the certificate. The OCSP Responder then can parse the CRL to determine the revocation status, and send the appropriate response to the client.

    For how to configuring OCSP, please refer to the article as below:

    http://blogs.technet.com/b/askds/archive/2009/06/24/implementing-an-ocsp-responder-part-i-introducing-ocsp.aspx
    http://blogs.technet.com/b/askds/archive/2009/06/25/implementing-an-ocsp-responder-part-ii-preparing-certificate-authorities.aspx
    http://blogs.technet.com/b/askds/archive/2009/06/29/implementing-an-ocsp-responder-part-iii-configuring-ocsp-for-use-with-enterprise-cas.aspx
    http://blogs.technet.com/b/askds/archive/2009/08/21/implementing-an-ocsp-responder-part-vi-configuring-custom-ocsp-uris-via-group-policy.aspx

    Thanks.

    Friday, September 26, 2014 1:25 AM
  • Thanks for the details. I suspected initially that the CRL was too large. But why doesn't the GPO work when I request it to be increased? That was part of my original question.

    When the admins of CAcert removed the redirect for the root CRL, I stopped getting the error in certutil that says HTTP_301. Then it started downloading the CRL, but monitoring using Network Message Analyser, I see a TCP RST sent from the local machine, indicating that a thread closed before the download completed without actually closing the socket (else it should be TCP FIN). And certutil indicated that the certificate was offline.

    Changing the GPO to 90s should have been enough, the average download time was about 20s. But Outlook 2010/2013 ignored this and so did certutil. As Technet documentation indicates this is part of the OS, either I was fiddling with the wrong GPO, -t 30 didn't do as advertised, or that wasn't the problem at all. Look at the original log, you'll see that everything was "verified", but still resulted in offline. Why?

    The links above are for setting up my own OSCP, this is not what I want to do.

    Thanks & Best Regards,
    Jason.

    Friday, September 26, 2014 6:21 AM
  • Hi Jason,

    Thanks for the reply.

    According to the logs, we can find the revocation check failed for this leaf cert, not all are pass the check.
    -----------------
     Revocation check skipped -- server offline
     Cert is an End Entity certificate
     Leaf certificate revocation check passed
     CertUtil: -verify command completed successfully.
    ----------------------

    From the logs, it seems that you didnot publish the CRL to DeltaCRL Location, this is not recommended. I think you can try to publish the CRL to DeltaCRL, and renew a new certificate to check if it helps.

    For your issue, I think the local client failed to download the CRL in this period. have you try to set GP to 180s or longer?

    Thanks

    Tuesday, September 30, 2014 1:40 PM
  • Hello Bryan,

    It's clear that this certificate couldn't be checked because it's error status shows so:

    CertContext[0][1]: dwInfoStatus=101 dwErrorStatus=1000040
      Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA
      Subject: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc.

    But this certificate (the self-signed root), is OK, it's error status is zero.

    CertContext[0][2]: dwInfoStatus=109 dwErrorStatus=0
      Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA
      Subject: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA

    I don't understand how you get to your conclusion.

    Secondly, while it's not testable at this moment (because of a redirect), when I requested the admins to remove the redirect, that's the log I got. It would download in 20s, I had the GP to 90s. And you also see in the logs:

      ----------------  Certificate CDP  ----------------
      Verified "Base CRL" Time: 12
        [0.0] https://www.cacert.org/revoke.crl

    So it should have taken 12 seconds.

    What server is failing? how do I get more information? The logs only show that something is failing, but everything of interest is 'verified'.

    Tuesday, September 30, 2014 2:08 PM
  • Hi Jason,

    I tried to test with my cacert certificate now but the redirect is again in place. Without the redirect, HTTPS is used in the CDP which - as per RFC 5280 -  CAs SHOULD NOT do due to potential issues with circular references. Relying parties "MUST be prepared for the possibility that this will result in unbounded recursion" - so it would be undertstandable if certutil would not follow all the recursions.

    The CRL needed to validate the certificate chain for the CDP HTTPS URL is again the same CRL as the same CA with the same CDP has also signed the server's certificate.

    So I think what happens here is that you can download the CRL "once" - for the purpose of validating the e-mail certificate - but you cannot download it "twice" - or actually infinitely often - as it would be required to validate the certificate used for SSL in the CDP URL(s).

    Elke

    Tuesday, September 30, 2014 5:06 PM