Field Report – PKI Spotlight Rocked My World Again

This is another blog in the series about how PKI Spotlight is changing how customers realize improved operational efficiency, enhanced security, and better visibility of their PKI and HSM key management environments, and how it’s changing our world as PKI practitioners. All the names and specifics are redacted to protect the confidentiality of all involved – other than me!

HSM management has never been easy – is that all about to change?

In another one of those “pinch me, is this true” moments, we regularly work with our PKI Spotlight customers through our Co-Management program to ensure their environments are working properly. We deployed PKI Spotlight a few months ago and the customer team was seeing improved visibility and reliability with the tool. That is the goal after all!

Earlier this week they reached out as they were seeing a large number of event entries for their Entrust nCipher HSMs. The Windows Application log was reporting error messages repeatedly of:

ERROR: NCCNG_NCryptGetKeyProperty: unsupported flags 0X40000000 (supported 0x00000040). 

These were from the nCipher CSP source, Event ID 3s as you can see in the log snippet below.

We were initially concerned as we had deployed our latest version of PKI Spotlight that included Is-Alive. This feature goes way beyond simple service state verification to ensure a CA is healthy and operating and involves verifying if the CA cryptographic keys are accessible and usable. So we were naturally worried we were the cause of the issue.

After escalating to the dev group and confirming we could reproduce the issue we began to triage the remediation. It was at this point I reached out to our friends at Entrust and asked them about the nature of this error. It turns out Entrust is aware of the problem and it has nothing to do with our code or even PKI Spotlight. In their Security World software, they have a bug that is causing this reporting any time components use provided libraries to access the CA key. They indicated the issue was addressed in 12.8 of their software and a hotfix was available for a few specific older versions, specifically: v11.72, 12.4, 12.71, or 12.60.11.

Armed with this information we needed to know the best course of action for our customer. To get this information in the past we had to sign onto each and every client that is using one of the HSMs to determine what version of the software was installed. But wait – we have PKI Spotlight! Ah-ha! We were able to ask the customer to pull up the HSM Security World status window and in a few seconds, we had the security world version for all of the monitored clients. Sure enough, none of them were on a version with the fix.

So while it would have been nice to be on a supported client version, we were at least able to extremely quickly gather the information we needed, and make the determination that the client software needed to be updated to include the fix. Life-changing quick information!

For additional details, if you are experiencing this same error message, in addition to either installing 12.8 or applying the hotfix (provided by Entrust support), you will also need to set a registry entry on the clients to disable this superfluous error reporting:

The fix included with SW 12.80+ contains an updated caping component supports suppression
of the error messages generated to Windows event log when unsupported
property value is set to dwFlags property of NCryptSetProperty() or
NCryptGetProperty() API call.

(1) Please add the following registry key:

HKLM\Software\nCipher\CryptoNG\NoErrOnUnsupportedFlag

Then set it to "1"
Please be noted that 32 bit application would refer to another
key under 'Wow6432Node' and same key must be added there as well.

Or you can simply set this global environment variable as follows:

NCCNG_NOERR_ON_UNSUPPORTED_FLAG to 1

(2) restart nfast server service

With this registry key set, our CNG provider would stop generating
error event log when given dwFlags property contains some unsupported
value.
But other behavior is exactly same as before and application would get
'unsupported' return value from API call, also would generates warning
event log when 'unknown' value is set to dwFlags property.