“Good day.
Is it possible make request like this “”curl –cert test.crt –key test.key”” on .net framework 4.7?”
This blog post is about programming and its purpose is to have a link to direct developers for explanation. Inspired from this list:
Disclaimer: TL;DR.
Microsoft has a long history of their proprietary cryptography subsystem called CryptoAPI. It was first invented in Windows NT 4.0 in the way it is widely used these days. *nix-based platforms use standard key storage format defined in a number of RFCs, often in an unencrypted format, PKCS#8 or PKCS#1 (which is a subset of PKCS#8). Operations with keys (encryption, signing, etc.) were implemented in separate libraries such as OpenSSL. Any library that implements PKCS#1/8 could work with such keys, thus providing a way to use custom cryptographic libraries on a single platform.
Microsoft invented and implemented their own philosophy around their cryptography subsystem with Cryptographic Service Provider (CSP) concept and proprietary key format. CSP is simply a box with named encrypted keys inside. Each CSP is responsible for key stored inside and provides an abstraction layer between client (key consumer) and certificate keys. CSP stores keys in an encrypted form, thus access to private key raw file doesn’t give you anything useful. This is how Microsoft provides a kind of key security. Instead of raw access to key material (that prevents from key leak in some degree), you use standard CryptoAPI calls and ask particular CSP to use named key to perform cryptographic operation (encryption, signing, whatever else). In some cases, you can export key material in standard format, such as PKCS#12, sometimes not. This behavior is governed by key export policy and this is another story. In theory, this concept was intended to protect keys from leaks, make developer’s life easier by abstracting cryptographic operations via a set of CryptoAPI functions (most of them are defined in crypt32.dll library) to operate with keys.
After years, with the release of Windows Vista and Windows Server 2008, Microsoft rebuilt entire cryptography stack from scratch and improved it all around, by:
This new stack was named a Cryptography Next Generation or CNG, or CAPI2 and backed by ncrypt.dll library. Providers within this stack got new name to distinguish from legacy CSPs – Key Storage Provider or KSP. CSP –> legacy crypto, KSP –> modern crypto. Plain and simple.
Built-in Windows components and services got native support for CNG: ADCS, ADDS, EFS, IIS, RDS, Internet Explorer, etc. Almost all what was shipped with Windows OS and what wasn’t based on .NET was compatible with CNG in 2006. A limited number of external products got support for CNG. Most popular was Microsoft Office 2007 which natively supports CNG when installed on Windows Vista and newer OSes. But most external products that were either, .NET-based or had .NET interface were not compatible with CNG, because .NET didn’t support CNG at that time. And its support came many and many years after CNG become native in Windows. M(B)illions of developers and IT administrators crushed their heads while battling with keys in attempt to get the right one for their application.
It was .NET 4.6 when .NET-based life become different. Cryptography stack in .NET can be divided to two eras: before 4.6 and after.
Before .NET Framework version 4.6, cryptography support in .NET was Windows-only and sticks to legacy CryptoAPI library calls. Easiest (and, possibly, the only) way to access the certificate’s private key was:
public class Class1 { public Class1() { var cert = new X509Certificate2(...); var privateKey = (RSACryptoServiceProvider)cert.PrivateKey; privateKey.Decrypt(...); // or privateKey.SignData(...); } }
An X509Certificate2 class has a PrivateKey property of AsymmetricAlgorithm type. AsymmetricAlgorithm class is abstract class for any asymmetric algorithm and defines only few relevant methods. Access to actual algorithm implementations is done via explicit algorithm groups: RSA, DSA, ECDsa, ECDiffieHellman. These classes had only one platform-specific implementation. In case of RSA it was RSACryptoServiceProvider. So, no doubt, previous example worked in 99.99%. In very rare cases you could get InvalidCastException: in 0.009% it was ECC key and in 0.001% it was DSA key.
Note: this example throws exception, when you access PrivateKey property and private key is stored in KSP. This means that KSP keys explicitly not supported in X509Certificate2 ecosystem before .NET 4.6.
.NET team was criticized for poor to no CNG support for a long period after CNG release. As the result, major .NET-based Microsoft products still (as of early 2020) don’t support CNG keys. Entire System Center product line, Exchange Server, ADFS and many other products still doesn’t support CNG storage, while CNG is about 14 years around us. Cool story. .NET team added a very basic CngKey class in v3.5 to access CNG storages and keys. Not so much and this class wasn’t integrated with X509Certificate2 class in any way.
Only with the release of v4.6, things become real and CNG support was greatly improved by adding CNG implementations for asymmetric key algorithms: RSACng, DSACng, ECDsaCng, ECDiffieHellmanCng. Using these classes, you can fully access CNG keys in your .NET applications. And here is a little puzzle: .NET has two implementations for RSA keys: legacy RSACryptoServiceProvider and new RSACng. You can’t know at runtime where the key is stored: in CSP or KSP and depending on key storage, a cast to different types is required. And this cast must be done at compile time, because RSA abstract class didn’t have methods to perform cryptographic operations. Kudos to .NET team since they found a quite elegant way to solve this puzzle: They added cryptographic methods to abstract base classes and now you can use them without knowing exact implementation: legacy CSP or modern KSP.
Compare them:
Instead or fixing X509Certificate.PrivateKey property, .NET team added extension methods to X509Certificate2 class
Since 4.6, this is the recommended way to access public and private keys. What you need to know in advance – which method to call: RSA, DSA, ECDsa? You can’t know this at compile time, but you can check it at runtime by checking the algorithm OID in public key: X509Certificate2.PublicKey has Oid that identifies the asymmetric key algorithm. Drop this property to switch statement and call appropriate extension methods to get the right key.
While first example still works in 4.6, direct access to X509Certificate2.PrivateKey and X509Certificate2.PublicKey.Key properties is now discouraged. You shall access keys only via mentioned extension methods. No discussions, no exceptions. Otherwise, you are a candidate for new thread on StackOverflow as I referenced in the beginning of this post. Don’t believe? Check next section!
These changes not only solved the support of CNG in X509Certificate2 class, but helped to move this class to .NET Core and other platforms with very different cryptography subsystems and add platform-specific implementations for asymmetric algorithms. So far, so good.
In 4.7 (and all versions of .NET Core on Windows), .NET team made things even worse, by changing the default type returned by X509Certificate2.PrivateKey from RSACryptoServiceProvider to RSACng. I already mentioned that this access is strictly discouraged starting with v4.6. Applications now crash at a runtime when you attempt to use example I posted in “Dark Ages” section. That example uses compile-time explicit cast to RSACryptoServiceProvider, but SURPRISE, the type is changed to RSACng in 4.7! Check and mate.
The whole point of this post was to explain why:
public class Class1 { public Class1() { var cert = new X509Certificate2(...); var privateKey = (RSACryptoServiceProvider)cert.PrivateKey; // use key } }
is bad! And:
public class Class1 { public Class1() { var cert = new X509Certificate2(...); RSA privateKey = cert.GetRSAPrivateKey(); // use the key } }
or
public class Class1 { public Class1() { const String RSA = "1.2.840.113549.1.1.1"; const String DSA = "1.2.840.10040.4.1"; const String ECC = "1.2.840.10045.2.1"; var cert = new X509Certificate2(...); switch (cert.PublicKey.Oid.Value) { case RSA: RSA rsa = cert.GetRSAPrivateKey(); // or cert.GetRSAPublicKey() when need public key // use the key break; case DSA: DSA dsa = cert.GetDSAPrivateKey(); // or cert.GetDSAPublicKey() when need public key // use the key break; case ECC: ECDsa ecc = cert.GetECDsaPrivateKey(); // or cert.GetECDsaPublicKey() when need public key // use the key break; } } }
is the right way to access the private key in .NET Framework 4.6+ and .NET Core (all versions). Happy programming with .NET and X509Certificate2!
“Good day.
Is it possible make request like this “”curl –cert test.crt –key test.key”” on .net framework 4.7?”
Sorry, I don’t know what that command does.
Any idea if GetRSAPrivateKey throws Invalid provider type specified with .Net framework 4.6.1 and of course the oid value is of RSA .
Do the certificate contain the private key? What “certutil -store my ” says?
Got he issue from another blog of yours. Certificate private key is deleted. Used crtutil to find the diff between the 2 RSACng keys. Thanks.
Good you were able to narrow the issue root.
Hi,
how to set the wrigth cardReader (eg. “Microsoft Virtual Smart Card 0”) if there are more than one card reader in system.
If i call “var privateKey = (RSACryptoServiceProvider)cert.PrivateKey;” than the first Card Reader in System is used (Private key of certificate was imported into “Microsoft Base Smart Card Crypto Provider” wit certutil -importPFX
during access to the private key a pin is required
I don’t think it is possible to specify reader name when acquiring private key handle.
What is the correct way to use CNG keys/containers to access the private key if it is only available on a smart card? I want to try to sign some hashed data using a smart card private key using CNG code with C# if possible. I already have the pin from user input to a textbox on my form, so I don’t want the pin dialog to appear. I have this working with some restrictions (one smartcard only – with two it only works on the first one found) using CSP code but want to change to the newer format.
“You need to use CngKey.SetProperty API to set PIN for your key.
1. Acquire private key handle for certificate (must be RSACng)
2. Acquire CngKey from RSACng
3. Call CngKey.SetProperty and specify “”SmartCardPin”” as property name.”
Thanks. I will try and get that working. I am trying: var pvtHandle = cert2.GetRSAPrivateKey(); Which gets me seemingly valid data. (I can see it in the debugger) But I am not sure yet how to extract the handle correctly. I appreciate the help.
Key handle is stored in pvtHandle.Key: https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.rsacng.key?view=net-5.0#System_Security_Cryptography_RSACng_Key
I get following error when calling CngKey.SetProperty:
Internal.Cryptography.CryptoThrowHelper.WindowsCryptographicException: ‘Der angeforderte Vorgang wird nicht unterstützt.’
Is this due to Device driver / Middleware? Error only with Acos card, working with Gemalto card.
I don’t know, insufficient details on what you are trying to do. I would recommend to ask the question with code on StackOverflow.com forums
I don’t know, insufficient details on what you are trying to do. I would recommend to ask the question with code on StackOverflow.com forums.
Is there any way we can export the private key obtained using the above methods to a file in base64 format ?
it depends on whether private key is exportable or not. But this question is outside the scope of the post. I would recommend to ask the question on professional forums such as Stack Overflow.
what is the best alternative if we want to allow few users to allow read permission to private keys. In the past we used the CspKeyContainerInfo that was available inside the RSACryptoServiceProvider
CNG supports ACLs through KSP properties: https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.cngkey.getproperty?view=net-5.0 and property is called “Security Descr” (see: https://docs.microsoft.com/en-us/windows/win32/seccng/key-storage-property-identifiers). And update security by calling https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.cngkey.setproperty?view=net-5.0.
Thanks for the informative article. It explains well how to export all the private keys of existing certificates and there encryption methods.
I am currently trying to use [System.Security.Cryptography.X509Certificates.X509Certificate2] and CNG to create a cert on Powershell 5.1 / .net 4.8 so it signs the private key in RSA. It always uses the old CAPI method. I was wondering if you knew a way of forcing it .net 4.8 to use CNG?
Problem is described in 2 links below:
https://stackoverflow.com/questions/69196724/difference-between-powershell-5-1-and-powershell-7-certificate-encyption-using-s?noredirect=1#comment122330957_69196724
I’ve been unable to confirm the following with .NET Framework 4.7 (and 4.7.1):
> In 4.7 (and all versions of .NET Core on Windows), .NET team made things even worse, by changing the default type returned by X509Certificate2.PrivateKey from RSACryptoServiceProvider to RSACng.
I’ve tried with both CSP and CNG keys/certificates and for CSP X509Certificate2.PrivateKey returns a RSACryptoServiceProvider while for CNG it throws the following exception:
> System.Security.Cryptography.CryptographicException
> HResult=0x80090014
> Message=Invalid provider type specified.
> …
> This exception was originally thrown at this call stack:
> System.Security.Cryptography.Utils.CreateProvHandle(System.Security.Cryptography.CspParameters, bool) in Utils.cs
> System.Security.Cryptography.Utils.GetKeyPairHelper(System.Security.Cryptography.CspAlgorithmType, System.Security.Cryptography.CspParameters, bool, int, ref System.Security.Cryptography.SafeProvHandle, ref System.Security.Cryptography.SafeKeyHandle) in Utils.cs
> System.Security.Cryptography.RSACryptoServiceProvider.GetKeyPair() in RSACryptoServiceProvider.cs
> System.Security.Cryptography.RSACryptoServiceProvider.RSACryptoServiceProvider(int, System.Security.Cryptography.CspParameters, bool) in RSACryptoServiceProvider.cs
> System.Security.Cryptography.X509Certificates.X509Certificate2.PrivateKey.get() in X509Certificate2.cs
> …
> Program.Main(string[]) in Program.cs
Can you please provide more details on how to obtain a RSACng from a call to X509Certificate2.PrivateKey?
As I said, you shall not access X509Certificate2.PrivateKey property anymore when on .NET 4.7 and newer. You should acquire private key only by calling appropriate extension method, for example, X509Certificate2.GetRSAPrivateKey() for RSA keys.
The problem here is that sometimes we don’t control the code, and cannot rewrite it. Still we expect it to work with KSP using the same methods we used for CSP. It would be nice if there was some flag or registry setting to control this behavior during the transition period.
> Still we expect it to work with KSP using the same methods we used for CSP
this is not true. CSP and KSP use very different CryptoAPI native functions. CSP uses “Crypt*” prefixed functions, KSP uses “NCrypt*” prefixed functions.
You say that since .NET Framework 4.6 this is the recommended way to access keys, but is it an official MS recommendation? Can you provide a reference?
> Instead or fixing X509Certificate.PrivateKey property, .NET team added extension methods to X509Certificate2 class:
> GetRSAPublicKey(X509Certificate2)
> GetRSAPrivateKey(X509Certificate2)
> A pair of methods for each asymmetric key
> algorithm
> Since 4.6, this is the recommended way to access public and private keys.
I don’t have any official references from Microsoft and I doubt they ever exist. This blog post is derived from my personal experience in dealing with certificates in .NET
Thanks for the nice article.
How to get the private key if the hardware device (dsc, smart cards) is marked as “Non-exportable”. Otherwise, get the error “Access denied”
It may be a permissions issue, not key exportability. You don’t have permissions to access particular private key.
Thanks for the informative article.
When I try:
ECDsa ecc = cert.GetECDsaPublicKey();
ecc.ToXmlString(includePrivateParameters: false);
I get error “XML serialization of an elliptic curve key requires using an overload which specifies the XML format to be used”.
Any idea how to solve this. Thank you in advance
You have to use this ECDsaCng: https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.ecdsacng.toxmlstring?view=net-6.0#system-security-cryptography-ecdsacng-toxmlstring(system-security-cryptography-eckeyxmlformat) and specify desired XML format.
When use the code below, windows throws me a dialog with the following message,”Windows security smart card select a smart card device” does anyone know why this happening and how to prevent it?
CspParameters parameters = new CspParameters(1, “Microsoft Base Smart Card Crypto Provider”);
AsymmetricAlgorithm asymmetricAlgorithm = new RSACryptoServiceProvider(parameters);
string pubKeyXml = asymmetricAlgorithm.ToXmlString(false);
X509Store x509Store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
x509Store.Open(OpenFlags.OpenExistingOnly);
You are using “Microsoft Base Smart Card Crypto Provider” provider and its name clearly says that it expects a smart card to store and access the key. Use different non-smart card provider.
Good Morning, I have a problem with this solution.
RSA cspp = p12.GetRSAPrivateKey();
int intKey = p12.PrivateKey.KeySize;
var dataBytes = Convert.FromBase64String(cypherText);
var plainText = cspp.Decrypt(dataBytes,RSAEncryptionPadding.OaepSHA512);
return Encoding.Unicode.GetString(plainText);
The error is: System.Security.Cryptography.CryptographicException: ‘The length of the data to decrypt is not valid for the size of this key.’
Help me!
I have the same problem.
:Exception in Signing For Document:xxxx1431454976 | Message: Provider DLL failed to initialize correctly. while signing the document using GetRSAPrivateKey
You may need to examine exception stack trace and determine whether your signing tool supports CNG keys.
Hi Vadims,
Nice article!
I work with a provider that assigned me the same RSA certificates on a yearly basis. So far so good.
Recently, they switched to ECC. I repeated the same steps: adding the certificate to the store, marking the private key as exportable, adding IIS_IUSRS to have full control on the private key and no matter what methods I try to access the private key, I get exceptions (like “Keyset does not exist”) or nulls. Their customer support is non-existent and the developers are kept behind a steel wall so I am completely stuck.
I just don’t know if the certificate or my code is trash. I use .NET Framework 4.6.1, shall I upgrade?
Cheers,
Peter
The code must be upgraded as well in order to support KSP and ECC. Unfortunately, I can’t tell anything based on the information you provided.
Hi Vadims, I had read some documents and noticed there is no private key in X.509 Certificate, so does it work when call .GetRSAPrivateKey() method when initialized X509Certificate2 instance from a .cert file?
no, this method will return null if there is no private key.
Hi Vadims!
When installing an application, I would like to ensure that the user it’s running as has access to the private key of a certificate installed in the local machine certificate store and specified by thumbnail.
This can be done using file system ACLs on the appropriate item in c:\programdata\microsoft\crypto\rsa\machinekeys. Previously, PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName provided the info needed to select the correct entry.
I’m currently at a loss on how to automate that and am relying on a prompt for the admin to set the required access manually using cert manager.
Any help would be greatly appreciated.
Thanks, Martin
You can use the following approach to retrieve CNG unique key container name: https://www.sysadmins.lv/blog-en/retrieve-cng-key-container-name-and-unique-name.aspx