Avoid single point of failure with PKIAAS

[mjourdan]

My CA is stored on a Nitrokey HSM2 which requires m-of-n authentication, using other HSM2.

In order to protect from device failure or loss, I would like to:

• be able to use the CA with a secondary CA HSM, identical to the first one
• give my colleagues the ability to use PKIAAS with their own personnal HSM

For the first goal, I used SmartCard Shell to import both DKEK and private key on my secondary HSM. I wonder if there are any more necessary step required in PKIAAS though.

For the second goal, I did the procedure below.

Steps to reproduce

I registered my personal HSM to my self-hosted PKIAAS instance. In order to give my colleagues the ability to use PKIAAS, I did the following:

• went to “Home”
• clicked on "Create TrustCenter Subject (Trust Center)
• plug a personnal token to authorize
• click on “Add Token”

Then I log out of PKIAAS.

Expected results

My colleagues are able to connect to PKIAAS with their own HSM.

Actual results

Colleagues are asked for their PIN, and after successful authentication they got the following error:

How could I reach both of my goals?

[sc-hsm]

The “Add Token” adds a HSM to the Trust Center, e.g. to store trust center keys. This has nothing to do with the personal token used to authenticate in the portal.

Personal token for authentication must be either added in self-registration or by enrolling a person in the trust center (“CA / Enroll Person”). Once you enroll additional people under the same trust center, you can assign them the required role, so that they can see the trust center and act accordingly.

Please remember, that the first person (in this case you) has ultimate rights initially. Any further enrolled person will have no specific rights. You will first need to open the subject view and give them the required roles with “Manage Role”.

The error message is a bit misleading (and already corrected). It should report, that the token is already assigned to a non-person subject, i.e. your trust center.

You can fix this by removing the token from the trust center in the service request where you added the token.

[mjourdan]

Thank you very much!

[mjourdan]

So, I did add an authentication token to the trustcenter, and associated with the appropriate CA.

Then, I went to the new subject, clicked “Subject / Manage roles”. I selected all roles and clicked “save” then “apply”.

After logout, I tried to loggin with this new key, and still have the same error message. Did I miss something?

[sc-hsm]

What do mean by “So, I did add an authentication token to the trust center, and associated with the appropriate CA.” ?

If you mean by “authentication token” the token that contains the authentication key for Public Key Authentication towards the trust center HSM, then that does not play a role in the PKIaaS. You can just plug it in when authentication is about to happen.

If you want others to access the portal, then they need to have an account that is linked with a token, used to login.

Typically, the individual token you give to a key custodian also contains the authentication key for PKA. So you can use the same token to login to the portal and within the portal authenticate the trust center HSM.

The only affiliation between the trust center HSM and the token used for authentication in PKA is when you prepare the trust center token and import the authentication public key. No need to configure that in the portal.

To sum up:

1. You give a token to key custodians
2. Key custodians generate an authentication key pair for PKA
3. You initialize the trust center HSM and import the public keys from 2.

Now you have setup PKA in the HSM.

Next you add the trust center HSM to the trust center subject. Now you can use the tokens from 1 to unlock the trust center HSM in the portal.

If you also want to use the same token from 1 to log into the portal, you need to either self-register the token (like the first user) or enroll the person in the trust center.

The portal maintains a link, which token is associated with a subject. A subject is anything that can have a certificate (Person, System, TrustCenter etc). Some tokens are associated with Persons, so they can then log into the portal. And other tokens are associated with systems, then they can be used to store keys required by systems.

[mjourdan]

Loggin into the portal

What do mean by “So, I did add an authentication token to the trust center, and associated with the appropriate CA.” ?

I mean I expect any of the other key custodians to be able to connect to PKIAAS with their own HSM, so that they can use the CA without me, as long as the quorum is fulfilled.

To sum up:

You give a token to key custodians
Key custodians generate an authentication key pair for PKA
You initialize the trust center HSM and import the public keys from 2.

Now you have setup PKA in the HSM.

Yes, I did all that in SmartCard Shell.

Next you add the trust center HSM to the trust center subject. Now you can use the tokens from 1 to unlock the trust center HSM in the portal.

If you also want to use the same token from 1 to log into the portal, you need to either self-register the token (like the first user) or enroll the person in the trust center.

I made additional attempts. After removing the individual token from the TrustCenter, loggin out, and unplugging every token except the individual token, I was asked for self-registration of this token, which is weird to me because I did previously enrolled it. Anyway, I could log in with a second personal token so that’s fine.

Signing a certificate with a secondary HSM

Now that I am connected with my newly added personnal token, I go to “Subjects / TrustCenter”. There I see my two CA tokens. I plug in the second one, and authenticate with the quorum of personal HSM. The status is the following:

2 authenticated public key(s) in 2 of 3 scheme

I see I also have the following roles :

• TrustCenter Operator
• TrustCenter Manager
• Registration Officer
• Certification Officer

I imported a csr, performed the different steps to get to the point of having a CSR approved, but not being able to actually sign it:

If I recall that’s the page where the buttons to sign the certificate should be. Why could they be missing, as the CA HSM is correctly unlocked and I have the required roles (I think)?

[sc-hsm]

It should work, if user 3 has the role Certification Officer and the HSM is unlocked.

Did user 3 do a fresh login after granting the role ?

I will check here if there is a bug.

[sc-hsm]

Just checked it, works like designed.

Maybe the HSM lost the authentication state ?

We are often seeing this on Windows. When the HSM token is connected to the portal and you insert an additional token, then Windows Plug&Play mechanism touches the session between the HSM token and the portal. This only happens on Windows and can be fixed by disabling Smart Card Plug & Play.

The secure link between the token and the portal is using secure messaging and the token will reset the authentication state, if a plain APDU is send outside secure messaging. This is the case, when Windows Plug&Play sends some APDUs for probing. It can also happen with ill-behaving crypto middleware that grabs freshly inserted smart cards and wildly probes around with APDUs.

Another workaround is to connect the token to the portal using a different machine. Internally we use a Raspberry PI to permanently connect HSM tokens to the portal. We then use the RAMOverHTTP client to connect.

[mjourdan]

Did user 3 do a fresh login after granting the role ?

Yes, this happens after a fresh login.

I did more tests, attaching the CA HSM as “HSM 0” and the personnal HSMs as “HSM 1” in ocf, so that the CA device would not be affected by unplugging personnal HSMs. Didn’t work.

I thought maybe it could be an issue with the CA. When I do the procedure with the primary HSM hosting the CA, the button to sign certificates is shown. When I use the backup HSM, the button is hidden.

So I tried to export certificates from both devices to check if they match. They do:

$ diff cert_pp_ca_racine_1.pem  cert_pp_ca_racine_hsm_2.pem
$

However, I can’t read any of the exported files with openssl:

$ openssl x509 -text -noout -in cert_pp_ca_racine_1.pem
Could not read certificate from cert_pp_ca_racine_1.pem
803BAC8C6D7F0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:../crypto/store/store_result.c:151:
Unable to load certificate 

Is it possible the impossibility to sign has to do with the content imported on the secondary CA HSM?

[sc-hsm]

The certificates on the HSM and in the portal are DER encoded. You need to use “-inform der” to read those in OpenSSL.

Just to understand what you are trying to do:

1. You have two HSMs for the CA, both initialized with the same DKEK Key Domain, so that key material can be share between the devices.
2. Both HSMs are added as trust center HSM and are visibile in the trust center view.
3. You have two personal HSM devices for login into the portal and to do public key authentication
4. You generated the X509 signer and issued a root certificate. The private key was initially on on of the HSM and was then copied to the other device.

So you should see both HSMs in the trust center view and you should be able to see the authentication state of each HSM. Depending on which HSM is currently connected, you should be able to perform public key authentication using the personal HSMs.

When you switch to the service request for signing and the sign button is not visible, what is the authentication state shown in the trust center view ?

[mjourdan]

The certificates on the HSM and in the portal are DER encoded. You need to use “-inform der” to read those in OpenSSL

It looks like the output is not correct then. I clicked “Export Certificate” in Smart Card Shell (as per the screenshot below) and got a file with a .pem extension, containing the following:

-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----

If I try to read it as .der anyway, I get the following:

$ openssl x509 -text -noout -in cert_pp_ca_racine_2.pem -inform der
Could not read certificate from cert_pp_ca_racine_2.pem
Unable to load certificate

Just to understand what you are trying to do:

Correct. In between steps 1 and 2, I exported the .wky from the first HSM, and imported it to the second one. I actually have 3 personal HSM for pubkey authentication though.

When you switch to the service request for signing and the sign button is not visible, what is the authentication state shown in the trust center view ?

DENK0301362 is the HSM I imported the key material to, DENK0301364 is the original one (which I can use to sign certificates).

[sc-hsm]

Sorry, got it wrong: The certificate you created is in the portal only and can be downloaded in DER format. The Smart Card Shell supports PEM Format, but the certificate you are seeing is not the X.509 certificate from the portal, but the CVC certificate for public key attestation. That format is not known by OpenSSL.

Let me create a similar test setup to see if there is a bug in the code locating the key domain.

[mjourdan]

Sorry, got it wrong: The certificate you created is in the portal only and can be downloaded in DER format. The Smart Card Shell supports PEM Format, but the certificate you are seeing is not the X.509 certificate from the portal, but the CVC certificate for public key attestation.

Oh indeed. I didn’t get the portal never actually stores the certificate on the device. Trying to compare both HSMs, the portal does seem to store the pubkey on the first device, but not on the second one, even if both do share the same key domain:

First one:

Second one:

[sc-hsm]

I guess the pp_ca_racine is a key you created and copied onto the second token ?

When you create a CA in the portal, then the private key is created with the Subject Public Key Identifier as key id. The key label always starts with “:X509 Signer”.

I see three keys with X509 Signer on the first token, but none on the second. You need to copy the X509 Signer keys.

The portal in its database stores for each Signer the Key Domain UID and the Key Id. When you copy keys from one HSM to the other, they have the same Key Domain UID and the Key Id is transferred as part of the backup.

[mjourdan]

I guess the pp_ca_racine is a key you created and copied onto the second token ?

Yes, I created it with SmartCard Shell with the assumption it would be the key that PKIAAS would deliver a certificate for.

I see three keys with X509 Signer on the first token, but none on the second. You need to copy the X509 Signer keys.

How could I do that? In SmartCard Shell, I tried both right and left clicks on every single item of the tree. More specifically, I tried the lines starting with “DKED with KCV”, “4:X509Signer 01”, and “Key Identifier”, and I could not find a way to export anything.

[sc-hsm]

You’re saying, that the context menu for pp_ca_racine offers “Export Key and Certificate”, but for the X509 Signer is doesn’t ?

I assume that you were logged in.

[mjourdan]

Well, I wasn’t :x

After login I could export and import the keys to the second device:

However, in PKIAAS, after I authenticate with the second CA HSM, the buttons to sign the certificate are still missing.