Nitrokey Pro with OpenSSL 1.1.1, TLS 1.3, and RSA-based certificates

I’m trying to set up OpenVPN with a Nitrokey Pro for client-side authentication as per the subject line. Notwithstanding the endless nettlesome quirks and bugs that needed to be overcome, I’ve arrived at the conclusion that it’s not possible because:

  1. TLS 1.3 mandates PSS padding when using RSA-based certificates. Since the Nitrokey Pro only supports PKCS padding, it therefore cannot be used directly (or at all - see #2 for more details).
  2. The OpenSSL function called by OpenVPN will perform software PSS padding and pass the result to the pkcs-helper library with request for a raw (aka “unpadded”) signature (when using the latest git version of OpenVPN and pkcs-helper - older versions will croak at OpenSSL’s request for an unpadded signature). Presumably, the choice of software padding was made for compatibility reasons, since cards are more likely to support unpadded over PSS signatures. Since the Nitrokey Pro does not support RSA-X-509 (i.e. raw/unpadded), this operation returns with “CKR_MECHANISM_NOT_SUPPORTED,” as expected.

As the token supports neither raw nor PSS RSA signatures, it seems pretty obvious to me that the subject-line setup is fundamentally impossible. Even if we fall back to TLS 1.2, we would simultaneously need to revert to an earlier version of OpenSSL to avoid the raw padding scenario. Neither of these actions are especially appealing from a security perspective. Are there any plans to implement RSA-X-509? Or RSA-PSS? I know this departs from the OpenPGP standard, but it would make the card more functional without any real sacrifice in security.

1 Like

Hi @dominic.schaub!

Incidentally at OpenSC there is already a discussion about it, with a possible solution. Please take a look at the link below, and let me know whether it works for you: