Nitrokey HSM 2 - AES Encryption - Init/Update/Finalize supported?

Hello,

We have a python script using python pkcs11 library used to encrypt files with AES.

First, we had an issue (that was not here with SoftHSM2): We can’t encrypt more than 1024 bytes in one call:

02.09.2025 16:07:39 [126276004871040] Function sc_hsm_C_Encrypt called.
02.09.2025 16:07:39 [126276004871040] Function sc_hsm_C_Encrypt fails with rc=64 "Input too large"

I understand that embedded devices doesn’t have infinite RAM and it’s OK.

Our files to encrypt are huge, for example: ~60KBytes.

Then, I tried to use Init/Update/Finalize mecanism (with another python library: PyKCS11) : The idea here is to encrypt data in 1024-byte blocks.

But then when the function encryptUpdate() is called, I get this error:

.venv/lib/python3.10/site-packages/PyKCS11/__init__.py", line 1483, in encryptUpdate raise PyKCS11Error(rv) PyKCS11.PyKCS11Error:
 CKR_FUNCTION_NOT_SUPPORTED (0x00000054) 

Question:

  • Is it possible to encrypt “huge” files (60KBytes) with AES 128 using the Nitrokey HSM 2 + python ? (without Smart Card Shell)
    • How ?
    • Update/Finalize mechanism is not supported ?
  • Is it possible to do it with Smart Card Shell ?
  • Is there another ‘secure’ way to do it ? (without Smart Card Shell..)
  • Is there a document somewhere with these limits ? (supported functions, max length supported for each mechanism, etc.)

Here some logs from the sc-scm engine when init/update/finalise functions are failing:

03.09.2025 12:05:32 [128935629114240] Function updateSlots called.
03.09.2025 12:05:32 [128935629114240] Function updatePCSCSlots called.
03.09.2025 12:05:32 [128935629114240] SCardListReaders: Command successful.
03.09.2025 12:05:32 [128935629114240] SCardListReaders: Command successful.
03.09.2025 12:05:32 [128935629114240] Found reader 'Nitrokey Nitrokey HSM (DENK04022760000         ) 00 00'
03.09.2025 12:05:32 [128935629114240] Function updatePCSCSlots completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function updateSlots completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getValidatedToken called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken called.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken called.
03.09.2025 12:05:32 [128935629114240] SCardStatus: Command successful.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_GetSlotList completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_GetTokenInfo called.
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getValidatedToken called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken called.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken called.
03.09.2025 12:05:32 [128935629114240] SCardStatus: Command successful.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_GetTokenInfo completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_OpenSession called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function updateSlots called.
03.09.2025 12:05:32 [128935629114240] Function updatePCSCSlots called.
03.09.2025 12:05:32 [128935629114240] SCardListReaders: Command successful.
03.09.2025 12:05:32 [128935629114240] SCardListReaders: Command successful.
03.09.2025 12:05:32 [128935629114240] Found reader 'Nitrokey Nitrokey HSM (DENK04022760000         ) 00 00'
03.09.2025 12:05:32 [128935629114240] Function updatePCSCSlots completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function updateSlots completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getValidatedToken called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken called.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken called.
03.09.2025 12:05:32 [128935629114240] SCardStatus: Command successful.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function C_OpenSession completes with rc=0.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getValidatedToken called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken called.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken called.
03.09.2025 12:05:32 [128935629114240] SCardStatus: Command successful.
03.09.2025 12:05:32 [128935629114240] Function checkForRemovedPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getPCSCToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function sc_hsm_login called.
03.09.2025 12:05:32 [128935629114240] Verify PIN using provided PIN value
03.09.2025 12:05:32 [128935629114240] C-APDU: 00 20 00 81 Lc=06(6) ***Sensitive*** 
03.09.2025 12:05:32 [128935629114240] Function encodeCommandAPDU called.
03.09.2025 12:05:32 [128935629114240] Function encodeCommandAPDU completes with rc=11.
03.09.2025 12:05:32 [128935629114240] Function transmitAPDUviaPCSC called.
03.09.2025 12:05:32 [128935629114240] SCardTransmit: Command successful.
03.09.2025 12:05:32 [128935629114240] Function transmitAPDUviaPCSC completes with rc=2.
03.09.2025 12:05:32 [128935629114240] R-APDU: rc=0 SW1/SW2=9000
03.09.2025 12:05:32 [128935629114240] Function sc_hsm_login completes with rc=0.
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function C_Login completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_FindObjectsInit called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Search Filter:
03.09.2025 12:05:32 [128935629114240] CKA_LABEL = 636D6E5F6B65795F69647830 "cmn_key_idx0"
03.09.2025 12:05:32 [128935629114240] CKA_CLASS = 4 [0x4]
03.09.2025 12:05:32 [128935629114240] Function C_FindObjectsInit completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_FindObjects called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] objectsCollected=0
03.09.2025 12:05:32 [128935629114240] *pulObjectCount=1
03.09.2025 12:05:32 [128935629114240] Function C_FindObjects completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_FindObjects called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] No objects in left in search list
03.09.2025 12:05:32 [128935629114240] Function C_FindObjects completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_FindObjectsFinal called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function C_FindObjectsFinal completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptInit called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function sc_hsm_C_EncryptInit called.
03.09.2025 12:05:32 [128935629114240] Function sc_hsm_C_EncryptInit completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptInit completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptUpdate called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptUpdate fails with rc=84 "Operation not supported by token"
03.09.2025 12:05:32 [128935629114240] Function C_Finalize called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)

We can see:

03.09.2025 12:05:32 [128935629114240] Function C_EncryptInit completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptUpdate called.
03.09.2025 12:05:32 [128935629114240] LockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] UnlockMutex (0x606382929340)
03.09.2025 12:05:32 [128935629114240] Function findSlot called.
03.09.2025 12:05:32 [128935629114240] Function findSlot completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function getToken called.
03.09.2025 12:05:32 [128935629114240] Function getToken completes with rc=0.
03.09.2025 12:05:32 [128935629114240] Function C_EncryptUpdate fails with rc=84 "Operation not supported by token"
03.09.2025 12:05:32 [128935629114240] Function C_Finalize called.

This line: Function C_EncryptUpdate fails with rc=84 "Operation not supported by token"

Is there any possibility to make it working “with the token” ?

Is there something to configure when we create the keys in the HSM or the token ?

@sc-hsm :

Can you confirm or deny that C_EncryptUpdate() is not supported ?

Can you confirm or deny that the Nitrokey HSM 2 can’t encrypt more than 1024 bytes in AES-CBC ?

Thanks

The sc-hsm-pkcs11 module does indeed only support C_Encrypt/C_Decrypt with up to 1024 input. C_(D)EncryptUpdate and C_(D)EncryptFinal are not supported. The input is verbatim passed into the HSM for processing.

If you need to process larger inputs, you must call C_Encrypt multiple times, with the IV set to the last cipher block. This is the normal CBC chaining mode.

Processing larger inputs for AES encryption is rarely used, which is why not all variants are implemented.

Thanks, it works.

For people looking how to do it with python, here is a working example:

from asn1crypto.core import OctetString
from pkcs11 import lib, KeyType, ObjectClass, Attribute, Mechanism

import tool_crypto as tc

PKCS11_MODULE = None
PKCS11_TOKEN = None
PKCS11_PIN = None
ECC_CURVE_SIZES = {
    b'\x06\x08\x2a\x86\x48\xce\x3d\x03\x01\x07': 32,  # secp256r1
    b'\x06\x05\x2b\x81\x04\x00\x22': 48,  # secp384r1
    b'\x06\x05\x2b\x81\x04\x00\x23': 66,  # secp521r1
}

def check_config(pkcs11_module: [str, None], pkcs11_token: [str, None], pkcs11_pin: [str, None]):
    if pkcs11_module is None or pkcs11_token is None or pkcs11_pin is None:
        raise ValueError("pkcs11-module, pkcs11-token and pkcs11-pin are mandatory!")


def get_token():
    check_config(PKCS11_MODULE, PKCS11_TOKEN, PKCS11_PIN)
    return lib(PKCS11_MODULE).get_token(token_label=PKCS11_TOKEN)

def encrypt_method_2(ealgo, key_label, iv, plain):
    check_config(PKCS11_MODULE, PKCS11_TOKEN, PKCS11_PIN)

    try:
        with get_token().open(user_pin=PKCS11_PIN) as session:
            sym_keys = list(session.get_objects({
                Attribute.CLASS: ObjectClass.SECRET_KEY,
                Attribute.LABEL: key_label,
                Attribute.KEY_TYPE: KeyType.AES,
            }))

            if len(sym_keys) != 1:
                raise ValueError(f"{len(sym_keys)} keys found for '{key_label} label!' (must be only one!)")

            key = sym_keys[0]

            if ealgo == "AES-CBC":
                mech = Mechanism.AES_CBC
                block_size = 16
                chunk_size = 1024
                cipher = b""
                current_iv = iv

                for i in range(0, len(plain), chunk_size):
                    chunk = plain[i:i + chunk_size]

                    if len(chunk) % block_size != 0:
                        raise ValueError("Plaintext must be padded to a multiple of 16 bytes before encryption")

                    enc = key.encrypt(chunk, mechanism=mech, mechanism_param=current_iv)
                    cipher += enc
                    current_iv = enc[-block_size:]

            else:
                raise ValueError(f"Unsupported encryption algorithm : {ealgo}")

    except Exception as e:
        print(f"[ERROR] encrypt() : {e}")
        cipher = None

    return cipher

def encrypt_method_1(ealgo, key_label, iv, plain):
    check_config(PKCS11_MODULE, PKCS11_TOKEN, PKCS11_PIN)

    try:
        with get_token().open(user_pin=PKCS11_PIN) as session:
            sym_keys = list(session.get_objects({
                Attribute.CLASS: ObjectClass.SECRET_KEY,
                Attribute.LABEL: key_label,
                Attribute.KEY_TYPE: KeyType.AES,
            }))

            if len(sym_keys) != 1:
                raise ValueError(f"{len(sym_keys)} keys found for '{key_label} label!' (must be only one!)")

            key = sym_keys[0]

            if ealgo == "AES-CBC":
                mech = Mechanism.AES_CBC
                block_size = 16
                cipher = b""
                current_iv = iv

                for i in range(0, len(plain), block_size):
                    block = plain[i:i + block_size]
                    enc = key.encrypt(block, mechanism=mech, mechanism_param=current_iv)
                    cipher += enc
                    current_iv = enc[-block_size:]

            else:
                raise ValueError(f"Unsupported encryption algorithm : {ealgo}")

    except Exception as e:
        print(f"[ERROR] encrypt() : {e}")
        cipher = None

    return cipher

def encrypt(ealgo, key_label, iv, plain):
    return encrypt_method_1(ealgo, key_label, iv, plain)

...

WARNING:

  • I tried different methods, but this is horribly long to sign huge files (in my case : 55.8KBytes + 28.8KBytes + 656.6KBytes + 496.4KBytes + 92.4KBytes)
  • Each 1024 bytes blocks takes 4-5 seconds to encrypt :scream:

EDIT (just to keep a trace):

  • If I use encrypt_method_2 for AES-CBC encryption, it can encrypt 24 packet of 16 bytes per seconds (according to pcscd logs): 24*16 = 384Bytes/s
  • If I use encrypt_method_1for AES-CBC encryption, it can encrypt 1 packet of 1024 bytes in 4-5seconds. 1024/4=256Bytes/s
  • Conclusion:
    • I don’t know if the Nitrokey HSM 2 is known to be so slow OR if there is something in the setup that slow down the process.
    • The encryption finish faster if we use small packets instead of big packets.
1 Like

Interesting, that you get such a bad throughput.

With a Mini-USB token and the AES Performance Test I get

CBC (1216 bytes) with 100 iterations
Elapsed 24838 ms / 248.38 ms

The test can be found in the Starterkit.

What does it report for your token ?

Hello,

Here is the result:

The results are similar to mine, but slightly worse, but this is certainly due to the size of the encrypted packet. Because as mentioned in my previous message, the larger the packet to be encrypted, the less efficient it is (that’s what I see).

Also If I understand your are not testing with “Nitrokey HSM 2”.