Expected Performance for AES-256-CBC with Nitrokey HSM 2

7ritn · January 14, 2023, 1:21pm

Hi,
I currently have a Smartcard HSM, which even though not in the specifications, can perform AES 256 CBC ciphers with sc-hsm-embedded. The Smartcard HSM can encrypt 32 Byte of data within 15 ms. I was wondering if the Nitrokey HSM 2 would be a significant upgrade to this performance. What kind of performance can I expect? Unfortunatly only assymetric cipher performance is listed in the specs for the Nitrokey HSM 2.

Thanks in advance.

saper · January 16, 2023, 1:05pm

Which hardware/firmware revision of Smartcard HSM do you have? Nitrokey HSM 2 is Smartcard HSM 3.5 pretty much the same.

7ritn · January 16, 2023, 1:30pm

I don’t really know. When I list information about the token using PKCS#11 I get the following:

  token label        : SmartCard-HSM (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : login required, rng, token initialized, PIN initialized
  hardware version   : 24.13
  firmware version   : 4.0

saper · January 16, 2023, 2:01pm

Looks like you have a newer Smartcard-HSM (revision 4.0) and Nitrokey HSM 2 is an older one (3.5-something). If you share your benchmark one of us could run it on Nitrokey HSM 2 to find out.

7ritn · January 16, 2023, 2:19pm

Well I don’t really have a benchmarking tool, just cloned https://github.com/CardContact/sc-hsm-embedded and modified the test source file (src/tests/sc-hsm-pkcs11-test.c) slightly so I could measure the time for the encrypt and derive key operation. Here my patch:

--- sc-hsm-pkcs11-test.c_OLD	2023-01-16 15:12:46.788364355 +0100
+++ sc-hsm-pkcs11-test.c_NEW	2023-01-16 15:12:43.295038445 +0100
@@ -37,6 +37,7 @@
 #include <string.h>
 #include <ctype.h>
 #include <time.h>
+#include <sys/time.h>
 
 #include <sc-hsm/sc-hsm-pkcs11.h>
 
@@ -1050,8 +1051,8 @@
 	char *tbs = "Hello World.....";
 
 	CK_BYTE signature[512];
-	CK_BYTE plain[1216];
-	CK_BYTE ciphertext[1216];
+	CK_BYTE plain[32];
+	CK_BYTE ciphertext[32];
 	CK_BYTE iv[16];
 	CK_ULONG len;
 	char scr[1024];
@@ -1102,8 +1103,17 @@
 
 		printf("Calling C_Encrypt()");
 
+		struct timeval start, end;
+		gettimeofday(&start, NULL);
 		rc = p11->C_Encrypt(session, plain, sizeof(plain), ciphertext, &len);
+		// stop timer.
+		gettimeofday(&end, NULL);
+		// Calculating total time taken by the program.
+
 		printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+		printf("Encrypt took : %ld micro seconds\n",
+			((end.tv_sec * 1000000 + end.tv_usec) -
+			(start.tv_sec * 1000000 + start.tv_usec)));
 
 		bin2str(scr, sizeof(scr), ciphertext, len);
 		printf("Ciphertext:\n%s\n", scr);
@@ -1750,8 +1760,17 @@
 	rc = p11->C_GenerateKey(session, &mech_genaes, secretKeyTemplate, secretKeyAttributes, &hndBaseKey);
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
 
+	struct timeval start, end;
+	gettimeofday(&start, NULL);
 	rc = p11->C_DeriveKey(session, &mech_derive, hndBaseKey, deriveTemplate, derivedAttributes, &hndDerivedKey);
+	// stop timer.
+	gettimeofday(&end, NULL);
+	// Calculating total time taken by the program.
+
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+	printf("Derivation took : %ld micro seconds\n",
+		((end.tv_sec * 1000000 + end.tv_usec) -
+		(start.tv_sec * 1000000 + start.tv_usec)));
 
 	if (rc == CKR_OK) {
 		printf("Derived Secret Key:\n");

and then run with src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin <user_pin> --invasive from project root

7ritn · January 16, 2023, 3:39pm

Somehow my previous reply was hidden by spambot (probably because of the link to the github repo). I don’t have a specific benchmarking tool. Just cloned the CardContact/sc-hsm-embedded project and made some small modifications to the test source code (src/tests/sc-hsm-pkcs11-test.c)

--- sc-hsm-pkcs11-test.c_OLD	2023-01-16 15:12:46.788364355 +0100
+++ sc-hsm-pkcs11-test.c_NEW	2023-01-16 15:12:43.295038445 +0100
@@ -37,6 +37,7 @@
 #include <string.h>
 #include <ctype.h>
 #include <time.h>
+#include <sys/time.h>
 
 #include <sc-hsm/sc-hsm-pkcs11.h>
 
@@ -1050,8 +1051,8 @@
 	char *tbs = "Hello World.....";
 
 	CK_BYTE signature[512];
-	CK_BYTE plain[1216];
-	CK_BYTE ciphertext[1216];
+	CK_BYTE plain[32];
+	CK_BYTE ciphertext[32];
 	CK_BYTE iv[16];
 	CK_ULONG len;
 	char scr[1024];
@@ -1102,8 +1103,17 @@
 
 		printf("Calling C_Encrypt()");
 
+		struct timeval start, end;
+		gettimeofday(&start, NULL);
 		rc = p11->C_Encrypt(session, plain, sizeof(plain), ciphertext, &len);
+		// stop timer.
+		gettimeofday(&end, NULL);
+		// Calculating total time taken by the program.
+
 		printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+		printf("Encrypt took : %ld micro seconds\n",
+			((end.tv_sec * 1000000 + end.tv_usec) -
+			(start.tv_sec * 1000000 + start.tv_usec)));
 
 		bin2str(scr, sizeof(scr), ciphertext, len);
 		printf("Ciphertext:\n%s\n", scr);
@@ -1750,8 +1760,17 @@
 	rc = p11->C_GenerateKey(session, &mech_genaes, secretKeyTemplate, secretKeyAttributes, &hndBaseKey);
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
 
+	struct timeval start, end;
+	gettimeofday(&start, NULL);
 	rc = p11->C_DeriveKey(session, &mech_derive, hndBaseKey, deriveTemplate, derivedAttributes, &hndDerivedKey);
+	// stop timer.
+	gettimeofday(&end, NULL);
+	// Calculating total time taken by the program.
+
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+	printf("Derivation took : %ld micro seconds\n",
+		((end.tv_sec * 1000000 + end.tv_usec) -
+		(start.tv_sec * 1000000 + start.tv_usec)));
 
 	if (rc == CKR_OK) {
 		printf("Derived Secret Key:\n");

After compiling I just run the test with src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin <user_pin> --invasive. Beware that to run the encrypt and key generation tests --invasive is required.

saper · January 28, 2023, 11:26am

Nitrokey HSM 2, firmware version 3.4:

Derivation took : 105025 micro seconds

Can’t get encryption result yet, because one test is broken as of Fix report of AES key sizes (Fixes #36) · CardContact/sc-hsm-embedded@4f26f20 · GitHub

saper · January 28, 2023, 12:13pm

Looks like one needs at least AES key on the token to run those tests.

Now I got this:

Derivation took : 105620 micro seconds
Encrypt took : 85725 micro seconds

github.com/CardContact/sc-hsm-embedded

sc-hsm-pkcs11-test: "Find a private key after login" test failure on an empty token

opened 12:00PM - 28 Jan 23 UTC

saper

As of https://github.com/CardContact/sc-hsm-embedded/commit/4f26f202b28d5fe1c91a…7a5b84061dd5b385b440 I get the following running the tests on an freshly initialized Nitrokey HSM 2: ``` Script started on Sat Jan 28 11:55:01 2023 Command: ./src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin 000000 --invasive PKCS11 unit test running. ``` and then: ``` Find a private key after loginCalling C_FindObjectsInit - CKR_OK : Passed Calling C_FindObjects - CKR_OK : Passed Calling C_FindObjectsFinal - CKR_OK : Passed - CKR_ARGUMENTS_BAD : Failed ``` Quick analysis: At the time the `testLogin()` https://github.com/CardContact/sc-hsm-embedded/blob/4f26f202b28d5fe1c91a7a5b84061dd5b385b440/src/tests/sc-hsm-pkcs11-test.c#L2833 is run, there is no private key on the token at all. If I run the following sequence instead: ``` radziecki> sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 000000 Using reader with a card: Nitrokey Nitrokey HSM (DENK01045710000 ) 00 00 radziecki> /usr/local/bin/pkcs11-tool -l -p 000000 --module ./src/pkcs11/.libs/libsc-hsm-pkcs11.so --keypairgen --key-type rsa:1024 --id 10 Using slot 0 with a present token (0x1) Key pair generated: Private Key Object; RSA label: ID: 10 Usage: decrypt, sign Access: sensitive, always sensitive, never extractable, local Public Key Object; RSA 1024 bits label: ID: 10 Usage: encrypt, verify Access: local radziecki> ./src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin 000000 --invasive PKCS11 unit test running. Calling C_GetFunctionList Calling C_Initialize - CKR_OK : Passed Calling C_GetInfo - CKR_OK : Passed Calling C_GetSlotList - CKR_OK : Passed Calling C_GetSlotInfo for slot 1 - CKR_OK : Passed Slot manufacturer: CardContact Slot ID : Slot description: 1 : Nitrokey Nitrokey HSM (DENK01045710000 ) 00 00 Slot flags: 7 Calling C_GetTokenInfo - CKR_OK : Passed ... ``` then I finally get ``` Unit test finished. 455 tests performed. 0 tests failed. ```

7ritn · February 2, 2023, 9:23am

Thanks for your effort. It’s astonishing how much worse the Nitrokey HSM 2 performs compared to the SmartCard HSM. 85 ms for 2 blocks of data. That means the Nitrokey takes 70 ms longer to perform the encryption than the SmartCard HSM.

saper · February 2, 2023, 8:48pm

You are using the newest SmartCard-HSM 4.0. This uses what I think is a different, new chip using flash memory, not a slow EEPROM. Probably with the SmartCard-HSM 3.x the results were the same, since this is what is inside of Nitrokey HSM 2.

saper · February 2, 2023, 9:37pm

Here are some interesting benchmarks, JCAlgTest - Comparative table I think both our smartcards belong to the category similar to one of the listed, but I don’t know exactly which.

7ritn · February 9, 2023, 10:32am

This makes me wonder why anyone should buy the Nitrokey HSM 2, when the SmartCard HSM 4K USB-Token costs slightly less and has better performance.

saper · February 10, 2023, 1:13pm

Smart-Card HSM 4K 4.0 is a new thing. Getting SmartCardHSM 4.0? - #2 by daringer Probably no new “Nitrokey HSM” under the Nitrokey brand.

On the other hand, there are some initial problems:

github.com/OpenSC/OpenSC

SmartCard-HSM V4.0 occasionally fails to generate secp521r1 signature

opened 08:45AM - 25 Oct 21 UTC

closed 07:15AM - 27 Oct 21 UTC

CardContact

### Problem Description Signing with an EC key on secp521r1 fails occasionall…y with CKR_DATA_INVALID. This issue only occurs with V4.0 devices. ### Proposed Resolution Fix sc_hsm_decode_ecdsa_signature() to correctly separate 512 from 521 bit signatures. ### Steps to reproduce Generate secp521r1 key pkcs11-tool --login --pin 648219 --keypairgen --key-type EC:nistp521 --id a1 --label a1 --usage-sign Repeatably run echo "111111111111111111111111111111111111111111111111111111112345678" | pkcs11-tool --login --pin 648219 -s -o tmp until the error shows up.