Expected Performance for AES-256-CBC with Nitrokey HSM 2

Hi,
I currently have a Smartcard HSM, which even though not in the specifications, can perform AES 256 CBC ciphers with sc-hsm-embedded. The Smartcard HSM can encrypt 32 Byte of data within 15 ms. I was wondering if the Nitrokey HSM 2 would be a significant upgrade to this performance. What kind of performance can I expect? Unfortunatly only assymetric cipher performance is listed in the specs for the Nitrokey HSM 2.

Thanks in advance.

Which hardware/firmware revision of Smartcard HSM do you have? Nitrokey HSM 2 is Smartcard HSM 3.5 pretty much the same.

I don’t really know. When I list information about the token using PKCS#11 I get the following:

  token label        : SmartCard-HSM (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : login required, rng, token initialized, PIN initialized
  hardware version   : 24.13
  firmware version   : 4.0

Looks like you have a newer Smartcard-HSM (revision 4.0) and Nitrokey HSM 2 is an older one (3.5-something). If you share your benchmark one of us could run it on Nitrokey HSM 2 to find out.

Well I don’t really have a benchmarking tool, just cloned https://github.com/CardContact/sc-hsm-embedded and modified the test source file (src/tests/sc-hsm-pkcs11-test.c) slightly so I could measure the time for the encrypt and derive key operation. Here my patch:

--- sc-hsm-pkcs11-test.c_OLD	2023-01-16 15:12:46.788364355 +0100
+++ sc-hsm-pkcs11-test.c_NEW	2023-01-16 15:12:43.295038445 +0100
@@ -37,6 +37,7 @@
 #include <string.h>
 #include <ctype.h>
 #include <time.h>
+#include <sys/time.h>
 
 #include <sc-hsm/sc-hsm-pkcs11.h>
 
@@ -1050,8 +1051,8 @@
 	char *tbs = "Hello World.....";
 
 	CK_BYTE signature[512];
-	CK_BYTE plain[1216];
-	CK_BYTE ciphertext[1216];
+	CK_BYTE plain[32];
+	CK_BYTE ciphertext[32];
 	CK_BYTE iv[16];
 	CK_ULONG len;
 	char scr[1024];
@@ -1102,8 +1103,17 @@
 
 		printf("Calling C_Encrypt()");
 
+		struct timeval start, end;
+		gettimeofday(&start, NULL);
 		rc = p11->C_Encrypt(session, plain, sizeof(plain), ciphertext, &len);
+		// stop timer.
+		gettimeofday(&end, NULL);
+		// Calculating total time taken by the program.
+
 		printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+		printf("Encrypt took : %ld micro seconds\n",
+			((end.tv_sec * 1000000 + end.tv_usec) -
+			(start.tv_sec * 1000000 + start.tv_usec)));
 
 		bin2str(scr, sizeof(scr), ciphertext, len);
 		printf("Ciphertext:\n%s\n", scr);
@@ -1750,8 +1760,17 @@
 	rc = p11->C_GenerateKey(session, &mech_genaes, secretKeyTemplate, secretKeyAttributes, &hndBaseKey);
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
 
+	struct timeval start, end;
+	gettimeofday(&start, NULL);
 	rc = p11->C_DeriveKey(session, &mech_derive, hndBaseKey, deriveTemplate, derivedAttributes, &hndDerivedKey);
+	// stop timer.
+	gettimeofday(&end, NULL);
+	// Calculating total time taken by the program.
+
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+	printf("Derivation took : %ld micro seconds\n",
+		((end.tv_sec * 1000000 + end.tv_usec) -
+		(start.tv_sec * 1000000 + start.tv_usec)));
 
 	if (rc == CKR_OK) {
 		printf("Derived Secret Key:\n");

and then run with src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin <user_pin> --invasive from project root

Somehow my previous reply was hidden by spambot (probably because of the link to the github repo). I don’t have a specific benchmarking tool. Just cloned the CardContact/sc-hsm-embedded project and made some small modifications to the test source code (src/tests/sc-hsm-pkcs11-test.c)

--- sc-hsm-pkcs11-test.c_OLD	2023-01-16 15:12:46.788364355 +0100
+++ sc-hsm-pkcs11-test.c_NEW	2023-01-16 15:12:43.295038445 +0100
@@ -37,6 +37,7 @@
 #include <string.h>
 #include <ctype.h>
 #include <time.h>
+#include <sys/time.h>
 
 #include <sc-hsm/sc-hsm-pkcs11.h>
 
@@ -1050,8 +1051,8 @@
 	char *tbs = "Hello World.....";
 
 	CK_BYTE signature[512];
-	CK_BYTE plain[1216];
-	CK_BYTE ciphertext[1216];
+	CK_BYTE plain[32];
+	CK_BYTE ciphertext[32];
 	CK_BYTE iv[16];
 	CK_ULONG len;
 	char scr[1024];
@@ -1102,8 +1103,17 @@
 
 		printf("Calling C_Encrypt()");
 
+		struct timeval start, end;
+		gettimeofday(&start, NULL);
 		rc = p11->C_Encrypt(session, plain, sizeof(plain), ciphertext, &len);
+		// stop timer.
+		gettimeofday(&end, NULL);
+		// Calculating total time taken by the program.
+
 		printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+		printf("Encrypt took : %ld micro seconds\n",
+			((end.tv_sec * 1000000 + end.tv_usec) -
+			(start.tv_sec * 1000000 + start.tv_usec)));
 
 		bin2str(scr, sizeof(scr), ciphertext, len);
 		printf("Ciphertext:\n%s\n", scr);
@@ -1750,8 +1760,17 @@
 	rc = p11->C_GenerateKey(session, &mech_genaes, secretKeyTemplate, secretKeyAttributes, &hndBaseKey);
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
 
+	struct timeval start, end;
+	gettimeofday(&start, NULL);
 	rc = p11->C_DeriveKey(session, &mech_derive, hndBaseKey, deriveTemplate, derivedAttributes, &hndDerivedKey);
+	// stop timer.
+	gettimeofday(&end, NULL);
+	// Calculating total time taken by the program.
+
 	printf("- %s : %s\n", id2name(p11CKRName, rc, 0, namebuf), verdict(rc == CKR_OK));
+	printf("Derivation took : %ld micro seconds\n",
+		((end.tv_sec * 1000000 + end.tv_usec) -
+		(start.tv_sec * 1000000 + start.tv_usec)));
 
 	if (rc == CKR_OK) {
 		printf("Derived Secret Key:\n");

After compiling I just run the test with src/tests/sc-hsm-pkcs11-test --module src/pkcs11/.libs/libsc-hsm-pkcs11.so --pin <user_pin> --invasive. Beware that to run the encrypt and key generation tests --invasive is required.

1 Like

Nitrokey HSM 2, firmware version 3.4:

Derivation took : 105025 micro seconds

Can’t get encryption result yet, because one test is broken as of Fix report of AES key sizes (Fixes #36) · CardContact/sc-hsm-embedded@4f26f20 · GitHub

Looks like one needs at least AES key on the token to run those tests.

Now I got this:

Derivation took : 105620 micro seconds
Encrypt took : 85725 micro seconds

Thanks for your effort. It’s astonishing how much worse the Nitrokey HSM 2 performs compared to the SmartCard HSM. 85 ms for 2 blocks of data. That means the Nitrokey takes 70 ms longer to perform the encryption than the SmartCard HSM.

You are using the newest SmartCard-HSM 4.0. This uses what I think is a different, new chip using flash memory, not a slow EEPROM. Probably with the SmartCard-HSM 3.x the results were the same, since this is what is inside of Nitrokey HSM 2.

Here are some interesting benchmarks, JCAlgTest - Comparative table I think both our smartcards belong to the category similar to one of the listed, but I don’t know exactly which.