Hi,
I am building U-Boot signed with an RSA key on a Nitrokey but when I run fit_check_sign it fails verification when it checks the hash. The verification uses the RSA public key combined with the signature to rebuild the original hash value. It checks this hash value against the hash of the bytes in the image. So it’s possible there is something wrong with the signature (even though i know it comes from the correct RSA key on the correct nitrokey).
Is there something about RSA keys or signatures generated on the Nitrokey (or HSMs in general?) that might prevent the hash from being rebuilt from the public key and the signature?
When i build U-Boot from an on-disk local RSA key created in openssl, it passes verification.
Thanks in advance,
Doug