hey,
generally the HSM2 should have far more mechanisms available apart from sha1, see pkcs11-tool -M
outputs:
❯ pkcs11-tool -M
Using slot 0 with a present token (0x0)
Supported mechanisms:
SHA-1, digest
SHA224, digest
SHA256, digest
SHA384, digest
SHA512, digest
MD5, digest
RIPEMD160, digest
GOSTR3411, digest
ECDSA, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-SHA1, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-SHA224, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-SHA256, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-SHA384, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-SHA512, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed
ECDH1-COFACTOR-DERIVE, keySize={192,521}, hw, derive, EC F_P, EC parameters, EC OID, EC uncompressed
ECDH1-DERIVE, keySize={192,521}, hw, derive, EC F_P, EC parameters, EC OID, EC uncompressed
ECDSA-KEY-PAIR-GEN, keySize={192,521}, hw, generate_key_pair, EC F_P, EC parameters, EC OID, EC uncompressed
RSA-X-509, keySize={1024,4096}, hw, decrypt, sign, verify
RSA-PKCS, keySize={1024,4096}, hw, decrypt, sign, verify
SHA1-RSA-PKCS, keySize={1024,4096}, sign, verify
SHA256-RSA-PKCS, keySize={1024,4096}, sign, verify
SHA384-RSA-PKCS, keySize={1024,4096}, sign, verify
SHA512-RSA-PKCS, keySize={1024,4096}, sign, verify
RSA-PKCS-PSS, keySize={1024,4096}, hw, sign, verify
SHA1-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify
SHA256-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify
SHA384-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify
SHA512-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify
RSA-PKCS-OAEP, keySize={1024,4096}, hw, decrypt
RSA-PKCS-KEY-PAIR-GEN, keySize={1024,4096}, hw, generate_key_pair
So my assumption would be (w/o having any experience with a Windows CA) that the tooling is responsible to use any of those. In order to help, you might also want to share more specifics or even the exact commands so somebody else can reproduce your issue.
best