Nitrokey HSM / Smartcard HSM in remote location

Maybe someone got an idea for a very special use-case.
I have a CA (based on EJBCA) that is using a Nitrokey HSM to issue x.509 certificates.

That CA is not located in a location where I normally am. So in order to issue a new cert, I need to drive there, plug the Nitrokey HSM into the server and sign the certificate in EJBCA.

Does anyone have an idea of how to do something like that remotely in a secure way? Like plugging the Nitrokey HSM into my own computer and connecting the remote CA to it via a network protocol?

Maybe via SSH or something similarly secure? I have done a little research but could not find something that convinced me (especially security wise).

Any hint or suggestion would be highly appreciated.

That is what the PKI-as-a-Service system was designed for: You a run a service somewhere in the cloud, but keep the keys locally.

The basic principle is, that a small local client connects the HSM to a webservice, that service in turn opens a secure APDU channel with the embedded SE. After that it’s like the HSM is connected to the server.

Unfortunately that only works with a deeper integration into the service and not with PKCS#11.

An alternative might be USBIP or PCSC over SSH.