Not the group signer is device specific, but the way the Smart Card Shell constructs the certificate chain that is presented to the HSM on which a key domain is created.
If you generate a group signer on one device and upgrade the firmware, then the device gets a new device certificate. After importing the group signer, the device certificate of the instance with the old firmware is missing from the certificate chain written to the .KDM file.
When you sign a key domain membership the following certificates are written to the .KDM file
DICA A -> Device A -> Group Signer -> KDM
The DICA and Device certificate are taken from the HSM on which the group signer is located.
When you migrate the group signer to a different HSM, then the chain may look like
DICA B -> Device B -xxx-> Group Signer -> KDM
as the DICA and Device certificates are not moved. The Group Signer certificate can not be validated against Device B, so the chain is invalid.
I guess we need to provide a mechanism in the key manager to retain that old DICA and device certificate associated with the group signer, so that we can complete the chain written to the .KDM file.