I suspect the problem is that @Peacekeeper has Terminal > Secure Keyboard Entry turned on.
What happens is that on macOS, processes can request exclusive access to keyboards, e. g. for password entry. The window server then blocks any other process from receiving keyboard events or querying the devices (some background on the APIs: https://developer.apple.com/library/content/technotes/tn2150/_index.html). I assume the Nitrokey App uses some form of pinging the stick as a HID and is then lead to believe that the stick has been removed if the ping times out.
This is not limited to Terminal, btw. The system also uses this to protect standard password text fields in the UI, so every time an application asks for a password, the Nitrokey App goes bonkers, which is quite annoying.
I would really prefer the Nitrokey App to be less spammy regarding connection status, maybe something more subtle like a grey vs a coloured tray icon. The notifications are intrusive without really conveying any information, they catch a users attention only to tell them what they just did, and on top of that are wrong most of the time.