NitroKeyPro and Keybased authentication

So I’m fairly new to more secure forms of key management, I’ve been used to storing my keys inside key files on my computer.
Recently I wanted to try and see if I could setup SSH authentication to my webserver using a key stored on my NitroKeyPro making my keychain more portable and secure in the process.

I followed this guide xmodulo.com/linux-security-with-nitrokey-usb-smart-card.html
pretty much step by step but noticed that in the end, I did not need my NitroKeyPro to be inserted into my computer at all for the authentication to succeed.

I have a feeling that upon exporting my key it somehow got added to my local key storage making the NitroKey redundant but I am not knowledgable enough about the exact workings to be sure.

Would anyone be able to help me ensure that I can only SSH into my web server while my NitroKey is inserted into my computer?

Notes:

  • OS: OSX El Capitan 10.11.4
  • NitroKeyPro
  • Even while the NitroKey is inserted into my computer it does NOT ask me to enter a pin when I attempt to SSH.
  • OpenSC 0.15.0
  • gpg 2.0.28

I tried removing from ~/.ssh the following:

id.rsa
private_key.pem

after attempting to SSH to my web server again I get:

I assume this indicates that the SSH session cannot find my key to authenticate with,
I checked if my computer was detecting the NitroKey by running: gpg --card-status and received card information like I would expect.