What probably happens: your PC/SC daemon has Policy Kit integration enabled and one has to enable the smartcard usage for the particular user: