It is based on the public/private keys authentication. Please look details in the proper literature, or the OpenSSH documentation. Briefly:
- It is not possible determine where one connects the first time without confirming the fingerprint of the public key from another source (e.g. read from the webservice website). Then the fingerprint is stored locally;
- If the public key will change on the subsequent connections, one will know its a wrong/phony server.
- If the private key leaks, then anyone can act as a trusted party.
About HSM competitors and the market, not thinking too long I guess this is niche, which does not give great return for the investment at the moment (a lot of research, verification and testing). And once the product fails on the market, the company might go out of business due to black PR, so big risk as well. Encrypted communication is just started to being widespread too (the TLS/https to all sites was added just recently - last years).