(a) they reversed the public + private parts of the key, and were upset when I communicated the public part of the key in cleartext
(b) they speced that the string being encrypted could not exceed 8 bytes ......
I tried so very hard and very patiently to explain to them what they were doing wrong, but they confidently insisted on their implementation. To deter fellow devs from trying this, I left loud comments in our code:
> So these guys are totally using RSA Crypto wrong. Though it's a PK Crypto system, they insist on using it backwards, and using signatures to send us cateencrypted values, and we send encrypted values back to them. It's dumb. I suspect someone read through the PHP openssl function list, spotted RSA_encrypt_private and RSA_decrypt_public and decided to get overly clever.
> This consumes a public key, and uses it to 'decrypt' a signature to recover it's original value.
> To further deter use, I will not add additional documentation here. Please read and understand the source if you think you need to use this.