I've done this, myself. It is really easy to accidentally hit this connect button when swiping on the iPad.
It was a pretty horrifying realisation that I'd become one of those random inviters - now I have much more sympathy to people I've never heard of inviting me to connect! I refuse to use the mobile site again, so I haven't checked if it's been fixed.