I didn't want to be the first to say it, but it does feel very inappropriate and gay to ask another straight man that I barely know to hang out one-on-one.
Which means that you need to meet potential friends through and with a group which makes it a lot more difficult.