Approach the issue orthogonally. Open a conversation about something else, then ask, "Oh did you ever get a call about that reference?"
1. If he denies receiving the call, that's him.
2. If he becomes unfomfortable that's likely him, but you need more information.
3. If you receive any other answer, press on.
"Yeah, the recruiter gave me a run down on what everybody said so far..." Now wait. See if that elicits anything. A change of topic without answering would be the typical "poor man's" lie.
If there's no obvious indication, you're now in a position to ask "informational" questions. First, restore their impression of anonymity. Slip in, "Yeah, but the reviews weren't matched up with names." If there's an obvious sign of relief, that's likely your person.
"Do you remember what questions they asked, by any chance. A friend asked me to give her a reference, and I want to avoid any potholes." You can circle around the topic like that for a few questions. A typical honest person will quickly become bored by the topic. A dishonest person will politely, and fully give as many answers to your questions as you want.
That's a start. Try to ask this face to face. Trust your instincts here, not your reason.
Remember your brain has a tremendously complicated apparatus for detecting lies. But this apparatus operates subconsciously. To utilize it, you'll have to become comfortable with observing your own responses. Does something "seem" off?
Another useful tactic is to simply ask twice. Lies are hard to remember, and they shift over time to explain away new evidence. The truth remains constant.
Remember, this doesn't have to be perfect. You're really just trying to probabilistically improve the chance that you successfully eliminate the bad reference.
To be honest, however, I doubt the person even realizes that their reference sunk you. As others have suggested, it was probably an accidentally lukewarm, "Oh yes, I can confirm so-and-so worked here. He's a hard worker, and he has my recommendation." That's pretty much the boilerplate HR reference, for the exact reason of avoiding liability.
I wouldn't even bother with social engineering. You're more likely to damage your social network than gain anything of use.
Pick yourself up, find another gig, use different references, and move on.
Remember, you're in one of the most in demand fields in the economy. Relax and find someplace that doesn't ghost you over a hiccup.