Probably SBCL.
I haven't worked much with Gambit, but have fooled around with both Chicken and Bigloo. Of the two, I found Chicken to be the friendliest (it also seems to have a somewhat more active community). Another possible advantage of Chicken is that it's directly supported by SWIG. There was some activity on the Chicken mailing list a few months back about porting some game libraries. I don't know what the current status of that might be.