Forum software like vBulletin and Invision often has this feature built in, and I've used it on a forum I help run. Unfortunately, after writing four or five custom questions, I soon found server logs showing spam bots blowing through the questions in seconds -- I suspect that since this is a common enough strategy, it's worth their time to pay someone $0.10 to pick the correct answer, then save the question and answer pair in a database somewhere for future use.