My friends and I worked on this problem for a hackathon and even managed to get 3 letters that could be recognized! The judges were unimpressed and the win went to a group that copied a project off of instructables.com :)
If you'll suffer a shameless plug, a friend and I made an (imperfect) ASL learning project at TAMUHack one year. If you're interested: