Nathan Sprague has a mostly-working implementation at https://github.com/spragunr/deep_q_rl