I’m also curious if an AI could process the screen feed quickly enough to compete in first-person shooter games. Seems like it would be difficult without extremely high end hardware for the foreseeable future?
- take a screenshot
- run massive skeletal detection on it to get the skeletons of any humanoids
- of those skeletons, pick a target closest to the player
- for that target, get coordinates of head node
- run a PID to control the cursor to where the head node is located
- move the camera one frame, repeat the whole process. If you can fit that pipeline to 16ms it can run in real time.
Someone has even rigged up a system like that to a TENS system to stimulate the nerves in their arm and hand to move the mouse in the correct direction and fire when the crosshair is over the enemy.
We are definitely already there.
holy shit that's amazing.