Part of my idea for this course was that you could use these methods on any game. You don't need anyone to make an API, you read the frames in, send direct keys, and you're all set.

That said, there are various things like scripthookpy that let you communicate with GTA V to do things other than AI with Python.

I am personally more curious about the AI aspect, so I figure any game that I can play visually, an AI can too with these methods.

In a way, you do. The inputs are well defined (mouse and keyboard), and the outputs (graphics) can be captured fairly easily. Interpreting those outputs is a lot harder than interpreting a stream of "character A at point (X,Y,Z), character B at point...", I agree.

What I want is a full public API to control all aspects of the current view and character.

At a minimum, I want to be able to send positional data to control the orientation and position of the head, and at least one of the hands (ragdoll physics and IK can take care of the rest). That way, I could control where I look with my head, and where I point or grab with my arm - independently.

Ideally - you'd have both arms/hands free - plus you'd need inputs to control other aspects.

For the head, if you crouch down, it would know this, and adjust the body model as well (or perhaps those parts could be controlled as well). The goal?

Better immersion for VR - but platform agnostic; the thing is, I don't want future games to be beholden to a particular or set of HMDs (ie, this game only works with the RIFT, but not the Vive, etc). I also own a nice older kit of magnetic trackers (an old Ascension Flock) that I could use, along with some older HMDs (as a contributor to the KS, I also have the original DK1, plus the later KS backer CV1 - but both of these aren't well supported under Linux, which ticks me off). Basically, let me (or anyone else) control the character in full, and just provide a way to output two stereo frames (don't even distort them - let third-party software deal with that - because you might have an older HMD that doesn't need the distortion). Then get out of the way.

I doubt that we'll see such a development though.