Rather than type every letter, a more efficient system is to represent sounds with keys. This is how stenography works. Each stoke of the keys is mapped to a series of characters; a letter, a grouping (such as 'sh' or 'ing'), or a complete phrase. Using such a system would allow one motion, for example for 'b', to map to a whole word, such as 'be'. Something as simple as that would reduce your risk of injury by something like 50%.
A major difference between stenography and typing is that steno is chorded: the signal is sent when the keys are released, not when they are depressed. I'm not sure how that would work with the full body.
Check out the OpenStenoProject for more about steno. Their community would probably enjoy discussing a full body steno theory (if they haven't already) .
Take care and thanks for all the effort that went into producing the video! It was a lot of fun :)
The Charachorder One is one example
The best part was that the speech was also signed for the deaf (that part was normal) and subtitled by a ..., well, ... stenography trainee.
He or she had all wrong, could not keep the speed, was mixing words and skipping whole parts. At some point they seem to have said to themselves "oh fuck it" and were typing more or less random words.
The whole country was with them, hoping that they will go on typing until the end (which they did). That was a memorable event and people actually learned about stenography and the fact that they apparently use a special keyboard in the form of a butterfly.
Please let me know if you want to join forces! You can find my contact information at: https://github.com/mristin.
I figured that I don't mind my kids playing computer games as long as they move. The first stage for me is to find a workable approach to DOS games. In particular, I thought about adapting ski-leu to race games such as Outrun & ilks.
These could also be of interest for inspiration as well - basic idea this Youtuber is doing is to use console gamer joystick re-mapper hardware to convert joycon acceleration readings to button presses/button combos.
Can you Lose weight by playing Breath of the Wild? - Ringfit Controller Mod Explained Version
https://www.youtube.com/watch?v=HYn3pWXMPF8
- Today we talk about the Ringfit Adventure controller mod for Zelda: Breath of the Wild. With this mod you can Exercise and play BotW at the same time.
I made a Ring Fit Adventure Mod for Mario Kart - Controller Bending
https://www.youtube.com/watch?v=CNZSfunw85A
- Today on Controller Bending we mod the Ring fit controller to play Mario Kart 8 Deluxe with. With this Ring-con mod you have to run to drive. Literally. Squeezing the ring-con activates items!
I'd imagine a lot of similar mods could be done via computer vision instead of using joycon accel readings.
I think the key would be to show up to a gaming console as a PS4/XBox controller instead of just a keyboard. I think there are Python libraries to do this [1] or just stick to Steam games that just need WASD controls. This allows you to tap into already huge library of high quality games and just focus on the OpenCV movement-> joystick button remapping part.
[1] Something like this - although it might be Windows only https://pypi.org/project/vgamepad/
I implemented a couple of mini-games using a dance mat as well:
https://github.com/mristin/dance-a-mole-desktop
https://github.com/mristin/judo-dance-desktop/
https://github.com/mristin/dance-cat-to-mouse
https://github.com/mristin/burpee-frog-desktop
https://github.com/mristin/dance-runaway-desktop
Dance mat could be even easier to integrate as it acts just as a normal joystick. The problem, though, is that many games expect reaction times achievable with a keyboard & mouse, but leg movements are a bit slower.
The problem with the dance mat is that you can not use it in a school class as you need extra equipment. It worked well in a dojo for my judo classes, though.
https://github.com/mristin/ski-leu-desktop/releases/download...
https://github.com/mristin/pop-that-balloon-desktop/releases...
If you want to run the code manually, install it in the virtual environment. Then you can simply run commands `ski-leu` or `pop-that-balloon`.
Btw., I use pop-that-balloon for basketball dribbling training, and a friend uses it for boxing. Have fun & please let me know if you use it for yet another sport :).
Edit: removed bullet points
You could probably combine semaphore keyboard with some sort of stenography software for faster input (e.g. Plover https://youtu.be/KZGuBV1xe64)
>>Downward Dog Driven Development
Imagine teaching some ML your personal body movements, and then you could have it transcribe text into Semaphore-output video (or live rendering)... At least well enough that Semaphore-input could read it?
Here’s hoping this ends up in a Vulf music video.
Instead of MediaPipe try training a simple NN with a quickstart asking new players to assume the positions for fine tuning? :P
I'm not sure how well it will work yet, but for a game controller I might try to have users record their own custom motions - as in, "show me a few of your chosen [high punch] motions to match against in-game."