Creator here. Thank you for sharing!
> My research goal is to train models using various hardware telemetry data with the hope that the models learn to interpret sensor inputs and control actuators based on the insights they glean from the sensor inputs. This research direction may open up exciting possibilities in fields such as automation, space, robotics and IoT, where L2E can play a pivotal role in bridging the gap between AI and physical systems.
This part was easy to miss but quite interesting, could you expand a bit here? What does L2E stand for?
I am trying to replace the 2.4ghz controller on my electric skateboard to make 0 to 5kmh and braking more pleasant and maybe use gyroscopes to do away with the controller altogether. What would tokens be in that case? Do you create a CAN style representation and feed that to the llm? What kind of throughput do you foresee being possible on which hardware?
Telemetry in sense sensor streams, both command and responses.
Then we'll just train a small model for long enough. Then we will see how it would respond.
That's the plan sort of.
Straightforward control problem of bringing the board from 5-0kmh smoothly?
Figuring out exactly which model to use and how, may take some work. Also understanding control theory will allow you to do things like traction control, etc.
um...
cat /dev/llama
Our goal is to write a proper kernel module to implement three things:
1. a character device 2. 1st backend is a LLMZip ie you write to say /dev/l2ezip, you get a compressed stream out 3. 2nd backend is a LLM, ie you write a prompt to say /dev/llama2, you get a completion back
So the 1st backend could be useful for compressed telemetry The second backend could be useful for IoT LLM, or our ambitious plan of responding to telemetry, ie take action, such as control motor speed etc.
We need to go deeper!
Qemu:
qemu-system-x86_64 -display gtk,zoom-to-fit=off -m 512 -accel kvm -vga virtio -cdrom l2eos.iso