Flying a Drone in First Person View Using the Oculus Rift
laughingsquid.com
laughingsquid.com
I'm in the FPV hobby for over half a year and fly quadrocopters and model airplanes using video goggles. The Rift currently has one important problem that, until overcome, will prevent it from gaining traction in this area: latency. It doesn't have an analog video input, and having to digitalize and process the video for the Rift introduces a delay into the signal that makes controlling your model uncomfortable at best, impossible in many cases.
Because of this problem, pretty much everybody uses analog video for FPV these days. There are goggles on the market made specifically for the job: FatShark series, the Zeiss Cinemizer HD and others. Some of them are HD and 3D capable (Cinemizers), many are fitted with built-in motion trackers (e.g. FatShark Attidude SD) that are easily bound to servos on the model controlling camera movement (pan-tilt setups), and there are external motion tracker modules easily used with the goggles that don't come with them. With no delay, flying even very fast aircraft is actually much easier in FPV than using traditional line-of-sight control. The latency problem is worse than it might sound: e.g. the GoPro cameras introduce a very small delay on their analog outputs (<50ms), and even this is absolutely noticeable by an experienced pilot in FPV flight and influences the control the pilot has considerably; the best latency currently achieved with various FPV setups using the Oculus Rift is in the 200ms region.
Another advantage of using "traditional" motion trackers over the Oculus Rift is that they output PWM signal, ready to be fed into your R/C transmitter and then used on the model with servos to move the camera.
Now, I understand that HD and 3D is eventually the future of this hobby, and the Oculus Rift might very well at some point become a very viable goggles and motion tracker assembly. But until the latency issue is solved, it's not going to gain a wide audience.
One cool thing is that the past couple of years have seen some serious disruption in the field. Open source firmwares for controllers are a thing now, and it's had nearly the same effect on RC that the WRT54G had on the consumer router business - you can now do things with very cheap hardware that used to cost hundreds or thousands of dollars.
Some links for you:
ER9X Firmware - https://code.google.com/p/er9x/
FrSky, a company doing a lot of interesting things in terms of radio equipment - http://www.frsky-rc.com/
The aforementioned RC Model Reviews - http://www.youtube.com/user/RCModelReviews
http://rcgroups.com/ http://flitetest.com/
RCGroups - contains endless gold for enthusiasts and hobbyists and mad scientists, alike. I suggest you get right into the forums in whatever subject interests you, and start looking for the build threads. This site, alone, contains enough information to get you seriously hooked on RC again.
FliteTest - fun, easy to watch videos on getting into the hobby. Also, these guys produce easy to make planes that will give you all the basics - me and my 6 year old recently got the FT Basics kit (nutball, delta, flyer) along with an FT Spitfire 2-pack, and we're having the best time of our lives flying a nice variety of cheap, easy to build/repair, planes. We'll probably go FPV in a little while ..
The latency on the military drones is actually far greater due to the latency on the satellite connections. The military drones are controlled on take off and landing via radio, but switch to satellite control for the mission. The lag does indeed affect the control. US military drones crash all the time. [http://www.zerohedge.com/news/us-drones-dropping-flies]
None of this matters when you can just get the Fed to print more USD and buy US treasury bonds with them and you just use that money to buy more drones.
If that could be brought down--say, by an FPGA or Paralella or something, maybe it wouldn't be so bad.
I was really hoping the cameras would be mounted on a 3-axis gimbal, with each axis controlled by the Oculus motion sensing.
The Oculus is a great option for for first person flying, even with using it as a regular 2D display, $300 is pretty cheap compared to most head mounted displays.
The latency was small, on the order of 10-12 milliseconds, but it took me months to overcome the weirdness of opening your mouth to speak but not hearing anything until ~1/100th of a second later. I shudder to think what it would be like to have this in your visual cortex but with a longer delay and also while your eyes are telling you that you're floating above the ground.
Here's how I would do it: The craft has two wide angle cameras. The Oculus viewer is at the center of a sphere in 3D space. Project the video feed onto the inside of the sphere, and keep the position of the feed anchored to where the camera is currently positioned in relation to the craft.
Moving your head moves the frustum immediately, and the camera position lags behind, so if you moved too quickly (say, turned 180 degrees) the frustum moves right away, but the video only comes into view after the cameras are repositioned. There could be numbered checkerboard or other pattern shown wherever the video feed is not projected, so spatial disorientation is minimized.
That way you get the cockpit feel and immediate movement in the Oculus, but avoid the uneasiness that latency would give with a more simplistic approach.
Another option would be to use 360 degree cameras to fill up the entire sphere, and then use the binocular cameras to provide high detail.
We had the people from Velodyne who make the 3d Lidar for googles self driving cars present last night and demoed their newer 1kg unit.
Something better which I've already seen is a gimbal[1] (which you can build for <100€) with one or two (for 3D) gopro cams. Weighs less and is way more awesome.
[1] http://s3files.core77.com/blog/images/2013/04/freefly-system...
Addendum:
This is the sort of steadiness you get with a gimbal btw: https://www.youtube.com/watch?v=NmSFZydbVnc
I wonder if it would make you nauseous looking through the Occulus?
The cool thing about the rift is how modern components and graphics capabilities allow for a light VR device to provide a very immersive experience.
This is basically being used to show video from the drone's camera. No stereoscopic image, no controlling using the gyro/accelerometers. Also, low latency, that is one of the most important aspects of what makes the rift experience interesting, certainly wasn't part of this experiment.
No point in posting a video showing new/trendy devices being used in such a meaningless way, just for the sake of combining cool toys
Did you even click the link? Of the three sentences, the second one starts out with "Using its two cameras", and the two cameras are shown within the first 10 seconds of video.
>What is the point of this?
The stereo view allows for depth perception in flight. They're getting fish eye lenses, and one thing that's hard to express about VR helmets like this one is the sense of space around you, even without stereo vision. This will create a feeling of flying that no monitor could ever hope to achieve.