Anyone using this.. what have you built?
I built:
1) Virtual camera: The robot was equipped with 6 fixed cameras. You can position virtual camera anywhere in the coordinate space and it constructs it's image with the information from the real cameras.
2) System for semi-automatic external calibration of cameras from LIDAR sensor. External calibration is the process of estimating the position and orientation of the camera. This system was not as precise as calibration using calibration patterns in lab, but it was useful when the robot was in the wild and you wanted to adjust the camera or strap a new camera on it quickly. However, to be frank I am not sure ft the system was ever actually used that way. I left the lab after finishing my thesis.
The ROS ecosystem is reasonably simple to understand and definitely makes things much easier. Apart from having some issues with compatibility between versions and incomplete functionality I remember it as quite a nice platform. On the other hand I have not tried any of it's competitors if there are any.
can I read your thesis, or (dare i ask) source code?
I will try some hasty writeup ;)
1) Create a virtual image plane (or cylinder/sphere for panoramic camera) in the 3D space. For this you need to specify the virtual camera's position, orientation, focal length and FOV.
2) Define pixels on the image plane. You need to specify the image resolution and just create grid on the the image plane. Each pixel corresponds to a 3D point in the world coordinate system.
3) To get a color of each pixel you project it's corresponding 3D point to all real cameras. Usually only some of the cameras see the point. Each camera that sees the point gives you a color according to the color of the pixel in it's image. You have to perform blending of these colors. What worked well in our case was to give more weight to cameras where the point projected close to the center of their image.
As you can see, there will definitely be some artifacts, usually more the farther away the virtual camera is from the real cameras. However, it worked to my surprise quite well. It was even possible to look at the robot from 3rd person even though all of the real cameras were on it.
Also it is crucial to have well calibrated cameras. Apart from their projection matrices you also need to consider their radial and tangential distortion which was very significant with our camera models.
https://www.wikiwand.com/en/Pinhole_camera_model
My startup is using it to power an autonomous customer service robot. ROS modules handle things like pose calculations, trajectory planning, collision avoidance, and basically all other interaction between software and hardware. From my perspective as a developer on the user-facing software, it's basically a simple event bus and RPC framework that has a load of prebuilt datatypes and modules for handling common robotics tasks.
Caveats – a lot of modules seem to follow pretty sketchy coding practices compared to what I'm used to in other industries. It seems to be a bit of a paragon of bad design. There also seems to be a lot of version confusion – lots of packages seem to support different subsets of versions, with most of the community possibly on older releases. There's an ongoing ROS 2.0 effort that I think uses MQTT as the transport. But I'm not a ROS expert, so take the above with a grain of salt!
One thing with ROS is that reimplementing it is not particularly difficult: I have, for various reasons and projects (in some cases before ROS was available) implemented basically all the core components of it (to similar levels of completeness and reliability). So the cost of doing it yourself is actually not all that high (and I think people overestimate it).
27 people are listed on their staff page.
Wall cleaning robot (still testing): https://www.youtube.com/watch?v=4dMjt_Pj4og
There are some nice projects on https://discourse.ros.org/c/ros-projects, I really like Kyler Laird's Tractobots: https://www.youtube.com/watch?v=oPI7f5kQRKo
It facilitates work by not having to rewrite a bunch of code every time. Also, most robotic related libraries like OMP, or TF are supported.
At my current job we don't use ROS but being able to look at the ROS drivers of commercial arms is a big help in making our own drivers.
https://scholar.google.com/citations?user=fMDLYCUAAAAJ&hl=en...
Use the 'cited by' link to see thousands of robot projects that use ROS:
https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14376...
(I'm not affiliated with Open Robotics)
I also used it as an intern at a robotics company to use a microsoft kinect as a depth sensor for helping the robot perceive space in three dimensions.