A Lens-Less Camera Built Specially for AI and Computer Vision Programs

Link to paper in question (with the images you wished were in the article):

https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-18-2...

Edit: Fixed link

This reminds me of The Expanse (Nemesis Games). In the future (~2300s), they use an "expert system" to recreate images of stars obscured by dust clouds. It creates a "computational lens that couldn't exist in the physical world". In one instance, the main characters use this technology to recreate images from a hand-terminal (smartphone) whose screen has been shattered.

Relevant excerpt: https://books.google.com/books?id=A_-zBAAAQBAJ&lpg=PT239&ots... Read the paragraph after the lightning bolt.

Genuinely excited about the applications of this technology to astronomy.

In the extreme, the logical extension of this kind of application is that over time every surface could be turned into a lens-less camera, by applying a thin photon detector film to it and using AI computation to transform low-quality photon sensor data into photos for human or machine consumption. It could very well be that in the future, any wall, glass pane, or other surface could be cheaply configured to watch/record its surroundings.

This reminds me of the "metric cameras" that are used in photogrammetry.

Circa 1940 that field was mostly about measuring things on the ground but then it became 95% about aerial and space photography.

Cameras intended for normal photography are optimized for sharpness, metric cameras are optimized for making correct measurement. That means the lens design is a bit different.

I've been curious about the side-view camera in my Honda in which I'm not sure that the "focal plane" is really a plane. I guess some day I'll find a wrecked Honda at a junkyard and get a mirror assembly to play with.

Looks really cool but might be sensitive to tiny perturbations because the inverse function is over fitted to the optics which are basically random noise. I am thinking temperature changes, scratches and knocks.

Another trend towards lower-power and lower-fidelity in AI-driven computer recognition systems. Last week, IBM announced it was developing better 8-bit AI hardware as an alternative to 16- and 32-bit systems, speeding computations and reducing hardware demands.

https://venturebeat.com/2018/12/02/ibms-8-bit-ai-training-te...

I guess once the human aspect is removed, the sensor and processing systems can be tailored for the computer aspect of recognition tasks, which isn't necessarily as powerful or high-definition as we expect.

I've been expecting some means of a display screen being the camera (pretty theory: phased array optics, a sensor at each display pixel). Interesting development.

> So, Menon asks, “If machines are going to be seeing these images and video more than humans, then why don’t we think about redesigning the cameras purely for machines? Take the human out of the loop entirely, and think of cameras purely from a non-human perspective.”

From a technical perspective, this is neat.

From a societal perspective, I want the images an AI sees and the AI's decision-making process to be auditable by humans.

I don't have the link handy, but this reminds me of the 3D reconstruction from images taken through a pane of glass with water droplets on it. Each droplet acts like a little lens with a slightly different perspective and you can use those differences to reconstruct the 3D structure of the scene.

Direct PDF: https://www.osapublishing.org/DirectPDFAccess/F9AF468F-C6F8-...

Extended light sources like LEDs close to the planar sheet of course couple to some TIR guided ray paths in the glass. But seeing far away objects like with a camera ... probably only those that are visible at larger angles to the plane normal. The camera is probably blind to the angular space close to the plane normal.

Makes me think of Dawkins' demonstration of eye evolution, where he shows the usefulness of even simple photo-receptor designs: https://www.youtube.com/watch?v=Nwew5gHoh3E

This is a really interesting concept. While this initial prototype is rudimentary, I agree with them in this important shift to building cameras "good enough" for machine-usage.