Converting a fisheye image into a panoramic, spherical or perspective projection | Hacker News Reader

Converting a fisheye image into a panoramic, spherical or perspective projection

paulbourke.net

186 pointslnyan27 comments

Transformations like these have wide applicability. Performing optics in software so to speak, is more flexible and cheaper than grinding lenses and building special cameras.

I was charged once, with transforming images for projection onto a sphere through special lenses. I didn't know the lens transform, the manufacturer kept that secret. I did know that the intensity across the spherical surface was of uniform brightness - the manufacturer boasted about this. With that bound I could guess the transform, as each pixel of the source image would have to illuminate an equal surface area of sphere once projected.

The manufacturer advertised a service to do this transform on your image, but they wanted $10K for each run! Once I presented my insights and a little math, they capitulated and did it at cost. I never got to write my projection software in the end.

Hey, that's a nice case study!

Could you give me some pointers on where I could start studying about the math involved? Are those topics usually covered by good books on computer graphics?

I loved "Multiple View Geometry" by Hartley & Zisserman, but also "Computer Vision" by Rick Szelisky is pretty comprehensible and covers a lot of (pre-ML) stuff.

My goto, which has literally saved me weeks of banging my head against a wall, is:

Schneider, P.J., Eberly, D.H., 2003. Geometric tools for computer graphics, Morgan Kaufmann Publishers

indispensable

Sounds like a classic f-theta lens design. I'm sure you were right about your transform, but it's also possible to equalize illumination by other means, like apodizing filters.

Not sure if this was shared already, but here's an example of the reflection of Buzz Aldrin's visor unwrapped into a 360 image: https://www.reddit.com/r/space/comments/ooexmd/i_unwrapped_b...

A few years ago, I was working for a startup in Pakistan which was making small (5-10 person) portable planetariums. The idea was to make astronomy education accessible to people from every socio-economic background. Paul's work on this [1] was amazingly helpful to get our projection done well, and put on successful shows.

Also, shout out to the European Space Agency for releasing high quality Planetarium documentaries under the CC license [2].

[1] http://paulbourke.net/dome/ [2] https://www.eso.org/public/videos/archive/category/fulldome/...

I bought some interesting photos from an artist who has done some things like this, sort of in reverse:

https://kovasi.photo/

no connection to him other than as a customer.

By the way, printing on aluminum seems like a really nice way to mount art on the wall

Thanks for sharing ... his photography is surreal.

Paul Bourke's webpage is a true internet gem for 3D reference information. I don't know how many times I must've landed on his page on 3D file format specifications.

Me too. An amazing resource.

Paul was kind enough to meet with me a few times, probably over a decade ago now, when I was a lost student trying to work out what ideas were worth exploring (I guess I still am).

He showed me some photogramatery work he was working on, I was amazed at how smooth everything was running given the precision and having worked with highly detailed tins and pointclouds myself on that days high end hardware. I asked him what kernel/engine he was using. He casually said it was his own. Which (having done toy ray tracing in C and dabling in openGL) blew my mind but also made me realise it would be absolutely necessary in order to do this kind of work with the type of control he needed. Everything from the ground up.

One of the few academics/researchers I've met who is 100% legit.

> If "straight" lines are not straight that normally means the fisheye center or radius are not specified correctly or the angle is not defined correctly.

Is it possible to make the computer guess the correct value with AI? I guess a method is to identify the artificial objects like walls and doors, and assume they have straight lines. The nice curved tops of the windows may be a problem. And probably there must be a special case for circles like the clock.

Calibrating using known objects should be possible. Most road sign dimensions are specified in national laws, motor vehicle headlights must be a specific distance apart, etc.

For a single image, you'd need some heuristics to distinguish curved objects from camera distortions. If you have multiple frames, you can SLAM your way out of the problem (essentially, match features and solve a big fit which optimized feature positions, camera positions and camera distortions). But by far the easiest option is to just calibrate it, through https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.h... or one of the many alternative methods.

What would be good to see on this page: an example of the correction of unwanted fish-eye from a portrait photo of a face.

You could fix bad profile photos and such without the need to upgrade to more expensive optics.

The applications of this are interesting to say the least.

Something i am currently working on is using fisheye images of the sky to predict cloud motion and to then estimate solar PV power output in the short term i.e. one to five minutes out.

It sounds simple but it isn’t yet even a child can observe cloud motion and predict to some degree when the disc of the sun will be obscured. Part of the inaccuracy is in the distortion caused by the fisheye lens.

I would be interested in seeing how well this works for subjects at infinity i.e. a fisheye lens pointed at the sky.

Wow, some of the distortions at the end of the article are really mind bending...

For simpler fisheye and defisheye effects, I wrote this web based tool a few years ago which some may find useful!

https://jywarren.github.io/fisheyegl/example/

Back around 2006 I used "panorama tools" to do the same sort of thing. It had this cool feature of being able to work out lens calibration if it had two photos from the same viewpoint but different angles. It did however require careful tagging of reference points on each image.

The logical successor to panorama tools is hugin these days, and yes it is able to perform these transformations very easily. The control point generation can be done automatically now, although I admit it might struggle a little with fisheye images.

I just got stable-diffusion running and thought the title of this would make an amusing prompt.

https://imgur.com/a/9TgW7qy

This means that open software for handling the raw files from 360 action cams is possible! Would be amazing if kdenlive et al could do it natively.

Just throw it into im2im stable diffusion (joking but it also would likely literally work).

I have seen these pages few years ago. But I could not find source code for these examples

> The source code implementing the projections below is only available on request for a small fee.

That shot is so iconically inner Sydney. Thinking Glebe or Paddington

Has anyone tried asking DALL-E to do this?

Reply on news.ycombinator.com