PhotoScan: Taking Glare-Free Pictures of Pictures | Hacker News Reader

PhotoScan: Taking Glare-Free Pictures of Pictures

research.googleblog.com

286 pointsstablemap64 comments

Worth noting that people who were unsatisfied with the low res results of PhotoScan (a poor 3mp), so modded it to bump the final resolution to 12mp https://www.xda-developers.com/modded-version-of-googles-pho...

Would be great if you can just take a video and it would automatically capture the keyframes relevant for this type of transformation.

The end game would be that you put your phone in recording and on some stand, and you just have to slowly turn the pages of your photo album. One can dream.

analyzing every frame of a video is very computationally expensive and the video resolution is often much worse than the photo resolution on such phones. But in principle the same techniques could be used so i don't see why it shouldn't be possible at some point.

I think there were some good counterpoints to what you said. But the most important one is.. USER.

People will record the video differently hence, adding more complexity to solving, comparatively, a simple problem.

One person might do the movements really fast, which leads to blur, which would mean they now need to add de-blurring. Other person might even miss an angle that was required. I think this is likely why panoramas are made in parts.

I bet that's probably the easier way to implement it, however the resolution on the camera is almost always better than video.

Is this very different from how it works? From what I can remember from the last time I used it, you point it at the image, and it gives you a set of points to center your camera around, automatically capturing the photo at those points, so you never have to press any buttons.

Turn the pages and wave the photo album back and forth :D

That would be extra processing analyzing frames trying to find the right ones. Again, the biggest problem with this is doing it on the phone at a reasonable speed.

Interleave a lower resolution higher frame rate video with the maximum frame rate of the largest size image (5-8 fps). Process the images in the background when the phone is wall power, and or push to cloud. It would be nice to do an incremental first pass to see if one has high enough quality source material.

Or use a hotword, e.g. "scan" after each page turn

I tried this but the resulting image either still had glare or was super low-quality. Too bad, really hoped it would work.

Have you tried it again recently? The new update allows for higher quality.

Nope, tried it last 4 months ago.

Same conclusion for me. I had much better results using the default photo app, while taking care of avoiding glare.

Yeah, just tried it. It was good enough that I would use it if I was at a friend's house or something and wanted to take a picture of a picture, but otherwise I'd use a scanner. Far better quality, both in terms of sharpness and colour balance.

What I really want is a sheet feeding photo scanner that scans the front and back and won't destroy old photos, or even better, a company that will rent me one for a few months. Then I can scan the 1000s of photos my parents have, and send it back.

That would be handy. You would think such a thing must exist. There are certainly companies that will digitize all your photos for you, including some cleanup in editing software, but I believe they charge ~ $0.75/photo. Anyway, they must have some decent hardware.

I just finished scanning a batch of my old high school photos, but I didn't take a lot, so it was only a few hundred. Managed to fit three at a time on the bed of my all-in-one printer/scanner, but it was tricky because there was a dead zone along the edges, so they had to be in the middle, but spaced out enough that the software could auto-crop them. Still not TOO bad, but I wouldn't really relish doing it for thousands.

Doesn't seem like an insurmountable task for a week of netflix and a ScanSnap scanner[0]. Personally, I have the s1300i and scanned thousands of pages of notes.

[0]: http://www.fujitsu.com/us/products/computing/peripheral/scan...

I looked at the ix500, but couldn't find much in the way of people using it with photos, a) quality, and b) how well it handles photos rather than paper

How did you hold the phone? Based on the article you'd think it works better if you held the phone at a slightly different tilt every shot, to ensure non-overlapping glare

Same - blurred patches, and generally very low resolution :(

Really nice challenge for computer vision: given a video of a camera moving around above a photo, with a variety of (unknown) distances and angles, for each frame, find the photo, merge pixels to give super-resolution. Even with all the advances lately in vision research we are still not there yet.

There was some talk of the maximum resolution being 3MP, but that through some hackery this could be increased to 12MP.

I didn't see any mention of resolution in the blog post, and many comments here and elsewhere about low quality.

Is the resolution still capped?

From the news I heard, this new update they talk about allows for higher resolution (on faster phones).

I tried this just now - yes, it removes the glare, but the resulting image is very low-res compared to just using the camera app.

So I really like it... but...

It makes really blurry photos. Every photo I take looks like someone turned the pixelate filter on.

Any suggestions... I'm doing this at night with lights on... will try it tomorrow during the day to see if more light is the answer.

EDIT: The photos I get back aren't only blurry, compared to photos I take with my phone camera, but they are about 2/3rds the size. iPhone 6S... I'd rather it take a bit longer on the compile and get me a better quality image.

Example. http://i.imgur.com/63tgrwK.jpg

I used PhotoScan in December to digitise a lot of my parents photo albums. I was really impressed with the ease and results. Right now I would consider using anything else.

I found MS Office Lens a tad better when I tried both a few monthes ago. Something in gPS bothered me but I can't remember what. That said this upgrade seems quite serious.

you would consider...or you would not consider using anything else :-)

Oops, not consider. I keep doing that!

But that said, I've not explored what else is out there.

*wouldn't

maybe

Had bad results with an avg smartphone camera.

It sucks when science creates something cool but it doesn't work in the real world.

There's also this problem https://www.reddit.com/r/Android/comments/5dbo07/google_phot...

> After testing PhotoScan on a variety of pictures using four different phones, I'm sad to report that we're not getting our camera's worth from the app. The long dimension of scanned pictures is always scaled (up or down) to 2,000 pixels, compared to the 4,000-pixel capture resolution of the 6P (12.3 MP -> less than 3 MP).

I noticed the limited resolution, as well.

If you are looking for real "archival" imaging, this app probably won't suffice.

On the other hand, it's nice to have on your phone to grab the occasional image e.g. from a picture on the wall behind glass or something similar. As another commenter mentions, maybe to grab an old photograph out of an album and put it on Facebook or the like.

Users should also note, its cropping is not always accurate -- larger contrasts produce better results. But, after it grabs its initial image, you can adjust the cropping -- including outward, to the limit of what was initially imaged.

P.S. I have a Nexus 5x. I'm guessing the app may work relatively well for me also because I'm using a phone that the Google developers are particularly likely to specifically support and have used/tested against.

Since this appears to be a small project -- maybe a 20%-er or the like? -- at a guess, results may vary significantly depending upon phone and camera module. At a guess -- I've no idea.

Having an average smartphone is where you went wrong - you need to persuade your employers that you really need a Google Pixel XL. Get one and then you can expect cool smartphone apps to actually work, albeit on the maxed out Google phone but not on your own normal phone.

In day to day use you may find the big phone with its 'best camera ever' and full suite of sensors to not be practical compared to your svelte normal phone. The added bulk, the fear of dropping the bulk or the likelihood of theft just puts you off carrying it around. But when these exciting science experiment apps come along then you are good to go, no need to make space for the app on your normal phone (where it probably won't work anyway) you can put it on the big phone. Even more fun, you can share the big phone around without fear of your personal messages popping up on its screen - a guest phone as it were.

The latest and greatest Beta version of Android is also default on the Pixel series, I didn't even have to install PhotoScan, mysteriously it was already installed on mine when I had the 'must try this' moment a few minutes ago.

Android phones are a bit like office PCs in the 90's when you could have some 386 running Wordperfect in the same room as some Pentium machine used for CAD, sure they all use some variation of the same operating system but that 'flight sim game' would only be worth attempting on the latest/greatest hardware and totally useless on anything less.

I used PhotoScan (iOS) to scan about 20 photos, and it worked surprisingly well. The quality was great, no clipping or unexpected cropping. Really impressed.

It also worked quite well for me on a Nexus 6P.

Wouldn't it be easier to arrange your lighting to not reflect back up to the camera (i.e., like a copystand)?

Copystands don't look cheap:

https://www.bhphotovideo.com/c/buy/Copystands/ci/711/N/37151...

Copystands that are large or have integrated lighting are quite expensive. I use a $45 smaller copy stand with a pair of $7 lights clamped to the shelf above my desk. If you have a real camera with a tripod mount, it's a very reasonable solution.

Wow, they are expensive for what they are.

You'd think there'd be lots of cheap ones coming out of China, but I couldn't find any with a quick search.

I had the same thought. Disable the flash. Use a copystand. Or, just use a scanner (the last three personal printers I've owned double as scanner/copier).

This is a lot simpler and all you need is your phone. It's obviously not meant for pro/enthusiast photography or advanced users.

It's a perfect way for a not-so-technologically-savvy parent to scan their old photos and put them up on Facebook.

Copystands are not free. This app is free.

There are dozens of apps already on the play store that will do geometry correction for images taken at an angle. No stand required. My partner uses one for working in archives.

Easier for the developer or the user?

My takeaway is that the twitching hip is disturbing.

https://4.bp.blogspot.com/-Q5bnNur88ME/WPfbie00jGI/AAAAAAAAB...

I get good results just taking pictures of pictures outside in bright sunlight, as long as the sun isn't directly overhead. They are evenly lit, plenty of light so not grainy, and no glare.

Very interesting. I had similar problems when photographing my paintings, and while the best photographs are taken with a good DSLR and a polarizer that can end up becoming a project in itself.

Many comments the output pixel count, no one seems to have noticed (1) the white border (2) ad at bottom (3) JPEG artifacts (block and mosquito noise on edges, dull and noisy colors, color smear) excessive compression. No wonder: 158kbytes for a 1399x2184 picture is 0.05 byte per pixel which ... may be enough or not depending on your quality expectations.

It might be that photos that are sent to Google get saved with decent compression (0.5 byte per pixel instead of 0.05) in DCIM/PhotoScan. Yet what if I want to keep ridiculous old haircuts private? There's no explicit "save locally" feature. :-/

Oh great, a third PhotoScan. http://forums.culturalheritageimaging.org/index.php?/topic/3... (One is an extremely popular SfM package, and the other is an obscure SfM package.)

I would be interested to see a side-by-side of the original photo, and a (high quality) print out of the photoscan image.

The new capture mode with no glare removal has much higher resolution, still does the cropping and rotation.

I tried this and my result was really low resolution and had a giant blob of glare on it. Had really high hopes from all the technology behind it, but it didn't work in my case.

Same here - both glare and low resolution have been problematic. I see that someone has tried to deal with the resolution, though: https://www.reddit.com/r/Android/comments/65p6n9/google_phot...

Incompatible with S7 edge :(

Do you know if you have the Exynos or Snapdragon version? I hypothesize it depends on the latter's Hexagon DSP.

I have Exynos. Feels bad.

PhotoScan [1] seems quite fancy. I wonder if they have super-resolution and post stabilization flow de-blurring in the pipeline for "simple" photos.

[1] nice initials btw

it looks like a new thing, but 2 weeks ago I stumbled upon this via a link in Google Photos app and tried it. Resulting picture did not look better then one taken without flash in a lighted room using camscan app.

brilliant bricklaying. Is this really the best thing CV researchers should be doing? There are so many unsolved problems and they're working on removing glare from old photos?

This is neat. I've got a ton of these "pics of pics" of old family members so this definitely appreciated.

Who's the guy in the sample pic? One of the researchers?

Reply on news.ycombinator.com