Hacker News Reader

I think you have somewhat unrealistic expectations of what even a decent photographer could achieve with a graduated neutral density filter (not to mention tilt/shift lenses... maybe 0.1% of even professional photographers use those). The photos in your blog post would have been really hard to achieve with "traditional" photography methods to cover the huge dynamic range (a graduated neutral density filter won't cut it for a street opening facing the sky). Most experienced photographers would employ some form of exposure stacking in that scenario, same as the iPhone did. Of course having more control over the blending would be great. I think if you tried to take the same pictures with a traditional camera you would be surprised by how good the iPhone pictures are compared to the effort it takes to achieve a comparable result.

Oversharpening is a curse in the industry to be sure, unfortunately to most people an oversharpened image looks "better".

Side note: I'm surprised by how heavy-handed the HDR looks in that particular set of photos. I'm used to seeing modern smartphones do a much better job of auto HDR in most situations.

Oh no doubt - it still takes experience and technique to achieve a good effect with those tools, and of course they can't be used in all situations so other techniques are required; but it still seems to me that "out of the box" the software is applying HDR like techniques when it isn't needed, or it would look a bit better if every part of the exposure wasn't normalised so much.

It seems that the software is doing a negative film photographer's trick of "expose for the shadows and the highlights will take care of themself" but rather it's "expose for the shadows and the HDR will take care of the highlights" - ending up in this overprocessed looking fake sky effect.

It's like there can't be any ambiguity anymore in a photo due to the exposure, the shadows and the highlights can't clip, we have to see everything. That seems a big change in mindset to me.

I've been giving this some more thought and I think in many ways the photos from modern smartphones are more similar to paintings than what we are used to from photography. They try to create a visual representation of what our visual cortex would perceive the scene to look like.

A painting of a room with a window would typically show both the room and the scene from the window "perfectly exposed", like the iPhone is doing with your skies.

This looks very strange to someone used to images from more traditional cameras, and the artefacts like halos stand out.

Photography on the IPhone is good enough for the majority of users. There’s a physical resolution limit on lenses. Hence why the sensors have stayed the same size.

The iPhone offers strong post processing tools for photos and video that make it easier and more accessible.

I’d love a smartphone with an Arri Prime lens equivalent, but it would never fit in my pocket

Photography on any half-decent smartphone is good enough for the majority of users.

Agreed. I mean, hell... Polaroids were good enough for a large subset of people once upon a time. We're well beyond that!

This. I have no problem with the Apple enthusiasts diving into the details, but really, 98% of the people taking pictures with their smartphones are happy with iPhone 5 quality photos.

I maintain that the 6S remains all that the majority of users need. Anything added after that was just to sell phones. And I’m a big Apple fan.

False, the SE came out after the 6s

Took my 6S to Italy and the pictures largely were great. But I wish I had my 11 at the time, the low light photos alone would have been worth it.

You can live vicariously through Federico Viticci's iPhone 11 Pro photo series!

https://www.macstories.net/stories/iphone-11-pro-in-rome/

I have the SE 2016 (same camera as the 6s) and improved low light photos is the main feature I'm looking forward to when I replace it.

This is really cool. One of the best things about Rome was exploring at night. I took plenty of awesome pictures with the 6s, but this takes the cake. Thanks for sharing!

I took some pictures of some distant mountains with my iPhone 5 and I CRANKED the zoom to frame the mountains the way I wanted them. All the the resulting effects, plus the twilight of Alaskan summer, made intriguing pictures, like those shots in movies of the mountain the group is striving to reach. I'd say painterly but I'd flatter myself. I had a nicer camera available but not in hand.

That’s because they’re only looking at the images on their phone.

As an aside, I’m a professional photographer and my greatest wish would be that people shop around for a photographer on a desktop or at least a tablet. Incredibly tiny effective image sizes can hide a lot.

Just like Instagram has developed its own aesthetic, so has "shot on an iPhone". I fully agree with you, those images have the typical "surface blur" look that all tiny lens + backlit camera combinations produce.

I'd say it's even worse with video. My phone can make 4K videos, but watching them on a 4K TV is painful. They'd probably look better if you would downscale them to 1080p and then re-upscale them to 4K. They'll be a bit more blurry, yes, but they won't have this artificial flat look everywhere.

Memorizing high-frequency patterns and then reproducing them is one of the very few tasks that AI is actually good at. As such, one can use the raw HDR exposure data from a phone and the noisy raw image data from the sensor and produce naturally-looking high-res images from it. The way that it works is that the AI has memorized the small-scale details from millions of good photos taken with large lenses and proper equipment, and that way it can replace the noisy phone sensor data with its noise-free counterpart.

However, such an AI currently needs 10GB of GPU RAM and it runs for 2 seconds on my 1080 TI for a 4K image. So we'll need another 10x in mobile GPU performance before it becomes feasible to integrate such technology into camera apps. And then of course another 20x for live video processing.

> However, such an AI currently needs 10GB of GPU RAM and it runs for 2 seconds on my 1080 TI for a 4K image.

Which process is this? Got a github link? Are you talking about AI upressing? I haven't found it improves noise

Sadly, I don't have a publicly available link for this.

But no, this is not AI upscaling.

Instead, it is representing the input image in a feature space trained to store natural images and then reconstructs the output from that intermediary representation. If chosen well, the intermediary will store details relevant for high quality images, but discard those specific to low quality images, like noise.

It's called a convolutional autoencoder with bottleneck.

Of course you can also send phone photos to the cloud for cleanup.

Unless it were available for post-processing a RAW in the cloud over 5G? I could imagine an extra-premium version of Google Photos doing that...

Actually, that might be a pretty nice addition to Adobe's Lightroom cloud service. It already syncs between my desktop and my phone, so why not run some auto-improvements upon import.

But I believe for most users, it needs to be close to instant on the device. Nobody wants to wait 5 seconds to see if their selfie turned out OK.

I think these models will drop in requirements pretty dramatically - there is a lot of work in finding sparse subsets of NNs that actually do all the work for example, and embedded hardware is getting a big bump in neural network inference due to massively parallel FMA blocks being added to their DSPs.

Also remember that the majority of academic work is done based on benchmarks that don't include inference time or hardware requirements. As these things start becoming useful without these restrictions industry will start tweaking the cost functions that are being minimised to make it useful in the real world too.

There's also one big area where photos have noticeable artifacts even viewed on the phone: portrait mode. Almost all portrait mode photos will have areas that are blurred when they shouldn't be, or in focus when they shouldn't be. Part of an ear or piece of hair that's blurred out. Or the background between your fingers is in focus.

Interesting to note is that the portrait mode that comes with iOS and Android are radically different, built on completely different technology. So you will see different artifacts on each.

I've found the iPhone's portrait mode to be near unusable despite looking at first-glance like it might be a great shot. Hair and ears are always an issue (and not something you can avoid in portraits very often...) and another classic example is with a wine glass where the surrounds blur but the vision through the glass is sharp.

To be fair the examples in the article are trying to show what happens when you use this post-processing in situation it was not designed for, sometimes going to extremes to deliberately demonstrate failure modes, so it would be surprising if it didn't have weird artifacts. None of those photos are using it as intended.

I agree some HDR skies in your shots do look weird, but frankly that's probably just HDR. How do you find the phone's photos when you switch off the effects you don't want?

First I looked at your pictures and appreciating these are tests I thought they were nice. I liked the composition in pretty much all of them, thought they were generally pretty good pictures and couldn't see what was wrong with them.

Then I read your description and if you hadn't mentioned the skies look added in I wouldn't think twice about it – now I can't stop seeing it. It really does touch on this feeling I've had towards my own photos for some time, thinking something is off but I don't have the expertise to understand what. I think you may have just nailed it for me – thanks!

A couple of points. On this. Phone cameras are mainly used by people who don't intend to do any (or only very minor editing). Overall, both IOS and Android can produce great results even with mediocre lenses/sensors. Computational photography basically enables them to compensate for that and machine learned configuration of editing steps a pro would do manually actually produce nice results. Of course these results are not intended for further post processing and lot of the applied edits are lossy in the sense of compressing dynamic range, clipping blacks for contrasty look and feel, blurring out noise & detail not visible on a tiny screen, etc. But considering iphones have very high resolution screens these days that should still be fine for that and simple prints. But lets be real, most of this stuff gets produced for and consumed on phone screens by people who are not even close to being decent amateur photographers.

But for anyone who is used to dealing with proper gear (cameras & lenses) and having the freedom to make very subjective artistic choices for things like white balance, local contrast, tone mapping, noise reduction, etc. what a phone does is always a compromise. And also it's a 1 size fits all solution optimized for mass appeal, which usually means "oooooh blue sky, pretty saturated colors, super sharp/contrasty photos", which as any postprocessing enthusiast knows are all variations of compressing dynamic range (i.e. super lossy steps).

I fall in the last category but I do appreciate the best effort my cheap Nokia 7 Plus camera app makes. I occasionally use open camera to get raw dngs for editing in Darktable but they actually do a decent job with the jpgs given the noisy original. My shiny new Fuji X-T30 produces much nicer raw files with tons of dynamic range and detail that simply is not there on a mobile phone. All of the AI stuff that happens in a phone you can also do with a decent camera but it requires lots of specialized tools and tweaking (e.g. hugin for hdr stuff).

But the best camera is the one you have with you and for most of us that's usually a smart phone.

While I agree with you, it seems that most people around me just don’t see it. Even photos in portrait mode which don’t hold up on a phone sized screen are seen as good to many of my peers and I’ve become the weird one for asking people to take pictures without excessive processing included.

I wonder if in a few years we’ll look back at the photos from this time and think they aged poorly.

I mentioned this in another thread recently, it I think most people do notice when they see the difference. I take a lot of photos with a real camera and sync them all to my iCloud Library. When showing someone a photo, they almost always comment, even at very small sizes or before the full resolution version downloads, “Wow, you took that on your iPhone?!”

People see phone photos all the time so they’re used to it and normally fine with it, but given an opportunity, they will notice and appreciate quality photos.

I've considered switching to a dumphone and camera, but anything that doesn't take "live photos" isn't something I'm going to use, at this point. It's the last feature added to a computing device that made me go "holy crap, that's like actual magic!" when it was announced, and went on more-or-less perform as advertised.

live photos are the few seconds records ?

They capture a second or two of (lower-resolution) video frames before and after the photo itself, plus audio. The UI on the iphone is that you hold your finger on the photo and it plays the "live" version. Under the hood IIRC it's just a short video + still bundled together in a container. The effect is reminiscent of the magical photos in the Harry Potter films, and when looking back at photos of one's kids from years prior it really is magical. To take them, the UI is the same as taking any other photo, except that you must know to keep the camera steady a second or two before and after taking the photo itself (it'll still be captured just fine regardless, but the "live" part will be a blurry mess if the camera's moving) which is the real genius of it—no extra effort, really, and it's automatic, provided live photos are enabled, so you don't have to decide whether you want to take a video versus a photo, you just take the photo and get a tiny little video to go with it.

Are the other frames lower resolution? The short videos are quite low frame rate. On iPhone X at least you can choose a different frame to use as the “main” photo. They don’t seem to be lower resolution.

They were 1080p/24, last time I checked when I was adding support for them for PhotoStructure.

Picking a different frame doesn't impact visual quality on a mobile screen, but it was quite a noticeable IQ drop on a 4k/5k display.

Things might be different on newer iPhones and the latest iOS.

I don't have any specific knowledge of Apple's actual implementation, but I've always assumed it used the full-resolution still image as a keyframe (I-frame), and the video clip as B-frames before and afterwards, compensating for changes over time. Choosing a new key photo may be full resolution, but is likely to be degraded in sub-perceptual ways.

It does pull multiple full frames off the sensor in rapid exposure tho, and chooses whichever one it thinks best (blur, ML, whatever) for the initial key photo.

I think there's a (totally normal and full-res) photo and separate video, if you look at the actual files. I don't know how that works with the ability to choose a different still.

On my iPhone 7 Pro, yeah, they're much lower-res. Might be different on newer phones, I don't know.

You made me realize that I’ve had live photos disabled for a long time. Sad that I missed out on so many “magic” moments but I’m glad it’s on now!

Ok, I forgot the term. I did see that on my cousin's phone and it felt "special". In these days where nothing jaded me it was quite a surprise.

Windows phone had this feature first and I remember it was my go to feature to impress iphone and android users with, together with the low light ability which was comparable to the iphone X at the time of the iphone 5 (on lumia 920). I went from a lumia 950 to an iphone se (first gen) and the decrease in photo quality and featureset was dramatic.

Too bad windows phone 10 was such a slow and buggy mess, and microsoft had by that point pretty effectively managed to kill the app ecosystem by rebooting it three times.

I've always found 'proper' digital cameras are also in their own uncanny valley compared to negative film. Especially with older sensors, (and as an amateur with very little skill) it is so easy to blow out highlights with a digital camera in a way that looks completely unnatural. It is like its own poor emulation of film.

In a way the iPhone and recent HDR techniques on smartphones improve on this as it's now quite unusual for a phone to overexpose highlights. The tone mapping could be improved to lessen that 'fake HDR' effect you can see in your examples, but that doesn't seem like it would be too difficult.

>unusual for a phone to overexpose highlights

Although this is really quite recent. It used to almost be a hallmark of my iPhone that, unless I took care, things like clouds would be blown out. I was somewhat prepared for this as I mostly shot slides pre-digital for a long time after I didn't have easy access to a good darkroom any longer. And slides don't handle over-exposure well at all.

>>Get them on something bigger though? Look a little closer? Use decent resolution monitor? A reasonable sized print? They look bad.

So I don't have an iPhone, but I do currently have in my home office two pictures taken with my OnePlus 5T, both printed on A3 size paper and framed, hanging on my walls. They look absolutely perfectly sharp and I'm constantly impressed that something so good came out of a smartphone.

Here's one of the pictures itself: https://drive.google.com/open?id=1iVenWHP4TesmhTWAY2qMDQs21u...

Here's how it looks printed: https://drive.google.com/open?id=1h6P5ZbNjQ5FD9nkzdB_TnfsA4l...

I think Apple specifically targets TikTok, Instagram 'video' crowd with this device who earlier used Android or an older iPhone.

Videos shot with iPhone SE 2 is indistinguishable with iPhone 11, which by itself was class leading. So, naturally people are going to take notice of better quality videos on these platforms.

This strategy is likely going to be successful in US, what I'm doubtful about is with > $550 price tag whether iPhone SE 2 would make it a viable choice in developing markets; even though TikTok, Instagram are quite big there.

They all look fine to me on a 17" screen tbh. I couldn't give 2 shots about professional photography, and neither could 90% of the people viewing your Instagram.

That's pretty much the age we live in, the best Youtube videos, for example, have extremely bad videography and no audio processing from a professional PoV, but they get millions of views because they're funny to most people.

I mean, should I even try to mention the POTUS? (commence the downvotes heh)

I have no idea why people's standards have regressed, but they did.

I don't know what are you talking about. I am a regular person, who never spent time obsessing about photography, and all the pictures in the article on the left (I assume they took them with the phone) look very very good on my 40" 4k screen.

Author here.

If you're dealing with HDR artifacts, it sounds like you aren't shooting RAW. Shoot RAW, and you can process it however you want.

I installed another camera app a couple of months after getting the phone, which allows me to do that. The phone has essentially replaced my Fuji x100t for one project, mostly because of its wide angle lens.

AFAIA the default camera app doesn't allow you to shoot in RAW, it doesn't even give you the fundamental manual expsure controls, and that's part of my point - the defaults are designed to make it easy and make the photos look good on the phone, which is what 99% of users want.

I'm absolutely fine with that, BTW, but it's rather frustrating that the out of the box camera app doesn't include the ability to control the three fundamental exposure factors. Not even hidden behind an "advanced" setting or anything.

The system camera app isn't meant for people who want manual controls. It's the modern version of a Polaroid. The way it works is fundamentally incompatible with manual controls, and adding a separate manual mode would confuse the vast majority of people who have no interest in advanced photography.

I'd argue the other way - the camera on modern phones is a massive selling point, I know many people that choose their phone based on it. It therefore makes sense that the out of the box should include manual controls somewhere as an option.

I know Apple like to push the whole "shot on an iPhone" thing, and they want to make sure the default app is giving the "best" quality. So maybe that's the reason they don't include them?

And to take your analogy further - I have a Polaroid 195, it has full manual controls. Of course, it was aimed at pro photographers who wanted those controls, and sold at a premium.

And to prove my point, if you ask someone old enough to remember a Polaroid camera to describe it, they'll describe the SX-70. It was revolutionary for how it put photography into the hands of non-photographers. The 195 doesn't even warrant a footnote in history.

>larger aperture lenses on larger sensors

Nitpick: smart phone cameras have very wide apertures (f̶2̶.̶2̶ f1.8 in the case of the iPhone SE).

True, but since fstop is a ratio (length of lens over width of aperture) at this scale the more important metric in image quality is sensor size. You can compare the depth of field in a 35mm shot with a 4x5 image made at the same fstop to see the wildly different dof characteristics.

When it comes to the overall quality of the image, counting both the size of the sensor and the absolute diameter of the aperture is counting twice. The amount of light that hits the sensor, for a given field of view, is entirely determined by the absolute diameter of the aperture. Bigger sensors collect more light because you can use a bigger absolute aperture for a given f-stop and field of view. Or if you prefer, a larger absolute aperture collects more light for a given f-stop and field of view because you can use a bigger sensor.

Obviously, the format has implications for DoF, but those can be either positive or negative. When I shoot 4x5, I spend most of my time fiddling around with movements trying to get everything in focus :)

Exactly. The thing that really matters for light gathering is Etendue -- the product of the area of the source and the solid angle that the system's entrance pupil subtends as seen from the source -- which is conserved (in a lossless optical system).

[1] https://en.wikipedia.org/wiki/Etendue

There's more nuance than that, because sensors have a certain fixed amount of area, given a resolution, dedicated to control circuitry. Larger sensors make the control circuitry a smaller fraction of overall area.

The focal length of the lens and the aperture diameter is what controls depth of field. The sensor size doesn't actually matter; you can crop a photo later (which is like using a smaller sensor) and the depth of field doesn't change. The way it gets involved is that a certain focal length with a certain sensor size will result in objects in the photograph being a certain size. That means that a 35mm camera will use a 50mm "normal" lens, and a 4x5 camera will use a 150mm "normal" lens. Attach a 50mm lens to a 35mm camera, take a picture, replace the camera with a 150mm lens on a 4x5 camera, and objects in each photograph will be about the same size. But, the 150mm lens will obviously produce more blurring away from the focal plane; that's just what longer lenses do.

The iPhone's telephoto lens has a 6mm focal length, and so tends to produce an image with more in focus than a lens that's 25x longer.

(Why do phones use a 6mm lens and not a 150mm lens? Because they don't have 150mm to spare between the lens and the sensor.)

To add to what you said, the depth of field calculation relies on the circle-of-confusion that you pick (which is based on the viewing distance and the size of final reproduction), and is therefore also closely dependent on the resolving power of the sensor, and thus pixel density. In addition DoF also depends on the focusing distance.

>(Why do phones use a 6mm lens and not a 150mm lens? Because they don't have 150mm to spare between the lens and the sensor.)

That is not true for today's lens formulas. You're probably thinking of old singlet designs from centuries back. For e.g. reverse-telephoto lens designs allows you to make lens much shorter than their focal length. Its all about getting the final magnification by positioning positive and negative lens elements, and ofcource other corrective lenses appropriately in the lens formula. This is why you can use "tele-converters" on lenses to increase their magnification (and thus their effective focal length)

It's f1.8 for the current iPhone SE model. Front camera has f2.2 though.

Ah yes, thanks.

Modern phones use heavy post-processing to compensate for sensor size. Works okay.

seeing as how most of the photos taken are meant to be consumed on the phone anyway, it's like bagging a mirror-less camera because you wanted a proper banner sized photo

> Get them on something bigger though? Look a little closer? Use decent resolution monitor? A reasonable sized print?

Most people do none of these things though. Or do them so rarely that it’s all but pointless.

Something that I have wondered about is the photo quality of film photo enlargements vs. digital (which I guess is just zooming).

A film is continuous right, so no amount of digital encoding would truly encode a strip of film. Of course there is a point after which it doesn't matter, but I am more interested in the technical look and feel of film vs. digital. A few years ago I looked at a film enlargement to A4 in a scrapbook and I have to say it had a different feeling about it. There were also funny artifacts of the kind that make superstitious people believe in invisible spheres that ehm are not invisible on film...?

If you contrast this with a CD recording, the bit rate is limited even if it's an audio CD due to the way they were recorded. Vinyl is supposed to be limited to 48 bits, but perhaps you can engrave in an analog way?

Film is certainly not continuous—its spatial resolution is limited by the size of the individual light sensitive crystals that capture the image. These appear as the grain structure in enlargements.

Does this hold for all types of film? For example, how small can the grain structure be?

It can be quite small, but I believe it largely depends on the ISO rating of the film. Larger grain = more light collected.

I shoot wetplate collodion sometimes and that has an effective ISO of under 2 and the detail is astounding - especially for a process that’s 150 years old.

Interesting question! Actually all photographic imaging systems run into physical limits set by the quantized nature of light. So even if you could shrink things down to one molecule, you’d have other problems first, principally an effect called “shot noise.”

Essentially, a film grain or a cell on a CCD sensor is like a “bin” that collects and counts photons. As you make the area of the bins smaller, there are fewer photons collected in a given image, and statistical fluctuations from sampling become more relevant. You get more “noise” in the final image.

See https://en.wikipedia.org/wiki/Shot_noise

With the exception that holographic film images the diffraction pattern produced by two interfering laser beams, so typical grain size there has to be on the order of tens of nanometers. It can certainly be done, just is pointless for normal film =)

Those physical limits are surely though at a much finer resolution that common high end digital cameras?

I understand that you are saying that you see the effects even if you are not a the "molecule" level as you say, but I would think that the hard theoretic limit of photons should be at a finer granularity than high end digital cameras.

Vinyl is limited to around 12 bits. It has a dynamic range of around 70dB. Vinyl sounds "nice" because recording and playback add distortion and noise at the low end, which gives bass and drums more weight, while the top end is smooth and extends higher than CD - which sounds like "air" because any intermodulation distortion in the amp and speakers will fold it back to audibility, even though nominally it's ultrasonic.

Chemical film creates distortions of its own. Colour film often pushes the reds to make skin tones, lipstick, and sunsets warmer. You can choose how much grain you want and B&W is often shot on fast high-grain film to give it a gritty textured look. Landscapes need slow micro-grain film to look as lush and detailed as possible.

And so on. The point is chemical film isn't any more real than a digital CCD.

Digital consumer displays all have colour distortions and dynamic range limitations, and so does chemical processing. Even if you apply no deliberate processing to an image, by the time it ends up on paper or a web page it usually looks quite different to the scene you'd see with your eyes.

So yes - the "feel" is different. And that's a creative choice - or it can be, for the professionals. Polaroids look very different to medium format.

Amateur phonecam users get one-size-fits-all enhancement whether they want it or not, in addition to some basic editing options. If they didn't their lifestyle shots on IG would all look lacklustre, and they wouldn't be happy about that.

Assuming that you have the technical ability, how many pixels do you need at how many colours to completely simulate film? I think that should refine my question.

Your common high end DSLR definitely outresolves 35mm film nowadays, especially at higher ISOs. It’s not a 1:1 comparison, but film does have grain and that grain size is the “limit” of the data.

That said medium format film and larger would still be hard to beat.

As far as tonality, a modern sensor should also have greater dynamic range than almost all film. They do behave differently though - highlights are harder to overexposed on film but dark areas are easier to underexpose, compared to digital.

In theory, if you had the money and incentive, you should be able to develop films that are much higher resolution than current digital hardware? Like the large format films you refer to.