A Billion Pixels a Second: Inside Apple's iPhone 16 Camera Labs
cnet.com
cnet.com
Doing 120 fps video at 4K so that any chosen frame looks amazing without artifacts is really quite an achievement.
The microphones were actually more interesting to me, that you can get lavalier performance from the tiny mics in the phone that are physically far from the person being recorded is seriously clever.
Getting this to work some of the time is already an achievement but I think people underestimate how much work goes into making it work across all different scenarios.
I rarely carry it because my cellphone is usually good enough but it’s good for travel and for lower light areas where cellphones struggle. It won’t fit in my pants pocket but it fits in my jacket fine.
When I first got it, I didn’t know when I should carry it and when I shouldn’t but now I know and don’t even bat an eye about it.
I started using iPhone Pros a few generations ago for the better camera, and it’s been great for random snapshots. The adage about the best camera is the one in your pocket is true.
But, I still have a mirrorless (Olympus) and some vintage 35mm cameras. The mirrorless is great for photos that I want to look very good (portraits, landscapes, mixed lighting, etc). The film cameras are a pleasure to use, but I’ll freely admit that using them is a bit like preferring a nice Seiko to a smart watch or basic digital Casio.
Anyways, with the prices of used cameras being so low, you can get a really nice, compact mirrorless setup for $500 or less. And fixed lens 35mm rangefinders from the 70s can be had for $200 or so. I won’t claim it’s a cheap hobby, but it’s no worse than golf or cycling.
I was talking to a friend who is a very good photographer last fall who used to always have a camera on him and he says that, like myself, he only uses his cameras for specific purposes these days. Mostly his smartphone is fine.
Nobody is claiming a world record from Apple …
>Doing 120 fps video at 4K so that any chosen frame looks amazing without artifacts is really quite an achievement.
Apple silicon is extremely impressive. In this case however dedicated cameras can still do more, which I felt was a relevant observation.
what trade offs do you accept for that mirrorless camera to do all of the same functionality while still taking an above decent looking image for the vast majority of its users without all of those fancy lenses fixed or interchangeable? As the saying goes, the best came you have is the one you have on you. Only snooty photo/video types care about your comment. Most videos used by phone people are only ever viewed on that device or other similar devices of others viewing it on whatever app they shared it. To even consider them comparable is just not an honest take in the slightest. I say this as someone with several DSLRs and a couple of cinema camera bodies in the next room.
Show them one shot from a cheap modern mirrorless and the smartphone is over for them. You don’t even need a high end body. And yes you can do a lot on body and yes they integrate with the phone as well. So that doesn’t kill any of those use cases. They augment the situation.
Currently using a Nikon Z50ii + 18-140mm zoom. I don’t change the lens for ref. I shoot JPEG and edit on my phone. Most of the stuff goes on facebook or gets looked at on my Mac. I don’t use LR or shoot raw. The thing literally tethers to my phone.
IME this only lasts up to the point they have to lug that brick with them.
Don't get me wrong, I'm firmly in the "my 8 yo m4/3 kit wipes the floor with any phone it's not even funny". Which is all the more the case for a modern FF kit. But even my tiny Olympus is huge compared to my iPhone. Only my winter coat with fat pockets can fit it. It otherwise needs an actual bag (might work with some women's purses, I don't usually carry anything at all).
Guess which one I have 100% of the time with me? I've found that the iPhone in hand while outside takes 100% better pictures than the Olympus kit in the drawer at home.
It's always a question of compromise. Before my Olympus gear, I used to love my FF Canon gear. Built like tanks. Until it got old carrying all that junk around, and it started gathering dust somewhere. Now I'm happy with m4/3 when I'm relatively serious about my photography, and I'm happy enough with what my iPhone produces when I only feel like having a pair of jeans and a shirt on.
Would a new Sony something-or-other wipe the floor with the Olympus? Possibly. Enough for me to lug it around? No way.
The new stuff doesn't weigh a lot. It's not FF bricks any more. Nikon stuff is mostly magnesium and plastic.
But, be that is it may, take a look around you. How many people do you see on any given day taking pictures with actual cameras, whatever the format? I live in Paris, a city flooded with tourists, whom you'd expect to be more likely to put up with carrying a somewhat inconvenient camera in exchange for better pictures. I can count on the fingers of one hand the people I've seen taking pictures with actual cameras in the last few months. And I walk or bike to/from work, along one of the most picturesque parts of the city. However, you can see people taking pictures with their phones all day, everyday.
---
[0] Some of the bodies are actually surprisingly small. A sony A7C2 is "only" twice as thick as my Pen-F, the other dimensions being similar. But the body itself is useless without a lens. And while wide-angle offerings seem somewhat similar (though there's no equivalent to the 8-25/4), the tele end is much larger, even comparing a variable aperture to a fixed f/4 offering from Olympus. And primes are an absolute joke if size matters.
I’ve been considering a m43 for that reason.
Is that the case? I think it takes one to be a photography enthusiast/nerd to care enough to carry a camera with them. Most people I meet just want a phone "with a good camera". Phone cameras in this sense do not compete with real cameras, it is pretty obvious one can get a better camera if you remove the constraints that a smartphone's physical body brings. I think it is pretty illogical for one to think that smartphones have as good cameras as a good real camera, having all these space constrains and whatnot, so I doubt people really believe that the phone cameras are the status quo. Some people may be "disappointed" by their smartphone's photos (because they are able to objectively determine it) when they travel but most won't get a real camera because essentially they do not care that much as to buy/carry an additional object (or they do not have the money).
For me, I am just happy that my phone now takes photos "that look good" even if I have no idea or skill on how to take good photos really, though I can appreciate if there is somebody with me with a real camera who does. And even if I had a real camera, I doubt I have the skill to take something better than the software-modified one I get from my phone.
This was my one https://imgur.com/a/rPCJwcR (thanks Google!)
Lots of people have this day and decide they'll keep that phone a couple more years and buy a camera with the upgrade cycle money. I think when I am travelling around 25% of people have mirrorless or DLSRs now compared to 5% a couple of years ago. A 10+ year old Nikon D3100 with kit lens is still a better camera then a 2024 iPhone Pro and doesn't cost a lot of money.
Across Europe at least people have a lot of distrust of their phones as well. I suspect some of that is driving adoption. I even see film cameras regularly now as well (!).
Wow this is really bad. Was it zoomed? If this is the case, it is interesting that it is still 4k resolution and companies prioritize "4k" even with horrible processing or whatever artifacts are these, rather than reducing resolution eg when zooming. Similar with low light, where binning the pixels could increase light sensitivity.
If it is not zoomed in, it is much worse.
I have an iPhone 15 Pro now which is ok if you are careful but has lots of other wierdness instead.
Most people take pictures on their phone and look at pictures on their phone only. They won't return home to realize the picture is trash because their viewing medium is only capable of displaying a little more than a million pixels.
What is not 'real' about an iPhone camera's lens? They're very sophisticated. (See e.g. here https://news.ycombinator.com/item?id=30557578)
https://m.dpreview.com/articles/2128193923/a-distortion-of-t...
The average person isn't at all worried about photo quality and is happy with their 3-4 year old phone.
At best, they're smiling and enduring your diatribes about camera quality and then going back to their iPhone Instagram snaps.
This strikes me as just a reflection of the ad campaigns. Apple promotes “better cameras” for every new iPhone, almost exclusively in their ads, so it’s not surprising that’s what people would say. With every new phone being just an incremental upgrade, hyping up the camera is the only way to get people to drop a $1000 on a new one. Most of these 38% won’t be able to tell the difference between a phone pic taken 5 or more years ago.
But that actually seems to be a very reasoned response to consumers asking themselves "Why should I upgrade my phone?" And over at least some timescale--maybe not every model--a better camera actually seems like a pretty reasonable answer for people who care.
Older phones gets slower, either because of the dying battery (that could be replaced, but many of these people are on subsidized phones so upgrading is baked into their plan), or software bloat slowly creeping on.
When camera boot up, shutter lag or post processing starts to take one sec or two, moving to a newer phone is the obvious option.
That is an arbitrary statement, that came from your biases and not observation. Maybe in best of lighting some 5-year-old photos maybe as good as latest flagship.
If digital cameras were so good they would not be disappearing.
I'd even go as far as claiming that this processing "taints" images in a very specific manner, and that images taken on today's smartphones will age very badly. I am no computational photography expert, but even I can tell that a photo comes from an iPhone: the bokeh looks uncanny (unnatural, one-dimensional, often with visible contours), the white balance biases towards yellowish tones, the amount of selective sharpening/micro-contrasting is uneven.
Other manufacturers (Samsung, Google) have comparable although distinct biases that may comfort your ideals of aestheticism or not, the point is that it's not feasible to get neutral takes out of those devices, because they are not technically able to do so. Computational photography is what they are and what defines them: true you can pull RAWs out of them (which are mostly only so in name), but that serves to reveal how-much is processed out of it.
Anyhow, I am not comfortable with the status-quo, and that drove me to buy a mirrorless camera a couple years ago: I don't even care whether it might take worse photos than my phone (it doesn't) or be harder to handle (quite the opposite!), I just want to opt-out of Apple's/Google's opinionated ride on modern photography, and take back the ability to make photos that look natural although somewhat imperfect.
Interestingly, I may be part of a small but growing niche: https://www.lux.camera/introducing-process-zero-for-iphone/
Yeah, at the last step there is a default filter (that can be changed) and you might not like that, but you can take photos in RAW and change them as you wish after the fact. Sure, the RAW photos are not as 'raw' as there is plenty of processing behind them, but that's also the case with most modern cameras, as far as I know. E.g. is it a lie to use adjacent sensors' data if we know that there is some vibration of the whole sensor? Then we might also call the first photo of a black hole "fake".
Unless you can back it up by showing me the actual code that's running on those devices, I won't believe you. Keeping an eye on the Halide (an alternative photo app) blog over the years convinced me otherwise.
Theory 1: People treasure memories and have been let down by pictures tnat don’t age well from cheap film cameras, then early digital cameras, then phone cameras. Phones have gotten good enough to replace separate devices for most people, but the cameras are still not as versatile. People will spend money for incremental improvements because the payoff will last decades and benefit their children. Apple invests heavily in camera tech to satisfy this need, and in marketing to communicate the benefits.
Theory 2: sheeple buy whatever marketing tells them to, those rubes don’t see any real benefit.
I just don’t see how anyone who enjoys even casual photography could go for theory 2.
Newer tech of course looks better, and people can definitely tell the difference when comparing them side-by-side. But there’s also an element of the reality distortion field when it comes to convincing people that they need to upgrade from a two year old phone to the latest release just because of the camera upgrades.
The 808, while a great high resolution sensor, processed stills at 5MP after pixel binning, compared to the 48/24/12MP of the iPhones. It did have a non-binned mode but again for stills.
https://en.wikipedia.org/wiki/Nokia_808_PureView
The majority of the post is actually about video and the comparison is a lot more dramatic there.
A 4k frame from a modern phone ,like the iPhone 16 pro, is ~8MP.
The 808 could “only” do 1080p at 30fps (62M pixels a second). Compare that to 4k at 120fps (995M pixels a second). That’s a far cry from 1Bn per second that you’re claiming. Impressive for the time though.
There’s a significant difference here.
The sensor on the 808 was amazing. The real meat of the tech here is the image processing pipeline that can process everything off the sensor for further processing.
The binning and other features were enabled (poorly or not isn’t part of the discussion) by a chip that could do about 1 billion pixels per second; that’s the issue; Apple doing it 12 years later and hyping it is what I was commenting on.
The processing you’re talking about could not handle 1Bn pixels a second except for maybe the very frontend of the pipeline. It could not handle it beyond that, which is where the meat of any ISP is.
It’s easy to dismiss the technology if you allow yourself to dismiss the majority of what needs to be done past the sensor read out.
Can't see what your point is.
This is really about hiding their entire SIRI recording your conversations without your permission.
The cult of apple kinda says everything wrong about the valley to me. This isn't magicians at work with secret spells, this is just cutting edge tech that is at the point of being commodity. Making something appealing to idiots has drawbacks, less repairability, less durability etc and now even apple is slowly reverting even this under govt changes...
And I bet if I drop my phone, it would be a lot easier to drive to one of the five Apple Stores in the metro area to get it repaired than it would be for you to get your camera repaired.
That’s not even considering the fact that it could service a drop better than your camera and is water resistant
Definitely need citations for this.
https://www.imaging-resource.com/news/2013/05/23/camera-meet...
Can you imagine any modern phone not surviving a 3 foot drop?
If you do want to record those kinds of sounds, the term of art is "field recording" and there's tons of good information available on how to do it well.
For now my approach is to use a external microphone with a longer cable and a notebook, and this works, but it would be nice to make this work with the iPhone.
What do you use for storage?
The most recent idea was trying to use it for sounds but I had the mentioned issue. Does the lavalier works for environment sounds or you only use for voice?
e.g.: https://www.mi.com/global/product/xiaomi-11t-pro-120w-xiaomi...
Because when you pick it up, your hand is covering up the middle of the phone. However the top quarter is unobstructed.
> And why do the cameras keep moving around?
Easy way to tell models apart.
is that the reason or just a convenient side effect? the last two models had to move the lenses around slightly so they could capture whatever they called their stereo AR/VR type acquisition so the pupil distances worked correctly. after that, there's only so many ways to arrange 2 or 3 lenses.
For instance Xiaomi is a lot more serious about the photo part (and they also sell a lot less as well, this model was only noticed in photography circles)
Low end model https://www.dpreview.com/products/sony/compacts/sony_dscqx10
High end model https://www.dpreview.com/reviews/sony-cybershot-dsc-qx100
Sony has consistently been innovating in this space but has always had basically insignificant market share. People keep complaining about mundane updates and AI crap but they won't buy phones from the one manufacturer that bucks this trend. MKBHD has a couple of videos about it (regarding the Xperia 1vi iirc).
My Sony phone has a 4k 120hz oled screen, notification led, headphone jack, fully manual camera mode, no bloatware or ai crap, great battery life (never had to charge my phone during the day). They had the battery save features years before other manufacturers. They've been doing this consistently for years. I don't know how they keep doing it, but I'm glad they do.
Most users would want to use the screen of their smartphone as a viewfinder. That either means using both hands or requiring a way to attach the thing to the smartphone.
Also, to do the image processing, that cylinder would need to have most of the processing power of the iPhone and, thus, a fairly large battery.
This is one of those things that doesn't seem like it should matter, but it does. If the lens is mounted in the exact center of the body, the images come out looking unbalanced. To produce balanced images, you have to offset the lens. Even very expensive pro mirrorless bodies are offset; that is, if you look directly down the center of the lens, you'll notice there is more camera body sticking out on one side than the other.
This is called the chirality of the optical path and it is surprisingly difficult to predict analytically. Companies will typically design the optical path, prototype it, and mount it on a jig to precisely measure the chirality. From this, they design the body with the proper offset.
Chirality is more noticeable the smaller the sensor and the shorter the lens. So on smart phones, which put tiny sensors behind wide-angle lenses, they have to get the offset just right. This explains why the lenses are in slightly different places on the body every time Apple updates their cameras.
It was also posted by a person with a huge karma. I want to understand what happened.
Every mirrorless body has a center marking for the middle of the sensor, so the camera can be mounted exactly centered on a tripod. It's actually important for photos to be exactly on axis if you want to do panoramas or stitching.
The only reason the body is not exactly symmetrical is engineering and ergonomics. Many point and shoots of the past in fact had the lens exactly in the center.
And "chirality of the optical path" is not anything related to this, in fact the term is not usable in this context at all.
People's face look perfect on my mediocre Android though.
I'm never going to buy or use an iPhone. Even the questionable advantage which was supposed to be the iPhone's camera is fake.
Due to the general flatness of the lens there is a lot of distortion around the edges which is digitally removed after the photo is taken. This isn't 100% perfect and causes some rather uncanny looks in some of the photos. You can use this for artistic effect but it looks crap mostly. Generally if you're using a proper camera there's a big chunk of glass in front of it so the main part of a portrait is well outside the distorted edges of the frame so it's not noticeable. Even new cameras use minimal lens corrections in body as well to eliminate this.
As for the white balance, Apple never seem to get this right. The colours are always slightly too orange / warm and vivid and never quite match reality in experience. You can crank the warmth down a bit after in photos.app to kill some of it.
Oversharpening - everything is too sharp. This makes the image pop out but nothing more. It's a terrible curse on smartphones. Not much you can do about it. Even shooting ProRAW on mine is oversharpened.
If you turn the True Tone feature off in Display/Settings it looks a bit better as well. That seems to completely mung viewing any photos later, giving them sometimes an over-blue tone.
Urgh all this is why I bought a mirrorless. Smartphones are really not very good. Even good ones (mine is a 15 Pro). Mine gets mostly used to take photos of an AirBnB when I leave it now or where the car got parked.