A UI Experiment with the iPhone X’s Front-Facing Camera
fastcodesign.com
fastcodesign.com
The said effect and the head tracking principle was already used in the New Nintendo 3DS consoles that was sold millions worldwide. Not only that - 3DS' top screen has a true "parallax barrier" screen, means it displays a 3D picture with depth and all.
Also visually the scene does move accordingly, mostly because the top display is tecnhically two displays in one and they render the scene from 2 different angles.
The only thing in common is that both have head tracking, that's all.
And also didn't the Amazon Fire Phone have head tracking?
Also found this, done on an iPhone in 2011: https://www.youtube.com/watch?v=7saRtCUgbNI
Or this table using a Kinect in 2012: https://www.youtube.com/watch?v=2CbiOikirrg
Both have their advantages, but they're not the same thing.
That being said, the Nintendo DSi (the predecessor to the 3DS) did have a trompe l'oeil 3D game, called "Looksley's Line Up". It had a ton of charm, but the DSi's camera wasn't powerful enough to support the feature well. For whatever reason (price), Nintendo sticks with low quality cameras. Still, I've always hoped that Looksley's Line Up will eventually make its way to another system.
Or could it support, for instance, rotating the handheld?
I worked this into an iOS display demo in ~2009 using their basic face tracking that made for like zombie face apps and shit. The RealSense is a MUCH more solid experience, but this is not new at all.
[1] https://www.youtube.com/watch?v=Jd3-eiid-Uw&t=164s
[2] https://tastenkunst.github.io/brfv4_javascript_examples/
I suspect they look much better in demo videos like this because there's no depth information in the objects surrounding the phone either -- the person's hand, the floor/desk behind them, etc. This makes it much easier for one's brain to map "pseudo-depth" onto the entire video.
In real life, you have depth perception for everything except the images on your phone's screen, making a demo like this look "flat" even at its most convincing. Real 3D requires two distinct visual inputs; this is a gimmick.
It was a lovely little taste of what might have been.. although we're still waiting!
And for expert UIs, one doesn't have to emulate reality. So you might exaggerate parallax, be non-linear or discontinuous, etc. There might be a much larger vocabulary available, than merely "look through the small portal" and "make secondary content conditional on viewing angle". Maybe.
Why might you want this? Consider your primary "desktop" environment shifting into HMDs. But you still need to mix in screen time, to give your eyes a break. It would hypothetically be nice to have similar UI idioms on screen. Especially as your tooling becomes less and less like current 2D ones.
[1] old similar demo (broken on FF) http://auduno.github.com/headtrackr/examples/targets.html video: https://vimeo.com/44049736 [2] current tracking-only demo https://trackingjs.com/examples/face_camera.html [3] eye/gaze tracking (demos at bottom; but for me, they require good lighting, and beard removal) https://webgazer.cs.brown.edu/ using https://www.auduno.com/clmtrackr/examples/clm_video.html [4] opencv.js https://docs.opencv.org/master/d5/d10/tutorial_js_root.html [5] color tracking https://trackingjs.com/examples/color_camera.html
But a lot of people read only the comments (or the comments before the article) so your comment does add value
Is it a good HN comment or not? :D
Admittedly the article did talk enough about the other thing that it’d be pretty hard to miss if you read the whole thing, but I think we can forgive it still. Perhaps the parent to your comment opened the page and saw the video but didn’t bother to read the article. I think that’s fine too.
I always read comments first and in fact quite often I read the comments only. As long as we don’t derail the discussion completely by starting a top-level comment about how we perceive something based on the title alone I think it’s fine. That includes it being fine to mention things that you are reminded of by the title as long as you phrase it correctly. In my opinion of course.
I think what would be even neater is to use the head tracking from the notch to compute a transformation of the back-side camera to simulate that you can see through the iPhone. Of course this wouldn't work when the iPhone is too close, or if the angle is too wide, but in other cases, this would make AR really good.
This isn't true. What we got for it is Face ID and the consequential removal of the home button which then allowed for the frameless design.
Not saying it's ideal. But the author has missed the point slightly.
Let's tame the article title a bit huh?
>Norrby is far from the first person to play with trompe-l’oeil (also called “parallax”) in the modern age. Johnny Lee, now at Google working on augmented reality, first made his mark by turning a stock television and a Nintendo Wiimote into a wild 3D game.
annoying bait title.
I do think the notch is bad UI but it can be worked around. I would have more easily accepted a narrower band of black across the top and bottom but Apple isn't losing sales because the rest of the phone's tech is that good.
I do look forward to the day when the components are barely distinguishable from the display.
Personally I find Samsung's current designs prettier than the iPhone X, though not by much - thing's gorgeous.
You’ll probably see more coming soon. Ours was in review by Apple for over a month.
?! That's crazy talk - I've not seen a review process for a new app take more than a few days in recent memory.
I guess using the Face ID API marked your app for more thorough review?
At least you finally got it out!
Yes, apple is harvesting your high-quality biometrics using this. 5 years down the line someone will leak the data and your face will end up on some weird porno series.
The biometrics are stored on each iPhone, in a secure enclave with severely limited access, and they're only useful for identifying (slowly) someone at close range.
So, yes, theoretically in the future someone could hack all iPhones everywhere using one of the most sophisticated attacks ever, and collect data that might conceivably be useful for high resolution cameras to identify faces at close range. Maybe. Although Face ID uses infrared, so would the high resolution cameras need to have that, too?
Anyway, that's why I felt it was a stupid comment.
Considering the fact that this exists because a) Apple was unable to come up with any real innovation to sell the new product and b) people seem to be really to lazy to type a number and rather give up that information.
Data Minimization, Data Avoidance are a thing.
> "using one of the most sophisticated attacks ever"
As if we haven't heard that speech before...
The more likely attack vector is simply capturing scenes through the front camera. That would give you, most of the time, an image of the face of the user, not just depth mapping information of questionable value.
But really, nearly everyone shares images of themselves online, so even that's of dubious value.
I'd challenge you to find any security expert who agrees that Face ID as Apple implements it can realistically result in useful biometric data being leaked to a hostile party. Apple supplies whitepapers documenting the secure enclave, I imagine there's one for Face ID.
Now, the question of whether Face ID is secure enough for any given user's needs as a local authentication is a perfectly valid question, and clearly for some users the answer is no. But, again, it's optional, and that's not at all the threat under discussion.
You make it look like it's easy for the normal Apple user to switch to Android. In fact it's quite the opposite and the whole situation is even worse because most of them won't even be aware of the dangers. The major reasons for people I know to chose an IPhone over an Android is the "ease of use" (resulting from the fact that their first smart phone was an IPhone already and they never tried anything different) and because they are "so confused with all the options/apps/general possibilities on Android". Those are the people who need to be especially protected. They are caged within a locked environment of a single US company. This alone should make you think.
> But really, nearly everyone shares images of themselves online, so even that's of dubious value.
This sentence together with this high tech approach you demonstrate on the rest of the comment is mind-boggling. As it's the most common approach of companies/individuals to abolish digital privacy all together. The old version of it was "I'm not afraid of X because I have nothing to hide". Horrifying, but now I understand where your attitude comes from.
Being born in a oppressive state, this is where I would actually use the word "stupid".
> I'd challenge you to find any security expert who agrees that Face ID as Apple implements it can realistically...
As I've wrote above. It may be that FaceID is not a big deal right now. We don't know it for sure since it's all locked down but we assume it. There is however still the ARKit and all those APIs using those depth/facial mapping capabilities. Those becoming the new standard for popular apps is just a matter of time and since you've already given rights to use the camera, those features will be (or are already?) a nice extra. So you see...we don't even need to reach out to possible changes from the paranoid government governing Apple and their data under and awaiting some patch to allow the access to FaceID data. It's far more accessible.
I wonder, would you allow your phone to take a drop of blood for authentication or where does your privacy actually start?
No, I'm saying Face ID is not mandatory on an iPhone X. You can use a passcode.
Nor am I saying privacy should be sacrificed on the altar of technology. I do my best to stay away from Google, and I try not to let Facebook know any more about me than necessary (and every day I contemplate ditching it, but there are a few important reasons to stick around).
There are plenty of ways to do biometric security wrong from a privacy standpoint. I trust Apple to do an earnest job of doing it right, because they have positive incentives to work for their user base rather than being a data collection/ad selling company.
And if biometrics aren't where you wish to place your faith, you can simply not enable the feature.
FYI: you can have an Android phone complete without a single google app or the google app store. LineageOS is the most popular alternative OS. There also other stores you can put on your phone. Like F-Droid, which hosts open source apps.
Biometrics, yes. AR-based depth mapping, no.
https://developer.apple.com/app-store/review/guidelines/#dat...
5.1.2 Data Use and Sharing
(i) You may not attempt, facilitate, or encourage others to identify anonymous users or reconstruct user profiles based on data collected from depth and/or facial mapping tools (e.g. ARKit, Camera APIs, or Photo APIs)
Let's wrap this up here.
- A user issues his fears based on a technology that is the topic here
- he gets downvoted into oblivion but no comment follows
- I trigger a comment by stating the obvious behavior prevalent on every single article posted here that may be or even is critical towards Apple
- you declare the users comment stupid based on your assumption that a single software use of the general feature may not be misused. Even though you can't know that because we are talking about a closed system and the APIs allow that without a possibility to opt-out (if you have already granted general camera permissions).
- you further state that users don't need privacy either way because they gave it all away. Which is actually the only really stupid statement in this discussion here
- after all that you even go so far damning another US company based on actually nothing. A company that allowed the world to develop their own open source operating system and app world after you've done everything to protect a company that provides you with a system you actually know only what they allow you to know about.
macintux, I couldn't have wished for more to demonstrate what is wrong here. There is a quasi cult behavior in the Apple fan base turning people into marketing machines ready to drop everything to protect the brand while condemning everybody else. You owe the guy an upvote. I don't care.
Then I am glad to have been of service.
Using face positioning vs the accelerometer is interesting, but without seeing it in action, is it that much more compelling?
These broad statements irritate me. I don't hate it - I actually think it integrates super well with the UI and makes the X look unique among the sea of identically looking glass slabs. And I don't even own an iphone.
No one I know cares very much, they think it's neat, they all wonder at the manufacturing of the oled screen, but like you, they like the extra screen real estate.
I feel there must be some sort of 'too cool for skool' bubble that journalists live in.
https://www.theverge.com/2018/3/4/17077458/iphone-design-clo...
That’s because the only purpose of the notch is to differentiate Apple phones from every other smart phone. It’s all about branding - https://medium.com/swlh/the-iphone-x-notch-is-all-about-bran...
It’s there to have a place to house the various front facing sensors/emitters while sharing verticals space with usable screen area.
Once you decide to do that you have to answer the question of how to deal with the border line between the two.
Obviously, you don’t want something ugly.
(1) hide the border. This means anything and everything displayed on-screen next to the notch has to be on a black background.
(2) create a non-ugly border and let it show.
Perhaps the decision to go with (2) was influenced by branding considerations but it’s a secondary decision, not the “only purpose. And (1) isn’t exactly a great option, regardless.
No, it's not. It's either for differentiation or to be able to (like the Essential PH-1 before it) claim an “edge to edge” screen (even though in Apple's case this excludes a big part of the top edge), but it's not to make room for the “technology”, which obviously would fit just fine with a rectangular screen and a top bezel.
Your argument that "the technology [must not] take up more room than needed" is simply false. I can point to plenty of Apple products where the technology was fit into a bigger box than needed so that the box didn't look misshapen.
Maybe this is one of those things that seems awkward until you make some direct comparisons with the alternatives.
Somehow we’ve had sensing instruments facing us for years now without notches. And yet, here we are. It’s funny how Apple can get everybody talking about their stuff isn’t it?
Surely that's survivorship bias? I dislike the notch so much that I choose to not buy an iPhone X despite being due for an upgrade.
In my experiments with those, the main drawback was ergonomics (device of varying size depending on the sensor, connected to a computer via at least one cable as you moved it around the thing you were scanning).
On a phone or other standalone/self-contained bit of hardware, I'd imagine it makes things easier.
Well, if you’ve used a Kinect, then you’ve used the iPhone’s tech, because Apple bought PrimeSense, which developed the initial Kinect for Microsoft.
My guess is that the only reason this demo is relying on the iPhone X is that Apple provides a facetracking API, so there’s no need to write any code or use external libraries to do eyetracking with a normal front facing camera.