Apple announces new accessibility features, including eye tracking
apple.com
apple.com
This excites me so, so much! I can't really use my phone as a passenger in a car without getting motion sick after 1-2 minutes. This seems like it might be a promising thing to try.
I don't sit in enough EVs to tell if they're the same.
In the racing/track world, any time spent off the accelrator or brake is literally leaving time on the table.
The idea being; you are going at 10/10 until the exact point you know you need to go 4/10 at an apex and apply full braking. Then getting right back up to 10/10.
Similarly, I notice that i do a lot of this "sine wave" driving (IE , gas, brake, gas brake) when I'm heavily loaded with passengers or on a curvy road where I don't quite know the ratios. So I'm often not expecting to need to brake so heavily but, for safeety of course, need to.
Re: your curvy road, I would say at that point you might just be driving too fast for the load in your vehicle. When passengers are onboard, a smooth ride will beat everything 99% of the time. If I have to suddenly break for a sharp corner, then I apologize and slow down so it doesn't happen again.
Does make me wonder what the difference is there.
If I’m riding somewhere to go get dinner, I have to sit in the front passenger seat. After dinner with a full belly? Throw me in the back and I’ll be fine.
Unsure if it actually works though, my personal test results are mixed.
I used to get bad nausea from aggressive physic-y VR games. But I heard people claim it can be grinded through. So I did that, and they were right. I can do VR games without needed to vomit although it’s still uncomfortable.
However… I am now much more sensitive to non VR motion sickness. :|
I was expecting to see much better image descriptions, but they've already announced a ton of new stuff for plenty other disabilities. Having haptic music will be awesome even for me, adding another sense to the music. There are just so many new accessibility stuff, and I can't wait to see what all is really new in VoiceOver, since there's always new things not talked about in WWDC or release notes. I'm hoping that, one day, we get a tutorial for VoiceOver, like TalkBack on Android has, since there are so many commands, gestures, and settings that a new user never learns unless they learn to learn about them.
There's certainly a non-zero number of times this could go horribly wrong (I'm guessing CarPlay doesn't have any sense of who is in the car with you) and defaulting to not reading them out is the safest option (but they could definitely add a toggle for this, yeah.)
Maybe it's not feasible for you but if you're ever near an Apple Store, you could definitely call them and ask whether they have an accessibility expert you could talk to. In the Barcelona Apple Store for example, there is a blind employee who is an expert at using Apple's accessibility features on all their devices. He loves explaining his tips and tricks to anyone who needs to.
I’m particularly interested in the motion cues and the color filters for CarPlay - I have color filters set up to enable every night as kind of a Turbo-night shift mode (deep orange-red color shift), would love to do the same for CarPlay.
I also completely forgot iOS had a magnifier built in!
With great power comes great responsibility.
Hopefully accessibility features are never artificially segmented to higher priced devices.
If it requires a chip that supports specific operations, and entry tier devices have an older chip, that wouldn't be a violation.
Source: purchased devices until finding one that worked, since Apple docs indicated the feature should work on all iPhones that can run iOS 17.
Edit: audio descriptions in Magnifier are non-functional on iPad Air, working on M2 iPad Pro.
Over here in Europe, where there's no lawsuit culture, some laws (not necessarily accessibility-related, our legislation in that area is far weaker) are violated almost without repercussions. When the government is the only party that can bring charges and the government is complacent / ineffective, nobody actually gets charged and nothing gets done.
There's also the problem of incentives, if you can get a lot of money from a lawsuit, you have a far better incentive to sue and find a competent lawyer. You may even deliberately look for and sue violators as a moneymaking scheme, some companies in the US do this. This puts pressure on businesses to comply with the law. Even if you're piss-poor and never sue anybody, you still benefit.
If this lawsuit culture doesn't exist, the best you can do is write a report to the government and hope they actually act on it. Many people don't know how to write these, and since there's no money in it, getting a lawyer to do it for you is an expense that nobody is going to help you recoup.
The people handling these reports don't help either, they're usually 9-to-5 salaried employees who aren't judged too hard on performance, so they have far less of an incentive to actually pursue cases.
Google is worse at this, Talkback multi-finger gestures used to be Pixel and Samsung exclusive for a while, even though there was no technical reason for it.
Apple has a different problem, many accessibility features aren't internationalized properly. Screen recognition still has issues on non-english systems, so do image descriptions. Voice Over (especially on Mac) didn't include voices for some of the less-popular languages until very recently, even though Vocalizer, their underlying speech engine, has supported them for years. Siri has the same problem.
Shit. I’m turning into one of those negative downers. I’m sorry. I’ve had too much internet today.
> Apps must respect the user’s permission settings and not attempt to manipulate, trick, or force people to consent to unnecessary data access. For example, apps that include the ability to post photos to a social network must not also require microphone access before allowing the user to upload photos.
Lots of iOS apps today really want to mine your address book, and constantly spam you with dialogs to enable it, but they don't go as far as disabling other features until you grant them access, because they'd get rejected once someone noticed.
Unfortunately, poor Americans cannot taste the freedom that Europeans have to be abused by developers.
To compare this to a non-monetized accessibility feature is a bit disingenuous.
The whole notion of ‘fairness’ is that gatekeepers should allow others to compete on even footing. That can be fulfilled by granting EITHER everybody OR nobody access to the market (but not: only yourself).
A big clue is that the world in it doesn’t make a ton of internal sense and the episode makes absolutely no effort to smooth that over. The questions it raises and leaves open without even attempting an answer are on purpose. You’re supposed to go “god, bicycling to make little pellets? Just to buy useless trash? God, why? It’s so pointless,” or, “they truly look down on and are mean to the people who get demoted, even though they’re basically guaranteed to end up that way someday, when their health or good fortune run out? Just cruelty for no reason that’s highly likely to hit them some day, too? That’s insane!”
… because actually it’s about now (or, the year it came out) and the point is to get you to connect the dots and realize we’re doing the same crap and just don’t notice. It’s only “sci fi” as some sleight of hand to give you an alien’s eye view of our own actual society as a way to prompt reflection. The core story of the corruption of a “revolutionary” to serve the purposes of the system is some bog-standard media studies stuff pertaining to today, not sci fi. The reveal that they could just fucking stop all this and go outside, it’s not some alien world or a post apocalypse after all is yet another thing that’s supposed to make you go “what’s wrong with them? Oh. Right. It’s us.”[1]
Long winded way to say: we can’t “end up” at Fifteen Million Merits future-world, because it’s just our own world we’re already in.
[1] Some read the final scene as depicting yet another viewscreen, but this is such a wildly weaker reading as far as the how effective the scene is that I can’t believe it’s intended that way.
Even outside of this, even able-bodied folks can be disabled due to illness, surgery, injury, etc. So it's great to see Apple continuing to support accessibility.
Recommend creating a Shortcut to toggle this setting.
Huh?
It's not built-in?
Android (or at least Moto) has it for years, auto enable on the schedule or the sunrise/sunset.
Dim backlight + Red color filter can make the screen almost invisible to those around you.
It’s attached to an automation for sleep and wake focus it works really well. I added an explicit run requirement for wake so that I could sleep in without getting blasted in the face with white. There’s a notification with a run button which I can wait until I’m truly up to hit.
https://www.icloud.com/shortcuts/4bcfd8fc02074316aaae3503d07...
The app is overall darker and more "strict" with how it handles content (esp. images) though: https://untested.sonnet.io/Heart+of+Dorkness and midnight.sonnet.io
Overall, reduced the number of photons is a fun metric to play with when building something/messing with prototypes.
I don't care how cynical it sounds, user experience became user exploitation a long time ago. Big Tech have been running that gimmick at too-big-to-fail scale for the last decade or so.
And don’t get me started on hidden command line settings.
<cough> Reduce Motion. Is it an accessibility feature or does it just get rid of an annoyance and is good for everyone?
Yes--some people can become ill with certain types of motion on a screen [1].
[1]: https://www.a11yproject.com/posts/understanding-vestibular-d...
Personally I forgot about it on my latest phone because I had a fresh prescription for my glasses and didn't need to enlarge the font at the moment :)
From outside, it feels like these are the only people with the freedom to improve the user experience at all. So they have to hide their work in the Accessibility preferences.
There's nothing that inherently "locks out" people who don't have a recognized disability from exploring these features. Furthermore, most of Apple's "accessibility" features are related to Vision/Hearing/etc (and categorized as such), so I think it's reasonable to consider them accessibility features.
Clearly based on other comments here, plenty of people discover these features and find them useful.
I'm glad the configuration is there, but it's hard to discover, so in and of itself it isn't accessible to find.
This isn’t “hidden”. It was even called out and demonstrated in the iOS 14 keynote.
Most people don't go into that menu to look around for things they might want to use cause features that almost everyone could benefit from are alongside settings for people with visual and hearing impairments.
The bar is pretty high. There are already hundreds of settings. Each one adds cost and complexity. So even if it's a good idea, the leadership might say "no".
Now let's say this same setting makes a big difference in usability for some people with disabilities. You just want to put it in accessibility settings. It won't clutter the rest of the UI.
You just turned your "no" into a "yes".
Growth at all costs, I guess.
I don't see optimizing for that as bad. If they think we'll love the product more by making it better for a given audience, especially if I'm in that audience, I'm happy. Does that mean this company now gets richer? Perhaps, and that's fine by me
They really earn consumer loyalty.
“When we work on making our devices accessible by the blind, I don’t consider the bloody ROI”.
>They’ll make features like this because it’s the right thing to do
While the positivity is great to see -- I'd temper that expectation they're simply doing the right thing and definitely acting in their perceived interest. People can and often do the right thing -- large companies rarely do.
There are companies which try (and succeed) to be honest "citizens". Berkshire and Costco have quite good reputations.
I'd also like to see what sort of weird stuff people come up with to use eye tracking. Games could use that in interesting ways.
When I hear eye tracking I immediately think of advertisers targeting me based on what I look at, NOT quadriplegics using Apple devices.
Maybe I'm a cynic
One is Zoom: I hold down two modifier keys and scroll, and can instantly zoom in to any part of the screen. It is extremely smooth, the more you scroll the higher you zoom, instantly and with high frame rate. Great when you want to show someone something, or just zoom into a detail of something if the app doesn’t support it or is two cumbersome. Or “pseudo-fullscreen” something.
The other one is three finger drag on the trackpad. Why that isn’t default is beyond me. It means that if you use three fingers, any dragging on the trackpad behaves as if you held the button down. It’s so convenient.
But sounds like something accessibility options may also provide, and I can see how it may be better for a lot of people.
In turn, I am interested why you do think that your way is (apparently universally) better?
The other way is better mostly because it doesn't interrupt the drag when you move fingers up (e.g., when you've started drag right at the right touchpad side and don't have enough space), though there is an invisible timer hack that tries to address it, and since 3 fingers take more space on the touchpad, so you're losing range
> coffee with the left hand
Still possible with the keyboard, though loses on convenience, likely better to dedicate some area of a touchpad for a touch+move to drag
> with only one hand (including right click: two fingers),
This doesn't make sense, literal right lick is 1 finger of 1 hand, same as bottom-right corner tap
And it's not universally better, don't make it up, just better than what I've explicitly mentioned
> Still possible with the keyboard
While drinking it? When using the mouse, I’m often relaxed, sometimes for example having my forearm lying in front of the keyboard.
> > with only one hand (including right click: two fingers),
> This doesn't make sense, literal right lick is 1 finger of 1 hand, same as bottom-right corner tap
What doesn’t make sense? Right click is two finger tap for me, which is super common. I don’t see the downside either.
Your way has advantages, my way has advantages, so far I don’t see a strict ordering that applies to everyone or even every situation.
> What doesn’t make sense?
You listing that gesture as some kind of alternative while the default is already 1-handed
> I don’t see the downside either.
The downside is that scrolling with two fingers might misfire when you change your mind and cancel scrolling right away, so place 2 fingers and lift them up, thus triggering a right click
> so far I don’t see a strict ordering that applies to everyone or even every situation.
And you'll never see it with your eyes closed. You need to actually try this stuff to be able to judge it, to see whether you drink coffee often enough as a % of your computer time to make the worse gesture better because it frees up your hand and the other one-hand alternative is not as convenient, or whether the mistakes of breaking a drag gesture on a bunch of files/objects in a graphics app leaving them in the wrong place are bad enough
Meant to write “no mobility problems”, but too late to edit now.
System preferences -> Accessibility -> Pointer Control
Then turn on the "Head pointer" option.
I imagine that could be pretty awesome for deaf people.
The vision pro has six mics, which likely enables it to locate the sources of sound in space by the amount of time it takes a sound to reach each mic (i.e. acting as an "acoustic camera"). Tablets and phones aren't going to have that hardware unfortunately.
And yeah, obviously wearing a headset is a pretty big compromise that's not always (or maybe even often) going to be worth it. This was more of a though of "the hardware can probably do this cool thing for people already using it" than "people should use the hardware so it can do the cool thing".
I use SimDaltonism, to test for color-blindness accessibility, and, in the last app I wrote, I added a "long press help" feature, that responds to long-presses on items, by opening a popover, containing the label and hint. Makes testing much easier, and doubles as user help.
Edit: I seem to be missing something as this is getting downvoted. I genuinely cannot use HN under 150% zoom so thought this was a basic comment I was making.
You could integrate a differential equation by lining rocks up in a large desert as a computer, but you wouldn't say that solution is "accessible" to a human in the same way it would be with a CPU, a monitor, and functioning sight.
If people made it "big enough" to be inclusive by default in some websites I'd have to zoom out as well. So my point is that to me is more important to zoom in correctly (many websites don't), than be "big enough" to start with.
I always end up double-checking whether the ‘unvote’ link is present. If it says ‘undown’, I know I’ve made a mistake (or vice versa).
I wouldn't downvote you, but I think those who did have read your comment as saying that it (hn design) is fine and they disagree.
(I'm not saying that's what you've said)
Ironically because of addons I now use firefox exclusively on mobile.
I can only hope that the algorithms take into account that there's a decent chance someone trying to hit a link or access the comments will accidentally report a post instead.
I still do it all the time. It's a problem on both desktop and mobile. I've send mail about it to dang before and did get a response, definitely a known issue.
I appreciate the information density, and that HN hasn't adopted some "contemporary" UI that's all whitespace and animations.
(And yes I agree the buttons are awkward)
It was on Accidental Tech Podcast #415, 26:19. The topic starts at 25:40
While on the topic, I can hear music but almost never understand lyrics; at best I might catch the key chorus phrase (example: the words "born in the USA" are literally the only words I understand in that song).
A few months ago I discovered the "karaoke" feature on Apple Music, in which it displays the lyrics in time with the music. This have been game changing for me. I'm catching up on decades worth of music where I never had any idea what the lyrics are (filthy, that's what they are. So many songs about sex!). It has made exercising on the treadmill, elliptical, etc actually enjoyable.
JSYK Spotify has had this for years too, its just under the "View Lyrics" button but it does highlight the sentence/word karaoke style. It used to be a "spotify app" back in that genre of the service.
One thing I wish Apple would implement is some kind of gesture control. The camera can detect fine details of face movement, it would be nice if that were leveraged to track hands that aren't touching the screen.
For an example, if I have my iPhone or iPad on the desk in front of me, and a push notification that I don't need obstructs the content on the screen, I would love to be able to swipe my hand up towards the phone to dismiss it.
Even on Pro devices, audio image descriptions stop working after a few cycles of switching between Image Magnifier and apps/home. This can be fixed by restarting the app and disabling/enabling audio image descriptions, but that breaks the use case when an iPhone is dedicated to running only Image Magnifier + audio descriptions, via remote MDM with no way for the local blind user to restart the app.
On-device iOS image descriptions could be improved if the user could help train the local image recognition by annotating photos or videos with text descriptions. For a blind person, this would enable locally-specific audio descriptions like "bedroom door", "kitchen fridge" or specific food dishes.
Are there other iOS or Android AR apps which offer audio descriptions of live video from the camera?
I am surprised CarPlay didn’t have voice control before this though.
BTW if you are plugged in to CarPlay and take a screen shot, it will include the hidden CarPlay screen.
Precision is the issue because we are mostly moving our eyes in about 8 directions, there’s no precision because we don’t know how to measure focusing of our eye lens with a camera yet (unless that too is just a matter of getting a picture).
Squinting would be the closest thing to physically expressing focusing. So the camera needs to know I’m looking left with my eye, followed by a squint to achieve precision. Seems stressful though.
Gonna need AI just to do noise cancelling of involuntary things your eyes do like pupil dilation, blinking.
It’s also pretty common for Apple to preannounce some smaller features that are too specialized to be featured in the WWDC announcements. This gives them some attention when they would be lost and buried in WWDC footnotes.
It is also probably an indication that WWDC will be full of new features and only the most impactful will be part of the keynote.
I am completely serious. I work with a lot of older adults, many of whom have cognitive challenges, and the lack of a physical button to press as an escape hatch when they get confused is the #1 stumbling block by a mile.
I don’t understand Apple’s intense internal focus on FaceID only. FaceID with TouchID as an option when a device is flat on the table or when your face is obscured is so much nicer.
I don't think this is a user-focused decision, I believe it's a profit-focused one.
It isn't. Just a few days ago the new iPad Air has Touch ID.
Only iPad Pro uses Face ID. For iPad Pro users who use it for work and unlock it hundreds of times a day when in an office or during a commute, Face ID is vastly superior.
I have naturally very dry skin, and no matter how much moisturizer I used in colder months, I had to constantly have it relearn my fingerprint, to the point where I would just give up and use a passcode instead.
Face ID, on the other hand, has been flawless since my first phone with it in 2018 (minus some mask-related challenges during the pandemic). It and ProMotion are basically the two reasons I bought an iPad Pro over an iPad Air.
On early models it was actually quite common for the button to stop working after a year or two. The durability has since improved, but habits die hard.
Then when they see younger people just randomly explore-clicking to get things working--because their experience is that it's a safe tactic--that can gets misinterpreted as expertise, leading to: "Wow, kids these days just know technology."
It would be so great if more resources could be allocated for such things within the Linux ecosystem.
Surveillance technology: >:(
Surveillance technology with the word "accessibility" in the title: :)
an iphone constantly broadcasts your location to third parties, can deduce where you work and live, understands the unique shape of your face, has a built-in microphone and multiple kinds of cameras, stores all of your conversations, stores all of your photos, stores your health information, can figure out when you go to bed.. all on a completely closed and proprietary operating system.
it’s like asking “why hasn’t anyone mentioned that we’re all using a website right now”
* on Android, in the builtin popup menu for copypasta, the COPY button is wedged in tightly (by mere millimeters) between the CUT and PASTE buttons on either side of it. A harmless one packed in between two destructive, lossy ones, and usually irreversible.
* the size of an adult human fingertip is no secret (and in fact 99.99% of humans have at least one!)
* Google's staff consists of humans
* Google supposedly hires ONLY "smart" people and their interview process involves some IQ-like tests
* Android has had this obvious anti-pattern for many years now. perhaps 10+?
I'm not fully against it, but it's good to keep in mind the implications.
https://support.apple.com/guide/mac-help/navigate-your-mac-u... - Full Keyboard Access (accessibility feature, goes beyond just tabbing between elements)
It's annoying that tabbing between UI elements is off by default on macOS. It's one of the first things I turn on with a new mac.
* readers are those glasses you can pick up at the drug store for $17.99.
everyone rejoiced. nobody implemented anything useful.
soon, advertisers, google included, was selling ads packages by "viewability". lol.
can't wait for safari to lead with "eyed" ads.
> all data used to set up and control this feature is kept securely on device, and isn’t shared with Apple.
It's not an API adopted which apps can use, it's a system pointer, like a mouse or trackpad
Does this line actually mean anything? Press releases are so weird.
> We believe deeply in
"We", in this case, refers to Apple and as a company can’t believe in anything, it specifically refers to their employees.
> the transformative power of innovation
A new feature will bring forth change
> to enrich lives
In order to make someone’s life better.
So far so good, at least no synergies being leveraged for 10x buzzword transformation. Let’s see if it passes the test: There’s a bunch of new accessibility features, that certainly fits the bill for innovation. As anecdata, at least one of these will drastically change how I interact with my devices, so there’s the transformative power. I will be able to use these new accessibility features to improve both the way I work and how I privately use technology, so one could argue my life is being positively enriched by this transformative power brought about by innovation. Do they "believe deeply" in this? That is only for Tim Cook and his employees to know, but they’re one of the few companies pushing advances in accessibility past the finish line, so they might!
Just because it's not a game changing feature for some people, doesn't mean its primary function isn't advertising.
Microsoft added a similar feature to Windows about seven years ago.