Apple rejected my dictation app for using the accessibility API
mitmllc.com
mitmllc.com
However, I would like to point out that Apple isn't totally wrong here because the accessibility API unfortunately is way too broadly scoped, and because of that you literally get access to everything on the computer like you you can screenshot listen and and move the cursor... This is completely ridiculous and the proper engineering solution would actually be to phase out the accessibility API and replace it with something that is narrowly scoped so you can grant specific permissions individually.
However, Apple, being Apple, is obviously not doing anything, and instead says no accessibility permission for anything that isn't demonstrable accessible. Now, there are obviously some exceptions because Apple is not particularly well known for applying its rule consistently and granting big exceptions for itself. However, they do have a valid point on privacy and data protection. And I say that as somebody who ended up distributing my MacOS app outside the App Store because I only got approval for iOS.
That said, I would definitely appreciate if Apple would gradually improve its developer program experience, because compared to its hardware lineup, the developer program is nothing short of abysmal.
You'd lose all window managers, things like alfred and textexpander, screenshot tools, computer use agents, etc.
I want apps to be able to do that!
Is there an opinionated reason not to break out capabilities?
If you have a disability and need tools to use your computer the last thing you want to do is have those things not only off by default but complicated and involved to turn on.
accessibility.screenshot accessibility.paste
and whatever else there is. that completely removes the issues for apps like this.
The issue is with Apple's UX. Apple insists on asking permission for every little capability an app wants. So I would have to say "yes, allow this app to take screenshots" and "yes, allow this app to read the clipboard".
I wouldn't be surprised if, in the near future, Apple forced people to click "yes, allow this app to read the clipboard from app X" and then separately "yes, allow this app to read the clipboard for app Y" and so on for every single other app on my machine.
Apple does not allow you to say, "yes, I trust this #$@-ing app, please allow it to do whatever it needs."
If you don't have use of your hands you want that. The whole point of accessibility APIs is allowing arbitrary control of your computer via novel means. One of the big selling points of Dragon Natually Speaking is the ability to tell your computer to do things based on descriptions without a mouse. "open outlook", "click compose", "select subject", "type foo", etc.
And no the solution here is not computer vision with an LLM. Text and buttons rendered on my computer exist in memory somewhere as text and buttons. We should not need to convert them to pixels and back lossily to recover text and buttons. We should just expose things to the accessibility API and not guess.
Also, even if you hypothetically wanted to use computer vision with an LLM… what API is that LLM going to use to take screenshots and click on stuff?
Are we sure about this? At least on windows, NVDA works fine with chrome and any electron apps.
If you're worried about people not trusting payment to you, might be worth seeing if you could implement this, so anyone who bought on the app store can still access the full feature set. Cuts you out 30% like, but better than nothing maybe.
https://en.wikipedia.org/wiki/WinDirStat#Version_history[10]
Everything I ever added was kept, and I was permanently banned. I created [ciation needed], started the admins noticeboard, reworked the USA Patriot Act article, wrote numerous articles for WiR with extensive referencing, contributed to peer review and good article reviews, and a shitload more, but nope. Not good enough.
Why anyone would contribute to that cesspool is anyone’s guess.
I can assure you, there are those on Wikipedia who committed far worse offenses and they remain.
Like I say - a cesspool that doesn’t respect article writers.
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_no...
So no, not the pot calling the kettle black in any way. She had her coiterie of followers, and she was banned as a bully. They enabled her.
You can't, because that never happened.
But hey, keep defending BrownHairedGirl. You seem to be fine with her constant abuse!
* BHG then made a thuggish and threatening post on your user page:
* BHG later characterised this like "oh but he was trolling me and I merely asked him one time on his talk page to stop (which is allowed)", which is bollocks because the "one time" is a long, angry screed written in a menacing tone.
And yet, no action was taken against her. I wonder why that was? And yet I was the one who was indefinitely banned. There was clear bias, and Wikipedia lost a good contributor. One I would say did far more than she ever did (she dealt with categories and dead links, and one article relating to Ireland).
Incidentally, if you want to know what the poster missed... it was that it was a one-way interaction ban. BHG was never, not even once, admonished for the way she treated me. In fact, her supporters actively encouraged her to attack me. When she attacked me, I was unable to defend myself or respond. Does that sound like procedural fairness to you?
Funnily enough, she did this to one too many people and was banned. I was banned by her supporters, not by ArbCom. It's what is known as a kangaroo court.
But hey, fairness and impartiality on Wikipedia was something that was lost a long time ago. Wikipedia is continuing to slide into obscurity. The ones who are active are most interested in drama, and not content. I feel sorry for those who are there for the content.
But when I hear people really complain somewhere I do tend to assume they were trying to smuggle in some specific messaging and got caught.
Some pages have somebody guarding the party line, anything that goes against that gets reverted. (How can putting an accurate link behind a piece of text be wrong??)
Pages without such guardians I've never had an issue.
In other words, Apple is abusing their position by defining overly broad permissions so that they can deny them and pressure people to fork over more cash to them.
Checks out, what's the problem? /s
Where I was more frustrated was how much this limited the potential usability of the iPhone app. Because of app store restrictions it is a far worse app ... though like in your example, still useful to a degree.
I can only hope they use the new CEO as an opportunity to seriously re-evaluate their entire approach to how they work with developers, though I'm not actually expecting them to. If anything, with the increase in apps being created via AI tools I worry they will go the other way.
If you're not, ask your representatives why you don't get the same rights.
On top of that, the app is completely optional: if you aren't comfortable giving it those permissions, don't install it?
Lots of shady and well-known developers (like Dropbox) are notorious for trying to weasel their way into getting Accessibility permissions, so they can do god knows what with them to your system.
Those two desires should both be fulfilled.
Worried about grandma installing shady apps? Enable parental controls on her phone.
Install some GNU/Linux distro and you can do whatever you want.
I just installed PopOS on a laptop recently, and… it just worked. There’s an app store for noobs that I think installs flatpaks. GPU drivers just work. Whole disk encryption. Everything just works.
I don’t see what else my grandma that just uses Facebook would need. Maybe automatic updates?
If you and your grandma only rely on the computer for its web browser, then good for you. You have flexibility that is not afforded to most people. But that's not how a person's phone works; phones dig a lot deeper into one's lifestyle, intentionally so. The walled garden was constructed to keep outsiders out, but now it seems the primary purpose is keeping those inside hostage.
I own more (and have them running right now) machines with linux than anything else and yet I'm not saying people can just switch. The problem is usually not "can do at all costs" but "can do with a reasonable addition of extra steps/relearning/tool does not exist/etc". There's some nuance and when I have some spare time I will (again) try to switch that one machine, but "it just works" maybe can also mean you're not using it for a diverse enough set of things.
In my case the reasons are actually quite boring: some hardware I couldn't get running and some (maybe minor) things that drive me nuts. The hardware is kind of a deal breaker atm. And yes, some people do a lot more weird things at home, my work machines were running Linux for 90% of the time since 2010ish.
i call bullshit. i have worked in very big orgs. changing a single icon can cause a deluge of support tickets.
She loves it. Zero problems. It's been a week and she's using it just fine. No lifestyle upheaval.
I couldln't imagine having the time to set it up as a daily driver that handles my daily workflows, hardware needs, etc. Terminal in OS X is a close enough approximation out of the box and goes beyond it in DX (IMO) with very little additional setup.
I know this will be an unpopular opinion.
No - moving to far away areas is not the right analogy. After all you need to have use cases where those huge companies do not control your business. So the alternative is to avoid becoming dependent on them; or cut off the dependency when possible.
You get a channel for installing apps, where someone vetoes random apps that want to have access to control your whole computer and potentially steal sensitive data?
>Install some GNU/Linux distro and you can do whatever you want.
And any random app can get total control and steal your data, unless you know how to enable restrictions. I'd rather have restrictions as the default, and for the most naive users who'd follow every app prompt, and then cry about their lost work/private documents/money, no way to bypass them.
I dont wanna start a war over this btw, even though it may not seem :)
On a personal computer, they "can't do much" to the things you can trivially re-create by reinstalling anyway. Apps, system files, etc.
They can however do everything to your own files, steal your documents, bank account data, and more.
That a progran run as you without root "can't do much" made sense for multi-user Unix services, not for a personal computer and your own files.
>Wonder why the most important systems in the world and big tech's servers run GNU/Linux? There's a reason
Yes, and it's not because "unless you log in as root, the apps can't do much" on your personal laptop.
Then don't install apps and use the web, mobile sandboxing is much weaker compared to any modern browser.
But we're talking about a case where you also want/need to use native apps. Not necessarily browser. Even if "it's the same" in this case, not all apps are equally viable as browser pages.
Not to mention performance/privacy/etc wise web apps are a regression.
Privacy-wise, web apps are also much better than native apps, less apis to exploit and also you can use an adblocker to kill tracking which as far as I know, doesn't exist on any mobile platform at the moment, even on GrapheneOS (I'm not talking about basic DNS shenanigans but actual automated native tracker removal)
I'd only agree about the performance part and even then, with the regression of mobile performance in the last few years (Material 3 / Liquid glass), it's not as clear cut as it used to be.
> But we're talking about a case where you also want/need to use native apps
Well if it's your only criteria then, the bar to get accepted in your average mainstream Linux repo is much much higher than any current mobile platform.
I tried very hard to switch to Linux full time some months ago, but I couldn't find a way of getting Microsoft Office to work satisfactorily. There are clever packaged versions of Outlook and Teams, but I need full native installed versions of Word/Excel/Powerpoint, and there just wasn't a good solution. That was a deal breaker, sadly, so I'm back on Mac for the time being.
Other examples would be some of the popular games with anti-cheat that requires Windows.
I'd argue that installing and updating apps on MacOS is simpler than on Linux distros because most apps have built-in auto-updates (or you can just drag the app to the applications folder) instead of having to rely on snap / apt / insert your package manager which may a lot of outdated and unmaintained packages and apps.
I get that some people are unfairly targeted but some other times it's people being (extremely) naive or just playing dumb
"Hey you know what would be cool? If we named our bluetooth speaker company bee oh emm bee!!11"
Build system woes are almost always solved by deleting build cache & artifacts and trying again. Often necessary after messing around with deeper dependencies.
This API is sensitive. I imagine Apple is particularly stringent as to how the access is justified. Not how it uses it but how the reason for using it is explained.
It's not like someone tests the app and all api calls to deem them reasonable or not.
I wrote about this here [1], and even called out some of the most common disabilities that clearly benefit from it.
They also don’t allow microphone access from keyboard extensions in iOS, which prevented me from shipping an iPhone target (that I really wanted for myself tbh).
It’s a shame and I can only assume they’re trying to protect their built-in dictation, which historically has been laughably bad.
Imagine a banking app, and for example an IBAN field.
I'm using https://github.com/cjpais/Handy whichseems to be doing exactly what this app does, and has a very similar background story (author couldn't type die to injury).
The problem from Apples perspective could be that there is a ton of tools that require access to the accessibility API because they want to do stuff that Apple have deemed a security risk and the only way to do it is by abusing the API. Some of these are also because macOS simply lacks certain APIs.
I think Apple overreacting due to previous API misuse by other apps.
Have fought similar demons lately, feel your pain.
I got an ortholinear keyboard that looks like a rectangular grid, just 12 by 4 keys around 10-15 years ago.
I don't recall the last time I felt pain in my hands, completely gone.
Screw Apple and their persnickety, controlling myopia.
Unless the platform is iOS, in which case it's "comply here or gtfo".
update: You're right, this is a real bug. The Direct version's auto-paste hardcodes the QWERTY keycode for V instead of translating for the active layout, so Dvorak / Colemak / AZERTY users would all hit it. The MAS version is unaffected (clipboard-only; the user presses their own Cmd+V, which is layout-correct). Fix is going into the next release. Thanks for the careful read.
[0]: https://boltai.com
Make apps for device, which are 100% owned by people.
net::ERR_CERT_AUTHORITY_INVALID
Microsoft was almost broken up over not allowing third party programs to use certain APIs. Apple abuses their dominant position to suppress competition.
Edit: Ah, it's in the article, this is about AppStore distribution. Walled gardens are going to walled garden.
OP’s description in the linked article doesn’t say much more than this, so what am I missing with this particular app?
I just wish they weren’t so obstinate about people installing from other sources without signing/notarization. I understand it from a security standpoint but it’s also nakedly self-serving.
I’m glad that they’re fine with signing in this case.
If anyone here has more direct experience with this guideline, especially from the App Store review side, I would like to hear it. I would rather understand the policy than just guess at it.
I have no idea what they’re thinking. Insanity.