Apple Shortcuts is falling into "the automation gap"
sixcolors.com
sixcolors.com
This is an exaggeration, though. I managed to use Shortcuts even while writing this text, so it definitely is useful.
But why is Apple so goddamn afraid that if I get to do what I want, something terrible will happen? Why can't my shortcut paste, keystroke, or use SMS as triggers? I understand there could be unintended consequences, but to me at least, the limitations of Shortcuts as it is now are obviously not technical but philosophical/political from Apple. Give me keystroke shortcuts and proper triggers! I promise I won't abuse it, and I also promise to not run with scissors!
It could become interesting if there were some integrations with Apple Intelligence: the tasks you'd delegate to a shortcut are the kind of low-stakes stuff AI integration is a good fit for. It doesn't look like it's in the cards, though...
I also made a shortcut that lives on my home screen to open a notes page for today, creating a new one if it doesn’t exist already. That one would be more used if I could ever get a journaling habit to stick.
It’s certainly very limited and suffers a lot from a poor interface, but I don’t think people give it enough credit.
If you try to paste anything into the dev tools in Chrome, it forces you to specifically allow pasting because scammers have convinced people to do it over the phone to con them into something. How I’m not quite sure.
It's pretty obvious that automation for non-developers is not a priority at Apple.
Opening up automation is a double whammy for Apple:
- More savvy users are able to solve more of their problems themselves, reducing the LTV of their potentially most engaged clients who might buy less on the App Store as a result
- Some guarantees of the ecosystem crumble (banking apps don’t know anymore if it is really you who initiated that transaction, ID verification apps don’t know if that camera stream is really from your device, and plenty other things devs suddenly have to worry about)
As a final nail in the coffin, it also means that the networks that Apple sells to its _users_ are less reliable or have less guarantees as a result - you don’t know if the text you received was really sent now or scheduled in advance etc.
Of course a lot of these rules are not justified, user hostile or plain non-sensical, but what I’m trying to say is that from Apple’s perspective the consideration is not just “user owns device” but a lot of interplaying dynamics that do not seem to be in favor of empowering users.
I feel like this is the general business ethos of modern consumer-facing tech. They don't want to sell tools that serve you. They want to sell a service to chew your food for you as you accept whatever mush they feed you.
Then there’s JavaScript for Automation, PyObjC, and many other technologies that Apple never really invested in because they just don’t get automation.
Somehow, Shortcuts survived being acquired, but in the process they actually killed stupefyingly useful functionality like being able to run some automations directly from the Apple Watch (I used it to send automated SMS messages to query bus schedules based on my location, and it was awesome).
Someone put a lot of effort into designing that system. Then it was basically forgotten.
Then it was resurrected in partially zombified form as Shortcuts - and forgotten again.
https://9to5mac.com/2016/11/17/mac-user-automation-sal-sogho...
Sal Soghoian, the man who headed up user automation tools at Apple – including AppleScript and Automator – says that he left the company last month when Apple eliminated his position ‘for business reasons.’ … The decision to lose Soghoian after almost 20 years in the role, together with a statement that makes the future of user automation tools seem uncertain, is bound to cause concern among those of us who depend on them on a daily basis. The fear will be that Apple may gradually cease support for power tools as part of an increasing consumer focus.
- wall mounted 6" screen + voice control
- iPad on robotic arm
- FaceID door locks
- security cameras
So you add your automation crap to HA, scan a QR code from HA and everything pops up in HomeKit
For me that was a bit oof, as it literally adds _everything_. So I just use Apple TV as a HomeKit hub with the basic stuff added directly to it and a few specific devices (Shelly) bridged from HA using a whitelist.
This way the WAF stays high and Siri works properly.
HA still does the more complicated stuff (ATV is on pause -> make lights in the TV are a bit brighter. ATV playing -> dim the lights according to the time of day etc.
It was somewhat recently that I read something about what I’ve also felt personally, that people haven’t really warmed up to the whole home automation idea. At least speaking for my own feelings about it, it’s largely because it’s unreasonably expensive for most people, an unreasonably challenging task to set up even with perfect conditions of a single company’s products, unreasonable complexity that introduces unreasonable brittleness, and an unreasonable mental load on having yet another complex, brittle thing to have to manage. It’s the antithesis of what Apple stood for at a time.
Apple succeeded because it abstracted having to be an engineer to do things. It’s why so many engineers hate Apple because of the “walled garden” and are smitten with Linux and Android because you can accomplish the same thing in 1000 ways, but none of them frictionless or even clearly, simply and consistently documented. It’s something the engineer minded don’t seem to commonly understand, people don’t want to be engineers like you to live their lives. You go to a restaurant so you don’t have to cook, not because you have to make 1000 decisions before you even get any food, it’s why I eventually switched to iOS when I did, and it was honestly a kind of weight off my shoulders because I had gotten tired of the self-centered maybe even narcissistic nature of things like Linux and Android.
Maybe a measure of business and product success should be the rankings of how easy it is to explain it to your Nanna and her using it … Nanna Arena, anyone?
But even if Apple can inject its famous “just works” magic into home automation, it is combatting its own entropy that has been slowly introducing bugs, glitches, and brittleness over the years. Does anyone feel like calling for Siri 8 times and being ignored before having to hold a button or simply just do it manually in frustration, with a system that costs $10,000 just to turn on your lights?
This is the company that charges $1000 for a monitor stand that is a piece of aluminum, I can’t even imagine the price of a 6” wall mounted screen, let alone an Apple robotic iPad arm. Not to mention it seems unlikely to me considering Apple’s abandonment of things like AirPort and the fumbled HomePods.
It all just feels like a typical recent core nature of Apple, too late to the wrong party because the company has been not just taken over, but overtaken by people who want to ride a wave to feel good rather than do good effective work.
But that’s just me. Does anything I’ve said resonate with anyone else? I get hints and glimpses of it here and there when I can tell myself, “I guess I’m not just imagining it”, and maybe something is very off at Apple.
We're smitten with these systems because they do everything that Apple does easier and because they do useful things that Apple doesn't let you do at all.
Also, I have been noticing more AI-generated slop comments on this discussion board.
I have a shortcut for captive portals when you want to get on WiFi, a shortcut to dictate a quick search on Kagi (I am working on another for Kagi Assistant) and another to check the trains on the train website. Between these and apple wallet I can leave my phone at home.
I was trying to work with it to customize finder a bit and found it had so a lot of permission and settings config requirements.
Have the security changes and scrambling of System Settings made Automator harder to use?
They can run external tasks via SSH scripts or API endpoints, e.g. send audio to LLM, https://www.innoq.com/en/blog/2023/03/openai-gpt-on-macos-io...
For local actions, app-specific URL schemes can be driven from shortcuts, https://www.macstories.net/tutorials/guide-url-scheme-ios-dr...
In iOS18 on iPhone Pro with lidar, a shortcut can trigger Magnifier to provide audio description of live camera image.
NFC stickers or expired transit cards can trigger shortcuts by physical proximity.
I think it’s pretty cool that I can make a shortcut connect to a host on my home network via ssh and execute a command. To just trigger that using my voice and Siri when my phone is nearby.
I live in a country with stupidly expensive electricity, so have my home media center connected to a smart switch, which I switch off last thing at night. I am thinking of moving my Pi mini NAS setup with it’s various drives into that media center setup, so last thing at night I can get a shortcut to SSH to that host, do a clean shutdown, wait a short while, and then use the Hue connector in Shortcuts to cut the power. I couldn’t do all that with either Amazon Echo or the Hue app.
The saddest part is that both the Alexa and Apple walled gardens spent years constraining users from integrating with other ecosystems, but now they are all racing to give data to 3rd-party LLMs like OpenAI, Anthropic and Gemini, which can ... integrate with other ecosystems via APIs, MCP servers.
Users can now escape walled gardens, at the cost of multiple middlemen. If Alexa and Apple had allowed users to access other ecosystems, there would now be thriving customer communities with proven open workflows that could be sherlocked into commercial success by the platform owners. Instead, that value will accrue to foundation model companies.
That reminds me of an announcement maybe about a year(?) ago, when people on the internet were making a huge deal out of ChatGPT integrating with Wolfram Alpha like it was this game changer, and I just remember thinking... we already had this with Siri over a decade ago, with no LLM middleman necessary, until Apple decided to kill it and start integrating their "machine learning" into all their products around mid 2010s.
I could probably do much of this stuff with Home Assistant, but it’s always a bit too far down the todo list. One day…
If anyone knows of an app that gives access to live TV and actually cares about the basic functionality of the platform they run on, I'd love to hear about it and they can have my money.
Yet even then, I know I have particular need, I'm not the average apple customer. Most people never heard about shortcut and wouldn't care if it disappeared.
Working on shortcut must be quite time consuming and benefit something like 1% of apple customer, so I get that it's never really going to be a priority.
They would rather focus on artificial intelligence that the average Joe can use.
Also, a quick search shows Automation[1] as an open source alternative. No personal experience with this one, but will probably try it out soon.
[0] https://play.google.com/store/apps/details?id=net.dinglisch....
Also extensible - it can trigger scripts in Termux. You can install any command line open source tool inside Termux and trigger it on tasks.
eg When my home radiator thermostat stopped supporting IFTTT, I made a Python script to call their API and trigger it with Tasker. Works flawlessly.
https://www.flourish.org/2023/11/netatmo-smart-thermostat-ho...
Shortcut on iOS is like 2 step.
Step1: export pdf to images and choose dpi (lowest for maximum compression).
Step2: make a pdf of previous images.
I couldn't figure out how to compress pdf on tasker without using some convulated third party addon.
It's just so convenient having small little script that run well and easily when you're dealing with people that barely know how to use a computer.
Myself I'm often on the road and have to send pdf reports to customers, sometimes one doesn't go through mail because it's too big or the customer himself have issue to send them to other parties and ask me if I can compress it. It takes less than a minute on my phone and I admit having already done that while driving because I was late.
Good luck doing that holding car wheel in one hand and the computer in another :)
With shortcuts I can create a short and simple "siri, do task" and that's it, the shortcut runs no matter if it's my phone, watch, or homepod that hears me.
I experimented with the idea of going back to Android. Having to recreate all of my shortcuts in Tasker was one non-starter. (It not having a HomeKit equivalent was the other. Google Home is much more limited.)
Shortcuts can also sync between Apple devices via iCloud (though Automations in Shortcuts don't; another thing that drives me mad about the app), and they more-or-less work regardless of where they're running.
I had an iPhone for a few months recently, and I tried the Shortcuts app, but it was extremely limited and couldn't even do some simple things I needed.
I couldn't figure out how to do it using tasker.
Then there is the Modes and Routines app that allows you to do anything.
On iPhone, to add a shortcut to a picture, the Shortcut app asks me to give it the filename! Of course they hide the filename everywhere they can because apple. They could provide a browse like experience but no, type in the filename! How am I supposed to find the filename if you hide it from me?
You're right. Because the automation apps on Android actually work, and Shortcuts can most charitably be described as a hot mess that should be put out to pasture.
I have tons of Shortcuts. They're excellent when they work. It's extremely frustrating when they don't.
Maybe your Shortcut uses an action that is no longer available in the version of iOS you updated to (and wasn't marked as deprecated in the changelog). Maybe the app that the action relies on send an unusual response, which Shortcuts won't tell you because it's way of error handling is "leave the user more confused than they started with". Maybe Shortcuts decided to just keep itself running in the background forever for no reason (which is even more frustrating given that apps like Music will get killed by the memory manager _WHILE THEY'RE PLAYING MUSIC_, something that should never happen on the latest iPhone).
Despite it's shortcomings, it's still better and a hell of a lot easier to use than Tasker, it's third-party equivalent on Android (because it, surprisingly, doesn't have an automation tool built-in).
I could not find another use for Shortcuts because there aren't enough integrated apps.
However, I improved it by recording the info into a Numbers spreadsheet now. Makes reporting and calculations easier.
I can't imagine this, and would prefer that resources be spent shoring up Apple's increasingly shitty frameworks.
Overall, Shortcuts feels like a nice polished app, but it's definitely severely lacking compared to Automator.
Next problem: I had to set the alarm sound to play at high volume since the effect itself is relatively quiet. Woah! That volume is permanent and now after an alarm, everything plays loud. Ok, no problem. I’ll just have the shortcut get the current volume and save it to a variable at the beginning and then change the volume to that variable at the end. Oh, there’s no way to set the volume to the value of a variable? WTF? The HomePod has actions for resizing and rotating images, even though it doesn’t have a display, but it can’t do what I need it to as a speaker?! Alright, hardcoded 50% volume will have to do.
Final problem: My shortcut repeatedly waits a few seconds and then checks the state of a door sensor in a loop. It would be nice if I could have a condition to exit the loop early but keep running the rest of the shortcut. Oh, Apple made a whole scripting system for shortcuts but didn’t include a `break` keyword. Lovely.
I officially have a love/hate relationship with shortcuts, which leans towards hate most of the time.
Good enough for my small quality of life improvements.
Anyway, something I do appreciate about Shortcuts is that most shortcuts can be invoked via CLI, so one can do some interesting things with them.
Honestly though, I still prefer AppleScript and/or Swift for most automation.
Instead of this cumbersome UI they should have provided a UI where you just normal Swift and ability to generate that Swift from prompting.