The Sky's the limit: AI automation on Mac | Hacker News Reader

The Sky's the limit: AI automation on Mac

taoofmac.com

123 pointsphony-account71 comments

One doesn’t need to wonder why Apple couldn’t get something done. Apple seems to be in dire need of a shakeup at the top level, but seems to be incapable of seeing the need for that or being able to do that.

From the outside, it looks like there are just power struggles and fiefdoms held by the old guard that have resulted in the stagnation and the worsening quality across its operating systems. Even today you cannot search and find all the things in the Settings app on iOS, which has been a longstanding issue. ScreenTime is totally broken since iOS 18 and shows tons of minutes of apps that I open for just one minute everyday. There are many more irritating old bugs as well as new bugs being added regularly.

The recently rumored to be forthcoming “26” numbering of all its operating systems and the cementing of annual releases with new feature addition doesn’t bode well for improving quality. The way Apple’s software teams have been working, a tick-tock cycle of improvements followed by stabilizing every other year is the only way things can get a little better. That’s a pipe dream for me anyway.

I would wager sky.app will find themselves acquired by Apple before the end of 2026.

Again.

I am a bit late to the (upgrade) party, but ios 18… what happened?! If you scroll through dates in the Calendar app, random date numbers vanish. Huh. Search in Notes is super weird (shows old, fuzzy matches first). Photos app is just.. wow where am I? How is there no jailbreak yet? (Is there?). I have read here the main issue is fragmentation, but I am not sure how that (not) works. If one team builds, let say Notes.app why is it so broken? Because it recycles search libs from (broken) Photos.app? Sorry for the rant.

Edit: is this to upsell the AI?

Yeah! macOS gets more and more like Windows by the minute. I dread the day, in about 700 years, when you can hardly tell the difference anymore.

Thanks for mentioning this. I didn’t connect my general confusion in the Photos app with an upgrade but it makes sense.

The navigation is so unintuitive to me that I’ve killed the app just to try and get back to the most recent photos I’ve taken.

Apple products have mostly been very good. Even when I didn’t love the UX I could figure it out.

So many recent features just feel half baked (like replies in iMessage). I’m starting to wonder if they’re better off leaving good enough alone (not that that is a remote possibility).

The app is destroyed, but it has some candy coat that distracted me for clearly stating my confusion after the initial shock. One of the most insane interactions with the photo app now is that, the basic(?) view offers you a chronological view of your pictures, top to bottom, bottom being most recent. Fine. But any other section, pinned, albums, etc, the order is reversed! Not to mention every ‘section’ has its own visual effects going on, radically different between albums and pinned things etc. If this is what they mean with ‘disruptive’ i am out ;)

I think Apple will have trouble course-correcting because there currently isn’t a good OS for sale. Why improve in terms of quality, if there isn’t any competition?

Weirdly, I think they might be smarter than us. Despite being superficially uninterested in hardware, the general public seems to buy hardware, and then just use whatever OS comes on it. Apple seems to have aligned with this by easing up on software quality a bit and obtaining a reasonable (although not incontestable) claim at being the best hardware company with their M-series chips.

I’ve seen arguments that it largely comes down to the lack of a Jobs-like figure at the top maintaining a coherent, user-driven vision across the company and won’t hesitate to take a hatchet to projects and initiatives that aren’t furthering that vision.

Cook is excellent in terms of managing supply chains and has been good for the company’s bottom line, but he doesn’t have Jobs’ capability to discern what users want/need and what makes a good product, and I doubt he has anywhere near the level of direct involvement that Jobs had.

That’s not to glaze Jobs, he was anything but perfect but he was undoubtedly among the driving forces behind Apple’s quality and knack for building things that people want.

Oh wow, clicking on their "Join Team" button is absolutely amazing. Credit to them.. almost makes me wish I was a Mac developer to apply for a job. Almost.

https://software.inc/jobs

This gives me a black screen on iOS safari

Edit: working now

seems like this is an actual emulator not just a JS approximation

So snappy, it's almost faster to mac os 8 in a browser than it is to use mac osx.

For those who want a better explanation of Sky's capabilities, I recommend Federico Viticci's article (linked from the Tao of Mac article but the links are not distinctive): https://www.macstories.net/stories/sky-for-mac-preview

He goes fairly deep based on two weeks of usage.

People complained about the links being too visible, so they are only visible once per article now, and I make sure to highlight them on hover over a paragraph. I guess I will never please everyone :)

Ha! yeah, you can't make people happy all the time. And I mean, I found the link, so I can't complain too much.

Quicksilver[1], app switcher [2]… Sherlock[3].

Now that Apple knows what to build, they’ll build it.

1. https://en.m.wikipedia.org/wiki/Quicksilver_(software)

2. https://www.folklore.org/Switcher.html

3. https://en.m.wikipedia.org/wiki/Sherlock_(software)#Accusati...

Oh, weird, I was just thinking of Memory Shift[1] earlier this morning, hadn't thought of it in probably 30 years, and now that switcher article mentions it.

[1]https://www.atarimagazines.com/creative/v9n12/82_MemoryShift...

Or maybe they'll buy it, like they did with Workflow.

This is what Apple Intelligence should have been. Instead we got Genmoji

Shameless plug here - we are working in a similar space, and have started at the copy & paste layer. We are calling it TabTabTab[0]; while we have a waitlist as we are working on our onboarding, you can already give it a go on[1]

The main actions are option+c to copy, option+v to paste. Take the front page of hackernews and capture it with option+c (use cmd+a to select all, if text is selected we pickup text, otherwise we pickup an image); and then go to sheets.new or numbers and hit option+v, see your Spreadsheet getting built. It works everywhere.

There's spells option+s (custom GPT per app / URL) and agent mode (option+I) but give magic copy/paste a shot. More examples here[2]

[0] https://tabtabtab.ai/

[1] https://tabtabtab.ai/TabTabTab.dmg

[2] https://tabtabtab.notion.site/tabtabtab-user-notes?pvs=4

I like shameless plugs that are honest about it. I'll surely take a look :)

I will email you with the link, we don't have that bit automated. Here it is in the meantime - https://tabtabtab.ai/tabtabtab.dmg

It seems like a power tool for early adopters who are willing to take a few risks and something safe for the masses are two different things?

Apple has a target on its back. Anything they do absolutely will be hacked. There are unsolved problems securing tool-using LLM’s.

I wonder what Sky does for security?

It’s really obvious that AI is something that is very rough around the edges and companies known for delivering polished projects are having a lot of trouble delivering. For example, Google should have been the leader in AI, but I think OpenAI got first mover advantage, because they didn’t want to ship a product that hallucinates. If you look at the pre-ChatGPT days, all the news stories were what happens when AI goes wrong not forgiving the failures and accepting that in some areas it can get things really right in a more efficient way.

Every time I read what Apple is doing with Siri and Apple intelligence, there is power politics involved and some sort of major incompetence in terms of risk taking/assessment

I’m interested in something in this space that can work with local models. I absolutely do not trust inference providers with my data without very careful risk analysis.

Sky honestly does not even look that impressive from the video on their website. So it can save calendar events if you type something into a prompt. Wow....

What am I missing ?

> So it can save calendar events if you type something into a prompt

They didn't type it. They hit a button, it suggested adding it to the calendar and they approved it.

You can still find some of the flow forced, but I think you've missed the starting point here.

Nothing really! Saving one or two clicks for people that can't bother to learn any UI. The true automation I needed which leads me to leave macOS is workspace arrangement. Now I'm using Sway on Linux and I now that Super+1 is Emacs, Super+2 is the browser, Super+3 is where the terminals are,... Typing the same shortcut again brings me in the last workspace. And no sloooooow animations in the way.

This stuff is readily available in the macOS ecosystem, but I don't fault anyone for jumping ship for these reasons. It's the same reason I did, before I figured out how to do it in macOS land by way of being required to work on macOS anyway.

I know how to them in macOS, it's just not pleasant. I still use macOS, but for very specific workflows. My daily computing is on Linux where I got everything to behave how I like.

You can set all that up on macOS and disable animations.

Doesn’t macOS already have add to calendar when it supports dates? I too was confused why they used that in particular. Nothing they Showed made me go wow that’ll save me time. And what makes any of that “ai”?

It really seems like a solution in search of a problem.

It is inferring the UI content. There's a bit more to it than it looks at first glance.

What does "inferring the UI content mean". You said you have some knowledge of how it pulls it off, is it using accessibility API and apple events behind the scenes? Those are the only two APIs I know of on mac that can do this (besides just grabbing the pixels of a window).

I've always been hoping that the rise of LLMs leads to a resurgence in apps being able to respond to apple events, since it seems like a perfect fit.

It's not about the app, it's about it existing outside of Apple and their push for Apple Intelligence.

I mean the article claims that it's good - better than all other existing solutions, and hence why it's worth talking about in the first place.

It is _much_ better than all the other MCP servers I tried for the Mac so far (I've been doing one myself and know a lot about Mac automation, so I've benchmarked them, and none can do the kind of UI analysis I see in the demos).

To ask it to organize the download folder seems great - more examples: https://www.macstories.net/stories/sky-for-mac-preview/

My brother, you haven't drunk the right coolaid.

It's not impressive because text is a shit interface for interacting with a machine. GUI's are much better and we established this in like 1982.

Also what they are doing in that demo is inserting a machine shim in an interaction between people, which is what people, time and again, say they do not want.

Seems that it can also text your friends (so you don't have to)

i am building in a similar space, while i see where you are coming from I am extremely curious on what "a mind blowing" or "i need that" experience looks like to you from a desktop AI product

Unrelated to the article content:

The styles on this blog overwrite repeated links with the color of the main text (see `.duplicate`). What's the design idea behind this? I doubt it holds, since the links are still there for keyboard users, etc.

Either make your hyperlinks visibly distinct (ideally underline them) or remove them.

If you have already been there, why would you go again?

No human visitor opens all of the links from top to the bottom, so the repeat occurrence might very well be unvisited. After all, we have `:visited` for that.

But even if you are questioning why anyone would click it -> well why have the link there in the first place then?

I did that because people complained that the links were too visible (because the site is actually a Wiki, there are often several copies of the same link throughout the text).

Over the past 20 years there have been many schools of thought about it, and the only way I could get people to stop kvetching about it was to have the links become more distinct on paragraph hover but still render them in a distinct color (and oddly enough, the rate of complaints went way down when I shifted to dark red).

I have no sane explanation for this other than ascribing it to user-driven UX research over a massive amount of time (when compared to most other websites I know).

But (fun fact) the duplicate links inside paragraphs do wonders for RAG chunking and vectorization. :)

Ha, did you just update the styles based on my complaints? :)

I believe there are multiple reasons why the styles are better now. Even the first-use red links are much more accessible now. And you are now presenting the same content to the users using screen readers or other assistive technologies as for the typical users.

By the way, I think your analysis of the Apple situation sounds absolutely spot on.

And I did that with just plain HTML, which is madness in today’s web :)

I’m ok with owning a Mac if it only means that AI tools don’t have root access to the machine.

It's interesting that in the sky.app video it uses Google Maps to find a bar, not Apple Maps. Seems related to much of the discussion here.

first iteration Sky

second iteration Net

When has Apple’s strategy ever been to be first to follow on the latest bandwagon? I thought it was more about making great products which integrate design and technology. They have a launch cadence and they take their time, is that really the same thing as “failing?”

There are plenty of places you can find rushed AI projects. Perhaps being slower and smoother is itself a differentiator?

Um. What exactly does Sky do? Can anyone list the top 20 things?

It does just one thing really. It bridges the gap between what you want to do and how to do it.

What does this mean?

Simple. It acts as a dynamic catalyst, seamlessly aligning strategic intent with operational execution. By abstracting complexity and enhancing cross-functional synergies, it empowers stakeholders to realize outcomes without friction—transforming ambition into scalable impact through an intuitive, always-on intelligence layer.

Have you tried ZomboCom's AI solution?

We're talking about an Apple product here. We don't talk about that stuff.

We do periods. Because it just works.

What's a computer?

Last century, we showed you how to Think Different.

Now you'll never need to think again.

Can you provide an example or two demonstrating this?

So, it saves you some clicks

Based on the intro video on their website and the linked article, it's an LLM agent for your Mac, so in short, what Apple Intelligence should've been from the very start. Create calendar events, look up information, send messages, etc.

These all seem like things that are more work to instruct and AI to do then to just do yourself

Where a lot of these tools fail is "first instruction accuracy". Put another way, what percent of requests are fulfilled accurately without needing to clarify your intention to the AI.

For voice assistants to really work, they need to nail the request on the first attempt in 95%+ of cases.

When the first time success rate for a request drops too low, it indeed becomes way easier to just do it yourself rather than asking the voice assistant for help.

The fact that we haven't solved this problem for basic things like turning on or off a TV doesn't give me much confidence in the future of voice assistants. I find myself frustratingly repeating commands to Siri way too often.

But does it do it well, or are we focusing on how nice the Mac UI is?

Author here: it apparently does it _very_, very well. Based on the few sample interactions I've seen and read about, it makes Claude Desktop look like a Ford Model T in terms of polish and overall look and feel.

I'm not overly crazy about the logo and buttons underneath the interactive windows, but the way it can infer window content and structure from any app (either by leveraging Accessibility or by just hacking at it I'm not sure yet, but one of the demos requires good UI introspection) makes it seem much better integrated than anything else I've used so far.

In the showcase video on their site I saw it generate an AI reply to friend group.

  A.C.K.: Wanna get [drinks] before?
  
  Sky: Drinks at Trick Dog before dinner work for you guys?

That's kind of cool I guess. I can imagine it useful for those friend groups that sometimes get demanding or overwhelming.

I can also imagine it being really good at getting a high valuation and selling for millions/billions of dollars.

Do people want AI to surprise sign them up for social engagements?

Yeah, I mean AI bros from valley can partner with "small businesses" like bars, clubs, bowling alleys, restaurants and fill those rooms with more people by auto signing them. Life is all about experiences as a service. Not the boring old stuff that one can buy outright from the stores.

> for those friend groups that sometimes get demanding or overwhelming

okay now I agree with boomers

i don't need mac apps, i need chromebook with apple hardware.

Reply on news.ycombinator.com