Hacker News Reader

Comment by perfmode | Hacker News Reader

> We believe in training our models using diverse and high-quality data. This includes data that we’ve licensed from publishers, curated from publicly available or open- sourced datasets, and publicly available information crawled by our web-crawler, Applebot.

> We do not use our users’ private personal data or user interactions when training our foundation models. Additionally, we take steps to apply filters to remove certain categories of personally identifiable information and to exclude profanity and unsafe material.

> Further, we continue to follow best practices for ethical web crawling, including following widely-adopted robots.txt protocols to allow web publishers to opt out of their content being used to train Apple’s generative foundation models. Web publishers have fine-grained controls over which pages Applebot can see and how they are used while still appearing in search results within Siri and Spotlight.

Respect.

View on HN

When Apple inevitably partners with OpenAI or Anthropic, which by their definition isnt doing "ethical crawling", I wonder how I should be reading that.

They already partnered with OpenAI, right?

To use their APIs at a discount, so what?

Apple aren’t paying OpenAI anything:

https://www.bloomberg.com/news/articles/2024-06-12/apple-to-...

That's a big discount

That's quite a discount! ;)

The art of the deal

They paid in exposure, unironically.

Same way as the other parts of their supply chain I suppose.

In theory Apple could provide their training data to be used by OpenAI/Anthropic.

It isn't "apple proprietary" data to give it to OpenAI.

Also the bigger problem is, you can't train a good model with smaller data. The model would be subpar.

"Good artists copy; great artists steal"

- Famous Dead Person

I mean they also buy from companies with less ethical supply chain practices than their own. I don’t know that I need to feel anything about that beyond recognizing there’s a big difference between exercising good practices and refusing to deal with anyone who does less.

You shouldn't believe Big Tech on their PR statements.

They are decades behind in AI. I have been following AI research for a long time. You can find best papers published by Microsoft, Google, Facebook in past 15 years but not Apple. I don't know why but they didn't care about AI at all.

I would say this is PR to justify their AI state.

Apple used to be at the edge of AI. They shipped Siri before "AI assistant" went mainstream, they were one of the first to ship an actual NPU in consumer hardware and put neural networks into features people use. They were spearheading computational photography. They didn't publish research, they're fucking Apple, but they did do the work.

And then they just... gave up?

I don't know what happened to them. When AI breakthrough happened, I expected them to put up a fight. They never did.

>I don't know what happened to them. When AI breakthrough happened, I expected them to put up a fight. They never did.

Apple always had the luxury of time. They work heavily on integrating deeply into their ecosystems without worrying about the pace of the latest development. eg. Widgets were a 2023 feature for iOS. They do it late, but do it well.

The development in the LLM space was and is too fast for Apple to compete in. They usually pave their own path and stay in their lane as a leader. The impact on Apple's brand image will be tarnished if Google, Meta, OpenAI, MS all leapfrog Apple's models every 2-3 months. That's just not what the Apple brand is associated with.

> I don't know what happened to them.

Tim Cook happened. The fish rots from the head down.

It's more like "Steve Jobs died". I'm not sure anyone could have replaced him.

Thats a lame excuse. And a strange attempt on deification.

The predecessor is no reason for the current head to slack off.

One problem with Apple's approach here is that they were scraping the web for training data long before they published the details of their activities and told people how to exclude them using robots.txt

Uncharitable.

Robots.txt is already the understood mechanism for getting robots to avoid scraping a website.

People often use specific user agents in there, which is hard if you don't know what the user agents are in advance!

If you're using a specific user agent, then you're saying "I want this specific user agent to follow this rule, and not any others." Don't be surprised when a new bot does what you say! If you don't want any bots reading something, use a wildcard.

Sure, but I still think it's OK to look at Apple with a raised eyebrow when they say "and our previously secret training data crawler obeys robots.txt so you can always opt out!"

I've been online since before the web existed, and this is the first time I've ever seen this idea of some implicit obligation to give people advance notice before you deploy a crawler. Looks to me like people are making up new rules on the fly because they don't like Apple and/or LLMs.

I stand by what I said.

Apple are saying you can opt out of their training data collection using robots.txt.

But... they collected their training data before they told people how to opt out.

I don't understand why me pointing that out as "eyebrow raising" is controversial here.

It's not controversial, it's just not how the ecosystem works. There has never been an expectation that someone make a notification about impending crawling.

It might be nice if there were categories that well-behaved bots could follow, as noted above, but even then the problem exists for bots doing new things that don't fall into existing categories.

My complaint here isn't what they did. It's that they explain it as "here's how to opt out" when the information was too late to allow people to opt out.

I think that's disingenuous of them.

It's been common knowledge for anyone running a web server since 1994.

I don't think you are reading my posts in full.

Yes, but given the lack of generic "robot types" (e.g. "allow algorithmic search crawlers, allow archival, deny LLM training crawlers"), neither opt-in nor opt-out seems like a particularly great option in an age where new crawlers are appearing rapidly (and often, such as here, are announced only after the fact).

That seems like a potentially very useful addition to the robots.txt "standard": Crawler categories.

Wanting to disallow LLM training (or optionally only that of closed-weight models), but encouraging search indexing or even LLM retrieval in response to user queries, seems popular enough.

Assuming well behaved robots.

They documented it in 2015: https://www.macrumors.com/2015/05/06/applebot-web-crawler-si...

> Using our web crawling strategy, we sourced pairs of images with corresponding alt-texts.

An issue for anti-AI people, as seen on Bluesky, is that they're often "insisting you write alt text for all images" people as well. But this is probably the main use for alt text at this point, so they're essentially doing annotation work for free.

I think it is entirely morally consistent to provide alt text for accessibility even if you personally dislike it being used to train AI models.

It's fine if you want to, but I think they should consider that basically nobody is reading it. If it was important for society, photo apps would prompt you to embed it in the image like EXIF.

Computer vision is getting good enough to generate it; it has to be, because real-world objects don't have alt text.

I actually use Claude to generate the first draft of most of my alt text, but I still do a manual review of it because LLMs usually don't have enough contents to fully understand the message I'm trying to convey with an image: https://simonwillison.net/2025/Mar/2/accessibility-and-gen-a...

Context not content.

Why would photo apps do what's "important for society"?

Annotating photos takes time/effort, and I could totally imagine photo apps being resistant to prompting their users for that, some of which would undoubtedly find it annoying, and many more confusing.

Yet I don't think that one can conclude from that that annotations aren't helpful/important to vision impaired users (at least until very recently, i.e. before the widespread availability of high quality automatic image annotations).

In other words, the primary user base of photo editors isn't the set of people that would most benefit from it, which is probably why we started seeing "alt text nudging" first appear on social media, which has both producer and consumer in mind (at least more than photo editors).

> Why would photo apps do what's "important for society"?

One would hope they're responsive to user demands. I should say Lightroom does have an alt text field, but like phone camera apps don't.

Apple is genuinely obsessed with accessibility (but bad at social media) and I think has never once advocated for people to describe their photos to each other.

> An issue for anti-AI people, as seen on Bluesky, is that they're often "insisting you write alt text for all images" people as well. But this is probably the main use for alt text at this point, so they're essentially doing annotation work for free.

How did you come to the conclusion that those two groups overlap so significantly?

Well that's easy, I read their posts where they say it.

So you found a couple people expressing this conflicting view and assumed it applies to a larger group? Doesn’t sound very reliable to me but I see this all the time and it makes sense if you look at it as a mechanism to explain the world .

This is a well known fact. A bunch of AI researchers tried to migrate to the platform from Twitter but got a ton of hate and death threats from other users so they went back. Bluesky has a pretty strong anti-AI bias and the community of folks talking about it despite that is very small.

  > this is probably the main use for alt text at this point

Alt text gives you 2k characters. All I gotta say is there's quite a bit of poisoned data

Respect, but its going to be terrible compared to every other company. You can only hamstring yourself so much.

Respect actions, not words and PR.

Gotta polish that fig-leaf to hide Apple's real stance towards user privacy: arstechnica.com/tech-policy/2023/12/apple-admits-to-secretly-giving-governments-push-notification-data/

> Apple has since confirmed in a statement provided to Ars that the US federal government "prohibited" the company "from sharing any information,"

I mean if you throw out all contrary examples, I suppose you are left with the simple lack of nuance you want to believe

All examples contrary to what? Admitting to being muzzled by feds?

Take all the space you need to lay out your contrary case. Did the San Bernadino shooter predict this?

You literally said that we should disregard this example and focus on the “real” situation as evidenced by a different example.

It is exactly the same thing as saying “if you ignore the heads, these coins really always come up tails”.

Does the Chewbacca argument method ever work these days?

Reply on news.ycombinator.com