Hacker News Reader

Comment by olliej | Hacker News Reader

It's really amazing when we still get those text ones and nowadays you can literally select the text in many of the images and copy/paste into the input field.

View on HN

I've never seen a text version that lets me select the text, that's bizarre

I’m assuming that he means that on (for example) Mac you can select text from any image and copy paste it.

https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use...

100% correct, I assumed windows also let people do that given text recognition is apparently trivial now (to the extent it's annoying - trying to drag images and get text selection instead is annoying :-/)

Windows has gone sort of the opposite way, copy/paste is now often hindered if the engine recognizes the string to be sensitive or otherwise un-wise to copy to your clipboard.

I've had a few instances on Windows 11 and surrounding software where ctrl-C as well as the context menu entry for 'Copy' were greyed out for this reason when skimming through logfiles, presumably because there was something about the line that triggered the MS "that's a password!" regex; stuuuupid stuff.

I think Windows now does it as well, but of course (as all things Windows) it works only in very few apps (forget Win32 ones, for example).

@_rutinerad got it - on macOS you can select text in any image, and I just assumed you could do that on windows as well (I figure in the context of linux it would be much more dependent on specific configuration so unilateral assumptions on behaviour would be questionable).

It's honestly annoying as it frequently interferes with dragging images out of safari, except on those occasions when I do want the text when it's super useful. I think the iOS interface just tells you there's text in an image or photo and gives you the option to copy it rather than cursor based selection you get on Mac.

[edit: from other comments it sounds like windows can do this but it's not always present, and not present in all circumstances, which makes me wonder how many cases in cocoa/uikit/swiftui it does not work]

Am I the only one paranoid enough to think that this means Apple is now indexing even the text content of images stored on it's users computers?

That's literally a feature of the platform. If you open the photos app and type text in it will give you the photos containing that text.

If your concern is "apple is harvesting my data" then no. All of apple's various analysis systems ("AI") are entirely local. This does mean you get a bunch of duplicated work as every device redoes the same analysis but on the other hand it saves you from "how do we defend against a compromised network".

> All of apple's various analysis systems ("AI") are entirely local.

Even if that were true (I couldn't say, and I don't think anyone who doesn't have access to Apple source code and production systems could either) that wouldn't preclude Apple harvesting the results of said AI analysis.

In fact, doing the analysis on users' devices would represent a shift of that processing from cloud to edge, representing a significant savings for Apple or anyone else in a similar position.

Apple's literal marketing message is that they do all the processing on device. It's not a shift for Apple, as apple has never done this analysis in the past, and only started doing the analysis once it could do it locally.

The use cases we're talking about also don't work on a cloud based analysis, as you can't have text selection block on network uploads (generally slower than downloads), and it would require uploading every image you open to apple which would presumably be a lot of traffic, and an obvious privacy nightmare. It would also break for users who turn on the e2ee everything mode for iCloud.

It's wonderful (for Apple's bottom line) that you believe these things. I assume you can show source code and provide access to production systems to verify?

Reply on news.ycombinator.com