Google seems to have made an official post on Reddit describing the feature set in detail:
https://www.reddit.com/r/Android/comments/1tb8xls/introducin...
[Edit]
And, the feature set references the 'AI mouse pointer' from this Deepmind blog..
https://www.reddit.com/r/Android/comments/1tb8xls/introducin...
[Edit]
And, the feature set references the 'AI mouse pointer' from this Deepmind blog..
Most people don't really seem to care about data collection when it comes to AI usage. A lot of people who will feed Gemini/ChatGPT/Bing/Claude/shady clusters across the internet for bargain bin prices/Mistral every detail of their lives will probably be fine with Gemini as long as it doesn't interfere unnecessarily.
You can point or select anywhere on the screen and it understands and searches the context. If you select a text block, even text inside an image, it allows to copy or search the text online. Otherwise it can search the image.
I use it often. It's intuitive and fast even on non-flagship phones.
I'd wager their A/B tests went well enough to warrant a port from phones to their new "Chromebook".
Do we use the same Android Gemini assistant?
Because the one I use does that and it has object detection smart enough to be intuitive. It usually gets it right when I point something on the screen. And when it doesn't, I can circle around the thing or just click again.
This Instagram post for example, it automatically highlighted the entire person, but I wanted to know about the shoes. I then clicked once on the shoes and it knew exactly what I wanted and gave me the info in about 2 seconds:
This is useful to non tech savvy folks. Not just to us hackers.
Object detection is mediocre at best. Circling things and using their AI editing features works, but the artefacts confuse Lens and other image parsing systems. Extracting objects from images usually mostly works, but it's not on par with what Apple had long before Google built it.
The difference remains that the Gemini app on Android requires activation. You cannot tap a button or click a link while you're on the Gemini screen.
The video isn't on the linked page anymore, but it's here: https://deepmind.google/blog/ai-pointer/ and here: https://www.youtube.com/watch?v=pZNzfQLgGsA
It's an absolute privacy nightmare for most people, but if we ever get enough RAM and compute to run this stuff locally, I think this can actually make a new paradigm for user interaction, something with lisp machine self-customisability but for people who don't know anything about computers.
And if it doesn't work, it'll be the most horrific, messy, useless UI humanity has ever invented, and we all get a new funny meme to laugh about Google. Win-win!
That assumes you intended to use AI. People are going to accidentally upload random private content to google.
At least one DE I've used (MacOS? KDE?) even had it as an official macro that would make the pointer 10x bigger when you shook it
> If a friend sends you a picture on your phone and you need to email it from your laptop, the file is just there — no need to email it to yourself.
So are there really people who will email a photo to themselves from their phone to… send the photo in an email?
Interesting to note that there is no mention of processor or operating system in that post. I’m guessing that it’s Android in a laptop form factor which I suppose might be something that some people would want, but I’m not one of them.
My wife and I have home offices at opposite sides of the house with hardwired desktops and Wi-fi APs, but we can't AirDrop to each other as we're out of range for it.
If you want to send a photo to your friend from your iPhone, just click on the photo and click the "share" button, then you have many options, including sending it via Email..
What am I missing?
There is also localsend, but plainapp needs to run in only one device.
Can be self-hosted, if that's a concern.
(There's a longer backstory, but this is, suffice to say, frustrating.)
I all the time use my phone as a camera (esp. for coin photography) than e-mail the photos to myself as the most convenient way to get them on my desktop where I can edit them with GIMP etc.
When on wifi, the photo backup upload starts immediately. If it doesn't (possibly due to your settings, this used to be my issue) you can manually open the photos app and tap the backup now button.
Google Drive would be another option to transfer, but would be more work (about same to "share" as email, but less convenient to access on desktop).
The e-mail way is actually quite convenient since on the desktop you can just download all the photos you sent in one go - they appear as a zip file that you can then just extract to your working directory, rather than having to save one at a time.
You install this on your phone (iphone or android), as well as on your computer (linux, windows or mac), then you can send files directly between any of these as long a they are on the same local network.
You just select your photos, click on "share", then select LocalSend and it'll show you the list of destinations to choose from (wherever you are currently running LocalSend), click on one and you're done!
On the receiving side you can select the directory where files will be transferred to, and if you set "Quick Save" option to "Favorites" then you can make your phone a favorite to avoid being prompted whether you want to accept each file it is sending.
So, it's literally just: select photos, share->LocalSend, click on destination
I got mad when I bought a Chromebook thinking it was a cheap laptop I could install any OS on only to find it was boatloader locked and the model I bought hadnt been cracked yet. Say nothing of all of Google's recent practices with Android. This whole thing just sounds like the plague.
...as computing shifts from operating systems [to intelligence systems](TKTK)...
edit: hah, maybe someone from Google saw my comment. This has now been fixed and TKTK replaced with https://www.reddit.com/r/Android/comments/1tb83gy/making_and...
I'm really enjoying reddit just completely roasting the entire concept in the comments.
I had an instance of it this morning: Claude proposed a shell command containing a URL and it used this format, which is broken in context.
I mean if you think about it, the type of person to own an android phone and care enough about phones to join a community is pretty much guaranteed to only be a tech geek.
A disaster from the first step.
This is actually a good use case for AI. My university sends a lot of newsletters with several events in free text format; all I want is to be able to select one of them, have an LLM parse the title, date, location, and category, and put it in my calendar.
Still, I'm sceptical this will work. Samsung phones supposedly have this same feature, and it works 1/10 of the times. Pasting it to ChatGPT and tell it to add the events to my calendar works fine, but the bottleneck is always the project managers in charge of the UI. Of course, having a small local model and being able to choose my own right-click items like I could in 1995 would be an actual solution.
Like... my dude that's way even slower than drag&drop the text on a light right next to it!
Same later on about changing the calendar appointment from whatever to 8pm... he is behind a desktop with a mouse, just input the number or click on the arrows to adjust.
I bet some people will mention that those are "just" simple to understand examples or that it's great for accessibility ... but it's not. It's not reliable enough for complex cases and not reliable enough for accessibility. So... yes JUST basic examples that are slower than other means.
PS: I did prototypes using voice and pointing in XR and yes that paradigm IS powerful, it's just being multimodal.
Yeah but what about Windows Explorer? They've been passively blocking SMB access forever at this point (by disallowing ports below 1024).
I would not be surprised if Googlebook's file browser goes via the cloud.
May chaos take the world!