Rabbit: LLM-First Mobile Phone
rabbit.tech
rabbit.tech
And then there are so many questions about the hardware capabilities of this device:
- where is the inference running? I don't believe it's on the device. And if it's in the cloud then why make the claim it is under 500ms? Is that just a "don't you guys have low-latency 4g at home?" moment?
- how is that tiny camera capable of parsing a small-text table with 100% accuracy? the optics just don't allow it to.
- what's the battery life with that kind of usage? If inference is running on device it must be very low. Just thinking that my GPU pulls 50-100W on average (with spikes to 200W) just to suggest code I still don't think rabbit is doing anything on device. If it's cloud based then 4g is also a battery destroyer. Maybe that's why the device is so big: huge battery inside.
The "teach" session was definitely cool. But at this point it must be magic because there's no way that thing browsed to a discord server, authenticated with hallucinated credentials and it just worked.
it seems that this device doesn't run anything locally, everything is in the cloud.
The answer is absolutely to do these types of products on the devices we already own. There is only one thing stopping this type of innovation and experimentation.
Not sure if the Rabbit device is the answer but I definitely see why people are trying.
Thing is these AI devices will become actually interesting when they start being observers in your life too and pre-empting rather than the siri style ask and response and that just isn't possible on a locked down OS like iOS.
For apps w/o a web interface, presumably they have an Android emulator running somewhere and they controlling it with their LAM and feeding the results back to end users. They can even feed sensor data from the Rabbit phone (e.g. GPS) to the emulator.
It is a brilliant solution to the problems they are facing, but their cost model must be extreme. "No subscription required" means I am wondering what their "pay as you go" pricing will be!
There's no 5G support either, unless they've purchased an external modem. So even if you do live in an area with low-latency 5G, you aren't gonna get it.
I wouldn't be surprised if this turns out to be running just a cheap Android fork that's locked in kiosk mode with a single app (or webapp).
Same vibe as Humane not using the projector for almost anything and then explicitly saying "You dont have to use it" in an interview. Like why add it then
This is a product sold by Rabbit - and they hired TE to help as a design agency.
But it would be an omission if we didn't also include the fact that TE hardware often has weird flaws like OPz cases never fit together properly even years after release and the more recent problems with shipping the ko2 sampler and broken faders.
e.g. in this very device, the thin stem of this scroll wheel implies it's actually shaped like LEGO minifig heads, like 中, which doesn't inspire confidence wrt lateral plays. It could easily be made 土 or 王 or even 凸 shaped internally with only the middle part exposed, which not just eliminates play, prevents debris ingress, but also makes areas around wheel edges hidden so quality control around those areas are less of an issue. Or, the shell seen here is shaped like a bathtub with a lid closed at the backside, which means the cross section of the tub has to be ever so slightly tapered towards the front, which affects grip when picking up. And, the backside has no bumps nor there are creases in the side, which further compromises pickup. Or eg the painted wheel. It's going to be PITA to paint.
Granted I'm biased feeling always defeated by superb aesthetics of their products - but their design always seems, how do I say, not as deeply connected to engineering ideals just as it is with aesthetics as seen in Sony or Apple products.
But really, this is screaming to be an app. There is literally nothing this thing can do that the phone in all of our pockets can't. Especially integrated with our smartwatches for ease of use. So why have a separate device for it? It makes no sense.
I think the hardware is just there to make it not 'just another app' in the press, really. And that the app version will come soon after the physical product releases and will take over 99% of the installed base.
And of course if this takes off they'll be quickly acquired by Apple, Microsoft, Google or Samsung. I bet that's what they're aiming for.
https://www.theverge.com/2024/1/9/24030667/rabbit-r1-ai-acti... (via https://news.ycombinator.com/item?id=38933819, but no comments there)
https://techcrunch.com/2024/01/09/can-a-striking-design-set-... (via https://news.ycombinator.com/item?id=38932575, but no comments there)
A minor convenience when it works well. A major inconvenience when it doesn't. (And it doesn't take a lot of imagination to come up with nightmare scenarios.)
Love the gadget, though. I feel like I want to eat it or build a Lego castle around it, or just look at it more.
There's a fundamental tension here: either you limit the LLM to a set of fixed actions that a user can individually understand and confirm. Or you let it figure out what to do and how to do it given a higher-level goal.
In the first case, it's limited and not really better than a well-designed site or app. In the second it's powerful but can run amok.
E.g., did it really fill out that four page order form for that e-bike you asked it to order? Maybe it uses the debit card instead of your credit card and your checking account is overdrawn. Maybe it sets the delivery address to your brother's address. Maybe it orders two bikes or 10 or the wrong model or wrong size.
OR
It asks/confirms each step of the way so there's little chance for mistakes.
This is an inherent problem with any kind of delegation. You either micro manage the details or trust your agent to get them right.
Ultimately, an LLM only has the capability to take a string as input and output another string. Any other functionality (Uber, search, travel, etc) has to be manually programmed in via APIs. For the foreseeable future, a fixed set of actions is the only way to do this reliably and is what this device does so talking about the second case of an LLM run amok is a moot point.
Pretty compelling price, and I'm certain the vision of AI agents that can use any existing app or website to take actions on your behalf is the future of computing. But there's no room in my pocket for a second device. I don't see how a device is going to succeed when an equivalent app for existing phones seems around the corner.
i mean there are technical limitations and tradeoffs to running LLM-size models locally. doesnt help to ascribe it to lack of foresight when it is a known Hard Problem.
Actually the only device with such features is a recent smartphone. Wouldn’t this be better as an app?
Ah already have ChatGPT installed, that'll do :-).
It's just the fact that the models that can currently run on a mobile device are not effective enough. It is about the most memory and compute intensive type of application ever. In particular, models that can reason or follow instructions reliably and for general purpose are too big to run quickly on mobile devices.
They are putting models on phones, but they do not have general purpose assistant capabilities.
This may change in the next few years though.
Rabbit means to solve that by creating a "LAM", a "Large Action model", which is a service by Rabbit that will click interfaces for you. I'm not sure this is the right approach - if it is successful, it will lead to more centralisation around Rabbit.
I agree this is a problem, but I feel a better approach would be to have a market of agents that for a small fee actually handle the whole transaction for you. So there might be multiple parties that say they can buy Delta Flight DL101 tomorrow 21:10 for various prices - some might be a service like the Rabbit LAM, others might be booking platforms, and there might even be airlines themselves. And now an agent-concierge that you choose once at the start will look at all the parties, and then pick and buy the right flight for you. This will make the problem a problem of an open market, where good speedy service is promoted, and prices get ever lower. And if the Rabbit LAM gets outcompeted by an ever better speedier solution, that would be a good thing. (This will also allow us to move away from our current dreaded attention-based economy where e.g. a booking websites tries to exploit your required presence during waiting times, which the LAMs would also solve, but, like I said, let's not move towards more centralisation.)
This seems to be a Langchain wrapper, where the Langchain is a prompt + retrieval based on a few documents.
ex. `https://github.com/joaomdmoura/crewAI-examples/tree/main/sto...` ``` BrowserTools.scrape_and_summarize_website, SearchTools.search_internet, CalculatorTools.calculate, SECTools.search_10q, SECTools.search_10k ```
https://openadapt.ai is an open source app that runs on your local machine that clicks interfaces for you -— but only for repetitive tasks that you show it how to do.
What I'd like to see is the Smalltalk approach: data providers that are able to send/receive messages, and can be connected together to achieve a goal. Even better if the connecting is done by the "machine" after I issue a command.
The LAM is a genius hack to get around the thousands of closed gardens that apps have created.
It also may have been easier than teaching an LLM how to make tons of API calls, and if done right I presume their LAM adapts to UI changes, vs writing integrations against breaking / deprecating APIs.
90% of use cases will be covered by an official API.
They’ll cover the other 10% with “teaching”. Essentially you telling the AI what the lazily written markup actually means. Then they save it into an automation template. QA teams have only been doing that for the better part of 3 decades.
I know a company that employs a building of a 1,000 people doing nothing but performing 1 click. So they put a human in the scraping /automation loop so they don’t violate the site/services TOS.
Good luck with that.
VC backed and publicly listed companies need endless growth, user-centric systems like what Rabbit is offering break those business models apart. Which is why I predict everyone is going to be fighting super hard against making UIs that just get shit done.
Watching the keynote, I found myself thinking how unhappy Uber would be with skipping over interacting with them entirely: there's no "Uber experience" you have when you're in the car, so what do you get from Uber that any random company with a tie in to Rabbit can't get you?
Option 1: a shift in devices/model like Rabbit pull the magic carpet out from under companies like Uber, and everything becomes purely transactional.
Option 2: rabbit-like market creates exclusivity-based need, to ensure Uber is the number-one (or only) rideshare choice, so it doesn't matter that customers aren't "experiencing" Uber. Uber relinquishes the experience to the agent (unlikely).
Option 3: Uber et al wage war against agents and make their use impossible
I can't find the exact ~2010 article from before being bought by Apple. I remember in an interview they were talking about making a web agent that could operate and perform tasks on any website, to avoid being locked out by APIs.
They have the right hardware for it and they have all the motivation, with their focus on on-device processing. OTOH they also have a pretty bad history with their AI assistant.
This isn’t a case where they had to choose one or the other (not saying that you’re saying that). It seems like they were genuinely caught off guard.
They let almost the entire original Siri team walk didn’t they?
They (Apple) are now in the position of being seen as laggards, caught with their pants down by companies releasing products whose core conceits are built atop inefficiency of their "core" models.
For both Humane and Rabbit, it's hard for me to imagine that there's enough "there" there for these to be beyond niche products that don't get starved out by the border-expansion of Apple / Android over the next few years … but I would have also guessed that A+A would have been further out front of this.
The most expensive app I pay for is Telegram Premium at 30$ per year. And that's really worth it for me for the translation otherwise I would have thought twice.
It’s the dedicated appliance that’s the genius part and what makes it feel like magic.
Yes, google or apple will probably ship LLMs on their phones eventually and make an overall more compelling product than Rabbit _if_ you're only willing to have one device.
Considering the relatively low price (and assuming they actually deliver it in a timely fashion) I can see this being useful as a pocket assistant between now and whenever google et al eventually catch up, and potentially past then if rabbit keeps innovating.
Lyu hit the nail on the head when he said that smartphones are mostly for entertainment/wasting time these days. I definitely know of friends who want to (or try to) go smartphone free, but the apps mentioned in the keynote make it a large inconvenience. A $200 device that offers much of the convenience of a smartphone without the distraction sounds like a good fit for that crowd.
I'm very glad they exist, but I'm always a bit flummoxed by what they make and manage to sell.
[1] https://www.thelightphone.com/shop/products/light-phone-ii-b...
This isn't a phone though right? The title seems wrong.
For example, if I wanted to do a simple find-and-replace on my 2021 M1, I'd have to use something like grep (a tool originally written by Ken Thompson in 1973—a half century before my laptop was built) and look up flags and syntax I inevitably constantly forget. We should have been past this at least 15 years ago.
I'm not sure what conclusions to draw from this observation however.
I would argue that it’s not that we’re bad at using tools that we designed for humans, but more that multiple shifts in the tech world over the last 50 years have intentionally (and unintentionally) broken those tools. AI does not need to use the tools at all, it knows the knowledge that the tools gate-keep and it can write code that does the data manipulation that underlies the most powerful software.
It essentially side-steps the whole mess that is the current tech industry (and to be honest a bunch of other industries).
Dogs were used to rotate roasting spits in Britain during the Early Modern era: https://en.wikipedia.org/wiki/Turnspit_dog Does this mean humans were inferior to dogs at the task? Absolutely not. It was just a more efficient alternative based on what was available to them at that point in time.
I think that was always the dream, even in early 20th century sci-fi. It's not a failure, it is a success.
However, in my opinion it's interesting that the dreamers underestimated the difficulty of robotics. Almost everyone thought that the manual jobs will be the first ones to be replaced. Right now it seems the other way around.
Pretty much any text editor worth its salt has find and replace, regex compatible, features. Even the extremely rudimentary TextEdit on your MacBook supports it.
The fact that foundational tools like grep still work over half a century later is a feature, not a bug, of our field.
How many cloud based services (most if not all of which rely on those foundational half-a-century-old building blocks, by the way) that are pushed to customers today will still be around in half a century?
Even a typical Product Manager, let alone a regular layperson, would have no idea where to start with regex, so it's a super weird thing to bring up. Of course grep is a great tool, just how the hammer is a great tool (invented around 3 million years ago); but we've moved past that. You can't use a hammer to do EUV lithography.
Supporting regex covers that extra mile that some advanced power users might need, since you were referencing obscure grep flags.
What kind of search and replace are you doing that is akin to EUV lithography?
Why not burn a couple hundred watts to do find and replace (/s)?
As a bonus, those capabilities don't require uploading your data to the cloud or risking LLM hallucinations.
Ignoring that half those services are dependent on the cloud already, this is such a bold and hard problem. A solution would be magical but LLMs are our best chance.
Without clearly defined protocols (eg SMTP, CalDav) and a ton of boring to write and use case specific glue code, you’re really dependent on LLMs to make those M*N connections from one service/format to another. An LLM can turn any current or future input format into any current or future output format. That’s really powerful and basically unsolvable at scale with the tools we have now.
Yeah except they actively don’t want you to have access to your data. They want you locked in to their shitty platform. And these companies will actually fight your ability to get data out, like Reddit or Twitter when they shut down their public APIs.
I want easily pluggable/connectable systems for everything. I want to see all my messages and all my contacts and all my stuff in interfaces I choose. The future is definitely not searching email, Whatsapp, Signal and Instagram to find the message my friend sent me the other day.
We agree that those things are things we do want, but do you really think we’re getting any of them here?
I think it’s reasonably legitimate to go: cool idea, I want it on my phone, as an app.
There is no reason for this to be, and indeed, mostly isnt a hardware device; it’s mostly a cloud service with a mystifying and useless custom piece of hardware tacked on the edge.
You’re doing on device inference? Cool, sign me up, you have my attention.
You have a cloud service for AI integration with existing apps? Mm… like, how is that different from any of the other AI startups?
If you differentiator is “my SaaS comes with a plastic box and no subscription fee” you gotta expect people to laugh, it’s a joke.
What does the oldness (and long usefulness) of grep have to do with the problem you described, which is the very hard problem of translating natural human speech into machine commands?
And of course, looking at Linkedin, the founder(s) are NOT convincing either.
Or maybe this is another VC-backed sale price :)
"No subscription" definitely sounds good to get people in the door but it seems short sighted in the long run, can't be making much on $200 and the more it's used the more that margin is eaten. Presumably they will roll out a sub at some point for extra features but then you get the backlash of "You said no subscription".
EDIT: it's 100% the razor model, they want the device out there so they own your interaction with "service providers"; i.e. they take a cut of everything to do. Middle men.
It's not much different than Sony selling PlayStations at a low price but making money off the games.
I also have a hard time believing it will work as well as they say it does gen-1 at a $200 price point. I'm very skeptical.
The teenage engineering design is nice though, looks like something out of the movie Her.
It's a vimeo embed
Edit: My feedback is that it's too slow. I wish the features were shown off before I had to get a layman's education for several minutes. I'm also really concerned that his uber will arrive before the pizza.
I like the device. It seems easy enough for my elderly father to use, and he won't lose it as "one of the websites" he can ask for help. There's something important about the focus a physical device brings; he can remember how to use it, where it's at, etc. The GPT UI I built him gets lost in his bookmarks, and he forgets it exists.
Like safari on iOS used to turn any video on browser to a standard video player window with normal controls.
Funny enough, I think them releasing only a video perfectly aligns with their product and why I think this will fail. They seem to have completely overestimated how often people want to read or write and underestimated the number of scenarios where needing to talk or listen to my phone is inappropriate.
I'm assuming if you had this phone, you'd feel like just as much of a psycho if you had to dictate every command to it while sitting in that same waiting room.
I’ve read the whole thing and I still have no idea what it does or what problem it solves. I’m not watching a Keynote to find out either.
Who made this?
The design is by Teenage Engineering but is not sold by them.
Problem to solve? Why would that matter? It's a fashion lifestyle product.
- the keynote video isn't scrollable behind the giant "share" button - the text on the keynote page is incredibly difficult to read (small text + very low contrast) - the homepage is full of dizzying animations that don't tell me anything about what the "phone" is actually supposed to do - "How it works" is just a set of animated cards. Once you figure out they're clickable (hoverable?), they spin around to reveal... half-formed ad copy? I wanted to know how it worked.
I'm willing to believe this is a real product only because of the "research" page, but it feels more like an aspiring web designer's portfolio piece. The flashiness of it all makes it really difficult to figure out what anything actually is.
While this thing does have a touchscreen keyboard it seems to me like the biggest selling point of the phone is actually something I don't want to use most of the time. I can't really see this succeeding for that reason.
1. https://github.com/vignshwarar/AI-Employe#:~:text=we%20don't...
Declarative interfaces that allow you to describe what you want and use agents to go out to different services and chain them together is a cool idea:
I don’t want to spend time using dozens of different apps with different (often poorly designed) interfaces.
Having a push to talk hardware button instead seems less clunky than a “hey siri” key phrase (I use Siri dozens of times a day but unfortunately ‘raise to talk’ feature on Apple Watch has never worked well for me).
I’m curious how their LAM works with interfaces being updated- if they need to retrain with UI updates or if it’s flexible enough to be stable with UI changes and new features etc.
I currently use ChatGTP sessions to dive into various topics I’m interested in, and explore ideas- I do like the idea of dedicated hardware that would allow this, but it’s something I imagine I’d keep on the coffee table at home, I don’t want to get a dedicated SIM and data connection or carry around another device.
They’ve raised $30M and I wish them well, I hope they survive and have me as a customer in the future.
Let’s say Apple were to do this. Apple would take years to get it right but when they announced it would be with a date with an accuracy to within a certain quarter. And it would just work on day one. Of course it would be saddled with subscriptions, but it would be real.
It just doesn’t smell right that a team led by a guy who can’t manage to pronounce his own first name clearly could think through all the issues and have a coherent vision within a year of when LLMs really took off.
Another possibility my paranoid mind comes up with here is the team/founder caught wind of some half baked research from inside Apple and decided they would try to pull a fast one and whip out a product before Apple could. And by the way, again of course it’s vapor, until it’s not, but we are still at the “it is” stage.
this doesnt qualify for vapor - they have shown the device in use, and announced a ship date.
(Click the LinkedIn icon for more info)
Don't know the current status.
Weird ad hominem. Your entire comment reeks of a bundle of biases.
A neat toy for hobbyists, but most people can't justify carrying two devices. And as soon as you add phone functionality to it... you've just made a new (more awkward) smartphone.
My takeaway of their angle is: the companies making the apps and services that are used a lot often have incentives that aren’t aligned with their users. People want to see baby photos, Facebook wants to sell ads, people want to find information, Google wants to sell ads, people want to see what their friends are doing Instagram wants to sell ads. The “LAM” promises to wrap a lot of stuff under an interface.
The device itself is just the access point.
I like the angle. I don’t see it working with today’s quality of LLM understanding human intent nor with all of the siloed platforms.
This also runs into the same core issue as voice assistants, which is that it's way easier to transmit complex thoughts/actions on a nice big visual UI than it is to convert those thoughts/actions into verbal instructions.
For example, when people order food from an online menu, most people are not fine with "get me the closest pizza available". They're usually considering complex tradeoffs between lots of factors: alternative restaurants nearby, how visually appealing is pictures of the food, health content, other options on the menu including their prices and content and how well they complement other dishes, etc. Figuring out how to express these preferences via a back-and-forth with an AI is more tedious than just looking at a visual interface and letting your subconscious process all this information into a choice.
I can be driving in my car, say "OK Google, open Spotify" and Google happily replies "OK, opening Spotify on your TV".
Lovely.
I have a Spotify playlist called "kid's music".
"OK Google, play the Spotify playlist Kid's Music", and it'll go and play someone else's public playlist called Kid's Music.
Even sending messages barely works.
"Ok Google, send a message to <wife's name because I cannot say "my wife" due to how unintelligent Google assistant is> on Facebook Messenger saying ...."
"Sorry, I cannot send messages through Facebook Messenger".
It can read messages from Facebook Messenger, but not send them, lovely.
I had Music playing on my phone, I wanted to know what song it was. "Ok Google, what song is playing", soon as I say that, the music stops and now Google assistant is listening to nothing.
Right now I cannot say "OK Google, go to Discord channel <foo> and summarize the day's conversation".
> which is that it's way easier to transmit complex thoughts/actions on a nice big visual UI
Depends on the task. Some are better on a UI (I want to see pictures of the dishes before I order), some are better with voice or natural language input.
Even on my PC, I could see some tasks being easier with an LLM
"Close all code editors I haven't touched in a week."
"Clean up my browser tabs by closing out all YouTube, reddit, and hacker news tabs. Keep open any tabs involving AI research or LLMs."
You've got absolutely insane market saturation by two major players that have some of the most advanced AI teams on the planet.
Almost nobody is going to buy an "LLM-first phone," they will just wait for Apple or Google to put an LLM into their existing phone.
1. OpenAI is building in a brand new space. Mobile phones are well established. LLMs as a product for consumers are brand new. While it’s true Rabbit are trying to merge LLMs with mobile, the elephant in the room is the mobile incumbency. The existing players just have to add LLMs to their existing dominant platforms and Rabbit is done for. OpenAI, on the other hand, was unopposed launching ChatGPT. They had a genuine technical edge and consumers were hungry for it. Is everyone hungry for Rabbit’s concept? Give me a break.
2. OpenAI raised an order of magnitude more capital. Rabbit’s $30M isn’t going to get them much farther than a prototype device. My impression is the founder here managed to convince some VCs to give him money during the boom times and leveraged the generative AI hype train more recently. But where is he getting his next round? The one that he will need to actually make phones at scale. That will cost billions ultimately, and the incumbents own the supply chain he needs to access. His effort is all but doomed.
3. OpenAI’s formula was easier for a startup to master. All they needed was money for the best AI engineers and scientists and money for GPUs, and they could create a blockbuster product. Rabbit needs the top engineers as well as extensive capital for manufacturing and distribution. There is a reason that hardware favors massive scale and a reason why hardware startups tend to focus on pinpoint innovations. The energy barrier is extreme.
These are three reasons why Rabbit is in an entirely different situation than OpenAI was circa 2022.
Both of those companies had it handed to them, like, literally got completely smoked by OpenAI, a company with a thousand employees +/- in San Francisco. The giants are incredibly vulnerable, just like the giants that Google and Apple disrupted such as IBM, Yahoo, AOL, etc.
I worked with one of the (many) teams at Microsoft who worked on Cortana.
The way the team leader explained it to me is that Cortana could do a lot more, but internal corporate politics prevented it. Rather than implementing the best solutions to user's problems, they had to do things like ensure Bing search handled certain results, to make sure that team stayed happy.
Or to take it to the extreme, if someone at Google came up with a device that directly beamed 100% correct search results into your brain, Google would never release the product because of the loss of search ad revenue.
By my reckoning, this approach is probably ten years ahead of its time. I simply cannot imagine that the tech is there right now to make this nearly as seamless as it will need to be to actually supplant the UI paradigms we have today, and it's going to take billions invested by the duopoly to get to that point.
But I really do think this is at least a fuzzy picture of where we're headed. Your iPhone in 2034 won't look like this, but you'll likely be able to trace some things back to it. There are a lot of pieces missing that we barely even know are missing yet, but it's incredibly exciting to see a startup try to jumpstart a step change like this.
Apple and Google presumably know about LLMs so I don’t really see this device being very influential.
I could imagine replacing my iPhone and my laptop with something Rabbit sized and a lightweight set of AR glasses.
But they need far more than the toy “OS” that Rabbit are demonstrating.
Perhaps at some point technology will allow glasses to have screens that are functional without making you look like, as they called them ten years ago, a Glasshole. Ten years may be enough time.
Anything that even threatens those business models will be strangled before it gets released.
IMHO Microsoft has the most to gain here. If they manage to turn Apple and Android into platforms that interface with MS made/ran/hosted LLMs, MS wins the day.
The problem is monetization, these LLM services are expensive, and telling users they need to pay another $30 a month to use their phone will be a hard sell. Hopefully LLM hosting costs go down, but if hosting LLMs becomes too affordable, MS's giant war chest of $ and hardware become less of an advantage.
The other danger is Apple/Google throwing a bunch of restrictions in place, but all the anti-trust scrutiny they are under right now could give them pause on attempting that.
i don't think apple or google are well-positioned to build this revolution, because they are too conservative and too bought-in on the old interaction model.
(if jobs were still around, different story. alas.)
they already have a habit of replicating the most useful apps natively
i don’t see why not here
replace siri once and for all, while running a local llm server for apps to tie into instead of an api to chatgpt
It will. I don't know which VC did the due diligence but the CEO once founded a scam company called RavenTech and the CTO just dropped out of CMU.
They used to be called CyberManufacture Co. and was selling NFTs. You see where this is going?
Even if Google builds something like this, gets market share and sells at a profit, it might still be a net loss for them because of all the advertising money and click tracking data they won't be getting.
Google definitely has the expertise to build this, but they also have an extremely risk-averse attitude resulting in layers upon layers of bureaucracy, and this product is literally cannibalizing their most important markets. I can't see this going over very well at all the internal reviews.
Apple was a lot leaner than Google when they build the iPhone (which was also cannibalizing one of their main products, the iPod), but they still had to make a completely independent team with no oversight except for Steve Jobs to get it right.
We also know how bad Google is at hardware. They seem to have gotten marginally better in recent years, but Pixels are still far from successful, even in countries where they are sold, which honestly isn't that many in the first place.
I'm very bearish on Google in this fight. Apple probably stands a much better chance, they've already proven they can cannibalize their own products, they need to do far less of that in the first place, as most of their revenue comes from hardware sales and subscriptions (which is perfect for something like this), they do have a lot of hardware expertise and they've already made major steps towards AR glasses, which might end up as a better form factor for such a device than a phone. Their pro-privacy attitude might be an impediment, this stuff works a lot better if you run it on a beefy GPU in the cloud instead of a tiny, battery-conserving chip in your phone, but I hope they find a good enough compromise.
I might even have tried it myself if Siri didn't suck to much. My experiments in this regard were...discouraging. Perhaps she radically restricts her use to things Siri understands (unlikely in her case) or she just knows how to speak in a way aligned to siri's expectations).
Though the other night I did awake to her her firmly saying "flashlight. flashlight. flashlight." for a bit followed by "shit" and a bit of fumbling at her wrist.
there are tools that exist that teach agents what to do, but the ability to have them on command and return a nice little summary in a neat UI is actually kind of compelling. It also feeds the company routine information to fine tune agents and maybe results in a dev ecosystem of some kind? need to learn more about how the system prevents some of the concerns hear like taking undesirable actions on your behalf
I'm not sure how exactly they handle the communication between that browser and the phone, but at least my sense of this is that they are doing what I imagine is "standard" browser automation (which is much more advanced with LLMs) and then something to reinterpret the output for the phone. You can do a lot with the mobile view of the sites, but they seem to be going further. (For something like Spotify I wonder if they are doing something site-specific)
The use of a browser offers interesting possibilities for auditing what is happening. With something like GPT Plugins you can see the JSON, but it's not really equivalent to what you as a user understand.
Looking at them using Discord, it makes me think they must be doing automation through vision, detecting pixel locations, etc, as Discord is deliberately hard to automate.
Voice chat with your computer is possibly the worst input method. UIs exist for a reason. Clicking a few buttons is much faster than a conversation. The future of computing doesn't look like a bot that's automating some website UI actions.
A legal aside: if an airline wants to make a sucky UI because its trying to upsell, should it be legal to allow users to skip over it? Is a company's website any different to their physical storefront?
At first it seems a bit like a kludge, but I don't really see any other way to create the dream sci-fi robot assistant. It has to be able to do things as you, and it would be impossible to manually integrate with every website or service.
I just want something that I can give my 93 year old grandfather so he can order a ride and get reminders about his prescriptions. Rabbit seems like a device that misses an audience that actually needs a simpler smartphone.
It has to be able to make phone calls and text on WhatsApp/Signal too, though.
Now if multiple in the same area were using this device, it may get chaotic/confusing, but no moreso than multiple people having loud voice conversations in one place.
I definitely see the appeal. I do not like owning and using a smartphone. A more functional device with a pared back interface is exactly what I would want, and I think they’re _is_ a decent slice of people out there who agree. You see that in the dumb phone and minimalist phone market that’s popped up.
That said my experiences with LLMs are that they are woefully underbaked for this kind of thing, and it’s a really tough sell to me on a privacy basis as well.
But for older folks I’m not sure I agree that the market is missed. I suspect the idea would be that you would manage the web portion for your grandfather and then he can just chat to the device. Unclear if it can make calls though, that does seem like a miss!
seems rabbit surely is giving a miss to this audience who needs simpler device
thisssss... many of the comments are wondering why this thing is a hardware device instead of an app and i think this is the answer
[1]: https://play.date
That is the problem doing end user hardware tied to a technology that is changing very fast and where most of the innovation may be out of your sight, being carried by your main competitors.
There are other worries, such as privacy and keeping the cloud alive, though, but the hardware doesn't seem to be the limiting factor here.
That's because it is. Telling my phone to "edit images on Photoshop" (one of their examples) just doesn't seem feasible in the majority of interactions.
Being able to directly email an autonomous agent or even CC one later into a conversation with instructions can be a game-changer. Not only would the replies be faster but the result could be cheaper compared to a human equivalent.
One use case which comes to mind is, which was also demo'd, is making trip arrangements. In the demo, the spoken request to book a trip to London was long and precise; no way I would get that right the first time on any push-to-talk device. At my company, we use a travel agent for business trips which we email and they book everything and send over receipts and tickets. I could just as well be emailing a LLM/LAM.
Call me weird but I don’t want everyone around me essentially hearing every thought I’m looking up in real time. Feels creepy.
When Apple implement something remotely similar to this into iOS (WWDC this year, anyone?), it will likely render products like this obsolete.
Honourable mention, Teenage Engineering crushing it with the hardware, as usual.
Not sure I want to replace any other actions yet. Probably someday, but touch + apps is actually just incredibly good. A sub-1yo can start learning to use a smartphone / ipad, and a 6yo can put together an uber eats order. I'm not saying it can't be improved on, and I fully want a competent voice assistant in my phone. But it's gonna be real hard to do better than what we already have in our pockets on the things they're already good at.
So this presentation fell flat is for me are when it was talking about doing things pretty much everyone does just fine on their phone. But it was very exciting to imagine a ChatGPT+ device sitting around the house for anyone to pick up and interact with.
> no subscription required
This does not compute for me.
seems like it's connecting to external services but i'm not sure why siri couldn't do that.
Until now i was under the impression i'd need to hustle more to earn cash & build the prototype. But that won't be fast enough. I can't watch people that just don't get it but try to imitate apple (website, keynote, lol) with just some bullshit.
The more i watch from the sidelines, the more i am reassured that we are building something incredible that nobody sees coming. But we are not located in the EU, not the US. So i am not sure what i should do.
> But we are not located in the EU, not the US. So i am not sure what i should do.
Keep the team in the EU, founders move to US, become a global remote company, raise funding in the US.
They must have had a prototype to get their initial $20M funding (Oct 4) as well as another $10M announced 10 weeks later (just 3 weeks ago).
see https://www.rabbit.tech/updates/rabbit-raises-20m and https://www.rabbit.tech/updates/rabbit-raises-additional-10m
But the actual rabbit device in the keynote could have come later. The cloud backend including the LLM is independent of the rabbit device. They could've demo'ed an app on an iPhone to show the VCs - there's nothing rabbit-device-specific that an iPhone or Android phone couldn't do.
I'd like to see it do more than help me buy things. Teach mode seems to be the main way to do productivity tasks?
Who actually interacts with a device like that? Is it an important problem enough to be paying $199 while I can do the same thing (often even better) with my phone?
https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...
No. I could go deep into "rare language, weird slang, local dialect" hole, but simple "No" would be enough. Their website is exceedingly non user-friendly, too
When you press the button on the side, you speak as you would type to ChatGPT, but now the LLM can do arbitrary things as you.
Absolutely Not!
I wonder how hard it would be to root this device and use it for the hardware.
Based on the rabbit keynote showing a colour screen - take a Playdate device, turn 90 degrees counterclockwise and reduce the height-now-width since you don't need a controller, just a roller wheel, pushbutton, and rotating camera.
Quick web search states a Playdate device has a 168MHz ARM Cortex M7 and a 400x240 LCD B&W screen.
Hell no. This is no better than a gameboy or whatever the current Nintendo console is.
Besides that, there are too many animations where a few sentences or pictures would do. Elsewhere, thick walls of text, sometimes barely indistinguishable in color from the background.
The site seems to be an omen of the product itself: cool in concept, but touch-and-go in implementation.
I tried to go into landscape to fix it, qnd I get a black screen asking me to go back to portait mode.
I don't know how you can get such basic things so wrong.
It's like those old hard drives "designed by Porsche" etc.
all in all.. fake and trying to copy apple. Can't people be original anymore?
Is this an early April fools?
- Not sure if people would want to "talk to" a device in public.
- Anything and more that this device do can be done by a smart phone that people already have.
- I feel like people prefer using a UI interface instead of speech.An example to this behavior might be using delivery apps instead of calling a restaurant.
- Looks too wide to fit into most pockets.
Probably not the first model but the next one for sure if it can text and phone too.
I think there is a market for this and even though I despise talking to a device, I can see the use and I can see how it could bridge a world which is getting too complicated.
It seems like a fatal dilution of focus to have to worry about the design and logistics of a fancy dumb terminal widget when you also have to get the software/AI/app integration stuff right.
Just make an app with text and voice interaction. Accept that the thing in our pockets with a screen and an internet connection is going to be a smartphone. You will not build an own-hardware moat with these weird little bits of e-waste.
Which is sad, because I'm sure there's room for a lot more innovative devices in the world outside of a single glass rectangle in your pocket that everyone must plug into in some way. The economics of the industry just makes it very hard for them to survive, and we all lose out because of it.
Genki Things is another...
I'm sure there are others?
Care to expand on that?
I can't speak for the deranged expectations, hype cycles and back-lashes over the last 5 years. And I think Meta's R+D budget is pretty hard to justify. But I don't think that reflects on their current product range (which could have been matched with a much smaller budget - and to some degree has been)
You just don't hear about them because they're not selling to you. They make business, commercial, and industrial hardware.
Consumer hardware is very hard because consumers are extremely demanding of hardware. Just look at how difficult it is to convince people to spend even $5-10 on useful software or sign up for a $100/year SaaS product with near zero marginal cost per customer. Consumers are really hard to please and consumer price points are difficult to serve.
https://www.youtube.com/watch?v=22wlLy7hKP4&ab_channel=rabbi...
This smells fishy.
push a button and talk to the ear buds.
On wifi at least. without a camera.
Not sure if any models are trained with app screenshots and interactions quite yet though. Shouldn't be long.
if the whole damn point of suffering with your goofy wildly colored mostly-incompatible mobile device is a talkable LLM, then "shake to keyboard" is a failure.
at $200 i'd rather just buy any of the plethora of sbc mini devices with cell phone antennas and play with LLMs myself without the black box that I can't touch. a uconsole or similar is , generally speaking, cheaper AND open-ish -- it's a fool's errand to trust a non-paid subscription service to continue existing.
The future of manual human interactions is certainly limited. (It will definitely end up looking like movie Her).
OpenAI is in best position to redefine the edge OS’s that get us there.
Has some CPP vibes.
Just like for games, "no, I don't think I will". I'll let the quirks be ironed first then I'll see the reviews.
The Android crowd, the Apple gang, etc... Once you are member of one ecosystem, friction is high enough usually to keep you in.
Moreover, many of the eco-customers wear more than one device.. phone, watch ...
So... why would I buy into yet another ecosystem with a third device to carry?
And no... most eco-clients will not abandon their current ecosystem that easily.
The 360 degree camera is quite nice, though...
i feel a revolution shouldnt require so much tethering to other services through the hardware you are looking to replace?
if they keep iterating towards a truly standalone product, I think they'll hit all the notes we've been waiting for from a post-cellphone portable device
or can I make an AI agent to set it up for me :)
They had to get Jesse to do this keynote with his arms in the air the whole time?
The AI in the device could be installed on any smartphone surely?
Are they pitching the device to try and make their product seem more tangible and get buy-in from more people?
====
Do you ship outside of the United States?
Yes, we currently sell to customers in the United States, Canada, United Kingdom, certain countries in the European Union (Denmark, France, Germany, Ireland, Italy, Netherlands, Spain, Sweden), Korea, and Japan. We will ship r1 devices only to mailing addresses in those countries/jurisdictions.
When will my r1 be shipped?
We expect to begin shipping r1 to US addresses for Pre-Sale Purchases in March-April 2024. Shipping to non-US addresses for international orders and fulfillment of Pre-Orders are expected to begin later in 2024,but we do not have a more definite estimate at this time. Shipping estimates are subject to change based on our manufacturing capacity and component availability. An exact shipping date is not available beyond the expected shipping window listed above. Please be assured that we will fulfill orders as quickly as possible in the order they were received. We will update this FAQ page with regard to shipping within the United States and shipping for international orders with the latest information on shipping window estimates as soon as we have that information.
====
Think I had forgotten TE did the design for that
Only after getting the users should smartphones be replaced with a bespoke device. The only need for doing the device first is if the service is effectively commoditized so need to differentiate somehow.
- It should be a Smart Watch!
(or smart glasses, perhaps later)
With a companion web dashboard to be used from the user’s smartphone, tablet or PC.
It's just "ooh, pretty pictures, buttons and camera." Seriously? I'm supposed to trust a product from a company that can't even tell me what it's about without making me sit through a 25-minute keynote? Are they out of their minds?
"push to talk button" ok, so?
"far-field mike" ok, so?
"360 degree rotational eye" so what?
"analog scroll wheel" that does what?
"usb-c plus sim card slot" which gives me what?
There's a video but I shouldn't have to watch a video.
There is zero information on the web site except an overly long video badly aping Apple's product announcement style. Including as-seen-on-TV "doing things is so hard" crap.
Hell, the FAQ is entirely about purchasing, not about the "product." It's therefore also meaningless. Rather than harsh I'm being generous. This web site is useless.
Since it's all cloud-based and has a far-field mic, I would also ask people around me not to use this device, if they had one, and turn it off if they are around me.
As far as I can tell, this product is just a bugging device.
> rabbit OS operates apps on our secured cloud, so you don’t have to. Log into the apps you’d like rabbit to use on your system through the rabbit hole to relay control. You only need to do this once per app.
So things don't run on the device, but on the cloud. Unfortunate.
so... you also still have to have another PC and operate apps on it.
Honestly that's the best solution for a V1, in terms of security, privacy, speed of development, etc.
The target audience is early adopters, that definitely isn't people who want to use Spotify/Uber/etc but don't have another device.
- Did it really scan the whole table to add a new column? As in, OCR? What would happen if I had more than one screenful of data, and who in their right mind would trust the OCR'ed output?
- How can it access the Discord account for the "teach mode"? Is it stealing your (sensitive and expiring!) cookies to reproduce the actions on their cloud? Or worse yet, running instructions from the AI on your local machine?
- It shows structured results from Spotify and Uber on the r1's screen. Does it have custom integration with all those apps? What happens when the API changes (Twitter), or they don't support an app I need (local taxi), or the app is averse to third-party integration (WhatsApp), or I need an "advanced" feature (payment methods on Uber)?
- Why in the VC hell are they basing their revenue on a one-time sale of a clearly unnecessary electronic gadget (my phone already has all that!), when their biggest variable costs will be recurring cloud expenses?
Yeah there is a screenshot of a list of logins floating around, uber and spotify were on it.
Wouldn't be too interested in having it logging in to my account on a cloud machine on other things.
That's what I caught too. It mentioned somewhere on their site that you'll have to provide login credentials through their Rabbit Hole dashboard panel of all services you want to use.
It breaks the services' ToS but I guess most people don't care much about giving their passwords. There's no other practical way to integrate with multiple services.
> Is it stealing your (sensitive and expiring!) cookies
> running instructions from the AI on your local machine
Based on my interaction with LLM enthusiasts, there are plenty of people would happily bet their career, give up all their credentials, and grant full access to all their devices to LLM if they are promised something cool. See the stories about lawyers giving LLM-generated materials to the judge for an example, or journalists submitting LLM-generated articles.
Yes, this occasionally ruins someone's day. No, this would not stop true enthusiast from trusting the model again.
USER: "hey siri [equivalent], please renew my CD for another twelve month term."
PHONE: "Did you say "transfer the contents of your entire bank account to the yakuza?"
USER: "fuck! no! cancel!"
PHONE: "Transaction confirmed. Current bank balance: zero."
Will it change the world? Unlikely. But there is definitely a niche that will love this, even if it's just for the form factor and design rather than anything practical. I'd also expect it to fit into TE's range of musical equipment in some form.
It looks like they'll also be allowing people who create their own automations to sell them in their marketplace. Presumably they'll take a cut of the sales here.
As an example, yesterday my daughter asked for a song from Snow White.
EDITED FOR CORRECTNESS (originally I said I asked for Heigh Ho from Cinderella)
In the car I say, "Hey Siri, play Heigh Ho from the Snow White Soundtrack". Siri: "Sure, here's Snow (Hey Oh) by Red Hot Chili Peppers" (try it yourself!)
Why are Alexa and Siri still so useless, inaccurate and inconsistent? Why can't I yet ask Siri to "book me a ride via Uber from location X to location Y" or "reorder the same thing I got last Tuesday from Uber Eats"? I assume it's down to compute costs, but I would absolutely pay an additional subscription fee for more intelligence behind these voice assistants.
I’d imagine something like siri or alexa isn’t evolving as fast since the server side cost at their scale would be astronomical..
It probably doesn't help that "Heigh Ho" is from Snow White
For example, I often wish I could just queue up a song to play next like I can over touch, but there's no elegant way for me to do that without picking up my phone in the car. If my wife were riding with me, all I'd have to say is "Hey play [XYZ] next", a feat yet unmatched by any voice assistant
> Prompt: Give me the lyrics for Heigh Ho from the Cinderella Soundtrack
> ChatGPT: "Heigh-Ho" is actually a song from the soundtrack of Disney's "Snow White and the Seven Dwarfs," not "Cinderella." The song is famously sung by the seven dwarfs as they head to and from their work at a mine. Here are the lyrics:
*lyrics omitted, but they seem correct
Edit: I tried it a couple of more times with the same prompt and this is not consistently reproducible, so yeah, Siri is still pretty bad at this.
"phones have too many apps so buy this toy device and also keep your phone because you're still going to need it to do most things"
Also, being an alternative to the smartphone seems sort of a point of this device.
1. at 13:13 there's a demo ordering a ride 'to home', then the user requests a car change to fit six people, and what's shown as a seamless switch from UberX to UberXL also updates the destination from the home address to LAX airport.
2. at 14:05 the device confirms and recites a pizza order, but the screen displays "chesse" with a typo while the voice reads out "cheese", so either the audio or the visuals is faked. My guess is that all of the on-device graphics were hand-written and hand-animated which would explain both mistakes.
I stopped watching at that point. Am sort of sad to see Teenage Engineering associated with a product that seems so sloppy and/or shady.
Like where is this? This is what I want. Not some voice assisted way of booking an Uber or some shit.
Agreed! That’s precisely why we created https://openadapt.ai.
Was very hard to sign up.
If it’s only for super rich people, they already have a human PA
1) Price seems too low for unlimited LLM usage and there's no monthly fee... so maybe you are the product?
2) Website has no information on the people behind this.
3) In the keynote, the demo of logging into to other services like Spotify appears to just be stealing the auth tokens from the laptop and shipping them to the R1 device. Not a good sign.
4) The founder's voice-over insists that "we value privacy" and "we do not hack" and "we do not create fake users." Protests too much?
5) They're not really addressing the trust-building necessary to convince people to use this like a personal assistant. It's waaaay too opaque.
Neat product, though. Maybe they're just launching super early, nothing is figured out yet, the website isn't done, etc.
I don't think the tokens are getting transferred to the device. It seems everything happens on the cloud, so the tokens must have been stored on the cloud too.
It's a 2018 spec Android device. $50 of electronic parts, $150 for the custom case.
It just looks like OAuth to me?
It only needs to last a few months until all the devices are palmed off to kids. There is a 0% chance this makes it to year 2.
> 2) Website has no information on the people behind this.
All the better to run with the money. Given there won't be a second device, they should have charged more. I suspect they charged less because it simply performs _that_ poorly.
> 3) In the keynote, the demo of logging into to other services like Spotify appears to just be stealing the auth tokens from the laptop and shipping them to the R1 device. Not a good sign.
It's worse, it's not processed locally at all. That means that it's all done in some random cloud.
> 4) The founder's voice-over insists that "we value privacy" and "we do not hack" and "we do not create fake users." Protests too much?
I've never murdered anybody and there's no point looking in the forest!
With AI you are always the product - how else can it be "trained".
> Record your actions, explain them with your voice, and play them to rabbit OS. LAM will learn the nuances and create a rabbit that can be applied to various scenarios.
> What if you create a rabbit that could be useful to others? You can monetize and distribute it on our upcoming rabbit store.
This is neat right up until "monetize", where everything flips from being a potentially cool community group of reproducible actions, to a pile of spam and bot-generated bullshit
Extra negative taste points for wanting a "cool SUV" for a trip to London.
What feels "dumb" about it?
Is the perception of others towards you when you do this? Because that ship has sailed, people Vlog on the sidewalk and do TikTok videos in the subway, and Facetime their mom or brother at the grocery store or in the mall and have long video conversations with each other out loud..
I'm not saying this to be dismissive. As a GenX'er it took me a while to normalize this behaviour even to myself, but I've accepted now that these kinds of interactions with devices are just everyday occurrences and literally no one cares.
They can be obnoxious for sure, but just using TTS/Siri on your phone is like the nicest version of this interaction I can imagine. ;-)
Not without me giving them the "Have you escaped from an institution" look they don't. And you judge yourself as you judge others. If I were to say "Add eggs to shopping list" on the bus, I'd instantly feel the stares of a dozen people as I look on my phone in shame (That's whether or not I'm alone on the bus). And my shame is justified too.
But I'd do it in the car, or the kitchen, if it was reliable and didn't require sharing anything with anyone.
For the longest time it was literally the only thing I used Siri for (cooking timers), but now I use Siri for setting reminders and other similar transactional stuff on my phone, and I use voice dictation for texts all the time in the car, and even sometimes when I'm walking in the cold and just don't want to take my gloves off..
And it works great. And it doesn't feel weird at all.
For example, adding things to our grocery list. It is quite handy to just say “add butter to the grocery list” when I am looking in the fridge. I am able to not stop cooking, but add things we need to the list as I find we are low.
I have found that relying on these devices to have consistent usability when the processing is all done on the cloud and out of my control was a foolish mistake. I do wish I could revert my google home to ~two years ago.
https://www.theverge.com/circuitbreaker/2017/11/16/16664846/...
There's no way this isn't a nonfunctional brick in 18-36 months when they go bust.
2 reasons:
- It will be better as a phone. People will not use it and the initial hype will die quickly.
- It is seemingly useful but useless in practice because many things are impossible to do. The demo is already so fake.
With the two above reasons, the momentum and funding will dry up quickly.
Check back in a year.
Come too think of it ... maybe that's the plan: to be bought up by a big tech company.
ALL AI wearable companies should return their money to investors, and wait for the AR glasses by Apple.
AR glasses are the ultimate form factor. And as much as I hate monopoly, Apple has the right app/dev ecosystem, and will make Siri work.
The AR glasses race is between Apple and Meta Platforms.
This 'LLM-first phone' looks like a solution in search of a problem. But we'll see.
Yes, but that device is a smartphone. I can't see a future in which standalone devices that are simply ChatGPT API shells are actually viable, unless they add some significant capabilities that a phone app or built-in phone assistant doesn't have.
Not to mention the price. This is a fifth of the price of most flagship smartphones.