Generative UI and outcome-oriented design
nngroup.com
nngroup.com
> In the future, generative UI will dynamically create customized user interfaces in real-time.
That's fairly unambiguous in saying that such things don't yet exist.
But unfortunately this is a sure shot way to rake in the big bucks. Lot of companies budgets are unfortunately still controlled by people who fall for buzzwords and who don’t have the time or inclination to look a bit deeper and understand core concepts and tradeoffs. So they refuse to look at things a little critically.
The big problems which are blocking these products from leaving the impressive demo stage is hallucinations, quality, and constraints. To my knowledge these are all solvable, albeit expensive.
A Q&A engine who's every answer should be reviewed by an expert or else?
A code generator that might generate code that has serious logical bugs or making calls to functions and libraries that don't exist or even might have syntax errors so to be useful, you have to have the above or average knowledge of that language to correct the output?
I spend most of my academic years studying type systems. Unless you work on space crafts or blockchain tech, that level of security is not needed. Modern typesystems is about DX rather than assuring correctness.
It is easy to be pessimistic on new technology. Sometimes it is merited. Oftentimes it is noise and needless rant.
They get almost exactly what they had in mind every single time.
I use ChatGPT and Copilot for software development daily and it's not giving me any functions that don't exist. Tune your custom instructions. And yes, sure - it shouldn't be used by junior devs in the current state. But it accelerates the seniors so much it's unbelievable.
GPT-4 is fantastic at taking a detailed requirement/question from an expert user and giving a good answer. Curiously, it sometimes doesn't respond well if it perceives one as a non-expert user - it'll start lecturing on BS, or avoid giving an answer.
Nowadays, all the “articles” they publish are a promotion for their training offerings without too much substance.
This article is absolute nonsense and I'd expect more from N/N, which I usually respect.
Not only does it forget everything about good UX, it also shows a lack of understanding of AI, how it works and where it's useful, treating it like some magic fairy dust that can be sprinkled about and magically solve all problems.
A system that completely reconfigures itself based on what it predicts that particular might want, based on biased or incomplete data, sounds like a terrible idea. Support alone would be a nightmare – if the interface changes for each customer, do the guides and videos also change? How would customer support know what the user's looking at?
However, I do like the idea of being able to tell a system the result you want, then it showing you the best way to achieve it. For example, few people go to an appliance website to look at washing machines for fun. They go because they want clean socks. A system that could help them find the right machine to clean their particular socks quickly might be valuable.
Generative UX sounds fancy but falls apart at the slightest bit of actual thought.
> A system that completely reconfigures itself based on what it predicts that particular might want, based on biased or incomplete data, sounds like a terrible idea.
This sentence shows a quite off understanding of the AI developments for this type of applications. This would be the same as saying that ChatGPT is entirely useless as each character is responds is merely chosen from a probability distribution of the previous characters. While the assessment is technically correct, it diminishes the fact ath it has huge value.
I also see that you have a lot of concerns by trying to predict how this affects other parts of a product (customer support, etc.). While I understand that this type of uncertainty is uncomfortable these things usually fan out alright (or at least profitable).
A comment like this sound like an 70s postman who says that electronic "internet" would never work because people would not be able to communicate with the rigour of handwritten letters.
I am not saying that this article is right or wrong. I merely say that a bit of curiosity would be beneficial when assessing new ideas - that can also be the difference of being in or out of the job market in 4 years.
An A/B test needs thousands of samples in some cases to make a determination about which is preferred. A generative UI would be like running 100 simultaneous a/b tests for your users.
I would think you would need a significant amount of traffic on the order of millions of users to be able to determine to some degree that one UI configuration is better for a particular type of user.
Without the data to know what works the AI will essentially just be randomly generating UIs for each user.
Defining an outcome is actually one of the hardest problems in AI and even in organizations. While you think startups can easily just say "make the UI that gets the most revenue" that isn't necessarily true. For example if a terrible user experience results in more short term revenue but your users die out, that isn't a good outcome for the company. Or maybe you have a free tier that you want people to use for network effect, etc.
So it is very hard to have a single number to use for outcome. If every user has a different UI, then it also becomes hard to spot things that go against the company's interests, etc.
That's all after preprocessing those PDFs, because there were some characters and sequences that blew up the works. That took some hunting. I did learn that FOSI processors love injecting x00 null characters into the PDF for some godforsaken reason. Didn't know that. Don't know why, though, and probably never will.
Falling back on older school techniques - from stats, text mining, topic models and other arcana - for some tasks is probably going to be worth it, for a while anyway.
Most users can't use any of this stuff, it's too CLI-ish for the vast majority of the people doing actual work. I suspect this is the gap the OP is hoping to insert themselves into, that gap between these . . these so-called "AI" things . . and the average user, but they're going to need to do a better job of scoping out just how big a job that is.
Now take all this with the following boulder o salt: we can't use any of the big cloud things, everything we do here has to be brewed up on prem, by people like me who do not know what we're doing. So we're not exactly AI superstars. Odds are everything I say is irrelevant for the outside world, with all your Geminis and whatnot.
With the more opaque systems where we don't have access to the models, we can't.
This lack of shared similarity in the workspace is also another big issue with Apple Vision for instance, but nobody really uses it more than a few hours to get to this realization.
> This individual example may be plausible without genUI, but not at scale.
Nothing about that example seems implausible at scale. It's just a combination of personnalisations, saved settings, and learnt user preferences.
Maybe no one puts that particular example together to that level today, but there's nothing stopping someone doing it today without generative ui.
I doubt that UIs are going to be tailored to every and each person.
But if I need to use word on some other machine, lest it requires to log into some account which I don't think should be required... The UI better be streamlined.
The same way google sheet and microsoft word are not too dissimilar. That's for a good reason.
As another side note, the idea of a UI that changes every time you open the site/app is, frankly, ridiculous. Not sure what kind of UX researcher or even user would want that. Seems like a saner approach would be one of two worlds below:
1) Each app has a carefully designed foundation on top of which user-customizable components exist. This is already common, sans GenAI-powered component creation.
2) There is no longer a concept of an "app." Users instead interact primarily with a smorgasbord of APIs, via their highly personalized but temporally consistent, AI-assisted personal OS.
That's what we have now, already. GenUI would be the opposite, where if you wanted, the UI could never change and it would stay the same if thats what you want.
Totally agree the concept of apps will break down.
Companies want to control what you see so they can control their revenue. Why would they want to expose their data and utility to UIs where they have limited ability to monetize?
The insights is the ability to emit knowledge in different dialects. An example is understand legalese. I am not able to do that as I have no degree in law. But LLMs are hugely effective at taking me through a legal filing and understand what I need to put into each field.
The hope is that developments like this will ensure the most efficient interface for a given group of people: Arabic speakers get an interface that take right to left writing into account. Legal people get a language they undrstand and laymen get an interface that caters to them.
I see a lot of critical comments around this. In my opinion, this will turn out to be an accessibility thing (Probably mandated by the EU in some years). Just like it is considered good form to support text to speech software for vision impaired people is it today.
The window app is mostly a proof of concept for now, as it runs only on Windows, you can see how it works in this video https://youtu.be/RJYv5PUz9bY?si=ZzrXL4HJq7ANL6H6
The language out self is in natural language, if that interested you, check out https://github.com/plangHQ
Interesting ideas, though from companies I've worked with, there is a huge cadre of UX people that know nothing else and would likely fight this tooth and nail.
None of it makes any sense if you stop to think about it for even a second. Anything from ever-changing UIs to the actual state of AIs to the basic systemic issue that you can't properly describe all the outcomes a person needs from a product without it being self-contradictory (and ever changing to boot).
Edit: this will not stop multiple startups from chasing VC money using this idea
It seems to me that a generated UI does the opposite of that, and leads to a lot of uncertainty...
... How does support help them, or even troubleshoot problems when the user describes what they saw and did?
... How can one person instruct another in accomplishing a task when the interface is different?
... How can you document the instructions for accomplishing specific goals?
... How user-unfriendly is a UI that not only changes per user, but could change over time as more data is gathered about how the user is interacting? Especially if the user needs to do something out of the ordinary [for them], the means of doing it can get buried over time.
And LLMs are exactly what is needed - they know the consistent and no-surprise answer to anything. The UI will adapt to serve your needs, that's how it can help you. The web we see today is full of dark UI patterns and junk content. A generative UI could filter, rank and present the pile of garbage in a clean form.
Social networks and big website operators control what we see, in what order we see it, and how it is presented. All of them carry vectors of attack. They abuse the public. For example FB was always changing where it put various user settings, and resetting them from time to time.
An AI agent that creates a generative UI can guard your preferences. I see these gen-UI agents as running locally in privacy. They interface with the web to interact and gather information, and distill it into bespoke UIs. They overrule all the power games publishers played with users and try to prevent users inadvertently leaking private information, an AI firewall, especially needed now that the web will be filled with synthetic text and other AI agents. Few people will still look at the raw web, we will wear a protective mask of gen-UI.
For some values of "short-term" and "quite a few"
Dark patterns is another type of "outcome oriented design" where the outcome is the user subscribing/not canceling/paying needlessly. Adding "strategic automation" will doubtlessly just make this problem worse.
Yeah this is going to be so great for usability. “Where did the “foo” button go?” Oh right, the AI has magically decided I am better off with it somewhere else. “Why did the interface change completely today?” Oh right, the AI is having a moment. This feels like a usability nightmare.
This screams upper-management consultancy, doesnt value the work that goes into design and UX and would love to cream some money by pretending they could fire an entire department.
Seems like predictable should beat customized for something like a gui.
This is what the future looks like, but without dragging and dropping.
It's just a bunch of blocks stacked in grids, columns, and rows. This is what GUI and UX has become. Just black text on white rectangles, because it needs to adapt to every form factor, be accessible, be internationizable, be blahblahblabhlabblahblahblah. It has to be generic.
GenUI is the next step in a very depressing trend of sucking out all the color from websites and applications, and just the thought of it happening makes me very sad.
I'd have a much better "user experience" if I found a website that turned my cursor into a sword.