The Elements of UI Engineering
overreacted.io
overreacted.io
Over the years, I have grown convinced that designing and implementing UI by hand simply doesn't scale. There are too many things to consider, too many different users, too many preferences, too many possible states. Responsive design isn't just about screen sizes anymore, it's also about the user's language, culture, disabilities, input, context, preferences, connectivity, knowledge, focus, etc.
We can't expect every restaurant, bank, festival and airlines to implement their own apps, and consider all of the above. You won't find a dark theme in the Domino's app. Why do we tolerate these compromises in the name of branding? Why should UIs be tightly coupled with the services and data? Why don't we have general purpose clients?
I think the job of service providers should be to semantically annotate their data, so that a general purpose client can dynamically render it. All of these UI concerns would only have to be dealt with once, and we all would be able to sleep at night. Just let business people do business, and let UI/UX people do design.
For all the downsides of amp, it’s demonstrated that letting go of asthetic vanity in the name of fundamental soundness is a good trade to make.
So to bridge the gap you really need a function that can translate your expression into reality by understanding what parts are important for your design to work and what isn't, and making it work on any platform/client/medium. The type of function that can approximate a potentially extremely complex reality, that can be trained incrementally given examples (bugs)... Sounds a lot like modern machine learning to me.
HTML and CSS started as a solution for publishing text-based content, like the olde magazines but on the web. For that they are splendid: text rendering and basic input handling are taken care of, and wide variety of output devices were supported since day one (HTML 2.0 without tables works excellent on phones).
Then web 2.0 and web apps happened, web idioms began to change, all of people's activity with computers started moving to the web. And it turned out that to create complex UIs for all of that you have to fiddle with rather low-level primitives and handle interactions between them, because outside of text layout HTML primarily knows about divs and a handful of input widgets.
This may be good because high-level widgets will be developed independently and will evolve faster (and maybe better) than if they were built into browsers. But in the meantime we have to live with the chaos that we have.
I don't think that having some good built-in widgets (like a better dropdown, menus, DataGrid) would prevent third parties creating their own custom versions but it would help 99% of users that need the basic stuff (like a dropdown that can have icons)
Thankfully, modern UI development is blessed with some really professional UI toolkits: elements UI for vue or Semantic React for instance do a great job of making us think in terms of generic interactions instead of reinventing the wheel.
This means we get to focus on improving the existing as the project warrants which is significantly more rewarding.
You mean like HTML? We tried that already and people disliked the lack of branding ability.
What people? Brands or users?
Then Singapore Air or whoever comes out with a new product/service - you can book time in an onboard shower. The 'standard' doesn't allow for ancillary service bookies, so how does Singapore Air get this to their customers? Go through the 'standards body' and wait for that, rather than just building it into their own app that they control end-to-end.
In reality, there are too many variations between different companies offering the same service, that it would be rather infeasible to have a single contract they all follow (without compromise) to develop these provider-agnostic client apps.
In a way, browsers and native mobile SDKs is the solution to that problem - browser/OS vendors have created clients for common APIs and widget toolkits to create interfaces. The 'hard work' in delivery an application has already been done (no need for the bank to worry about HTTP, or building widgets like text fields), leaving them to delivery 'just' the layer on top of that.
I've been thinking lately about the "design system" trend that is becoming more and more popular where companies want more control over styling and behaviour to make their UX unique to their brand as well as consistent across all their products. Ready-made 3rd party components like the excellent react-select don't really fit into this world as general purpose components like this naturally have to make some choices regarding styling and behaviour. No matter how customisable they are, in the end they rarely integrate well into a UI based on a design system.
This makes me feel like the abstraction is all wrong. Rather than aiming for fully-functional, out-of-the-box components that cater for all manner of general purpose requirements, how about a library/framework that focuses on a set of primitive components that deal with the lower level concerns like layout, scrolling, positioning, etc. Maybe the abstraction could be more like "composable shapes" than "ready-made components" or something along those lines.
With this approach, you wouldn't ever start out with something like a ready-made Autocomplete component, for example. Instead, you would always build a custom Autocomplete and have complete control over styling and behaviour, but it would be built from solid foundations using some form of the "shape" abstraction. That way, you can focus on making the component's styling and behaviour consistent with the design system without having to worry so much about layout, accessibility, scrolling, positioning, etc - as all of these are taken care of by the framework.
https://github.com/paypal/downshift
I'm glad to see this trend.
The more I think about this the more I feel like I'm describing exactly how "Qt Quick"[1] works, the declarative user interface markup language that is part of the Qt Framework. Of course, it's not a web technology but it would be interesting to see a web-based framework/lib based around some (if not all) of the ideas found in QML.
I had to look that one up. I find it amusingly ironic that an accessibility initiative uses an abbreviation that renders the name incomprehensible.
A modal, for instance, cannot be an independent isolated component. Because at the very least it needs to display an overlay over the entire page, which means that the modal trigger must have some way to communicate with an element at the root of the document. Those are architectural decisions, not UI ones.
And that's just one example.
There were a lot of UI elements that were obviously needed if you were going to use a browser as an interactive app platform, but were easily passed over when minimal forms were considered sufficient.
I could see, for examples, a consistent WYSIWYG edit-box, a dropdown menu construct, a select-with-manual-override, consistent date and time widgets. Instead we got a bunch of inappropriate elements glued together with CSS and JavaScript to sort of work, but in unpredictable, non-native ways.
Whenever we can't fit everything on the screen at once, we use some of the patterns below:
- scroll view
- virtualized list
- tabs
- drawer
- master-detail
- page navigation
- modal navigation
- alert
- tooltip
- combo box
- collapsible
- carousel
- gallery
It wouldn't make sense to use any of these patterns if we had ∞ sized displays.
Would you call all of the above patterns "navigation"? Why not? Isn't navigation just a way to reach content that isn't immediately accessible? Why don't you think of scrolling through a list as some sort of navigation? I think you should.
It really helps to re-frame all of the above patterns as simple layout strategies. Layouting is about putting content where it belongs. Whether that content is visible or not (covered, collapsed, out of bounds) doesn't really matter.
Let's consider the classic master-detail example that so many people struggle with:
Tablet (stacked on X axis)
+-----+---------+
| | |
| M | D |
| | |
+-----+---------+
--------X--------
Phone (stacked on Z axis)
+-----+
+-----+ | /
| | M | /
| D | | Z
| |----+ /
+-----+ /
The only difference between these two examples is the stacking axis. That's it. It's the only thing that should change when resizing a window. You don't need to recreate a completely new layout using frames and pages and what not. Re-framing the problem just makes everything much easier. Navigation is just layouting.Like:
-Normal (stacked / Z) navigation is still special since it happens always on a full-screen scale.
-While you can stack your own UI the way you want, you still have to be able to really navigate to another page or app.
What about if you could navigate on widget scale? Like every web widget would have a different URL? Instead of having a single monolithic view we would have more independent views.
Or what about if we render external links as regular content? Should <a> and <iframe> be just a single element and just give different CSS render hints. If we can compose final user experience from multiple applications what kind of "higher order applications" we can do and what this means for application development?
Here are some examples of state transitions that can be augmented with animations:
- reorder items in a lost
- add/remove an item from a list
- expand/collapse an element
- hover/press/disable a button
- show a popup
- show an inline error message
- open a drop down menu
- open/close a burger menu
- change the burger menu button to a back button
- change the scroll offset to a new item/anchor
- increase the height of a text field
- show/hide the top menu/navigation bar
- add an item to the cart (and the count badge appears/increases)
- increase the value of a progress bar
- image goes from loading/placeholder to loaded
As you can imagine, most of these state changes benefit from transition animations (scale, translate, opacity). We just add linear interpolation to a discrete change.
Can you think of a good reason to use different techniques and APIs to implement route transition animations and button state transitions? I can't.
Once you think about using gestures for continuous transitions (swipe to go back on iOS), it makes even more sense to think of these components as physical overlapping sheets of material, with their own weight, inertia, grip/transition/friction, rails, anchors, springs.
Consider these interactions:
- swiping from the edge to reveal a side burger menu
- swiping from the edge to reveal the previous page
- swiping down to dismiss a bottom sheet
- scrolling down to reveal additional list items
- swiping horizontally to reveal actions under a list item
- dragging horizontally to move the thumb of a slider around
- pinching to zoom-in on a picture
These are all types of continuous navigations. They don't require animations because you're continuously animating them using touch. These interactions should be easy to implent. Programmatic discrete state changes should automatically infer transitions based on physical characteristics of these materials.
What's a route? Should the currently selected tab be part of the route? Should the expanded/collapsed state of a widget be part of the route? Should the visibility of a popup be part of the route? Should the vertical scroll offset be part of the route? Should the zoom level of a map be part of the route? Should the open/close state of a burger menu be part of the route? I think the concept of a route doesn't make a lot of sense if it doesn't capture the entire state and history of a person's interaction with an app. I see no reason why the browser history/backstack should discriminate against different navigation patterns, and only store pages. Adding all state changes to the history makes it easy to use the back button to dismiss a popup, close a burger menu, close the keyboard, etc. Heck, all apps should have universal undo/redo functionality.
Another specific type of transition people are struggling with are shared element transitions. For example, you tap on a thumbnail and it seamlessly animates into a detail page with a larger version of that image. This is easier to do as a layout transition than as a route transition.
A last thing to keep in mind is that layouts don't need to immediately create and render all of their elements. We can use virtualization, to only materialize what is currently visible. For example, a list of 1000 items will only materialize the 10 or so items it can display at once, and will dynamically create/recycle items as the user scrolls. The exact same strategy can be used if we're stacking items on a Z axis. For example, we could create and render the top 2 items (so that the previous item is immediately visible in a swipe back to reveal scneario), and only create and render other items as they get closer to the top of the stack.
A route is the thing that I send to another person or save myself so that the specific piece of information I am looking at can be found again. There is a decision that has to be made, though. My selected text is never part of a route, but that might be the information I want to share. Conversely, when I open a hamburger menu to click the share button, the hamburger menu's open state is never the information I want to share.
Another consideration: does the route encompass the concept of the information being displayed, or the actual information? We typically solve this with a "permalink", where one route represents the concept (feed, newest, etc) and the permalink represents the specific information.
For now, it's just a bunch of things I figured out along the way.
The name of the game is failure aversion. Frameworks and NPM packages for everything. When I interview JavaScript or UI developers this is my first discriminator. Why write original code or reinvent the wheel when somebody else has your simple solution behind 50mb of external code that you didn't write? If you are that fearful coward who believes in not writing original code I don't want to work with you. Have a nice life and go work somewhere else. I would rather work with somebody willing to take a chance on problem solving.
If you are going to down vote please mention why. Don't be a troll. Hacker News tries very hard to not be an echo chamber.
And what's funny is that if you hadn't been a jerk the original point of Frameworks and NPM Packages for everything is a good point and probably would have been well received. Because, the fact is that, sometimes it IS better to do it yourself instead of using the package and most wouldn't. Sometimes.
On the other hand, if the existing libraries/solutions don't fit your use cases and you spend more time customizing them to work than you do developing your product, or the existing solutions are out of date, unsupported, or appear slapped together, it would certainly be worth working on your own solution to see if you can find a better way. Likewise, experimenting with custom reimplementation is a great way to understand the problems libraries solve in an in-depth way, as Dan suggests, and you may even end up with something better. You just need to be realistic about whether it makes sense to make it yourself or utilize what exists.
> But we also need to make it easy for product developers to do the right thing. What can we do to make accessibility a default rather than an afterthought?
Yes! We accessibility advocates have been wishing for this for decades. In the context of web development, application developers should rarely, if ever, have to reach for the ARIA role attribute, because they should be able to re-use existing rich widgets rather than implementing custom ones. I'm hoping that Web Components-based toolkits like the new Ionic 4 will help here. Then project boilerplates should include some kind of accessibility testing by default, so developers will have to go out of their way to ignore it.
The dream is framework agnostic Web Components that "just work" in a modern browser. You can make it happen with a bit of polyfill javascript. My worry has been spending a bunch of effort making them and then having to maintain separate sets for all the different frontend "web component-ish" frameworks so that people would actually use them.
So if you don’t end up using a modern popular framework, you have already invented one that is undocumented.
As the QA gain mastery over accessibility the developers are on the hook to implement the proper controls. It is easy to get upset when developers look like a bottleneck and hold up releases, but nobody gets upset when QA identifies potential liability and a shitty product.
[0] https://getbootstrap.com/docs/4.1/getting-started/accessibil...
I think of JSX as a declarative way to redefine HTML and to make it fit to the desired design. Sometimes it's quite possible to find a common behavior of a component and it can become part of a library, but that's not always the case. I like to use react components as a slim layer and prefer to do the work (side-effects, data processing etc.) somewhere else (purely functional modules, redux-saga etc.). My goto-rule to create components became the following: "if I start naming my component with something else than with layout-related terms, then I'm using react as the wrong tool for the job". React is just the view layer, a data processor that puts the view-data into the DOM. And as such a view layer it's sole responsibility is to render conditionally, the responsibility that your linked library tries to extract. I personally prefer to read through early returns in the render method of a component as opposed to following the order of an array.
There are behavioral UI problems in an SPA that are not related to the DOM itself. React can't do much about that and I think the problem of entropy stems from there: In the app at work one of the biggest struggles was to keep a global lock for a modal. This modal can pop-up to the user and show a notification, confirmation, form etc. Modals should never overlap with each other, give the user all the time she needs, but should also follow priorities (i.e. a push-notification to cancel the session being one of the highest). The difficulty in managing this lock was that actions from everywhere can pop-up that modal, be it a push notification over websockets, functions called from the native side, failed/succeeded API calls or behavioral inputs that should work differently all over the app (i.e. barcode scanning a product will search for a product in one view, will add it to the cart in another, or will prompt for a follow-up action if no product could be found). Thanks to redux-saga, we can make use of actions not solely to update the store state, but can additionally (if ignored by the reducers) use them like a message bus in a concurrent system. So with redux-saga (and inspired by elixir) I could make use of the actor-pattern and build a supervisor saga that keeps this behavior maintainable, but there is way to much complexity in this.
Point being, react and redux do a great job at managing entropy as long as it's used in the correct way. I think each component that doesn't take props can in fact be the starting point into it's own (micro-)application. But I think the biggest difficulties are external influences for an SPA - those interconnected influences that make the web so attractive for an application.
EDIT: typo
[1]: https://html.spec.whatwg.org/multipage/semantics.html#meta-t...
You didn’t write a fetch ever, you subscribed to a data feed from the data feed repository.
To prevent redundant fetches, the data feed was served by a cache, which then filled cache entries by doing the fetch.
Doing a POST invalidated the whole cache.
Which triggered fetches of everything, but only once/API. The fetch return pushed out to all the subscribers. The subscription was driven by the JSX, so only visible items had active subscriptions. Essentially, you wrapped your UI with the fetcher component, during your JSX display tree into a data dependency tree.
Optimizing the cache invalidation was a problem we ignored for later, because the fetch overhead wasn’t too bad, and we went from doing 40 fetches/page without the cache layer to 10, so the back end never noticed.
Entropy handling should be the goal for 2019 UI/Frontend engineering.
It can certainly still be greatly improved, but I think we've already made great strides in entropy handling on the front-end in recent years.
Virtual DOM is rather a desperate move - to support components life time constructor/destructor events componentWill/DidMount / Unmount.
What if you will be able to define in standard HTML/CSS something like this (as it is supported natively in Sciter)
// css
div.mycomponent {
behavior: MyComponent url(components.js);
}
// script in components.js
class MyComponent : Element {
function attached() { /*constructor*/ }
function detached() { /*destructor*/ }
… other custom component specific methods …
}
With that simple mechanism you don't need virtual DOM and its overhead at all. Component binding requires only inclusion of that CSS.But it is still far away from standardization.
Sciter uses this feature almost 10 years.
All these 10 years we have libraries of reusable components and so no need for React.
Same is about flexbox and grid: https://terrainformatica.com/2018/12/11/10-years-of-flexboxi...
Virtual DOM is not just about componentWill/DidMount/UnMount. It's when updating the view when a component's inputs (props) change that Virtual DOM shines.