Why we decided against GraphQL for local state management
tech.okcupid.com
tech.okcupid.com
MobX is probably the best way I've found to manage reactive state with React, Inferno, etc, but still, it's a huge piece of of software for just this purpose.
I first moved to Mithril because it doesn't need reactivity. You solve the same problem than React/Vue et al in just 10kB. No need for an external router or a state management library. Your state is just vanilla JS. Mithril doesn't need to know when a particular piece of state has changed, it just tries to re-render everything and the vdom takes care of the rest. The performance is comparable to Vue 2 and React [1] but the DX is lightyears ahead. It's a sushi chef knife though, you really gotta know what you're doing[2].
Now I'm using Svelte because it's even better. It does have reactivity but the compiler just figures out the dependency graph so you don't need to ship a huge thing like MobX to do that at runtime. You also write a fraction of the code compared to all the other frameworks I've used which is Svelte's best feature. This compiler approach is a game changer IMO.
[1] https://krausest.github.io/js-framework-benchmark/current.ht...
[2] What I meant by this is that since your app is essentially vanilla JS you're on your own to architecture the thing. If you don't know what you're doing you will shoot yourself in the foot.
I have a nagging problem that keeps coming back to me though. The declarative state model is the holy grail for UI design, but if you try to mix in any form of imperative logic, it all falls apart. All of a sudden you need to understand and plug into poorly documented and opaque component lifecycle APIs in order to insert your imperative code and have it work correctly. And that really messes up the nice clean solution, making it look more like a chimera of two state models.
Some of that is fixed by libraries that create components native to your framework of choice, implementing the appropriate lifecycle hooks as necessary. But now you're relying on a library that adapts one library to another library, and now you've got a dependency sync problem on top of it all.
It wouldn't be such a problem if there weren't so many extremely useful libraries that depend on imperative code. Personally, I've run into massive problems trying to use D3, Leaflet, and Bootstrap, as well as a handful of others. I'm sure there are thousands more.
Not trying to take away from your point. When these declarative state models work well, it's a beautiful thing to behold. But there's always some point where the abstraction leaks.
https://svelte.dev/tutorial/onmount
https://reactjs.org/docs/state-and-lifecycle.html
https://blog.logrocket.com/introduction-to-vue-lifecycle-hoo...
I found you have to do some extra work to keep track of things to convert from declarative style to Leaflet’s imperative style, but overall I actually had a pretty good experience with it. Feel free to ping me on Twitter (same username) if you have any questions.
Can confirm. My team of mostly fullstack devs that prefer backend ran into lots of issues with too much freedom in Mithril.
Ummm... no, state management is pretty much the only hard problem in programming.
Then you realize the "6 screens" actually turn in to a 50 step state machine that shares state between screens and changes screen logic conditionally on other screens, and the designer created sketches for the happy path.
And your "really simple straightforward state management" now has you going through 10 steps and 3 screens before you can arrive your state 11 which you are working on, you can't unit test the logic because your state is tightly coupled to UI and the HTTP stack, you can't reason about state updates because it's happening all over the place and is written by 3 other people concurrently, and coming to the project 6 months later makes you quit on the spot.
I've written plenty of WinForms to JQuery UI to know that "simple state management" is anything but.
Redux has initial overhead but down the line it pays for it self in these scenarios many times over. Going over requirements/flow with product owner and then writing tests for a reducer specifying the expected states is incredibly powerful in ironing out miscommunication in requirements and much faster than sending iterations of UI app to testing.
Like the parent said - the hairy part of the app ends up being dealing with the state machine - shuffling data over the HTTP is trivial, CSS/layout/animation are not that hard if you know what you're doing and don't need to support ancient browsers.
Completely agreed about Redux and other over-engineered state management
By the way did you ever look at http://meiosis.js.org for managing state?
The article touches upon current Redux as well, such as when it says "If done right, it can definitely be a breeze to work with", which I agree with. And the "if done right" part has gotten a lot easier, as Redux has finally decided it's ok to provide opinionated approaches that guide you towards that happy path. I really do think if Redux hadn't been so concerned about being opinionated way back when, it'd be far better received today. But better late than never.
If it's simple just put handle async stuff in situ. If it's more complex, use custom middleware. Thunks, sagas, etc are all anti-patterns. The single worst thing the Redux docs did was give the impression that middleware was some kind of advanced functionality only useful for library designers. Most of your app logic should probably live in middleware.
The "redux toolkit" or whatever doesn't help with that, it only reifies questionable practices. Skip it. Write a simple utility for generating actions/types, and then go about your business.
Thunks are simply an approach for writing reusable async logic that has access to `dispatch` and `getState`, without being tied to a specific store [0]. While I do think more people would benefit from writing middleware for their own particular use cases, most people just want to have a place where they can fetch some data and dispatch an action containing the result. Thunks make that straightforward. Thunks are also by far the most widely used async middleware across the Redux ecosystem. These are all reasons why we settled on thunks as the default async middleware included in RTK [1]. (It is worth noting that with the advent of `useDispatch` and hooks, you can write some fetching logic directly in a `useEffect` call vs a thunk, but there's still benefits to using thunks in many cases.)
RTK has specific support for thunks in two ways. `configureStore` automatically adds the thunk middleware to the store setup [2], and we have a `createAsyncThunk` API [3] that handles the common pattern of dispatching actions based on the results of a promise.
However, nothing about that requires that you use thunks with RTK. You can still add whatever middleware you want to the store, whether it be sagas, observables, custom middleware, or something else.
> The "redux toolkit" or whatever doesn't help with that, it only reifies questionable practices. Skip it. Write a simple utility for generating actions/types, and then go about your business.
I'm afraid this is entirely wrong.
RTK encodes the best practices recommended in our Style Guide docs page [4], and includes APIs that simplify your Redux logic considerably:
RTK improves your Redux code in many ways:
- `configureStore` lets you set up a Redux store in one line with good defaults built in, including automatically adding the Redux-Thunk middleware, enabling the Redux DevTools Extension, and warning about accidental mutations
- `createSlice` generates action creators and action types for you automatically - all you have to do is write reducers and given them reasonably descriptive names. In addition, it uses Immer internally to let you write "mutating" reducer logic that is safely turned into correct immutable updates, so no more nested spread operators.
- As mentioned, `createAsyncThunk` handles the typical use case of dispatching actions before and after making an async request - just fetch your data and return a promise, and it'll dispatch actions automatically.
- `createEntityAdapter` provides prebuilt reducer logic for typical collection management operations, like `upsertMany`, `addOne`, `removeAll`, etc.
So, RTK _is_ that "simple utility for generating actions", and more. It's an official package from the Redux team (ie, myself and the other maintainers), you can pick and choose which of its APIs you actually use in your app, and you can mix and match which parts of your Redux logic are written with RTK with parts that might still be written with other approaches.
[0] https://blog.isquaredsoftware.com/presentations/workshops/re...
[1] https://blog.isquaredsoftware.com/2020/02/blogged-answers-wh...
[2] https://redux-toolkit.js.org/api/configureStore
None of what you're saying matters in the scheme of things. You can get everything you need from
const { types, actions } = createActions([ 'ACTION_NAME', ...]);
The middleware approach is more straightforward than thunks and far more maintainable. Reifying that as "best practice" is only going to continue to spread this anti-pattern because no real application is about "just grabbing some data for a bit" and inevitably that one api call expands into many, not to mention all of the other side effect related and asynchronous functionality that one has to deal with in user interfaces.
I've confronted him once but to no avail, I even got downvoted by the community. :)
https://www.reddit.com/r/reactjs/comments/fvuwpl/handling_si...
edit: i see that's happening to you too here lol, godspeed
Other than that, I just spend too much time on social media :)
const todosSlice = createSlice({
name: 'todos',
initialState: [],
reducers: {
addTodo(state, action) {
const { id, text } = action.payload
state.push({ id, text, completed: false })
},
toggleTodo(state, action) {
const todo = state.find(todo => todo.id === action.payload)
if (todo) {
todo.completed = !todo.completed
}
}
}
})
export const { addTodo, toggleTodo } = todosSlice.actions
export default todosSlice.reducer
https://gist.github.com/weeksie/04f41329d0015e2128209df9f9de...
I skimmed the debounce middleware from this guy Nir Kaufman who has some pretty good things to say about front end architecture. Generally with Redux we have a system that can absolutely provide real event-driven CQRS style front end so it seems crazy to ignore that in favor of managing async thunks like it's 2004 and we've all just discovered how to put an XMLHTTPRequest on an onclick handler or something.
Outside of that core.effects file, I put all of the async app logic in custom middleware as well. YMMV but using a framework for replacing const declarations and switch statements is overkill.
The broader point is that if you're going to create utilities like that, it's often worth just rolling your own. When you attach a core part of your application to a framework you've signed up for the ride. I realize it seems like I'm being an asshole to acemarke, who appears to be a perfectly lovely dude, if a little passive-aggressive, but I sincerely feel like pushing libraries for problems that shouldn't require libraries is making our ecosystem worse.
Can you clarify what you mean here?
From my perspective as a Redux maintainer, most of the concerns I've seen expressed about Redux over the last few years really didn't focus on whether it was "opinionated" or not. It's been a combination of:
- Not understanding the original intent behind Redux's design and the historical context that led to it being designed that way [0]
- People being told they _had_ to use Redux with React, even when it really wasn't a good fit for their use case
- The "incidental complexity" of common Redux usage patterns [1], most of which were due to the way the docs and tutorials showed things (writing `const ADD_TODO = "ADD_TODO"`, separate files for actions/constants/reducers, etc).
- Changes in the ecosystem leading to other tools whose use cases overlapped with why you might have chosen to use Redux previously
All that said, yeah, we've made a concerted effort in the last year or so to be much more opinionated about how you _ought_ to use Redux, based on how we've seen people use it. Those opinions are built into our new official Redux Toolkit package [0], which is now our recommended approach for writing Redux logic, and the Redux Style Guide docs page [1], which lists our guidance on best practices and patterns for using Redux.
I also just published a brand-new "Redux Essentials" core docs tutorial [2], which teaches "how to use Redux, the right way", using our latest recommended tools and practices like Redux Toolkit and the React-Redux hooks API.
We unfortunately can't wave a magic wand and get folks to update the seemingly endless parade of Medium tutorials and Youtube videos that show the older approaches to using Redux, but I'm hopeful that the emphasis on using RTK will make it a lot easier for folks to learn and use Redux going forward.
[0] https://blog.isquaredsoftware.com/2017/05/idiomatic-redux-ta...
[1] https://blog.isquaredsoftware.com/2019/10/redux-starter-kit-...
[2] https://redux-toolkit.js.org
[3] https://redux.js.org/style-guide/style-guide
[4] https://redux.js.org/tutorials/essentials/part-1-overview-co...
I think this led to a lot of confusion and uncertainty with Redux. "I'm using it, but I'm not sure I'm using it effectively." I think a lot of ineffective implementations caused a lot of developers to get a really bad taste. I can unfortunately say at my previous company, just mentioning Redux would make just about every developer cringe. I hated that.
But I feel like a lot of what is making Redux more successful these days is the more opinionated releases like the toolkit and the new docs. Since they've only been around about a year now, I think it's reasonable to conclude that before their existence, Redux didn't have a strong opinion a new developer could lean on.
It's similar to React, which also tries to avoid opinions. I can appreciate and understand that approach (for both React and Redux), but I do wonder would React be better if there was a stronger opinion on things like CSS, routing, state, etc?
I don't mean to criticize. I love Redux and am very grateful for it. This is just how I experienced it over the years.
People on whom it was forced on, and who disagreed with those opinions, were VERY vocal, and pushed most of the other crap, which fragmented the community and muddied the message.
From my point of view as a end user, the only real difference between now and then, is that the vocal minority is now the majority, and they shifted focus to match (which is a good decision). Obviously I can't speak for Mark, but it's how it looks like as an external observer (pun not intended)
This is most glaring for the whole "do a GET and store it in Redux" thing, where if there's no actual data mutation or derived data or anything else that actually requires or makes use of a global scope, you should just be using a memoized API call function (or a library that abstracts away the common use cases of one, like SWR).
I know people that advocate putting everything in redux. They are quite serious about this. Then you end up with reselect everywhere and the whole goddamn universe is memoized. No one seems to know how React renders (or re-renders), so memoize everything! Yes! That's the ticket.
The overuse of Redux and the React hook brain damage stems from the tide finally turning against OOP. Haskell did a number on the OOP paradigm. So everyone is afraid of encapsulated state and localized logic. Then you end up with React hooks. Instead of making React's object API better, they choose to make functions worse. So everyone is pretending they are doing functional programming when in reality they are just using functions that are bastardized with this weird flavor of dynamic scoping and all the subtleties of that. But I digress.
Today, there are a thousand different ways to do web development wrong and practically no way to do it right. Starting with misconfiguring webpack and working up the stack. It's a true complexity quagmire. But surely there is a SaaS or open source toolkit (with adjacent commercial services!) coming soon to help guide us out of the fog.
Just kidding. This hell of ours is by design.
I'd be really curious to see how you'd make the object API reusably composable, given that that's one of the basic reasons for hooks existing as they do.
I'm gonna go out on a limb and guess that maybe 1/3 of those Redux-using apps probably aren't really benefiting from Redux (things like using it _just_ to avoid prop-drilling when you could just use Context, writing reducers that are nothing more than `return action.payload` with no real logic, etc).
We've always had an entry in our FAQ section that gives some advice for when it might make sense to put a particular piece of state into Redux vs keeping it in component state. As I rewrite the Redux docs piece by piece, I'm trying to add some additional clarification and emphasis on when you should actually consider using Redux overall.
Still, the bulk of the concerns I've seen over time fall into the categories I've listed - I just really don't remember seeing many comments from folks who were concerned that they "weren't using it effectively" as a particular pain point. Perhaps that was being felt, but expressed in different ways? (I definitely think a lot of people _haven't_ been using it effectively, but didn't necessarily realize it.)
In any case, the highly positive response to RTK and the docs work tells me we're headed in the right direction.
My next task is to rewrite the existing low-level "Basics/Advanced" tutorial sequence to remove outdated references, show simpler patterns, and improve explanations. I've got my notes on things I want to change here:
https://github.com/reduxjs/redux/issues/3855
If you or anyone else has any particular feedback on things you don't like with that existing tutorial or would like to see covered better, please comment and let me know!
You’re not solving any problems, you’re just promoting your framework.
He makes an effort to address specific points, doesn't get defensive or angry, and seems to take criticism and feedback seriously. He has responded to my own critiques of Redux in a reasoned way.
I think that if someone is going to express claims or opinions about a library on HN then we should be happy to see an expert response, rather than letting the discussion become one-sided. Sure, there could be a bias, but I think we all win by having both sides represented.
Do we really want to discourage that?
The problem with his style is that he parades himself as an expert but he's emphatically not an expert. He's a guy with an agenda to push his open source framework. I understand where the bias toward politeness comes from, but that makes groups like HN susceptible to this kind of pretense to authority. He's actively making working developers' lives worse by promoting this stuff and it's frustrating.
Perhaps I shouldn't give a shit.
Erm... I've been a Redux maintainer since mid-2016. I oversaw development of React-Redux v5 and v6, wrote v7 by myself, and directed development of the hooks API in v7.1 over the course of multiple 250+-comment issue threads.
I've written the Redux FAQ, "Structuring Reducers" usage guide, the "Style Guide" best practices page, almost all of the Redux Toolkit docs, and the new 25K-word "Redux Essentials" real-world tutorial.
I've written about 150K words of Redux-related blog posts, including tutorials, history, technical architecture, and best practices.
I've given multiple conference talks on how Redux works and how to use it.
Even if you don't agree with the approaches I'm recommending that folks use...
if _I_ don't qualify as an "expert" on this topic, who _does_? :)
If you have actual substantive technical arguments for other approaches we should be recommending, please feel free to open up an issue to discuss them. I'm genuinely always interested in ways we can make it easier for people to learn and use Redux.
How many complicated applications have you built with Redux at their center? Because that's the part that's missing from your list of qualifiers for being an expert.
But both were pretty complex - true SPAs, no routing, lots of complex client-side state manipulation going on.
In fact, the first one is actually what inspired everything that I showed in my "Practical Redux" blog tutorial series, since I couldn't show off work stuff publicly.
But hey, if that list of stuff I've done isn't good enough for you, apparently no one in this world qualifies as an "expert" in any topic :)
You are almost certainly an expert in Redux's internals but that's not what we're discussing. In fact the root of the problem is exactly that you've conflated one for the other.
I've looked at hundreds of other codebases, ranging from beginner apps to complex enterprise-level apps. I've talked to thousands of people who are using Redux in many, _many_ different ways, and catalogued hundreds of libraries people have made to add on to Redux. I _know_ how people are using Redux in practice, and what kinds of problems they're running into.
I'm quite serious when I say I'd like you to write up an issue with some specific concrete feedback listing your concerns with how we currently teach Redux and recommend people use it, especially since I'm working on an ongoing revamp of our core docs. I obviously can't guarantee any _changes_, but I'm very interested in having substantive discussions in a more suitable venue than HN comments.
Whether or not you choose to take me up on that is up to you.
The issue is not that one or another approach is always better, it's that the entire act of lobbing another framework onto the stack that a dev has to familiarize themselves with only adds to the confusion.
Rails worked because it contained everything in one package. The success of that model poisoned everybody's mind into believing there had to be "opinionated" frameworks to manage other (extremely simple) frameworks and has contributed greatly to the mire of misdirection that is modern front end development.
It's a case of not even wrong.
how is that?
You can write a short story about this on your blog, now that you’re a “full-time writer”.
Tangentially related, they linked to RecoilJs which I wasn’t familiar with. The presentation on the RecoilJs site finally helped me say “Ok, I get the local state problem”. Worth a watch and reading the previous HN discussion on it [0-1]
The new version just looks like yet another Tinder clone, and doesn't resemble anymore what Okcupid was good at: Good profiles, which allowed to get an initial impression about people beyond "I like their looks".
The entire internet is being optimized to either outrage you or sexually arouse you as quickly as possible.
To me, it seems that we're in a precarious position. I wonder what happens when we reach the limits of dopamine saturation, if there is such a thing. Otherwise, it seems Huxley was right -- "man’s almost infinite appetite for distractions" is a dangerous thing.
I absolutely hate how accurate this is. It feels more and more that the optimization and efficiencies are making the internet less useful but more addictive.
Many posts are written about end results. "Look at this thing we built that is cool and works well." Thanks for sharing the full story, OkCupid.
The tools for client-side state management with GraphQL are really powerful, but require a lot more involvement than something like a standard state management store, IE an action that calls your GQL endpoint and stores the data in state.
async function fetchUsersAction(state) {
const users = await gql.getAllUsersQuery()
state.users = users
return state
}
This approach has worked well for me in a number of fairly decent-sized apps, and it seems to be a pretty common one.
Its rough because Apollo does so much for you, its wonderful, but there are so many sharp edges and pitfalls - from the buggy devtools to the lack of/incorrect docs, major unanswered github issues & constant API breaking changes even on patch/minor version bumps, its been an uphill struggle all the way. All that said, it still seems like the best tool for the job, Relay still has a huge learning curve & has already had two major branches with no refactor path between (Classic/Modern) & the numerous other implementations all lack things Apollo gives you for free.
If your usecase is simple, using Apollo to query a GQL server with a well defined schema doing little to no client state handling it can be pretty straightforward, but the complexity ramps up crazy quick as soon as you start messing around with caches or persistence or local state.
The Apollo folks seem to have a handle on things but hopefully I/we can start contributing back, if not with feature work than with documentation
In the past I had to write or figure out my own Redux solutions to store data from queries in the state and now that comes out of the box with Apollo. If I want to manipulate this state locally, I can just do a client.writeQuery in a standard JS function (no resolver bullshit). In the rare event I want purely client-side state, I can use the same method with the @client directive. I've never really needed resolvers as I can contain the logic in my `update` handlers or state manipulation (action) functions that look and behave like standard JS.
Caching is great out of the box. You get all variations you might want (cache-first, cache-only, cache-and-network) and it's very easy to configure. Not sure what problem you ran into. Maybe server-side?
State persistence is a whole different story. I've found it's hard to do well due to Apollo's ROOT_QUERY combining all pieces of the state. It requires almost an all-or-nothing approach or a lot of hacking to get it to work. I've spent a lot of time searching for proper solutions and even built my own, but they've all been insufficient. Considering local storage persistence brings XSS vulnerabilities as well, I've decided to focus my efforts on performance for now which helps both returning and new users.
Personally I love Apollo for its state management and will continue to integrate it in more projects. My biggest gripe at the moment is writing maintainable client-side queries. The code gets quite verbose, especially when you try to do proper (TypeScript) typing without splitting up your code into tiny fragments all over your codebase. And keeping the types in sync with the queries and server-side schema takes a lot of time and effort. I've mostly been thinking about solutions to auto-generate client-side query types, this would increase type-safety as well.
The learning curve isn’t that really that steep, it’s just made artificially so by the poor docs. It’s unfortunate that the Relay team doesn’t really prioritise getting new users, because it’d motivate them to explain things better. But there’s not actually that much to learn, you can be up and running and productive with it pretty quickly.
Its like your favorite screwdriver that you somehow always manage to cut yourself on - you're extremely grateful that you have this tool, it lets you tighten screws even in the narrowest spots, you just wish the handle wasn't so damn sharp
This adds extra overhead on every single mutation as opposed to Redux, where your the current state is automatically passed into your reducer, and you don't need to worry about writing anything after you update - the store handles that itself by updating the state to the result of the reducer.
I don't mind Redux for local state management, but it's a little overkill if you aren't using it already to handle API state.
Using Apollo on the client side makes me want to pull my hair out though.
Redux still fills that role very well.
Personally, I don't like the "write client-side resolvers mimicking the server resolvers" approach. I'd much rather have a database that is synched with the server and listens for new changes. Once you have your "offline-synched-db", you get offline, optimistic-UI & real-time for free.
If you mix everything in one then you have a crazy ball of goo that’s rendering the whole UI tree everytime something small happens. Sure vdom is fast but you’ll still feel it for large apps because generating that vdom may call expensive functions over and over.
Svelte is nice but I haven’t used it for a large project. I do swear by having a good separation of those states. It has served me well for over a decade building UIs in different frameworks.
When component state changes, only that component changes. E.g whether a dropdown is opened.
When appState, changes the whole app tree is vdom diff rendered. Immutability helps here. Props/context are threaded all the way down. E.g the current light/dark theme
ServerState should be subscribed individually by components that need it, and only the components looking at that state render. (E.g List of items from GET call, when you update the item, an update only needs to happen in that one place that keeps its data synced with server)
This means that Apollo is essentially incompatible with Redux unless you are willing to duplicate the data in the Apollo cache into your Redux store. Such duplication is a waste of resources, and also defeats the point of using Apollo to begin with, since you are circumventing most of its feature and just using it as a plain GraphQL client.
I think Apollo is trying to do too much, and a much better architecture is not to couple your GraphQL query infrastructure to your state management infrastructure. This can be done by using a more low level GraphQL client and then choosing one of the many amazing state management libraries out there in combination. This way your GraphQL client does not dictate your state management layer.
The holy grail of "fixing race conditions" and avoiding async bugs is RxJS in my opinion, which is why I am working on what I call a "stream management library", its a state manager where your "state" is represented as RxJS streams https://rx-store.github.io/rx-store/
To remedy this, we're currently moving away from it. Either to Redux or just plain useReducer (TBD).
This is the exact thought I had recently when fighting the Apollo cache.
That's two months they spend learning GraphQL, two months introducing a mystery-meat binding layer and learning that, and two months debugging it all and pulling their hair out.
And no, using a giant third-party lock-in toolkit like Hasura or Apollo is not a response.
I don't see any problem using Hasura for prototyping. It offers pagination, aggregation and other things that usually only matured REST APIs offer. So basically it's possible to post-pone the Web Application-Backend, see what queries make sense and later "hard code" that with the preferred framework. At least that's how I'm approaching a side-project right now and I'm really happy that I can focus on the open tasks and keep the mundane work for later.
Drupal however has a history dating back to 2001!
And if we’re picking Drupal because of its age (and if not then why Drupal, of all things? What’s the overlap with GraphQL?) why not go back further and assert that someone who’s great with C and CGI could do even better?
That said, the presence of GraphQL is a great signal that a team is infested with architecture astronauts, so I suppose it has its uses.
As in industry, we're okay with a frontend JS framework - React (or vue or angular) being part of the baseline set of investments you make when building a website. It's just considered the cost of doing business. (But that wasn't always the case! I for one don't miss using jQuery...)
I imagine GraphQL will get to a similar place - where the investment is more well understood and accepted.
(Of course common sense still applies - if a website is simple enough that it doesn't need a frontend js framework then don't use one - same goes for graphql, don't use it if you really don't need it)