Hacker News Reader

Comment by thedavidprice | Hacker News Reader

Co-founder of RedwoodJS here. We are so excited and proud of what Redwood has become, both as a project and as a community! Whether you are setting off to start your side project or looking to become an open-source contributor, I'd like to personally invite you to join us.

With Redwood, no one has to go it alone.

If you have any questions along the way, don't hesitate to reach out to me. I'll be watching comments here. And my DMs are open everywhere.

Join the Redwood Community: https://redwoodjs.com/community

So very excited to see what people with Redwood

View on HN

I have a simple question -- I've looked at GraphQL in the past and it seems extremely limited when you start wanting to do more complicated types of joins for performance, etc. This is such a fundamental thing to any webapp in my mind, and yet when I ask this previously people are often like "uhhh yeah...... use joinMonster I GUESS?"

Has GraphQL improved? I literally can't fathom using some kind of external plugin just to load data using more complex queries for performance reasons.

ever considered db views instead of doing a bunch of joins at fetch time?

Yes, but not every query should be a view (materialized or not) because the ORM doesnt support things like lateral join, cross join, etc.

Why can't you just build a special GraphQL query with the SQL query you want? I mean having something like

    query {
        myCustomQuery(...params...) {
            id
            whatever
        }
    }

backed by

    async function resolver(ctx) {
         const rows = await db.query(sql`SELECT yadda as id, badda as whatever FROM dabba JOIN dada ON ...`);
         return rows;
    }

Well if that is possible, that's great, but when I've asked similar questions before the answer is always to use some external library like joinmonster or whatever.

This is easily possible with the low-level graphql NPM package which is the base of many frameworks. Most give you an option to get through the abstraction and do it.

How refreshing to see something else than foo and bar

Nothing about GraphQL causes a join monster.. that's just unbounded limits on the front end queries.

You could validate against having too many layers of nesting in one query, validate pagination, provide abstract relationships for the many nesting and special case those joins, etc.

It's basically a question of how to efficiently query data. There's always ways, but it might require some trimming of allowed graphql queries.

I was referring to this: https://join-monster.readthedocs.io/en/latest/

> You could validate against having too many layers of nesting in one query

The GraphQL Server in RedwoodJS does query depth limiting out-of-the-box. You can configure it, but by default the depth is 11 -- because all the best things go to 11.

[1] https://redwoodjs.com/docs/graphql#query-depth-limit

GraphQL is designed around CRUD fetching patterns - so typically you would have a query per data type slice, and that query can be as complex as you like. In order to avoid N+1 queries, you should use dataloader: https://github.com/graphql/dataloader.

You can also join additional data to reference in deeper resolvers, but that's an antipattern AFAIK.

The latest release of dataloader happened more than 2 years ago.

Does it need more work? I'm genuinely asking, I've not used it

Dataloader is a super simple idea + library; I doubt it needs any updating. Edit: great video on it by creator: https://youtu.be/OQTnXNCDywA

It's a pretty reliable indicador of disuse.

FWIW, the Guild (the group behind a lot of the more popular GraphQL libraries) has recently take stewardship of this package, and it looks like there are already a few recent PRs merged into the repo.

Lack of updates also isn't necessarily an indicator of disuse. It currently has close to 5 million weekly downlaods, roughly 1/5th of express which is probably the most popular library for building APIs in node.

Postgraphile turns postgres schema into GraphQL schema, can do, filtering, aggregates, computed columns, filtering on computer columns etc

It really depends on how the resolvers are written, GraqhQL is just a interchange mechanism like RESTful APIs . You can do same things good or bad on both.

Depending on the framework that use for your DB-to-GraphQL layer (like Hasura / join monster/ PostGraphile / subZero etc) there can be some there can be some limitations on what it can and cannot do, but that is not really a GraphQL problem.

The GraphQL specific gotchas for me where

- For most part component level queries works well, occasionally batching in higher component is important for performance, for example instead of executing a secondary query on each row of the table ( say fetching photos of user profiles from a different store) doing it one level up can speed things up.

- Caching has to be solved very differently than RESTful APIs, GraphQL has everything in one endpoint as POST and you cannot leverage CDN's to cache the canonical resource URLs

- Less verbosity or nested request-responses could be important for UX performance, using HATEOS style discoverability usually results in many APIs calls 2-3 levels or more deep, in RESTful systems that is not a major problem as you are caching API responses in nodes in dozen locations/CDN PoPs near the user but that is not possible in GraphQL, so you have to keep mind of verbosity, that can also impact how you store and index your database and how you configure and run replicas etc at scale.

---

Anecdotally I have used Hasura as GraphQL frontend for an existing mature PostgreSQL db and APIs, (1 tb+, 1000's of concurrent users), and it scales fairly well even with 4-5 level nested joins or fetch ten of thousands of nested records. Hasura also does subscriptions reasonably well and they don't use triggers for implementing it.

Some areas that needed a bit work around for me - recursive CTEs, cross-db joins, performance of very large counts ( postgres problem - using HLL or tuples from the planner if approximate values are fine.

My utmost gratitude to you David, Tom, Peter & Rob, getting on the RedwoodJS train early on has been a great choice of life changing vehicle. Also, very excited about that launch week! Can't wait to see the community grow.

So, going through your tutorial and coming up on an issue/error... when I scaffold the 'posts' I keep getting an error:

    API listening on http://localhost:8911/
    api | GraphQL endpoint at /graphql
    api | 15:23:32  Server listening at http://[::]:8911
    web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request 
      localhost:8910/graphql to http://[::1]:8911/ [EADDRNOTAVAIL] 
      (https://nodejs.org/api/errors.html#errors_common_system_errors)
    web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request 
      localhost:8910/graphql to http://[::1]:8911/ [EADDRNOTAVAIL] 
      (https://nodejs.org/api/errors.html#errors_common_system_errors)

Any chance we could be of help over in either:

Redwood Forums: https://community.redwoodjs.com

Redwood Discord: http://discord.gg/redwoodjs

Have now, direct link if you want to take a gander: https://community.redwoodjs.com/t/tutorial-issue-error-occur...

Thanks!

The address the api server is listening on is different from what the proxy is pointing at?

  [::] -> [::1]

What’s your business model?

RedwoodJS is an open-source project. We are in a unique position to receive funding from Preston-Werner Ventures, which allows us to develop the codebase as well as cultivate an amazing community.

Although it's not a typical for-profit business model, we are sustainable and will enthusiastically continue building Redwood to v2 and well beyond.

You can read more about all of that in today's blog post from Tom (aka mojombo): https://tom.preston-werner.com/2022/04/04/redwood-v1-and-fun...

What’s your business model?

Millionaires Toy Project.

Much nicer for all of us than most millionaires' toys.

Instead of giving an edgy reddit-like repeated response in clear pursuit of karma, perhaps you could read the link the GP comment just gave you.

> So my million dollar spend on Redwood development comes with no strings attached, except that we continue to focus on building the best app framework for startups.

It's a personal investment and a passion project. Unless they're lying, there is no business model.

I wasn’t being edgy and I did read the link. A one-off donation isn’t a business model.

> Although it's not a typical for-profit business model, we are sustainable

If it’s sustainable, there's clearly something other than a one-off donation. What is it?

Right now there is no business model, on purpose. This is why I'm personally supporting it. I think what David meant is that we are sustainable because I am committed to sustaining the project. Our goal is to find a properly sustainable model in the future that meshes well with our mission and ambition. It will be exciting to figure that out!

Good on you, one could've said the same about Next.js. Technically there is no business model, but one has been built providing additional services around it.

Congrats, it looks damn awesome.

Next.js was an OSS project before ZEIT (now Vercel) hired on the developers, if memory serves. Next.js was certainly not the main vision for Guillermo as far as I know, but it ended up becoming a very large focal point due to its widespread success. Now, Vercel has an entire ecosystem - Next.js is certainly a 'product' in that regard.

Preston-Werner is an investor in Netlify, RedwoodJS was built with Netlify in mind (though it supports others hosting companies but not to the same extend).

Netlify is competing with Vercel that has Next.js. So basically RedwoodJS was built to sell more Netlify hosting plans the same way Next.js was for Vercel. It is therefore safe to say that RedwoodJS will continue to exist as long as Netlify is a thing... unless Next.js completely owns the market and RedwoodJS's reason to exist makes no sense because of too few users.

Not everything needs a business model. Not all projects are businesses.

Sure – so when somebody asks what your business model is, do like mojombo did and say there’s no model and there’s a patron instead of doing what thedavidprice did and say that it’s not a typical for-profit model but they are sustainable. One is a clear and simple answer, one is a weird dodge.

Any details you can share about why the project is using rather obsolete tools (yarn, webpack)? I understand they're old and stable, but I took the project for trying to be on the leading edge.

These were state of the art tools when we started work on Redwood more than two years ago. We're investigating other more modern alternatives, but swapping them out is a finicky and time consuming project and we've been focused on primary features. We will keep evaluating our dependencies and you may see some changes in v2!

Thanks for the open answer, appreciate you checking out the question.

Can't help but giggle at the downvotes.

Tell me more about "obsolete", because I don't think you mean what that word means?

22M weekly downloads – that's pretty much zero. So yeah definitely obsolete.

It means no one has written a "Why we switched to webpack" article in years so that means it's completely obsolete and useless.

Hacker News logic 101.

Webpack is obsolete?

Man, this field really does move quickly.

I don’t think Webpack’s obsolete, it’s just got a lot of competition these days, most of which is easier to use.

webpack was older when rollup came out than gulp was when webpack came out. if not for create-react-app's impact, we'd probably all have moved on 2-3 years ago

Wow. Weird reasoning. Never thought someone could be so openly ageist about… a module bundler.

Could you maybe provide a better argument for why you think we’d all have moved on by now? Aside from ‘wow its not as new as the new thing’?

I don’t know much about rollup, it doesn’t seem to be that different, but some of the newer tools are rust and go based and better tuned for performance during development. JS tooling is generally slow and weirdly complex, and people seem to acknowledge these issues and improve on them.

that's just how our industry behaves. we like shiny new conveniences and tend to want to adopt them as soon as they're viable.

this moment with postgres, node, webpack, babel, scss, and react feels off in a way. there are genuinely more capable alternatives to all of those that take less time to learn and work effectively with, perform better, cost less, and are trusted by big institutions. rails never felt this entrenched when it was the leading default, for example.

there are a lot of positives to this that non-web engineers have seen for a long time--education becomes better and more accessible, people can have more stable and successful careers, and groups can take on more ambitious missions--it's just surprising from how i've seen web engineers approach their trade over the past 15 years

Yarn too, apparently. We'd better go tell..... everybody :)

What do the cool kids use instead of Yarn these days?

Dunno, they don't invite me and my yarnie buddies to their parties.

Ha! My thoughts exactly. If I had more time it would make for an interesting post about all the lessons learned as we navigated the pains of Node.js in all its forms.

Lesson One — Building and Bundling are hard but don't make it harder.

We (by that I mean Peter and Danny) took a hard look at new tools, and, well, they let us down. We reverted a few steps that took us months to figure out. But we learned. And someday we'll make a move to different tooling. But in our opinion, those tools are not ready yet. (Side note: I read a tweet about how buggy SWC is with Next. Anecdotally, although we're stoked for the future of SWC, it remains as a draft PR yet to be merged in the Redwood Repo. We also reverted the primary part of our ESbuild implementation. So the tweet jived for me with our experience.)

The hype is all about speed and performance. But I think the reality is fairly simple:

- implementations become unintelligible spaghetti over time

- everyone has been burnt by having to manage this themselves

- poor implementations lead to diminishing performance

Turns out the existing tools are fairly robust. But to use them we spent an epic amount of time tuning and refactoring. It's complex. But it isn't broken. If you're willing to let good enough be good enough, set a benchmark, hit it, etch it in CI, and call it good.

Lesson Two — Let go but pin the packages

Note: We chose to have @redwood packages include all the dependencies and config and integration under the hood. Redwood apps have very little package overhead out of the box. This means we have to tame the dragon behind the scenes.

The "cool kids" seem to be moving to pnpm (and/or Yarn pnp). I've been burned many times by the monstrosity that is node_modules, but, to be fair, it's also fairly incredible what we're able to achieve. I don't think any one package manager is going to be the cure for what ails us. But I do think it's possible to MakeItWork regardless of what you choose:

- pin your packages because the dependency of your dependency is your enemy

- lean hard into CI; then lean harder

- stay current; give in and let Renovate update

Yarn 3 is a phenomenal tool. (Plus the core team is top-notch.) And we're not even using pnp yet. We do have a long way to go and yes, a faster install would be great and a smaller size on disk and etc. and etc. But, again, I think people have been burned by having to figure it out themselves and finding things unmaintainable in the long run.

---

I'm tired and likely not making much sense anymore. I might have lost the thread, but I think the overall point I was trying to get to is this --> zOMG I'm so tired of talking about build tools and package managers and chasing incremental performance gains and can't we just focus on features that move the needle and have fun again? Oh, and PIN your damn packages 'cause that "^" is nothing but trouble!

Thank you and good night.

Non frontend expert here trying to learn.

What would you recommend instead o yarn and webpack?

I have been using vite and npm in my small frontend projects.

Older versions of npm (6 and below?) had many issues. Recent versions are more predictable. npm is fine. yarn 1 has always been a solid choice.

There's nothing wrong with webpack except its lengthy build times. It's not a knock on webpack, it's more a knock on using JavaScript (node) for everything. Vite uses esbuild which is coded in Go. Next.js adopted swc (Rust) and hired the lead.

I recommend Next for large react apps and vite for apps that would otherwise use create-react-app. Obviously if you're doing Vue, use vite.

Reply on news.ycombinator.com