Hacker News Reader

Comment by blueslurpee | Hacker News Reader

>Sure, in the ’80s, it might have been possible for one person to do it all: systems administration, database, coding, design. But in today’s world, database administrators, infrastructure engineers, coders, designers, project managers and others — whether on staff, on contract or with a vendor or service provider — must work together to execute projects quickly, securely and reliably. Tech today is a team sport.

I find actually the opposite to be true. The leverage of individual engineers is only growing with the advent of SaaS and the proliferation of API's. Then of course there are the stories of small engineering teams with Unicorn exits, e.g. Instagram, WhatsApp, etc.

View on HN

I want to preface my thoughts with that I wish the "team sport" mentality was more common in the dev world.

Unfortunately, I have to agree with blueslurpee here. I frequently join a team and find there are 1 or 2 devs who "know everything" and it is near impossible to pull the information out of their heads and into mine or someone else.

This might be isolated to the areas I work in, but I do find that the "10x engineer" myth seems to live on and prosper in far too many teams I've worked on. Despite how discouraging it is for the rest of us who want to play a "team sport".

How much of that is because those people aren’t being team players vs them learning things differently?

In my experience the people who “know everything” either wrote the code from scratch (thus understanding it at a deeper level than the quick summaries people would use to describe it) or just read.the.damn.code. This is a problem I also deal with working with junior team members who would rather ask a question about how something works rather than read the source. Even if I do try to explain it, I can’t explain it to the fidelity and nuance encapsulated in the actual code. Nor can I explain the context as well as clicking “find references” does.

As such I am not sure those people who know everything are secretly zealously guarding their information. Any brief explanation of something technical is simply too low-res compared to reading its implementation. So even if that person is super nice and tries to help as much as they can, I doubt anybody would learn as much as them just from secondhand information, as opposed to reading/writing/modifying the source themselves.

> In my experience the people who “know everything” either wrote the code from scratch (thus understanding it at a deeper level than the quick summaries people would use to describe it) or just read.the.damn.code. This is a problem I also deal with working with junior team members who would rather ask a question about how something works rather than read the source. Even if I do try to explain it, I can’t explain it to the fidelity and nuance encapsulated in the actual code. Nor can I explain the context as well as clicking “find references” does.

Depends on the shop.

The trend is toward people trying to standardize and automate as much of the development process as possible (so that they can get rid of domestic employees who cost to much and fetch it all over to people in exploitable-countries with less valuable currencies).

People who ask questions, push less ambitious changes, push buggy stuff that barely squeaks past the unit tests and then go back and improve it later if someone complains.. all of that leaves activity metrics that check boxes on the reports of people who don't care whether devs read and understand the existing code or not.

I share my screen and do an initial source code walkthrough of the overall structure of the project / component, and make sure they can build and run the code, before asking a junior team member to read the code themselves and figure the rest out on their own. It's worked really well, in my experience.

> This is a problem I also deal with working with junior team members who would rather ask a question about how something works rather than read the source. Even if I do try to explain it, I can’t explain it to the fidelity and nuance encapsulated in the actual code. Nor can I explain the context as well as clicking “find references” does.

As a junior, this is a problem for me too. I don't know when it's the right moment to ask something. "When I've been stuck for an hour to half a day" seems to be a good heuristic, but it's not perfect. I don't really know what's the best option for me. If I ask later, I learn more by myself, and improve my skills in figuring out a codebase. I also let senior engineers be more productive. If I ask earlier, I'm more productive. But I take more of the senior engineer's time.

Another heuristic I tend to use is "for code stuff, try asking later, for business stuff, try asking earlier", since business stuff is often harder to figure out. We have code that describes the behavior of our product, that I can read and try to understand, but we don't have code that describes the behavior of the business.

Same dilemma when adding comments. We code in a relatively "plain" style, so "how" comments are mostly useless. "why" comments are more useful, but people tend to not really update them.

I think a way of solving partially that would be to be clear about the meta stuff. For example, I could ask if I'm asking too much or not enough questions, if my productivity seems okay. The senior engineers could also communicate about that. Even sharing how they do themselves stuff would be useful. For example "I tend to look at the code for X minutes before asking someone.". As a junior, I could multiply this time by 2, 3, 4 and do the same. Or ask the seniors about how they did things when they were new to the codebase.

To give a bit of context, we have more than 10 MLOC at work on a 30+ years old codebase, ~200 engineers and half of them are there since 3 years or less.

Yeah, knowing when to ask questions is itself a skill. I didn’t mean to insinuate in my comment anything negative towards junior engineers or imply I hate when they ask questions. But I’d say knowing how to read code and understand it on your own vs recognizing when something is almost unknowable without asking someone is a hallmark of a more mature engineer. But I stand by my insinuation that people aren’t deliberately hoarding knowledge, they just learned what they know effectively (via primary sources and doing) rather than through hearsay or lectures. Or they “know how to know”, they seemingly know everything because they know how to find answers to any question.

What you say about business logic is definitely true. In general the more something is an “emergent” property or itself only a high level concept (eg “Why do we use Mesos?” “How do we implement load shedding?”) the better it is to ask. The more implementation specific (“Why won’t this compile? What does this error mean?”) the worse it is. It’s not only about the time to find something out but also how it can be found out - whether it’s a web search or codebase search away, or not. The other problem is the senior people you ask may not always know the answer of the top of their head, and in answering your question will have to answer it themselves - typically you want to avoid that too.

For one I would always be more than happy to truthfully answer a mentee’s question regarding if they are asking too much.

> I didn’t mean to insinuate in my comment anything negative towards junior engineers or imply I hate when they ask questions.

That's not how I took it, sorry if this wasn't very clear but my point is that it's difficult on both sides, and it's interesting to share our struggles to see if we can find better ways to transmit knowledge.

> . In general the more something is an “emergent” property or itself only a high level concept (eg “Why do we use Mesos?” “How do we implement load shedding?”) the better it is to ask. The more implementation specific (“Why won’t this compile? What does this error mean?”) the worse it is. It’s not only about the time to find something out but also how it can be found out - whether it’s a web search or codebase search away, or not.

That's a great heuristic.

> The other problem is the senior people you ask may not always know the answer of the top of their head, and in answering your question will have to answer it themselves - typically you want to avoid that too.

I think that may be a good thing in some cases, as long as you don't demand/need the answer now and the question is good. If there's something that even senior people are not clear on, that's an opportunity to clarify things for everyone. I think that's what people mean when they talk about the value of new hires being able to see surprising stuff that feels normal to the people inside.

> For one I would always be more than happy to truthfully answer a mentee’s question regarding if they are asking too much.

That's great!

The issue is, it's faster to build a feature yourself than to coach a new hire to build it.

So if an org doesn't value mentoring and growing engineering talent, you'll end up with a few engineers doing most of the work and an org that magically can't scale.

Then you'll see the hiring side buying in on the cargo cult and trying to hire seniors with resumes matching their existing devs, then whining they can't hire them for cheap. At this point the business velocity has reached it's top speed.

I have the opposite problem: colleagues who don't want to learn how to do anything new.

I work alongside a principal engineer who refuses to learn anything beyond what they learned 5+ years ago. It seems they got to the Senior level and just gave up learning. Makes no sense to me and is a detriment to all the more junior engineers in our team.

>Unfortunately, I have to agree with blueslurpee here. I frequently join a team and find there are 1 or 2 devs who "know everything" and it is near impossible to pull the information out of their heads and into mine or someone else.

This is a classic defense mechanism for engineers trying to shore up their position.

There's a tug o' war going on between many companies and their employees wherein the companies are trying to make the employees expendable and the employees are trying to make themselves indispensable.

I found it mostly went away when I worked in companies where most people treated their current job as a stepping stone to the next rather than as their final destination but in companies in 2nd/3rd tier cities where everybody is married with kids and looking for stability....yea

The worst offender was a guy who was massively in debt, paid a shedload of child support, was not a great programmer and didn't interview well. He was sometimes openly hostile to my attempts to get information out of his head.

> I found it mostly went away when I worked in companies where most people treated their current job as a stepping stone to the next rather than as their final destination but in companies in 2nd/3rd tier cities where everybody is married with kids and looking for stability....yea

My experience of this, is that working with colleagues with families meant things were more stable and predictable.

In comparison, the people treating the job as a stepping stone developed things for their CV, rather than for the good of the organisation.

I can see how your experience can also be true however.

I dunno about 10x engineers, but there definitely tends to be a person who knows everything on many teams. I noticed over time that if I need to ask a random question about project A, I always ping the same 1-2 people from team A for most As... then I feel bad that I'm always bothering them, start asking someone else, get vague/incorrect/lazy answers, and go back to the same person anyway ;)

I don't think it is always a senior person though... it's often a junior engineer who's been there a while.

if your team relies a lot on senior engineers knowing things, there's a problem with your technical architecture. The knowledge should be look up-able, and tied in a clear way to the functionality.

I'm one of the folks with lots of knowledge atm, and there's a corollary that the moment you teach somebody enough that they know things, they leave the team. Putting that knowledge into databases, code, and config files is much more durable, and can have new things built off of it

Why do you feel a "10x engineer" is problematic in a team? You can have different types of players in team.

Any article that uses the terms "coders" and "IT" is a red flag to me. I've never seen that terminology inside organizations that are actually capable of shipping software.

> Then of course there are the stories of small engineering teams with Unicorn exits, e.g. Instagram, WhatsApp, etc.

I'm surprised the author didn't consider these as well. Some of these stories are almost a decade old at this point, they should be well known by now.

We didn't have APIs 10 years ago?

I agree with the article. As we build more complex products we need to be more specialized and work together.

Maybe not the case if one is working at a startup trying to cobble something together that looks like a working product by customers and investors.

We can build very complex products today so much more easily than 10 years ago.

There are services that will give you everything, from a managed DB to distributed CDN around the world, to one-command deploys... I feel like, if I only knew a business idea that didn't totally suck, I could create a whole new product that could scale to millions of users on a weekend.

Somehow it feels like a marketing leaflet for a CORBA/DCOM based product.

"I could create a whole new product that could scale to millions of users on a weekend." Sounds like an extremely appealing business idea.

It's not that we didn't have APIs, but it's the reality that they have penetrated every sector beyond early-to-medium adopters. Even regulated companies are moving to the cloud and using public web services at this point.

Ten years ago these companies would have been playing around with these things for sure, but not using them for their core business. This is creating demand for generalists within that huge section of the economy that isn't a top-tier pull for talent because the demands are different. These demands can't sustain frontend devs who could be working at Facebook. Eventually they will need specialized folks, but for now that continues to come from outside consultancies, just like it always has for this set of companies.

Yes, this is more accurate. 10 years ago most of the stuff was JSPs / JSFs and other UI+Backends that were fused by it's platform.

We had APIs since Sun RPC, they just keep being rebooted.

The 2021 version is REST/gRPC for SaaS products.

REST? Didn't you get the memo? We've all moved on to GraphQL because Facebook use it and that means we all should use it.

It also enables our UI components to easily fetch data themselves. Because that's a great architectural pattern that we've discovered.

It's unironically true that GraphQL in combination with database introspection & query tools like Hasura is a far superior pattern to REST-based CRUD.

Additionally, as a UI developer, I understand you are trying to make it seem like GraphQL promotes anti-patterns, but being able to make a single query which is deeply nested and therefore can maintain state for an entire sub-tree of your app is a really good state management pattern. Less coupling, separation of concerns, and usually significantly reduced network activity since you just need to keep the GraphQL state aligned, which is actually specified unlike how to sync a REST resource.

Do you actually have any experience with this tool?

Hasura is great. GraphQL on its own, unless you're offloading all of it to something well-maintained and capable like Hasura which will (hopefully) automagically save you from the time cost and risk of GraphQL, is almost always a bad idea.

"Using a protocol you don't understand without a tool that implements it is a bad idea"

Listen, I get it, we all want to hate on NIH and cutting-edge stuff, but what you've said is not an objection. GraphQL is a complicated protocol, but it offers a very clean interface between clients & servers, and when you put in a strong implementation you can generically access data that before you needed to specifically access.

I don't even know what else to say. There's a time cost and risk of every single thing you'll implement, you haven't given any insights into what tips GraphQL into a superbly expensive and risky area... Especially because Hasura is completely free and open-source...

> what tips GraphQL into a superbly expensive and risky area

The TL;DR is that implementing it without something that automagics away most of the work, does some clever stuff to save you from various pitfalls, and is very heavily battle-tested is approximately as bad an idea as exposing your SQL server directly to the web browser, except that you could do that for free. "Oh no I couldn't, I'd have to do so much work to make sure that access is safe—there's no way that would pay off, and I'd probably still manage to deploy DOS or data access vulnerabilities!" Yep, exactly.

It's also of dubious benefit vs. other options if you fully control both the front-end and back-end and aren't planning to allow lots of 3rd party users of your API—though even in that case Hasura, in particular, may still be a win despite GraphQL, not because of it.

Well, a tl;dr that says implementing a heavy protocol poorly is a bad idea is once again a non-statement.

> It's also of dubious benefit

Laughable. Comparing to the silly comparison of "direct SQL queries", both have lots of dangerous pitfalls, sure, but direct SQL queries give you no wins in-and-of-themself. For implementing GraphQL on the other hand, you get schema definitions, discoverability, strict typing across a network call, a large set of tooling you can plug in that "just works" because GraphQL is a protocol designed to tackle modern web issues, and more. GraphQL is not just "exposing your database directly", I think it shows a lack of experience with it that you think that way

My assertion, broadly, is that GraphQL is incidental to the good things that have grown up in its ecosystem, that GraphQL is per se not especially good or interesting, and that attempting to replace a traditional REST interface with it, without also bringing in a large stack of supporting tech that you didn't need before, is almost certainly a bad idea that will end in tears, because it is heavily dependent on those things to not-suck. Nonetheless, the hype has companies burning money on it, as is always the way with hype.

I would agree that it's really, really easy to improve on JSON, which is a curse on our industry.

And yes, I have experience with GraphQL. I think it has very little useful or cost-justifiable place unless you buy in to a heavy and very capable 3rd party system to manage at least the server-side portions for you, automagically. It's similar to buying into Rails. Absent that, yes, it's very much comparable to letting browsers supply SQL queries to your server, in that only with incredible effort could you avoid creating a vulnerability gold mine when implementing such a thing, because you're starting at a scope of "everything" and narrowing it down, rather than starting at a scope of "nothing" and building up from there.

[EDIT] and on the topic of vulnerabilities, given what it does, it'd be a sucker's bet that several popular implementations aren't just waiting to be a key player in "someone downloaded the entire database of 100 major sites that use this library/framework" hacks in the not-so-distant future. It is very much a "use with care, only if you need it, and with both eyes wide open" sort of thing. Again, I agree that Hasura's temptingly-handy, and that JSON and other non-statically-typed boundary-crossing data structures are practically always god-awful.

> [I assert that] attempting to replace a traditional REST interface with it, without also bringing in a large stack of supporting tech that you didn't need before, is almost certainly a bad idea that will end in tears

I literally brought in Hasura, and completely replaced my REST interfaces with it. There is no "large stack of supporting tech", I added a docker container to my small set of services (which already included PostgreSQL). Again, Hasura is free and open-source, and incredibly powerful while still easy-to-use. Setting up authentication is also a breeze.

Further, to imply that a query GENERATOR is in any way comparable to a query RUNNER is just foolish. Cut it out. You can't claim that it's a "sucker's bet" the whole ecosystem is just exposing unsecured databases, especially not without a single link supporting this having happened.

To conclude, I guess I find that I would strongly disagree with your approach to things and your reasoning behind it, and I find your claims like "JSON and other non-statically-typed boundary-crossing data structures are practically always god-awful" hilarious when I just told you I can treat GraphQL data as a statically typed boundary-crossing data structure thanks exactly to the GraphQL & implementation thereof you rail against.

Take care, good luck with whatever solutions you are using.

I'm not praising REST at all. REST as it was de-facto used was in my opinion a hot mess that was poorly understood and had developers focus on the wrong things (e.g. pretty urls).

I still don't think GraphQL as state management is a good idea for anything surpassing moderate complexity. And yes I've worked with both GraphQL and Apollo. I can see the merits for sure; for instance discoverability and being able to generate types from the API are great. I don't like overcommitting to Apollo though and using the overly sized client. Something is wrong when your state management is bigger in filesize than the SPA framework you're using.

> I still don't think GraphQL as state management is a good idea for anything surpassing moderate complexity

This statement is a bit hard to break down. What is "moderate complexity"? For example, I have implemented layers where GraphQL queries feed into mobx to manage some additional application state that never persists. I didn't even use apollo client for this, I just fired off a REST query.

Apollo client, on the other hand - I don't really know what you mean about "overly sized client", I am under the impression you can get it for 12-30kb, while React+DOM is 100-160kb (sorry not sure which numbers are gzipped here). I additionally think that it's a very light layer in terms of managing state, when I speak of "managing state via GraphQL" what I really mean is knowing when the client-side cache needs to update, and this is way easier than REST or for that matter anything else I've seen. In this regard, you can sync the server-side and client-side data easily and directly, and then use whatever state management solution you wish for your application.

Ok, besides it all, if you're just making fun of everything, that's fair, but it seems strange to mock something new / popular / powerful without any mention that it does things better than the old popular solutions

> For example, I have implemented layers where GraphQL queries feed into mobx to manage some additional application state that never persists.

Not sure what you want to say with this because it sounds like you used Mobx for state management in that case. Or do I misunderstand you?

> I additionally think that it's a very light layer in terms of managing state

According to bundlephobia:

React + DOM = ~42KB Gzipped

@apollo/client = 39KB Gzipped

For reference redux + redux thunk is like 2KB

Xstate is less than 20KB

> what I really mean is knowing when the client-side cache needs to update

Aha, and that sounds sensible. I agree. There are lots of other things in Apollo though that you apparently aren't using.

> > For example, I have implemented layers where GraphQL queries feed into mobx to manage some additional application state that never persists.

> Not sure what you want to say with this because it sounds like you used Mobx for state management in that case. Or do I misunderstand you?

Yes, I kind of botched it, but it's my point - GraphQL as state management can mean a lot of things. It can mean Apollo, client-side cache invalidation, use of subscriptions, etc.. I think that I found your statement of GraphQL breaking down above moderate complexity confusing because it was unclear to me what way you meant this in.

RE: apollo client, it's interesting to see that it's large, I agree that contrasted to XState or Mobx or especially Redux it is quite large, I would be happy to see that number come down, but I also am not so scared of 40KB I can serve from cache given I'm rendering "dataflow editors" of some complexity

Finally, I agree I barely use Apollo, I find its graceful property to primarily be cache management but greatly prefer mobx for true application state management because it is blazing fast / the most efficient rendering strategy

I haven't read enough about GraphQL (or even used it) to debate its relative merits, but I find it very interesting from "business architecture" point of view.

For instance, assume the future of IT looks like it does today, only ramped up x10. Imagine the majority of business value is driven disparate API's/Services managed by companies or organisations, with numerous and diverse clients that can "plug in" for their respective needs.

GraphQL essentially allows you to simplify the work that any client (internal or external) needs to do to tap into your interface, provided that you invest the appropriate resources on the backend. (which you would, if that were your core business proposition and a piece of your competetive advantage)

If companies continue to digitalize (is there any doubt) then a company's value will be encompassed and driven by the quality of its API, and importantly, it's relative use of connecting with many diverse clients. Therefore backend API's that support and interface to digital products the best will win.

I see GraphQL supporting this in that it trades simplicity and a uniform mono-interface on the front-end for complexity on the backend. I think a lot of firms will look at that trade and call it a great deal.

Ugh, no. The problems of just exposing your database really come out after you try living with it.

We wanted to make schema changes on the back-end, but the front-end queries were tied to it. Those queries were in apps run by different groups, with different goals and timetables, but we couldn't break them. If we had a defined API, we could be free to modify any back-end implementation details that weren't directly exposed. We couldn't because every detail was baked into consuming apps. What was our API, and where did it end? Now it was the entire database!

When we had to source data from elsewhere, or provide it via logical abstraction, we ended up piling hacks on top of the data structure's "API" from the back end that was now replicated in the front-end in order to "maintain it". Maintaining it really meant "providing an API that was the haunted ghost of our previous database schema "API" because now other apps/clients expected that "API".

Who’s exposing their database? Why? You’re exposing a view over your data that may or may not live in a database.

None of your points apply directly to grapple, all apply to a REST based architecture. More so, I’d say.

You're not exposing your database though (or at least you shouldn't be): you're exposing views. The set of views that you expose are your API. But it's not tied to your database schema than any kind of API (not less either though, if your database schema really doesn't match your API, your performances are likely to suffer sometimes, but that's true with any kind of API).

(I am in no way a GraphQL evangelist btw)

AWS makes it way to easy to connect a database directly to an API, both for API gateway and app sync.

It sounds like your main problem is that you cant get your callers prioritize making changes.

Using rest doesn mean you can stop supporting a call pattern because you feel like it. Somebody else still needs to do work before you can drop your translation layer

We don't need to imagine, that was the scenario being sold for CORBA/DCOM, then it was sold for SOAP/XML-RPC webservers,...

We even got stuff like Bizztalk and BPEL to orchestrate API calls with graphical workflows.

The circle of IT fashion.

Yeah you're right, silly me.

I was on a call just yesterday about how we should consider moving from one solution to another simply because the new one 'uses graphql which is so much easier and so much more powerful'

Some interesting & fruitful replies in here. There is definitely some nuance that deserves to be unpacked. Yes, a well-oiled team with competent leadership and an effective culture can do more than an individual alone.

However the premise of their statement is that it used to be more feasible for one person to "do it all", whereas now things have transitioned towards a more team-oriented environment.

I observe this trend patently in reverse.

i agree. the overhead of teams and external dependencies is often just too high--you're far better off if you can keep things small but mighty

It specifically says in what you quoted "or with a vendor or service provider." If you're able to get away with only hiring people who don't understand the intricate details of filesystem drivers, overlay networks, and hardware virtualization because you just consume your infrastructure from a cloud provider, you're included in "team sport," where your team includes the cloud provider who hires a whole bunch of people with expertise totally different from your company. WhatsApp didn't stand up its own networks or build its own phones. It stood on the shoulders of giants who knew how to do that.

Well said, I agree with you.

Found the engineer that needs a project manager. :)

I think most devs just haven’t had a good product manager. It can be shockingly effective, but only when they care about you, rather than managing you.

Going to echo this sentiment. The project I'm on now is the first time I've worked with a PM that gets whats going on and actually manages the product instead of just sending PMs after hours asking for some "small" feature to be squeezed into the sprint. It makes a HUGE difference, and I can honestly say I'm much less stressed out now than in previous projects.

interesting. i've cycled through some projects with and without PMs. I've not found 'good' PMs to make a huge difference compared to 'without', but definitely 'bad' PMs impact things negatively. It's hard to define what the 'bad' is, but micromanaging, forcing process where it doesn't need to be, arbitrarily enforcing rules when convenient, etc are all just negative impacts. It would be more beneficial to just not have that person at all. I've had some good PMs, and while I didn't notice much difference in my own performance/delivery/stress/etc, I think it gave some other folks somewhat more confidence/assurance.

As someone else eloquently put it, "With people management, the ceiling is low, but the floor is lava."

>stories of small engineering teams with Unicorn exits, e.g. Instagram, WhatsApp, etc.

Are there others? Seems to be a common theme with those 2.

Reply on news.ycombinator.com