"You don't want to write "raw" html/js/sql/Java" Uhm I thought that was my job.
It's not your job, any more than a firefighter's job is to be a hose operator, or a nurse's job is to be a syringe operator.
Your job is to produce software that meets the requirements of the business, as a member of a larger engineering organization which needs to be able to function well enough to deliver that software.
Why are frameworks popular? Because they provide a standardized way for multiple engineers, and multiple engineering teams, to collaborate effectively on the same product -- and they make it easy to hire people who already know the frameworks.
If everyone is writing "raw" JS/HTML/SQL/Java/whatever, they are actually building their own individual, unstandardized frameworks.
That's not good for supporting software when the author leaves, and it raises the barrier enormously for onboarding new people. Above a certain scale it's just not workable.
You can be cynical about this and say that it reduces engineers to interchangeable cogs in a machine. Or, you can understand why things are the way they are, understand how to succeed in this reality, and build a fulfilling career path for yourself by providing value that a mere cog cannot. I've worked with a lot of successful, happy engineers who are not cogs, and what separates them from the others is not coding skill -- it's that they all understand this reality.
I for one believe that the best solutions I've produced over the years were things that side-stepped the popular or established frameworks and were more customized and focused solutions. They gave the business an undeniable edge over the cookie cutter competition, and by being well versed in the full stack, my team was able to implement new features at a fraction of the cost.
Anyone who knows the underlying technology can pick up our work in not much more time than learning any off-the-shelf framework simply because our solution has fewer moving parts and the relationships between the business logic and the bedrock are direct and intuitive.
Yes but that's different from being a cowboy coder or a cynic who rejects the concept of frameworks in the first place. You built a new framework customized to your business needs. To do that, you needed to understand the context of the business and your tech stack, understand why existing frameworks didn't fit, and build a new one that works well. It's a good example of not being a cog, which is what I was encouraging in my comment.
Of course, "framework" might have different definitions for different people. To me, by definition, a framework tries to be more generic than a custom solution, has more features that appeal to more than one use-case, and is designed to be flexible and fit multiple projects.
The solutions I've had major success with were none of those things because the benefits we got (deeper knowledge of our stack, faster iterations, and features no one else could touch) way outweighed any perceived benefits of using something mostly off-the-shelf.
No one can judge their own code. We all think we've written great stuff that other people hate working with. Even your team can be bad at judging it, because they can reach out to you if they get stuck. Those are awful ways to judge how good code is.
In my experience the best measure of good, robust, well-written code is the opinion of the team maintaining it a year after you've left. If those people are singing it's praises and happily working on it rather than complaining about tech debt and saying they want to do a rebuild, then you can say you did a good job.
Custom is best sometimes. And that's ok. A well thought though and implemented 'bespoke' framework (literally just a system) has no value based disadvantage over a public, well known, framework / approach.
there's different criteria for judging. Business criteria and business success is one (good) criteria, but there's also the criteria of good craftsmanship (which may contribute to business success indirectly by making future maintenance easier). This criteria is hard for a non-technical business stakeholder to judge, but easy for the maintainers.
The 'common framework' approach works great if you've solving a common problem. That's why we use libraries for certain modules or functions of a system. When your business is, even just operationally, different in some way, bespoke is often the best way. Make it good and it doesn't matter a sniff if it's an established public framework or not.
There was a post recently on hackernews talking about how startups have an "innovation limit". That is, if you try to innovate too much, your scope grows beyond your reach. I think the idea applies here as well. Building everything yourself takes a lot of time. If you can position yourself to leverage the advancements of others, you can move much faster.
most business think themselves different, esp. in their IT infrastructure. It's closer to hubris imho.
If you have an internal/back office app, it doesn't require a custom UI framework to be created.
The only place where a custom framework should be created is if the domain of the software is custom.
Netflix: https://netflixtechblog.com/building-the-new-netflix-experie...
Facebook: https://blog.risingstack.com/the-history-of-react-js-on-a-ti...
Edit: find out what framework (although it didn’t work on some urls I tried): https://www.wappalyzer.com/
Some of the popular open source frameworks are derivatives of what these companies had already developed internally for themselves, e.g. React.
[1]: https://medium.com/dev-channel/a-netflix-web-performance-cas...
That is only "technically" true, in the way that if you develop any kind of sufficiently complex program then you will have, by necessity, built something which could be called a framework.
Realistically, though, your point is invalid and misses the mark.
To say that programmers shouldn't write pure code without an existing framework is akin to saying that a house builder shouldn't use bricks and lumber to build a house; they should buy a pre-fab kit instead. Which is, of course, nonsense unless you want the shittiest quality most generic house possible.
That's not what's been argued.
The overall thesis is that developers end up using higher-level abstractions, and between wasting time writing and debugging your custom high-level abstractions and simply adopting a feature-complete, stable, and standard high-level abstractions, you'd have a hard time trying to justify wasting time rolling your own when others already did all that work.
That's only true when all the abstractions you need have already been implemented by others. Most of the time though, in addition to off-the-shelf abstractions, you need to build new abstractions which are specific to the business problem at hand; and shoehorning those abstractions on top of an existing framework which was not created with them in mind may be more work and may provide a less robust, way more complex solution.
Not really. More often than not, any framework already ships with far more features than the ones you'll be using, with the added benefit of already being production-ready and extensively tested not only by the maintainers but also by everyone else already using it.
When you roll your own... Well, good luck.
> Most of the time though, in addition to off-the-shelf abstractions, you need to build new abstractions which are specific to the business problem at hand (...)
You're confusing implementing higher-level abstractions which allow the low-level details to be ignored with actually implementing your own business logic.
The grandparent definitely isn't, and that's a bit of an uncharitable interpretation. I'm sure there must be a tiny minority of developers out there that doesn't ever have to write higher-level abstractions, but in the real world, writing both "business logic" and higher-level abstractions is the norm for most jobs. Only toy projects and the most menial development jobs escape that.
Even frontend, which someone erroneously claimed above that is "99% isn't custom" really is. The two major libraries, React and Vue, and the rising one, Svelte, really aren't frameworks at all, and require in practice not only a lot of custom stuff but also higher-level abstractions. Again, the exception is toy projects and menial dev jobs.
Just because 10% of your codebase is uniquely unsuitable to the 'standard' introduced by the framework doesn't mean that you can't use it for the remaining 90%. And even for that 10%, it's probably not so alien that some of the functionality provided by the framework is still relevant.
What they call "building your own framework" is simply what I think of as programming. You design a program, it will have a structure. Why is that "forbidden" and something that programmers are thought of as unable to do?
It may make sense if custom framework is a magintude order better, but if it is the same as a standard framework or worse then it is not worth to spend time on it.
If you want an analogy, this is like a builder claiming that he is going to make all bricks and nails himself. You don't usually write your own standard library, why would you write your own framework?
And software engineers already have their toolset. The real analogy would be telling all the builders that they now have to use those shitty multi tools that are 10 screwdrivers and a hammer all in one.
If you feel your job is to find neat ways of writing Java or C# or wtfever, you are limiting yourself. You should be looking to offload any repetitive/tedious task to automation, and from there figuring out how to leverage ML to make it efficient.
The "work" you speak of is going to be obsoleted in the next decade, if it isn't cutting edge CS theory it should be done by the machine.
The point of using frameworks is that you can be sure that many of the bad choices the people who wrote it made early on have by now been tidied up. You don't have to worry that Dougie the intern's custom HTTP router is actually all sorts of ballsed when you go to set up your first DELETE endpoint, because Dougie didn't get to touch anything that fundamental to the stack.
I do get off the boat when it comes to leftpad and stuff like that, don't ever pull external dependencies for trivial things. But an ORM or a UI framework? Those problems have been solved hundreds of times, absolutely no need for anyone to solve them again unless you're that good that thousands will move to using your solution instead of the existing ones.
The framework isn't bricks and nails. It's a prefab house plan. You follow the instructions and you get a house out on the other end. If you need to go outside what the prefab covers, you have to fight the framework because now you are drawing over top of an already printed out structural plan to try and do something different.
The equivalent of bricks and nails in SWE is using libraries. Use a library to do a thing. You still need to design the program (drawing the floor plan), formalize how the parts interact (drawing the structural plans), and actually put the pieces together (building the actual house) but you don't need to worry about the small details like how to perform X algorithm (make the bricks), write a messaging standard (make the nails), or handle the intricacies of concurrency, distributed data structures, or drawing to a user interface (making the concrete and mortar). Your libraries do that for you. All you need to do as a developer is focus on writing neat functional kernels that do "the hard stuff", wrapping all your "hard stuff" and libraries together in some OO class to implement a certain functionality, and then stringing the bits together with some imperative glue to tell all the pieces to work together. Past that is orchestration & administration and that's Ops' responsibility.
Unless all you need is cookie cutter, don't use a cookie cutter. Fighting the cookie cutter will be far harder than using the tools for the job(cleanly separated, well documented, and interoperable libraries) to solve specific problems.
What they overlook is the time and energy wasted in the framework cottage industry: making dependencies talk to one another, aligning version numbers, major version upgrades being forced on them (or risk being insecure) and the time spent searching Google, SO, and GitHub issues for support.
As a bonus, I know that every dev I hire with experience with that framework is also going to know how these things are set up, and they can focus their onboarding time on what actually needs to be unique to the problem we're solving.
Regarding the downsides:
- a framework doesn't necessitate additional libraries any more than rolling your own does (arguably this is worse without a framework, because unless you don't use libraries altogether you have a hodgepodge of libraries to solve common problems that don't have a lot of incentive to be cross-compatible, and a framework really can't be incompatible with itself)
- Any half-way popular framework is going to ship security updates for past major versions (and again, if you're using libraries this is more of a headache than updating a single framework)
I think this is the heart of the issue. Of course everyone thinks their problem is a special snowflake, otherwise they wouldn't invest resources into solving it. The job of us developers is to distill the essence of it and generalize the rest. But it's something I've always struggled to communicate with the business side. Either I "just don't get it", or they take generalization to mean I want to drown them in configuration.
Yeah it just doesn't work that way, you need to understand the domain and understand the system, and it takes a while to learn in a new job. This is just a fact, for every job, also software engineering. Stop this weird fantasy of replaceable cogs on the assembly line, you are not henry ford.
Documentation or (preferably and) built in language convention are the winners here as frameworks come and go.
I’ve had to come in before and deal with someone’s custom ORM or logging framework.
I’ve made the mistake myself of writing something that was a cross cutting concern and then later found out that there was a popular third party package. I replaced my implementation as soon as feasible.
Why are you assuming that everything a software engineer creates will always be bad? That's an unbelievably deprecating attitude. Do you ever see any other professions do this? UX? Product? Writers? "everything we always try to do will always be bad" and we have to try to safe us from ourselves? What?
And this "framework" would be custom made to solve exactly the use cases at hand, and nothing else by definition would be better at those specific use cases.
And why would you optimise so much for onboarding and hiring anyway, at the expense of so much continuous wasted time and less output?
When my product owner quit, the new guy was working in parallel for three months, because "of course, they have to learn the ropes". That's normal for every single job basically, except software engineering.
Why not just spend a couple of days, or a week, to give an introduction to your new hires, so that they understand and learn how you have structured your program?
The only "problem" with that, is that you have to stop acting out this farce of the computer genius with the superior intelligence, in your hiring and onboarding process.
Not necessarily, but it usually ends up badly. I have no doubts most of my peers could write a fine custom framework, but it's going to take a whole team of them at least a year to come up with something decently ergonomic, tested and documented. How often is the business side of the company going to be on board with spending that amount of resources on a solved problem?
Instead developers decide to fly under the radar and do it anyway, but then run out of steam while the framework is still an untested, undocumented prototype and the pressure to shift focus to features becomes too great to resist. And by that they have doomed the whole company to a bottomless swamp of technical debt, with seniors getting fed up and leaving, and being replaced with juniors with no clue of what's going on, making the onboarding problem magnitudes greater than it was in the first place. And if someone has the genius idea of a big rewrite, we all know where that's going, especially since the only people they have been able to retain the whole time are either inexperienced juniors and mediocre seniors, so the cycle repeats again.
No, because they would not make a framework, it would not require all of this work, because it's not designed to be generic and reused by others.
Simply write a program. Not a framework. Has this idea been completely lost? Solve your own use cases only. Don't try to generalise it and optimise it for reuse by someone else.
Would you write your own operating system or programming language because it might be better suited to solve your particular use case or use a general purpose language?
Of course, building your own home is possible but that could be a shack made up of sticks and mud or a stack of bricks and mortar with no base/foundation which can fall apart at any moment.
This is exactly the problem, you are trying to use a standardised blurprint, to build a custom architectured house. Where you should probably make a choice here, either go standard or custom.
> Of course, building your own home is possible but that could be a shack made up of sticks and mud or a stack of bricks and mortar with no base/foundation which can fall apart at any moment.
Why would a highly skilled builder build a shack of sticks and mud, or a house without foundation? Why do we assume that every software developer is an incompetent clown?
Because they make it very quick to get a 70% working system that you can sell to people (internally too) before the people with the capacity to get a 100% working system gets their first version.
So, no, they do not make sense for making prototypes. They do make work when you want a 70% solved problem and won't care about never reaching 100%. On the other cases, you still need to use them, because if somebody gets those 70% first, you will have problems, but they make no sense at all.
In “raw" html/js/sql/Java.
You add value when you optimize, debug, and refactor the business operation. Increasingly in old-economy companies, but especially in Silicon Valley companies, the business operation is largely defined in code. So this is something software engineers can do, through well-chosen code changes.
Outside of startups, engineers rarely do this alone. There are executives, business analysts, product managers, data scientists, etc. with very similar tasking. But you want to get into a position where these people see you as a full intellectual partner, not as an implementation resource to type in their already-baked ideas.
"General-Purpose Tool-Building Factory Factory Factory"?
"Why I Hate Frameworks": https://www.gwern.net/docs/cs/2005-09-30-smith-whyihateframe...
It depends.
If you have a nail, and you are using a jackhammer of a framework to pound it in, every developer that comes after you has to learn how to use that jackhammer unless they have previous experience with it. And honestly, most complex frameworks have so many different potential usage patterns that future developers will still have a heck of a learning curve. (I've never seen two uses of redux that remotely resembled each other! After using redux for 3 years I can easily find redux code that I have literally next to no idea what it is doing!)
If you use a hammer, even if it is a bespoke hammer, future developers just have to learn how to pick up that hammer.
Obviously at some point it pays to switch to using an Industry Standard(tm) solution, but for many problems plain old JS or HTML may very well be appropriate.
For example, I was making ad campaign landing pages for my now defunct startup. To ensure speed I wanted them to all be static HTML. I could have used a static site generator of some sort, configured templates and all that jazz, or, instead, I could just write HTML + CSS for 3 landing pages. (Actually I only had to write the CSS once).
Now if I needed to programmatically generate dozens of landing pages, then sure, adopting some static site generating framework may have been worthwhile. But in my case it wasn't. Raw HTML was the correct solution.
there's no mantra that should be followed without thought in software engineering. Frameworks helps in a lot of cases, and should be used - but this decision require someone with experience to make, and this experience has to have been hard earned from previous failures or projects.
Feels like people shoehorn Gatsby too much into those kinds of situations. Bless you for keeping the dream alive - every time I see well written and small HTML/CSS/JS code on a landing page I want to scream.
Despite knowing React and being familiar enough with Vue, Angular etc. I still get imposter syndrome when I see a site's markup and see weird artefacting that comes from some framework that I have never heard of.
That's an extremely open question, and I wouldn't agree that frameworks, in any way, "provide a way for people to collaborate" any more than a whip does. They seem to me to limit the ability to write the code you want for no value other than providing confirmation bias.
The idea that the same thing being outputted on the screen will have two implementations by two uncommunicative engineers that vary in such a way that they become impractical, impossible or even at all harder to debug, is simply a myth. I highlighted the word "engineer" because that's the key point - if you are hired to solve a problem, you are probably going to solve it the easiest and most convenient way. You are trusted to write code, not use an API. I find it hard to buy the idea that 20 year old aspx code is any easier to debug than pure JavaScript written around that time.
As far as why frameworks have become the norm, I would personally put it down to, in no particular order, the following ideas:
- Some are genuinely useful - functionality is limited in the parent environment's API and must be implemented in some way. This is rare, in my (mild) experience.
- The company had learned the lesson that contractors (especially ones that win the bidding wars) write horrific and unmanageable code. Therefore, it decided that a cost-free way to alleviate this issue is to force all engineers, permanent or on-contract, to use a specific framework.
- The above point spread between companies as people (directors of engineering) shifted around, word spread that someone was able to make significant cost savings etc.
- They are in place to allow one to follow idioms better. I recently interviewed for a company that followed, quite closely, the clean architecture book by Martin Fowler, with their own internal framework. They had a good plan in mind - they wanted components to be interchangeable, usually at compile time, and needed to ensure that they followed procedure in order to write code that was manageable at scale in order to achieve this. They made their own framework and have used it for 10+ years. This proper, elegant use of a framework is very rare.
I do not mean to say that frameworks are useless - however, JQuery has always been absolutely worthless to me in my personal endeavours - which often included quite bespoke DOM manipulations - the exact place that JQuery is meant to shine. To me, it is a great example of something that was phased in as a Band-Aid solution to people writing terrible code as they had no ownership of the product.
I will admit that I use little helpers, such as
function $(usuallyASwearWord){
// Yes I really use element by id for this when writing my own fun code. Not query selector.
return document.getElementById(usuallyASwearWord);
}
When using a framework, you get most of the benefits right away, and most of the costs happen much later, when abandoning the framework is costly.
Besides pure technical quality, you have to consider more murky issues: who wrote the framework? how long will they be around for? Does their problem match your problem? Will your interests remain aligned in the future? Is there a succession plan? Will new hires be familiar with it? What's the learning curve? Does it integrate with the other N tools you are using?
Ideally, the decision to use the framework or not should be made by someone who has used it before. This is often not the case.
My personal preference is to always use simplest tools that get the job done.
I wish that hiring technical writers to write docs was more common. Most of the new libraries I have to use have documentation that is so poor, I have to use stack overflow + write test cases.
Currently, I am in the middle of taking over a go code hairball from someone who left, and who loved external dependencies. Out of 31 direct dependencies, 11 are not at release 1.0 yet. Many have poor documentation. I believe that when I am done with it, our code will have fewer dependencies, and be half the size.
> it raises the barrier enormously for onboarding new people.
is.. a little ironic. I've never met any of the gazillion frameworks out there that have made it easier to get people up to speed, despite promises to that effect.
Hmm, i'd say that there are frameworks that actually make things easier and better, and some that are much like the incomprehensible Rube Goldberg machines that the previous poster complained about.
For an example of this, you needn't look any further than the Spring framework in Java, perhaps the ultimate example of enterprise bloat of poorly understandable (unless you spend all your time digging into the internals) and overly abstracted non-solutions for problems, and that's even before you pull in 50-100 dependencies for making your dated monolith do whatever the business wants.
There is an absurd amount of XML configuration needed, except that sometimes you need to mess around with annotations in your source code as well, except for those cases when instead you'll want to just overwrite some class with your custom logic inside of the methods, which will be initialized by some Eldritch abomination of class loading and reflection, or maybe you'll need to change a bunch of properties in a config file, with all of this leading to long and arduous error messages that will make you waste hours digging through Stack Overflow.
I was in charge of migrating an older Spring app to Spring Boot, which attempts to alleviate some of these issues (and there was also the fact that the particular Spring version was EOL at that point) and after wasting more time than i'd like to admit of an error after an error after an error, it was decided to just update Spring itself as far as possible and let it chug along in a barely alive state.
The framework wasn't a way of managing risks or "getting things done", it ended up being a risk in of itself and was a massive pain once the system needed to be supported, maintained and also have new functionality developed for 5-10 years or so. That's not to say that there aren't better frameworks out there, but there definitely are some that will be more trouble than they're worth, unless you want to have a person or two full-time just to wrangle them in order and continuously maintain and update them, which for many of the smaller orgs out there is unacceptable.
> If everyone is writing "raw" JS/HTML/SQL/Java/whatever, they are actually building their own individual, unstandardized frameworks.
I've also seem bespoke frameworks be used and those were generally worse - while there was the occasional good idea in them and they could be decent to work with in a limited set of circumstances, getting help when something goes wrong with them was impossible and you were stuck reading the source and mucking about in it. Why is that bad? Because with the larger and more popular frameworks out there you can instead look up solutions for problems or integrations that someone else has written.
I'll admit that pulling in dependencies carries certain risks in of itself, but you'll very quickly realize how nice it is to let someone else step on rakes before and learn from their problems and mistakes, instead of attempt to translate code comments from Lithuanian to your own language when prod is running into issues, because the original devs are long gone and there is no documentation, nor examples of how to use anything in place.
Thus, it is better to use frameworks that have tens of thousands of collective man hours put into them, or even more, as long as they don't suck.
So, in the end, it's probably better to use frameworks that keep you closer to the code that is actually running, like Spring Boot (where at least parts of the config can be done in plain Java), or even look at ones like .NET (the newer ones are a bit like Spring Boot), rather than DSL hell.
You can put a breakpoint in your Java code and see why something isn't initializing properly. You cannot put a breakpoint into your .properties, .xml or .yaml file (or even feasibly find where to put into the over-abstracted configuration read mechanism, or find how it connects to the bit that is actually initializing your data).
Finally started another project. No frameworks this time. Started with html and single js file until I needed more.. and solved the problems step by step. I'm at the point that I now see the benefits of react, but only by doing it this way could I possibly learn why.
That's why I feel bad for the new generation of developers. When I started developing in 2000s it was enough to know low level abstractions, some design patterns, etc. Along the way I've seen the context in which for eg. MVC was revolutionary, and all the latest frameworks actually solved a lot of real problems (SQL query in PHP garbage all over the place on backend, jQuery/MooTools/ExtJS/Coffescript/AngularJS, supporting IE6, etc.)
A lot of modern tools make sense to me because I know the context they came out of. I feel we've done a shit job of preserving this context and lesions that lead to current frameworks - this will eventually lead people to rediscover all the mistakes that were hit in past, but also saddle us with legacy burden because it's hard to distinguish what's relevant and what's a result of historic constraints.
The reason I like batteries included frameworks is I get all that for free and ideally it's already following best practices so I don't have to worry about security, about account recovery, about versioning new data/features, about scaling, etc
with the exception of security (i take it you mean authentication), everything else is unlikely to be provided by a framework like spring directly - you'd have to choose something to provide it or write it yourself.
You jump from Rails project to Rails project and they all work the same way. You know where everything is immediately. It's fantastic for teams and long term maintenance between different devs because the learning curve of each project is virtually non-existent.
But Rails has gotten bigger and more complicated over the years, which makes it tougher to make the case.
Except lots of projects today use very diverse structures and the meat of the code is often on complex libraries like ActiveInteractors, Trailblazer Operations, stuff from the Dry-RB ecosystem, etc, or sometimes (hopefully) simple Service Pattern classes, if you're lucky. Sometimes all of them at the same time. For lots of companies, Rails projects in the real world has ceased being easy to grasp for a while.
Certainly set the standard for a long time though.
1. There is software engineering, and then there’s working at the digital factory.
2. A good way to set yourself on the path that leads to the former is to get intimately familiar with the HTTP protocol. A good way to start is to reject fancy frameworks, and to get comfortable reading RFCs.
Being the guy who really knows HTTP is a powerful career move. Becoming that person almost inevitably kicks off a virtuous cycle that steers bright/motivated devs well clear of writing REST endpoints for the rest of their working lives.
Being intimately familiar with HTTP is fine, reading a lot of RFCs is fine, but there are plenty of well respected, well compensated, influential engineers who do neither, and have no need for either one.
I concede that this may superficially resemble the aberration you describe, but insist that it’s actually very different.
The HTTP comment seems off-base. HTTP is neither particularly important or interesting as a protocol. You can spend a half-hour with netcat and have a few conversations with your web server, and you're set.
The concept more broadly is spot-on. The "I don't need to know [x] to do my job" crowd is universally mediocre at their job.
For a programmer, the set of things one layer down is important, be that cache/memory hierarchies, low-level programming, operating systems, databases, compilers/interpreters, or otherwise. So is one layer up, which is usually business and product understanding.
I'll also mention: Mediocrity at one's job is okay. There are plenty of wonderful people who have work-life balance and do interesting things outside of their job, and just want a pay check. There are people happy to be working on systems I'd be miserable working on -- payroll, banking, inventory, etc. -- and I'm glad they're around. A few of them are among my favorite people too.
My problem is I find the knowing the lower level stuf way way more interesting. Heck I'm doing a udemy course on network programming which arguably won't help my career, I just have a secret shame about not really knowing networking. (Plus it's really interesting)
And comment about netcat happens to be exactly what I’m condoning.
As for mediocrity, sure. But it should be voluntary, not imposed by a lack of options.
HTTP has no interesting theoretical basis, and doesn't really generalize to much. Like most early internet protocols, it's kind of random and ad-hoc, with incremental hacks and improvements like keep-alive added later. You can pick up any RFC -- POP, IMAP, or whatever, and get just as much out. That's not nothing; they're worth knowing, but here's a short list of protocols and data standards which which have a theoretical basis which will teach you something deeper:
- git. Brilliant data standard.
- SQL. And relational database theory.
- SGML. Yes, the original. Much juicier than XML in design.
- Cryptographic protocols. Pick up a copy of Applied Cryptography, and read the chapter.
- A survey of the early RPC and object embedding protocols (OpenDoc, OLE, etc.)
- TCP/IP
- Any of the various compression and/or error correction protocols.
- Printer standards, especially PostScript
- Video, audio, or image compression standards.
.. and so on.
Some of those require months of study.
Overall I agree with your gate-keeping comment - that's how people interview/hire. I disagree that it "has no place" - if I'm hiring a mid-to-senior engineer that's web-capable I am 100% gate-keeping to make sure they have an intimate understanding of HTTP + are capable/comfortable reading RFCs.
It's like a series of tubes. That's why SWE work involves so much plumbing and getting deep into all sorts of $#!+.
Also, React is not slow at displaying tables. If it’s slow, you can use plugins to hide cells which are not in the viewport.
(Old farts like me manage a billion records and display tables of 7k rows x 100 columns in a webpage in mere milliseconds, a joy that React developers will never understand. Why do you need more than 10 lines if you have pagination, they say).
Are you doing something fancy with HTTP?
They don't have to read RFCs in bed before sleep, but they should at least know the gist of what they contain.
Interview 0:
"I expect them to know the basics of HTTP and to be capable of referencing/digesting an RFC"
Interview 1:
"All devs should be up to speed on the mysterious world of character sets, encodings, Unicode, all that stuff"
Interview 2:
"Any real programmer can reverse a string in C without a buffer"
Interview 3:
"I expect anyone working with k8s to understand the cgroup fundamentals."
Interview 4:
"No really. What is html5?"
Real programmers know how to do both even if the don’t do them exclusively.
Maybe it's because I live outside the United States, but it seems like it's much harder to build a "pathway" into work that demands that intensive, deep understanding of engineering on computers. At least one that'll pan out in my 9-5.
The supply of problems that require deep knowledge vs framework Kung Fu... my impression from ground level is they're tilting more towards GitHub martial arts every year.
The supply of companies outside the United States with a deep interest in these topics is sparse, to say the least.
In my role as a software architect in a small company I've been booting out various "frameworks" and overly invasive libraries, because it turned out they are not delivering much, while requiring attention and forcing us to bend our code to satisfy their demands.
First ones to go were various cloud libraries: AWS, Google Cloud, CloudStack. It's a plain REST. I estimate that the code that talks to Google Cloud directly is shorter than it was when the library was involved.
Docker went (it's a REST with a few quirks). Kubernetes client went (same). GitHub client went.
gPRC went (fortunately we're not required to talk gRPC anywhere except a few endpoints, and it turned out that Protobuf is not hard to implement if you don't have tons of schemas, and gRPC is HTTP/2+Protobuf).
Finally I booted Mongo client library, because all we needed is a few requests, and implementation of Mongo wire protocol + BSON ended up costing us ~300 lines of code.
Remaining libraries earn their keep: there is cryptography, XPath, JSONPath, compressors, userland IP stack, complex serialization formats. I thank the authors of these libraries, and due to some reason these libraries are also the least complicated and most compatible version-over-version.
What's not to like about the SimpleBeanFactoryAwareAspectInstanceFactory [1]?
[1] yes, this exists
However, a lot of people have an interesting view of things like frameworks, ORMs, and other abstractions. To some, it feels like pure all-or-nothing; but what if it didn’t have to be?
Even the biggest hater of SQL database abstractions will admit there are reasons why someone would want them. But I wonder if there is an answer somewhere in the middle that could go both ways. For example, sqlc[1] provides basically all of the type safety that overbearing database frameworks might, but it lets you write plain old SQL just as you normally would, and integrates cleanly with your existing migrations solution. It doesn’t solve everything, but I have a feeling more people would like tradeoffs like these: an enormous amount of complexity does exist, to say, parse SQL and emulate DDL transactions for the sake of typing, but it stays entirely in the tool itself, and the resulting code and interfaces are completely simple and obvious code.
I believe that there are multiple better middle grounds waiting to be discovered in any given niche where the options currently feel like barely-worth-it trade-offs.
[1]: https://sqlc.dev
There is only one reason, and it's that people don't understand how to use SQL, and that's because it's not taught in computer science degrees, and this also has strong ties to the battle between sun and oracle.
And this makes database developers expensive and hard to hire.
There is no reason to use an ORM if you know SQL and database modelling. The relational model is by far the most powerful tool to solve CRUD use cases, which "web development" is, and that's just simply a fact.
What the ORM will do for you is automate some boilerplate mapper code, which might be a bit boring to write, but is very simple. And the downside is that it will make all of your database modelling, querying, access etc orders of magnitude more obscure and complex compared to just using SQL. And now you will waste enormous amounts of time fighting with the framework to debug the ORM and find out which kind of weird SQL statements it generates, where you would have complete control over this if you just wrote your own.
The obvious solution that would benefit everyone, is to simply train your developers to use SQL, if they don't already know it, and let them take a course if they need to. Not to slap on some stupid framework and make a huge unmanageable mess of the whole system.
Unfortunately, I immediately think you have misunderstood. While ORMs and other database abstractions may be a boon to folks who don’t know SQL, or at least some may think that, it is not the only reason that people use them. It may not even be one of the bigger reasons.
Type safety is first and foremost a great example. Wouldn’t it be great if you could literally not ship SQL code that has typos, syntax errors, or other query errors? Unit tests catch those, but you have to actually write them, and they still can’t help if you hold things wrong. (Like using sprintf for SQL queries, god help us. Yes, that is a mistake no professional should make. But if you add enough layers of complexity, you might just fool someone who knows better into doing basically that with extra steps. There are some ways to combat this of course, like PEP 675[1].) In some environments you might accomplish this with compile-time linting or advanced metaprogramming. However, ORMs also do this right out of the box generally, at least in languages where it can be done.
But even then, that doesn’t scratch the surface of what’s possible. Mind you, with sqlc, we’re still talking about making raw SQL queries. But those SQL queries contain tons of information that gets duplicated by the programmer. For example, it is possible to statically know what types of inputs and output columns any given query will have based on just the schema and the query itself. sqlc gives you this information. It generates correct wrappers for SQL code using only raw SQL queries as input.
This is not the only benefit of abstractions of course. And sqlc doesn’t give you all of the benefits of an ORM. However, that’s exactly what I’m trying to say: a lot of people have an idea in their head about what an abstraction must entail, but there’s probably better tradeoffs that are genuinely not susceptible to the same issues that you’re imagining. Case in point, I have a feeling you interpreted my post to only be regarding ORMs, but even the text you quoted is more general—I said “database abstractions” for a reason :)
How would you be able to validate data that you can map to your tables without hitting the db?
If you wanted to add a new row of data to one of your tables, you would have to hit the db, get row columns and make sure that data fits.
Or you could write out your table structure separately outside of the db, and validate against that, but that would just leave you with two sources of truth which is an antipattern.
Or you could just use an ORM.
I'm fluent in SQL, and understand the solutions that ORMs solve - it's not just a pure abstraction over sql queries, but the tooling and utility provided wrapping around that is what makes it so powerful.
In the case of C# Entity Framework, the ORM abstraction- LINQ - is a first class language feature with full compile time type checking and auto complete.
The same LINQ query can work across databases and in memory collections including RDMS’s and Mongo. Eventually in any object oriented language, you’re going to end up doing “something” to convert relational data to objects.
This is only true for the most trivial of use cases that will not really bring any value to your users. As soon as you get more advanced feature requests, you need the power of the relational model, which is now abstracted away behind a much less powerful interface.
Edit: more advanced, or even just a higher number of orthogonal use cases against the same data.
I think the whole framework situation in general is much better in microsoft land because it's one company that designs the whole ecosystem, and overall experience, and also the people with probably the most experience in software development. I even think VB6 is superior to the standard open source web development stack used today. The only part where MS seems to be behind is version control.
How can MS be behind in version control? TFS has had git support for ages and MS owns GitHub.
Moreover, many ORMs give you the tooling to validate data before hitting the db, which is also another huge plus.
I mainly work with Python on the backend, and I've recently switched from SQLAlchemy + marshmallow + alembic to SQLModel and both have been amazing to work with.
I would never dream of setting up tables through plain raw sql ever again.
If it is the case that I'm moving onto a different language, as Go may be it, I'm willing to learn the ORM, granted that it deals with migrations and validations out of the box.
In general, it seems like the easier it is to do something in the scope of the framework, the harder it is to bend the framework to do something outside it's scope.
We've all seen this with frameworks that make 95% of the work really easy, and the remaining 5% is like pulling teeth. You're bending the thing where it's not meant to be bent.
(And that's when new frameworks are made. :) )
But if you go lower, under the framework, it's way more flexible. As a dev, you can explore and create instead of fighting. Which is great.
But really we're just on a different turtle. Raw JS has a really hard time with some problems when you try to bend it wrong.
Our job is to take the right technologies for the job and make them work, no matter how abstracted they are. The pain is merely from the wrong tool. And, truly, it sucks when you have to do that.
All that said, every other framework I've ever used in my personal projects has fine documentation and community support, including their peripheral deployment tooling. My experience at work seems like an outlier to other tools I see. Comments like the original post and OP make me think everyone is working in legacy systems based on aged tech from the cowboy era.
Every dependency is a liability, especially the kind you can't remove without rewriting your entire project. Adding a library to my project is a very serious thing for me.
I mean it isn't your job and it shouldn't be. Your job is to make software handle with consistency some business process or become an extended memory of some business people.