The primary issues that make frameworks flawed | Hacker News Reader

The primary issues that make frameworks flawed

timperrett.com

59 pointsEwanToo42 comments

This: "Concepts such as Functor should not be scary. Many engineers in our industry suffer from a kind of dualism where theory and practice are somehow separate, and formal concepts like Functor, Monad and Applicative (to name a few) are often considered to "not work in practice", and users of such academic vernacular are accused of being ivory tower elitists. Another possible explanation might be that engineers (regardless of their training: formal or otherwise) are simply unaware of the amazing things that have been discovered in computer science to date, and proceed to poorly reinvent the wheel many times over. In fact, I would wager that nearly every hard problem the majority of engineers will encounter in the field has had its generalised case be the subject of at least one study or paper... the tools we need already exist; its our job as good computer scientists to research our own field, and edify ourselves on historical discoveries and make best advantage of the work done by those who went before us."

Now you know why programming is not considered engineering. There are plenty of programmers who pride themselves in not caring about "theory" ("I didn't go to school and now I'm making money building cool stuff!!") and thus history is repeated.

You don't need much CS theory in order to develop and maintain a CRUD app. I've been a programmer who gets paid to write code for 11 years now, and I'd say most of the code I've seen wasn't that much complicated than a CRUD app.

Also, I don't feel the need to be called an engineer. I just write code, hoping that it would work most of the time, under as many diverse circumstances as possible and trying to meet the client's needs. It sounds trivial, there's no "theory" in it, but sometimes is hard as hell to do all these 3 things right.

Some insights inspired from theory help a lot with writing CRUD apps. (Also of course, inspired from outside and then explored further by `theoreticians'.)

The most successful of these are now invisible. Either because they are available libraries or so ingrained in the background of what we are doing.

See for example regular expressions, parsers, garbage collection, databases, file systems, CPUs.

For CRUD apps, "Out of the tarpit" (http://shaffner.us/cs/papers/tarpit.pdf) is an interesting paper that I hope describes how the mainstream programmer will build CRUD apps in perhaps ten years time. (I already used a variant of their techniques in a previous job, and it was rather pleasant.)

But yes, in practice knowledge of theory often acts more as a gatekeeper for getting into the likes of Google than you will use it. But when you do spot some opportunities to replace endless workarounds and tedious manual monkey-work with an elegant overarching principle, it often makes all the hours spent studying worth it.

As an great practical example: shake (http://shakebuild.com/) is a build-system that was born out of our frustrations with Gnu Make at Standard Chartered Bank.

Similarly, Bloomberg used to use C++ to describe financial derivative contracts. Those derivative contracts don't fit well into class hierarchies. Switching to an embedded DSL approach embedded in a functional language made the software easier to write and more robust to common errors especially when maintaining.

> See for example regular expressions, parsers, garbage collection, databases, file systems, CPUs.

Of course we're all standing on the shoulders of giants. So does the milk-man, who goes door to door using a vehicle of which he doesn't know much about how it was built/what its underlying principles are (is it a Otto-based engine? or a Diesel one? does the milk-man need to know the laws of TD?), but that vehicle is paramount to the milk-man's financial success, so to speak.

All I'm saying is that we don't all need to be like Rudolph Diesel, Nikolaus Otto or James Clerk Maxwell, we can still do our jobs as programmers perfectly fine the same as a regular milk-man does, we don't need to know the theory behind CPUs, garbage collection and the like in order to write code that does stuff.

Don't get me wrong, I get what you're saying and understand your points, I just think that right at this moment the programming world needs more "milk-man-like programmers", because if we build walls around our profession ("you can only be a programmer if you know CS theory" and the like) then nothing good would come out of it. I mean, we will certainly build some cool, nice programs, but the world right now doesn't needs just some, few "cool, nice programs", the world needs lots and lots of code. In order to have lots and lots of code we need more programmers.

It is precisely this attitude that has hampered the development of programming into a respected profession.

The world does not need lots and lots of code or programmers; the world needs less but better code and programmers, quality over quantity.

But this objective is thwarted when the idea keeps being thrown around that "everybody can, and should, code." There is already a glut of bad code developed by "programmers" unaware of what they do not know and unwilling to continue learning to improve.

Bad code, be it un-secure, un-performant, or un-usable code, is turning off consumers, and the earlier we sit up and seek way to improve code and product quality, the better.

> [...] the world needs lots and lots of code. In order to have lots and lots of code we need more programmers.

Nah, we can also have our programmes write code for us.

Snark aside, we are somewhat in agreement. We just have different personal preferences: I am happy to spend lots of time studying mostly useless theory for fun, and the ability to rescue me from endless tedious and unreliable hacking/patching every once in a while.

You are probably happier to deal with the tedium, and occasional blowing up of your software at runtime (Python, I am looking at you!)---in exchange for not having to waste all your time on CS theory.

The difference between a milkman and a programmer is that a programmer can build her own little robots to do all the tedious task for her. (And in most cases, by `robots' I mean little software tools, and not so much any hardware.)

The milkman can come up with some ingenious things too, eg a better route, or perhaps a particularly clever way to `tetris' the bottles and crates in the car---but it's not a core part of his job.

In contrast to the sibling commenters, I am happy about every milkman-programmer there is. That means less competition for my own breed of skills.

And that's the attitude that leads to the cure glut of IOT hardware with significant security bugs or no security at all. Build it fast and ship asap is not a sound plan for any software,and even less so for something that needs security. Anyone with an incentive of formal CS theory education will write better software than the same person before they had that education.

When's the last time you saw a milk-man?

I think this is a case of not knowing what you don't know.

I don't really think it has much to do with going to school. I think theory gets ignored because some portion of an engineering team has no interest in learning it, so we cater to the lowest common denominator because its the less expensive option, at least in the short term. It's actually the passionate people who managed to get into this without a degree that have been the most interested in new ideas in my experience.

That's not true, technical colleges award bachelors in engineering for applied computer science in economics, media design and, indeed, computer engineering.

It used to be the case, circa the 1990's, that you only saw libraries and example code. Sometimes a "skeleton project". Anything beyond that was exceptional. Having lots of open code available online was a new idea.

The thing that made the framework prolific - and it happened in tandem with the rise of Wordpress and other CMS systems - was having so many small, disposable projects. Rather than copy the same skeleton over and over, you could run a cute "new" or "init" command to do the same thing, glom together some plugins, tutorials, and styling, and instantly have something that looked the part - provided you didn't actually test it on a actual business case. Since many web site projects were and are still primarily fancy landing pages, with simple interactivity, frameworks became hugely appealing as a way to ensure a degree of polish and maintainability.

But frameworks became marketed as the way to make web apps, and here I think our enthusiasm went astray, since apps pretty consistently need to work through all their infrastructure from first principles, even if they're doing something relatively small. The "microframework" approach of bundling together unrelated libraries or interfacing their functions to a common protocol has remained a popular one, and that is most likely because it better satisfies the needs of app makers.

He is right. Give me a specialized collection of functions any time, aka, a library, but I will never again use a framework, which is a half-finished program, of which the users are supposed to figure out how to fill in the blanks. A framework imprisons your own code within confines and limitations of what will often turn out to be some kind of imbecilistan.

"a framework, which is a half-finished program"

I think you might have another point here. If the framework wasn't a half-finished, but reasonably complete, program, then the criticism of the article wouldn't fly. It's perfectly OK for a finished program to be non-composable.

Emacs is a good example. It is a text editor first and framework second (although it's hard to believe, because people joke about it). It's not composable, merely extensible. Emacs has all faults that the article criticizes about frameworks; however, since it is usable on its own, it works out positively in the end.

Frameworks are demonstrably not broken, as millions of individuals and companies use them, successfully, as part of their projects and products.

Why do software developers equate "this technology is imperfect" with "this technology is worthless"? It's the kind of hyperbolic, emotional thinking that I would expect from a child, or Donald Trump, not people who supposed to be rational, analytical engineers.

I read the "broken" in the OP title as conceptual problem, not a practical one.

"as millions of individuals and companies use them, successfully, as part of their projects and products"

This was true for COBOL in its heyday, too (well, maybe not millions, but there was less computers in the world, too). But would you use COBOL in a new project today, even if it's not actually broken?

I think the question that is left unanswered here is - if not frameworks, what should we use?

Conceptually, there are only two solutions either:

a - write everything from scratch, which I think we all agree will be disastrous.

or b - compose tons of libraries together - for a typical webapp, that means connecting your ORM library to your routing library, to your template rendering library, to your authorization library, to your authentication library, session management, caching, deployment management, logging, monitoring , etc...

Obviously, the big giant downside is that it takes really long to connect all those things, and you usually do it badly and break abstraction layers anyway (not to mention security).

You end up spending 10 hours on every one of the dozens of things you need to make a proper webapp instead of spending a few hours setting up most of your app with the framework of you choice and spend 30 hours on that 1 or 2 damn situations that don't quite fit into the framework's golden path.

Really, you can work to create better frameworks, but the cost-benefit analysis will always sit with working with a framework rather then joining things together - unless the project is trivially small or you have enough resources to practically build your own framework. And that's before we take into account new hires and the reduced cost of training if they already are familiar with the popular framework you chose to use, or the ability to create plugins/extensions/add-ons to a framework that crosses concerns (For example a plugin that can deal with both add a model and a new route).

I've found that without a framework, there is very little to actually connect. I use something like spring-context for stateful things like datasources and queue connections, for clean setup/teardown. In the controller, I call a service which returns a plain map with data, and pass that data to a mustache template. I tend to just invoke stuff and pass around data. What is the difference between `mustasche("foo.mustache", myService.baz())` and some implicit render logic a la Rails? One extra line with the template name instead of a framework that implicitly resolves the template?

Setting up a new app usually takes me 10-15 minutes and 100-150 lines of boilerplate (read config files, create spring context, wire up controllers, ...), which is definitely worth the (low) cost in my opinion to achieve the blissful state of absolutely no implicit stuff happening beyond my "main" being invoked. This win gets bigger the more the project grows.

You're overestimating the complexity of (b). Composing libraries is pretty trivial, and in my experience easier than learning to do meaningful work in a framework, and certainly easier than working around framework limitations.

The problems with (b) isn't the composition, which is easy, but the availability of quality libraries. Popular libraries are often written (and re-written, and re-written, and re-written) as plug-ins for specific frameworks. For example, it used to be hard to find a JavaScript drag-and-drop library that worked independently of JQuery.

That said, I'll take that pain over the pain of a framework. When you get to the interesting bits of an application, it's so much easier to evolve my own design than to work around framework design decisions.

(b) can look like this:

Luminus is a Clojure micro-framework based on a set of lightweight libraries. It aims to provide a robust, scalable, and easy to use platform. With Luminus you can focus on developing your app the way you want ...

http://www.luminusweb.net/

http://www.luminusweb.net/docs/useful_libraries.md

If you don't use an existing open source framework, you end up writing your own custom framework from scratch - And the outcome is not usually as good.

Then the next developer has to learn how to work with your custom framework from scratch (which is not as easy as learning the popular open source one).

All projects will have custom code in it, the business logic, that other unfamiliar developers will have to learn, regardless of whether or not a framework is used.

If instead of a framework you use a platform, like Ring on Clojure, Facebook React, JVM servlets, etc, then all a new developer needs to know is how the platform works and then get familiar with the code that has been written so far. Typically, these platforms have minimal implicit behaviour, so everything that happens is traceable via some kind of "main" entry point.

What part of this is "writing your own framework"? I don't feel that I write my own framework, I feel that I completely avoid the framework and the framework related problems highlighted in the article.

If you aren't using a framework, you are writing your own framework. Moving from project to project will be a pain, for you and others.

Disliking frameworks is not very different from disliking DRY, conventions and abstractions.

You can also write a library, which is a different thing than framework. What you're saying is a false dichotomy, based on different understanding of the term "framework".

I think what mainly distinguishes framework and library is that framework dictates you conventions outside its own code, where library does not. This dictate can be as simple as "I must be in charge of running the program, i.e. the top-level thing".

See, the conventions are not bad per se, however, they do not compose. So bundling conventions with potentially reusable code, as frameworks do it, is a wrong idea.

In my experience, I've been able to completely avoid the framework, rather than building my own. I end up with pretty much the same controller and db access code, absolutely no implicit logic, a "main" entry point (in my case typically in the form of a @ServletContextListener class), and maybe 100-150 lines of boilerplate - i.e. the stuff that is usually implicit, such as reading config files, connecting to a database, etc.

I do wonder if some issures (especially about composability) are actually caused a layer deeper, in the programming languages.

For example, java has some very peculiar ideas about typing and metaprogramming that encourage a "plugin" approach an tie your codeto 3rd party libs stronger than would be necessary.

Some examples:

- Classes that implement an interface now have it as a dependency: If the interface class is not present, your class won't compile. This even though the interface class usually doesn't contain any additional information (exception being "extends" clauses)

- the same is even true for annotations - even if those annotations don't even end up in the compiled class at all.

- the language itself offers an extensive range of tools to package your code into discrete units with well-defined inputs and outputs. It offers significantly less tools to wire those units together and actually make then into a complete piece of software. In the end, if you roll your own solution, you often get to choose between "bowl of spaghetty" or "needlessly complicated". Frameworks stepped in to solve that problem (or at least make it look as if all the complexity were justified).

- the reflection API encourages adding your own semantics to java language constructs and in a way extending the language - except there is no way to coordinate reflection usage by different frameworks.

Saying that Monad should be used instead of frameworks, because frameworks don't compose?

Monads don't compose, and the workarounds are ugly and still a topic of active research.

I can't upvote this enough.

<rant> I want to use PFP techniques. I have debugged numerous bugs after which I wished that the return type of a function encoded the effect the function has.

But, boy, do they make it hard?! I really do wish that PFP advocates stop deluding themselves and make a better pitch for why and how what they advocate be used by the industry. The first version of Wadler's "Monads for Functional Programing" [1] paper came out in 1992. The paper "Monad Transformers and Modular Interpreters" [2] came in 1995. Swiestra's free monad paper [3] was published in 2008. The Eff Monad paper [4] came in 2013.

Think of all the advances computing has made since 1992. Programmers in the industry have worked hard equipped with their tools deemed primitive. Hell, people have caused real world revolutions armed with nothing but PHP. Yet, the PFP folks haven't gotten around to compose multiple effects in any sensible fashion ... since nineteen ninety two! But that has not stopped them from pointing out with derision how everyone else is doing it wrong.

> It is imperative to understand that the need for composability in our software tools is an absolute requirement. If we as an industry have any hope of not repeating ourselves time and time again, we have to change our ways.

Yeah, right - and which decade can we expect the PFP solution to be delivered? </rant>

[1]: https://cse.sc.edu/~mgv/csce330f16/wadler_monadsForFP_95.pdf [2]: http://haskell.cs.yale.edu/wp-content/uploads/2011/02/POPL96... [3]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.101... [4]: http://okmij.org/ftp/Haskell/extensible/exteff.pdf

It's kind of a circular argument here. "Frameworks, which I define as things with these problems, have all these problems." A more insightful article or even series would start with the problems and then focus on how various frameworks and libraries either fall into the trap or transcend it.

I disagree with the headline quite strongly, but a lot of the negative points made in the article resonate with me

It sounds like the author doesn’t really have any idea what a “framework” is, but has heard several things they dislike called frameworks, so the presented thesis is “frameworks considered harmful” when what might be more accurate is the ”the following design patterns have this set of problems.”

No, he actually has a very good idea what crappy frameworks are, because he wasted a bunch of his life working on the Lift web framework.

So what is your definition/understanding of the term "framework"? Can you give examples of something widely considered to be a "framework" that doesn't suffer from problems described by OP?

I have to say, I disagree with virtually every conclusion here.

I don't doubt that there are some bad frameworks, or indeed that some developers have a habit of reaching for the closest framework even when it's not the best solution to a particular problem. This was most obviously demonstrated to me once when several applicants for a developer role submitted a simple CLI coding exercise as an entire Rails app, despite the complete lack of that requirement.

We all use abstraction to model problem domains in terms of familiar concepts. There are different levels of these abstractions, and different levels are used in different places where it's appropriate. Sometimes using no framework is suitable; sometimes a small framework is suitable, sometimes a grab-bag of libraries, and sometimes a larger framework.

I've used Rails a lot. In some ways, it's a pretty standard implementation of well-known concepts – REST, Active Record, MVC et al. But it removes the work and friction of selecting an ORM, routing library, template renderer, asset pipeline, session management and so on, instead providing workable standard solutions for all of these. In many cases, this is completely fine – there are a class of web applications that Rails is pretty great for. Using it reduces the amount of custom plumbing that is required, and that can be both a short-term and long-term win.

On the other hand, some applications don't fit into a particular framework's constraints. Building a server to do some basic websocket communication would probably not be a good fit for Rails, for example, and one might consider building it on top of some libraries in Go, or Node, or something else. Maybe sometimes a larger application has specialised requirements that mean a framework would be an unsuitable choice, and it's better to compose modules in a different way. These different scenarios come up all the time!

Avoiding the use of frameworks, in my experience, can often result in developers building their own buggy implementation of 50% of what the framework offers. If the problem exists at all, it exists in the unsuitable usage of frameworks.

I think it depends on the platform, and libraries available on said platform.

On the JVM, I create webapps using a @ServletContextListener class, which essentially is a "main" that gets the servlet context passed to it. You could call this implicit, but it is at least the only implicit thing you need. I have 5 lines that read in and left merge some config files. I manually invoke a spring context for setup/teardown of stateful stuff like database connections, queue connections, etc, and initialization of core services using these. Some more lines to set up the Jersey controllers on the servlet, maybe asset-pipeline or something else for handling static files, and we're up to a total of 100-150 lines of initialization/boilerplate.

The win here is that there is absolutely nothing implicit about my app besides from the invocation of "main". My main beef with frameworks is the implicitness. Great when it works, but I'm so tired of hitting some kind of bug where some implicit thing stops happening for whatever reason and having to trace it down in the framework bowels.

In my experience, there is very little difference between learning a framework vs learning a platform (JVM servlets) and a few core libraries (spring-context, Apache's data source for DB connections, JMS for queue stuff, Jersey for servlets, spring-security for auth).

I don't think frameworks are flawed, but platforms and languages.

I mean, just look at the Web. DOM, Web Components, Indexed DB or WebGL could all be considered features of the Web-Platform and they seem all to have awful APIs.

Most frameworks, yes, not all

It's like overcomsumption when using too many frameworks, it ends up bad

The last good framework I used was Metrowerks PowerPlant.

"Frameworks are the worst thing to happen to software development in the last decade (and possibly ever)."

His site is using bootstrap.

Reading that article on the phone is what's broken. It could really do with a framework that will stretch it to 100%

This is when Firefox reader mode comes very handy!

Reply on news.ycombinator.com