Too Many Tools and Frameworks (2015)
mrmrs.io
mrmrs.io
Each ecosystem around a set of technologies essentially has its own "Herfindahl index"[1] of sorts. For example, the Ruby ecosystem's index might be fairly high, simply because Rails is so dominant (to the point where Ruby and Rails are often conflated by outsiders). On the other end of the spectrum, the current JavaScript landscape seems to have a fairly low index; there are a huge number of tools with very few clear winners at this point in time.
I suspect the best/healthiest ecosystem is probably somewhere in between that of Ruby and JavaScript. Rails may have ultimately hurt Ruby by its dominance, just as the current volatility in JavaScript tools seems to drive some people away (or at least add frustration).
Of course, I have no numbers to back any of this up - it's just an observation (one skewed by personal experience, no doubt).
I have often wondered whether the level of complaint relating to Javascript framework proliferation is explainable as due to language structures (whether notably present or notably absent) or unusual initial conditions of the JS community/ecosystem.
Is there even a field of techno-psycho-sociological studies, and is there an equivalent of the Sapir-Whorf hypothesis for programming language ecosystems?
Regarding JS frameworks, I think the barrier to entry is still low, and given the suffering involved for any given tool-chain, I'd imagine the temptation to roll ones own is high.
Finally, JS framework fates are tethered to that of the browser. The latter changes frequently, so the former ends up in a refactoring perma-loop.
Could you elaborate on that? What kind of browser changes are happening so often and are so deep that frameworks need to be refactored every time? (My experience is quite the opposite.)
What developers should really do is contribute and make the existing projects better. Everyone will thank you for it.
There are thousand working on linux (the kernel). How many of them do you know?
Having a patch or two in angular doesn't seem to move the needle.
Getting something on Angular means you were able to get your changes approved and merged (team work)
Situation is better if you can have a bigger project or more people using your something.js
Much of what we work is all with people and on large codebase, so if you are contributing to existing project, it is actually helpful for interviews. Also it takes weeks to get a PR accepted, so if you actually were successful in getting your feature accepted, it shows your resolve to contribute the feature because you want to contribute and not because you want to do a PR stunt for your career (and not get annoyed by the constant changes which the authors ask, which are close to a million)
Many more JavaScript tools and libraries have websites that are visually appealing than for any other tool or technology that I've seen. Since the informational content is usually not of a similar or higher quality in my experience, it stands to reason that appearance is more important to developers of JS tools/libraries. Which could imply that peer recognition or career building is a more common, or stronger motivators in this particular demographic (of JS tool/library authors)
I'm also entertaining the thought that JS development might be a more solitary work (on average), which could conceivably also influence if you choose to cooperate rather than build anew yourself. Most JS tools were also not very big, as they had to be delivered to the client, and as such more amenable to being built by a lone developer or a small team.
The only thing I known for sure is that in 20 years of coding, nothing have confused me as much as the plethora of JS 'things', not even DCOM or the almost unequivocal praise of Hibernate (for a while)
I think you read too much into it. The front-end of a website would be important for tools which help making front-ends of websites, don't you think?
I agree with your other points, but this one is easily explained by the fact that people who specialize in Javascript tend to be in the front-end/UI development field, and thus are more likely to have an artistic background/sensibilities.
However, there is also something to be said for rewriting work. There's the concept of "plan to throw it away"[0] I think a lot of these projects start because people are just trying to learn. Then they share them and get built upon. You then have an ever increasing amount of technical debt, eventually inspiring others to write a simpler version and then that becomes the defacto and so it continues...
I think my favorite example of just stupid stuff that keeps getting propagated is the Date type in javascript - the month field starts with "0", but the day field starts at "1"[1]. Like what is with that?
Even with everyone contributing to Javascript that stupid thing still exists because you can't break the internet to fix it (ask the Python community how changing a lot of syntax works out - although Python wasn't even that bad). That's why people write new stuff, because they want to make things simpler. Not really something to complain about IMO.
The unique stuff I learned is probably just these four languages: BASIC, C, C++, Python - I learned it in that order and now I can pretty much do w.e. language I need to use. Because all frameworks are based off the same stuff. Go is easy, Javascript - no problem!, I do full-stack work and data science in my day job it's really not hard at all. All the frameworks make it even easier.
But it's also difficult to decide if it's worth it, especially if you've found a real deficiency. The creator of http://propelorm.org said to me that his framework gets him €120k+ offers and that he recommends building open source projects that are in demand.
So I don't think it's possible to give a definite answer what developers should really do. Maybe bit of both?
And it grow more un-fixable as longer and bigger is the project.
For example: Fix C. Or javaScript. or Angular.
Is impossible to do it without change what them are. And when the trouble is intrinsic the only sane option left is restart.
In fact, stick to old is as worse (and maybe more) then rewrite. The trouble is when rewrite not change enough to be worth it.
This isn't a problem by itself, the problem is the ever-changing "clear winners" and hordes of people using them with no understanding of their trade-offs.
I hear complaints like this a lot: "It was so easy to-do this with jQuery, why is this JS-framework-de-jour so complicated?" - well, why didn't you start with what-you-know and explore their limits litte bit, so you would know what you really needed?
With Ruby and Rails, you have a single answer to many use-cases, with JS and the npm, you have many answers to each use-case imaginable and choosing by popularity is very damaging.
I think this description matches the Python web ecosystem quite well. Historically there were the Z(ope)-ish projects, which still exist and are being used (and they work quite nice); Django obviously dominated for a long time, but in more recent years some healthy competition ensued (Flask and it's highly modular approach for example, but also Pyramid), which resulted in a lot of reworking and improvement for all frameworks. Django now vs. Django 2010 is a vast difference in developer and user friendliness. Some specialized frameworks popped up (Tornado comes to mind), but overall there isn't a heavy fragmentation of the ecosystem (bad) but no one dominator either (bad).
Web frameworks: Spring, Java EE, a lot of solid smaller ones.
Build tools: Ant (mostly legacy), Maven or Gradle.
Then for almost any niche there's usually 2-3 clear contenders, production-grade stuff.
For such a huge ecosystem Java is remarkably not-fragmented.
Right now it seems very clear that React, Redux, and Webpack are winners.
The difference, I think, is that many of these tools are simple enough that if you have a new idea you will simply write a replacement instead of trying to work with the maintainers to push out a dramatically different new release.
Language communities are in an arms race to create the best platform for the Cambrian Explosion of tools necessary to support a dramatically growing and diversifying developer population. Languages with a proliferation of tools, where developers are experiencing choice fatigue, have the incentive and the usage data necessary to design the meta-tools that will make development in such an environment easy. Languages which rely on small selection to solve the problem will remain niche languages.
We're just waiting for some innovations in package/community/docuentation/source control management to crystalize and make the whole problem moot.
But the issue for me is how employers handles it. No, I don't know Angular. But I have used Backbone, Ember & React. So do you really need an Angular expert, especially when Angular is only a few years old and is completely changing in Angular 2?
Should employers insist on extensive trade knowledge in a specific framework, or should they be able to hire based on understanding of development and architectural principles, and give a new employee a little bit of leeway to get up to speed on their stack.
If it's the latter, then you can relax, and focus on reinforcing good practices and understanding the theory about what you're working on, and your resume will continue to build.
If it's the former, then you have to constantly pad your resume with skunkworks projects simply so it doesn't seem like you're getting "left behind." And that is exhausting, and not very useful to anyone.
Why do you ask?
It astounds me that hiring for JS frameworks actually is that specific, considering you're likely passing up on people that are simply better problem solvers.
This post really sums up my experience, and I've done plenty of Javascript before (I just haven't kept on top of client side frameworks):
https://hackernoon.com/why-learning-angular-2-was-excruciati...
If I were to hire someone to work with me, I would try and find someone who has experience with Angular2. Angular2 vs JavaScript are effectively different technologies compared to Ruby vs Rails.
I'd actually say for someone completely new (e.g. locked in a basement doing C or Java for the last 20 years), learning Erlang would be far easier than learning modern JavaScript and Angular2.
The decisions they made going from Angular1 to Angular2 are a big reason I've never really bothered with Angular.
Just on a base level, if a system isn't properly documented, that does make a case for finding someone with indepth knowledge.
But a company using a system that is very poorly documented as their core technology is a big red flag for me.
I suppose that might be the case. It feels incredibly counter-intuitive, but it's not inconceivable that the JS frameworks stray that far from "normal JS".
Were I in this situation as a employer (and co-worker, necessarily, not only in a managerial position), I would probably make the decision to use other technologies and compile to JS, in that case. There's very little to gain from using a framework that by necessity is tied to a crummy language if I can skip using that as a source language anyway, and then I'd just go with something sane like Elm instead.
(Bear in mind, this is coming from someone who doesn't actually work with front-end development, so take it with a grain of salt.)
To clarify, what I mean is that someone who can't learn Elm, PureScript, Bucklescript or the like, very likely isn't a very inquisitive person and isn't someone I'd like to work with. On top of that, someone who can't see why you'd want to write in any of those languages instead of JS very likely isn't someone who I would want to work with, for fairly obvious reasons.
CircleCI chose to use ClojureScript and Om instead of using React directly. NoRedInk chose to use Elm instead of writing JavaScript directly. Others have chosen ScalaJS.
It's not popular in the startup world, but plenty of companies chose (and continue to choose) to use GWT to build their in-browser user interfaces. I worked at one of them back in 2010. I had huge doubts about the Java-to-JS compilation process. I'd mostly only seen ugly GWT apps cobboled together from the UGLY base components GWT shipped with. As it turned out, we were able to created our own good looking components quite easily. So long before Angular or React existed, we were able to create a nice an easy to understand component based web app, with business logic split out into services and wired together with dependency injection. It ended up being a sane choice that enabled us to continue developing safely and sanely as the app became larger, and it performed very well on both desktop and mobile browsers. GWT wouldn't be my first choice (or second, or third) in 2016, but it ended up being a pretty good one in 2010.
I'm not saying that choosing a compile-to-JS language that has very different semantics than JS is necessarily sane and practical. But I think that plenty of real world successes demonstrate that this choice can be simultaneously sane, practical, and successful.
The first set is normally seen as "hard", because it is very easy to shoot oneself at the foot with them, and because experts tend to be attracted by them. The second set is often seen as "easy" because the complex features are normally there to stop novices from harming themselves.
The unexpected thing is that "simple" is much easier to learn than "complex", independently of what other adjectives come bundled with them.
1 - Yes, the other 2 combinations exist. "Simple and powerless" does not make the headlines, and "complex and powerful" is very, very rare.
Often they opt for what's "hot". Which is the worst possible approach. Or for what it seems easy to hire in (which isn't much better).
I think yours is a dangerous attitude, I've seen a lot of mess produced by people who come in to angular knowing other frameworks and their output is terrible.
Part of that is because angular is different, it's not very nice and it has obscure knowledge needed to make it play nicely with unit testing.
If you take an ember / backbone / whatever expert and give them a few weeks angular training they'll rock a todomvc but full apps often end up with controller-as-a-god-object.
Genuine experts would be really nice to find, but it's really difficult because frameworks aren't given time to mature before the smartest people have moved on to other frameworks.
You're right that employers should hire on general understanding and give time, but the required time is typically 6 months and a failed project or two. Many employers don't make and don't want to make that kind of human-capital investment anymore.
The "rockstar" who knocked something up pads their resume with another project with a hot new framework, got experience with another new set of tooling and can leverage that to move somewhere else to produce more working but hard to maintain code in another hot new framework because their CV is padded with experience showing they're a quick learner, but the company left behind pays the cost in maintenance.
A genuine angular expert will likely be well employed for a while come helping troubleshoot and clean up the mess left by the cool crowd.
p.s. I am not an angular dev, this complaint can be leveled at a lot of hot new tech, particularly but not just the javascript front-end world, it equally applies to people who knocked up go microservices at a .Net shop, or hastily introduced mongo to replace oracle, etc., etc.
A possible approach is to let developers new to a framework on a prototype or toy project first, and, very important, throw away that prototype.
Yes, that means you might have to wait two or three weeks longer until you get started with the real project, but the result will be so much better.
But that's one of the prices you pay for using an immature technology. For a more mature one, you could read a book on successful usage pattern of the technology.
Wouldn't a depth of knowledge of dev patterns (and anti-patterns) be enough?
I don't have to know anything about Angular to know that's a bad idea.
Outside of a hire-an-expert consulting gig, I really would say the problem here lies almost exclusively on the engineering management, or lack thereof, and broader staffing decisions. I don't think it's necessary to have a project fail for someone to understand a tool[1] but the ability for someone to get up to speed is a function of the time invested in things like mentoring and code-review.
Yes, many companies make the short-sighted decision to skimp on that investment but that needs to be recognized and fought as a management problem – any project is going to suffer in those conditions even if your entire team is staffed with experts because understanding the actual business problems is as hard as the tech stack and if you can't carve out time to think about a few controllers, there's no way that the rest of the problem doesn't have problems which are at least as bad.
1. If true, that would be a huge sign that the tool has major design failures and is not suitable for production use rather than an argument against hiring people who aren't already experts.
One of the most common complaints is that the biggest web tools are libraries, not frameworks. React, for example, is very emphatically not a framework. To develop any reasonably sized application, you need to bring in a number of other libraries as well (such as Redux and/or Apollo).
I think it's very interesting that even though the expert and experienced programmers who create things like backbone.js and React have openly and explicitly designed them not to be frameworks, people continue to call them frameworks, and not disparagingly. There seems to be a huge appetite for frameworks. From the outside (and definitely projecting my own inexperience) I would guess that a lot of people feel lost architecting a web UI from scratch. They don't want building blocks. They want a way, a path to follow. A lot of server programmers felt the same way twenty years ago. Adopting a framework meant giving up freedom that they didn't want in the first place because they didn't know what to do with it.
But I'm coming at it from a very outside/inexperienced perspective, so I feel like I should leave it as a question rather than a comment. What do you think? Why are frameworks considered such a good thing in web UI programming that widely used and admired libraries end up being called "frameworks" as a term of respect?
It's objectively not. To build any reasonable application you'll also have to choose, at the very least, a state management utility as well.
That being said, I'm sure there are beginners who call React a "framework" but that can easily be explained by the simple fact that they are beginners: they don't really understand what React is or even what the differences between libraries and frameworks are. They assume any stack has to be built in some sort of framework, so React is that "framework."
> Why are frameworks considered such a good thing
The only thing nice about framework is that they make the main workflow obvious. But things becomes scary when you walk on the wild side (backed by experience with Django and Django RESTframework)
The problem with reactjs is that facebook was using reactjs and its "architecture" for big apps and now they try to make it work in a more general context.
> in web UI programming that widely used and admired libraries end up being called "frameworks" as a term of respect?
Backbone is framework, saying it's a library because you can use only the collection model and use whatever method you want to fetch your data or whatever template language or bind events yourself is like saying you can replace the URL routing or template language of Django on your own. Doable but not recommended.
I think backbone is framework not a library it tackles several issues (class system, views, and middle end) and it does it badly but that's a topic of ~another post~ the past.
Historically Javascript community has favoured the do-it-yourself framework hence or because there is a lot of libraries you can choose from or because the needs of everyone are so different that's it's difficult to create a miningful framework. Mind the fact that Google previous "framework" named Google closure was a stdlib kind of framework compared to the competition of MV* framework like emberjs or angular it did not force a given workflow.
But is this pressure justified? I see two possibilities:
a) The latest crop of the top 10% tools really makes us much more productive than last year's crop of the top 10% tools. So the best developers will always filter through a lot of new tools to find the best 10% and benefit from them.
b) After accounting for productivity losses incurred by constantly relearning and rewriting everything, the lastest best tools have no productivity benefits. They are just a distraction from the product we're supposed to be working on.
I don't think there is ever a time in history when this question can be answered unequivocally, and therefore it is never possible for good developers to stop looking for better tools completely.
However, I feel that at no time in the past quarter century has the pendulum swung so far in the direction of (b) where front end development is concerned. I realize this is very subjective.
Frameworks can help teams create a baseline standard. A custom framework can be something that you need to educate other developers with, better to have an external framework at times. But at the rate of churn, depending on the team and product, some frameworks makes sense and some don't. Monolithic frameworks are harder to change, micro can swap more easily.
Really in the end, all frameworks and tools are abstractions to help accelerate development, but sometimes they don't, and one case of that is constant churn. New frameworks are a healthy sign for active development but they also have a cost associated with them. Frameworks can also abstract away too much and shroud the real platform/standards which leads to lock-in. Each project and product has needs that some frameworks fit and some don't, real development/engineering is deciding what a product/project needs.
I don't know why people compare gulp and grunt with webpack, they solve different problems.
Web development seems unusually prone to this trend, but I've seen it happen with software in general. I hypothesise that the reason for this happening to web development in particular is largely because the basic problems have already been solved long ago, so all this churn is created by people who are "oversolving the problem", having nothing else to do than to think of new --- but not necessarily better --- ways of doing it.
Introducing complexity must feel productive to a lot of people, and I suppose some derive pleasure from being able to learn about, build, comprehend, and modify such complex systems. But I don't. Making websites that require several MB of JS and the latest browser features to display a few KB of static text? I just don't see the point.
Related: http://countercomplex.blogspot.com/2014/08/the-resource-leak... and discussion https://news.ycombinator.com/item?id=8679471
I had a much better list, then the previous Poster, because it solve my problems the best way.
Did you need my list now too?
I am absolutely sure, that somebody else had a better list.
Where is the progress in that?
The specifics are supposed to be automatically handled by preprocessing and deployment tools, or only cared for in the optimization/final polish phase.
90% is also roughly the failure rate of startups. Pareto principle?
There's no equivalent of the basic problems of carpentry in the profusion of javascript libraries. Many of the libraries written in the last year solve problems created by other libraries.
People forget that the reason Rails was such a revolution is that it took all the fifty kinds of crap that people used to build an application and made it available in one package with good defaults. The good defaults part of that is absolutely critical.
The problem is not "bad libraries", it's a complete lack of curation and direction.
The long term answer isn't to have fewer options or barriers to entry for new frameworks. The right solution is to make better ways to browse and select between all the options we have available.
Can we curate libraries algorithmically? Or do we need people to do it? (And if so what should their tools look like?)
I think you're missing the point. The problem is not "which library is best at x y and z", the problem is that nobody agrees on what X, Y, and Z are.
It's a definitional problem, not a curation problem. Curation is easy once you have a yardstick to measure things with (pun intended).
Again dropping back to the metaphor of woodworking, one can clearly evaluate the results of using any given tool against a thousand years of physical history - no such comparison is possible with current libraries, especially in the rat's nest of javascript, and that's fundamentally the issue.
Meanwhile, the hammer you use is no longer supported, and breaks every time you pick it up, but using a new hammer means changing every single nail in the parts that you've already completed.
Then the job ends, and you look for a new job, they don't think you're qualified because you used the same chisel for two years, and don't think you'll know how to use the new chisel design that came out 3 months ago.
This made me lol, thank you ... Those poor carpenters. Thousands of years of metaphorical abuse.
Also the fragmentation of efforts to improve the tools is a waste of resources. Instead of choosing a few directions and delivering, everyone knows better and it's doing his own version of different, but usually not better, tool. That will help anyone else? "Yes!", they will say. "I don't care", they think.
As an agile guy, back in the day I wrote an Agile story tracker. Fun times. Then I saw some other Agile project management tools. Then for a while it seemed like every week or two I was getting emails asking me to evaluate yet another agile/to-do list tool.
It seemed that programmers were unable to grok the agile story concept without making their own tool to use it.
But I saw the same kind of thing in the code I wrote. I wouldn't just solve a problem; I'd solve a class of problems. Generalization was awesome! The more I generalize and account for edge cases, the more value I am creating! Woo!
Many of us programmers, myself included, are constitutionally incapable of just solving the exact problem we have using the minimal amount of code and then moving on. We confuse coding with value creation, so we build frameworks and tools out of everything. If you build a list sorter, that's okay, but if you build a 30KLOC list uber machine, that's better, right? After all, it does more. You worked harder at it.
The second thing I saw, which was even more painful to realize and accept, was that it was very easy to get "upside down" in a toolset or framework and not even be aware of what's going on.
The first time I saw it was when I was teaching a mainframe team Visual Basic, back in the 90s. We were building a simple form with a textbox and button. I had showed them the concepts and they were a smart bunch. I started the exercise, but a couple of developers were stumped.
Why?
When I walked over, the problem they had was the properties page. Sure, they got the concept of drag-and-drop. But once the button was dropped? There were like a million options in there! What was the right font size? What did this thing mean? And so on.
I thought that was pretty silly and tried to forget it. But then I noticed in my own work that I'd get sold on a new set of tools. For sake of argument let's use the old Infragistics web controls. They do everything! Amazing control and look-and-feel!
I'd drop them on a web page wanting to create a smart grid. But the grid didn't do X, and in my mind, X was what needed to happen. So I'd poke around at the help file. I'd go online. In some cases I'd tear apart the javascript and start fixing it myself. Hey, it's only web programming. I can do this.
But holy shit that took a lot of time. On my next project when I needed a grid? I dropped a HTML table in. Just added whatever functionality I needed directly from JS. Not only did it take less time, but the resulting codebase was easier to read.
Then I started watching the teams I coached. They very rarely talked about actually solving problems, or language/foundational issues. Instead they talked about tools and frameworks. This tool or framework would do this cool thing if you tweaked it like this. Joe spent 2 days trying to get Y to work.
For many, many teams, buying into the framework means buying into a complexity and cognitive overhead that's siomply not relevant to the actual problem they're trying to solve. I don't know. Maybe it makes them feel cool or something. One of the smart kids. But speaking as both a participant and an observer, it's whack.
tl;dr So sure, let a thousand flowers bloom. But your job isn't arranging flower baskets, it's helping somebody who's down feel better. You might not even need flowers at all. Stop focusing on the tools and start focusing on the solution. Then the tools will work themselves out.
The point of that story is that it takes a long time to properly understand a problem domain well enough to start solving problems in it at the right level of abstraction. A framework can "understand things for you" and just work if you set it up right, but the cost is that you lose the freedom to replace that framework with something smaller and lighter.
I tend to wait until I've solved a problem in similar ways three[1] or so times before I generalize it.
[0] Because other developers want to solve for a more general problem. [1] Three is such a comfortable number, isn't it?
First is an over-abstracted mess that may or not work before it eventually collapses. Brainstorming.
Second is a rewrite aiming for compactness, that could end up impossible for anyone else to understand.
Third pulls just the right amount of abstraction out of the terseness of the second pass.
I treat it like a feedback driven control-system. Too much abstraction? Dial it down. Not enough abstraction? Dial it up. Repeat until good enough.
If I stop at 1 or 2, the code is poor. If I don't oscillate, it takes more passes.
<View cls="ba jcfs pa2">
<Text cls="white tc">
Something
</Text>
</View>This is also true of Startups -- most of them die, but that doesn't mean they didn't add value in validated learning (positively or negatively).
Elon Musk has often stated that he thought Space X had very little odds of success, but he continued because he believed he could create value that someone else could build on.
It's really hard to tell these days.
After some experience with node.js - I'm weary to pluck anything that's not in the 'top 10' most established frameworks.
You need organisation. Which is again time/money/resources.
Four words in and CrocodileJS is already lumped onto the 4/5ths heap of unusable frameworks (Node is an immediate non-starter in the same vein as MongoDB). Just to give you some perspective and perhaps help you see the irony.
http://www.javaworld.com/article/2079190/scripting-jvm-langu...
> CrocodileJS is already lumped onto the 4/5ths heap of unusable frameworks
in this context, "framework" seems to refer to CrocodileJS, not Node.
The README says yes but the amount of work that has gone in to it...
why am i not surprised
We even have smart light bulbs -- my door light turns itself on and off at dusk and dawn.