“Don’t Reinvent the Wheel, Use a Framework” They All Say
mogosselin.com
mogosselin.com
In this context : you're either comfortable not knowing, or you're not.
What I mean by this : some of us are comfortable not knowing how X system works (temporarily, of course). Some of us can start building things with a web framework and gradually gain understanding from the top down. This is OK.
However, others (and I include myself in this camp), are completionists. Abstraction is an annoyance, and not knowing is like an itch that needs to be scratched. For such people, having significant gaps in understanding a system model is a real problem. Super duper high levels of abstraction is a real problem.
Different people, different learning styles. For any beginner in any field, it's important to acknowledge how you learn best and then pick a learning strategy that's compatible.
I found it very true that different people learn in different way from my effort on sharing the details of programming and math with fellow co-worker and interns too.
Try make 30 people grab the same details (programming/details) and you will learn a lot about what you teach about and also on how people learn.
When i understand that, i ultimately understand the reason why there is so much debate and on going discussion about some particular details: much of our confusion were caused by lost of context in communication.
[edit: typo]
I think this a common fallacy that many developers fall victim to. Our entire modern lives consist of using things we don't understand, by this logic you should also not fly in a modern aircraft because I am sure you don't fully understand how they work. Better yet, you should refuse to drive your car.
>Super duper high levels of abstraction is a real problem.
Not at all. If an abstraction saves you a ton of time, you should accept it by all means. If you are your boss whose money is being wasted because someone refuses to use a time saving abstraction, I am sure that person would not be your employee for long.
If I were engineering a car or a service tightly integrated with one, I would be pretty interested in knowing how cars works.
> If you are your boss whose money is being wasted because someone refuses to use a time saving abstraction
That's a bit out of context, don't you think? The article was making a claim regarding how beginners should learn web development - not how businesses should be run.
However, since you bring it up - in my experience misunderstood abstractions are at the root of many, if not most, security vulnerabilities. Trusting the system without understanding it is often quite irresponsible.
But if you aren't, you don't. If you're building a web application, you're the delivery driver, not the mechanic. You just need to know how to use the car.
That's a flawed analogy. Delivery drivers aren't responsible for configuring or providing maintenance on the packages they deliver.
However, this is drifting pretty far out of scope of the original discussion, which concerned learning strategies for aspiring developers.
While I think it's a good idea to have an understanding of the systems we build on top of, that's just my opinion. We'll have to agree to disagree.
Understanding the cause and effect between your interactions with the flight stick and throttle is one thing, understanding the physics behind it is another entirely.
Most all photographers know basic exposure rules, but there are some very good photographers that don't know much beyond that. They rely on trail and error to get consistent results and probably know their tools better than someone who spent that time jumping around to different tools but know the concepts really well.
As an aside, the best thing I did was force myself to use a 1963 Minolta rangefinder, with one film type and speed only, for an entire year. Manual everything. This allowed me to really learn those concepts properly, but it was still steeped in trial and error; reading the concepts, then coming up with them afterwards from first principles is easily the best way for myself to learn a topic :)
The only example I can think of is that I've heard Paul McCartney can't read or write sheet music.
Some people (like myself), feel the need to understand the underlying technologies in order to become competent. I don't feel like that's the only way to competency.
After reading about the application of the Newtonian dynamics on life via wikipedia, honestly, the best description is probably a mix of both; Newtonian dynamics explain the action on the bottom of the wing, while the Bernoulli principal (directly related to the Euler derivation of Newton's second law) explains the acceleration of the airflow over the top of the wing and the negative pressure generated there.
Its worth noting that the negative pressure generated by the Bernoulli effect is greater than the positive deflection pressure, else planes would have a hard time stalling since the greater the angle of attack. Stalling is caused by the separation of the airflow over the top of the wing from the wing itself, resulting in a loss of lift, at extreme angles of attack (which would thus present greater angle and surface area for deflection).
Abstractions are a basic requirement for any sort of advancement, there is no inherent virtue in knowing what's under the hood unless you have a good reason to dig deeper.
When you get sick you take drugs that may actually kill you, you don't say I am not going to take this pill because I don't know how it works. Heck, I have no idea how the lowly Tylenol works and I take it all the time:)
I disliked web frameworks when I was a "beginner".
> It's fun to think that completionists exist, fun like believing in the tooth fairy or the little rat.
That's a bit nonsensical. Plenty of engineers feel strongly about having a solid understanding of the systems they work with.
It's not a bad feeling, but feeling that way doesn't magically enable you hold all the nuances of the output of billions of lines of code in your head.
I've noticed that most engineers would rather rewrite than firmly understand.
I am weak on the hardware level. I know more than most programmers I've worked with, but embarrassingly little when talking with actual hardware people. I've done a little work to rectify this, but have miles more to go and I'm not sure if I'll bother.
I don't consider myself extraordinary in this regard, so people who know the stack at some level of depth is not all that surprising. I take it as a given that there are many people who know what I know plus hardware. Completionists aren't unicorns (at least not for any sane definition of it. No one knows a modern operating system 100%, for instance).
That said, knowing what the kernel does with sockets is not very helpful for writing a web page. In my experience the vast majority of benefit one gets from being 'full-stack' is knowing one level deeper than whatever they are working on.
If you are writing a performant web server it absolutely helps to know what the kernel does and if you are writing a web language framework you probably ought to understand how the web servers you rely upon generally work (for instance, threading models, etc). You usually don't need to know two levels deep very well and almost never three levels deep. Usually the intervening abstractions are complete enough that those details eventually get papered over into something else.
I worked on a few large scale java web apps and regularily had to find and fix performance issues. I'm not sure if it would have been possible for me to easily dig into GC internals or find bottlenecks in Spring itself, without some lowlevel understanding.
So first of all you need some kind of lowlevel understanding to understand a GC. Hell I've talked to Java guys that couldn't explain the difference between stack and heap.
And you need to be able to support the "few tens of megabytes of libraries" (which surprise, are not just a few tens of lines of code) for real projects. If shit hits the fan and you can't google the solution, you need to dig into the framework code.
Which is (I think) is why the OP sais, don't use a framework if you couldn't code it yourself.
Sure you can work yourself top down the stack, but chances are (from my experience with others) that since Spring just works (in most cases) and for small apps never really have to deal with the GC, they think it's okay to not understand that stuff.... until shit hits the fan.
You have to start somewhere - and, sooner or later, you have to learn to debug other people's code or systems. (Which is actually pretty easy with modern tools; you don't need a deep understanding of spring to find a performance problem if you have a decent profiler)
Again, I'm not talking about a full blown JVM with feature parity and all the JIT optimizations. But actually given docs and some time even implementing a JVM should be doable, sure it will be slow but interpreting bytecode is really not that hard.
You just can't do this if you don't have a basic understanding of C or other lower level languages.
Also your CV probably sais Software Engineer and not Hardware designer. So I'd say hadware bugs are out of scope.
If we're talking about experienced developers then sure, at some point people should learn that stuff. But as a beginner it's impossible to understand everything at once; start at the highest level, the level you're actually going to be working at most of the time. There'll be time to peek behind the curtain and learn the details of what's going on at lower levels later - and good programmers absolutely should do this. But I don't think people should start off without frameworks any more than they should start off in assembly.
> Also your CV probably sais Software Engineer and not Hardware designer. So I'd say hadware bugs are out of scope.
If that's what counts then my first job title was Java Developer :P
Use an environment that matches what you want to learn, and one that promotes good development practices. But given how much modern programming is framework-based, that likely means using a framework.
These projects allowed me the opportunity to work on smaller subsets of problems than implementing a full-on CMS would have exposed me to as well as exposing me to many, many patterns I never would have encountered if I had to invent my own.
It was much more instructive to me to have to fix a small bug or implement a small feature that it would have been if I had to implement everything from the bottom up. I was also frustrated by many things in WP, but I believe that I could have been just as frustrated at any number of things in any system: that's just part of learning.
That said, I have always really enjoyed programming, and so the bad patterns and poor design choices that I see in wordPress weren't a big limitation on my learning, and I have since worked on a lot of of different systems, many of which have added new idioms to how I express myself in code.
I can see how people I know and work with don't progress very far in their programming skills because they can rely on frameworks or other code that they don't understand. But I have learned a whole lot by copying other folks' idioms.
I'm currently working on a series of articles (maybe a small ebook, unsure yet) on developing applications in PHP -- sans frameworks like Zend, Laravel, Symfony. All those force a way of thinking that is very powerful if you're a good software engineer and web developer, but allow for a novice to build things while not learning the "right" way to do it.
So, PHP 5.5, Composer, and Packagist, combined with the PSR4 mean that a complete novice, and a hardened web dev, can take small composable libraries and create powerful reusable applications and libraries themselves, with best practices and great code. It's the anathema to PHPs historical shittyness. It's how I've been building my current startup. It's the future of PHP web dev, side by side with those big frameworks for when you need the big guns and a large team, etc.
It's made developing in PHP awesome, and I never thought I'd say that. Now the issue is in educating others to give it a try :)
Frameworks and libraries are two different points along a continuous line (not even endpoints). Not black and white, just different shades of gray.
I haven't looked at Zend in a long time, but the difference between Zend 1.x and a pile of libraries was marginal, yet Zend was labelled an "enterprise grade" framework.
(Not dissing Zend, this was a deliberate design approach.)
We can fix this, though. The best part is, the skills learnt and code written while doing so are completely transferable and directly usable by the big frameworks too! So, it's a win-win :)
For what it's worth though, yeah I totally agree with you. It's not an either-or situation.
Its actually something we've been doing for an API at work. The main reason was so we can easily port legacy code [read hundreds of includes/ and that type of fluff] into a PSR compliant system pretty much on the fly without having to jump through all the hoops of a framework - all while still having access to everything in packagist.
This guy's article reminds me of the guy who did that "Programming is Terrible" talk on Youtube, where he was describing an anecdote about the fellow who decided he'd build everything from scratch because he'd understand it better. That's kind of dumb - programmers have spent the better part of the last 50 years solving hard problems to get to the point where we are now. Why do we have to rehash their hard problems when there are still tons more hard problems we could be applying ourselves to?
I started out trying to learn and build using Django but it degenerated into exactly what the author claimed about automagic frameworks.
I switched to Flask/SqlAlchemy and was able to grasp MVC much quicker. It was more work to do some stuff, but getting a little lower than Django helped me learn what was happening in routing/views/controllers/etc. Using SqlAlchemy meant I could stick within my realm of knowledge with respect to Python without all the fancy stuff within Django.
>getting a little lower than Django helped me learn what was happening in routing/views/controllers/etc.
Definitely. Armin nailed the degree of abstraction necessary to make MVC pretty easy while giving you some indirect insight about what's really going on.
user = User.objects.create_user('name', 'email', 'pwd')
user.save()
So my advice overall is to learn the core by writing stuff from scratch as much as possible and really understand how things work. Then, pick a framework and do the same. So even though you will write crappy bad quality code, you don't need to release that in production. Win-Win here.
Even if you throw out of the ORM and go with writing SQL queries by hand, do we need to understand how the database will parse the SQL, how it stores the data on disk and how it executes the query?
The main reason to understand SQL when building a web application is really that the ORM doesn't provide a full abstraction due to the differences between the relational model and the OO model. If we could solve that then we could happily never use SQL again.
Yes, I've thought that, too, but it turns out that, beyond a certain relatively low level of complexity, that's a great way to produce SQL queries which take a half hour to complete.
> but helpful to understand how an ORM converts to a SQL query.
Certainly, because there's a degree of complexity beyond which your ORM's going to start getting pathological and you're going to have to fall back on querying the database directly, which see point one.
The presumption here is that everyone wants to be a great developer; this mostly isn't true, most developers simply want to complete a task and care only about how quickly they can do it.
Good development practices and project structure, generally good code examples, fewer bugs and tested code and re-using code are all very important things in any development, and frameworks all help with that.
If you are starting some of your very first web development, I'd encourage you to play around with some low level Node.js HTTP stuff, or even write a web server in another language, to get a feel for how the request/response model, HTTP header/body sections, etc, all work.
But after that basic introduction, dive into a framework and learn it well. The projects I know that were developed 'without a framework' are more often than not poorly structured, poorly documented, and difficult for new developers to learn their way around, precisely for the reasons given at the end of the article.
If you evaluate frameworks for your use-case, and realise there isn't one that suits you well, then by all means, go and make your own, but don't just make your application - make a framework for your application, and then use the framework, it will result it much better structure and better software engineering practice.
You are right! I didn't mean that I would like beginners to code 'Production Code' without any frameworks and do all their stuff themselves.
Especially if it's a big application, this would mostly result in 'unmaintainable spaghetti code'.
What I meant by beginners: "People learning to code", not "Beginners doing a production code without any help from more experienced programmers" ;)
> The projects I know that were developed 'without a framework' are more often than not poorly structured...
So why not teach developers how to structure code properly? Teach them about coupling, cohesion, benefits of immutability and layering. Make them feel when modules should be broken up, and how they should do it. Expose them to different architecture patterns used in larger applications. This allows developers to make better decisions about code, both as they write and refactor it.
It seems that so much of modern day pop-programmer culture is what framework they should know, rather than how to build/choose the correct framework for their needs. Perhaps this is yet another triumph of shiny over time-tested wisdom.
People have been thinking for a long time about how to build large applications that aren't horrible to maintain. Don't ignore it because kids on the Internet are excited about ultracool.js.
And is it really nessecary to write all the update / insert queries for my dayabase?
It's about an industry that insists on re-learning ancient lessons over and over again. It believes things like "we don't need design patterns, we have Ruby!" and "we don't need to choose architecture, we have Rails!" It constitutes a sort of intellectual deference to That Which Has Been Made And Agreed Upon to Be Good Enough. Meanwhile, the foundational knowledge becomes something that is ignored or looked down upon, because it's not shiny.
I want developers who are not afraid to rip up parts of the stack and make them better if they come up deficient. This requires deep knowledge and ignoring the rabble of people who insist that you "just use X." How do you think LLVM started?
1. They remove functionality (developmental power) from their underlying technologies in exchange for making a few types of features slightly easier. 2. Their abstractions are leaky - they cannot be used to their fullest potential without also understanding their underlying technologies. 3. They add overhead to a project both in optimal performance and technical debt.
So why does everyone flock to frameworks? My best guess is because of a failure to have resources available to teach web development beginners the basic technologies and how they work together to create simple, clear, powerful network applications.
When you finally have a single product that is worth optimizing, then you can dive into the deep details. But mostly you have dozens of throw-away products before you get the lucky one.
We don't ask doctors, plumbers or auto mechanics to "just get the job done". We ask them to do it right.
For example how many doctors know how a CAT scan works? They trust that the machine is doing its job correctly, and focus their expertise on interpreting the results. And a lot of auto mechanic work today involves reading codes off the car's electronics and looking up the fix. Do you really think every auto mechanic understands the full hardware/software stack of all the cars they fix?
Abstraction is not a problem; it is a productivity multiplier. But only if it is done right.
The biggest problem with software abstractions these days is that their quality is so low that they can't be trusted. Imagine if CAT scan machines were built to the quality standards of Django or RoR. People wouldn't stand for it.
The difference between a repairman and a mechanic, in classical terms, is that a repairmen fixes things with tools someone else made, and a mechanic can make tools. They must understand how machines and tools work, and a doctor (of medicine, a physician) is supposed to have general knowledge of the functions of the body, of the diseases and ailments that effects, and the technology currently available to try to help.
In the Django example up thread (or typical newbie Rails app, etc) an "app" is quickly spawned that the dev may not how to secure, update to new requirements, internationalize, or make perform fast enough to useful.
Somewhere in these mixed metaphors is a discussion about the difference between programming, computer science, and software engineering ..
I don't know. Do you? Or are you just assuming that most of them don't?
We're building software here, not saving lives or building something physical. Software is virtual, it can be refactored and fixed numerous times until the desired performance and bugs are all worked out.
Doctors and anyone building something physical don't have this luxury. You can't tell a doctor to "refactored" a heart transplant so they have to get it right the first time.
Frameworks are great for getting stuff done, which is exactly why I do recommend them to beginners. Beginners are learning code/a new language for a reason. Usually that reason is to get stuff done. They don't need to know how the insides of these frameworks work to do that.
As these beginners progress, as they start getting curious or need, for example, to know why getting your ORM to count the records in the database is faster than getting your ORM to pull all the records and counting the number of records in the array, this is when these beginners need to think about re-inventing the wheel.
I'm a firm believer in 'you should re-invent the wheel once and once only', but never ever ever use the wheel you just built. That way you should understand what a framework or any lower layer of abstraction is doing for you. Whomever wrote the framework will also most likely have a better solution that has tested more edge cases than you did, you have less code to maintain (yay less teechnical debt!) which is why you never use your own wheel. Re-invent the wheel so you have a better understanding but that's it.
If we keep expecting beginners to understand every level of abstraction that we have created for them they'd be writing a web-server in assembler, not solving any of the problems they have, become disinterested and giving up or never ever ever shipping any of their code ever.
Practically, it doesn't matter if you start with a framework: work with stuff long enough and you will either have to figure out the internals of your tools or you won't. If you do, the you learn, if you don't, it wasn't relevant anyhow.
Web frameworks consist of a lot of things besides pure programming. And these beginners barely understand programming...hell, they barely understand basic file systems that don't involve folder icons. And so all of this stuff...databases, MVC design, object-oriented concepts, dev ops, debugging...oh, and the entire world of front-end design...is thrown at them in such a way that there is no way they can decouple what are mostly orthogonal concepts.
It'd be fine if these beginners couldn't write a optimized SQL query in place of ActiveRecord...that'd be perfectly fine to work with. But I'm talking about a much deeper level of confusion. I once worked with a Rails self-learner and he wanted to throw up a single web page, with a sortable table that had a trivial number of rows (i.e. the data could all fit on one page without being cumbersome)...he could not, for the life of him, figure out how to do that in such a way that didn't involve creating a new Rails project and deploying it to Heroku.
It's not that I'm against teaching frameworks...it's just that for beginners, it seems like a very convoluted way to do so, and one that could be very unproductive in the end.
On the other hand, you don't need to use all these things.
When I learned Java, we were told don't worrry about "public static void main" just now, it will become clearer later. It did. I don't remeber them ever explianing that line, but explaining all the parts of it at different times.
Likewise, using Django or Rails, I am sure you can have a basic development server set up, and just let it call your code much like pressing a button would have done in visual basic in the past. You don't need to overcomplicate things.
> The problem is to always tell beginners (or let them think this is a good idea) to start with a framework / cms because “you shouldn’t reinvent the wheel”.
Learning a CMS and Framework are not same. Learning CMS is user stuff, but learning a framework is a developer stuff.
> When they should use a framework:
> When they need security features (login, session management, etc.). It’s often a bad idea for beginners (and even experts) to write code related to those areas and use it in a live application. But you still should know the language basics.
You can't provide security by learning a language basics. You should read books, articles about web security (independent to the language)
> 1. Customization can be difficult in complex code.
> Take for instance Liferay. Liferay is a big web development platform. If you want to start a membership site...
Your example is not a framework.
> 2. Debugging is usually more difficult when using frameworks.
> Frameworks are good at throwing unreadable meaningless errors.
Are you sure you have used a framework before ?
tl;dr Framework and CMS (or portal) are not same stuff.
Would you tell her to learn HTML, CSS, PHP and Javascript from scratch or would you recommend Wordpress?
If my Mom wanted her own website because she wanted a place to share her writing, or some photos, I'd suggest that she install Wordpress, find a free theme and work with it until her site is set up how she wants it (or just give up and invite me over to help).
On the other hand, if my Mom wanted to build a website because she was interested in learning how to develop web applications, I'd likely have to start her off with HTML and CSS (then introduce Javascript once HTML/CSS sink in). When my Mom really wants to learn something (especially something technical), she needs to see how all the component parts are actually built so that she has a full understanding. A full understanding of the smallest components makes it easier for her to understand bigger concepts. When it comes to web development, understanding how a document full of angle brackets can render as a web page would make everything else fall into place.
To complicate things, if my Dad wanted to learn how to develop web applications, making him start at HTML/CSS/Javascript would almost guarantee failure. My Dad operates best when you throw him in the deep end. He would do best if he downloaded an open source Rails project, figured out why it does what it does, and then modified it to do something different.
I struggle when it comes to the why. My Dad is extremely mechanical and he was a police officer. Perhaps working with a police force caused him to look at events as a byproduct of an entire system? Or, maybe this is straight up nature and his brain is just built differently? Or, maybe his parents did something in his first few years of life?
Take Bootstrap for example. A beginner starting with frameworks might think: "Cool, if I add class this and class that, I get styling and layout". When in reality you need to know that class name means nothing, it is the CSS in the back that is doing everything.
Start basic, otherwise you will never know how exactly the frameworks work. Plus, you will never be able to write your own frameworks/libraries/code since you didn't learn it!
PS: The answer "just use a framework" = add a whole new file to use 10 lines of code and bloat your website with extra files and slow load times.
I made A Soft Murmur [1] without ever having written a line of code before. (There are literally comments in the code like "oh, so that's what an array is".) I originally planned to use jplayer [2] for the audio because I believed it had very good cross platform support. But after I dug down deeper (and wasted quite a bit of time), I discovered that for what I actually wanted to do (control multiple audio streams from the same set of controls), jplayer had poor cross-platform support: for example, on iOS every audio stream has to be started by a user action, so it's not possible to play multiple streams by tapping once. There were a number of problems like this. The issue was, because I was so much of a novice, I didn't know how to assess jplayer's limitations to the degree that a more experienced developer would have before choosing to use it.
Similarly, I used jQuery Mobile [3] to create mobile-friendly sliders for audio volume control, but then spent a long time working out to disable all the other things jQuery Mobile does by default.
I ended up using raw HTML5 audio elements, and I kind of wish that I'd implemented the sliders and the rest of the controls in vanilla js rather than jQuery and jQuery Mobile. I would have learned more. As it is, my understanding of what jQuery actually offers is hazy because I'm unsure about how much I could have done without it.
My takeaway is that until you know the basics of a language, you can't tell what tools you need or the limitations the tools you chose will impose on you. I think it's far better for beginners to start using the absolute minimum set of tools (in my case, hand-coded HTML, CSS and JS) and then add more complex tools when one encounters a problem one can't solve with what one already has, or to optimize something one already knows how to do.
Do we really want to force people into learning how to wrangle strings from an <input> field into valid dates or how to manually CSRF proof their forms to do basic web-dev?
The difference between Web development and desktop development is the HTTP protocol which is responsible for security issues and other behaviors (sessions, cookies) that you need to understand if you want to build solid applications.
Just the difference between client and server side validation is often not understood by beginners, which is a big problem.
If somebody rely on the framework to do all the job, he's going to have security issues in his applications and other weird behaviors (double submit when refreshing a page, etc.)
The point of a good framework should be to pave over the cracks like double submit bugs, since they are not really things than anybody should be wasting brain cells on.
2. Writing a web application without a framework is a good learning experience. For anything but small-scale local learning experiences, the risks and costs of not working with a framework are significant. [It is probable that I, with my ego, would "do it wrong" and that a community of developers has arrived at a far superior solution.]
One of the best explanations for what advantages a framework offers over basically just writing your own framework I've found is in the Symfony2 Book: "Symfony2 versus Flat PHP". [1]
[1] http://symfony.com/doc/current/book/from_flat_php_to_symfony...
This is why frameworks aren't great for beginners to learn on. So many decisions are taken a way from them they don't gain an understanding of what is really happening.
jQuery is a DSL which makes it a bit tougher to categorize. You can use it as a library, but in general it will end up taking over more of your application structure and flow control if you use it well.
Modern frameworks have a level of built-in mitigation for the common stuff like XSS, SQLi and CSRF and so on.
The problem is that most developers don't even understand all the things their framework is doing for them.
Non-trivial PHP code written without the aid of a framework is a great morale booster for penetration testers.