Hacker News Reader

You're right, but most replies to you have overlooked the customer:

Customers, paying customers, are accepting Javascript apps. Java was never good enough for that to happen except internal to large organizations where aesthetics and user experience didn't matter.

There may be a zillion crappy, similar overlapping Javascript libraries - and to my background it seems like a massive waste of time - but the fact remains that people, everyday people like my mom, sister, uncle and aunt are all spending real money for services accessed through Javascript. It is good enough.

Should I tell them: "You know that website you like so much, e.g. reddit- it would be so much better if it was a native app on your PC written in C++, so, like, don't use it. Demand and wait for someone to re-write it in a proper language with libraries that have a proper pedigree. Then you can run it on a 300 MHz computer instead of 3 GHz."

That'll never work! The javascript/PHP/whatever else stack that implements reddit is absolutely adequate for them to browse cat pictures and interact with their magic tricks communities.

Now, diehards like me cringe because of the waste of CPU, but guess what - CPU power is no longer a scarcity. There are only philosophical and environmental reasons to wring our arms about using less CPU power. And possibly these concerns are non-sensical. Are we wasting the world's resources making fast CPUs so people can use inefficient sw libraries? Who knows. But either way, the point remains:

Javascript's customers and users, and the services they demand and pay real money for all support Javascript and nothing other than a fanatic religion will convince them to change their ways.

It makes me mental, but I've even been learning Javascript over the last few years. It's about the same speed (and variable scope) as the BASIC I used on the Apple 2 in 1984, but so what, I'm paying the rent with it.

Customers are accepting web apps. They don't care if the apps are written in Flash, js, BrainFuck, Haskell, or morse code converted into asm.js via a transcompiler written in APL.

But development is not a democracy, and the fact that n thousand projects use a certain stack doesn't mean the stack is fit for purpose.

Take the security horror that is Wordpress. It's popular because it's easy to use, but from a professional point of view it's an infosec plague and server killer.

The quibbles about js have nothing to do with its popularity. They're more about the overall lack of professionalism in software development.

The underlying issue is scope. We don't have a cathedral, or a bazaar. What we have is exactly one (1) planetary network, made of a loose affiliation of barely cooperating nodes and applications.

The original Internet infrastructure worked because the RFC process meant that the core features were designed and refined by peers. Hobby coding in the bazaar removes all elements of peer review.

But the alternative isn't a UNIX-style cathedral, it's a revised set of standards leading - eventually, I guess - to a planet-scale operating system and shared library framework that takes away all the cruft by making it unnecessary.

None of the current stacks are the right way to make that happen.

So the issue isn't js - it's more that web stacks have become de facto operating systems for industrial computing projects without any of the rigour, peer review, or oversight of a well-designed industrial gold standard OS.

And it seems that no one - or at least no one that Tim O'Reilly's authors talk to - is even thinking about how to create an OS of that quality.

Yeah. There's a great recent post by Jeffrey Ventrella on "Slow Programming"[1] highlighting the problem that comes from the "fallacy that all engineers are fungible"[2] and throwing a bunch of programmers immediately into an iterative development cycle without proper up-front planning.

I think it's important to note that big reformations, like Agile replacing Waterfall, are reactionary; by necessity they tend to be overly so; to promote change they demonize the old way. Unfortunately there's usually a core of good ideas that get thrown out as well.

I don't think the Waterfall methodology works for most products, but it made up-front planning and careful consideration part of the design process that's missing in a lot of projects today. I think a balanced approach is needed; just enough up front planning to provide a strong, clear vision to the team, consideration for the tools and processes that will be used to get there, and someone with the vision and technical knowledge to lead it. These are some of the points I like to hammer out before launching into a project:

  * Design - what is the product supposed to look like?
  * Testability - how do you intend to test your code?
  * Documentation - how will the code, product be documented?
  * Tooling - what's your preferred dev tool chain?
  * Code architecture - What's your design philosophy?
  * Prototyping - straight to product or prototype first?
  * Delegation - how to structure for future delegation?
  * Security - how to audit the security of your app?

These points often seem to get left as an afterthought, but these broad strokes have worked pretty well for me and still allow for iterative development and functional refinement.

  1. http://ventrellathing.wordpress.com/2013/06/18/the-case-for-slow-programming/
  2. http://www.datamation.com/career/article.php/3757311/The-Myth-of-the-Interchangeable-Programmer-Can146t-We-Just-Offshore-Him.htm
  3. http://pragdave.me/blog/2014/03/04/time-to-kill-agile/

Thank you for this comment! I would also add something like "System architecture" to your points, because that often affects how things are developed.

I'm curious about the details. How, exactly, do you hammer those points out? One big meeting? Multiple meetings? A week or so "workshop"? How detailed are the answers to, eg. the Design, Documentation, and Architecture questions? Wireframes? Mock-ups? Architecture diagrams? Who do you invite to the planning session(s)? Do you re-evaluate things periodically using the same points? Do you run major new features through the same process, or just slot them into ongoing iterative development?

I'd be very interested in any elaboration you're willing to go into.

Sure. First off, I should say that as the lead developer I took ownership of these decisions. Design by committee is certain death. I didn't ask for permission; I sought input from various stakeholders (often one-on-one) and made it clear that while the final implementation might differ from their suggested approach, I would be proposing solutions that took everyone's basic concerns into consideration. After gathering feedback, it would be several weeks before a new iteration was put forward for review. Stakeholders could use and interact with it. More feedback would be solicited and then the next iteration would begin, again taking the feedback into consideration but not making any promises. I had managers who were furious with me for not kowtowing to their demands, but I'd produced a very successful prototype (featured at Blackhat 2013 and Mobile World Congress) and they grudgingly had to admit that the results were better than anything the previous design-by-committee approach had produced.

  * Design - what is the product supposed to look like?

Good design is not part of the DNA of large enterprises. This makes it low hanging fruit for upstart competitors to come in and impress clients with the usability of their "slick" design. Because purchasing decisions are increasingly being influenced by the people who will actually use the product, this gives them a tremendous competitive advantage. In large software teams, its typical to begin design with the data model, and progress through successive layers of code until the data finally goes splat on the users screen in a great meaningless pile. Smaller companies, especially ones with a legacy in web application design, have figured out the value of designing the User Experience first, and then developing the lower layers. This is not without tradeoffs; a new use case that you never envisioned can force changes to the data model that break compatibility. I found that working from both ends of the stack (prototyping the UX while simultaneously considering the data model and future use cases) was beneficial. Anyway, it's amazing how much software actually gets built without anyone even thinking about what it should actually look like to the end user, until the very end. When managers come from the era of punch cards and PDP-11, well, this is just how it is. You have to fight that mentality by pointing out the contracts they lost because the product looks like crap.

  * Testability - how do you intend to test your code?

It amazes me in this day and age that there are still product managers who believe automated testing is a waste of time. I set out a goal of having 100% test coverage, which I maintained for quite a while. Gradually the test coverage declined as new use cases needed to be supported. It simply wasn't possible to cover off every error case or alternative configuration of the product, but I still keep coverage between 80 - 90%, with most of the uncovered code being error handling corner cases that are not supposed to happen in production, or code that exists purely for running in a non-production environment. Incidentally I use Mocha and jscoverage for this, but there are lots of options. The important thing is to choose one. Tests are triggered from npm, which is in turn triggered from make, which is in turn triggered from Jenkins every time code is submitted to our repository.

  * Documentation - how will the code, product be documented?

Initially I set out with a goal of having the project be "self documenting". I settled on having an API that roughly follows HATEOAS principles - the idea being that given the root URL, a user could easily discover and navigate their way through the rest of the API with a web browser. I then used tools like RESTClient and POSTman along with Selenium IDE to automate some REST API developer tests. The output of these could be cut and pasted into an OpenOffice document that I submit to our code repo along with a PDF version. This isn't as fully automated as I would like, but I've got an excellent Install Guide, Developers Guide and User's Guide that can be easily updated and are readily available. They were created early in the project, in tandem with development. Tasks were not considered complete until the documentation had also been updated. QA helped ensure this.

  * Tooling - what's your preferred dev tool chain?

This is purely a personal preference, but I use Sublime for most of my JS related development. Outside of that, I'm a dedicated vim user. I dislike big IDE's because I feel people and projects become dependent on them and they foster unnecessary bloat and complexity because they don't require a developer to hold the whole design in their head. An ongoing problem is poor quality code getting into the repo and not being flagged until a bug surfaces or a build fails. I wanted to move that feedback much further forward in the development process and make it immediate, to catch problems before they ever got submitted. For that, I wrote a sublime plugin for jslint (Sublime-JSLint) with a lint-on-save capability. Any time a developer saves a file, they get immediate feedback on whether it passes the linter. This gets repeated on the build server, and lint warnings will literally break the build. As a result, all of our JS code is very consistent.

  * Code architecture - What's your design philosophy?

In addition to web-first, mobile-first, and API-first approaches, you might have strong ideas about MVC architecture, REST API best practices, abstraction oriented architecture, OO, Functional programming, etc. Being trained first as an artist, my preferences are pragmatic. I want strategies that will get me from concept to product as quickly and easily as possible. I've found an API-first approach with an abstraction-oriented architecture to work well. I began blogging about REST API best practices as a way to gather information on the topic. I also wrote a couple of pretty bad "REST" APIs before I started getting better at it. Now there are frameworks like Sails and LoopBack that will guide your API design.

  * Prototyping - straight to product or prototype first?

I do believe in prototyping first, but you should be willing to view the prototype as throw-away work. Prototypes are often thrown together quickly and incur tons of "technical debt" almost intentionally. Trying to productize a prototype can be more difficult that simply starting from scratch. A lot of times managers will wonder what the hell is taking so long... they saw a working prototype and they've already got customers lined up to take it, so where's the product? They don't realize the prototype is all smoke and mirrors running off a flat file and your personal Google API key. You may need to explain that the prototype is not suitable for productization, and why. On the other hand, if you do want to go from prototype into production, you need to be careful when you build the prototype not to incur the technical debt of a quick-n-dirty mockup, and figure out the path for getting from prototype to production. Things like replacing the data access layer with something that talks to a real database, and having a proper configuration file, etc.

  * Delegation - how to structure for future delegation?

API first design is helpful because it provides a sort of contract by which various components can interconnect. Although I initially wrote the API, server, mobile app, and web interface to the product, the API allows any of the client applications to be forked off into a standalone project. Eventually we did this with the mobile app. This is where the documentation started to prove its worth. The point is to think about how the project and the team is likely to grow, and which parts could be handed off to another team, and how to define clear boundaries with a clean interface between them.

  * Security - how to audit the security of your app?

It's often just assumed that applications will be secure, and then of course they aren't. Session hijacking, SQL injection and cross-site scripting vulnerabilities have been present in most of the web-based user interfaces of projects I've reviewed. Enterprise sw developers tend to be unaware of what these attack vectors are and how they work. There's no substitute for being knowledgeable about web security but there are tools like Nessus and Skipfish and various penetration testing tools that can help. I guarantee they will turn up security issues you never knew existed. If you can automate the process, great. If not, making a security audit part of your QA process is important.

I don't think it is the culture of 'speed' necessarily, but rather one of 'feedback loops'. With javascript in the browser you can see and experience code-reload after an F5†. There is no compilation time; no real detriment to importing another non-used library if it's not compiled into your code.

Opposed to using a JVM language, where everything is statically compiled into a single container, including another library (or upversioning one) will impact compilation times and oftentimes trigger a complete re-build, often taking minutes before you can see the output of your code.

Minutes versus seconds when dealing with code correctness really helps, and is what is needed most when trying to 'hack it together'.

† Note: This is the same with test suites: karma tests are often instantaneous and spock/junit tests sometimes take 30-40 seconds before they start to run, depending on environment.

>With javascript in the browser you can see and experience code-reload after an F5†. There is no compilation time; no real detriment to importing another non-used library if it's not compiled into your code.

While that may be true, typical JavaScript projects require a compilation step. Plain JavaScript is starting to get somewhat rare, and projects typically use a compile-to-js language like CoffeeScript or TypeScript, or perhaps ES6-to-ES5 compile step. And then you need to minify that code since it's not recommended to run a non-minified "debug" version during development. And if you're project is any complex you're probably also compiling a single .css file perhaps from .less, compiling .html from .jade and packaging it all up in a different public folder.

The point is that typical non-trivial web applications need a build step.

Why would you minify your code during development? Isn't minified code for production?

Because minified code functions differently from non-minified code. It helps to detect bugs caused by minification. There's also a general principle which states that the application shouldn't be developed in a "debug" mode, because the debug mode might cause the application to execute differently from production.

As I understand it, minification is basically removing whitespace. Why would that make the code behave differently?

Minification can also involve changing variable names; sometimes there are dependencies on names that are unknown to the minifier. Also, the removal of whitespace can cause lines to combine in interesting ways if there aren't semicolons between them and if the minifier doesn't automatically add it.

http://stackoverflow.com/search?q=minification+bug

JavaScript minifaction is much more than that. It renames function parameters, does all kinds of fancy tricks to refactor code and so on. AngularJS is one example where minifacation breaks the code if you don't use the "special" minification-safe pattern.

On the off chance that whatever minification library you're using broke something somehow.

Was about to write the same thing. None of the complaints in the article were about Javascript. It was more about the current habits that have evolved around it. Granted, I think the PHP world has had a good 10 years of dealing with limitless code libraries, it just wasn't as well organized with bower, npm, grunt...etc.

Shouldn't the response be to look for managers with programming background and (at least some) understanding of the process?

You don't have to be Michael Schumacher to become responsible of an F1 team. But you have to understand basic staff and hire proficient personnel to do the rest.

I think it's just a matter of recognizing what software development is, and what it isn't. Somehow large, well-managed corporations are able to deal with creative teams in marketing, in design. But when it comes to software development it just seems to me we've been stuck into the wrong bucket. We sit at keyboards, we type, and make computers do things, therefore we are 'IT workers.'