The Joel Test for 2017 | Hacker News Reader

The Joel Test for 2017

myers.io

43 pointssigna1172 comments

>Do you use source control?

>Do new candidates write code during their interview?

The author suggests removing these two rules because they're ubiquitous, that everyone does them.

I hate to inform the author, but he's sadly mistaken. There are plenty of companies still out there that aren't doing these things, especially with interviews. Also, I think the ubiquity of them is a reason to include them, because it helps draw attention to those companies that are deficient in some way.

Yes! If you interview for a company and they don't use source control that is a giant red flag.

I generally think the same these days, but it made me recall 10-15 years ago when I was consulting and considered a company having no source control a place where I could provide immediate value. Now it just scares me right out the door.

It's funny how quickly that's gone from, "Oooo I can help you!" to "you guys are helpless."

The value is also depending on them understanding why it is useful. And if they do, would they not use it already?

They are in a mess, they hire you as a consultant, you explain VCS, they get it, everybody wins.

They didn't have to understand in advance. They just had to be smart enough to know they needed help.

I was literally going to say the same thing. The reason these questions are in there is to whittle out people immediately. And yes there are developers and companies out there who do exist that do not use any VCS.

Probably the better thing to ask a candidate as a follow-up question is why is version control important.

I believe this question is as valid today as it was in 2000. From the beginning of my career (in 1993) everyone company I have worked for has used source code control. Not to do so was even in 1993 a huge red flag. Sadly there were companies in 1993 and also today that don't.

The tools have got better in some ways and also worse (e.g. complexity of git). But before subversion there was cvs, rcs and visual source safe.

I still dont see this as a red flag. Even if source control isn't used yet, you can add it once you start - it isn't expensive or complex any more. That's what makes you a valuable hire.

It should be a red flag. It can easily be that everyone that suggested it got shot down for 'disrupting the process', or that they couldn't find it in the budget or schedule to implement it.

This.

One of my most frequent freelance clients was a web developer who farmed some modules out to me to be implemented. From what I could tell, he used FTP for deploying changes, and did not use any kind of version control. I kept him on as a client because I needed the work, but this lack of proper tooling caused numerous headaches. In retrospect, it probably would have benefitted both of us to condition any work for him on his adopting source control for that project.

Replacing one-step builds with CI isn't a good change. As someone who's having the pleasure of getting test environments up and working stably for several different large startups let me just say: having a one-step process for doing this helps immensely. Forget CI: if you can't get your project up and running, you can't confidently make any changes in it to begin with. I think that Spolsky rule is more relevant now than it was when he wrote it.

Further: I don't think Spolsky missed CI so much as that he predicted it. So, maybe, if you wanted to 2017-ize the Joel Test, you could say you should have hourly instead of daily builds, or something like that.

Spolsky was, I think, pretty far off the mark about hiring. I'm fine with keeping a bullet in the list about carefully-designed hiring processes, though; striking that, because "everyone does this right", would be lunacy. Virtually nobody hires right.

The Hallway Usability Testing thing always seemed to strike me as Spolsky promoting an idea he'd been talking about with CityDesk, putting it on par with "having a bug tracker" and "using source control". Losing the rule altogether would be fine with me. Suggesting that every team have dedicated UI/UX people is silly: many good teams don't, and even the teams that I work with that do don't engage those people on every (or even most) UI/UX changes.

Even doing daily builds sometimes has little value. There can be days in my office when there is no release going on, but everyone is working on some feature or other, but there's only one testing/staging environment.

The one step build is far more impactful.

Hallway usability testing I interpret as dogfooding if possible. Some companies have the entire company dogfood the entire app every day, and it creates a lot of 'hallway usability testing' on its own.

He proposes removing the rule about "quiet working conditions" and even says "There is no right or wrong answer for which is correct." On the contrary, there is a lot of scientific evidence that quiet enhances concentration and productivity.

This is my main point of contention with the article as well.

It's more than just quiet. Having a totally open work space creates a lot of visual distractions as well. People walking around you all the time is far more distracting than the noise in my experience.

I believe you are speaking for yourself, now. On the contrary, I find open space much better. I like to see people around me. Talk to them. Exchange ideas. And if I want to keep to myself, I just put on the headphones. That also contradicts the "silence" argument. I'd gone mad in a silent environment. I need a music. It's all really individual and one definite answer does not exist.

Re-reading the original Joel test, it is clear that Joel means quiet in the sense of free from interruptions.

There has been an ongoing debate on the benefits of having developers in separate offices for many years versus have developers in an open plan office. There are arguments in favour of both sides. Joel, however, has plumped for having developers in their own offices.

So in this case I believe that Joel's (opinionated) views remain as valid now as they did when he wrote them. After all, even though there have been improvements in noise cancelling headsets (and office etiquette now allows wearing them), it doesn't stop interruptions from other developers.

There is no "ongoing debate." "Open Workspaces" are the cheapest possible way to house workers and are almost the anthesis of what scientific studies have recommended.

Have you ever considered that the management types arguing in favor or open workspaces-- usually work in an office themselves? The last VP I worked for, bought the engineering team a fucking table to work at -- stuck it at the end of a hallway, and literally spent hundreds of thousands of dollars on his office.

The engineering team heard the message pretty loud and clear.

Let me start with an assumption that is often accepted to be true about developers. Many of us are thought to have ADHD or be on the autistic spectrum.

People with high functioning autism, especially those that fit what has been known as the Asperger's subset, are very prone to oversensitivity to external stimulation. In people with this oversensitivity, sounds seem louder, lights seem brighter, and everything is more distracting and more nerve-wracking than for the average person.

I think ADHD and distraction is mostly self-explanatory. In addition, it can take even longer after a distraction to get back into the deep concentration needed for complex tasks. However, these folks often if not distracted can stay in that deep concentration zone a very long time.

So when research shows (and it does) that a quiet workplace is more conducive to work that needs long periods of concentration and we're as a community pretty certain that we have a large portion of our group who are more easily distracted and has higher costs for being distracted, how is there any argument against giving a low-distraction workspace?

square footage in trendy downtown office spaces is expensive ;)

That's certainly a factor in contention with a less open workspace. It's not actually an argument against it, but something that should be balanced against it as best is possible.

So are developers - and good developers even more so :)

If you have a medical diagnosis, can this fall under ADA protection, requiring your employer to provide appropriate accommodations?

I'm not a lawyer, and what is an appropriate accommodation certainly leaves room for interpretation. My understanding is that yes, either diagnosis qualifies under the ADA. How much weight that carries for any specific accommodation I have no idea.

My employer values offices. Most aren't very big, but most of the company works in offices.

Also, to favor the team dynamic and lower contention on conference rooms, offices for a team surround a conference room for that team in a "pod" architecture. You enter each team's office suite from a hallway into the conference room, and then go to the individual offices from there. You only need to book a conference room elsewhere in the building if there are enough guests from outside the team to overflow your own pod room.

It's helped a great deal over having offices right off the hallway for purposes of avoiding distraction and also for getting the team meeting and hack session space with no waiting.

> However, a company that doesn’t get at least 10 is effectively guaranteed to be a terrible place to work.

Seems a bit harsh. None of the companies I've worked for has scored 11 or 12, but the only thing that stands out in my mind as "terrible" was at one company where we had to integrate with SAP. And that company could have easily scored a 12 and still been terrible simply because that SAP interface was remarkably bad. In fact, the only ones that I would consider dealbreakers (source control and quiet working conditions) are no longer on the list.

Seems a bit high horse ish

I agree. Just because a place scores 12/12 doesn't stop the managers from making the job a living hell in other ways. On the flip side: my current job doesn't even score a 10, but it's far from 'terrible', and no one would dare leave because of deficiencies from these questions.

> Yes! If you interview for a company and they don't use source control that is a giant red flag. reply

Hallelujah!

At Cygnus in 1991 an employee named Rich Pixley tried to convince us to adopt source control (CVS was the state of the art at the time). John Gilmore and I were staunchly opposed, and I think Michael Tiemann was neutral. I don't remember what the employees thought. In any case John and I finally acquiesced in ill humor and only under the conditions of a number of absurd and nonsensical requirements (e.g. "must integrated transparently with emacs"). IIRC Rich simply ignored our bullshit and set things up and John and I realized we had been idiots.

Eventually we had to write the client/server CVS implementation because of a source control horror story. And I think as a result the FSF ended up adopting source control too.

I don't think I would work with anyone who doesn't use it any more, at least for software. On the CAD side (Cadence, Solidworks etc) the situation is still dire.

Ohh, someone who knows my pain! I deal with SolidWorks EPDM on a daily basis, and I am continually amazed by how terrible it is. I long for proper source control with branching and merging and sensible handling of local files and all the lovely things that have been figured out in the software world.

Not to mention SolidWorks keeps the absolute pathname of objects in its proprietary files (EPDM patches them up at check-out time) so it's unreasonable to try to keep them in a conventional source control system. Grrr.

I'm the original author for this post, and there are a few things I've realised since writing it (despite the fact that it was only 2 days ago).

The first is as allwein says below: Just because I made an assumption that source control is ubiquitous, it doesn't mean that it actually is, and it probably shouldn't be removed from the list.

The second is with regards to quiet working conditions. I've seen here, and on Reddit, that most people seem to vehemently disagree with me. I didn't imagine that there would be this level of disagreement. The popular opinion is that everyone should have quiet working conditions. My statement was that working conditions are tied to open office vs closed office, and it was a personal preference. So, my question to HN is: Am I wrong in thinking that this is a personal preference and that it benefits everyone to have private offices?

I think it should be revised to, "Do the office conditions satisfy YOUR needs?"

Some people like having lots of noise/people around them (all the noise blends together at some point) or they just put headphones on anyway so they don't care. Others want their own office, or could do a shared office. It's such a personal choice these days.

From what I've read, it seems that open offices reduce productivity. The only real advantages I can think of are projecting coolness and saving money on office space.

Yes, you are wrong, if we are take the test for what it's worth--that is, Joel originally defined quiet as a value, and you have to assume that that's still correct. Otherwise, Joel doesn't know what he's talking about ("Noise is sometimes good!") so just drop the test completely.

Anyway, the baffling contemporary move towards open offices only increases the importance of the rule.

I think private offices can be helpful, sometimes.

Anecdotal: At a previous job, I found a coworker quite furstrating, and he found me frustrating. The CTO told me the two of us were loud enough to be distracting the others, so I immediately tried to moderate myself… but the fact we were still aggravating each other was a contributing factor to me deciding to leave. I had enough respect for everyone else there that I was embarrassed to be noticeable in that way.

Part of the way the Joel test works is to determine if companies value developers. Companies that will pay for developers to have their own offices are likely to treat them with respect. Companies that treat offices as status symbols and then deny them to developers are likely to treat developers with contempt.

The same applies to the "Do you use the best tools money can buy?" and "Do you have testers?" questions.

You can theoretically have quiet and open offices. Think of libraries and their quietness rules.

This argument is no different when Joel first wrote the test, so to say the question is "out of date" isn't very accurate.

Furthermore, I would suggest that five years ago people were arguing about whether open plans were better. Seems to me like that argument is over, with the resolution being "nope, but they're cheaper!"

I personally often like to work in public areas. Having an office doesn't keep me from doing that, but it does let me retreat when I want to.

I am quite sure that noise level is a personal preference. I know people who use industry grade ear protectors (only few) and much more who just use headphones with loud music. Note that open office is not only about noise, though.

My answer would be - open office is the best, as long as you can have quiet time when you need to focus (headphones, protectors, whatever suits you.)

> Can you make a build in one step?

vs

> Are all builds handled automatically by a Continuous Integration server?

Are two very different indicators. And both are, least in my mind, equally important. One measure the holistic approach to source code including everything necessary to produce a functional system. Checkout, build, use.

The other is a measure of how much care is given to the overall process of software engineering. How much automation the tests receive, how much attention is given to deployment etc.

Software construction must be tied intimately with the software code itself for thousands of good reasons. Too many times have I seen a team jump onto the CI bandwagon; abandoning all notions of build scripts to let this new shiny thing perform it's magical incantations to build and package the system. A checkout alone is no longer sufficient to gain a fully functional system one can test locally, the CI becomes in an integral part of the software code but without the process the rest of the code receives (reviews, versioning, source control etc).

With the multiplication of commercial tools flaunting how easy it is to automate all this that one does not even need a programmer anymore, I fear software construction will become harder and harder to do without the help of these external systems.

So yes, do team have a single operation to build (that is construct) a workable system is still very much relevant.

And yes, do team use continuous integration to automate their test and delivery process is also important.

> know many people who like the idea of open offices. They promote more effective and direct communication between members of a team, often leading to faster and better solutions to problems

There is no world in which this is true, it's just interiorised worse conditions of working at this point.

I don't understand the reasoning for throwing out the noisy work environment question. It can be optional for sure, but so is every other question. Use your judgement, but keep it in the list for most people.

The "best tools money can buy" is too objective.

What you're really trying to get at is that you'll get a decent computer, not a hand-me-down laptop from a salesman who couldn't sell anything.

Joel was a Microsoft guy, so I'm pretty sure he was looking for Visual Studio and MSDN licenses, as well as the latest Windows instead of 95/95 (or 3.1!).

These days it isn't as big a deal, but back then until about 2010 the more expensive machine really made a huge difference. In a time where most people were still running pentium 4s, we had quad core machines to develop on. Compiling was 4x faster than the average computer sold.

These days a 3 year old computer really doesn't make that much difference.

Spent a whopping $300 each to max out the RAM (16GB) and get an SSD for each of the 3 developers we had. Improved their productivity so much. It previously took almost 2 minutes just to open a project.

16gb of RAM minimum is important. SSD hard drives as well. A big monitor or multiple ones if you're into that thing. Ergonomic mice/keyboards if you need them.

That's still around $1000 or so for a decent machine. When Joel first wrote that it would have been more like $3,000. A $3,000 machine is probably overkill for most developers now - as evidenced by the fact most sacrifice performance and get laptops willingly.

Just for yucks, I took a look at some old purchase orders for Sun engineering workstations (single-user). They used to sit on everyone's desk at a well-funded engineering outfit.

I saw an Ultra/1 (170MHz) for about $15K in 1996, and a Ultra 60 (360 MHz, 1GB RAM) for $19.8K in 1998. It's gotten a lot cheaper.

Unless it's a struggling startup, I'd still expect a good $3-4k for developer comforts, unless it's something I already owned (like my favorite ergonomic mouse).

And yet I just left a job with a 4 year old "desktop class" computer with a "spinning rust" HDD and 8GB of RAM. Something that 4 years ago would have been deployed in a call center. Did I mention I didn't have local admin as a .NET web developer? This was a company that prided itself on its "progressiveness".

Programmers don't leave companies, they leave computers!

You jest, but IMHO the state of a developer workstation (computer, desk, chair, room) speaks volumes about the organization.

You get points for taking his background into account, but don't forget, Joel was an advocate (and AFAICT, real practitioner) of other "tools" like comfy chairs, big monitors, and quiet workspaces: https://www.joelonsoftware.com/2006/09/07/a-field-guide-to-d...

So true. I once took a contract at a smaller company where everyone had newer MacBook Pros but the owner refused to let anyone get an external monitor because the laptops "already had one built-in."

You're going to do all your development work on this Pentium 4 with 1GB running Visual Basic 6 and Windows XP, with a single 19" monitor. Via RDP.

I'd love to see one tailored a little more to web development.

Also I never understood asking this question:

> Do new candidates write code during their interview?

If you're there for an interview, you're about to find out first hand.

I mean, it's possible that this question could be asked during an HR or nontechnical screen.

I didn't think of that, but whenever I asked Joel test questions of HR, they usually couldn't answer them other than if they use source control and if they have testers on staff.

I think this question is more geared towards evaluating the level of sophistication that a technical organization has prior to having an interview or even applying (which can often take significant time).

I've been at plenty of jobs where either new candidates (or existing employees) have talked a good game but when they finally show up to work, can't code their way out of a paper bag. The hope is that writing code during an interview helps prevent those situations.

The other thing this questions tends to tell you is that you'll have a technical employee likely involved in the interview, which is good for all sorts of things from finding out about general work environment to getting details on the answers of the rest of the Joel questions like "what VCS?" and "build system?".

I wonder if the Joel test is still the gold standard for most people.

Especially "Do new candidates write code during their interview?" seems to run contrary to the much decried whiteboarding (not taking sides right now, also it's not about paper vs whiteboard etc)

I still think the questions are a useful data point, just never agreed with the "10 and less is a failure", because it's just too broad and some of the points the teams I worked fulfilled 100% in spirit, but never to the letter of the test - but then you're in wishy washy gray area again. (One of my main gripes is that if you compare Excel (iirc that was his team) probably has "a single build" with dozens? of developers - so I'd rephrase it as "all your projects have a releasable HEAD/master branch" but I had people disagree on that premise.

"write code during interview" was always about real coding, using computer. No paper, no whiteboard, the best way to do it is to give the candidate access to some real code, like a company's internal project, and real task, like a nasty bug to solve, or a small new feature to implement.

I had an on-site coding interview at a place where they asked me to implement a small iOS project on my own computer doing things however I normally would. Had access to documentation, SO, could search google for whatever. They gave me a couple hours to do as much as I could - there was another dev around that could answer questions I had. Afterwards, I walked them through what I had done, answered their questions, responded to feedback, etc. This was code interviewing done right in my mind.

Whiteboards are for design, computers are for code.

Good point, I had totally forgotten we had that at a past company. Give the candidate a computer, a simple task, someone to ask and check results after 2h.

The joel test seems very applicable to a scenario where you have a large team of people working on a small number (or even just one) of big product-oriented projects.

I find that a good chunk of it doesn't apply in my job, where I'm one guy working on a large number of simple web applications.

The criteria "Do new candidates write code during their interview?" is something I consider pretty important. I would say it makes more sense if rephrased in a less polite way: will all my peers be able to write a working line of code.

Overall I have to say I'm really impressed with how the list has held up. The questions are pretty specific, and reasonably diagnostic of common issues that create spirit destroying dysfunction.

Another recent update, with a lot of common features: https://austinstartups.com/an-updated-joel-test-for-2017-315...

His list (with its diff):

    -1. Do you use source control?
    +1. Is your source control effective for the disconnected programmer?
     2. Can you make a build in one step?
    -3. Do you make daily builds?
    +3. Do you build, test, and deploy (somewhere) on every commit?
     4. Do you have a bug database?
     5. Do you fix bugs before writing new code?
    -6. Do you have an up-to-date schedule?
    +6. Does your source have a stable trunk from which releases can be cut at any time?
    -7. Do you have a spec?
    +7. Can all of a product's features be verified in one step?
    -8. Do programmers have quiet working conditions?
    +8. Do programmers have access to a door that can be closed for uninterrupted work?
    -9. Do you use the best tools money can buy?
    +9. Do programmers have the freedom to choose the best tools for the job?
    -10. Do you have testers?
    +10. Is testing the team's responsibility?
     11. Do new candidates write code during their interview?
     12. Do you do hallway usability testing?

I think this list is a bit too focused on big, dedicated software companies.

If you're spending a month or two making a throwaway bit on analysis, for example, maybe it doesn't need hallway usability testing and plugging into a CI system (although I'd argue for a makefile, which maybe does the same job).

Similarly, if you're not collaborating on a thing, maybe you don't need a schedule. You can do the job with an ordered list of tasks and a future date when you'll revisit that list to see if you need to ditch some things to stay within the budget.

And if your company only has a couple of developers to support the business, then you're probably not going to hire dedicated testers or UX specialists (how much longer before that gets replaced with some other buzzword anyway?).

And actually, all these things can be changed. If you go to a company that doesn't do something that you think is important, then make your case. If you're in a position where your case won't be listened to, that's a much worse indicator.

I wouldn't work without version control though. If I have to be responsible for some text then it's going in Git, be it code, configuration or notes.

On a different tack, I think the idea that a spec isn't useful any more because of agile is definitely a wrong one.

Your spec doesn't have to be absolutely rigorous, set in stone, or meet some standard. Writing down in detail what you intend to do and how it will meet your aims is really useful for non-trivial tasks.

I agree that the quiet working environment is still important, and the research that suggested the open office was better has largely been superseded. (Google, who started the trend, has moved away from it.) But this generally provides a much-needed update, and I'm going to put this on my "ask in the phone screen" list.

Nah.

Reply on news.ycombinator.com