Dark Matter Developers: The Unseen 99% (2012)
hanselman.com
hanselman.com
Conferences, in my experience, are mostly a waste of time. I can usually watch the keynotes online after the fact if I care to. I'm not interested in joining discussion groups and posting; like the vast majority of people, I'm content to lurk, but I'm not blind!
So yeah, I'd be very comfortable challenging a light-visible developer on pushing/using new technology and generally being aware of the landscape. I just don't care to talk about it constantly.
[had an extended comment here before but got downvoted by what I can only imagine are people offended by the word “lagging”. Reminder that downvotes on HN are for comments breaking etiquette, not to show disagreement]
Is that an accussation though, or a compliment?
"Newest technologies" are for early adopters and hype suckers. Engineers use proven technologies, not just "newer" for the sake of it.
It's part of an engineering mindset to trust proven technology, but it's also part of an engineering mindset to make sure you're staying up-to-date with critical changes in your field. Ignoring either for long enough will make you a worse engineer.
Sure, just like everybody has shat themselves one time or another, at the very least as a toddler, but perhaps also when they had to make a long run to a remote bathroom after Taco Bell. But there are really people suffering from severe diarrhoea too, and those are two distinct groups.
It's not that engineers are immune to hype. It's that the others are hype-magnets and hype-spreaders.
Engineers aren't some special class of human with special brains or pseudo-Vulcan logic that makes it impossible for us to be just as hype-spreading as anyone else.
This takes overthinking to a new level. Of course there are proven technologies - used with success for decades, known for their stability, etc. I'm pretty sure one can find arguments against any of them. But we do have a general (not unanimous) consensus, even if it hasn't been cast into stone. For example that Python is tried and mature for data science, or that C++ has proven itself in games, and Java in enterprise and application server programming, and so on.
>but who is going to prove them if not those very groups you deride
Actual domain experience from millions of developers, and observation of decade(s) of use, stability, and known upsides/downsides?
Nobody waited for hipster programmers to declare mature technologies. Their "job" is to hype new stuff. If it evolves into something mature, they go to the next shiny thing.
The dividing line between baldness and having your hair is also nonexistent, and yet both states exist and one can tell them apart if they see them.
https://plato.stanford.edu/entries/sorites-paradox/
>Those millions of developers didn’t hop on the bandwagon just because Java was sitting there, it required a lot of forward-thinking, high-skill people to get off the ground. And they were hipsters at the time.
Nah, I was there "at the time", and this is bad example. Java got traction because of marketing to corporate departments, not because of "forward-thinking, high-skill people" driving it. It was literally the first language that came with a full blown, tens-of-millions, commercial hype campaign.
In any case, the early adopters were neither "hipsters" nor more mature "engineer" types. It was the duped-by-hipsters types, but at that time enticed by marketing.
The kind of hipsters described above and in abundance 2023 (or even 2010) sense hardly existed back then (Java caught on during 1997-2000), and even when, very rarely, they did, they didn't have the means and captive audience to make their hype be adopted (people hyped things like Ada, Smalltalk, Lisp, Eiffel, and other lost causes). Back then it was mostly marketing driven hype, sold to corporate decision makers.
Modern hipster devs came about with blogs, social media, the proliferation of conferences, the rise of startups as we know them today when they recovered after the dot-com boom, and companies like Google and Facebook fighting it out for developer influence.
This definitely doesn't come off as "positive and a negative". By grouping them together (and casting them as an "other") you are devaluing them.
As for the others types that you mention, sure: the very purpose of the original comment was to deride people promoting hype.
Are we allowed to write off people's activity for any reason? Dislike certain trends people follow? Criticize some tendencies or bad habbits? If so, I chose to criticize hype-merchants and write them off (regarding that aspect of their life, I'm sure they could still be wonderful parents, or golf players, or whatever otherwise).
The question is, write them off as what? Because as we already established, it requires developers in order to make a technology “mature.” So how does it make sense to write those developers off, when they are necessary to the creation of the thing you want?
If anything, the latter leads to the opposite: tons of immature technologies, hyped to high heaven, used by tons of developers who don't know any better, and then discarded as a new shiny thing comes along. Discarded both by users, and their limited-attention-span creators.
This is in total opposition to the way I was taught engineering.
New technologies get used in products and projects all the time. There's a discipline called risk analysis where the pros, cons and risks of technological choices are weighted-in to take a decision on whether or not to incorporate them.
The more a given tech is used, the better it's understood, making it less risky to incorporate into larger products.
I’ve worked at tiny startups all the way to big tech and immediately had a few people come to mind when I read the description. A lot of them were really into sports or the arts or had a busy family, often this meant that outside of the function of our very specific role, they never really had much to say about tech or programming. That’s a really good thing though! I think it’s a drag spending all day with people whose interests are too similar to mine, it gets really stale. I’d much rather hear about how Dan’s hockey team’s doing than yet another debate about LLMs :)
Come now, if your definition is so strict then nobody is in that group. Its a very far cry from youtubers/guest speakers etc.
If you keep your real identity mostly hidden then you are in the dark matter group.
There's no upside to me talking about my current work and I might accidentally breach NDA, so I don't. I feel I can give detail-stripped war stories from previous jobs, though.
(Working on WinCE was very much being a "dark matter" developer, because there's almost no user community at all. And a good example of the NDA problem: you get some of the source to Windows CE, which you can't really discuss in public because it's copyright Microsoft)
And I never blogged or spoke a word about it, because it's absolutely forbidden both by rule and by company culture.
Therefore if you see say a Github user who hides behind their screen name, or a HN commenter who purposesly keeps their job details vague, that could be someone who doesn't want to be dark matter - but the company policies force them to hide that way.
“I'll generally take a standup mercenary who takes pride in his professionalism over an artist any day. When I hear 'artist', I think of someone who doesn't think it necessary to show up at work on time. More often than not their efforts, convinced as they are of their own genius, are geared more to giving themselves a hard-on than satisfying the great majority of dinner customers. Personally, I'd prefer to eat food that tastes good and is an honest reflection of its ingredients, than a 3-foot-tall caprice constructed from lemon grass, lawn trimmings, coconuts and red curry.”
that's such an unnecessarily disparaging remark that I kind of don't even know how to take it (or him) seriously.
They are great for recruiting.
I applied to some jobs and they didn't see anything and I got shot every time. So right now, I publish a lot more, I do this for myself, so not everybody can see me.
My hypothesis, people change the rules a lot and when things start to fall apart the rules. So right now, when I see nothing, there's a big chance that I don't going to this person serious.
> My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences. Where are these dark matter developers online?
> The Dark Matter Developer will never read this blog post because they are getting work done using tech from ten years ago and that's totally OK
They've chosen a way to express, what to _us_ on social media platforms, are the people whose voices are not heard. If you have chosen to take that in bad faith, please take a step back.
Scott is a fantastic teacher, I've learned a lot from listening to him and his podcasts. But at the same time, being in his position requires one to stay on the bleeding edge in some cases, to be able to teach newcomers emerging technologies and explain new features in Microsoft tech like ASP and C#.
When you're surrounded by tech bloggers, social media and the like all day, I expect it to shape your view of what the wider development community actually looks like - even if you're in it. As you say, I feel the vast majority of us just go to work, go home then carry on with our lives. Perhaps pop our head over the fence one in a while if something truly earthshattering is happening.
"Guys, did you know there are developers who are just developers and don't jump on the self-promotion-and-conference-speaker-train out of actually-writing-codeville at the first opportunity?! Shockingly, such people exist! They might—and you'll want to sit down for this—even not be bad at their jobs!"
Somehow using older tech is being conflated with being a lesser developer, and not being “cool”, when that’s kind of the opposite of the point of the post.
Scott Hanselman is projecting the world view conference-goers and blog-posters onto everyone else. He is obsessed with modernity, so everyone else must be using stable technology; he worries that his bleeding-edge tools are problems in need of a solution, so everyone else must be producing. In reality, any given 99% of developers is going to be using a diverse and varied mess of different technologies of varying maturity levels.
I don't think it's animosity at being called a lesser developer, but rather a baffled sense of insult at the claim that 99% of any group can be homogenous enough to teach lessons. Imagine a car enthusiast who drives a Porsche 911 writing the same article about the "dark 99%" who drive proven vehicles like a Toyota Camry (sensible passenger car) or a Mack Titan (gigantic truck used for pulling 200 tonne road trains) and are out there transporting. It's just a really weird claim to make given that the differences between his 1% and absolutely everyone else are dwarfed by the massive internal differences within that 99%.
Promoting adoption of the latest thing is the job of marketing, and his job for the past decade-or-more falls squarely into that category.
The people who go to car meetups and follow all the engineering and racing teams will be much closer to modern tech. Because it requires them to know about the latest developments in order to stay in the conversation.
What I disagree with is the % - it’s more like 90/10 instead, and most commenters here are probably in the 10% just for being on HN.
> Or they are just totally chilling and punching out at 5:01pm, but I like to think they are producing
...because everyone knows that you can't be productive during normal business hours, right?
-
"They are just not such a narcissists who need to sign up on every social media platform and constantly tweet about a shitty podcast and making shitty tiktoks to get a feeling of worth in their life."
FYI, this is waaaaay more judgemental than what you quoted...
I read the post initially as mocking the self-importance of the "1%". Also a bit judgemental, TBH, but in a friendly way. Who cares about what Scott Hanselman thinks, right? But the later parts indeed trend towards patronizing.
- occasionally reading a link from HN
- regularly reading discussions on HN
- occasionally reading other tech blogs
- never going to conferences
- never blogging about programming
- almost never coding in my free time (but doing other hobbies instead)
Am I in the 1% of visible programmers? I absolutely do not think so, largely because I don't blog or talk about tech. But I'm also not in some kind of weird sheltered place where I have no idea what's going on.
If we cut things off there it becomes a statement I find much more palatable.
Not everybody has the luxury of spending time thinking and talking about code - most of us spend that time writing code. Not that no one ever benefits from thinking and talking - quite the opposite - but the reality is, most employers don't want to pay most of their employees to perform that work. They'd prefer it be limited to a senior developer or manager role.
How do you know if you've never met them or talked to them?
"There's this group of people I've never met or talked to, and here are my assumptions about them." is a horrible thing to write. Baseless assumptions because you're some sort of 'thought leader' in web doesn't lend itself to saying backend dev, embedded devs, game devs, enterprise devs, etc are putting things out there.
This article can be distilled down to "I live in a bubble and can't be bothered to look very far outside of it. Why aren't those devs coming to me?"
Such a bad take.
By definition. If they tweeted or used facebook they would make their presence known and would no longer be a dark matter developer. Same with leaving comments on a blog, writing blog posts, or attending meet ups with other programmers.
Also, isn't this kind of a self-fulfilling prophecy? Once Scott sees a dev on Twitter or at a conference he must mentally move them from "dark matter" state to "not dark matter" state. By definition that means it's impossible to ever meet a dark matter dev, so they'll always be unknown to him.
Advice I see here is: If you are a creator, take into the account how your decisions will affect ALL users of your product.
To quote the footnote of the article:
> These people and companies all exist, I've met them and spoken to them at length.
Not to me, not to the majority of developers in the world.
Maybe to a bubble of people that do like to blog their drivel, the active twitter users and all others with unwarranted self importance.
Agreed - never liked the author, met him several times in person - he is as fake as fake can be, and is real passion in life is self promotion.
The article reeks of elitism.
Most of these developers, do their job, learn new stacks when the need arises to do so, their work is proprietary, and have hobbies completly unrelated to programming.
This is critical to the discussion. In every industry you see people that live and breathe by their jobs, and why not. But especially in IT, there's this weird syndrome of "the revenge of the nerd" where many teenagers were nerdy and UNcool and now they're in tech and they can have big positions in big name companies and whatnot and it seems that working in Tech is the new metric for professional achievement.
And it's so easy to see the narrative as "only monomaniac nerds make it" because that's what we're told and what we hear. But the truth is, many more people are invisible because their work in tech is just a function/job/utility rather than a revenge over their own teenage years and/or a complete dedication.
I personally grew from the former category to the latter and I'm so much better in my life. YMMV as always.
Lots of developers have various degrees of that "revenge of the nerds" syndrome, but many (including myself) get over it somewhat as you get through a few waves of cool new tech fads, crunch times, and boring projects that have to get done and it just becomes a job to do.
For example, I've been participating and trading in prediction markets as a form of income during the pandemic. It was originally just a hobby but I was between jobs when the pandemic hit. I imagine many employers would be turned off by this (and still have no idea how to present this as I'd like to get back into tech). If only due to a lack of understanding as it is still a very small niche.
And most code and projects I produce aren't really worth the extra effort to present publicly and would only serve as a noisy hiring signal. Would anyone really think it's impressive to see dozens of hastily written disposable web scrapers for very specific details from niche data/news sources? Or some code to browse a dataset in Pandas? Probably not.
> Maybe using ASP.NET 1.1 at a local municipality or small office
> But, as my friend Brad Wilson points out, those dark matter 99% have a lot to teach us about GETTING STUFF DONE. They use mature products that are well-known, well-tested and well-understood.
The last 3 - 4 years i use the shiny new tech things and i feel like i fall from one black whole to the next. Most of the time i fix things of shiny new things. These new things fixing problems, that was solved 20 years ago from dark matter developers. But hey, we have cool bloggers and facebookers, they told us, that these things are cool and a really joy to work with. Haha, they don't now anything.
You know things like hydration, "islands", SSR, yes right, things that we use 20 years ago. And now we are back here again, but wasted huge amount of time, reinventing 20 times the wheel.
Dark matter developers, you have done everything right!
The article is from 2012. I wonder if the author's opinion has changed much since then? It does seem that we don't pay enough attention to making things that we can keep running and improving for decades. These days I really favor mature tech. Things that have stood the test of time.
I have a lot of respect for things like Java and Ruby on Rails and even Windows COM.
And .NET has a core problem in exactly that space: They have a huge (profitable) market of black matter developers who like X (e.g. a visual designer) and a open source community which asks for Y (e.g. WebAssembly support). So .NET is always ripped apart between the Enterprise centric analysis (MS has agreements with big companies to analyze their code usage) and the analysis from the GitHub issue trackers.
I enjoy programming and have solo build systems that run businesses. However, there is so much stuff that interests me outside of programming that I don't care to commit time for projects that may never bring any value to my life.
I'm also sick of the internet machine and all the hype about some tech. I got burned by picking up something new after hearing about it on social media. Many times the YouTube tutorials are so basic that as soon as you start implementing that tech you discover bugs, lack of docs.
All new tech sucks when you are trying to "get stuff done". I had discovered bugs in books and was literally stuck for days trying to figure out how to resolve them and SO would return no solutions.
People in startup, or software development social media bubbles tend to get a very distorted view of things. Like that everyone is using react, or that everybody is doing SPAs, unless they riding very front latest trends where things are headed a little more back to server side (e.g. server components etc).
But both of those are nowhere near everyone. The whole point of this article is a reminder that what everybody is talking about on sites like this is far from a complete reflection of the space. Most bloggers like to talk about the latest tech, and most startups often use either the latest or near latest, but it is important to remember that the resulting perceptions are skewed relative to the sheer number of developers out there.
Does that mean I didn't have anything interesting to say? No. I think I had a lot of interesting things to say. It just means I'm not popular.
Likewise I don't bother posting on Hacker News or Reddit anymore because all of my posts have been mainly ignored.
Does this mean I don't know anything about new technology? How could I possibly not be aware? I read HN every single day. The last few contracts/startups I have done were with the trendiest stuff like GPT and Stable Diffusion.
You popular people don't know about me because I'm not popular. But don't confuse that with being out of touch or uninteresting. Popularity and merit aren't the same thing. Go listen to the Billboard Top 10 songs right now and tell me if you really think those are the best 10 songs in the world currently.
I’m not particularly comfortable, using terms like “dark matter” to describe folks. Many of these people probably far outshine me, in passion, skill, talent, experience, and expertise. A lot of them are likely not visible, simply because it isn’t a priority for them.
It takes effort to keep a high profile, and many folks find they have other things to which they may wish to apply that effort.
It's fun to test new technologies, but I can't in good conscience recommend them to companies that don't have the resources to actually be at the bleeding edge. Too many people look at the shiny new tech and don't consider the cost of using it.
My primary motivation is creating a good product. I get things done by managing a healthy mix of known technology with a sprinkle of new things so that the magpie developers can see something shiny. In my experience the new shiny things should be kept to a minimum until you either grow so big that your lost opportunities are insignificant, or you have several teams able to cover the bets in case s*t hits the fan.
Unless your startup is mostly technology (like a new database), you are probably better off using "boring" technology and focus on making your product stand out. Try to find a few dark matter developers who knows how to balance.
Similarly Azure is just a myth and does not exist.
btw, Windows Clusters can do a good job, albeit an expensive way to resilience, but not something understood from many of the Linux-only crowd.
We need more openness and less closed views, for sure.
Can you point me to how this stat was determined?
In Australia, I work as a consultant across many orgs, both large and small, and the percentage of Windows Servers is definitely over 75% here, but apparently we're an outlier. For example, Australia East is a top-tier Azure region because there are so many Microsoft-centric large enterprise orgs around here.
It also matters what you consider a "server" and whether you count "unique configs" or raw numbers. For example, a University might have 2K-3K distinct Windows Server VMs and a 5K node Linux HPC environment running an identical build as a single cluster. So do they have more Windows or Linux servers? Technically, more Linux, by the numbers. But their admins will spend 99% of their time dealing with the large number of distinct Windows VMs, and treat the Linux cluster as just 1% of their fleet.
In this manner, the FAANGs definitely skew numbers. They may have hundreds of millions of largely identical Linux servers, but they're treated as a few huge clusters.
And of course, startups created by ex-FAANG employees will generally be Linux-centric and all hires will be similar Silicon Valley types, reinforcing the notion that "only Linux is used for servers". Yes... in one tiny part of the world. I bet BSD is also more common... near Berkley.
Yes. Real-world developers, as opposed to serial new-tech-hype suckers, software convention panelists/social media influencer developers, and cushy FAANG employees.
Previously discussed here: https://news.ycombinator.com/item?id=31767890
Don't mistake their silence for lack of an opinion though! I work with them and do my best to encompass their views and requirements where I can.
Speak for yourself.
> and need reminders that our views, circumstances, and passions don't represent those of the majority
There is an army of self-appointed activists/moralists spamming every thread in every forum with such "reminders". I sure wish there were fewer of them, and that forums like this one had rules against repeating that trite remark 200 times per day.
But to me, in 2023, this post is such an odd take.
People not interested in being noisy on the Internet (or the walled gardens that today constitute the "public", attention-based Internet) in no way relates to _any_ of the numerous assertions he goes on to make about those hypothetical 99% of people.
Literally every take and even implied take (e.g. the silent 99% don't use technologies that iterate at high speed), seemed like a totally off-base generalisation in my experience.
As a counter point and friendly reminder, you don't have to feel pigeonholed by any or all of the assertions in this post just because you don't want to be noisy on the internet.
So, the article alludes to a spectrum of engagement, but uses a binary distinction (dark or not) as its driving metaphor which has created some cross-talk in the comments here.
This is a bit unfortunate because other metaphorical humor in the "grey developer" spectrum lies in wait. (Not just grey beards, grey matter brain stuff, etc.). Two of the comments to his blog (spelling "grey" both ways, lol) mention this.
Of course, even referring to it as a one-dimensional "spectrum" obscures the complexity that it is surely multi-dimensional, and indeed my quoted comment begins to enumerate independent axes.
One of those things is not like others. I read a lot of tech blogs/articles etc. I just feel a bit of imposter syndrome when it comes to writing/blogging. Not a lot of user groups or conferences (e.g. for Linux which I enjoy a lot) at my current location. For tweeting/facebook, I deleted my facebook account long ago and I never got the point behind twitter for tech content except for things like breaking news/product launches etc.
They are getting paid reasonably well either way.
You see, I'm a Corporate C# programmer from the EU. All I ever need exists on microsoft.com and stackoverflow.com, and when my 9-5 is up, I go home and I don't think about work for the rest of the day. Average age of my team is probably around 40.
Dark Matter Developers: The Unseen 99% (2012) - https://news.ycombinator.com/item?id=20599190 - Aug 2019 (90 comments)
I think we've learned that there's a lot more to developing software than "make XYZ work": can we onboard new devs without driving them mad (readable, logically structured, documented)? is it testable? do we have up to date dependencies?
I am the same type of person. Solving a problem and creating a great product can be very rewarding and is my primary motivation.
There have been times in my career where I have been thinking more about how I could sneak in a new technology in a project. In those cases it was a sign that the things I worked on were not motivating enough, or a bad work environment. I'm more aware of how these things affect me now that I have the benefit of hindsight.
Dark matter code can be clean or spaghetti. New shiny code can be clean or spaghetti. Implying that there is a causal relation between dark matter and spaghetti is disingenuous I would say.
Of course, code written for the new shiny framework from two weeks ago is at most two weeks old and thus hasn't had time to accumulate cruft. But that doesn't say anything about the quality of the developers.
I would say if code is really old and still receives new features, the developers must have been doing something right! Take (GNU) Emacs as an example, it's from 1985 or so I think, and now speaks LSP.