The Stupid Programmer Manifesto
hasen.substack.com
hasen.substack.com
Most of the things in this post could be the right answer if the project is a weekend side project that's going to get a few hundred views a month, or a website for a small business. Bob's Restaurant doesn't need a Google scale website built with microservices, Docker and SPAs hosted on AWS and S3.
But something like Google or Amazon or Netflix would. Use the right tool for the right job.
Plus, even a 'stupid' or '0.05x' programmer can be a rockstar at a small company or organisation where a lot of complexity isn't required.
Well, that and how most complex projects didn't start that way. A lot of the time, these things were built in a much simpler way, and dealing with increasing demand over the last decade or two caused them to balloon in complexity.
Theres no High Availability so in the chance it went down theyd be screwed and have no failover but makes you wonder if its cheaper just to lose traffic for a few minutes while theyre upscaling VPS resources or pay for all the HA scaling stuff running around the clock.
All that comes down to the large variety of training everyone in the field gets, from university to youtube tutorials.
It's noisy out there
The opposite.
> All that comes down to the large variety of training everyone in the field gets, from university to youtube tutorials.
A big problem is that a lot of the youtube/bootcamp ecosystem focusses on "teaching what they do at the FAANG" to lure people into thinking they could get a job there by enrolling/buying whatever training they are selling. And then these people repeat what they have been told are "the best practices".
Proper engineering degree teaches about gathering requirement and analyzing the problem space to understand what really needs to be done.
This. I don't even think you have to bring a degree into it, honestly. This is just at the heart of all engineering (software or otherwise).
And then they get jobs which serve to validate their learning paths (maybe not at FAANG), which gives them confidence about what they learned being the right way to do things.
>Proper engineering degree teaches about gathering requirement and analyzing the problem space to understand what really needs to be done.
Well sure, but a lot of non-engineers do that too. Where's the real distinction at?
A lot of non-lawyers can discuss law principles and explain laws pretty well; but someone who passed the bar I know for a fact are familiar with certain concepts.
Same for engineers.
- Home page - About Us - Menu - [Maybe 1 or 2 info pages] - A contact page (with or without a form)
That's something that can be easily done with static HTML and CSS, and hosted on just about any random shared hosting service you can imagine. Assuming it doesn't become a major viral hit, it'd probably use a couple of megabytes of space on the hard drive and under a hundred megs of bandwidth.
Maybe up that number a tiny bit if Bob decides he wants to run a blog on the domain and you decide to just do the whole thing in WordPress or Drupal or whatever else.
Either way, a site that small could be built like its 1980 and it'd be perfectly fine.
Early on the client wanted to make changes to the menu, and have a feed of events from their Google calendar. I implemented a Google Calendar feed script. Enter the world of dates in JS and learning how to use OAuth to hook into their feed. The client was getting confused as to why all this work had to be done for such simple little features and there was extra cost associated, and at the same time requested that they can CRUD menu items. I told them we need a CMS for that. They didn't like hearing the was more time and cost associated. By this time the statically implemented, elegantly designed Home | Menu | About site delivered the purpose of a visual mockup rather than a website that will help the business retain and add growth.
It could be said that this story is one of not planning and getting the spec right with the client, which in part is true. The part that was wrong is that the site could be this simple CSS HTML static site when in fact, even for a small business lacked the features and flexibility needed. Initially, the client and I decided that we would have me make the changes and updates to the static food and drink menu pages. This turned out to be very inefficient for both parties. Small businesses often don't want to pay hourly rates to have minor updates done on their website, which was the case because I don't work for free.
In the end the HTML/CSS site had a CMS, Database, and multiple external services. I learned from this moving forward and came up with the right stack for the scale of business I work with. However, in the late 2010's I noticed more small businesses opting into products like Squarespace and Wix, and selling clients with certain small businesses (nick nack shops and local bars restaurants) on custom site builds was getting harder.
Back to the original post. I get the point of the blog was to make a statement against unneeded complexity. Sure, however, it's important to remember that the work of producing digital products is inherently technical and complex. Developers should strive to be a smart programmer and spend time picking the right tools for the job.
So we ended up with great puzzles like how to link an order service with an inventory service to check if there's enough inventory to fulfill an order, all through a central event bus. Then of course the order service has to update the inventory if the order is confirmed. But of course, the order service also has to talk to the pricing service, and since the price of products varies by day, they'd have to remember the price at the time the order was made. Whose responsibility is that? No idea, that's someone else's service.
I've advised engineers to rewrite code before, mostly for resume building to help jumping ship to somewhere better. Basically, use the current job as a way to train on the stack the company you really want to work for is using.
If a company doesn't have a stock-based comp and sticks to prevailing wages/CoL there's no incentive to help the company succeed. Especially with non-technical management, the best thing to do for ICs is often to aggressively rewrite (in the language that's trendy right now and that better companies use) and get promoted by showing off a significant output. Then jump to a better company having spent a year training on their stack at the previous company's expenses.
so naturally it doesn't matter (in long term) what you should learn to get a first job, change jobs, change domains - prioritize ROI the tool would give you in future, a job market, and your sanity :)
It also creates an invisible bias towards worse companies in your next position, because smart companies are wary about this kind of behavior, and will be savvy to filter out candidates that have displayed it.
Great work happens at the intersection of employee's and company's interests!
Why not simply align the two? Compare the Xerox Alto with the Macintosh...
If you want microservices to work you shouldn't have anything central in them, and you should avoid async as much as possible.
One-way async dataflows work much better. That way each microservice is up with all the (potentially delayed) data it needs to respond to requests.
Typically one-way or two-way is not a choice you can make as an engineer, most processes in your business require two-way, because they are initiated by a user and user wants a definitive response.
The "potentially delayed" approach is very tempting for engineers, but it should be exception rather than the rule.
It
- dilutes responsibilities between services (who is responsible for delays? is service A producing too many messages or is service B too slow to process them?)
- makes your SLA vague (message was processed 3 days later, do we treat it as downtime or not?)
- requires more infrastructure & processes (every service has a queue, dead-letter queue, and a process to deal with dead letters)
- requires a ton monitoring overhead (what delay is acceptable? how do we even measure delay? what if you have different SLA for different messages? we'll have a monitor per message type?)
- introduces a lot of unnecessary complexity and rules (how do you deal with TOCTOU, e.g. admin deactivates a user, but by the time the message gets processed he's no longer an admin)
- ruins user experience (we received your payment information, but we won't immediately tell you that it's wrong).
Despite it's downsides, potentially delayed approach can be a fine tradeoff when it saves you 7-8 digits per year. Most companies never reach this phase.
By abstracting content the only thing that needed to change for new types of content was the content service and clients. Abstracting recipients which can be users/groups, meant that the only service that needed to care about this detail was the one that replicates sent to inboxes in the receiving service. Because of the use of content ids and user/group ids, this is all small idempotent/immutable metadata. The system was complex (yet became manageable over time) and onboarding onto each service was immediate.
I think few have seen well-bounded microservices' contexts leading to the idea that it's a bad distributed monolith. Also worth remembering that advantages of microservices 'done right' is large scaling of developers and isolate failures.
They probably already have such programmers. Actual 10x programmers are exceptionally rare. Having an entire team of them is extremely unlikely.
OP's approach might work for a personal project or a small company, but when you need to work in a large and complex system you start to appreciate the beauty of tools like Docker.
It's cool to question cargo cult, but don't throw the baby with the bathwater.
You should always be chasing the simplest solution for your problem because no matter how smart you are today and how much you understand the problem and the abstractions, a year or two from now you’ll be a different person.
You might have new team members, or fewer, new management, marketing goals etc etc etc. Complexity is inherently difficult to change. My chasing simplicity, you are optimizing for potential change.
Me neither.
All of these things involve tradeoffs. Multiple repos vs monorepo highly depends on your organization. Microservices vs monolith has been discussed to death. A simple, locally-driven deploy process is great for solopreneurs but doesn't scale to full companies. Load balancers might be required depending on your availability requirements. Storing all your data on disk is easier than maintaining a database, but aside from performance bottlenecks, it's harder to define a recovery process (RTO/RPO) in the case of a hardware failure. And as much as I dislike them, there are ease-of-use benefits in using dynamically-typed languages.
The biggest technical challenge in most software engineering roles today is evaluating the pros and cons of these types of choices and choosing the right one for your situation.
I think they are smart enough, they just don't want to, or don't see the value in it (even if they've made an uninformed decision)
> I’m not smart enough to figure out how to transform data between different layers of the system, so I just don’t. I use the same represetnation in the UI layer and the storage layer.
OK, well then you might need to gather a little more work experience and meet the repercussions of decisions like this one.
Or maybe the OP is a contractor and never has to deal with the repercussions of their actions.
Seems like data protection occurs way lower in the stack like https and encryption at rest.
So even if all you are displaying is the users name or initials you would still be sending things like SSN and credit card number to the front end
Sanitizing your inputs has been known about for literally almost half a century that should just be default for developers at this point.
Except if you're a "stupid programmer", in which such defaults are irrelevant to you. In such cases, one can only hope they're relying on tooling that sanitizes as much as possible for them.
Handing data from the user (untrusted input) directly to the backend unchanged is going to in 99.9999999% of cases also mean it's unchecked.
"I'm not smart enough to bother sanitizing my input, or to learn about stuff that's someone else's job like security" would fit right into this "manifesto".
E.g. does the SSN consist of 3 valid integers split on dashes? If not, it ain't a proper SSN. Catching that typeError is much safer than trying to roll regex or character allow/blocklists.
But also, the manifesto never mentions data structures.
It says
> I’m not smart enough to figure out how to transform data between different layers of the system, so I just don’t. I use the same represetnation in the UI layer and the storage layer.
Transforming data means actually changing the data, not the data structure that houses it. The author is explicitly talking about making changes to the data itself.
You should validate the data and sanitize the data...but if youre transforming data youre headed into a state management nightmare.
State management is the number one enemy and other than sanitizing and validating both the data and structure should be considered mostly immutable.
You don't want to store code unmodified in a database; that's how you end up with SSTI or stored XSS. You encode special characters in a simple, reversible way (as one example).
Similarly, you don't store passwords untransformed in a db, you hash them. That's a transform.
There are tons of examples like this.
> I’m not smart enough to figure out how to transform data between different layers of the system, so I just don’t. I use the same represetnation in the UI layer and the storage layer.
That has nothing to do with the data structure, is about how the data is being represented in the backend vs the frontend. It's explicitly about changing the data itself.
Stuff like url-encoding and escaping characters are examples of transforming data to be represented differently in the backend (where e.g. it's encoded or escaped) from the frontend (where it's displayed in "proper" formatting).
How are you going to validate it doesn't do something harmful?
And also, you keep mentioning sanitization as though it's not inherently a transform. Stripping out whitespace? That's a transform. Just because it's a one-directional transform doesn't make it not one.
Even worse, you may be running a mail server on the same machine.
That happens to every server exposed to the internet.
Unless there is a targeted ddos event its usually on the level of a query or two per second AT MOST or so.
Its something most people don't worry about because theres no way around it.
If you need ddos protection you put the server behind cloudflare or something like it.
Its totally fine to raw dog the internet without a firewall in my opinion if the server only has a single web server that you keep updated and sanitize your inputs.
Running other services on the same server without a firewall becomes horrible though youre right. lol
Managing state is the number one enemy on software.
Store it in one format in the database. Read from the database and transform it. Package that into a TO object and send it over the wire. Receive the TO object and repackage it into your local context Take the local context and repackage a bunch of parts of it into each view model as necessary.
Everyone just wants to bundle up data and throw it over a wall instead of working together to engineer end to end.
When it comes time to change your data model you can't without bubbling it all the way through the to the public API which may not be desirable.
Repackaging the data at every level means you only have to change the transformation at one of those levels.
For small projects this isn't a big deal.
But if you are working on a large project with multiple teams you have public contracts at multiple levels. You don't want to wade through 10 layers and 10 teams of changes because you change the way you store and compute some attribute.
Otherwise you're going to break something downstream and you're still going wade through all of those layers, and now it's less obvious what broke because everyone is re-bundling your data into their own formats.
And I am not advocating for dumping your DB rows directly onto the wire for the frontend to fumble.
I am just saying it's silly to write a frontend and backend in a super modular, decoupled way when they are actually just a single service.
For example let's say for whatever reason we were storing a duration as an int. But instead we decided to migrate it to start time and end time.
Do we need to force that change on everyone downstream and add a new major version API? Or can we just compute the old duration from the new attributes in the transformation.
Even in your example do we really need to change the frontend in this case? For small projects the extra boiler plate probably doesn't out weigh the benefits but for large projects it absolutely does.
If the data structure needs to be changed it needs to be changed at the beginning of the system typically the front end. And downstream code needs to be fixed to accommodate that.
Otherwise youve created 20 different state machines for each part of the system stacked on top of each each other each expecting a different data structure and returning a different data structure.
So changing anything after the system is sufficiently complex is an exercise in masochism and development will slow to a crawl to avoid breaking one of the 20 downstream black boxes.
There should only be a single data structure contract that all teams follow.
There needs to be a SINGLE data structure passed through the system originating at the beginning of the system. The data structure can be added to by code along the pipeline but never changed.
At this point you may as well just use a object or dictionary. The type doesn't give you any idea about the actual shape of the data.
> So at any given point you don't even know what data is present or not present depending on which point in the pipeline it has passed or not?
At least with a non mutating data structure with the public contract you can tell if it's passed through a part of the pipeline because a required field is blank or null.
With a mutating data structure that gets changed, say 20 different times throughout the system you have no idea if it's passed through some part of the pipeline or not.
Even better if somehow everything can interact via a centralized database where all state is stored even intermediate state. Its not just storage. It's also state management.
In dynamically-typed languages this is a nightmare, and this is one of the significant reasons I've stopped using them. Finding a bug six months to six years later from some attempt at this and finding a place that used the old type incorrectly when it was getting the new type, or because the programmer thought they were getting one but in fact got the other, or my favorite, the code is actually called with both types and nobody noticed until way too late... it's just something I've had enough of.
Remember I'm a 0.5x developer. If I have to write code to transform data for every kind of entity/object I need to store and have a UI to view and edit, that's way too much. I'm already very slow. No need to slow me down further by telling me I have to write so much extra code that does no useful work.
By way of example, I'll point to the classic example of a New User form UI: that UI will need 2 password inputs (one for the password, the other for the "confirm password") - but your User object/schema/DB-table won't have two separate string password fields - it'll have a single binary/byte[] salted-password-hash field, and you certainly must never show that in the UI.
As for your (reasonable!) concerns about having to write-out repetitive types/models that all share similar (though hardly ever identical) data/fields/shape: I agree it's tedious, but that's what we have scaffolding-tools for (granted, I often end-up having to write my own scaffolding and templaying tools...), so for a simple CRUD application just design your database and have the scaffolding tools take care of the rest, including stubbing-out a functional UI (and this is why Ruby-on-Rails was huge when it first came out: it took-away all of the drudgery in CRUD web-applications - but then all the other frameworks/platforms improved their scaffolding-stories and RoR is certainly less attractive in comparison now.
-----
Oh, and of course, ChatGPT et al. is also incredibly effective at scaffolding entire solutions from scratch, e.g. from last month: https://bit.kevinslin.com/p/leveraging-gpt-4-to-automate-the
But even in this case, I don't have two representations of the same object. Rather I have two different object:
Account (persisted)
SignupRequest (not persisted)
The SignupRequest is used to create the Account
The signup form on the UI is about editing the SignupRequest object. This object will be sent from the UI code to the backend as-is. The backend code will use it as-is (ignoring the json encoding/decoding).
There's a code path in the backend that takes SignupRequest and uses it to create a new Account.
>I don't have two representations of the same object. Rather I have two different object
Exactly! Your api contract and your storage are ALWAYS two different objects, because they serve two different concerns. Sometimes by coincidence they can share the same shape but there's no reason that they need to be coupled together and impossible to change independently, other than the fear of inconvenient "boilerplate" mapping logic. By doing this up front, and not even letting it enter your data model, you create a formal abstraction boundary; it's reserving the right to change two pieces of data independently. Also, mapping/transformation logic can often just be simple, pure, total functions; which are trivial test and maintain compared to anything that touches I/O.
I find this is never the case - usually for the exact reason you inadvertently sold as some kind of "benefit": "it's reserving the right to change two pieces of data independently" - because when you need to map from, say `SignupRequest` to an `SavedAccount` object you'll encounter data-members required by `SavedAccount` which cannot be sourced from the `SignupRequest` object - for example, supposing the UX people come to us and say we need to split-up the registration form into 2 pages, such that most fields are still on page 1, but the password boxes are on page 2. Now you need to deal with how to safely persist data from page 1 for use in page 2 (so using hidden HTML inputs between pages won't work because that requires POST requests to work, but both pages should be able to be intiially-requested with a GET request.
Not really. I have request objects for everything. "Search" is a request object. List pagination is a request object.
Every function exposed through the RPC API takes a request object and returns a response object.
The response object is often just a collection of objects straight from "the database".
A response to a paginated list request will contain a list of objects straight from the database, in addition to some metadata about the pagination (names: current page number, total page count).
The cruicial part is there's no "transformation" of data as it goes out from the database into the UI. There's some aggregation and grouping (an outer object that contains multiple objects), but that's about it.
Again though there's a subtlty: some transformations do occur, but they don't occur on the path from the storage to the UI. Instead, everytime I store a complex object, I also derive a "simple" version of the object and store it too.
When you request a list of objects, you get the "simple" version, and the UI displays them in summary format. When the user clicks one of them items on the list to see more details about it, the backend sends the "full" object.
Notice the underlying principle: the UI flow dictates how the storage layer stores objects.
This is the anti-thesis to the common wisdom, where the storage layer does not care about the UI, and it's the job of the intermediate layer to transform data for the needs of the UI.
That always pisses me off. I can copy/paste the password from the first field into the second. Hell, I use a password manager; I pasted into the first field, so I paste into the second.
It's just a check to make sure they match. But my passwords are complicated enough that I can't remember them long enough to type the characters in the first time; I literally have to copy/paste.
So this anti-pattern assumes that users are using some memorable password like their mother's maiden name, probably for all the services they use. Anyone using sane password practices is penalized with stupid friction.
[Edit] I have an even bigger gripe about asking me to enter my email address twice. If you really aen't sure I entered it correctly, send me an email asking me to confirm.
I think you're really, really underestimating how easy it is to make a typo when entering a password into a masked password box, hence the 2 fields.
Also, web-browsers don't let you copy-and-paste the password from one box into another - so I don't know how you're doing that: https://security.stackexchange.com/questions/149326/why-cant...
-----
I appreciate that you're just-as-annoyed with web-form tropes, cliches and irritations as I am - and when I'm building a system or UI for savvy people then sure, I'll do things like skipping the registration form entirely and just use OIDC federation or PassKeys or whatever the current-security-fad-of-the-month is - but my day-job requires me to write software for ...uh... "normal people" and part of that means having to weigh-up the support-costs of users who want a predictable and easy-to-understand that isn't too different from what they already know - and those normal-people are what pays my bills.
Point taken; perhaps simply having that widget that lets you see the password unmasked would be better, than forcing you to enter it blind twice (and getting it wrong twice). Fact is I don't try to copy/paste one field into the other; I paste the same stuff into both.
FWIW I'm a normal person. I don't know what OIDC is, and I've never been asked to use Passkeys (I'm proud to know nothing about Apple devices, and I don't entrust my security to "the cloud" or third parties). My password manager is local, backed-up locally. I'm already annoyed once I'm presented with a registration form, and only complete one when I'm forced to. Every extra annoyance increases my rage.
It was a very, very painful learning experience.
It's annoying, but I understand why that's done, so I don't get too upset about it.
But making me type in my email address twice? Grrrr.
And if you're a slow and unskilled developer, you want to minimize the the pain in the second phase.
(That goes for all developers, skilled or unskilled of course.)
You want your human representation to be specialized, highly redundant, and as large as needed so anything important is presented. Your users can't find data or keep it in memory, enabling them is your main concern.
Usually, those two sets of constraint lead to the same format on behind the scenes admin panels and nowhere else.
This is necessary because each of those things represent completely different needs, because they operate in different domains. In particular, the guarantees are different; the input must effectively be treated as having no guarantees, and you must check them all. Your internal operation may add additional guarantees it provides, things you don't need to check anymore because the mere act of being passed a value of a particular type means that the code can rely on this particular thing being true, thus saving me a ton of code everywhere. For the output, you don't care at all about any guarantees but you need to conform to what the external world needs.
In general, trying to cover all these bases with one structure is a bad idea which leads to pain. I've seen it many times, where developers try to overload one structure to do too many things.
In specific... it so happens that 95%+ of the time, covering all the needs with one structure works out fine. But I conceptualize this differently than you. I do not say to myself "It's OK, one structure is all I even need because anything else would be overcomplication." I say "It so happen here that the structures are so similar that I can conveniently elide them down to one without significant loss. I can do this because I have examined all the needs and guarantees and verified that they do not conflict."
But the difference is, as soon as they do conflict, as the codebase changes over time, I split them, leaning on the compiler to guide me through the process, because I know from experience it is not particularly hard. When you need to do this, it is easier to just do the split of types than to try to make one type straddle the gap. (Besides, once you have one type straddling one gap, the odds are by the time you're done it's going to be straddling more than one gap. This tends to happen precisely to those most central types.)
I bet you have at least one type somewhere that is suffering from trying to straddle too many use cases. But I would also bet you don't actually have many such types. It turns out that the majority of the time the elision is safe. But I think it is a useful perspective to still mentally model that as an elision and having separate types as the underlying model. I think the way you are advocating for thinking of it works fine the 95% of the time our models agree, but in that other 5%, someone following my system is going to be a lot happier than someone following yours.
I do a lot of relatively small programming, projects in the single person-year range. I do a lot of this sort of elision, because bringing the full power of generalized architecture to such projects can cost you a lot more than it gains. But I also do this sort of modelling in my head a lot, too, and when I notice a particular elision is starting to cost me something, I will on-demand unelide some particular architectural elaboration on the spot. In fact I am taking a break from this exact process to type this post, as I need to replace an increasingly complicated hard-coded structure of decorator-based plugins for a fully configuration-based model of arbitrary combinations. For any given such re-elaboration, it is perhaps more expensive to do than if I had started with that architectural feature in the first place, but across the full space of possible architectural features I win big versus starting with an architecture that is too heavy weight in many ways that I will ultimately never need in this particular project.
This doesn't match my experience of contracting.
Not caring about code quality and then leaving others holding the bag of sh*t is frequently what "ladder climbers" do, and those are, by definition, on payroll rather than contractors. For a contractor, there is no ladder to climb, but willingness to get one's hands dirty is definitely part of the job description.
Contractors are quite frequently the people who are brought in at extra expense to maintain mission-critical infrastructure when the employee who made it is no longer at the firm, or has moved on to different responsibilities. After having gone through several contracts, a typical contractor will often have a lot more experience with different coding practices and their respective outcomes and a deeper appreciation of good engineering.
The trick is to refuse to adopt the job-title "contractor" - simply refer to oneself as a "consultant" - it goes a long way.
Fun-fact: there's no rule against contractors/consultants earning equity in their clients' projects either: apparently I can artificially raise my hourly-rate by 20% during the post-interview negotiations, then sweet-talk the client into letting me have token equity in exchange for a 16% pay-cut. Simply do this for every small gig you get and eventually it works out as a pretty-nice personal-ladder to early-retirement.
...not that I want to retire, really - the idea terrifies me because I simply can't do anything except work :/
Also, remember another possible reasons why a company might hire a contractor: to have someone to blame for a project they know is failing.
One of my favorite things about contract work is that it's time-limited. This means that I can be 100% honest about bad dev processes, bad code, bad management, etc., because I won't have to worry about angry managers making my life miserable for any longer than the remainder of my contract period.
I can't count the number of times I've spoken up about something terrible, only to have the permanent devs privately tell me later "I'm so happy you said that. I've been wishing I could bring that up for years"
That's what I read into it, because that's what I identify with. I've met so many fellow CS students (back in the day) who cared about performance optimization. I never cared. I just wanted it to be simple, working and done. I cared about clean code because I had a hard time understanding anything else. I felt allergic to over-engineering because it portrays a whole host of thoughts/emotions that just hurt my brain empathizing with it how it would feel if I'd have those. So I kept things simple too.
Usually the repercussions aren't really there. When they are there, then they need to be there and it's usually a sign that your product and the place in the market you're at is evolving. Code is a living thing. Sometimes a big rewrite is necessary, but if that simple unoptimized code held up for 10 years (which is what I'm experiencing), then I'd argue that's okay.
Some are universal, but some, only apply to certain domains.
I'm not really a fan of tearing down others, but I do feel the industry has a lot of room for improvement. Not sure if these types of screeds will actually make things better, though.
Regardless of what he's trying to say, there are much much better ways to present these topics.
I was working on a feature for a webapp that I thought should be doable in a day, but I spent roughly a week on it. When I was done with it, I looked back and thought: wait, why did this take me the whole week? I couldn't come up with a satisfying answer.
So I just decided to accept that I'm not that productive. Maybe there are things I can do to improve my speed of implementing features, but for the time being, this is my speed.
I was going to make a writeup about coming to terms with that.
But somehow as I was writing (originally on Twitter) I linked it in my head to the way I hate modern dev culture: the docker and the webpack, etc. Programming is already hard, why make it 10x harder with all the tools that require complex configurations, etc?
I remember raising this point on HN and other places with other developers, and that there was always push back from people who swear by these tools.
So it clicked in my head: I hate these tools because I'm already slow as it is and I can't take it when these complexities slow me down even further.
Then as I spent more time thinking about the content, I thought this a manifesto worthy content.
But in terms of the points mentioned: I'm dead serious about all of them. Although what I actually mean might not be obvious from first glance.
When I said I write objects as-is, it appears that most people in this thread thought I'm writing individual files. This is not what I'm doing. I'm using a B-Tree backed key-value store and doing binary serialization. I also have a scheme where I make use of the properties of B-Trees to make indexing/querying possible. I have a whole write up about this topic: https://hasen.substack.com/p/indexing-querying-boltdb
For using HTTP as a medium for RPC, I also have a writeup: https://hasen.substack.com/p/automagic-go-typescript-interfa...
These alternative techniques took time for me to develop, so you could say that it would have been faster for me to just use postgres/docker/aws etc (the standard stack). But this is my point: I really am too stupid to learn them.
However, I would like to share a different angle, if I may. I don't think your speed is necessarily the problem. I think your time estimation skills might need some work.
It's easy for us to think things "should be simple". Unfortunately, the modern world of software is becoming more and more diverse and complex (a symptom of software "eating the world"). It's only natural in a world that has evolved from 8086's and terminals to interconnected-everything, multi-OS, multi-platform, multi-device magic is going to be orders of magnitude more difficult to build on at any reasonable scale of business. There are just so many small things that can go wrong!
I would gently suggest you stop being so hard on yourself and instead of beating yourself up for "being slow" just take each technology one at a time. Ultimately, the way to survive in this career is embracing change and staying sane while doing it. It's not easy, but it's the only way.
As for you being dumb, I would also like to say that it is clear you choose to deeply understand the topics you put in your head. Many folks "learn enough to get the job done" and very little more. It seems you have an appreciation for deep learning. This is not a bad thing, it is usually this trait over a long time that differentiates a truly senior engineer from an intermediate one. Your path may be slower, but it is probably more thorough and more informative in the long term. Keep in mind though, this breeds imposter syndrome in your mind. When you accept you still have lots to learn, it is easy to feel like you know nothing at all. I would suggest not letting this get to you. We all feel it, it is real, we're all just trying to do our best day-to-day.
Oh, and always 2x pad all your estimates at minimum, cause if we know anything about computing, it's that it always has hiccups.
It's completely fine to accept your situation without giving up on improving it.
The problem is when people refuse to acknowledge reality because they really hate certain labels and are too invested in not having that label apply to them.
I'm slow and it's ok.
I'd like to become faster in the future.
But for now, I'm slow, and that's ok.
I have met many, many brilliant engineers. They fly real fast and really far. The funny thing about it is that they tend to crash and burn sometimes too. Sometimes they even break orbit, but even then, someone has to stabilize that orbit from time to time.
Life takes all kinds of people. Folks who are willing to simplify things while ignoring fads are just as useful as those who jump into the unknown of new technologies.
What you think of as a weakness you need to compensate for so you can get work done is what other employers will consider an asset because you refuse to add complexity where it is unwarranted. What you call "slow" others might call wisdom.
I'm really not trying to cheer you up. I'm sharing facts from over a decade writing software being the "slow one".
He's the one who wrote BoltDB https://github.com/boltdb/bolt
I just put a relatively thin layer on top of it.
Now, to address your point more directly: I'm too stupid to figure out configuration, but not too stupid to figure out code. Code gets compiled and type checked. You can have tests, etc. Tractability for code is much higher than configuration.
With configuration, you have to be really smart and keep many moving parts in your head.
With code, you can be a bit dumb and lean heavily on the tooling.
Though there’s one thing I take issue with. „I’m too stupid“ implies that these techniques are impossible to understand, while I think it’s just a matter of being able to put in the time and effort to learn the intricacies. It’s absolutely fine if the resources (mainly time) aren’t available to you, but I don’t think it’s a matter of cognitive function.
Most of the techniques people use to manage complexity at scale are too stupid to actually do that. Their only benefit is that following them prevents people from doing something even worse.
It also prevents people from doing anything better. Instead we dogmatically follow patterns with mountains of useless boilerplate and unnecessary abstraction, but somehow it's considered less complex.
I hate software sometimes.
HTTP Verbs? You really can't get more basic. At least you could try to wrap your brain around the idea that GET reads and POST writes.
Not using SQL? Okay. You're spending a lot of time and effort hand-rolling your own shitty database.
The combination of things suggests to me they have can't or don't want to deal with with non-trivial mental models. Things like transforming data between layers or figuring out how someone else will read your code.
The next person to work on their stuff is going to have a massive headache figuring out what "made sense" to this person.
You mean… they’re stupid? Like they say in their title?
Now, writing an article conflating limitations or lack of desire with intelligence shows... poor self-awareness.
But yeah, I prefer not to say stupid when I can be more specific. :)
Every REST API codebase I've ever worked on would like a word. There's TONS of subtleties that determine whether you should be creating a GET, POST, PUT or PATCH for many different use cases.
And this isn't even taking into account the hundreds and hundreds, maybe thousands of endpoints I've come across that were obviously wrong, but some Senior or even Principal dev had somehow managed to completely botch it - NOT a rare occurence.
You can simplify it to GET = doesn't change stuff, POST = changes stuff. Basic HTML form tag stuff.
For the few things in-between like self-expiring links or counters recording how many times a thing was viewed can get tossed into GET because the data you're retrieving isn't being modified. (Search can be POST if you don't like query parameters in your urls.)
The biggest subtlety I can think of is deciding whether PUT to a place that doesn't exist is valid, and whether DELETE of a place should delete just the object or also the place too, if PUT is only valid for existing places. How to handle that subtlety is application dependent. Personally I like to architect RESTful services such that POST isn't necessary since it's hard to make it idempotent, but sometimes POST is the most comprehensible interface.
Also, I wouldn't bother using REST at all unless I was going to make everything fully discoverable via hypertext[1]. However that's not to say there aren't soundly designed non-RESTful ways to use the HTTP application protocol.
For http verb, I'm onboard with just using POST for everything.
The database is one area that you need more care. Even if just SQLite, storage persists and especially if this is for a company, needs to persist past you.
I've seen this before.
It crashed and corrupted most files because the author didn't understand how file IO really worked (he also created a ramdisk thinking it would be faster).
Fortunately, someone took a memory dump of the process some times before and most data was recovered.
I have come to dislike this growing trend that celebrates mediocrity and failure. Posting proudly that you are stupid, terrible and incompetent seems to result in applause and high praise. Why? Why is that a good thing? I hope this was intended to be a parody.
You may be experienced, but a large part of people in our industry are just out of school and don't have the experience to sort through it.
It's impossible to know if the author has imposter syndrome or not without some way of measuring his competence. However, I think "imposter" has become a loaded term as a result of the syndrome it's associated with. There are cases describing workplace incompetence, or not being qualified for a job, or being rejected for cheating at interviews, but there is no discussion of imposters in those cases. They're just deemed incompetent or not the right fit and moved on from. Actual imposters who game interviews and do manage to land jobs aren't called imposters either, they're called dishonest, but there's no "dishonesty syndrome."
So I hesitate to reach for explanations of imposter syndrome if I'm not actually qualified for something. It can become used as a crutch to fall back on like a lot of diagnoses, instead of addressing the root causes of qualification. Waking up on that cycle made me realize I have a lot to work on in some areas.
Not everyone can be the best and this seriously affects some people's self confidence, especially when they see the best around every corner.
In 2023 it's not good enough to be good enough, at least not if you believe what the people on the internet have to say so folks rebel against such things.
That's not what this is. It's a sarcastic critique of techniques the author thinks don't justify their weight. It's a comment on the cognitive load required by the plethora of complex "best practices" that often goes without questioning.
The multi-repo microservices thing really hit home for me. I only recently had to deal with this, and it was horrible. I finally made my peace with it, but it required adding an entirely unnecessary level of abstraction and techniques where merges are the new commits, and repos are the new directories, and so on. And each service was 90% boilerplate and like one endpoint with a few hundred lines of code.
Ultimately I think reducing the coginitive load of your devs, and making the build-test-debug loop fast, is the highest goal of project leadership, but I seem to be in the minority. If you don't like the "stupid" verbiage, I think one benchmark for a successful system should be usable by devs who are mildly drunk.
It reads very much like "I don't like learning new things" rants I see from time to time.
They've found one way they like doing things and stopped. They don't know how difficult any of the stuff they are talking about is. They just know it's something they don't want to learn.
Unironically.
I like learning good/useful/effective things.
Not "new" things.
Most "new" things are less capable variations of existing things that are better and more mature.
Some of this is because this is what I know, but some of it is because I'm too dumb to see the benefits of the added complexity. I'm not going to do "WEBSCALE!" things, more than likely, so why not just run a FreeBSD server and, if needed, stand up a jail here and there. Chasing the tech rabbit can be fun, but it is also exhausting.
The problem is time. A lone dev just can't take the time to setup the full stack every time for every project at every client. What to do?
Templates, scripts, pre-made configs ... soon you have a stack that will deploy itself and all you have to think about is structures and algorithms. And now you look like a 10x dev. Not because you "know everything in the universe," but because you leveraged technology ... much like our customers want to leverage technology.
The one of you who came up with that idea was the true -0.5x engineer.
1. Easy wins category: Don't repeat code, cause you will take twice to change it. This is clearly visible to even the author.
2. What I call the "tougher" win category: When you require more executive function like breaking a bad habit, managing cognitive load; in short understanding why it saves you time in the long run. A reasonable example of this is using typing in python. I and many of you know that you can catch errors way faster using types+pyre(for example) than writing tons of unit tests, but it takes an old python programmer longer to get used to putting in the types and some times remembering to set up and use Pyre to get used to the idea of how it saves time. Pre-pyre used to python dev might just think "I will write tons of unit tests".
There is obviously a psychological/mental health component to this.
As long as the disk written data is being backed up rock and roll!!
When they say 'compiled binary' are they using Java or are they writing web apps in C?
As others have pointed out: don’t use k8s to deploy a web app for your local soccer team. All of the suggestions in this article are fine.
It’s actually kind of smart.
Update: ... except the REST/HTTP part. It doesn't matter for a small friends-and-family website, sure. But if you're building something you want to have grow and stick around and be used by others you'll just be setting yourself up for problems down the line. I dunno why RPC has come back into vogue, it was a pain in the 90s and early aughts that I had hoped we'd abandoned. It doesn't play well with the wider web ecosystem, it's a pain to grow horizontally, and locks you out of some pretty neat tools that can take your website further as you grow without much effort.
Even if I have the experience, I still decide to use sqlite, single binary, local files, vps, monorepo. At least until market-fit is proven.
Some projects live long enough to become bloated, big, do-everything projects, full of experimental technologies that were forbidden to use at work.
Adding layers, abstractions, interfaces may lead to adding unnecessary complexity, or dependency. How to spot code written by a senior? It is dead simple, obvious. How to spot project managed by a senior? Is is easy to install, well documented, easy to kick-off, uses little-to-none resources.
Sure, you can orchestrate in K8S to spin up the front end, back end, and database instance, but one day someone has to troubleshoot that. That someone being a dev who doesn't work with K8S all the time - good luck.
This is demonstrably untrue and a hilariously bad take.
Docker absolutely simplifies things. Give me a docker-compose file any day over “install X version of Y and Z version of Q”.
Edit: I am actually not arguing just to argue and think about this a lot. A "real world" example is "well, I need to mock out AWS SQS locally and now I have to pull in a fake local queue" or "I need to mock out RabbitMQ locally", etc. The short answer to those things is that you mostly don't - test the endpoint handling queue messages independently / write integration tests for it if you have to. There is no point to having the whole system mocked on your local - SQS/Kafka/whatever will do what it's supposed to do for the most part.
Assuming conservatively that this is 1B/year and 100K users/day, that's still $273/user/day. What are you selling?
Inventory tracking and auditing for Fortune 500 companies in a SaaS before that.
sed -i 's/I’m not smart enough/I have not made the effort/g' {url-in-a-file}
Devs work in a performance business, and most of the time "smart" can be replaced with "effort". Heck, even "experience" often equates to "outcome from prior effort". If you don't care to understand something, therefore making you less "smart", we're often just talking about a choice in applied & focused effort.
Of course, there are caveats to this. Effort alone can't compensate for capability to learn (I keep this bookmarked: https://cdn.shopify.com/s/files/1/0535/6917/products/incompe...) And there are times where effort requires a lot, and we might not have the room/time to simply add that capability to our repertoire.
But all this said, I have found that being "smart" enough is more often just a status of where my efforts have been applied.
I am also a 0.5x programmer. I write dead-simple C; I don't even touch web development, which I consider more complex than C development with Valgrind.
In fact, I've spent nearly 3 years writing a build system. That's how slow I am, and it's made some acquaintances laugh at how long I've worked without result.
But in reality, I'm not just building a build system. I'm also building my own stack, a la [1], including replacing as much of libc as I can and changing the OS API's I use.
I think that soon enough, this stack and this simplicity will become my superpowers and make me a 10x programmer.
Being a stupid programmer means you won't be clever. And if you're not clever, you will be able to debug it, even though it is twice as hard to debug than to write. [2] Good debugging is a superpower in itself.
[1]: https://youtube.com/watch?v=443UNeGrFoM
[2]: https://www.defprogramming.com/quotes-by/brian-w-kernighan/
We're currently trying to simplify a monolith that is too hard to maintain, and the biggest issue for us that debugging is heavily impacted the call-stack size. Sometimes 100 functions before it gets to the point you want. All this due to very complex class and function hierarchies.
I'm responsible for a rewrite of some modules and I'm personally gonna stick with simple stuff.
This reminds me of a bit of advice from Austin Kleon that I've used frequently -- "Make bad art, too"[^1]:
> “Good” can be a stifling word, a word that makes you hesitate and stare at a blank page and second-guess yourself and throw stuff in the trash. What’s important is to get your hands moving and let the images come. Whether it’s good or bad is beside the point. Just make something.
This is a perspective I have to come back to, as an engineer building enterprise-scale things, when I'm working on small-scale projects. I don't need to use the same tools I use at work, I can pick "bad"[^2] architecture, I just need to build _something_.
[^1] https://austinkleon.com/2020/04/15/make-bad-art-too/ [^2] "Bad" in that I know precisely how and when and what would bite me in the ass when I try to scale it.
No shame either way, I think the juice isn't worth the squeeze for automatic testing short-lived code myself
(I love CI systems at work and wouldn't live without them, but that wait time :-(, so everyone tries to get the OODA loop on their local machine )
One idea I had would to build an Elm-like backend language BUT with PHP's "edit the file on the server and it runs" nature. Combine that with some source control, forking and more of a VSCode editor on the server, and you would have a nice DX for small projects.
All those features exist but they live in different languages/stacks, you can't have them all at once (I hope this is where someone replies "you say that... but have you tried X"!)
Good luck getting that to run in different environments because static or not, nearly every executable has dependencies.
> I’m not smart enough to figure out cloud services and auto scaling groups, so I just upload my static binary file using scp to a linux server I rent from a VPS provider.
Which of course you'll need to setup, patch, figure out how to restart apps when they go down and on and on. There's no easy out.
> I’m not smart enough to figure out how to setup all the databases and firewalls and load balancers, so I just embed the http server and data storage engine as libraries in my static binary.
And how do you back that up?
I get the gist of the article, but some things are not always replaceable with an easy alternative. Most alternatives also have disadvantages as well as their own learning curves.
Postgres, Docker, Infra as Code and Kubernetes are pretty much best practices on any development.
Don't kid yourself that the learning curve is not worth it.
Maybe he's using nix to parameterize his builds with the system architecture, but left it out because it didn't fit his idea for a blog post.
Go compiled with CGO_ENABLED=0 disagrees, especially since embedding was folded into the standard library.
> Which of course you'll need to setup, patch, figure out how to restart apps when they go down and on and on. There's no easy out.
Recompile, upload, restart. If the VPS server goes down, that is what their tech support is for.
> And how do you back that up?
Either use LVM snapshots and tar or sigstop/tar/sigcont, and rely on whatever embedded storage engine you are using to be able to recover from what will appear to be a bog standard recovery from unexpected shutdown. If you are nice, arrange for an API to be able to take a complete snapshot of your running data.
> Postgres, Docker, Infra as Code and Kubernetes are pretty much best practices on any development.
That is debatable. There is a whole lot to be said about the virtues of a single binary that has no dependencies beyond a vaguely modern Linux distro.
The primary difference between good and bad developers is good developers understand context. Every tool, technique, and practice were developed in a particular context to solve specific problems. Good developers understand the contex and limitations. Bad developers use the golden hammer or follow all “best practices”.
The OP calls all the technologies they dont use “bullshit”,so they are probably a bad developer. But a good developer might choose exactly the same set of techonologies for good reasons.
“Keep it simple” is a great principle, but it is also bordering on a platitutde because determining what is simple is not trivial.
Maybe that is the key, you are 10x if you are solving a real problem. But now you are spending that 10x on solving stupid problems with tooling, which is not aimed at the main problem.
(10x Programmer) - (9x time spent on tooling) = a 1x Programmer.
All that means is that sort of programming isn't for me. It doesn't mean I'm a terrible programmer.
That's very interesting, I've been looking for DB + web server + application server in one process solutions some time ago, but the only thing I found back then was Tarantool. I wonder which solution the author had in mind when he was writing this.
Even with that risk, for certain types of apps, this can be a way to avoid a database which might need a separate process to backup/dump etc, the skills of a DBA-type of person in some environments, adding yet another database to an enterprise "approved" list, etc. This technique presents a nice, clean view of real data and allows backups of just the live data directory itself. On SSD, it's also much more performant than you might expect.
It's quite possible to make the author's approach work very well with nearly zero long-term maintenance and no external database dependencies, as long as you are careful to manage the expectations and requirements.
I wouldn't have made all the same choices, but I admire the decision-making process.
Calling yourself stupid is stupid because the graph of those you->betterperson relations has no leaves, and is a totally closed loop. You’re the smarter, better benevolent version of someone else.
>> I’m not smart enough to figure out how to manage multiple repositories with shared code, so I put all my code in one repository.
> I'm not smart enough to manage a monorepo, so I put all my projects in separate repositories.
There's no reasoning behind the opinions other than "bc dumb".
I'd reverse that for myself. I am not smart enough to cross-compile my project or statically link, so I throw it in a docker container and call it a day.
Every time I decided to use filesystem as db for my program, a couple months down the road that turned out to be a wrong approach.
No, you're "brilliant." You know how to make something that's simple enough to get the job done.
You need serialization and deserialization when interacting with a DB, too. You're trusting a library to do most of that for you. This is the difference between a junior engineer and a senior engineer. As a junior engineer, you don't trust your own judgement and use a library. As a senior engineer, you don't trust the library writer's judgement, so you write it yourself.
What? No. As a senior engineer you realize that loading data from a database is a solved problem, has nothing to do with the core use cases you're addressing, and would be a massive waste of time to build yourself.
That would be really inefficient though. I vaguely recall database driven blog and forum scripts taking over because of how poorly the old solution worked at scale.
by the way, I'd say:
I'm too stupid to keep my configuration in order, so I use Docker.
> I’m not smart enough to figure out docker confiuration [sic], so instead I compile my web application into a self contained statically linked binary executable file.
That might be fine for something that needs to run once. What if it crashes? And what about everything else that's required. Database, centralized cache, queues, workers, load balancers.
You can't build a fully functional product without those and what are you going to do Install all of the dependencies on a VM, by hand? What if you have to migrate to a different machine for whatever reason? Are you going to write scripts to automate all that and test them and change them and test them again? What if your program crashes? Are you installing supervisor? Will you turn your nice little binary into a daemon and deal with the whole initd/systemd configuration? What about the development environment? Are you going to do all that again when you switch laptop or when you hire someone?
(I personally prefer a PaaS like Heroku, Vercel or Render but that's besides the point)
Complexity very quickly gets out of control. What if you could, instead, define what your project requires in terms of environment, services, dependencies, networking etc. and just give it to a magical (today we'd say "AI powered") DevOps entity that makes sure everything runs smoothly no matter on what machine it's running, assuming it has enough resources? Locally you'd just run a single command and the whole thing would magically come to life.
That's what Docker and Compose/Kubernetes are. A very convenient abstraction layer that makes it very easy to define your underlying architecture in a declarative way and have something else worry about how to get there.
And, let's be honest, it's not even that complicated. You get 80% of the benefits with (less than) 20% of what Docker etc. are capable of. I learned how to use it once 8 years ago and I'm still using it to this day, just occasionally googling the odd command or Dockerfile keyword. I've seen the same with git. So many people just memorize a few commands and can't be bothered to learn how it conceptually works.
I guess I just went on a long rant here but I'm really surprised how, in a constantly evolving industry like tech, there are so many people who are afraid of learning something new or to change how they do things, sometimes out of pure irrational spite or fear of whatever is new. Reminds me of the bit from Brett Victor's talk "The future of programming" where (paraphrasing) he says "binary people thought assembly was a bad idea and assembly people thought C was a bad [...]" etc. etc.