Programming culture in the late aughts (2022)
morepablo.com
morepablo.com
That sounds so spot on.
We do have a choice though. Static HTML, server-side templating, vertical scaling and /var/log still exist and work just fine. We don’t need to adopt the newfangled techniques unless the use case calls for them.
It is just so, so much simpler to debug and trace what's going on. It can handle a surprising amount of traffic and work before any actual issues come up. Nothing changes from an upstream vendor, at all. It really "just works".
That said, I'm very aware there's a sweet spot and that this doesn't work past a certain point. I just think that point is so far beyond where most people actually get.
A simple make file and maybe a load balancer can easily let you launch large web applications used by millions of users.
I do not understand where people are getting stuck here that they need all of this complexity.
How many users do these apps have?
When I started my current job, the project was veering into microservices hell by a team that did not understand how to build a web application. A rough back of envelope calculation of potential customer base and what the product was aimed at (B2B in renewables) told me that even with world domination, the backend of the application would at most have 50k active users ever. So we didn’t really need 30 different microservices all which could scale independently, maybe just 2 or 3 larger services.
10 microservices to support a 2 page static html webpage.
We "needed" microservices for "Scalability", but we will never have more than a few hundred users at a time.
We can't have simple logging of our services, because all logs have to go to Loki, and Loki only does full text search. So first, learn how to query with Grafana's querying language in order to do any searches at all, then figure out what you will ever need to search, and turn that into a prometheus metric, but wait, now we have to properly compose our metrics for queryability....
I need to implement retry logic in one of my microservices. Would be easily implementable with an Amazon SQS or RabbitMQ queue. But we can't have queues, because they aren't stateless. We need to use Kafka because it has the ability to scale to billions of events, never mind that we never see more than 10k events in a 24 hour period. So now we'll need to write custom logic for retries and managing state outside of the queue, which means we're back to using database tables, which means there's no point to using kafka to begin with for our application...
Our services build in gitlab to docker images which then are hosted in the cloud and managed by k8s in custom vpcs. Sure, makes sense. But also, can someone show me how the developers on the team are supposed to be able to run the system locally? What? Can't do it? Replicating the production environment for local builds is now too heavy a lift and has been deprioritized by management? So how are they supposed to develop again? Oh, they'll just never learn how the whole system works, and continuously make changes they assume are isolated to their specific microservice without realizing they've introduced an error into the wider system unless the lead/manager catch it? Got it.
I could go on. Architects choosing technologies over listening to their engineers, in order to pad their resumes with technologies that look good to other architects. Committees of middle-managers choosing the 'safe' enterprise technologies instead of what is actually needed.
There aren't that many projects that have "millions" of users.
This sounds way too specific to be a hypothetical
I took the choice: to run from the web with my hair on fire, screaming.
Towards embedded, where none of this matters, and where the crux of the whole thing is really, really important: its the atoms, then the electrons...
We don't need the web if the user holds our interface in their hands. Surprisingly enough, the web holds an iron grip on that face/phone interface, but .. one easy solution to all the cruft, is simply return to the roots.
Yes, this requires hardware production capabilities. Its not for everyone, then.
But I tell you, 40 years into this industry, it was so refreshing to abandon all of the fancy front-end stuff and just get straight down to the embedded realm. Nothing nicer than having the machine - and the user - all to oneself.
Of course, this edge case gets discussed further in the article, but that is just one of a few things this old-timer could say about what went wrong with programming culture... (*cough* *cough*..should never have removed the compiler..*cough* *cough*.. this opened the door for all the weirdos to take over what should've stayed in the OS .. *cough*)
But, be careful, its requires just as much rigour as any other market. Perhaps more so, in some ways. Tooling and methodology are a lot more important ..
Then the answer is something like this.
* nginx
* Java
* Macroservice architecture
* Vertical scaling
* Server-side rendering
* Minimal javascript, no session cookies. These things bump the cost of each page-load.
* /var/log with relatively sparse logging in the sunny day path
Like this is a pretty strict design philosophy. You can probably do more JS and front-end stuff than I do, but it should be understood it's not free. My page loads incur for the most part a single request each. There are websites where each page load incurs hundreds of requests. It's worth repeating: Back-end requests are not free. They are cheap, but they are not free. When you have a dozen users and they incur a few thousand requests, that may be within what your server can deal with. When you have a thousand users and they incur hundreds of thousands of requests, it's probably not as fine.
Even (or maybe especially?) popular products like Gmail are guilty of this, sending easily over 200 requests. That's 700ms(!) worth of stalled requests in the best case scenario.
Context switching is expensive.
Struggling with this because even in a small app in a Linode box, I don't mind just cp-ing my bins but there's always other things/versions they need. Thinking of bringing in Ansible to help me out here.
Deploying feels easier in the container, Kubernetes world
All it is basically doing is a git pull, symlinking in the vendor code (gem) cache, doing all the rails preflight ceremony, and updating a "current release" symlink to point at the new folder. Oh and it restarts the servers (nginx, rails).
At a previous job I had an 8-line shell-script that did almost the same thing and it worked for years.
I have a dist-directory, in it is a directory for with each release version, and a symlink current pointing to the current release. To deploy a new version, I create a new directory, unpack each service, and then redirect the symlink ... then "systemctl restart service-name".
You got rid of a lot but where did the functionality for those tools go? I understand wanting to get rid of containers & Kubernetes, but did you rewrite/bake the functionality in to your app? or maybe not care about monitoring and logs temporarily for now?
Kubernetes and the surrounding ecosystem can create many of the problems they set out to solve.
Knowing docker these days is a huge boon - it immediately removes the "works on my machine" problem, and honestly the amount of docker knowledge required to deploy a basic even crud app is about 10 lines that can be copy and pasted from stack overflow by searching for "Dockerfile for X"
(Never bothered figuring out why since the script ran fine on its own and was replaced with a proper application shortly thereafter.)
Is that a good tradeoff? I don't know; it seems to me you can get 95% of what Docker gives you with 5% of the complexity. Luckily the ecosystem has started to realize that and better tools have started to materialize.
There are other tools too, such as containerd (and more I don't recall the name of first-hand), but I don't have first-hand experience with them.
I feel this one. At a previous gig we had an entire project to try and pick a logging solution, and none on offer let you do the one fundamental operation that makes using logs bearable; full text search.
Thankfully many DAWs, video editing, CAD, servers and other useful software saw the value in multi-core workloads.
It’s a thing… just not for web folks most of the time.
But go back 20 years, and you predate Linux RCU. The OS Developers weren't laughing at the problem then.
Arguably the Linux Real Time patches are solving a similar problem, which could be said to be extracting maximum parallelism out of the code. For Real Time this isn't so they can run in parallel, but rather so a higher priority task can interrupt a lower priority one with impunity. Real Time Linux still isn't done yet, although it's getting close.
I believe it was the DEC Ultrix dev team who, reportedly, amused themselves after hours by making transparencies of Linux source code, projecting them on a screen, cracking open a few beers and having a laugh at the idiot mistakes those college kids made -- mistakes, of course, that you can avoid by choosing a real Unix OS.
It took a while for Linux to actually be taken seriously outside of home-lab tinkering.
It took a while and big companies like IBM and organizations like the OSDL and FSG (which became the Linux Foundation) pouring many thousands of person hours and millions of dollars into it. The Linux kernel has plenty of independent "amateur" (not paid to do so) contributors but it's also got a lot of paid contributors for who it's their day job.
It's an awesome mix of contributors but I think there's a definite generational shift around the 2.2/2.4 to a much more capable kernel than 2.0 and before. Had Linux not gotten the financial backing it did I don't know that it would be where it is today.
- a way to avoid blocking the UI,
- tool to gain some perf. on new fancy multi-core processors,
- making the code much harder to read and debug, which often makes them a victim in a cost-benefit analysis.
There were in fact attempts for new concurrency paradigms (like Software Transactional Memory) which however led nowhere outside academia.
So what are some other ways of handling multi-core code that you were taught?
Typical audio software, for example, has been multithreaded since the 90s (simply because the audio callback and the GUI must run in different threads). And in the mid 2000s these threads started to run on different CPUs.
> I think this narrative, while conventional, is bollocks. [...] Ruby's weird semantics are credited for how one person as able to use it to make a world-changing framework; to call Ruby a "boring" choice now is a testament to how successful the right weird tech can be!
Sorry, but this whole section is bollocks. I've never heard anyone call Ruby boring, and I've had one too many methods monkey-patched in from halfway across the universe to ever call Ruby "boring". Basically no-one seems to use Ruby for new Serious™ Programming™ Projects™ anymore.
This guy called it boring when justifying its use for discourse back in 2013.
> Ruby isn't cool any more. Yeah, you heard me. It's not cool to write Ruby code any more. All the cool people moved on to slinging Scala and Node.js years ago. Our project isn't cool, it's just a bunch of boring old Ruby code.
>Basically no-one seems to use Ruby for new Serious™ Programming™ Projects™ anymore.
Yes, and in so witnessing this, you are demonstrating how non-bollocks the original bollocks was and now is. Such is the cyclical nature of things in our highly stratified industry .. its one layer of bollocks on top of a shit cake, after another ..
This is opposed to Rust from 5 years ago, or e.g. Zig today.
That's how I typically use "boring technology" anyway.
I bet the total number of people who will learn a C++ GUI framework between now and the end of time is smaller than the number who know one now.
I was happy to work on anything algorithmic. It seemed like there was so much CRUD and foundational communication stuff happening. All the programming literature talked about "business logic". Now with the rise of machine learning, a new grad might have a better chance of doing non-CRUD work now vs. then.
The funny thing is that I want to stop doing that and have other problems instead, because I'm just tired of everything that has to do with what (mostly) web development (and other distributed systems) has become and this incessant either layering more things on top of other things or adding extraneous things.
I'm currently re-orienting around a possible switch to game development.
(For those who might think I'm being impossibly naïve; I have plenty of lower-level experience and have worked at 2 game studios previously, it's not that foreign of a world. I originally left to do consulting because it paid significantly better.)
There might be something a kin to old school hacking in the indie domain.
I will also say, having worked at game studios before I believe them to have a much better barometer for what is extraneous, so even the live services teams at these places tend to be a lot more sensible than your average web shop. Your mileage may vary, though.
Games as a service and de-risking large projects I'm not sure is that relevant? Someone has to work on the engine, the networking code, and so on, and this work is largely not really going anywhere. There have been approximately zero innovations in the last 20 years that remove the need for being able to write lower-level code and knowing the internals of a game engine, provided you want to make something half-way decent.
Even if game studios somehow manage to not do the work I'm interested in, I might just take an extremely long sabbatical while continuing to make games and work on those things.
There is an alternate future where I also try to create custom solutions as a contractor in performance-critical niches and still work primarily with backend stuff, but I'm not sure what that would mean.
Microsoft had a very strong hold on education during that period.
I didn't learn anything outside of the Microsoft Ecosystem: Excel and Word, yes, but also Access and VB.NET when I was trying to get into programming.
Having direct access to good reference material was was critical, and often that meant hardcopy.
But now I'm close to graduating university. Is there any kind of job where this kind of software is still more common? PHP web development, sysadmin work, writing systems software or native software, etc.
Unfortunately sites like square and social media have taken over the small time web dev companies where you used to be able to get your hands dirty at all levels.
I'd probably look around at digital marketing companies. Your tech skills won't be "praised", because they favor creative over technical abilities. But they'll probably throw a wide range of technical problems at you and expect you to be "fullstack".
During the last 30 years, everyone's focus has drifted up the stack, to higher and higher levels of abstraction and higher and higher level languages, to the point where we are totally divorced from the electrons and realities of the underlying hardware.
Honestly, I think that this separation of concerns makes much more sense.
SOM-based development (system-on-module) in the embedded Linux world means that being able to productively pick and place the right software components is a highly sought after skill.
There is gold in them thar' repo's, especially if your shovel is a SOM and your mule is a functioning supply chain direct to the customer ..
Don't fall into the "good ol' days" trap. There were more than enough problems/challenges/frustrations to go around back then too.
Of course, I can fire up DOS, NeXTStep, or Windows XP and get to doing it on my own, or hack on open sourced codebases from the time, but the key component is the teams of people doing it, and the common wisdom that they had.
Rather than thinking it was heaven, it's a hell I'd prefer over our current one, so to speak.
I have a lot of nostalgia for writing BASIC and the feeling that I got the first time I learned about HTML4 tables and ASP.NET. But you could not pay me to return to those technologies.
This is true, but I don't think for the reason the author may think. Basically, parallelism in a single process is hard as hell and nobody can do it reliably. On the other hand, concurrency is easy (mainly due to not sharing memory between threads). Basically we just achieved the parallelism we wanted by utilizing concurrency within individual processes, but the processes themselves are parallelized at the OS/hardware level.
We still get clock increases (5 GHz was once the exclusive domain of exotic platforms), but memory latency also hit a wall, and that’s why processors add more memory channels along cores.
Just trying to generate html templates dynamically with a $db object interspersed in is still a pretty rad & pure experience. Not much between you & the machine. Not that I've actually done that in the last two decades. I used a little ejs though & wrote a little build tool to turn ejs template like things into standalone modules.
I hesitate to comment since I’m nitpicking a parenthetical, but this weird little bit of classism totally broke my immersion in an article I was otherwise really enjoying. Middle class people overwhelmingly can’t afford housing, but let’s shame them for trying to find at least one kind of affordable luxury in life? We’re the industry making giant luxury phones people “””don’t need”””, but people would rightfully call me an asshole if I shamed them for wanting one.
A giant car is probably the most harmful-to-others luxury indulgence you could pick (and a significant factor in why people can't afford housing). I'm all for having a hobby and spending money on your hobby, but maybe pick one that doesn't kill other people's kids so much. And I'm baffled that you're calling this classism and then talking about middle class.
International travel. People need to get to work, nobody needs to backpack through Europe to "find themselves" (besides perhaps Europeans).
Probably less harmful; the per-year CO2 emissions are similar (depending how long your commute is and how much travel you do), and travel has fewer of the other negative externalities (taking up road space and city space, killing people directly).
> People need to get to work, nobody needs to backpack through Europe
Perhaps, but I don't see how putting out an extra n tonnes of CO2 by driving an unnecessary giant car to work is any better than putting out an extra n tonnes of CO2 on things that are more directly self-indulgent. (If anything I'm more sympathetic to wasteful consumption if the consumer is at least enjoying themselves)
That is why American capitalism/consumerism is bad. Our society would be far, far better served if people exposed themselves to different cultures, beliefs, experienced world first hand instead of through talking head media. If more "found themselves" they'd know life isn't about work and money.
Some people are simply refusing to harm themselves when they choose a large vehicle.
Getting in and out of a "regular" sized car routinely injures me. The injuries are painful, immobilize me for weeks at a time, and occur several times per year.
I also know several people for whom being in a compact car causes anxiety or panic.
Do cases like this enter into your calculus, or do you just assume everybody you see in a large vehicle doesn't much care for the rest of humanity?
Seek medical assistance, immediately. This is not a joke, I am not intending to offend you- you genuinely should seek medical assistance as in: right now.
> I also know several people for whom being in a compact car causes anxiety or panic.
Now I will intend to offend: Yes, lets make the problem worse! Humans without cars should feel even more anxiety and panic! Especially tiny ones which you can't even see.
I mean, pedestrians do, right? I guess it depends on the local driving culture.
My best friend is blind, and this is precisely how he feels trying to navigate the world.
But yeah, living in America, car ownership seems unavoidable for most people in most areas.
It's not a medical problem.
I'm just tall and small cars are the wrong size for my body. Like choosing a piece of clothing, the solution is to choose a vehicle that fits my body.
I mean, I could try to have surgery to change my height, but that's wrongheaded, right?
There is definitely middle ground - frankly, I don’t believe you are interested in hearing it because you have already decided that obscenely large cars that kill children easily are the only way you can feel like you're experiencing comfort.
The way you react to that is pretty rude.
I think you're being disingenuous, I have no obligation to be kind to your face when you spin such dramatic falsehoods.
But sustaining so significant injuries from getting into a car larger than a Lotus or a Mini Cooper is concerning enough to warrant medical attention - we simply shouldn’t get damaged so easily.
How does it injure you?
Edit, more context: I've routinely gotten in and out of small sedans all my life. When I was younger, this wasn't much of a problem, but now that I'm in my thirties, it's become very frustrating and dibilitating.
Unless your small car is a Mazda Miata or some other triumph of miniaturisation, it shouldn’t happen.
Some examples: Lights always blind me, they block almost all vision, and have very poor vision themselves.