Why Software Jewels Are Rare (1996) [pdf]
cse.msu.edu
cse.msu.edu
Perhaps the solution to making software jewels less rare is to make it easier for software engineers to make jewels by creating more sophisticated developer tools, so that way the quick-and-dirty approach just so happens to align with "the right thing."
So many frameworks, so many concepts repeating their form every 3-5 years...and only with pure luck you pick the one framework that will be there for the next 2 years to come.
Nobody refactors. Everybody reinvents. Everybody abandons. It's an endless cycle of catching your own tail.
If you read a software engineering book about patterns and architectures you'll quickly realize how many framworks reused a concept that is 30 years existing already, yet selling themselves as the hot new cool kid on the block.
I don't understand why lately architecting software doesn't result in refactoring anymore. I think that only through refactoring you can really understand and learn from your own mistakes.
> Everybody reinvents. Everybody abandons.
There's that saying about haute couture - "out of fashion" is just highly avantgarde.
That's why I build CGIs. They as up-to-date today as they were 15 years ago.
The process is not going to be commercially viable for many years simply, and this is why crap persists because that crap does solve real problems today.
You can check out the project: http://www.adama-lang.org/docs/why-the-origin-story
It's going to take a long time until I get to a 1.0 release.
Will it be used to build online board games (like an engine) or to "generate" one?
There's also another nice tool for simulating games. Not really in the scope if your project, but around the games topic. https://machinations.io/
As a developer, I can code the back-end for a board game without thinking about the failure modes for disk/db/network and I code as if I'm a single process (you know, the way we coded as kids). The mental model mirrors if you gather all your friends to play a game with the same keyboard and mouse.
As a service, it is a combination of EC2/Lambda for compute, Ably for notifications and connectivity, some DB for storage of state, and workflow for orchestration of people transactions. All within a single package.
As a game designer, my hope is that I can make it easier enough that the rules backing a game can be built and then evaluated with simulation and artificial intelligence. I have rough prototypes of this, but I implemented the back-end of Battlestar Galactica and had a random bot just play it. That's a remarkable achievement in a way, and now the question is how to train a system to search for patterns in looking at what decisions are made with a JSON document as the state of the game.
Commercial players are finally entering this space, with Roblox getting more attention and Unreal/Fortnite moving in this direction.
Are you sure you need a new programming language? It seems like a DSL or a framework would do for the problems you're trying to solve, and you'd save time for yourself by not writing a standard library, and for your users by letting them use a language they know.
A comparable framework for some of the workloads I envision would be https://akka.io/docs/ , but then you have to live within the framework or bad things happen. My idea with the language is that I eliminate all the bad things and keep thing simple.
Where I've scratched my head on this problem with existing solutions is that you need a lot of things to align. For instance, making it possible to coordinate players easy requires having a language support async/await semantics. This lets the server coordinate you playing a card which another player can interfere with, and another player could interfere with that.
The challenge with async/await in modern languages is what do you do with the process if the future takes... hours to return? How do you handle upgrades? operations in the cloud of cattle? This is where I started to re-invent a logger for board games, but then you must treat all your game code as deterministic. If you don't, then you are in for a surprise as the server replays the log. Instead of trusting myself, I'm forging the standard library such that everything is deterministic.
The final component that gives me confidence in needing a language is the reactivity. My language is basically a text-based Excel engine with tables, objects, and global values. This means that when a player makes a move, I simply need to manipulate the state appropriately and then everything bound to that state gets computed on the fly as needed. The state also is annotated with privacy controls such that important information doesn't leak between players, and this is something that is really hard outside of a language context.
All in all, I don't really need a language to ship a single game, but if I want a platform to enable anyone to build a board game, then I need to make it accommodating.
That's because craftsmanship in software has massively diminishing returns outside of a few niches.
Unless you're writing some VERY highly optimized kernel code, some physics logic that makes a simulator run correctly, etc... you're writing code that gets the job done until it's replaced.
A login form on a website works, until the business requirements for it change. Is it worth going back and understanding all of the fine details to make sure your changes don't break anything, or should you just rewrite it from scratch in an afternoon?
This is how I feel generally about craftsmanship. There are some places where it's warranted for long term stability and incremental improvement. But the power of software lies in it's mutability (else it's better and faster in pure hardware), so the vast majority of software will break due to changing requirements or dependencies, and will be cheaper and less buggy to just replace. There's no point in finely crafting these replacements because eventually they'll suffer the same fate.
linguae used the word craftsmanship, which covers more than just performance. Correctness is another obvious aspect. (We still write production code that is vulnerable to buffer-overflows, for instance.) Standards-compliance is another. Wikipedia has a list that applies here. [0]
This is analogous to other engineering disciplines. There's no market for supersonic civilian aircraft, but there are plenty of challenges besides speed to keep aeronautical engineers busy.
Even if you're developing 'just another website', poor craftsmanship might result in downtime and loss of customer data. It might also result in longer onboarding times for new developers. The list goes on.
[0] https://en.wikipedia.org/wiki/List_of_system_quality_attribu...
Assembler code for old 2D game consoles is unclear, partly because it's not structured programming, partly from rushing out the door and/or generational cruft, and partly from talking to awkward hardware interfaces, and partly from optimizing performance. I've seen C code with #ifdefs to pick different code on x86 and RISC CPUs, which might've made sense as an optimization with CPUs and compilers from 15 years ago. Duff's Device is faster on older processors, but sometimes slower on new processors and compilers.
From what I've heard, microservices address organizational problems more than technical ones, and dependency injection may be more useful in large enterprise projects (not sure, in my minimal experience it creates complexity).
I think that all of performance, clarity, elegance, and replacability (possibly linked to modularity) are values, and software craftsmanship can encompass one or more to varying degrees.
Am I misinterpreting your comment, or did you just list 3 distinct items?
1. easily understood and simple code
2. code easily maintained and extended by corporate employees
3. fast code.
In my experience, points (1) and (2) are one and the same. No tension at all. Easily understood and simple code is easily maintained and extended, by anyone.
When I write code, I never try to cater to some junior idiot. I write for myself, 6 months later. Yet people who see my code often compliment how easy to read, use, and extend it is. Junior and experts alike.
Even with something like a login form. If you keep on improving it with what you learn whenever you need to re-implement it or extend its requirements you end up with at least a re-usable "architecture", and most of the times you probably end up with big chunks of code that you can use, either the JS part, or html, or back-end logic. You now have a nice show/hide password field, your html is semantic, issues that were on the first quick and dirty were ironed out a bit with every iteration.
> But the power of software lies in it's mutability (else it's better and faster in pure hardware)
This doesn't seem at all right? The power of software lies in it doing what it's supposed to do, not introducing problems that might be costly or annoy users to the point they don't use it anymore or something better comes along and not doing what you don't expect it to do. And at the same time you're saying its value comes from mutability but then saying that it's better to rewrite it?
I also think that if you can rewrite it in one afternoon, perhaps it's ok, but I doubt the things where some semblance of quality and effort matter is when whatever is the software isn't rewritable in an afternoon.
> so the vast majority of software will break due to changing requirements or dependencies
This also depends. Exhaustive testing for the sake of testing is not good, but a good test suite that exerts the overall functionality of your software is almost always good and allows for much faster iteration down the line and pays itself in a short amount of time or iteration cycles.
You do realize that "mutability" means "ability to be changed", correct? It's value comes from being able to be rewritten piecemeal, as opposed to hardware where you'd have to design a new PCB, manufacture it, then snail-mail it to all the places it should be, then install it.
>> so the vast majority of software will break due to changing requirements or dependencies
> This also depends. Exhaustive testing for the sake of testing is not good, but a good test suite that exerts the overall functionality of your software is almost always good and allows for much faster iteration down the line and pays itself in a short amount of time or iteration cycles.
What you said literally had nothing to do with that except you quoted from my comment.
It's not any better. And has the added challenge that since it doesn't need to make money, sometimes projects keep going when they aren't doing very much useful on the order of the resources put into continued development. "Sunk cost fallacy" can be enormous.
Software is just hard.
Another context than the software industry is the "enterprise", where software is being created to support some internal business process (in the broadest sense of 'business process'). That has it's own pressures. But that software too ends up looking much the same.
The OP article really struck me, I have to think more about what to take from it, but I think there's a lot there.
This article to some extent suggests a kind of "software pessimism" -- is all software just destined to be a garbage pile? (Although the authors say "we can and must do better", the text doesn't leave me confident in that).
Software is just hard. It's also eating our world though. If it's all destined to be a garbage pile, we're in trouble -- forced to spend our lives fighting with buggy, hackable, inhuman software that acts like it wants to kill us.
one way or another, we probably need a lot less software in order to make it better software. That seems pretty unrealsitic, I realize.
I've thought before about, what would a "slow code" (like "slow food") movement look like?
He also has millions of dollars, thus, years to devote.
We might exploit this example to say about a slow code movement "people in the slow code movement would recognize at the start of a slow code project that they will be working in relative secrecy for several years".
> Perhaps the solution to making software jewels less rare is to make it easier for software engineers to make jewels by creating more sophisticated developer tools, so that way the quick-and-dirty approach just so happens to align with "the right thing."
That would go a long, long way, but culturally we would need to embrace quality over profits and I don't see that happening any time soon.
I do believe we will make some gains in the tooling due to open source/collaborative work a lot of us do for fun. Granted, without the cultural shift, it'll be a slow, gradual change.
It didn't have to be really good: It was better than any other digital music store out there!
Apple had some good UI ideas in the past. For a fleeting moment, they were the best. But eons ago they sold out the user to sell him a lifestyle/subscription system built around keeping people from leaving. This vastly compromised their UI designs to the point of making them frustrating and sometimes futile.
https://discussions.apple.com/thread/250315803
Software that loses data it fetched or produced is an accident. Software that actively search for and deletes data it has nothing to do with (in backups!!), is malware. People went to jail over such things.
Why specifically do you hold iTunes in such low regard? The bolt-on, beyond-music functionality has always sucked, but I didn't find it wanting from a music management POV. (Although I never used it for ripping. dBpoweramp for me, thank you.)
As far as the raw UI, they were shown to the door by Zune which was a massive failure! Couldn't Apple with its money and music business success have made Zune or something even better? Of course they could. They had an early lead. My point is they chose not to because openness and control in music (or anything else) is antithetical to their business model. This adversarial view of users is also what lead to terrible iTunes bugs like eating your music, which others also recall well.
Then how would you know how good or bad Apples software UI/UX is if you have only one frame of reference? It's good or bad compared to what?
I don't think you can easily make that claim.
The classic Mac OS and pre-Big Sur macOS were the best environments I’ve used from a UI/UX perspective, with Windows 95/98/NT/2000 being worthy runners up.
Then you need people who are willing to pay for the product that the craftsman creates. Corporations generally don't value craftsmanship. Consumers are not willing to pay the price. In the past, you had royals and aristocrats who would pay for works of craftsmanship for prestige. Who are the modern patrons of craftsmanship? I would love to see some Silicon Valley billionaires fund software craftsmen, but very few of them put their money to work in such impactful ways.
I have not found the same willingness to pay for quality amongst consumers, though. Counter-intuitively, the sort of hyper-knowledgeable "pro" consumers I come across are typically less willing to open their wallets than people who know nothing.
When we write software it lasts for years. It's not like anybody says a year in "oops Django was a bad choice, lets try this again in Node."
We want to care about craft, but we work on the same application longer than it takes to build most buildings. We more than any other craft are saddled with choices made years ago that were made with way too much information.
It's interesting to note that the companies behind half of my favorite programs were bought out and began distributing malware before shutting down.
And I can't substitute my GUI shell that easily so I keep the default and suck it up.
BLAS and friends would be a good candidate for old software.
And I'm not a zealot or anything. I just like this aspect.
Any leads?
If anyone's interested : https://www.sidefx.com/
The industry standard Autodesk's Maya, on the other hand, is a bloated POS. I've quit two jobs that made me use it and have taken it off my resume. Thinking about using it for doing my job makes my hands curl in disgust and pain.
Unkindly, it looks like a kludged together unorganized open source monster with functionality smooshed every which way without consideration for how it actually gets used. Slightly better than Zbrush, which is a ui abomination.
Overall I bet it's fine and dandy if you take the time to learn its eccentricities. I fear it's very low on discover-ability due to the lack of strongly opinionated and efficient visual metaphors. But I haven't taken the time to dive into it to make a full and comparative review. Nor would I. I'm old and don't have the time or ambition to do so.
tl:dr Blender seems cluttered but ok. Use Houdini if you can, but Blender if you must. Death to Autodesk.
Rarely are technics given first seat at companies. We talk about being product organizations or engineering organizations but I feel like in both cases there is still enormous political processes that most engineers face in their urge to do the right thing. And sussing out, helping the company internalize which bits are of lasting value, done well, &bought be enshrined &built upon, sharing & syndicating these notions out: it re-becomes political almost immediately.
I had a rather long-winded reply to "software is a losers game" that surmized a lot of the challenges & environments of software development together[2]. I think it helps explain some of why the field so rarely, clearly excels.
[1] https://www.tlt21.com/why-software-engineers-should-become-c...
what I think happens is that companies try to hire the best talent they can find for their money and tell fairy tales how they are modern and foward-looking company with the vision that is going to change the world - and of course devs will internalize that and will try to apply craftsmanship to their daily work, and then you will have problem of shipping on time and on budget.
if managers told exactly and upfront that the products must be hacked together and asap and the quality is NOT the goal, that the goal is to ship shiny new thing quickly to impress VC suits in next N months, and get their funding check and continue this loop until the company finds its exit in form of IPO/acquisition/etc - everything would be honest, upfront, efficient, and conflict free.
On the best places, a lot of the communication problems come from the fact that managers tend to willfully ignore the fact that low quality is a losing proposition at the long term, and developers tend to willfully ignore the fact that short term only gains have value.
On the worst places, communication problems tend to come from the fact that management wants to artificially prop the numbers before they get a way to jump ship, defrauding everybody else, so they simply can not be honest.
All the bonsai-like rules about software jewellery have zero value in those industries.
What those company owners and management departments care about is their main business line, as long as those boxes keep shipping or people keep buying their stuff, they couldn't care one second how the sausage is made.
It is easy to blame all the major consulting sweet shops for the quality of delivered work, while in most cases what gets delivered is what customers (e.g. management board) are willing to pay for.
Just ranting out loud, not targeting anyone.
More generally, I feel like tech needs some breakthroughs in the "create more value than you capture" (Tim Orielly 2008) sense, where it is tech itself that is the product, rather than a product built of tech, that is trying to be created. Exposing & opening the tech, making the tech accessible. This is a far off point, not one I'd want to reduce the discussion to, but I think in general we've had a lack of technical leadership that's deflated our general sense of how core, critical, & powerful this general drift is.
I very much agree, that it's all very complicated.
We don’t always do good enough... some times we need extremely well done due to the sensitive nature of the code.. it all depends.
There seems to be a polarized view of "jewels are worthless, just ship" or "we need more jewels". Both perspectives are dangerous.
Software-jewels have a place, but you have to be selective about what code needs to be that level of quality. And like so many decisions in software development, it's about the trade-offs.
If we're writing experimental code with a low shelf-life or internal tooling, we are better off writing code we expect to throw away completely. Writing high quality code here will take time away from making code quality high in areas where it's important.
On the other hand, if there are some subsystems critical to our business--we want them robust, extensible, and fast--they are good candidates for "jewel" quality. But you have to allow the time needed for a potential rewrite and more thoughtful architecture.
Except Google, Microsoft and Apple.