Magpie Driven Development (MDD)
nichesoftware.co.nz
nichesoftware.co.nz
Couple of points OP missed:
1. To prove that the new Rust service working fine, Magpie will stay extremely motivated, put 2x hours into it. You have to literally tear them off a keyboard. Now you got a devoted Owner, who is willing to take more responsibility of a project, isn't that wonderful?
2. Every rewrite serves as a good security review. Old unnecessary permissions will get abandoned, and only required accesses will be migrated. If Magpie is experienced and responsible, they will make it more secure than it was.
3. Every time I've seen a project rewritten (regardless of technologies) -- it became better. But only big companies can afford it. Magpie did it for you for free.
4. Motivation (aka pay less). If you see your team is bored, and updating their resume -- allow them to rewrite something on a tech stack they choose (and you can postpone promos till next year).
2. There is no security review if only one person knows that language. As well as that one person being new to that language.
3. mostly agree but as you state, not relevant to new technology decided upon by someone on a whim.
4. Do you really believe creating sections of the tech stack that only a single person is able to maintain will result in you not having to pay these staff as much? Those staff who you now have a critical dependency on and they know it?
Do you really think that developers shouldn't be paid what they're worth? Or that developers should hold down their worth so that you can pay them that much less?
which is why they shouldn't be allowed to artificially make themselves indispensable without bringing any additional benefit to the business
I've seen a semi-large project been rewritten 4 times at G, by the whole team.
When I came I literally couldn't propose a single thing to improve (there were only things that could be traded off for something else). It was very self-respecting to work on that project (though I didn't do any of those rewrites). And I respected managers there for giving resources for it to happen.
Yeah, that includes a lot of time learning. For that reason it is also exciting, because you’re having fun learning what you chose to and additionally fattens the resume.. that releases extra dopamine.
I'm not at all opposed to learning new shiny tech on the job. But it has to be a team effort, not one developer just deciding to forge their own path.
FWIW, I work with a team who has greatly modernized their skill sets together over the last few years. But they did so by choosing when to start with a new tech, and putting time in their sprints towards the learning efforts, training, tutorials, etc.
That's what horrors are made of. You absolutely do not wish your people overworked for any extended period of time. And if there is an drawback, hurdle down the road, instability, etc...
But if your brain is simply tired of doing motivating job, then it just reduces a level of motivation, until it gets enough rest. So it's a self-regulated system.
It let me think of how it works in my current team, where I enjoy being. Basically, we are all Magpies here, and the implicit agreement here is that every project is combination of code and experience (in form of utilized best practices, like writing README-s, leaving scripts for everything etc).
Everyone wanting to dive into another project is expected to be somewhat experienced in software engineering to know the best practices without asking (e.g. read the README, read the scripts), and everyone is expected to just read the code, it's all there.
After those two actions, there are little left for communicating, really. But I understand that probably doesn't work for most teams, as we accept only senior devs to our team.
PS: Also, I just noticed we never re-written a project here so far, mostly because older Magpies can explain why it's ideal for how it is, without resorting to "it works, so don't touch it" and "for historical reasons". It's written well enough so that even if I started rewriting, I would end up with the same thing.
Yeah and since the Magpie won't be around long (since it will soon find a new shiny job) someone will have to learn rust just to maintain this project.
We had these short Jinja templates that relied on long Python scripts to do the actual work of performing the substitutions, as the logic was too complex to handle otherwise.
Maybe the new maintainer will rewrite it in Zig or Haskell if they don't know Rust.
I've seen the opposite happen, a lot. In the late 00s, the trend in the industry was to rewrite all the things. Everyone was trying to move away from PHP-based solutions to something "modern."
Old code is battle-tested. That's valuable. Sometimes people reach for rewriting something that really just needs the plumbing updated.
> The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive
1. Cobol. Seems like most crowd here on HN thinks of it as a problem, contrary to what Joel says.
2. Security -- old code that's not being updated is often insecure. E.g. it requires old environment, that's not being supported anymore, so no security patches for you.
3. Motivation -- the older a project gets, the more you need to pay for its maintenance.
The manager was ousted or toned down a peg later on by a new, more willful manager.
I'll concede that a rewrite CAN improve a software and pull it into the new decade, but it's a big risk, a lot of effort, and a lot of re-discovery - always consider, if that amount of effort and money was invested in the existing codebase, wouldn't it have the same effect?
Another phrase that comes to mind is hype-driven development.
That how the prima donna started my current company. If not for the Magpie, I'd be working somewhere else now.
> Investigating, you find out that there’s no compelling business reason for the component to be written in Rust. Not much of a technical reason either. It’s just that the developer involved really wanted to learn the Rust language and related ecosystem of tooling, and they figured this work project was the best place to do that learning.
> Magpie Driven Development is when you have developers who want to use something new for no better reason than it’s shiny and fun.
That developer didn't use Rust because "it's shiny and fun". They did it to learn. Couple possible reasons:
1/ Mental hygiene. They're bored out of their mind with their current tech stack, and picked something different to prevent burnout.
2/ Principal-agent problem. They wanted to learn the new language, perhaps to use it for a side project, and opted to do some of the learning on company time.
3/ Professional development. They want to "sharpen their saw" and/or keep up to date with industry trends, and opted to do it on company time.
4/ CV padding. They consider changing the job, and are trying to gain some marketable experience.
5/ In-house improvements. There's a need for a change of tech stack in the company, of which the author of this article isn't aware. The developer did a project in new technology as a part of moving in this direction.
We can debate professionalism and ethics of any of those reasons as applied to a specific case. But I don't think classifying such situations as developers chasing shiny things is productive or insightful.
Certainly something that would probably get messaged at all levels of the company, not something you surprise drop on the rest of your coworkers and pull them kicking and screaming towards.
If a developer could create it alone, without anybody noticing, the company can replace it again if they learn it's not a good technology.
If someone else on the team wants to shake up literally everything with no consultation with the rest of the team on whether or not it's a good idea, now everyone else is getting dragged along for the ride without much consent, because now it has to be maintained and kept up to date. And it may not even be a good idea; individual developers are not infallible.
A service, isolated from the rest of the code, fully written in a week by a programmer without experience on the language. If that shakes literally everything, your place has a problem.
And it's the kind of problem a single developer or small skunkworks team can help you move out of, if they are given some freedom to experiment with architecture and settle on some good guidelines.
the least you could do is ask the rest of your team first. what's so contentious about working with your coworkers?
Managers don't have a clue on what you are set up to learn. That's not their job. If you have an environment problem, anybody more than 1 level up from you will at best ignore it.
> There's a need for a change of tech stack in the company
If your company is changing entire tech direction, managers should at least be made aware because they will need to plan for time. If this is being run Rambo-style by a few devs going guns blazing for the rest of the company, it's a serious problem, not least because the change in direction might be misguided without a big picture.
> Talking to the rest of the team is a good advice. Traveling the hierarchy up and then down expecting for permission is not.
You seem to be arguing against a strawman.
Before management can plan some migration, the technical people must know what else exists and how well it can be used on your environment. After your people run a few replaceable experiments, they can reach management and inform them of the costs and benefits of each choice, but going to management with "our stack sucks, we don't know how to improve it" is guaranteed to lead to a bad outcome, and defending a plan that you don't know how well it will work has a very high chance to turn out bad.
Dead-end Driven Development happens when the majority of a developer's time is spent working with technologies that hurt their career. The way you end up like this is often because the projects you were initially hired for have reached a maintenance state.
Instead of rewarding developers who have successfully completed the tasks they were initially hired for by assigning them tasks that benefit both their career and the company's goals, developers who meet management's expectations are often assigned to legacy projects that are riddled with obscure bugs, strange architecture, and bad choices. The end goal of these tasks is often to upgrade a system that uses ancient technologies in order to ease some customer's mind.
In this situation burnout eventually creeps in and the engineer faces two choices: either they start doing something that puts their career back in the right track (MDD) despite the consequences, or they find another job that improves their situation.
In the previous economy I would expect most people to pick option 1. In the current economy I'd expect an overwhelming majority to pick option 2.
In my limited experience, it takes up to 3 years to go from the start of a project to it being in a maintenance state. A lot of developers (consultants, self-employed, etc) have jumped ship before that, often around the two year mark.
So there's two - three years of work, it's 90% done, it's nearly ready to go into production... then what? Who does the actual work? Who does the maintenance?
I think anyone that picks a technology stack and architecture should stay long enough at a company working on it until they fully understand the consequences of their decisions. Too many people jump from one hype technology to the next, one greenfield start of a project to the other, but never have to deal with their own code and decisions ten years down the line.
And you KNOW a lot of software has a lifetime of at least a decade, sometimes / often more. There's critical systems that are still running with the decisions of a previous generation of developers, people who are now being asked to come back from retirement to maintain their old systems.
You can't call yourself a senior (or serious) developer if you've never had to deal with the consequences of your own actions.
Devs are not stupid.
If they wants to do Rust so badly, that's fine but they're now a trainer and need to teach the language to everyone involved and share responsibility of the new codebase. And if the other developers don't want to, then the developer needs to find a new job real fast.
Every few years, a new Head of Engineering comes in and says something like "let's use $TECHNOLOGY for this project" with the expectation that it will make the company more effective, I think. It's been GraphQL, microservices, clouds ("can we do this with Lambdas on AWS?" when we don't have any other significant infrastructure on AWS), and a bunch of other things.
On one hand, it's good for me to dabble in a lot of this stuff (learning the basics of GraphQL on company time helps my resume, after all), but I end up pushing back most of the time ("I don't think this is a great use-case for GraphQL. From what I understand, $X and $Y would be reasons to use it, but for this project, REST makes more sense because of $Z"). Meanwhile I'm just trying to get my work done and not make it a hassle.
If we had a good reason to use GraphQL (or any of the other things management suggests), I'd be into it, but I don't want to be wrestling with fitting square pegs into round holes for no benefit.
If you report directly to them, it might be worth a conversation to frame your team's work in a way that sells it as Cool New Technology that is being built.
I have a pretty good relationship with that manager, actually, and that's more or less the framework we use to sort of compromise. He gets his buzzwords and I get to get my work done.
I enjoy learning new things, don't get me wrong, but sometimes a CRUD app is just a CRUD app and trying to shoehorn other stuff into it makes it more of a pain in the long run.
I guess I just wanted to illustrate that this isn't only an IC issue; management can engage in MDD too.
A LOT of job adverts in the past years that I looked at had "blockchain" and "IoT" sprinkled in, not because they did anything with it, but because a lot of developers wanted to do something with those. Because "Java, Postgres, React" don't excite anyone anymore.
MDD - Migpie Driven Development
HDD - Hype-Driven Development
BWDD - BuzzWords-Driven Development
HNDD - Hacker News-Driven Development
RDD - Resume-Driven Development
CVDD - CV-Driven DevelopmentTo use precisely the tools and technologies that the project started with until the end of time.
"Don't have to rewrite the whole fucking thing with a new busload of developers every five years because the previous busload fucked off because they were getting bored" driven development.
Or more eloquently, Use Boring Technology: http://boringtechnology.club/
Ruby! No, wait, Haskell! No, wait, Erlang! I thought I'd never catch up to the industry.
While I learned a bit from Haskell and Erlang, I eventually realised that no-one hypes the 'boring' languages that people are using to get shit done every day.
People need to focus their CVs a lot more. Write the one or two languages you have the most experience in / with, and the one you really want to work with at the top - you don't need to mention you know (x)html(5), css, sass, less and scss - if you mention you have experience as a web developer I will assume you know these things and can learn the rest easily enough. And stop mentioning things you sniffed at in school or on your friday afternoon, if you haven't built products in it, it's not really important.
There are plenty of arguments in favour of allowing teams to use shiny new tech. Done responsibly, it can help with engagement, job satisfaction, retention, and recruitment.
I would say they were unprofessional and selfish. Not great as cultural fit neither.
Discipline should came first. Imagine you go to a restaurant and order a pizza and instead you receive sushi just because the chef wanted to learn a new cooking style..
Java and C# are safe bets. It probably won't get you the hip polyglot hype developer type, but I'm not convinced there's any company that thrives on those. Maybe for the POC or MVP, but long term you need a reliable tech stack.
I mean take Google; their backbone is/was C++ and they're moving towards Go, which is even more boring than C++ is.
From a business standpoint, that doesn't make a ton of sense. Development is meant to complete the business's objective of solving a problem for humans. It's not meant to make the developers enjoy their work more (though that's a nice bonus when it happens).
Chasing the new shiny puts the team in a perpetual state of re-learning how to do a known process. Maintaining and extending existing code in a known way is more predictable and more cost-effective.
> Development is meant to complete the business's objective of solving a problem for humans.
Sure, but every layer of management is making decisions constantly, including about what to direct development to do, that are in their own personal interest rather than the business' interest. If they can do that, the peons can (and should) too.
In my own life experience, the majority of my employers have genuinely wanted to work together to improve things for a large group of people.
I have never worked for an employer where it was obvious that everyone was only out for themselves.
I guess you mustn't have worked for very many employers. Or at least not in the USA. The sentiment is endemic where I am in Houston, TX and previously in CA
Most of those companies have been in the upper Midwest - either South Dakota or Minnesota. My latest employer is based out of North Carolina.
Interesting that you assume my experience is invalid because it doesn't match yours.
OTOH I still think that complaining that people at or near the bottom of the org chart follow incentives to achieve better personal outcomes, because it might hurt the company a little, is naïve at best—at worst, it's propaganda in the class war. "Workers, please limit your own career trajectory to help out the company. Here, let me (deliberately?) misunderstand your motivations and, as is typical, present developers as akin to children, not like the fully-formed adults in management".
Haha. No.
Your second paragraph... what is that in response to? How is saying that forcing other developers to follow the tech whims of some wannabe 10X is a bad thing "propaganda in the class war?"
The article, mainly, which reads very much like it's from the perspective of a project manager or engineering manager or something like that—as in "This morning you found out to your surprise that one of your senior developers" (emphasis mine), which seemingly-not-a-developer voice and POV it continues throughout. The (inevitable) infantilizing is here:
"Magpie Driven Development is when you have developers who want to use something new for no better reason than it’s shiny and fun."
Sure—except when developers do it because it's how you get ahead in this career. But that's something a rational person, not a toddler, might do, so we can't allow that that might be the reason. This sneering "oh, those childlike developers" tone is typical of management most places, when they talk/write about developers. I've even seen it in places that are pretty damn developer-friendly and tech-driven—it's subtle, but "oh, you know how software developers are" is deeply ingrained in management vernacular. I think it's a defense mechanism, mainly—and a very successful one, from what I've seen—because devs are paid like we're upper-middle-class but managers are, unconsciously or otherwise, very uncomfortable with us being in the same (or higher, maybe) social class as them, so take care that we do not rise in status on that front. I think that's how you end up with that kind of language being the norm, when managers treat of developers. "Well OK they make a lot of money but aren't they so immature and flighty and distractible? And god knows they can't be good at talking to people, or understanding how business works, or anything else that's our job. Now hold on, I've got really strong bad opinions about how this button should look and I expect to have my thoughts treated with deference and close attention. Why? Well, look at the org chart! That's why."
Anticipation and avoidance of this dynamic is why lawyers and doctors tend to resist having non-doctor and non-lawyer managers—and it's part of why those professions still, on average, carry significantly more social value than "software engineer". Pay's only part of the equation.
- Investors
- Workers
- Customers
- Community
A delicate balance has to be achieved by the company to meet everyone's need. If the company cannot achieve this, it will suffer one way or another.
However, characterizing workers as only needing to work with their latest interest misses out on a world of other (and sometimes conflicting) needs.
Not every developer in an organization is going to want to change languages and ecosystems every Tuesday.
Not every developer wants to work with whatever Linus, Netflix and other Googles have concocted for their unique reasons yet here we are.
this is itself a negative phrasing for the phenomenon. For one thing, Rust isn't just shiny and new - it has actual benefits.
Depicting "a perpetual state of re-learning" as negative/positive omits whether or not the process is actually improving or not.
> Maintaining and extending existing code in a known way
depends how debt-inflicted it is. In any case, this article describes a new service, so there wouldn't be the same issue as extending existing services.
In some very rare cases, rewrites can also be a good idea.
But choosing a new technology just because it's new and popular? Almost never a good idea.
Sure, but the given example states that as assumption. IRL how is that determined. The article states:
> Investigating, you find out that there’s no compelling business reason for the component to be written in Rust. Not much of a technical reason either
but doesn't detail if that was based on (or in agreement with) the opinion of the senior engineer who implemented the service, or the opinion of the author (whose role isn't that clear).
I'd also state that that while "choosing a new technology just because it's new" is bad (although could easily be a proxy for the features that the new thing actually brings - Rust isn't just a new rehash of existing languages, it brings unique features); "choosing a new technology just because it's popular" does have merit.
However, this approach doesn't work for everyone or for every business. There are many successful risk-averse businesses staffed by folks working ticket queues.
In the end, it's about the culture. An autodidact will struggle in a ticket mill. A ticket puncher will struggle in a skunkworks.
Please tell us how you incentivise your staff to deprioritise their skill growth in favour of your business? I hope to hear things other than "salary, benefits and bonus" but they have to be monetary.
Another option is to openly state during hiring that new technologies and approaches are not welcome.
Software developers: Act in their own self interest in a minor way that's not perfectly in line with he company's interests
Business bloggers: Hey now, cut it out developers! What a bunch of low-attention-span babies you are.
Is it really that common that people expect their current employer to subsidize their “skill growth” when they know they are just going to hop to another company in the near term?
Not judging —- just asking as an outsider looking in.
—edit—
Huh, just downvotes so I assume it is expected to be able to train yourself on company time…
Yes, that's common in professions.
> when they know they are just going to hop to another company in the near term?
That's a consequence of employers not willing to keep paying market rate over time. The only way to get back to that rate is to go somewhere else.
Also, most employers are expecting to hire already competent proffesionals, and invest nothing into training people in-house. So for each person that jumps ship, when they hire a replacement, they enjoy a subsidy of said replacement having secretly trained themselves on previous employers' dime.
Sigh.
I'm still the only person who really knows Lua at the company.
[1] I used Lua to write the testing programs for the product.
Of course it would be great for a company to be able to hire people who only learn new things on weekends and then 100% commit to and derive all their motivation from delivering the product, but I think that's quite rare...
In my experience, shipping requires compromise. For example, I have found it to be a very bad idea to schedule a ship project, based on shiny. It’s always a good idea to step back from the precipice, a bit.
The last two years at my last job, we were assigned to work with a startup, staffed by awesomely brilliant folks, to develop a particular kind of software, in one of the new, customized, LLVM-spawned, languages. I won’t get specific.
It did not end well. The devil is in the details. It was a great R&D project, but a lousy ship project.
The app I’m currently developing is being written in Swift (shiny), but using UIKit (b o r i n g), and traditional MVC (b o r i n g), instead of SwiftUI (shiny), and MVVM (shiny). SwiftUI was a thing, when I started the project, but it was still pretty raw, and I had barely started to work with it. The servers are written (for the most part), in PHP. They work great. If anyone has any doubts about the viability of PHP as a server, they should check out the “fishtank graph.” That was the graph that was released, recently, that showed PHP is several orders of magnitude more prevalent in Web server code, than any other language.
I call it the “fishtank” graph, because PHP is this line along the top, and every other language is represented by lines that look like gravel, on the bottom of the tank.
I may have been able to do this enormous app in SwiftUI, but there was no guarantee. There was no doubt, whatsoever, that it could be done in UIKit. If I write in UIKit, I write MVC. UIKit was designed for MVC. I’ve found that it’s always a good idea to use a tool the way it designed to be used. SwiftUI is good for PoP, KVO, etc., and different event-handling models.
It’s coming along great. The quality is amazing, as is the UX, and the performance.
I plan to ease into SwiftUI, by writing SwiftUI versions of my various UIKit widgets. That will be a much better way to learn up on it.
Microsoft tried to make this a thing 15-ish years ago with WPF/Silverlight ... is it experiencing a renaissance? Unfortunately I don't recall too much about its merits (or possible lack thereof) since the last time I read about it I wasn't sufficiently experienced with software architecture to understand what it brought to the table.
I’m ambivalent about it, but I have yet to do a full-size project with it (like I said, I feel that it is best to use classic MVC for UIKit development). I have used Presentation Manager (MS-defined) for serverside (PHP) code, but that’s different from MVVM for Swift-based stuff for Apple device programming.
We’ll see. SwiftUI is fairly radically different from UIKit. I basically like what I see, but I haven’t gone down that rabbithole, yet.
I’m very practical. I don’t get particularly distracted by shiny, and have no problems doing stuff like mixing cutting-edge techniques with 35-year-old patterns.
Of course you should have communicated more with the developer though to see what they've been working on.
The doing it in secret part is a far bigger problem.
But again, backend guys being backend.
I have yet to be wrong in immediately steeling myself for the worst the second I see the word.
I've found backend devs to be more conservative than front.
for example, any/all logic can be in the backend. A fn call fn(a), fn(b) or fn(c) in the FE can simply be fn_a(), fn_b() and fn_c() instead, but that would needlessly complicate the backend. If there is an object with N fields, and I want a subset, there are N*2 possible subsets, and whatever number of subsets I need (between 1 and N*2), I can either create that many BE endpoints, or a single endpoint that accepts a list of fields it should return - which is what I believe graph-ql is?
I personally see good use in GraphQL for 2 usecases:
- Aggregator over existing (!) microservices backend arch that has REST (plus maybe gRPC) in place, to allow FE people to move fast with their UI. In no way should new services be written in GraphQL, BE should still use REST (+gRPC)
- Prototyping/Fast clickdummy implementation; ideally a Snapshopt of an existing DB is used, and graphQL adapter for 1:1 entity mapping to allow 1-2 FE devs to quickly toss together a new FE (this should under no circumstances be used for anything even close to production)
Cases where it has been pitched to me where I see no increased value:
- BE team is too slow/lazy/defensive for new features (just have a GraphQL adapter and 1:1 map db entities to GraphQL entities -> please don't do that)
- having "typed" queries (you should just use typescript to strictly type your REST DTOs instead and optionally use a VM mapping)
Tl;dr: put your magpies on the top!
Going back to my current legacy .net project that i hate. Except there's no way I can stealth port it to rust or anything else so i'll just try to get the changes over with.
I hope you've at least had exposure to recent, proper .NET code. Don't judge the platform based on that project.
Right now I'm sometimes spending time maintaining something for a DCOM based semi-monstrosity and that does suck even if I've managed to clean out some old warts, luckily I'm also doing some more modern .NET stuff and that isn't too bad at all actually.
That said, new technology proof of concepts should always be done on non-critical systems first, and certainly not sprung as surprises.