We at $Famous_company Switched to $Hyped_technology | Hacker News Reader

We at $Famous_company Switched to $Hyped_technology

saagarjha.com

258 pointsWayToDoor99 comments

The post is great, but it's important to remember it's only a first step in $Famous_company's story. Some rewrites are resume padding and PR to get more hires, and a talk a big conference, but are rather harmless. Others succeed, and were good ideas. But other times, the result is a big failure. One of my personal favorites involves a $Hyped_technology having a well defined trap that then leads us to another article, 1 to 2 years later, explaining how they migrated away from $Hyped_technology, often to something boring.

But the really sad outcome is that $Hyped_technology is a big failure, but $Famous_company spent too much effort, and $Famous__technical_person too much of their social capital, to want to talk about it. Then all kinds of little startups that have heard the talks, read the blog posts, and then decide that if the solution was good enough for $Famous_company, it must be good enough for them!... except that it wasn't, and a lot of expensive engineers spent a whole lot of time making things work again, abandoning $Hyped_technology altogether. Unfortunately, the landmine documentation advertising the decision is still everywhere, including getting referred to by sales engineers of $Hyped_technology.

Similar to the USDA/FDA and the push for the "low fat diet".

Except that wasn't a mistake, that was due to an intentional misinformation campaign.

So are you saying that new technology is not hyped with misinformation? Write once run anywhere?

Write once run anywhere, was Java's slogan in the nineties. And I still find the amount of abstraction JVM provides fascinating. At least since it became well defined memory model, you can write complex pieces of software and they behave remarkable identical on a lot of different platforms. I am aware that the API border to the operating system has always been the weakest link regarding practical portability. But I wouldn't call that misinformation.

The mention of accessibility at the end reminds me of this very celebrated post from 2015: https://engineering.flipboard.com/2015/02/mobile-web.

The article described Flipboard's migration from a browser DOM-based app to an entirely canvas-rendered one, which made the whole thing invisible to screenreaders.

There was just a throwaway line at the end on accesibility: "This area needs further exploration."

> Ultimately, however, our decision to switch was driven by our difficulty in hiring new talent for $UNREMARKABLE_LANGUAGE, despite it being taught in dozens of universities

This was an actual pitch point to switch language and stack in a previous company.

At some point the CTO+HR dept made the calculation that it would cost less in the long term to rewrite our services in a new language and bring in cheap/abundant devs, than to bring in more people familiar with the current stack and continue scaling it, as those had an higher average salary on the market.

This also just came up recently at my company, that we should maybe rewrite because our technology hurts us in terms of recruiting... I'm not sure the people considering that realize how costly that decision would be. Is it really worth the millions and millions it would cost to rewrite into something considered more sexy? Probably multiple years of incremental work to get it done, and a massive hit to developer productivity and maintenance costs along the way.

I'm also reminded of when Twitter claimed that it was Rails' fault that they didn't have better moderation tools. One of the most hilarious excuses I've ever heard.

I bought a gps app several years ago after testing it out on my phone. It had everything I wanted and was fast and smooth on my phone. Next year I wanted to use it for a long trip but then it was updated, they had rewritten it from scratch. Most of the functions I liked were gone and the thing was slow like I have never seen. Battery drain was big, the phone got scorching hot. Now after 8 years it still does not have all the functions it had when I bought it and it's still too slow to be used even after changing to faster phone a couple of times.

And still people are so fast to convert to latest frameworks and do complete rewrites. Most of the time recreating bugs they thought they got rid of many years ago. Or just make loads of new bugs. Or make it so slow on old phones that it can't be used any more.

Evernote is pretty much ruined for me because of this. From a lovely, snappy native app to an electron monstrosity that's slow and barely usable.

Lucky for them their only competitor is Notion, another slow js app.

Probably they decided to ditch their native app for a cross-platform web based resource hog.

Yeah, think they said it was an electron app now.

Anything that was reliable in up-time suddenly becomes legacy I guess. It also forces Cloud Service Providers to pivot to supporting the new flashy solutions, and suddenly "old faithful solution" ends up looking like the old frail man in the corner. It's a never ending lesson in building sand castles...

I think Silicon Valley illustrated this concept best. Just posted this clip a day ago to my twitter, so I too have felt the searing pain of being an IT architect and watching the right solution decisions get snowplowed - https://www.youtube.com/watch?v=ySDX02WD0og

You left out the most important part. Sexy is a moving target. By the time your multiple years of incremental work moving it over are done, the new technology will be old and unsexy. Heck, in some cases the very fact that you(uncool company) use it will cause it to be unsexy.

Well, it depends on your goals.

Recruiting cheaper programmers will probably look nice on immediate results for a time, assuming the rewrite progresses well.

Whether the flood of bootcamp grads which, thanks to law of large numbers, will also contain a lot more bad choices and overwhelm your filtering ability, be actually good stewards for continued existence of things possibly critical to the business... is a completely different thing.

> Recruiting cheaper programmers will probably look nice on immediate results for a time, assuming the rewrite progresses well.

Possibly, but on the other hand, "adding more developers to a late project just makes it more late".

It all just feels like a colossal rathole to me. I'm feeling exhausted just thinking about it.

Possibly, but on the other hand, management usually sees this as a success story for HR ("Managed to fill every spot in days"), and a failure on the dev team ("The project slipped by months, despite getting all the people they asked for")

The not so hidden assumption that came with the pitch was that a bunch of the well-paid senior staff would leave for greener pastures (they wouldn't stick to maintaining a "legacy" stack). The savings were coming from both incoming and outgoing flow...

Or alternatively, there are some senior staff that are just plain bored with the current stack, and want the chance to play with something more exciting to them. Both are possibilities, depending on where the idea is originating.

I’m guilty of doing this at my current company.

I am a creative builder. There’s something inspiring about building things in a new format, the same way an artist might explore art in a new medium.

I get frustrated with the parts of the company that use “technology X” because it’s a “core competency.” I’d be more accepting of this argument if I didn’t notice a strong correlation between proponents of this strategy and people who often do bare minimum maintenance, purely market request driven, nothing inspiring or innovative or game changing. Then competitor does something cool next door and tue blame game starts as the pink slips issue.

Evolve or die.

There is of course the flip side of this coin, and the yen-yang of balance is what makes for the best strategy.

>Evolve or die.

The value in one's application is the domain specific part (the product). By switching the tools, which almost always are created outside of one's organization, you don't enhance your product.

With each rewrite you risk the advantages of your existing product and you will still be a consumer of third party language/framework, just another one yet without experience.

One interesting outcome of Twitter's claim was other startups moving away from Rails towards other stacks.

In my not infrequent rants inside the company I work for, I stand on one side of a major philosophical disagreement about old languages. The proponents of "keep coding in this old language" are also the people who want to retain the 5-10 year old OS, DB, and assumptions about external threat models and technical debt.

Suffice to say nobody I ask outside of this company wants to adopt this model: Retained old platforms plagues the banking and finance sector. A 1950s OS called LEO was emulated by a 1960s OS called GEORGE which in turn is emulated by 1970s and 1980s ICL/Fujitsu mainframes, all because the core finance app is in COBOL and two greybeards per bank are on $mil to keep it working...

Banking...

- Does that COBOL code do the thing(s) it's supposed to in a reliable and 100% predictable way?

My guess is : Yes it does.

So let's go down your path. You rewrite that COBOL program using your hot new recent language of choice. How many new bugs will be introduced? What other risks are there in this new language; CVEs? RCEs? Simple coding bugs which could cost that bank millions/billions/trillions - or just as bad, the bank's customers their savings/livelihoods?

Which New Hotness language would you rewrite that crusty old COBOL application in?

Rust? [0] Golang? [1] Something else?

There are times when it's better to leave the New Hotness to some new application rather than trying to replace a known reliable/predictable system with New Hotness. Especially for banking.

[0] https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=rust [1] https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=golang

I guess you're on the other side of the fence. I don't dispute that the code can be fine.. but Y2K is strongly applicable here. I would personally worry if my financial institution depended on a bytecode interpreter to run Cobol for a dead OS rather than reimplement.

Rust might be the sexy language but surely Erlang or Haskell would be the wise choices?

I mostly agree with your take. To jest, the other side of the coin is when you have a legit and critical reason to move your stack forward, it will be way harder than if you moved along with the times. The same way moving from a Solaris system from 6 decades ago comes with heavier costs and is risquier than from some old Redhat.

For the bank example, if you’re given another system to sync with and you’re still full COBOL, I’d assume it’s just a whole world of pain for years and years. Meanwhile the other banks move forward and integrate newer systems way faster and in more secure ways than you do, directly affecting your business.

Highly unlikely they will integrate newer systems in more secure ways using a new, unfamiliar language. In fact, many of the new platforms are becoming more of a black box that developers are unable to fix or understand when they need to fix or even recognize critical faults.

Big bang rewrites are dangerous, gradual migration is a better plan for larger systems.

> Big bang rewrites are dangerous, gradual migration is a better plan for larger systems.

I think we're on the same page. Yes, it needs to be gradual, and devs need to become familiar with new languages, which is why I am proponent of slowly "modernize" old stacks even if there is no immediate need to do it.

Every couple of years someone at work rewrites the email sending code in a new sexy language and reimplements the same 7 bit encoding bugs as well as other subtle email encoding issues. This happens even with ample documentation and code comments about why specific conditionals are set.

If a bug has been around long enough and often used, it is an undocumented feature that becomes a documented feature in the next rewrite.

I've been through this multiple times.

By the time you have something that is as good as what you had before the rewrite in the new programming language, the cost reductions will not be worth it.

This is akin to my pet peeve dev team blog post:

We had to scale our $STANDARD_LANG application. Turns out it wasn’t fun.

But you know what is fun? Writing tiny, non-essential MVPs in $SHINY_LANG. We haven’t tried scaling our $SHINY_LANG apps yet or deploying to production, but we’re excited about the future and we’re enjoying coding again!

It's even "funnier" how then they try to take $SHINY_LANG to its extremes and expect it to work flawlessly. And they don't try to understand it or work around it. "No optimizing, only performance."

I remember some people getting bit by some Go stuff early on. Things take time. It will get there when it gets there. Then they come to HN and create a "why $technology sucks" when they a) didn't read the manual b) didn't match their requirements with the capabilities of the lang c) had unreal expectations

Don't buy the shiniest Tesla then use it as your building contractor car. Especially when you like to overload the rear axis and pretend that's just not an issue.

Sums up the last two years of employment prior to my 'job elimination'. The company was acquired by a VC who brought in a new CIO. Everything was the cloud, webservices, etc. I contended that a stable application with 99%+ uptime, a team of six developers, and peak concurrent users of less than 10K was not an optimal candidate for webservices. We had had the application load tested by a very reputable firm and they asserted the platform could sustain nearly a million concurrent users. Given the nature of the business, even a 100 million customers would not lead to that level of concurrency. Typically, our concurrent users peaked at 0.3% of our customer base. So I have been comfortably retired for the last two years. And the company? Well, they 'deprecated' the application and leased a 3rd party solution that had all the right buzz words on their marketing material. So it goes.

Yeah but everyone who learns $FLASHY_LANGUAGE and $HYPED_TECHNOLOGY can put that shit on their resumes and after they've driven the startup into the dust they can land a job at $FAMOUS_COMPANY.

And really it'll be better for your career to embrace this and learn the tech stack by fucking up some unsuspecting startup and then bouncing somewhere big enough to actually use it.

Reminds me of this semi-related classic that elaborates on the urge to rewrite things from scratch:

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Joel never spent 6 months working on a very simple feature request that should have taken 5 minutes and a half dozen lines of code.

I'd like to see him say with a straigh face that it is a good idea to stick with VB6 indefinately.

There is a lot of wisdom in his post, but it isn't always practical or possible to evolve a legacy code base.

Every code base is legacy, in one dimension or another.

Google was a from-scratch rewrite of search engines. Facebook was a from-scratch rewrite of MySpace. Slack was a from-scratch rewrite of IRC. Chrome was a from-scratch rewrite of IE. The list goes on and on and on.

Any of the previous incarnations of those ideas could have iterated into success, but it turns out it was easier to spin up a new product (even a completely new team / company / culture) with a clearer focus / path to improvement / culture than it was to turn around a barnacle-laden ship. Sometimes a rewrite / rethink is the best way forward.

This is a wildly incorrect comment. Literally none of those were rewrites.

It’s a completely different scenario to do greenfield implementations in existing categories vs managing the literal replacement of existing software with the same thing written in a different language.

It’s not an incorrect statement. I’ve been part of successful rewrites. The key is to be your own disruptive competitor. Disruptive competitors are rewrites of your product that you yourself were either unable or unwilling to do.

This Joel blogpost is always trotted out as some kind of gospel, and I think it’s stymied a lot of clear thinking around the possible merits of a rebuild.

I don't think this is what Joel is referring to at all. He's talking about literally codebase rewrites. In all your examples, there was not existing code base that was rewritten. A company tried to innovate on an existing idea, yes, but that's not what Joel is arguing against here.

Plus in some of those examples, they leveraged open source libraries, so they reused preexisting code.

> Chrome was a from-scratch rewrite of IE.

Hardly. It used several external libraries, including those from Netscape and Apple, who also made web browsers.

All examples of survivorship bias. You don't know of all the "rewrite" failures because they failed.

I love that it ends with “We are hiring.”

All these stack write-ups are always either recruiting posts, or self-branding by the author.

To be fair i see more big companies stuck with outdated inefficient legacy technologies rather than companies evolving in line with technical progress.

Booking.com, partially all of the faang etc. to name a few

Not too sure about it. What I mean is that most of those software companies rise to the top in the last 20 years or so, and a lot of what is deemed worthwhile technology already existed at the time, be it Java, Ruby, Python or else.

What did evolve is the arrival of the client side frameworks. It's a change, but is it better? I don't think there's enough evidence today to definitively say so.

Previous discussion:

https://news.ycombinator.com/item?id=23144380 (1224 points | 2 years ago | 200 comments)

I keep the list btw XD http://kokizzu.blogspot.com/2016/12/list-of-tech-migrations....

or github repo, just in case someone want to add https://github.com/kokizzu/list-of-tech-migrations/

Very cool!

Wonder what the tipping point was to get the author to write this.

No particular tipping point, really, just a lot of reading the same thing over and over again. I think I mostly wrote it the morning prior to posting it? But in a sense it was “written” over a several years before I finally put it down on paper…err, a Markdown document.

It echoes the famous Instagram architecture talk. Almost exactly.

https://youtu.be/hnpzNAPiC0E

Another famous instance of $cool_startup hyping up and ditching a technology was Airbnb and React Native.

Except it ain’t satire!

Cue argument about non typed scripting languages with garbage collection not belonging on long running server processes, etc… “There’s this thing called the stack, a request response cycle fits inside it and actually http is stateless probably due to contemporary stack limits”

I agree, and found the article funny.

And by 'switched' we mean, of course, 'rewrote this little-used module of our Big Ball of Mud, there by increasing the total area of mud in our architecture'.

At the core it's what I see as a threefold conflict of interests: individual developer vs. product vs. organization

individual devs have to stay relevant, learn new technologies to up their market vaule products want a simple, reliable and working solution, not some new, but well known technology organizations want tech to be homogeneous over more then one product to be able to shift devs around, reuse know-how etc.

sounds about right! I got some of those vibes more than twice in my experience :D

I'm copying this template for my next CTO blog... . . . /s (/j?)

I think the author is on HN if I am not mistaken (saagarjha)

waves

why was $FAMOUS_COMPANY lowercased to $Famous_company in the title? That's weird.

Hacker News eats titles sometimes.

Love it, basically the core truth of most rewrites I have seen throughout my career.

Doesnt say how much it cost.

Anyone know?

Did I hear Rust?

What is the slur „techbro“ supposed to refer to?

That one of the founders is whiter than average and his parents were not poor, thus all his success obviously comes from one of his many priviledges, nevermind any hard work, determination etc?

(Note: am not white myself, I just prefer to judge people on their merit)

Where in the blog post did you see anything about "techbro" referring to a white person? Or anything about race in general?

Sometimes people eager to complain accidentally tell on themselves.

GPT-3 posting hours

Techbro is a mindset, not a socio-economic class. It's like the nerd version of an instagram influencer.

https://tvtropes.org/pmwiki/pmwiki.php/Main/Techbro

Interesting that "Tech bro" here is defined in terms of their relation to the classic nerd archetype.

I agree there are way less classic nerds - I don't meet too many devs these days that run linux or are really into sci-fi. But I also see hardly any tech bros. What I do see a lot of is a certain kind of "socially conscious" programmer. You know the kind - they love codes of conduct, they have cushy jobs but are super eager to tell you about how things work "under capitalism", they enjoy chatting on discord and probably don't know how to connect to IRC, they have a Mac, etc etc.

Is there a good derisive term for them?

I think most the original techbros are in management now.

I always assumed the label was derived from the huge influx of people into CS in the late 90's and very early 2000's because of the dot-com boom. I know there were a lot of people at my school at that time that seemed like they would have been in business or finance classes if not for the obscene money they saw thrown around by internet companies when in highschool or before choosing a major.

I mean, that's a valid choice and people are going to choose careers based on money, but if that ends up wildly changing the makeup of the people commonly in that career, you can expect the people that went into it for interest in the work to come up with some salty nicknames for those they see as interlopers.

> Is there a good derisive term for them?

I've heard a few, but they don't seem to stick: for a while, Social Justice Warrior was in vogue, and lately, I'm seeing a bunch of variants of woke, which I imagine are spoken italicized to signify vitriol and/or contempt.

Edit: I had forgotten cuck - though that's broader, and would be used as a derisive to all persona non grata not just the nerds.

The sjw, woke, c*ck are general labels, not tech specific (and some are too explicit). I laughed at the soydev in the sibling comment.

Cuck is a little more specific. A cuckold is someone who spends resources raising someone else's offspring. It's the other end of the derisive term "welfare queen".

I've heard them called soydevs before.

Ah that's a punchy term! Yes I encounter way more "soydevs" than "techbros".

Being "socially conscious" in a strictly political sense does not excuse one from dudebro status (or dudette, as the case may be). Especially when their political stance happens to be so clearly lacking in internal consistency, as in someone worrying about the supposed negatives of "capitalism" but not doing anything to effectively address them, even while working for a stereotypical "tech" business.

It's not lacking in internal consistency at all. People engage in political discourse Lite™ to demonstrate that they read the right books, or that they follow smart people twitter. It's another form of fashion, and when you take it farther than that it's seen in a negative light, especially in professional contexts.

Ah interesting! I didn't realise it was an unwritten rule that you don't take it too far. I feel Eliza Doolittle learning the etiquette and norms of my social betters.

> to demonstrate that they read the right books, or that they follow smart people twitter

Sure, but that's pretty much by definition a dudebro mindset.

I'm not entirely confident in the idea of concatenating two words for male to describe a behavior that is not exclusive to either gender. It doesn't make a lot of sense to do that.

What exactly is the problem with codes of conduct, critiquing capitalism, chatting on Discord, and using a Mac? Why the derision?

We were discussing derisive names for stereotypes in tech.

Yes, why the derision? Why is there anything to deride about any of those things?

IDK, why the derision for techbros? Or hipsters? Or anything else?

I see, that makes sense, thanks!

Then, why are the vast majority of the images of people in Google image search results for "techbro" or "tech bro" white men? Serious question.

https://www.google.com/search?q=techbro&source=lnms&tbm=isch...

https://www.google.com/search?q=tech+bro&source=lnms&tbm=isc...

> Then, why are the vast majority of the images of people in Google image search results for "techbro" or "tech bro" white men? Serious question.

For the same reason that an image search for "successful CEO" shows white men. Google's results are both a reflection and amplification of any input biases.

Sure. Google returns what's out there, and people put stuff out there that reflects their biases.

But, the claim was that "techbro" was wrongly being conflated with "white man." In this case, we're talking precisely about these biases. These image search results seem to be evidence against that claim.

A number of those images are from the TV show Silicon Valley, a show satirizing the industry.

Similar search in Google images also tend to match various stereotype (basketball player, hotel housekeeping, etc). Google will generally reflect what's on the Internet.

Right, and the fact that there is a popular TV show depicting such a stereotype is some amount of evidence that the original claim that "tech bro" and "white man" are wrongly conflated is wrong.

Search "tech bro" -tv -hbo and you get similar results:

https://www.google.com/search?q=%22tech+bro%22+-tv+-hbo&tbm=...

If anything, especially on the first page, the images get a little whiter when you add -tv -hbo to the search string.

Sure, but the bigger point is that Google is merely showing what the Internet reflects, which in general reflects society. Like when you search for "hotel housekeeper" or something positive like "CEO".

I thought it was a documentary.

techbro just means a guy who's an asshole :D

We at $IRRELEVANT_DEV_BLOG wish we were as successful as $FAMOUS_COMPAMY

Reply on news.ycombinator.com