Killing Off Wasabi (2015) | Hacker News Reader

Killing Off Wasabi (2015)

blog.fogcreek.com

56 pointsgk127 comments

I actually had an internship at Fog Creek in 2010. Ted Unangst (mentioned in the article) was my mentor. One thing I noticed about the folks there were that they were all crazy fucking smart. Joel had built this amazing recruiting machine that could pull down some of the real best of the best -- definitely a team that rivals any I've worked with at Facebook.

The drawback to this though was that the best are easily bored. FogBugz was an important cash cow for the company, but there were few truly novel problems to solve. This was not a "web scale" effort -- it was self-hosted software for tens to hundreds of seats.

So I speculate that they okay'ed the project just because it was an interesting thing to do that solved the underlying problem (albeit in a bit of a convoluted way). The simple cost/benefit analysis from a management point of view is simple; pros: your devs are happy and you get linux support; cons: it's a little more complicated and people make fun of you on the Internet.

Over time, Fog Creek came up with better outlets for this kind of nerd energy -- kiln was fun because there were mercurial nerds around; trello was a foray into server-side javascript. There was still a bit of that "the nerds rule" bad decision making that made FogCreek an awesome place to work, but it was more constrained.

Since then (and I'm way out of it now), a lot of the old school "just badass" programmers have kind of gone their own way and Fog Creek has hired folks who are better aligned with what they do and how they do it. I'm not sure if this is because of a masterful shift in recruiting philosophy or if the diminishing recruiting brand (due to Joel's blogging pause) led to the right kind of developers becoming interested naturally.

In either event, it was definitely a very interesting culture, and I'm happy to have been able to play along for a summer :)

I guess the dot com crash in 2000 helped them cherry pick talents.

A summer intern created their compiler ...

There was a time when one of their blogs (one of the founders, I think) touted the fact that they had never lost an engineer to attrition. It seemed like a strange fact to use to promote the work environment because, even if true and impressive over any length of time, it would certainly have to be retracted sooner or later. I know at least one of the original Kiln engineers has moved on and Trello has been sold.

Anyway, I was always curious just how elite of a team they managed to maintain, so thanks for your comment.

Edit to add: whoever the devs were and are, hat tip for a remarkably stable system that does what it’s supposed to do. Since Kiln 2.2 or so landed way back when, the FogBugz/Kiln combo has done everything we need for support and development. Much to their sales team’s chagrin, it can be difficult to justify an upgrade, despite the allure of feature and performance updates.

That article about zero attrition was written in 2007: https://www.joelonsoftware.com/category/reading-lists/tech-l...

At this point I think everyone who was working at Fog Creek in 2007 has left, including me. (Though I came back in 2013 and am still here.)

We have an excellent team, continuing to innovate on Manuscript and Glitch.

We spun off Trello, which was acquired by Atlassian for almost half a billion dollars.

We co-created Stack Overflow, which continues to be incredible.

Good point. Trello and SO are both phenomenal pieces of software that I use all the time and recommend to others. All of this stuff is fits a need and does its job reliably. Well done.

Brings back memories of the flamewar when it was created https://blog.codinghorror.com/has-joel-spolsky-jumped-the-sh...

Ha! And of course Jeff and Joel eventually worked together to create StackOverflow...

Reminds me of a post [0] on evolutionary architecture retweeted by Martin Fowler just yesterday:

1 Build for now

2 Choose tech based on ability to evolve

3 Evolve one use case at a time

Despite the (imho) horrors of having to maintain their own compiler, it seems Wasabi was a vehicle that supported the company's evolution. Once it became obsolete, the next evolutionary step was to get rid of it. Perfectly fine imho.

[0]: https://medium.com/@shoup.randy/evolutionary-architecture-27...

We can't be certain what would have happened had they chosen otherwise. But I find it very hard to believe they wouldn't've done better by switching to an existing cross-platform web language.

Spolsky wrote the (for me as a young developer) eye-opening article about why you should never rewrite from scratch. But even then, it was clear that Wasabi was taking this to an extreme. VBScript is one of the worst and most limited scripting languages ever. Using this as source and then transpiling into more powerful and expressive languages seems totally backwards. Then they brag that they are able to add new features like closures (which already exist in the compilation target languages). So they are creating a new, horrible, proprietary language, just to avoid rewriting the original source.

It seemed like a fun challenge to work on, but just like "lets rewrite everything from scratch", the kind of ideas developers come up with when they don't have to consider the big picture and overall cost.

Spolsky's wrong about rewriting from scratch, he took it way farther than was sensible.

The main problem most people make about rewriting from scratch or switching architectures or what-have-you is not doing it at all but thinking that it'll be a cheaper and easier transition that it will actually be and also committing too early to the new product.

A lot of this comes down to the arrogance of engineers and the devaluation of testing. The reality is that testing and real-world use have tremendous value in shaping products. Actual usage is one of the best ways to discover bugs, of course, and over time those bugs get fixed, leading to a more mature product. Similarly, actual usage is the best way to discover the gaps between the developers' design and vision for the software and what people actually want and need. More so when people trade hard-earned dollars for software. Developed software thus represents not just X number of hours of dev. work it also represents a tremendous number of hours of refinement, validation, and verification through "beta testing" (product use), in short: product maturity.

Trying to replace a mature product with an immature product is a disaster waiting to happen, even if the immature product is technically built on a superior foundation. The right way to do this sort of thing is to spend time running parallel development efforts, giving time for the new thing to mature and compete against the established product and allowing the market/customer base to make the transition as the new product gains maturity and becomes superior in practice not just in theory. Or, as people avoid the new product due to it failing to achieve its promises.

I'd take the view that a rewrite in place is always better than a rewrite from scratch. With a little effort it's always possible to implement the new thing piecemeal as part of the existing thing.

I think there's something to be said for biting the bullet and just doing a proper rewrite if the investment increases your market size. There's also something to be said for solving problems at the right layer.

How much time could have been saved if they'd picked a cross-platform language, e.g., Java or PHP, and done the rewrite once the market had been validated with what was essentially a prototype? Instead they tried to shoehorn it into what they'd already written by writing half-baked solutions at the wrong layer.

Don't get me wrong, it's impressive technical wizardry to get as much of it working as they did, but the work was always going to have a short shelf life. They've done well for themselves either way, but I wonder what might've happened if they'd rewritten it upfront.

Unrelated fun fact: most people never had any real Wasabi. Me neither :/

https://www.quora.com/In-Japanese-food-how-can-I-differentia...

I am busy growing some in a pot, for that very reason.

/me wonders if it died in the forty-degree heat today

If you want to try some real wasabi, you can get it from Oregon Coast Wasabi (aka Frog Eyes Wasabi):

http://www.thewasabistore.com/

They sell the graters too. (You don't want to use an ordinary grater.)

I haven't tried their wasabi myself, but heard about them a few years ago and have been meaning to order some ever since, so this thread reminded me to look them up again.

Wow, that project sounds like a total dumpster fire. Not that I know any of the details, of course, but I find it hard to believe that it saved any effort to try to write a compiler and a custom language/type system/code generator/etc versus just embarking on a rewrite of the code in question. To be honest I’m surprised that even in hindsight he seems to think it was a good idea, as I read it as a cautionary tale.

Wasabi could compile VBScript, which was the language they had at the time. It's actually a more or less identical approach to the one Facebook took with Hack, but Facebook had a lot more resource to make it awesome.

It started out as compiling VBScript but from my understanding it became its own beast, with all kinds of bells and whistles.

Yes, definitely there can be valid reasons to write your own language/compiler etc. Of course, the size of the Facebook code base, not to mention engineering team, put it in a clear class of its own. And the Hack project was run by people with extensive experience in compilers and type systems, not a summer intern... hah. But yeah, it’s not a bad idea in all cases. Facebook, google, Microsoft and many others have invented new languages and/or written compilers. And I’m sure there are valid reasons for a small organization to do it as well — but I fail to see what made it justified in this case. Then again the product didn’t go belly-up, so there’s that.

Downvotes? What did I get wrong here?

Writing compiler is not necessary super hard if you are ok with it being slow.

Still dead!

Hey Jeff. We’re still slowly rewriting old ASP-to-Wasabi-to-C# pages to this day.

We hadn’t open-sourced it, so this meant any investment had to be done by us at the expense of our main revenue-generating products.

I think it's cute that he thinks he could have open sourced his appalling kludge and gotten some free labor from unpaid OSS devs. He may have been right.

He does have the brand recognition, there's bound to be at least a few shops that would've picked the language because that's what Fog Creek uses. Which probably would have led to contributions further down the road.

Right. In other words, "He may have been right".

OP here. I believe there are still classic ASP apps in existence that could use Wasabi for the same purpose we did, to switch to more-modern .NET. They would presumably prefer an open-source solution to a proprietary one.

Reply on news.ycombinator.com