Hacker News Reader

Comment by gw98 | Hacker News Reader

I know the point of this article is to demonstrate how to solve a problem but the premise is a bad one and must not be confused with sound software architecture at all. In fact it's slightly painful reading it and I really wish people would stop writing articles like this.

The functional requirement here is to take some HTML, parse it and emit slack flavoured markdown.

Involving WASM / Rust / cross compiling stuff and fucking around with a build tool chain should not even be being discussed anywhere as a solution for this. I mean it's fine as a toy but the problem is that half the industry doesn't have any idea what's a good idea and what isn't from an engineering perspective when it comes to solving problems and this will be taken as gospel on how to solve the thing.

What we have here is a Rube Goldberg machine, not a cleanly solved engineering problem.

View on HN

The article is tagged cursed for a reason. This is not intended to be a good solution, it is intended to be _a solution_ that works better than you'd expect while also explaining the Unix philosophy and going through my entire thought process into making it.

The best kinds of hacks are the ones that look ludicrous but are actually somewhat fine in practice.

It's very clearly a toy example to demonstrate the idea, and it does so well.

> What we have here is a Rube Goldberg machine, not a cleanly solved engineering problem.

And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem. People have been using linked libraries to re-use code across languages since forever, it's fine. This solution isn't very different, it just makes shipping easier.

As a parting comment, "sound software architecture" is not a decided upon principle which can be empirically determined, and if it was we'd all be out of a job.

> People have been using linked libraries to re-use code across languages since forever, it's a fine way to solve problems.

People have been doing it forever, but people have also hated it forever. Twenty years ago you saw SWIG in a build, you'd know you were probably in for a bad time in an exciting new way.

I think SWIG had a different purpose. The author here generates and embeds WebAssembly in Go, to avoid building the same lib for multiple platforms (+ the bindings to call the low level c code). Maybe the tool wasn't good enough? Right now this is just WebAssembly which is proven to work on multiple platforms.

If the API is clear and documented, I don't see why this would be an issue, except for the fact it might be a little bit clunky.

It's not the first solution I would come up with, but the question would be: why not? Just because we're used to older and more traditional patterns, why not just to embed webassembly for low level stuff in your code?

> And what _we_ have here is unfounded indignation over a perfectly fine way to solve a problem

Absolutely no way is this a fine way to solve the problem. That is crazy talk.

1. It introduces additional toolchains into the solution when it is unnecessary.

2. It now means you need multiple language specialists to maintain it and associated communications and context switching.

3. More interfaces and integration means more fragility when it comes to debugging, problem solving as well as increasing the problem surface.

4. It massively increases the dependency stack which means there are now multiple sets of supply chain issues and patching required.

This makes no problems easier at all! It's even a bad last resort if that's all you have left to try.

Sound software architecture is very very well defined and this is definitely not it. I have seen entire companies burn by taking this approach to problem solving.

I'm really getting tired of solutions before problems and this is a fundamental example of it. Give us a real use case not manufacture a problem for it.

> 2. It now means you need multiple language specialists to maintain it and associated communications and context switching.

Sometimes you have that and you need to accept it. In this case the function was really trivial. What if it was a mega secret algorithm that needs Rust speed which you want to really write in Rust because XYZ? Or C++ I don't care. But not Go, which you want to use exclusively for microservices, for example.

> 3. More interfaces and integration means more fragility when it comes to debugging, problem solving as well as increasing the problem surface.

This is inevitable when you want to go that low level. Some companies use C++ for everything, some mix low/high level: how do you do in that case?

This article is about Go, but I could imagine something similar for Python: use something superfast under the hood, python for the high level part. It's really a common solution, which requires a bit of bindings left and right but it's worth it.

> 4. It massively increases the dependency stack which means there are now multiple sets of supply chain issues and patching required.

This is how software is done today. Maybe 30 years ago you would write everything in C and be happy with it. Today companies use at least 3 programming languages - at least that's my experience.

> This article is about Go, but I could imagine something similar for Python: use something superfast under the hood, python for the high level part. It's really a common solution, which requires a bit of bindings left and right but it's worth it.

That's not a bad idea really, the wasm/wasi option could be a nice intermediate step between "we can't optimize the python any further" and "write a native extension" (or god forbid use cffi or the cursed horror that is ctypes).

> Absolutely no way is this a fine way to solve the problem. That is crazy talk.

I disagree. I see this in a similar way I see Electron and actually you can make the same arguments against using Electron.

But guess what, Electron wins on practicality. It makes creating GUI apps much easier. That wins over any problems associated with the extra baggage of shipping a whole browser. It doesn't win it for everyone, but it does for the majority of people who just want to get shit done.

This appears to be a straw man. Nobody is trying to tell you to rewrite your software stack using this technique. The OP demonstrates a cool hack. The site we currently occupying is a place for cool hacks. I don't see the problem. As far as hacks go, it's far from the most egregious that I've seen, and even suggests a few thoughtful lessons about the future of FFI beyond the C ABI.

> The functional requirement here is to take some HTML, parse it and emit slack flavoured markdown.

That is solved already and not what the article is about

The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature.

That is what article is about.

The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting.

> The nonfunctional requirement is a self-contained binary that is not dependent on machine's own libraries or any extra files. That's not just mental excercise but a feature.

Let me add more to this: speed.

You have Rust/WebAsssembly in the mix and automatically you gain in speed as shown in the article.

I honestly don't understand the negative comments.

If you build a lib in C and then you use standard binding mechanisms, oh it's ok. In this case you leverage a great tool (cargo) to do the heavy lifting for you and bam you get a safe binary that gives you better performance and overall better tooling -> it's bad.... why?

I read the original version of the poster's comment and it was way more aggressive and more about "sound architecture". Maybe he/she can explain what exactly is wrong with this approach and what other approach should be taken instead?

> The "proper" engineering solution might be very well "just rewrite that small part in Go", but this approach is nonetheless interesting.

Why? In this case it's a trivial function. What if it's a strong algorithm that needs speed? Rust helps there, and it's faster than Go.

> Why?

Because integrating anything in Go is such a pain in the ass gophers rewrite everything. Thus it makes perfect sense to a gopher.

It's the flattening of everything.

For a system architect or senior developer it's very interesting to know I might glue a (not so trivial) Rust codebase into my Go program so easily. For people actually working on Go, Rust, WASM, etc., these are also experiments to evaluate the ergonomics, performance, etc. of all the tooling. For someone who wants to learn how FFIs work, this is a great tutorial.

But I'm certain I will now get at least one mid-level dev or interview candidate actually try to do this, in a similarly trivial case. And I will have to explain yes you already wrote the whole build pipeline but no we're not going to maintain it, and yes the blog post says it's fast but it's not really that fast, and etc. etc. It's bad enough every time I have to hear "it's just one Python script, what could it cost?", but the more complex it gets the more energy it takes to genuinely convince (rather than merely authoritatively declare) people not to do it, and the limit there seems unbounded.

This is a toy example to demonstrate a framework to build with. But I don't think it's not a cleanly solved engineering problem. Instead of reinventing xer Rust code in Go, Xe has simply taken the functional rust code and made go run it in a cross-platform and portable manner. Minimal effort from the engineer has been expended to solve this and C was not involved for FFI definitions either.

> Minimal effort from the engineer has been expended

Is the definition of "how do I get promoted in a company where I don't care if it survives the next five years", not "a cleanly solved engineering problem".

I don't really see the maintenance issue. The rust code seems maintainable and the go code seems to be fairly lightweight overall.

You missed the two entire toolchains and the third tool required to stick them together?

It's pretty difficult and expensive to build one stack, let alone three and onboard another tool.

Depends on how the rest of your stack looks. I wouldn't presume.

I've dealt with pretty much everything from steaming nightmare creeping Cthulhu desktop applications right into back end fintech stuff written in the dark ages over the last 30 years. At no point have I found this solution being applied where it solved a problem. I have seen it applied many times where it created problems!

Author here. My article is not supposed to talk about a good idea. It's meant to bring a bad idea to the table and explain why it works. I designed the function in question with the understanding that it would fire once or twice per 10 minutes. This means that paying a cost like 3.1 megabytes per invocation is okay. If this was intended to run _constantly_ (such as if it was a core part of a run loop), that's different.

I recommend putting this point up at the top and quite clearly explain that.

One thing I'm wondering about: what's the call overhead of wazero? Especially for function calls (rather than as a `system()` alternative)?

I understand that cgo has significant per-call overhead, aside from all the other curses it implies, does wasm/wazero have the same issue or not?

With the version of wazero I'm using right now, it's about 0.3 milliseconds to do a single call from Go to Rust and back in the best case on my hardware. I am told that a recent patch to wazero will increase this by a lot, so I'm going to try upgrading to that patch and see what differences it makes. I still think that it will be a bit slower than cgo, BUT the platform-independence and strict sandboxing makes up for it in my book.

I think the article is absolutely clear about it but also I think people will (are! in this very thread) ignore that part and make enormous messes that will hurt real users and someone else will have to clean up.

I’m not sure how to fix this but I’m also tired of pretending it’s not a chronic problem.

If you ship something featured in my blog into production that's your problem TBH. I can't be responsible for the sins of others.

I'm not going to blame you for anything, but I also think this is an odd viewpoint for a professional technical communicator to have.

The alternative is rewriting the code of Go into Rust or vice versa.

And if rewrite is Rust->Go way, also continual effort of porting any bugfixes in upstream lib.

I dare to say more complex toolchain is the easier and less time consuming part.

I wouldn't do it in this particular case (converting with mastodon powered html is probably simple enough) but it wouldn't be a terrible solution in some cases.

What's more interesting is if you use same method for app plugins, now you can compile anything in WASM and as long as it have right hooks it can be used in your app as a plugin

> The alternative is rewriting the code of Go into Rust or vice versa.

This is more work up front, but it’s better engineering.

> easier and less time consuming

Have some pride, jesus christ.

Depends on how well supported those stacks are. WASM is very well supported and likely to be getting tested/used/improved extensively as the years go.

I'd rather work on software that depends on three different tech stacks that are well understood and used by many, than software that depends on a single niche tech stack.

I'm not a "stick to a single stack for everything" kind of person, but here we're comparing Go or Rust, to Go + Rust + WASM. The first option is strictly and substantially less risky in this dimension.

I don't care for the eccentricities of the Rust community, but this is a good article that demonstrates an effective approach to invoke code from one language to another. The problem is a toy one that's not the focus here. The focus is a better means of doing interop

Reply on news.ycombinator.com