Zeroperl: Sandboxing Perl with WebAssembly
andrews.substack.com
andrews.substack.com
Get in loser. We're rewinding the stack - https://news.ycombinator.com/item?id=43014070
Readers may want to look at both articles of course!
I've wanted to use wazero to run my Exiftool [1] for quite a while. Just as I use wazero to sandbox dcraw [2].
But WASI Perl never materialized.
This may just be what I'm missing.
[1]: https://github.com/ncruces/go-exiftool
[2]: https://pkg.go.dev/github.com/ncruces/rethinkraw@v0.10.7/pkg...
The subhead sounds weird, but part 1 makes more sense and is pretty interesting. Perl has many modules to deal with file formats nobody has used since Perl's prime. It isn't totally clear to me if the goal is to compile the Perl interpreter into WASM or interpreter + modules. In any either case the goal is to re-use the original tools within new tooling.
I’m building a new startup and file metadata plays an important role. There are thousands of file formats, each format may have dozens of versions, and each stores metadata differently.
Our use-case also needs metadata to be present when a file is uploaded - extracting the data on our servers means we add considerable overhead to upload post-processing & we lose data that is useful to customers.
So we need to extract metadata client-side and staple it to the upload. Herein begins a journey of self-inflicted pain and suffering.
ExifTool is written in Perl.
https://andrews.substack.com/p/zeroperl-sandboxed-perl-with-...
As increasingly is the case, a good starting point is the CI workflow: https://github.com/uswriting/zeroperl/blob/main/.github/work...
A thrown exception handled with a try/catch block any other name...
What's in a name? That which we call a rose
By any other name would smell as sweet;[0]
He obviously he meant my perlcc, it is maintained for compatible perl versions and it works and is used in industry.
Support has languished because the language changed fast enough that it never kept up. It may be used in the industry, but for a modern Perl application (or anything past 5.10) it won’t work. Hence why it was removed.
Speaking of - libexif doesn’t support a lot of file formats, whereas ExifTool does.
If I always opt to choose the easiest path then I’m setting a precedent for myself that I will compromise on all my goals for half finished solutions that take me further from initial idea.
Maybe, yes, perhaps! But sometimes the problem at hand - the proboem specified in the post - isnt the entire desire or objective. "Actually be easier" abounds, but sometimes our scope & intent in the long run builds on the problems at hand.
Porting ExifTool may be easier (but then if you want to maintain it, that's a drain for life). But having perl that you can now run anywhere might be something this author sees other use for. Getting good at wasm or exhibiting their excellent systems skills might have been side objectives.
Whether we just do things to get to get results at hand, or whether we invest ourselves broadly to build a better world is a constant struggle for many engineers. This shows up in the comments time and time again as "would actually be easier". I'm sorry for hitting hard on this specific comment, but there's a Two Cultures problem here, and one culture trivializes the other endlessly in the comments. It's hard for me to state why trying hard & caring & doing extra matters, but I think that breed of people are the ones that I look up to, that make all the difference to me. And I really wish there was a good defense or rallying cry, something we could say when we get the inevitable "would actually be easier" that can capture the enthusiasm for setting ourselves up & building broadly towards our better worlds.
It's not always apparent how the side-quests will help us. But man, I can think of a half-dozens of upsides of the work here. You now have a portal Perl you can use anywhere. You don't have to maintain & keep a fork of ExifTool up to date. You've learned a ton about wasm file system interactions which could be useful for any portable systems. There's so many incidentals, that people just don't see when they blaze through to "but it could be easier."
“I must study Politicks and War that my sons may have liberty to study Mathematicks and Philosophy. My sons ought to study Mathematicks and Philosophy, Geography, natural History, Naval Architecture, navigation, Commerce and Agriculture, in order to give their Children a right to study Painting, Poetry, Musick, Architecture, Statuary, Tapestry and Porcelaine.”
—John Adams, 1780
“Hard times create strong men, strong men create good times, good times create weak men, and weak men create hard times.”
—G. Michael Hopf, 2016
plus love how you worked in side quest angle — more often than not the payoff can be quite high!
The only other option is a rewrite. Does that really sound easier?
> only other option is a rewrite
If you mean by hand, not necessarily, you could use a "code converter" tool to do the rewrite automatically. But it may be impractical; perl is a notoriously difficult language to work with.
Perl is pretty close to Ruby and Python; those may be the most obvious candidates.
A code converter that kindof-sortof-but-not-really transpiles (like Python's 2to3) would require a lot of debugging of ugly machine generated source code. Who wants to do that? Would probably be even harder than a full rewrite. (How many Python libraries were actually ported using 2to3?) Similar for using any translation an LLM might spit out.
So yes, by hand. And it's not obvious if a full rewrite of a complex library would be easier than porting its runtime to WASM, especially when said runtime has been ported to countless other systems already.
By completing this work, I can use ExifTool in any environment now - and more broadly, there is now a portable, embeddable, and sandboxed version of Perl that others can use in their projects. I can think of a lot of use cases, and helping other developers makes any hardship I endured worth it.
Yes, exactly, and thank you for that!
I don't know of any other general-purpose scripting language that can be run in WebAssembly. This could make Perl an interesting choice for writing sandboxed build system rules.