Show HN: Xc – A Markdown Defined Task Runner | Hacker News Reader

Show HN: Xc – A Markdown Defined Task Runner

github.com

113 pointsjoerdav44 comments

Nice to see other markdown-based task runners!

I'm the creator of mask which is another alternative, written in Rust. Our approaches slightly differ. It looks like xc parses the README.md file for commands while mask looks for a maskfile.md by default, though you can provide a --maskfile arg to specify any markdown file that follows the expected format.

https://github.com/jacobdeichert/mask

Why not a simple well commented shell script?

Most code editor will syntax highlight it a lot better.

In the docs for iommi we've tried both the documentation-centric and the code-centric approaches.

We started with the documentation-centric approach because we had a lot of documentation and wanted to check if the code examples in them worked. So I wrote a bad parser that extracted the code from rST and put them into python files that we then ran. This worked... ok-ish.

Then we switched to the code-centric approach that we use today where we write tests that we generate the documentation from. There are specially marked documentation strings plus some tags for parts of the tests that shouldn't be included in the generated documentation, and some other cool little features like saving down the output of test requests to their own html files and then embedding them with iframes instead of using screenshots.

I am a programmer so I like the code-centric approach more, but they are pretty similar. The big thing is that you need to control the file format/parser yourself.

Before mask, I used a custom bash command runner which relied on a directory structure to implement the command and subcommand tree. This was pretty simple and nice to use. So if it’s working for you, there’s no reason to look for alternatives.

Mask takes advantage of the markdown structure in a few ways. Headings define top-level commands and subheadings represent nested subcommands, which makes it extremely easy to structure a command tree. Also, mask checks the code block lang code (ruby, python, js, fish, etc…) and executes the script using that runtime as long as you have it installed. There’s other features, but those two are great examples why markdown works well as a command definition format.

> Most code editor will syntax highlight it a lot better.

Really? I thought most now understood markdown code blocks with language tag?

Let me ask a silly question.

Use case, I'm on Mac OS X (aka not Linux, not Windows). I have an Apple M1 processor (aka not x86_64). I have 8GB of RAM (silly mistake on my end, I know). Therefore, Docker + virtual machine based solutions are too expensive memory wise.

However, I also like to think in terms of "let me separate this functionality into say... a Kubernetes workload like a pod"

There's no good containerization solution for Mac OS.

Can something like `mask` be used to achieve basically a "poor man's k8s" for a long running service?

Basically, run these 5 or 6 services in parallel, let me be able to see their logs individually.

If not, I completely understand. I just can't tell if there is an actual need for a solution like this, if it already exists and I just can't find it, etc.

Have you tried using a process manager like supervisord[1] or pm2[2]? The former has a pretty clean declarative interface that you could view as a replacement for something like a docker-compose file or a k8s pod config.

You don't get any containerization/isolation benefits obviously, but it sounds like you've already accepted that.

[1] http://supervisord.org/ [2] https://github.com/Unitech/pm2

And another possibly lighter alternative is something like concurrently: https://github.com/open-cli-tools/concurrently

Foreman is a nice, easy to use local process manager. It's what heroku and such use with a simple Procfile for defining multiple processes, but it would work fine for running multiple developer tools, testing services/dependencies, etc.

There are multiple implementations of it, heres the canonical one: https://github.com/ddollar/foreman And heres a go version that's easier to install and use: https://github.com/ddollar/forego

Mask can't directly solve this problem by itself, it just runs whatever script you give it. If you can write a python/js/bash/etc script to achieve what you want, you can stick it inside a maskfile with the rest of your commands.

Regarding docker, I haven't tried it on M1 yet. However, I've been using Ubuntu multipass [1] for over a year now and I'm very happy with it. It makes it easy to set up and manage VMs for different projects, and it seems to run very efficiently on macOS in my experience. When a project needs a docker container like postgres, I just run docker compose inside the VM rather than running it directly in macOS. You can also limit the amount of CPU/RAM the VM uses to keep things under control.

[1]: https://github.com/canonical/multipass

This actually reminds me a lot of Gauge from Thoughtworks: https://github.com/getgauge/gauge

It's typically paired with Taiko for test automation, but generally speaking it's a markdown to logical instruction engine.

I dig it, but also worth taking a look at what the Thoughtworks team has done especially around the VS Code tooling and language server work that they did to bring intellisense into their Markdown templates.

Wow I had not heard of gauge until now but it looks really cool! Seems like a nice more general behavior driven development tool. Very cool!

Like "literate programming" + "shell scripts". Ideally the latter can be a better, cross-platform language (e.g. "```python" code fences).

I had exactly the same idea regarding ```python fences. I filed an issue if you wanna give it a thumbs up: https://github.com/joerdav/xc/issues/42

Your doc links are slightly broken. On the page https://xcfile.dev/task-syntax/ the links don't resolve, although the ones in the sidebar do. The former are missing the trailing slash that the sidebar links contain: https://xcfile.dev/task-syntax/task-name vs. https://xcfile.dev/task-syntax/task-name/

Thanks for pointing that out, I recently did a rewrite of the docs. Seems to be one that works locally but not on remote. Thanks!

It's interesting to see emacs features reinvented

this is a trival thing to achieve in emacs or org mode or babel in a number of ways.

Why is copy and paste hard to do though?

This comment appears in any submission, doesn't it? Doesn't matter what it is, it's probably a trivial emacs feature. :')

(It probably genuinely is in the infamous Dropbox thread somewhere - 'not only that but it's also built in to emacs'!)

Which suggests the wrapper is the blocker. (I have no horse in the emacs vim holy war; this argument could apply to macOS just as easily.)

Sometimes maybe, I can see that here. But a lot of the times its people pushing their flashy developer product, where the reason to, lets say, overlook Emacs is even more obvious.

> Doesn't matter what it is, it's probably a trivial emacs feature. :')

Org mode is far from trivial! when a commenter posts about org mode on an article like this, consider it an invitation to explore something deep and wonderful rather than a claim that the problem being solved is trivial!

To be clear, I was just paraphrasing OP - I'm not an emacs user (nothing against it, just never got into it.. and yes I do use vim, on Arch btw).

The idea was that even if you don't have xc installed or know it exists, then you could benefit from the Markdown based documentation required for xc to work, and in those cases you could copy & paste.

Maybe I’m outdated but why not use simple make files?

    PHONY: comment
    comment:
      TEXT=“yes, I also wonder why people don’t use simple makefiles” $(MAKE) publish
    publish:
      echo “$(TEXT)”

Some “simple” things like passing arguments to a task become incredibly complex.

I’ve been playing around with https://github.com/casey/just and so far it’s been very pleasant.

Mostly preference from what I can see. Some people don't like parts of how make works.

Personally I have no problems with makefiles but I see the appeal.

If you like to use make as a task runner, may I recommend you take a look at my entry into the ecosystem:

* https://github.com/TekWizely/run

Feels like make (by design) but purpose-built for managing small tasks.

The reason I prefer xc to makefiles is it is more discoverable I think, when you have a project with a Makefile you often have some documentation in the README that outlines what each Makefile command does. I thought why not cut out the middle man!

You can definitely add the documentation to your Makefile and have a target ("help" is commonly used) that displays it! Examples of such an approach abound.

My initial reaction was: why Markdown? That's (despite its name) a markup language, not some sort of data/configuration format. But then I saw that it allows you to consolidate the documentation and the tasks into a single place, and now I think this is a great idea!

Other comments have pointed it out, but the only issue I see is that shell commands aren't trivially usable on e.g. Windows. I'm sure there are good solutions out there for this, but I haven't found them yet...

Most tasks should work trivially on windows actually, xc is shipped with a shell interpreted.

The only constraint at the moment is if an executable is used that isn't available on the system.

I didn’t realize that, thanks! I’ll definitely give this a try…

A few years back on my project Takeoff (https://github.com/takeoff-env/takeoff) - a tool for creating docker environments for local dev - I used maid (https://github.com/egoist/maid) for exactly this, using a markdown file as a task runner.

I liked the idea, although didn't go much further with the project.

To the extent that posts like these evolve into discussing the merits of Make as a task runner, I would like to offer my tool for review:

* https://github.com/TekWizely/run

I built it to feel like make, but be better at managing tasks and wrappers.

If you are evaluating task runners and appreciate the simplicity of Make's syntax, I hope you'll give Run a try.

I authored a markdown player, for which one may run arbitrary code/commands, capture output, and self embed. Literate programming, md targeted.

https://deno.land/x/markdown_player@v1.2.3

OCaml users also like omd. Iirc omd is a bit ocaml specific tho

Why not Makefile ?

The key insight here seems to be that you usually need to specify the tasks for a repo twice: once in your task runner (Makefile, package.json, bazel BUILD file) and then again in your readme so that people can learn what custom tasks exist and what they do.

This tool tries out the idea that maybe if the readme was structured the right way, it could double as the task runner specification. That way, even if you don't have the tool installed (maybe because you're browsing the repo online instead of in your editor) you can still see what the tasks do alongside their instructions

Seems like i missed the point. Thanks for explaining

TIL Make is the Zoidberg of technologies.

Also non trivial Makefiles quickly become an unintelligible mess of rules both explicit and implicit, and an endless stack of variables that never feel fully defined.

I was going to make a funny comment about Make, but then I decided I can just link the Make documentation's page "The Two Flavors of Variables" [1] which links to definitions of the four different flavors of variables.

[1] https://www.gnu.org/software/make/manual/html_node/Flavors.h...

There are four kinds of assignment, the first applies to recursive variable flavors, the rest to simple ones. It's maybe something to learn, but why is this a bad thing?

People make good jokes but anyone asserting that official GNU documentation is anything short of wonderful needs to learn to read again.

Why not both?

Make is not actually that great of a task runner, but the syntax is easy and its ubiquitous.

I created a tool that allows you to define tasks in the easy make-style, but is purpose-build to be a task runner:

* https://github.com/TekWizely/run

Great work! I really like it!

BTW, it would be great if it had .env support!

Reply on news.ycombinator.com