Hacker News Reader

Comment by pdimitar | Hacker News Reader

I love `just` and have adopted it universally in all my projects. For what it does, it gets the job done fantastically.

That being said, I found myself needing a tool that builds a DAG of dependent tasks and automatically figures out what can be ran in parallel and what cannot -- obviously you have to spell out all tasks and who depends on what first.

Anybody knows such a tool?

EDIT: Apparently people did not get the hint that I believe `make` is an over-engineered pile of metric tons of legacy and I'll sooner slash my wrists than to learn it in full.

I did mean something ergonomic and easy to read and write. And no I'll never view `make` as such. I tried. Many times. I have better things to do in my life than to memorize exceptions of the exceptions.

View on HN

I’ve not tried it but this popped up on here a while back and sound like it might fit the bill.

https://taskfile.dev/

Thanks, this one has been in my radar for a while, I'll absolutely get to it at one point.

I think this is exactly the intended use case for Ninja. It’s discussed in this recently posted article.

https://news.ycombinator.com/item?id=42268310

Thanks. That article is fairly disappointing for not having even one simple example file though...

Julia Evans has a Ninja introduction[0] with simple examples. I tried it for awhile, but ended up going back to GNU Make.

[0] https://jvns.ca/blog/2020/10/26/ninja--a-simple-way-to-do-bu...

Just gave this a read. Impressive. I'll give ninja a try soon, I have a possible use-case for it. Thanks!

I wrote frof [1] for exactly this purpose :)

Designed to be ultra-simple and with minimal "config-file acrobatics".

It looks like this [edit, formatting]:

    write -> analyze
    build -> analyze

    write:     echo 1 2 3 > data.txt
    build:     compile_tool.sh > tool.sh
    analyze:   tool.sh data.txt

https://github.com/j6k4m8/frof/

Can you explain that one a little bit more to me, please?

I don't get the first two lines of your example well. They seem to show the dependency but which one is the default task, or how do you ask for a task to be ran?

You write the file and ALL steps are run in topological order so that a job never runs until its dependencies have run. i.e., in a tool I'll have `build.frof` as a separate frof file than `download-dependencies.frof`, perhaps. (If your preference is that those belong in the same file I'd be down to have PRs that support that! Should be very easy, I'm happy to try implementing this if there's interest.)

So for a file with those contents called `mygraph.frof`, you can (after installing) run `frof mygraph.frof` to kick off the jobs in the current shell (inheriting env vars etc).

[edit] maybe a clarifying example here: https://blog.jordan.matelsky.com/frof-render/

OK, so for the example in your comment upthread both `write` and `build` will be executed sequentially?

here they'll probably be executed simultaneously, since they both have zero dependencies and the machine can run multiple jobs at the same time. (can be disabled with `--max_jobs=1` or `-p=1`).

Here's another illustrative example:

    A -> B
    B -> C
    Z -> C

In this situation, frof will schedule `Z` to run in a parallel thread ASAP, so it will likely run alongside A... and if Z takes longer to run than A, Z will continue running when A stops and B starts. But C will wait for all other jobs to finish before it can schedule.

Nice, thanks a lot. Unfortunately I am quite swamped recently so I will definitely cannot help you with feature requests and testing but I have bookmarked frof and absolutely will be giving it a try.

Just one thing I would dislike... Python. How easy it is to run frof without having to fiddle with venvs and such?

no worries, good to know this would be a useful feature! I'll add it to my backlog.

    pip install 'git+https://github.com/j6k4m8/frof/'

and then

    frof myfile.frof

should work!

Was thinking about rewriting it in Go recently... :)

I've found prototyping in python followed by a rewrite in Go quite pleasant, would recommend

I'll try the vanilla Python route but knowing our mutual hatred, it'll crap the bed in 0.5s. :D We'll see.

> Was thinking about rewriting it in Go recently... :)

And then I might actually contribute. :)

Besides Make, I guess Bazel kind of fits the bill? It was very "Googly" last time I checked it out, but I think that was a decade ago and right when it was released, so it might be more fitting for not-Google nowadays.

I never looked at it but seen some fairly negative reviews here on HN. Any idea why? And why do you like it?

Imagine that instead of a make target listing its dependencies, you had to pull them out into a separately maintained BUILD file.

That’s not quite true, but it feels like it sometimes. Bazel is nice about seeing exactly what you need to rebuild if you touch a file. It’s very, very complex though.

In code terms, think of it as a framework that you have to embed your project into, not a Makefile or such that you’d drop into a project. That doesn’t make it bad and it has its niceties. You’ve gotta be prepared to pay for them with sweat equity.

Thank you. I heard similar sentiments RE: complexity and that's usually enough to turn me off of a tool.

Make?

Come on, be serious. If I wanted ancient sh-isms and bash-isms I would have learned make to 100% some 15 years ago.

I meant something ergonomic and easy to read and write.

> If I wanted ancient sh-isms and bash-isms

So don't, set make's shell to something else instead. It doesn't understand the recipes, it just dumps them to a file and runs $(SHELL) on them.

For a more extreme example, just to show what's possible:

   SHELL := python3
   
   .ONESHELL:
   
   foo.csv:
    import csv
    with open("$@", "w") as f:
        writer = csv.writer(f)
        writer.writerows([
            ['Test1', 'Test2'],
            ['Test3', 'Test4'],
        ])

I'm this years old when my life was revolutionized.

Not a bad idea, thanks. I did this a few times as well but when I analyzed the ROI I figured that just writing a simple-ish Golang program is just less confusing and more consistent in its totality when you ask yourself "do I really have to use Make and Python and, and, and...?".

So yeah, thanks for bringing visibility to this pretty decent compromising approach. It worked for me for a while but eventually I just went all-in to either use `just`, some _very_ short bash/zsh scripts, or jump all the way to Golang.

They are right, though, aren't they? I mean .. if you want something "modern", go ahead and learn Bazel. Make is quite a bit easier to learn, I'd say, and you don't need much (also no shell/bash) to express your DAG dependencies.

I'll agree on the DAG bit but I'll never use `make` again and I tried for no less than 10 years (on and off, not 24/7, otherwise I would have learned it long ago indeed).

I stay away from `make` almost religiously. Its complications _always_ find a way to creep into your file one day. Always. :(

So while they are technically correct and it's my fault for not saying I don't want `make` in the comment up-thread, I don't think my comment deserved the down arrows but oh well, I'll live through it.

Maven, Gradle, etc.

FYI to some people trivializing self-harm in a technical discussion is rather tasteless.

I disagree, fundamentally, that using a hyperbolic metaphor "trivializes" the underlying concept used.

Regardless, I have found over the last several years that attempting to scold people for not measuring up to your standards - ones they never signed up to uphold - without a serious attempt to justify them is strongly counterproductive.

Especially when it's couched in language that will readily be interpreted as snarky. One of the reasons people dislike the phrase "it's not my job to educate you" so much is that it takes for granted the presumption that the underlying idea is a subject of education, i.e., an objective fact rather than an article of someone's worldview. Prefacing a claim with "FYI" (i.e., "for your information") has the same issue. Taste, in the metaphorical sense used here, is definitionally not objective, and thus it is not possible in principle to "inform" others of what is or is not in good taste - only of some other community's standards for taste.

It's an exaggeration to illustrate a point. Still, thanks for bringing in the perspective.

Reply on news.ycombinator.com