Entr(1) – Run tests whenever files change | Hacker News Reader

Entr(1) – Run tests whenever files change

entrproject.org

108 pointsjsnathan45 comments

Another approach would be to use the Linux kernel's fantastic inotify() feature, which does not require running tools designed for other purposes...

https://linux.die.net/man/7/inotify

https://en.wikipedia.org/wiki/Inotify

There is also a less well utilized fsnotify feature for filesystem-wide notifications. Here is a cross-platform go implementation which also supports OSX: https://github.com/fsnotify/fsnotify

Hey! OP here. I submitted this exactly because I kept running into problems with inotify-tools, and other naive inotify wrappers, where tests would execute multiple times in quick succession, or stop executing on file changes, or other odd behavior.

Entr on the other hand works like a charm. You can see on the website that it details a number of edge cases with inotify that it works around. I'm not even sure it's an exhaustive list.

Note that that particular go implementation is prone to deadlocks.

Also, inotify is suboptimal for this use case. inotify has no recursion support, which means that the user must keep a watch on all files individually. There is also a limit to the number of inotify watches that can be in place at any one time.

A more modern, and more suitable interface for this use case is fanotify[0]. It doesn't provide all the functions that inotify does, but importantly it supports recursion, which alone would seem to make it a more obvious choice for this task.

0: http://man7.org/linux/man-pages/man7/fanotify.7.html

> Note that that particular go implementation is prone to deadlocks.

Any more information on this? I'm curious to read why.

inofity is no fantastic. Memory use scales with the number of directories watched. The program that uses the inotify system call has to keep a mapping of file descriptors to paths. Inotify is slow, inefficient and error-prone for recursively watching file systems.

fanotify is not an alternative because it cannot monitor file deletion or renames and it requires root permission.

http://www.lanedo.com/filesystem-monitoring-linux-kernel/

> The program that uses the inotify system call has to keep a mapping of file descriptors to paths

If it doesn't care what file is modified and just wants to run a command on any change, why would it need to do this?

In that case it's not needed. One could imagine that one would like to start a specific command. That is what 'tup monitor' does.

Another use-case is a desktop search tool.

http://gittup.org/tup/manual.html

I myself quite enjoy using the very nice fswatch[0].

https://github.com/emcrisostomo/fswatch

Indeed, I live by this one-liner which makes life so smooth:

    $ fswatch -0 -event ./main.lua | xargs -0 -n 1 -I {} lua ./main.lua

I made something similar with fsnotify recently: https://github.com/spelufo/on-change

The program discussed in the submission uses inotify

Err, not as far as I can see.

It has a header file that on Linux emulates kqueue using inotify: https://bitbucket.org/eradman/entr/src/1dc74ab543e2bced2994d...

Aha. Be that as it may, using inotify directly from the shell is still a fair approach which is worth putting out there.

Windows counterpart: FindFirstChangeNotification or SHChangeNotifyRegister().

entr is is what I eventually settled on after years of searching the perfect utility to build/run tests after file changes. After using gulp, grunt, watching_testrunner (which I eventually inherited maintainership of), watchman, watchdog (https://github.com/gorakhargosh/watchdog), sniffer (https://github.com/jeffh/sniffer)

entr is cross-platform on OS X, linux and BSD.

watching_testrunner has no BSD support

watchman is way too big and not domain specific enough to my needs

sniffer worked quite well but it required having a scent.py file everywhere

entr keeps it all in a neat, unix like package you can pipe files to.

I keep some example usage in my book, The Tao of tmux at https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-file-w.... In this section I demonstrate my workflow with entr(1) in a Makefile. The code I use in the example should work across OS X / BSD / Linux (note the utilities like find(1) may behave a bit differently across unix-like systems).

I use the Makefile w/ entr(1) in development on my projects like tmuxp at https://github.com/tony/tmuxp/blob/master/Makefile. tmuxp is BSD-licensed so you're free to work off that if you'd like to try it on your own project.

  ls -d * | entr make

Just a nit: this seems to be imperfect for filenames containing new lines (I know, I know). An xargs like -0, --null option could make it more robust. Then the equivalent of the above command would be something like:

  find . -maxdepth 1 -name '[!.]*' -print0 | entr -0 make

Edit: Seems like -print0 is not POSIX compliant. Is there any safe way to pass a list of paths using the shell in a fully POSIX compliant way?

Imperfect as well because any filename starting with a dash will be interpreted as an option to ls.

Nitpicking… or maybe not: if you take the habit of running `cmd * ` whatever the command, I guess you might end up writing dangerous things like `rm * ` in a script which will fatally end up running in a directory containing a file named “-R”.

On the other hand, since the target users are developers, they are in a good position to fix these newline-containing filenames. Even if they are POSIX compliant they shouldn't be encouraged :)

    printf '%s\0' * | xxd

What a coincidence! I tested it yesterday. Worked like a charm. Thanks!

One note: the browser reloading script was not installed on debian, so I downloaded it. It was not clear from the documentation if that was the intention.

A similar tool written in Rust is watchexec.

https://github.com/mattgreen/watchexec

This looks nice too! Very intuitive interface. Thanks!

Ruby developers have been using Guard for a long time now.

https://github.com/guard/guard

Am I the only one having issues with guard? It's performance has been abysmal for the most recent projects I've used it with.

I turn off autorun and only have it test what's changed.

And it also doesn't run coverage checks, it's only for running the test i'm currently working on.

But yes, the ruby infrastructure is pretty bloated i recon.

I use entr all the time. I usually do something like piping go files to entry to run a script to build and run my app or tests.

Much simpler than most tools I've seen like it.

> It is not uncommon for modern web frameworks to continuously hunt for file system changes and auto-reloads when run in single threaded or standalone mode. This functionality is superfluous if the application can respond to signals.

This is true if all that happens is a page reload, butamy frameworks are able to patch code or styles into the running page without a reload. This requires more than a signal handler.

Yeah this is a huge difference if what you're working on has state, being able to patch styles or modules makes development much smoother.

s/butamy/but many/

Many years ago I did a little utility I called autobuild[0] to do similar things. It has osx and windows support, but never got around doing linux support. I don't think inotify has an easy way of doing recursive filesystem notifications, having to manage the hierarchy and their notifiers, so it lapsed when I didn't need it at the time.

It felt like this kind of tool would/should be one of those standard command line tools always available, or the at least of those standard things everyone knows to install, like silversearcher/ripgrep etc.

[0] https://github.com/scoopr/autobuild

this is one of those problems I've needed to solve for forever, but which i've never explicitly looked up a clean solution for. I can immediately think of 3 instances in the past week where this would have been perfect.

Watchman from facebook is another great, single-purpose solution.

https://facebook.github.io/watchman/

We switched from watchman to entr on the project I'm currently working on because watchman was troublesome to configure and IIRC it completely detaches from the shell as well.

A long time entr user, I recently switch to modd for flexible almost makefile like configurability, and watching/ignoring of multiple patterns to name a few convenient feature.

https://github.com/cortesi/modd

I wrote a dead-simple version of this (that uses polling and doesn't give you filters for files) in python and called it `jonsnow` and threw it on pypi. I understand this is an insanely useful utility, but it's interesting that everyone seems to spin their own :)

awesome.

What would be really great is a fuse DAG.

Some paths should run programs on write, others on read.

The filesystem is already the lock provider for a lot of programs; we should be using it as the event processor.

This is inherently polyglot -- every programming language in common use has FS utils.

Next step: run tests whenever you enter a keystroke in your editor?

You're probably joking, but if you use a real-time linter, maybe you could run tests whenever the file you're editing has no errors.

You might even try to limit which tests to run based on the test coverage (from previous runs) of the changed functions or statements.

And then there's also ideas like LightTable's inline evaluation [1].

[1]: https://www.youtube.com/watch?v=7YIaHyTdjTY

Cross-platform alternative: http://emcrisostomo.github.io/fswatch/

Looks interesting; anyone know how well this works inside docker? For me watching files can often cause huge CPU usage.