Rebuild code automatically with "watch" and a makefile | Hacker News Reader

Rebuild code automatically with "watch" and a makefile

scott.mn

42 pointsgraue28 comments

The use of watch seems pretty pointless here, since you specifically don't want the original watch(1) behavior of clearing the terminal and redrawing.

Equivalent and no special tool required:

    while true; do make all; sleep 1; done

One problem that becomes apparent with a longer interval and longer rebuild times: if the make takes say 30s, and you're rebuilding every 60s, then your site will be updated every 1.5 minutes instead of every minute. Maybe not what you want.

A program you should really check out is trigger-listen [1]. It can run a command every minute regardless of how long the command takes to complete (as long as it takes < 1min). You can also trigger a rebuild whenever you want by writing a byte into a fifo. I use it to rebuild static sites when a certain branch is updated -- just do a trigger-pull in your git post-receive hook.

[1] http://www.superscript.com/trigger/trigger-listen.html

> One problem that becomes apparent with a longer interval and longer rebuild times: if the make takes say 30s, and you're rebuilding every 60s, then your site will be updated every 1.5 minutes instead of every minute. Maybe not what you want.

Easy enough to fix without resorting to a specialized tool:

  while true; do make all & sleep 1m; done

(Changing the sleep to a minute instead of a second, which may have been what you intended.)

That way you'll kick off a 'make all' once per minute regardless of how long it takes. That in turn might not be what you want if your make takes more than a minute, but then it's also simple enough to extend it thusly:

  while true; do make all & sleep 1m & wait; done

which will do the same but avoid having multiple makes running concurrently (if a minute elapses and the previous invocation hasn't finished yet, it waits for it before starting the next one).

    while true; do make all & sleep 1m; done

Welcome to orphan land. (^C to see.) Don't do this folks. The 'make all' invocations may also potentially overlap and corrupt your builds. To be fair, you fixed that issue here:

    while true; do make all & sleep 1m & wait; done

But the orphan issue remains.

On linux you can use inotify to avoid polling. I use such a rule for my makefiles :

    watch:
        while true ; do inotifywait -qe close_write $(SRC); $(MAKE) ; done

How do you use it? Like:

  $ make watch &
  $ vim

Or do you use separate screens?

I use separate screens. Usually I have one terminal for vim and one terminal with a shell for make, git, etc. Using this setup and a tiling window manager, I can see the build log whenever I type :w.

  while inotifywait -qe close_write $(SRC); do $(MAKE); done

This will stop trying to build at the first compile error.

No, it will stop when the directory disappears.

I use 'entr' for this, which uses kqueue(2) to avoid polling:

  * http://entrproject.org/
  * https://bitbucket.org/eradman/entr/

Neat, that looks like a much more well-engineered and less hacky solution. While I'm happy with what I have now, I've bookmarked this in case I need it later for a bigger/more complicated project.

I like it, nice and simple too, not even a configure script.

Created a tap for it on homebrew if anyone wants to use it.

brew tap mitchty/entr if you want to test it out.

There is also watchr: https://github.com/mynyml/watchr

Or you can use a build system like Tup, which uses inotify to notice when any dependencies change: http://gittup.org/tup/

My watch-make tool tries to do this "properly"; it uses GNU make's --print-data-base to figure out which files to watch for a change. You can run "watch-make test4" and it'll only run "make test4" when a file which is in the dependency chain for test4 changes, rather than any file in the directory tree.

It tweaks the PATH for child processes so recursive make invocations should work.

I've been lazy and not put yet made an npm package (it's implemented in node.js) or tested cross-platform. Guess I've no reason to be bored tomorrow.

https://github.com/andrewffff/watch-make

Awesome, starred it. I hope you do put it on npm!

If you're on mac, you can use fswatch instead:

https://github.com/alandipert/fswatch

It uses Mac APIs to monitor the file system and executes your command when it changes.

And there is https://bitbucket.org/ssaasen/spy if you want something that works on both Mac and Linux (disclaimer: author here). The "run" mode can be used for the "compile on fs event" use case.

I've used something similar before, but I've always found that after a while it makes me wary to save a file. Saving means a rebuild, meaning potential errors, leading to a let-down or sense of failure.

While computers have become a lot more stable, I still like to save often, and this kind of setup eventually makes you feel less like you want to save.

Interesting point. I usually don't mind breaking the build for a few minutes while implementing a biggish change, but I also commit often and can use "git diff" to find out what I changed and quickly get back to a working state.

A node version of this is to use supervisor [1].

E.g. in a makefile [2] for http://countw.tf/ (a great way to vent while exploring an old codebase with many err... sub-optimal choices)

  debug:
	browserify site/scripts/app.js -o site/scripts/app-built.js --debug

  watch:
	supervisor --watch site/scripts --ignore site/scripts/app-built.js --no-restart-on exit --exec make debug

1: https://github.com/isaacs/node-supervisor

2: https://github.com/joshka/countw.tf/blob/master/Makefile

Yet another fun use of watch, one of my favorite tools! I use it daily in combination with curl and grep to look for certain output of my JSON/XML services during development. The -d flag is especially useful in this regard.

  watch -d -n 5 "curl -s localhost:8080/some/service | grep status"

Along the same lines as the original post, I also use watch as an ad hoc, pseudo continuous integration tool for executing code that doesn't fit well within a REPL. [1]

[1] http://codeaweso.me/2012/09/using-watch-as-quick-continuous-...

The ability to do this based on file modifications is my favorite feature of the build tool SBT (a Java/Scala build tool). I really wish more build tools would integrate this natively. Usually this functionality only shows up as a plugin for a specific build task (such a Spork for Rails testing) or as a platform specific command line hack (which isn't as useful if the build tool has a large start up time).

If you're interested in this area there's a Ruby project which does the correct thing here, which is to only recompile/run `make` when a file changes. The original author has stopped maintaining the project. I've been working on a fork, called `observr`, and looking for contributors:

https://github.com/kevinburke/observr

For a nice, portable and generic version of the "do something when the filesystem changes" flavour of tool, check out the "watchmedo" command provided by watchdog (https://pypi.python.org/pypi/watchdog).

It's quite similar to watchr in ruby-land, but the executable is easier to use without creating a config file.

I've done something similar with inotify for compiling/concatenating CoffeeScript/SCSS/Haml in case anyone wants to use it: https://github.com/Tarential/autobuild-client

I suppose I should write up this ancient use of inotifywait and xpdf's -remote option on a blog somewhere:

https://news.ycombinator.com/item?id=5276477

Hah! I love it! Continuous integration with two UNIX utilities!

Reply on news.ycombinator.com