Show HN: Metalsmith – A pluggable static site generator | Hacker News Reader

Show HN: Metalsmith – A pluggable static site generator

metalsmith.io

174 pointsianstormtaylor46 comments

I like that static site generators are basically the Cups And Balls of our craft.

It's so well understood and constrained of a problem domain that we can now ignore the practical considerations and go all-out with the art itself. I feel like this design and api is a great example of that, much like Penn and Teller's Cups and Balls with clear cups[1] - wonderfully creative innovation within a completely and totally solved problem domain.

[1] https://www.youtube.com/watch?v=O_n3Zb3bW3g

One problem I see. On the one hand you say:

> All of the logic in Metalsmith is handled by plugins.

But on the other hand, you say this:

> Each plugin is invoked with the contents of the source directory, with every file parsed for optional YAML front-matter, like so…

The YAML parsing should be a plugin as well, some of us have existing JSON front-matter files.

Hmm yeah that's true. Tough call since that is the front-matter is a 99% case, and the idea of file's being able to have their own metadata is kinda core to the functionality. I wonder if I could just add support for both JSON/YAML by default. Would get annoying if people started asking for more front-matter types thought...

It's cerainly as sane as sane defaults come, but I agree that if the philosophy is 'everything-as-a-plugin' (another excellent choice for this kind of tool) then everything should be a plugin :)

Now to build something with this, I can see a 'dual write' plugin in my immediate future...

Thought about it a bit more (was actually convinced for a second) but then I realized that that would mean lots of the other plugins would be depending on the YAML/JSON plugins as well. And when you have dependencies between plugins like that things start to get too complicated for the benefits I think.

A JSON file is a YAML file, so what's the problem?

I'm not sure why this is getting downvoted (well, maybe the tone), it's true that YAML is a superset of JSON http://en.wikipedia.org/wiki/YAML#JSON

That's really cool actually.

You could have 'reading' and 'rendering' plugins. Reading plugins obtain the content and meta-data.

A possible use case would be generating pages from an RSS feed. You'd have to specify the meta-data separately in some config though.

(Sorry, really don't want to over-design something that seems simple and elegant. Just a thought.)

Thanks. Looking forward to trying it out this weekend. My current generator, which I'm generally happy with, is DocPad.

http://docpad.org

One thing I don't like about jekyll (unless I'm missing something), is that on site generation, the last modified time stamp of the files gets updated too even if the file content hasn't changed. This way a FTP program like Transmit will not be able to synchronize only the modified files.

Just about everything that supports ftp supports ssh+rsync. I'm not apologizing for the bug, but rsync+ssh is a sane default for synchronizing everything everywhere these days and sidesteps the problem almost entirely.

If this really works, I would love to see it replace the hellish jumble of team-editable documentation. I've seen Confluence, PBWiki, Google Sites, and a smattering of others used to no good ends...can we please switch to this now?

I'm actually in the process of converting our Segment.io docs to use it right now :) makes it way nicer for everyone* folks to just be able to edit Markdown, but still have the power to do lots of custom things to make the experience better.

* I was going to say for "less-technical" folks but then I realized that even technical people shouldn't have to be subjected to our current tangle of Jade files!

I agree, nice for writing/generating specs! Plus, check the raw documentation in markdown/markup/whatever you want into source control, right alongside the code, and reap all those benefits too.

Could be compared to assemble.io and stylistically reminds me of Gulp.

Gulp was the first thing I though of when looking through the examples.

Edit: In fact, could this whole thing just be a gulp plugin?

I don't see why not. My thought was "why even have this when you could just build the plugins for gulp?".

The problem with Gulp is just that it just adds too much extra cruft into the mix that isn't really necessary, mostly around running tasks from the CLI. Our general thought for build tasks is that all of that should be in a Makefile which will nicely handle mtime checks and everything for you, and is available on pretty much every setup out there.

The simplicity is nice because you can read through Metalsmith's source and really understand everything that it's doing very quickly: https://github.com/segmentio/metalsmith/blob/master/lib/inde...

It appears to be indeed both extremely simple, and extremely composable. How well does it handle large collections of files?

I haven't tried it on crazy amounts of files, but it's just using node's basic async I/O under the covers, and reading once. If you notice any sluggishness let me know!

It's also greatly impacted by what plugins choose to do. I had an extra clone call (literally cloning the buffers for each file) in the templating plugin at one point that like 50x'd the build time :) Everything I've done so far though has sub-second build times—quick enough that I've been building on every request[1] in development which makes things super simple.

[1]: https://github.com/segmentio/metalsmith.io/blob/master/serve...

This is really intriguing. I also recommend HarpJS (https://www.harp.io/) as well, which has some impressive pre-compile features.

This looks very cool. I like the focus on plugins, it would be pretty awesome to have a flexible static site generator with an active plugin ecosystem.

I like this design. It kind of goes in the same direction as my github.com/andrewf/filtdir while being significantly more refined. This seems to make the whole directory structure available to plugins, while my tool only works one file at a time. It might even convince me to switch.

A functional type approach. It will be interesting to look more into it. It would be nice to have a .map, .filter functions (if they don't already exist). So, if you don't want to rebuild everything you could do a .filter(htmlDate < mdDate) type workflow.

Not just a nice tool, but a great implementation. Simple, lean code, no promises or anything fancy.

The overall structure of the thing looks pretty promise-y to me.

How do I get this working on Windows? I've installed it with npm install metalsmith. Now what?

You’re right, the “Install It” section should make that clearer. Anyway, there is an explanation of basic usage if you follow the link to “CLI” (https://github.com/segmentio/metalsmith#cli) in that section. It says you can create a `metalsmith.json` file that lists source, destination, and plugins in the described format, and then run `metalsmith` from the command line to build your pages according to the configuration file. And I think you will probably also have to install any plugins you use beforehand. Plugin installation instructions are all in the plugin READMEs – they are basically all just `npm install <some-package-name>`.

But it’s harder to figure out how to create my own local plugins and make sure Metalsmith is able to see them. And the documentation should make it clearer how to use the JavaScript API, in the context of a static file generator, where most people are not thinking about writing a program. It took me a bit of thinking to realize that you would have to create a `whatever.js` file inside the directory containing JavaScript code with `Metalsmith(".")….build()`, and then just run it with `node whatever.js`.

Sorry about that! Just updated the Readme and website to hopefully make that clearer. And I've added Readme's to all of the examples[1] now too, some of which use the Javascript API and some the CLI.

[1]: https://github.com/segmentio/metalsmith/tree/master/examples

Somewhat relevant plug:

I have a similar weekend project that is aimed towards generating static markup. The syntax looks a bit like haml with simple to use mixins, includes, and variables.

http://rubygems.org/gems/aml

You should have linked to its home page https://abstractmarkup.com/. The RubyGems page does nothing to sell me on why I should bother installing your gem.

Did anyone else manage to install it? I'm getting a "shasum check failed" error.

The project scaffold generator part sounds pretty interesting because I needed it pretty often and eventually had to build that for myself.

Just republished! Can you let me know if you still see it? Sorry about that :$

Got the checksum error, too. Successful after the republish.

Everything is alright now, thanks :)

Couldn't choose between Jekyll and Middleman, but now I think I'm going with Metalsmith. No ruby, understandable plugin structure… I'm sold!

With sooooo many static site generators (all very well done too), would be awesome to have a side-by-side by-feature comparison table.

Apparently there are many aggregation efforts, but no by-feature comparison. https://github.com/jaspervdj/static-site-generator-compariso...

I fail to see the big advantage of this over its competitors.

Yeah it depends on what your use case is to begin with. If it's just the simplest blog with a running series of Markdown files, then really any of the static site generators will do. But once you get into trying to implement some more advanced features then you run up against the limitations of most (if not all) of them because they assume way too much up front. A couple real-world examples from us at Segment.io are:

Documentation - for our docs[1] we want to be able to use the same simple static site generator without having all of the blogging logic. Basically the nesting of the files should result in the nesting of the URLs. But we also want to be able to tie in metadata that we have in our database about all of our integrations. And we'd also like to be able to write custom handlebars helpers that turn a simple JSON object into a widget that renders API calls in any of our supported languages.

Academy - for our academy[2] I really want to get to the point where we can generate PDFs for each of our articles and being to re-distribute them that was as eBooks (or potentially for a collection of articles) because that kind of thing appeals to enterprises who are looking for guidance. And we could even end up doing the same thing with our docs pages. And then we also want to have custom handlebars helpers for

Blog - for our blog[3] we want just the most basic implementation, although maybe with some niceties about author metadata to load in avatars and such.

Whenever you try and get into additional features that weren't considered by the original "static site" (or worse "static blog") generators, you usually end up building really cludgey code, if it's even possible.

So with Metalsmith we avoid all of that, because the plugins can do whatever they want, and it's super trivial to add local plugins to the mix if you're cooking up something which you know is unique to just you.

And the last thing was that we were sick of having a Ruby dependency (with all of the associated slowness) just to build our blog with Jekyll. Basically was increasing build times by an order of magnitude.

[1]: https://segment.io/docs [2]: https://segment.io/academy [3]: https://segment.io/blog

This is a nit, but non-constructors should be lowercase.

It actually is a constructor, just lets you omit the `new` keyword if you choose since I think it's nice not to have to do that sometimes.

So simple, yet so powerful.

Very nice. Well done.

Awesome.

Reply on news.ycombinator.com