Sssgen: Make static websites simple again | Hacker News Reader

Sssgen: Make static websites simple again

ehuber.info

59 pointsedmundhuber34 comments

Meta: Looking at the code at [0], all of the code is at the root level. Please, don't do that. It makes it quite hard to do static analysis on the code (i.e. pylint), since merely importing the code to do the code tree walk will execute the code logic. Ditto generating documentation from the code.

It can create a fair bit of boilerplate (relative to your average Python boilerplate), but the `if name == "__main__":` guard keeps this under control.

[0] https://github.com/edmund-huber/sssgen/blob/master/bin/sssge...

Pylint doesn't actually import the code, just parses and processes the resulting AST, which is in my opinion how it should be done.

The guard is useful if your code is already composed of multiple functions/classes, which could be called from outside without running the full logic, but in this case I'm not sure it applies.

Fair enough, pylint and pep8 do not. Pychecker, pydoc, doctest, unittest... many other tools which are part of the typical Python development flow do.

Exactly this. This is not what "200-lines of simple Python" should look like.

Firstly this is super cool and much simpler than other SSGs, but whilst I applaud the stand against "convention over configuration", I read the docs and find all the magic. Files starting with underscore are ignored? Where was the configuration for that? Oh yeah, that's just a convention. Then there's the magic for "_inherit.yaml", which -- by convention of this tool -- is treated specially. You can't have it both ways.

You're absolutely right. I guess it's hard to be entirely magic-free. In this case, I'm pretty happy with the underscores thing since it's one pretty simple rule. Happy to hear alternatives, though. :)

I would view this as files with underscore being special and that is the convention. Being ignored is just the default special behaviour. Maybe in a future version there would be another file that does something more than being ignored.

Leave it to us developers to over-engineer easy stuff. Static Websites are simple. That's the entire point of them.

Every single change is instantly reflected on the page. A non-technical person can easily debug HTML (given countless resources ranging from MDN to W3Schools ~_~) than any templating language or what have you.

That said, for us devs this is yet another tool to play around with and maybe extract something useful for our own setups. Thank you.

Looks nice, but I don't think it will replace my current setup. I simply use makefiles (or rather mkfiles [0], because of the neater syntax) to administrate the generation of the site from markdown. That also makes it easy to add a deploy-target which deploys the site.

[0] http://doc.cat-v.org/plan_9/4th_edition/papers/mk

Mind documenting and displaying your toolchain? Would be interesting to see how you do it with Makefiles.

I'll do a more thorough write-up some time, but I do the obvious thing — use meta rules to generate the files using the appropriate tools.

I use Pandoc to convert to html and glue together the files. An index file lists which files go to the server. Some files are html only, others are also available as PDF.

Some of the things I do are less obvious. There is a rule to make the target «deadlinks» which iterates through the resulting html files and attempts to fetch the (first bytes) of each linke. The links which fail go into deadlinks.

There is also a target which lists spelling mistakes.

The deployment target is configured to fail if there are dead links or spelling-mistakes (there are also a few other sanity checks).

Did you implement stuff like a blog-aggregation-page and Atom-feed? How do you show the dependencies from aggregation to all aggregated items into the Makefile?

I tried using make, but then made my own 152-lines [0] of simple Python instead.

[0] https://gist.github.com/qznc/6fbdc37ad33e6853911a

I didnl’t try that since my site is not a blog.

Some files on my site depend on others in non-trivial ways. For them I just use an index file of dependencies, and cat the index file in the dependencies list. It is a bit tedious to update the lists, but I have sanity checks in place to tell me when something is missing.

It seems a case of "I didn't understand this engine's documentation, so I'm making my own engine to do this exact same thing".

Nope. The whole point of a "static site generator" should be for it to be SIMPLE. If it takes me more than 15 min to understand almost everything about it, I consider the said generator a failure. This is why I never bothered with Jekill or Pelican after trying to get them to do what I wanted and couldn't in less than 30min.

...and I fall back to my script which does "php + php -S to temporarily serve + wget to generate the actual static site".

sssgen seems like a breath of fresh air to me and I will actually use it. Thanks OP!!!

Also, thanks again for the sound decision of using Mako so I can actually have python code in my templates instead of wrestling with another "crippled" template language. And for jumping off the maddening "convention over configuration" wagon... I love configuration and explicitness for anything non-obvious

> The whole point of a "static site generator" should be for it to be SIMPLE.

This is incorrect. The point of a static site generator is to generate static site content. You may use it in a simple manner, but there are many projects and companies that maintain huge, complex websites using static site generators like Middleman and Jekyll (along with various proprietary static site generators), in combination with content managers like Contentful. I'm talking hundreds if not thousands of pages, ecommerce, massive blog, multiple content types, etc. etc. I've worked on such projects, as have many others around here.

I'm glad that enlightened companies are increasingly open to using static generators for such websites, since many (possibly most) of them don't need to run PHP or Rails if the site doesn't have dynamic content on the frontend.

For such organizations, the point of static site generators is to produce vastly more secure and faster sites that are much easier to maintain and less expensive to host. Not to mention the ease of caching.

The OPs project, nice as it is for a small, simple project, would be impossibly simple for such projects. Anyone who tried to use it for a project of the scale I've described would just end up re-writing an inferior version of Middleman.

ssgen sounds like a cool project, and it also sounds like you're using the right tool for the job in your projects, but don't make the mistake of discounting a tool as useless or too complex just because it's not the right one for your purposes.

Not taking a side here. :)

What do you think sssgen is missing that I could rip off of Middleman?

Or in general, what do you think is missing from sssgen to make it feasible to use for 100K+ -page sites? I'd really like to hear your thoughts.

I think your approach is great. There's clearly a demand for simple static site generators that favor configuration over convention, so unless you really want to pursue the Middleman audience, I'd keep on keeping on. But I'll offer a few thoughts for your purposes:

1. I think a small, well-tested core like you're doing is great. I also think for your purposes, Mako templates are the right way to go, since they allow super flexible logic in the templates.

2. If you want some inspiration from a kinda sorta similar approach, take a look at Metalsmith and its plugin ecosystem if you haven't already.

3. A great asset to any static site generator would be a collection of "starter themes/sites" that already have things like a static asset pipeline set up (using Gulp, Grunt, or whatever floats your boat).

If you really want to accommodate the bigger static sites, a few critical features:

1. Super fast compile times. This is why Hugo is rapidly gaining ground despite not having a plugin API. When you have thousands of static files, it's not fun waiting 10 or 20 minutes for each compile (Middleman and Jekyll have gotten faster, but they're still slow relative to Hugo and the Node-based static site generators).

2. With huge sites that are built by large teams, flexible templates like Mako almost become a liability, since careless team members can overload logic into templates. For these teams, you may wish to offer an alternate template choice, like Jinja.

3. Some kind of feature similar to Middleman's dynamic proxy pages, which would allow users to dynamically generate hundreds of pages basic of Contentful's API or their own JSON files. See: https://middlemanapp.com/advanced/dynamic_pages/

Finally, features Middleman's url helpers have proved to be a significant help on large static sites I've worked on, since the URLs are no longer hard-coded. There are a lot of small but significant features like this: localization extensions, pagination, etc. But if you have a good plugin API, all of these things can be created and offered as plugins.

But again, I think your current approach is just fine. If I were you, I'd perfect a simple core generator and then try to build up the ecosystem.

Thanks! Finally a static site generator that you can actually fully understand in 5min!

And the whole Mako + yaml + tree variables is so sweat and simple.

I like this project, mostly because it resembles similarities to my static site generator (not open source), with very similar concepts (except I'm using mustache), and slightly different internals, but about the same length, in Python.

Coincidentally, I've just started to re-do it as a python web app, that runs some limited dynamic processing (e.g. forms, logging, kudo-tracking, everything to keep it 100% self-hosted). I'm really interested if there are projects in this space that I'm not aware of, that I could just use...

Nikola has a plugin called webapp that's similar. it's kinda broken though and takes some fiddling, and is a bit buggy. I would be very interested in your project. I love static site generators but I hate having to remember all the commands. a web interface would be really nice.

I've started something like that too. Instead of a templating engine I am just providing a jsfiddle-esque interface that smashes all the parts together with a bit of magic on top to get tracking and forms.

I built my own simple static site generator in ~150 or so lines of Ruby. It's a fun thing to do.

That kind of inspired me to start a simple static website hosting service which is still in beta: http://www.statichosting.co/

I've tried all the other hosting options, and they are fine. I just never fell in love with any of them, so I made my own with a couple friends.

If you are interested in trying the beta, please sign up. We'll be opening it to more sites soon.

This is interesting, but even for a beta you should explicitly tell prospective customers about the limitations of your service:

- how big are the zip files they can send?

- any limitation on the outgoing bandwidth?

- will the static websites be public or will it be possible to configure some sort of basic authentication?

- etc. (I'm sure you get the idea)

Excellent points. I don't have all the answers to those questions. This is the best I got for now:

- I think the zip file size is nginx configurable and it's set to like 4 or 8 MB right now. - I don't have any bandwidth limitation at the moment, but if someone has a very high bandwidth site or wants to use the service to host/share files I'd have to cap those users somehow or maybe just charge more. - Right now they are public, but I'd be willing to explore basic auth if that's a requested feature.

In general. We are making a service that we want to exist and are trying to do the best for whoever decides to show up.

Thank you for those questions, I am too close to the product to even realize what people might ask, but those all seem obvious in retrospect.

> I've been asked to implement fairly complicated websites using static site generators

What does "fairly complicated" mean here? An example would be nice.

Sorry I didn't make that more clear. As an example, https://docs.fastly.com/ was my handiwork. More recently, https://teforia.com/ .

It is simple (for developers?), but not easily extendable. Simplicity is not an excuse for no plugin support nor not writing good documentations. And more powerful tools need better documentations. If you think other SSGs are not well documented, just call for help and improvement. Why do you reinvent the wheel without good documentations in mind?

Any way to run the site not compiled through python built-in server ? So I don't have to generate over and over to check out tag or template mistakes.

All of this has happened before and all of this will happen again.

So I have to create my own templates for atom feeds?

and so the cycle begins again

It's not documented and I don't want to read 200 lines of code.

Yeah, it's pretty funny that on the one hand, I'm saying "conventions make it necessary to eventually have to read the code", and on the other hand, I'm saying "read these 200 lines of code to understand how to use this static site generator".

That being said, for either Jekyll or Hugo, I've had to read >>200LOC in order to really understand a problem. :)

Reply on news.ycombinator.com