Static site generators need less rigid content APIs
fvsch.com
fvsch.com
As I tested myself several SSG, I (as many it seems !) thought « Well, none fit my needs so let’s make one »
My problem was that I need an interface for the final clients, which are no tech at all, to be able to edit content.
Solutions like Hugo are not possible as it would means install it on a computer at the client office, and make him/her understand how to deploy. No go.
As the post states, they would also have to learn some stuff. Even more no go.
So for those case, the solution is to use a CMS like Wordpress. Which I don’t want.
So I built my own, using the best of both worlds.
For the backoffice, for managing content, a PHP/MySQL classic, with a media gallery and a tinyMce. Pages are in categories which can have subcategories and pages, as deep as you want.
Each page must have a template.
A preview button (which is the key) and an export site button.
And that’s all.
Of course, it only suits my needs (as me or some clients like web agencies will be in charge of the templating) and have no means of going public, so don’t fit what OP was looking for.
But it works, it was a two days job to create, and as it’s all mine, I can add any functionality I want.
The idea is that Wordpress clients can be used to create content, but the output is a static site served from S3 and Cloudfront.
If you want to try it, send me an email: hello@perspect.com
There's nothing to stop you from using JavaScript in the browser to do any number of things.
There are already a few CMSs for static sites, much like the solution you've made. The old-school one is Movable Type, but nowadays there's also one by Netlify, Publii, Siteleaf and a few others. There are also plugins for Wordpress that generate static sites from it (meaning you don't have to expose WP itself to the outside world).
That said, making your own solution can be the best option.
There are other solutions, such as using wordpress or contentful to provide content and then regenerate the website when something changes. I think the creator of Gatsby is creating something like that as a business.
It doesn't handle authentication, so you'll need to put /admin under authentication on the web server level, but it allows you to edit and write pages, save them to drafts, view them as drafts, publish them, modify their metadata...
The only downside is image hosting, where the friction is still a bit too high.
2) Datomic is such a database – an accumulate-only log with explicit time dimension that can be queried JOIN-style, WHERE-style and GraphQL-style
3) Thus crud apps backed by Datomic are suitable to be delivered from CDN like static sites – rendered pages, API, everything
Here is such a static site backed by a database: http://www2.hyperfiddle.net/:markdown/ final rendered site: http://www.hyperfiddle.net/ – since it is server rendered React.js, obviously you can use React.js to render your pages, so you get a fully extensible markdown pipeline as you can see.
Here is my blog, a static site: http://www.dustingetz.com/ It is also a progressive web app, so if you navigate around with the browser dev tools open, you will see API requests served from disk since they are Cache-Control: Immutable. Yet, here is the live CRUD dashboard for it's database: http://dustingetzcom.hyperfiddle.net/:hyperblog!dashboard/
It's great for docs, for example: http://docs.hyperfiddle.net/ (We turned off server side rendering for this one but you can leave it on!)
Did I mention Hyperfiddle also does arbitrary CRUD database apps, live in your browser - queries, schema, views, everything?
dustingetz.com and the docs app both have embedded hyperfiddle editors, so they run in a development configuration. The 10MB is for the real-time editor which depends on the ClojureScript compiler and standard library. A production javascript configuration will bake/compile/dead-code-eliminate in advance and then asset sizes will be comparable to any ClojureScript project (which is to say, better than almost all JavaScript projects). We haven't gotten around to doing this yet. Thanks for checking out hyperfiddle!
I tried pelican, but failed as didn't find an acceptable way to write a content. I have a kind of blog with around 30 to 50 photos per post and there is no way to put into the text as convenient as in wysiwyg where it is possible to select the picture I need.
Thus I decided to abandon the idea to migrate to SSG. Even though I tried it and dreamed about it a lot :)
I've basically stopped using my static site as a result of media management being a pain. I'm probably going to do back to Wordpress or something (shock horror!). Wordpress isn't hard to manage at all quite frankly, it's just a different set of compromises compared to a static site. It's also super easy to manage rich media and as I'm falling squarely on the side of 'get shit done' it's looking like a good option.
It's trivial to cache it well and update it. Yeah it's not as secure but there's a risk vs usability judgement that has to be made. And I think I'd prefer to be producing content and having some good backups rather than just not use my site.
Send me an email at hello@perspect.com to try the service.
I agree with you that sitegen from git is just too cumbersome when the articles aren't very bare/simple. Also when there are multiple users contributing, especially non-technical users.
It's also a pain to self-host (especially compared to competition seeing as it's basically a requirement to go for a full VPS install) and is pretty opaque when anything goes wrong.
It's easy to use and slick, and I'm not OP, but it's not for me at least.
We also support uploading images to Cloudinary instead of committing them to your git repo to prevent repo bloat.
Depending on what you're trying to do, a non-CMS option that will remove some of the tedium of inserting all these images is to take advantage of Hugo's Page Bundles feature[1]. With this you could just group a bunch of images in a folder and iterate over them in your templates.
It sounds like this article is using "simple" to mean "easy", and "elegant" to mean "simple".
An SSG is simple, and it's easy for people who know "the command line, the plain-files-and-scripts philosophy, git and ssh" (which is not quite the same as "developers", though there's usually a large overlap).
for file in *.md ; do
echo " converting $file"
pandoc $file -f markdown -t html5 \
-H templates/header-prod.html \
-B templates/nav.html \
-A templates/footer-prod.html \
--template=./default.html5 \
-o ../out/${file%.md}.html \
-s --data-dir=./ \
--highlight-style pygments \
--variable=file:${file%.md}.html
done
scp -r ../out/* me@cloud:/var/www/html
Besides that I agree with the assessment that these tools are from developers for developers. In the end a static site generator is more like a special kind of build tool and less like a CMS (what non-technical users would need).
1. People often forget that "making your own" also entails a huge risk for a client. What happens if you go bust? We're constantly being contacted by organisations who were talked into a proprietary CMS and then left in the lurch when either their relationship with the agency broke down (which often happens in this situation because the agency know the client is locked into their CMS) or a company's lone developer left.
2. No content editor should have to use anything much beyond a Word-like interface, even if there are separate TinyMCE fields for separate areas in a page template. Even Markdown, which seems elegant and simple to developers, can cause all sorts of issues with non-technical content editors - and most clients outside the development bubble are non-technical. I feel like the lessons learned at the beginning of CMSes - for example, you shouldn't expect anything but pain from expecting clients to edit HTML - haven't been fully understood.
The one end-user benefit from SSGs seems to be that the site is packaged as static files and is therefore faster and easier to deploy, but is that really worth jumping through all these hoops? If any of these SSGs wants to reach a broader market, they'll need to have a WordPress-like interface that then publishes the static files and somehow manages to cope with dynamic things like search and filtering in an accessible way.
One service I've been interested by is https://www.hardypress.com/ (just tried it out, not affiliated in any way) which effectively takes a WordPress site and publishes a static version. They cope with form submissions in a similar way to Netlify (I think) and also deal relatively with querystrings that trigger a filtered listing. I think this is the only direction I'd be comfortable with for my clients because it gives them all the benefits of a user-focused CMS with a static site generator, while allowing them to take the site to one of the many other WP agencies if they choose to.
For example, in the past I have built a multilingual media database with Lektor, where items could be, say, books or movies, and depending on the content type I could enter a "number of pages" (int) or "duration" (time) or "author" (selection from a list of "person"-typed contents), and the admin interface would be auto-generated. I haven't seen a similar functionality in other SSGs.
Downside: For large sites rebuilding takes very, very long...
Rolling your own specific static generator is quite easy, the bulk of the work being gluing libraries together. What is worse, using an existing tool in a way it is not meant to be used or maintaining your own?
Exactly. I created a tool to help with this as well.
https://github.com/pauldotknopf/statik
It has no features, by design.
The entire "generator" is a silly Python script: https://github.com/tresoldi/tresoldi.github.io/blob/master/b... (the site is at http://www.tiagotresoldi.com/ , in case anyone cares).
The content API that is not limited to filesystem data puts it in a different category than other SSGs for me. They also seem to have the ambition to take on wordpress. It can involve some configuration though. It doesn‘t make the use case in the article trivially easy either.
I mean, how many dependencies/ceremony do we need to convert .txt > .html?
Webpack? Graphql? Client-side routing? I mean, if you want to have fun with shiny tech, go for it, but it really is the wrong tool for the job.
Gatsby gives you neither.
Give me a tree navigation. Oh, it's not supported? Now, in order to implement it, I have to figure nodes/graphql/webpack/fragments (I know the tech, but it it's _used_) to get a solution that I could whip up myself in 30 minutes if the tooling and techstach wasn't bogging me down.
There is a hidden cost to large peices of software with high ceremony. Most people ignore it because of the initial speed you get in setting up the project, but fail to realize the downstream time that will be wasted dancing with these large hammers.
I urge you to just take a look at the GitHub issues and ask yourself "How much blame can be given to the tooling/tech stack for this given issue?"
While I do know React well enough, when trying to work with local content I had to follow a tutorial that asked me:
1. To install 2 plugins 2. To copy-paste 3 different chunks of boilerplate code (configuration, the code that maps filesystem data into some kind of representation — maybe using GraphQL already, I’m not sure — and lastly the GraphQL queries to retrieve that data). 3. To personalize this boilerplate code, which requires me to learn GraphQL.
Other tools can do similar work without plugins, without having to learn a second language (GraphQL) separate from the core language that the tool is using (JavaScript), and without having to write code to feed content into a in-program representation yourself.
It could be that all this overhead is worthwhile because it enables you to do complex things. But from what I’ve read I wasn’t even sure that Gatsby could achieve my use case without me duplicating my information architecture as “feed content to the in-program representation” code, and I didn’t want to risk spending a second day only to find out that it couldn’t. (Your comments about tree navigation, as well as some of the GitHub issues I’ve read, suggest I was right to bail out.)
My actual takeaway is that Gatsby might be a good fit if you already know React and GraphQL (or already wanted to spend time learning those), and are working with remote content from a headless CMS and some other sources. But for local content only, unless you already know Gatsby or want to use that project to learn it, it’s needlessly complex and overkill.
And I also want to use headless CMSs or other data sources, in my current project a GraphQL source, which maps right in with the (very recent) schema stitching. So I think gatsby is the right choice for me, but I will carefully consider recommending it in the future. Being a VC backed company they may have more cheerleading than other open source SSGs going on.
"Just a txt > html converter" is what every developer builds for themselves at some point and hence the 300 different projects that already do that. I‘m glad someone is making a database backed (kind of) static site generator for once, even if it brings some code weight with it.
I read a blog post recently that really resonated with me [0].
> And dependencies? People easily add overengineered “full package solutions” to solve the simplest problems without considering their costs. And those dependencies bring other dependencies. You end up with a tree that is something in between of horror story (OMG so big and full of conflicts) and comedy (there’s no reason we include these, yet here they are):
But yeah, I don't mind where the data is stored. It's the html generation that people go overkill with.
Spend an extra day and build your own. It will fit you exactly. No more, no less.
> but saying it's a poor tool because it has a lot of dependencies is silly and immaterial.
You're missing also the time it takes to customize it. For example: https://github.com/gatsbyjs/gatsby/issues/4583
How complicated can it be to have simple tree navigation? Turns out, it's pretty complicated...
I could role my own in 20 minutes without the abstractions, but no, I'm fighting uncessaray abtractions/layers/tooling that gives very little value when it comes to converting .text -> .html.
It blows my mind that people don't see this.
Read this: http://tonsky.me/blog/disenchantment/
https://github.com/pauldotknopf/statik
> This is a simple tool/library. There are no opinions or abstractions, aside from the abstraction needed to host and export content. There is nothing preventing you, the developer, from doing what you want with your project. Parse and render markdown files in a directory for a blog? Build a user manual? What ever you want, you can do.
I used it for my resume: https://github.com/pauldotknopf/resume
See my other HN comment.
https://news.ycombinator.com/item?id=18011099
> I tried using Gatsby for a project of mine. The moment I tried to do anything not supported OOTB, it seemed I was fighting the tooling (and webpack) at every corner. It was incredibly difficult just to get a simple tree navigation.
> One thing I thankful for though is that it gave me a new-found love for the simple and non-flashy libs/tooling.
> In the end, I wound up writing my own static site generator.
> You may say to yourself "So you just wrote your own Gatsby!?"
> No, I didn't. I wrote a thin lib that you can register endpoints and extract them to disk. It does absolutely nothing else. The idea is that I will wrote my own markdown rendering, navigation, html/css, etc for each project. "But what about the time it takes to implement all the features you need!" The time it takes to implement these minor things take far less time in the long run, and I will never have to be in an endless fight with the tooling to get simple tree navigation. Every feature I implement is exactly what I need, no more, no less.
> Sure, it isn't as cool as React, webpack, etc. But I'm a lot happier.
I gave up on Jekyll when I tried to hook it up to hit & push each published site as a branch patented to the site branch & the corresponding master commit that was used to generate it.
I ended up fiddling with my SSG and exploring software designs that make it easy to add in weird extra behaviour. I really like how Dist::Zilla (Perl package publishing tool) allows so much flexibility and reuse, without many strong opinions of its own (mostly that packages are a collection of files and that prepare plugins should run before publish ones).
I made a static website with it recently and found it very quick to use, assuming you know React. No config or setup required (something rare on js projects nowadays), really just "npm install react react-dom next" and you are good to go.
Build a react component at "pages/about.js" and boom, yoursite.com/about routes to this component.
(At least, that how it used to be last time I used it)
I work for Forestry.io, and our product is a CMS that aims to ease the pain of non-developers managing content for static sites. We are committed to bridging the gap between copywriters and developers. I am a big believer in the potential for static sites to transform the web, but I realize there is a lot of ground left to cover.
• it is template-driven, via Go templates engine (the theme/wiki.tpl is the entry point, all other templates are loaded dynamically) — no need to put files in special place in hierarchy (unless you impose it on yourself);
• it has `glob` function for matching files by pattern, e.g. `glob "/20??-??-??-*.md"`, and can render the contents of files loaded this way;
• it has `matchre` function which allows simple editing of page contents (you should be able to cut contents before/after a `<!--more-->`, etc.); by the way, I kinda feel, that the three points above, taken together, probably already make it Turing-complete in some unholy, non-euclidean, blasphemous way, akin to C++ templates...
• the "only" caveat being... it's not actually a SSG (yet™). It's more of a "personal wiki/notetaking" app currently. But the next step and goal in its development is to convert it to a SSG, only I'm still pondering various approaches to go there. And... it's kinda on hiatus now, in that I'm not developing it recently. But if I will develop it, it'd be exactly to convert it to a SSG now, so you should be fairly safe to "watch" it on GitHub. If anything shows up in your inbox, most probably it'd be a set of commits converting it to a SSG. And this very thread on HN may, or may not, make me get back to work on that ;)
Please note it's also poorly documented as of now. Though it has some reasonable --help message contents, see: https://github.com/akavel/g-wiki/blob/fork/wiki.go#L49-L110
Also, for editing markdown, I personally like https://typora.io/ and, more recently, https://zettlr.com/
EDIT: You could actually already try running `wget` on it. This is actually one of the approaches I'm considering.
> Originating in simple blog engines, static site generators treat pages as a single content chunk (often in Markdown) with some metadata sprinkled on top. If you need several long chunks of content (say, a product short description, long description, technical specs, and a list of vendors), you’re out of luck.
I've struggled with this too. I'd really like to write in unadorned Markdown, because I'm using Trello as a headless CMS[0], but I also want the Markdown to map to an object containing discrete chunks of content.
The solution I settled on is to map level 1 headings to separate properties. The content that follows each level 1 heading is converted to HTML and becomes the value of the corresponding property - unless the content is a code block, in which case the code block is parsed as YAML to get a nested object instead of HTML.
I'm one of the Co-founders. We built TakeShape to be a more end to end static site CMS. TakeShape has a GraphQL API and Static Site generator that uses nunjucks for templating. You could also just as easily only use the GraphQL API with React or GatsbyJS's new native 3rd party API support.
TakeShape has built-in integration with s3, GCS, FTP and Netlify.
TakeShape comes after 7 years of running a design agency and trying to balance the best of many different styles of CMS and site creation approaches.
Also the author completely missed the point that everything is a "page".
I wish I had time to write a detailed reply to this post but looks like the author has made up his mind to puke on Hugo. I hope this reply serves as a warning that the author's warning about Hugo is very uneducated.
AMA regarding clarification about Hugo in that post.
And I've been using it for a very long time now. I actually wrote the original _index.md explainer page for the docs at the time the switch was made to stop it being 'abused' for section front matter. At least, I think that was the work around people were using, I forget after all the changes there have been.
Hugo is a powerful tool. I'm sure there are ways to do almost anything with it, and some of them answer the critiques in the article. But it's constantly changing (it's still a long way from version 1.0) and its power and flexibility bring a steep learning curve and their own way of doing things, which seems to also be what the article is commenting on in general with static site generators.
I am not saying that people are born knowing how to use Hugo. Hugo is powerful. If you want to tweak it (the theme), change how your site looks and all, you would need to understand how Hugo works, learn templating,etc.
If you are happy with one of the existing templates, you don't need most of that.
Nothing comes for free. You either learn or live with what an existing theme has. I think, this applies, to anything in general.
The section I wrote about Hugo, and indeed my whole article, is not purely about technical capabilities. It’s about user capabilities when using specific software, so there’s some focus on what the software can do, but also on the user’s perception of what the software can do and on the learning curve.
In my day-and-a-half with Hugo, I’ve read most of the docs twice, and have read some old issues and release notes. It looks like some of the complexity in Hugo’s design comes from adding features and redesigning some features over time. This process tends to accumulate cruft, and calls for compromise between conceptual clarity and backwards compatibility.
At one point I’ve read release note (for 0.20 I think) that said something like “from now on, everything is a Page”, but was surprised that different kinds of pages had access to different kind of data, somewhat arbitrarily. By contrast, if you start with a concept like “everything is a page and has a predictable feature set”, it’s easier to have this simpler design correctly implemented and correctly reflected in documentation.
There's always MDX.