Why I built my own static site generator
yakkomajuri.github.io
yakkomajuri.github.io
You don't have to be on the Node treadmill to create a static site generator. You can use something stable!
(I wouldn't recommend it.)
If it ever becomes too slow, maybe I look into using xargs -P or adding more '&' between commands.
Node isn't always a treadmill. JavaScript itself is very stable, much more than many other languages. The big thing you need is something to transform markdown to HTML. You can either write it yourself, or take one that's stable. The rest can be done in plain JS, without libraries.
Shell scripts are only as stable as the executables that they invoke (and portability is a separate matter worth taking into account, too—few people who mention stability address the issue of incompatible variations of shell and make across different systems). In this case, the upper bound is determined by peg-multimarkdown that you depend on.
I also built a blog engine on top of git using sh scripts and the core-utils in git hooks (https://p4bl0.net/shebang/fugitive-readme.html), but that was a long time ago and I don't use it anymore. It had a dozen users at some point and I even received a few contributions to the project. Fun times :).
I could add menu / link tracking pretty easily but I don't need it. My script is less than 15 lines long and can convert any number of markdown files into their own pages on my site, ready to serve.
I've done most of this in awk.
I also primarily target the Gemini protocol, so the web presence is a bit of an afterthought and that probably shows as well. I deliberately included every limitation of gemtext into the HTML rendering of the documents, including no inline links and no inline images. It does do some special rendering but very little.
It's backed by a git repository with gemtext files that get rendered into HTML. There is also a gmi->gmi conversion for gemini, as I introduced a few non-standard tags to instruct the page generator to add stuff like generating feeds for the blog part and so on.
The result is weird and not at all intuitive. I also realized this is basically off-brand Confluence except not as hard to navigate and 1000 times faster.
But yeah, the main point is experimentation. Testing ideas and seeing what sticks. Some of it does, but not everything. Like I have no headers for my web sites. My thought was that it's just a part you scroll by, a piece of pointless self-promotion. I wanted to elevate the content above all else. Turns out it's pretty confusing when you do that. Books have a cover for a reason.
Sure it’s not customizable at all: it can only generate my site. And that’s fine. I like it that way.
For example: I sometimes translate poetry, and I have a bunch of code that renders individual poems from plaintext (not Markdown, because newlines and whitespace _are_ significant): https://github.com/nathell/nhp/blob/master/src/nhp/poems.clj
My deploy script (it's just 5-10 lines of bash) mostly for some of my "old" angularjs(yes v1)
1) merge all js files (cat file.js >> app.js)
2) make sure any files that request/include JS and HTML files are not cached (generate new random ?rcc=random-num)
find -type f | egrep "html|js" | xargs sed -Ei s/"rcc=([0-9]+)"/rcc=$RANDOM/g
3) ssh-copy over to serverSexy ? Absolutely not , does it work ? Like a dream :)
Having the ability to just type "./deploy-aws.sh" and be done in 5 sec is an amazing ability to have for you project. Especially if you are SOLO founder.
When I push to the production branch, a github action does an aws s3 sync. My web service pulls static content from the s3 bucket.
It works well and i really like github actions for this kind of work.
Unfortunately, we typically lack the same enthusiasm for actually writing content for said static sites.
So you end up with 1,000,000 developer blogs with an average of 1 post.
The title of which is “how I built this blog using X”
After having gone through this charade many times myself (I’m a high level procrastinator), if your goal is to publish on the internet, do not spend time creating a custom static site generator.
In fact, don’t use a static site generator at all (you inevitably won’t remember how it works and will have broken dependencies years later when it comes time to pen post number 2).
Just sign up for a basic site builder (Webflow if it’s a company site, squarespace if it’s a personal blog), pick a not-offensive template, and be done with it.
Trust me, nobody will think less of you.
By eliminating the decision points around building something, you can force yourself to not waste hours comparing CSS frameworks and instead actually write something of value for other people.
I don't think that's an obstacle to writing. Let's be honest, the average blog lifetime is probably lasts less than a dozen posts. The ones that last are weird outliers, and that's not a problem from choosing the wrong technology.
It absolutely has been for me, and I'm willing to bet if you actually track your time, you might be surprised.
Obsessing over building is like a warm blanket for someone who's a developer by trade. Writing in public, by contrast, will make a non-writer extremely uncomfortable and vulnerable.
And humans are super good at pain-avoidance.
I spent roughly 200 hours trying to be clever about building my last blog (experimenting with static site generators, headless CMS options, markdown flavors, CSS frameworks, build process, etc) when I was first getting enamored with the static site world.
Fast forward a few years, it turns out I built a monster that I can't remember how to use, nor do I care to.
And I realized instead of getting my ideas into the world, I hid behind that security blanket of "building."
Seems pretty common that people have a romanticized idea of writing, and like the idea of being a writer much more than they actually enjoy the writing process.
My most recent attempt to rekindle my old blog turned into a write-a-SSG-that-can-import-tumblr and then a quest to recapture the nostalgic MySpace/text-mode feel, and no actual new blog entries.
Writing an SSG was procrastination at its finest. I did manage to migrate to ghpages and make the old tumblr and google-sites versions auto-redirect.
Its telling how, over the years, I've had to keep moving my free-hosted blog to a new provider. This is the third time. I'm hoping ghpages are here to stay and don't start doing a sourceforge ad angle or anything...
>Beyond all the extra stuff Gatsby ships with, a lot of Gatsby blog themes try to make things "easier" for you by abstracting away the internals and exposing a config.js file, where you add your name, a bio, some links, and Gatsby does the rest.
>But that comes at a cost. And that cost was made clear by me hunting the favicon file in the directory for a while only to find that some plugin auto-generated it based on a path for a profile photo you could set in the config.
Software design should adhere to the maxim: simple stuff should be simple, complex stuff should be possible.
When software adds so much complexity that simple stuff is no longer simple, it's time to revisit the design.
all: $(addprefix $(OUTDIR)/, $(addsuffix .html, $(PAGES))) $(addprefix $(OUTDIR)/, $(notdir $(wildcard imgs/*))) $(OUTDIR)/styles.css
$(OUTDIR)/%: imgs/% | $(OUTDIR)
ln $< $@
$(OUTDIR)/%.html: pages/%.page template | $(OUTDIR)
$(BUILDER) template $< $@
$(OUTDIR)/styles.css: styles.css
ln $< $@
deploy:
rsync -avh $(OUTDIR)/ $(DEST)You might want to have all html pages added to a <ul> in an index.html.
Then you might want a sitemap (similar solution to above).
Then you might want an RSS feed (and an atom feed).
Then you might want height and width attributes automatically set in <img> tags.
It's all doable, but soon your build script is a classic static site generator and dwarfs the Makefile in functionality and complexity.
If you're not going to do the absolute minimum for your site's users why bother making a site in the first place?
'make' is older than html by almost 20 years.
I write my website in Emacs org mode, then export to html. I like to think that I win on nerd points :-)
Now I'm using a Frankenstein's monster of org-pandoc and `cat`
I don't know what this means - I write a normal org document with headings and suchlike, and then export. For site structure I store each page as a separate org file and the html-export exports the links correctly.
It might just be that I am happy with the output so I didn't try to change anything, so didn't run into the problem you did.
For example, suppose you wanted a sidebar, or an absolute-positioned floating navbar or something.
I suppose that I would have to specify that I wanted to export a ToC with a maximum depth of 1. Currently all my css rules are in a different file (not managed by emacs/org) and I would modify the css in that file to ensure that the ToC appears in a left positioned div, or a floating absolute div, etc.
To be honest, I think that that will only work for a ToC. If you asked "how would you specify a list for the sidebar and another for the top menu navbar" I won't be able to give you an answer[1].
[1] Hey, maybe it can be done, but not as far as I know.
---
template: custom.html
---
# My normal markdown
Since it's a fairly standard tool, parsing it is trivial with npm packages[2] and you don't need to be manually parsing the strings: import fm from "front-matter";
const data = await fs.readFile(...);
const { attributes, body } = await fm(data);
// attributes: { template: "custom.html" }
// body: "# My normal markdown ..."
[1] https://jekyllrb.com/docs/front-matter/[0]: after seeing and deriding the code for wordpress. another, much bigger, last laugh there too.
I write Markdown and commit to a git repo, then a web hook in Caddy pulls and builds the static site, anything I commit is up in a matter of seconds.
I'd much rather spend what little time I have writing blogs to help people learn new things than write YASSG.
Someone below linked to a site with over 400 SSGs.I think OPs "why" was really "because I wanted to", after having only tried Jekyll and Gatsby (apparently).
Note: this only applies if you're planning to write your own HTML. If you're using a packaged theme, Hugo would likely be a great choice. I didn't want to use an existing theme, because all the ones I found either didn't do what I wanted or were bloated (or they had Facebook and Twitter logos built in...). For the record, my entire index page (including CSS and favicon) is 1/5th the size of just the CSS used in Hugo's tutorial.
> you add your name, a bio, some links, and Gatsby does the rest.
>
> But that comes at a cost. And that cost was made clear by me hunting the favicon file
1) it's more fun
2) it's good practice as a programmer
3) you get complete control over what content is generated (i.e. can minimize CSS/JS size)
4) you get complete customization - you can make your own way to template things that is convenient for you, or auto-import a particular file format you like to keep for your notes on disk, etc
5) you don't have to learn how to use wordpress
(It's not to say I think using wordpress is bad - if you like it, use it!)
Heck, my Wordpress blogs are faster than when I had static sites on shared hosting.
In other words, it does precisely what you're saying it should do.
I have a series of content websites, and at first I used Jekyll and it was fast to load, but everything took 10x longer to do than just using WordPress.
I now use WordPress on a VPS and served with Litespeed Enterprise and the performance (read: load times and CWV/GMetrix scores) is very very good - better than static sites hosted on shared hosting, in-fact.
It is great for non-technical people, but the risks are high.
I personally find that github pages is a better alternative now: most users can handle doing edits in github's web UI, and on commit the changes are auto-deployed to the live site and appear within 60 seconds or so. And it helps me to see their git history when they screw something up.
It feel like it's the best of both worlds, because it's simple to learn and customize, but there are plugins for the things you don't want to spend time writing yourself. Note: it is Node-based, so only useful if you're comfortable with JavaScript.
For example, I'm using plugins to: check for broken links, generate an RSS feed, and run a test server with automatic reloading.
But then I was able to easily add in my own code to handle relative links, generate Graphviz diagrams, and format dates.
One other recommendation: I hated almost every template language I ran across (Hugo's, Liquid, Nunjucks, EJS), but I'm thrilled with the simplicity of Handlebars (https://handlebarsjs.com/), although it is a bit limiting and the "block helper with parameters" syntax is strange (perhaps an indicator that I'm trying to do too much in the templating language!).
I have built 2 SSG's over the years, once in 2009 and one this year. So it's fun to read through someone's internal thought process on their motivations and decisions.
Someone put together this mega list of 460 SSGs that I found helpful for inspiration and learning: https://staticsitegenerators.net/. Also would be a good place to add Teeny, though not sure if they are still updating the actual website.
I guess I finally found out the theme I have been searching for.
Use your own favourite language etc. but I wonder how many people who call themselves devs would be able to do something equivalent in a few days, considering all of the little gotchas with parsing, performance, flexibility etc.
Maybe this sort of thing would be better than a normal FANG interview where they just want to see how you would actually get something done?
It was great when I started. I did some teaching back then and put my lectures in its own section, which I could re-build and serve to the local network for my students.
I no longer know how it works. I don't care to maintain it. It needs big changes to handle something like embedding a Jupyter notebook. And it depends on Python 2.6(!).
With hundreds of pages, and its own custom URL layout that I don't want to break, I dread migrating to a modern system.
Thanks to a custom build with kotlinxhtml and a bunch of other tools, I'm very efficient. High performance thanks to simple html+css and a little vanilla js, so the pages show up instantly with 38k+ monthly visitors on a low-spec cheap server. No background loading or rendering in js or whatever.
[1] https://www.project-daily.com/pages/an-experiment_1065.html
The JS configuration file is a dumping ground of modules and objects that aren't even consistent with themselves and now Nuxt 3 is out and within a matter of hours had 100 bugs reported on Github.
All I planned on was a basic markdown blog with some interactive Vue components for things like diagrams.
Little bit tired of Node based SSG's so I looked at the .NET options (my prefered stack). There used to be one called Wyam that rebranded to Statiq and now you need to actually buy a license for (typical .NET shenanigans).
I ruled out Hugo and Jekyll because their template languages are certainly less ideal. From what I saw it was common in both to have starting blocks in one file and the ending of the block in another. Basically one file would have <body> and then another random file would have </body> in it, what a mess!
So it's either back to Node based SSGs or write one in .NET (I once did start that, a bunch of markdown files would be processed with a markdown library and then passed to the AngleSharp library to generate the HTML).
https://astro.build/ is next on my list and seems promising so far.
Overall, despite there possibly being thousands of SSG's none of them actually seem to please a majority of developers. I think that's unique, because for problem xyz there's usually maybe a top 3 or 5 hyped up popular options but SSG's are a wild west.
You can start simple with MAKEFILEs, perl scripts, pandoc and other basic/builtin unix scripts etc... and go up to build a full SSG site in Node or Ruby / Python etc... with a million dependencies if you wish to.
*: every time you want to publish something you have to run a command , wait and push the output to a webhost configured properly, as opposed to installing something once and writing in a webform and your words appear on the site.
You can actually make your own themes with github pages
What I like about 11ty is that it comes with several template languages out of the box and you can pick whatever you want. You can also mix several languages.
Or you can run it in a bare-bones minimal version. I guess it just fits my mental model.
Furthermore, it doesn’t have constant updates. The API is quite stable (although 1.0 will be released soon, which requires some manual work). But I haven’t had to change a thing in a couple years.
Not affiliated, just happy with it.
Many Markdown utilities support front matter, which is designed to solve exactly this problem.
---
template: blog
other-key: other-value
---- because I've done it already and I haven't gained anything from it other than feeding my ego
- it makes me work on keeping it updated to "my custom needs" more than I work on the actual project that needs it
- it always gets outdated really fast, esp if it uses js a lot
- I've learned that it's generally a waste of time, time better spent on other areas of your business and existing tools are good enough
The other points I won't deny, though I sure liked feeding my ego.
Think you worded that incorrectly. It should be
> it always gets outdated really fast if it uses js a lot
I agree with that point. I wouldn't write a static site generator in JS for that reason.
That may be true, but that doesn’t say which language is better.
Jekyll sites tend to be lovingly configured but then rarely updated, I've found. I'm simply happy when I don't have to visit the abomination of medium.com frankly.
Fortunately, some of the extensible static site generators make this fairly easy to add as a feature. Here's an example for Eleventy (aka 11ty):
https://github.com/11ty/eleventy/discussions/1973#discussion...
I don't completely agree about Jekyll (it can be very light weight, requiring zero Ruby unless you need to extend things), but rolling your own solution for a personal home page is always nice.
It’s a little bit of a strange tool, but you can start by pointing it at an HTML file, and it will automatically build it, bundling and minifying any dependent resources and giving them a hashed filename for better caching.
If you want to do multiple pages, it’s simple; you can just link them. Parcel Bundler will pull the links and bundle them just like the first page, recursively. When it generates linked pages, it won’t make a hashed filename, so your directory structure will be left in-tact. If you want nice subdir pages, you can put index.html files into subdirectories, and voila. And as a bonus, any resources you depend on from two different pages will share the same bundled resource, as you would probably hope.
But HTML can get repetitive, so you probably want templating of some sort. For that, I tend to rely on Pug templates. They support blocks and template inheritance, so now you can do more advanced stuff without having to repeat yourself constantly.
Finally, if you wanted to use TypeScript or SCSS, you just, can. They will be compiled and bundled to JS and CSS transparently, exactly like you would probably hope.
And then maybe you want to do something more interesting than what it can do out of the box. Like for example, maybe you want to render Markdown files into a template (obviously relevant to this use case.) In that case, you can extend Parcel. My favorite shameless plug is a documentation site I have set up for a reverse engineering project which uses Parcel to render Markdown and Kaitai Struct files into templates which are interlinked with each-other. The resulting site is fully static and the build process is relatively quick. I don’t know of any cleaner way to do something like this!
The biggest downside of Parcel is that I did, in fact, need a custom plugin to render Markdown files directly into templates. Maybe there’s a simpler way or a plugin on NPM that serves this relatively common use case. But I just love having the page graph intertwined with the asset graph like this; it’s super powerful for building fully static websites. The only thing it feels weak for is if you want to generate pages from data somehow; Parcel is pretty tethered to pages being an asset, so as far as I know you can’t simply synthesize them. Still, despite this and other quirks, I really do enjoy the experience.
I've written my own static site generators before, too, and with all static site generators pretty much, I’ve felt unsatisfied with how they interacted with bundlers and compilers.
https://git.sr.ht/~evan-hoose/SSSSS
Is it garbage? Yes. But it's my garbage, and does what I need it to!
I feel like this is a design flaw: https://github.com/picocss/pico/issues/13