Comment by cxr | Hacker News Reader

Comment by cxr | Hacker News Reader

Parent Full thread

> it could have been phrased more nicely

That's an understatement. It's the kind of comment that should be in the grey. Comments that amount to pile-ons and smug dismissals—like the one above—are far more welcome here than they used to be.

> Spolsky's CityDesk. It was a Windows GUI that would FTP HTML and images to a static server!

Among the things that Microsoft first started pushing to GitHub around the time that they formed the .NET Foundation was Live Writer. <http://openlivewriter.com> Dave Winer's Frontier is open source, too, but has also rotted. Brent Simmons, the creator of NetNewsWire, set out within the last couple of years to recreate it as a modern Mac app, but dropped the project last summer.

> Although his blog also rotted due to CityDesk rotting! It's a hard problem.

I'm convinced that we've missed the obvious and have been doing it all wrong, due to predilection for certain systems and ways of working. If you have a personal homepage, you should have among the stuff hosted there a document that describes how your site is updated. This document should be less of a blog post than an SOP. But now make sure it's specified with enough rigor that you can feed the same document into a machine that can perform those steps automatically. The reason the "WordPress setup from 2004" works and "old-ass blogger.com site" work is because they're closer to this ideal than Jekyll or Hugo or the others are. The caveat is that the hosted services make it really hard to actually look over the procedures that the machine will run, and if they let you, then you'd find that they were spread out over multiple "documents".

Whatever the case, the thing not to do is to have this sort of thing live out-of-band elsewhere as a tool that requires its own configuration and setup (with both the tool and the configuration being subject to rot) accompanied (if at all) by a terse README. (Otherwise, if you're keeping that sort of thing elsewhere, then your homepage is not really a true digital home, is it? It's a Potemkin palace, and the place where those other things live is your real home.)

Yeah it's definitely a problem that you have this "elsewhere" thing that rots. Having some kind of online editor is one way to solve it, although it also tends to rot (who hosts that? Are you limited to PHP? Are you upgrading PHP? Did that company get sold?) [1]

I think a simpler way to solve it is with the now standard "infrastructure as code" pattern. For example, use Github Actions and similar platforms to build your blog.

Basically check in a spec for a container (Dockerfile) into VCS. It's done off of your machine, and it will "never" rot. You still have the cloud portability problem, and the "shell scripts and YAML are gross" problem. But those are solvable, especially for the relatively simple case of building a blog. I'm working on that:

https://news.ycombinator.com/item?id=25343716

[1] I think it's funny that VC's use non-startup platforms for their own blogs, because they know that blog platforms do not last.

https://blog.samaltman.com/

https://posthaven.com/

> now standard "infrastructure as code" [...] check in a spec for a container (Dockerfile) into VCS

This is an example of the "predilection for certain systems and ways of working" that I referred to.

> it also tends to rot (who hosts that? Are you limited to PHP? Are you upgrading PHP? Did that company get sold?)

That's not really what I meant. My reference to WordPress was probably more misleading than I intended.

I really mean it when I say a human-readable document that's more SOP than blog post—it's not a metaphor. Consider a static site generator for an existing site, and consider the steps it performs on the input to produce the output. Specify these steps as a single, written document—one that you can print out, even. If you're doing it right (in contrast to how we "have been doing it all wrong" up till now), then you should be able to ask yourself, "Is this document suitable for publishing a copy of it on my site itself (or printing it out and dropping at the bottom of a filing cabinet)?", and you should be able to answer "yes".

(This is where the reference to WordPress came from—it's not exactly a "single document", but it is a bunch of PHP files that you could conceivably stitch together and consider to be one and which you can take a peek at so long as you're self-hosting. But even then it's not something generally available as another piece of content on your site, because WordPress doesn't "reveal itself" as such, i.e. there's no associated HTTP resource by default that's associated with the underlying WordPress-in-PHP engine itself, so its likeness as an example is strained.)

That still doesn't solve hosting and portability, but it's a separate matter from the authoring tools (etc) experiencing rot. In practice, this can be covered by the inclusion of an annex to the above SOP (or a completely separate set of procedures) that describes how to get your generated files onto whatever host you're currently using. (In software terms, this section would describe an "adapter" and its use.) Static site pages are already pretty portable by their nature, and you don't really move around enough to worry about it, really, but if you do move, then just document the new process by speccing out a new adapter.

The takeaway is that if these processes are too elaborate to be documented in this way, then it won't work. But if that's the case, then they're already too complex, period—which rules out Docker and pretty much anything else that's "standard" in the devops world right now. I've prototyped something like this, and describing the whole thing that fits into a document weighing <500kB without even trying is pretty easy. My prototype is actually currently at 109kB, but that just covers the build SOP. Albeit, that is both man- and machine-readable (i.e. machine-executable), so it's detailed to a high degree of rigor. That's one long document that includes stuff like formally describing how the SHA-1 algorithm works, how Markdown gets processed, how templates and includes work, and so on. The adapter doesn't really exist, though; since it's a static site, it's just a note that says, essentially, "use rsync; here's the hostname and the exact command-line flags". In the document specifying how building the content from source works, I punt on specifying YAML, because a drastically simpler subset (i.e. colon-separated, key-value fields on separate lines) was able to accommodate 100% of how my frontmatter already looked.

I don't think relying on underspecced and inscrutable commercial systems like GitHub Actions is especially simple, resilient, or healthy. It seems to be subject to exactly the sort of rot mentioned before. Basically, what you said about VCs' blogs.

Hm I think we're coming at this from different angles... the whole "SOP" terminology confuses me, though I guess you mean "standard operating procedure".

I'm a programmer so when I hear "document on how to update something" I hear "untested set of manual steps". I don't understand why it isn't a shell script instead ...

----

I actually don't want to preserve an authoring experience forever. I expect that my authoring experience will change [1].

What I want to preserve is the data. Both the source data and the generated data. I have one git repository for each. And everything else is basically Unix tools that I can compile myself.

Ask me in 4 years about this stuff, or 10 years: http://www.oilshell.org/site.html

I bet it's all going to work, basically due to the "Lindy Effect". I can build everything from source even if people don't maintain it.

I actually wouldn't rely on Docker itself, I meant more a "spec for a container", but Dockerfiles now have multiple implementations like OCI.

The point of the shell layer is to NOT rely directly on Github Actions APIs, even while using the service. It's indeed a problem that all these cloud services have weird SDKs and command line tools.

However when you look UNDERNEATH these cloud services, you get a KERNEL and a SHELL. That is the "timeless API" I'm writing to. I already did that with continuous builds, and it would be similarly easy to port my blog toolchain to the MULTIPLE clouds this way, so I don't rely on one. (It wouldn't be easy for all problems, but it's easy for building static content.)

I realize this is all outside the norm and 99% of writers aren't going to use this toolchain. But it goes very well with the Oil shell project. Unix and document authoring are very closely related. Historically the first use of Unix was to create patent documents at Bell Labs.

I don't quite understand what you are getting at so I think you must have a much different use case. If you have an example then that would help. It seems like you might be talking more about a blog that lots of people are editing?

----

[1] Actually it JUST did after 4 years and at least a hundred posts. I used vim for 4 years, and now I switched to the online editor StackEdit because it has realtime preview. I wrote the last 2 blog posts in StackEdit and it worked great.

However the data remained exactly the same. Unix is data-centric, not app-centric. Apps are the things that rot.

> when I hear "document on how to update something" I hear "untested set of manual steps"

Why "untested" and why "manual"? I mentioned "detail" and "rigor" more than once and that it's possible to take it and feed it into a machine that will "perform those steps automatically".

I don't think kernel+shell are all that timeless of an interface to write to. They only barely pass for being portable in the here and now. (Oilshell's existence as a project is a testament to that. Its breadth and the depth of the pitfalls it has to avoid is another one.) Most of the time portability is an illusion that comes from most people around a given project being part of the same technological subculture, so they happen to use a common implementation. When you stray outside of that is when you run into breakage that no one is otherwise aware of. See also <https://pointersgonewild.com/2019/11/02/they-might-never-tel...>.

> I don't quite understand what you are getting at so I think you must have a much different use case [...] It seems like you might be talking more about a blog that lots of people are editing?

Nope, the use cases we're talking about are the same. I don't know where the lots of people editing parts comes in.

What I'm talking about is conceptually very simple (maybe too simple, leading to an assumption that there's a lot more to it). It's a document that describes in detail, just like any other tech spec, precisely how to process a given input and turn it into a directory of static resources that you are used to dealing with, i.e., the output of a static site generator. The only difference between it and other specs is that in this document you specify things with enough rigor that you can effectively "execute" the specification itself. I've heard Rob Pike say in a few talks that some of the Golang backends come from a tool Ken wrote where they can feed in PDF manuals from microprocessor vendors and get a working Go assembler out of them, but I'm not intimately familiar with that tool. See also <http://www.moserware.com/2008/04/towards-moores-law-software...> where some RFC-style diagrams constitute a part of the implementation of a working TCP/IP stack.

> If you have an example then that would help.

I might do a short (? I'd hope) screencast next weekend if you're interested in providing some feedback. Asynchronously and pre-recorded, not live. My first assumption is that when you see what this is all about, then you're going to be pissed off. Based on your description of your current setup, though, that impression may not be accurate.

> Apps are the things that rot.

This is where my comments about "predilection for certain systems and ways of working" and how "we've missed the obvious and have been doing it all wrong" come from. In trying to solve these problems, there's way too much emphasis on apps, CI configurations, or whatever mindtrap is ensnaring people at the moment. The "obvious" approach I refer to is first adopting a mindset where it's an imperative to include a functional specification as described. I.e., add it to the set of things that you intend to publish on your site. Once you do this, it's hard not to notice that there's now an overlap between your data and the "app" you're using to publish it. So the second thing to do is to eliminate the part that's most susceptible to rot.

The only difference between it and other specs is that in this document you specify things with enough rigor that you can effectively "execute" the specification itself

Hm if you have a concrete demo then sure I would watch it and give some feedback. (e-mail in profile) It sounds like literate programming for a shell script to me :)

The kernel + shell is the best we have, since there are multiple implementations going back years. For example FreeBSD and Linux are totally separate projects and codebases, and implement a very very large common subset (POSIX, which has absolutely everything you need for text processing and document authoring).

For example, I just tried out NearlyFreeSpeech, a very inexpensive pay-as-you-go hosting service, and I found it runs FreeBSD. If I really wanted to make my blog portable and timeless, I would port it to NearlyFreeSpeech, which I estimate would probably be one day of work (really, I only use the Python stdlib, etc.). But I'm comfortable enough that Linux is going to survive until in some form or another until I'm dead ... or if it doesn't, there will be some migration path for the bazillions of other users.

The point is multiple implementations: Unix has multiple implementations, CommonMark has multiple implementations, and so does HTML/HTTP. Those things aren't going away.

The reason that Spolsky's blog rotted is because he built it in a custom tool (which he wrote), which was built on VB6, which was built by a company which changed its strategy ... etc. The stack was too high, and too proprietary.

He switched to WordPress, but I'd argue that stack is too high too, even though it's more open source.

So I'm happy with my solution but sure I would look at different ones. But my feedback might be that it should be "tested" in the real world. Just like the HTML+CSS solution is NOT tested because the OP didn't actually write many blog posts with this toolchain!

Reply on news.ycombinator.com