Hacker News Reader

I think it's less a technological problem and more just that everyone who used to care about that site or its content no longer does. Or the company behind it has gone out of business -- who is going to maintain a website for a defunct organization, and why would anyone want to?

There's no obligation for a person to maintain anything longer than he wants to do it. Putting a blog online is not a lifetime committment. Interests change, or you simply realize nobody much cares about your online musings, and you move on to other interests.

But what can you do when you do care, to make your website as durable as a printed book?

You could literally print it in a book. With links suffixed by a page number [915].

Point archive.org at it (and make a donation).

Depends on if you want it to survive without you maintaining it. If so, something like a bog-standard Wordpress blog hosted by them might work until they decide they don't want to bother anymore.

Otherwise, some setup using S3-as-a-website or GitHub pages may work, but those also depend on the company maintaining that service.

If your entire website can be dropped into a ZIP file and served anywhere, you have a greater chance in it surviving, especially if the internet archive got a copy at some point.

But if you die and nobody cares about the content, eventually it will disappear.

> If your entire website can be dropped into a ZIP file and served anywhere, you have a greater chance in it surviving, especially if the internet archive got a copy at some point.

Well, your entire website can be a zip: https://redbean.dev/

Keep it as simple as possible. Static HTML that can be dropped on any web server.

HTML/HTTP may some day be obsolete, but will likely be around longer than anything built on complicated javascript frameworks or tightly tied to current web browser technologies.

One step in the right direction is to keep the representation of information simple, or to transform it to something simple for long-term archiving.

Frameworks and Browsers change. External services change. But, unless its stored with no backup and the medium fails, an ASCII textfile composed 50 years ago is still readable today, and will likely still be readable in 100 years. And thanks to UTF-8 being pretty much ubiquitous these days, we don't even have to limit ourselves to ASCII any more.

I am not saying to make every page a textfile. I am merely saying that one step to preserve digital information for generations yet to come, is to represent it in simple and robust formats. And if simple is not an option (eg. media will probably need to be use a compression format), then well documented and open formats should be used.

When a company goes out of business you will typically also try to sell of all its assets, and a good domain name might be such an asset.

It’s not uncommon to hear of rack servers with several years of continuous uptime. I wouldn’t be surprised if you could keep a website online for a decade without touching anything by using an LTS distro, enabling unattended upgrades, and running something like nginx.

> system font stack that matches the default font to the operating system of your visitor

If the list just matches the default font of the OS, is there a difference between using such a stack vs. not specifying a font? Wouldn't the browser naturally use the OS's default if font-family is unset?

> I've also been working on a background process that automatically captures full-resolution screenshots of every website you visit, creating your own searchable personal web archive:

How are screenshots searchable? They aren't plain text. You can't grep them.

The tool captures screenshots in addition to text.

That seems awfully simillar to archive.org wayback machine. I do like to see all these archival projects though, they are certainly worthwhile.

irchiver captures text on the page, and separately OCRs the screenshots (specifically, the screenshot from your viewport). So you can search just what was shown on the page, or what was in the page. Both techniques have pros and cons.

While archive.org is fantastic, it can only capture pages that are both 1) publicly accessible (i.e. no social media content) that it happens to crawl, and 2) static content (you're out of luck if the content you want is loaded dynamically, or changes depending on user input).

IIRC archive.org does save the JS and things it downloads, so you can replay them when you visit the archived site later.

I guess the difference in this case is that the JS on web archive relies on future browsers being backwards compatible, whereas irchiver relies on much less to stay timeless which is good. Although I don't think JavaScript will ever get a major update (as in breaking comparability) I believe relying on that is not a perfect way to archive web content. This kind of backwards compatibility breakage is something we have seen before with the deprecation of Adobe Flash and it could theoretically happen elsewhere on the web stack.

Agreed, I wish they would archive the DOM too like archive.is does instead of just the requests.

Why not though. On MacOS you can select text in bitmaps nowadays. So the tech is there to make a grep for pictures.

Interested personally in the font stack point. The system font stacks listed focus on font styles more than individual typefaces. If I want say, Bookman, is there any harm to long-term use to use a font stack tailored to that font?

There isn't, no, and there's no particular harm in self-hosted web fonts when it comes to long-term stability. Not only is it relatively unlikely that future browsers will deprecate web fonts but not anything else in current CSS (in which case there's a bigger issue than your fonts), it's even less likely that they'll do so in an entirely incompatible fashion. Assuming you've designed your font stack with decent fallbacks -- which you should -- then it's just like any other font stack, e.g., you may not be presenting in the typeface that you really want to, but nothing's going to break.

("Web fonts are okay, actually" is evidently the windmill I will keep tilting at on Hacker News.)

irchiver seems incredible. I hope there's a comparable product for other OSes one day.

I’ve been working on a very similar thing which runs on Windows, Mac, and Linux: https://apse.io

One of my ideas is called Wantsfiles at the root of domains. The Wantsfiles contains information about what you want and the price you're willing to pay. An order matching system matches your Wantsfiles.txt to Sellfiles.txt and transacts on your behalf.

So I could host link to my tgz on my Wantsfile and offer $5 to host the tarball a month. There we go a perpetually hosted website.

You could call it autohosting.

disk failure and os upgrades are a non-issue with the right technology stack.... but then you have other problems (more complicated setup)

Hm interesting, I read over your 7 guidelines [1] and I would say I agree 50%.

So the most lightweight way to keep something up and running is to make it trivial to port to many hosting configurations by simplifying the toolchain needed to rehost it -- I agree with this, although I would use the words "standards" and multiple implementations. The linked article doesn't appear to emphasize this.

1. Return to vanilla HTML/CSS

Again I feel the relevant issue is "standards", multiple implementations, and the fallback option of "old code" like GNU coreutils (i.e. taking advantage of the Lindy Effect). Not just just HTML/CSS (which certainly meet all of those criteria.)

I thought about this when designing https://www.oilshell.org/ and the toolchain

- The site started in Markdown but is now standardized on CommonMark [1], which has multiple implementations. So I don't see any reason to stick to HTML/CSS.

- The tools are written in Python and shell [2]. Both languages have multiple implementations. (Ironically, Oil is a second implementation of bash! My bash scripts run under both bash and Oil.)

- Python and shell rely on Unix, which is standardized (e.g. Linux and FreeBSD have a large common subset that can build my site).

This is perhaps unconventional, but I avoided using languages that I don't know how to bootstrap like node.js (has a complex JIT and build process), Go (doesn't use libc) or Rust (complex bootstrap).

On the other hand, C has multiple implementations, and Python and shell are written in C, and easy to compile. I'd say Lua also falls in this category, but node.js doesn't.

I feel like this is at least 60% of the way to making a website that lasts 30 years. The other 40% is the domain name and hosting.

This is a pretty a big site now, you can tar a lot of it up and browse it locally (well it's true you may have to rewrite some self links).

Of course, this is my solution as a technical user. If you're trying to solve this problem for the general public, then that's much harder.

[1] https://www.oilshell.org/blog/2018/02/14.html

[2] https://www.oilshell.org/site.html

2. Don't minimize that HTML

Mildly agree if only because it's useful to be able to read HTML to rewrite self links and so forth. Tools don't need it, but it's nice for humans.

3. Prefer one page over several

If you're allowed to use Python and shell to automate stuff, then this isn't an issue. I suppose another risk is that people besides me might not understand how to maintain my code. But I don't think it needs to be maintained -- I think it will run forever on the basis of Unix, shell, and Python. Those technologies have "crossed the chasm", where again I think the jury is still out on others.

4. End all forms of hotlinking

Yes, my site hosts all its own JS and CSS.

5. Stick with native fonts

Yes, custom fonts have a higher chance of being unreadable in the future. I just use the browser's font preference to avoid this issue. It's more future proof and in keeping with the spirit of the web (semantic, not pixel perfect).

6. Obsessively compress your images

Agree

7. Eliminate the broken URL risk

I think too few people are using commodity shared hosting ... I've been meaning to write some blog posts about that. I use Dreamhost but NearlyFreeSpeech is basically the same idea.

It's a Unix box and a web server that somebody else maintains. I absolutely don't care about CPUs, disk drives, even ipv4 vs. ipv6, and I've never had to.

The key point is writing to an interface and not an implementation. Commodity shared hosting is a de facto standard. The main difference between a tarball of HTML and a shared hosting site is that say "index.html" is always respected as /, and a few other minor things.

So I expect Heroku and similar platforms to come and go, but the de-facto standard of a shared hosting interface will stay forever. It's basically any Unix + any static web server, e.g. I think OpenBSD has a pretty good httpd that's like Apache/Nginx for all the relevant purposes.

Github pages also qualifies.

So I guess this is adding sort of a programmer's slant to it. To be honest it took me a long time to be fluent enough in Python and shell to make a decent website :) Markdown/CommonMark definitely helps too. I had of course made HTML before this, but it was the odd page here and there. Making a whole site does require some automation, and I agree that lots of common/popular tools and hosting platforms will rot very quickly. (and like you I've seen that happen multiple times in my life!)

And I think what your guidelines might be missing is a guideline on how to make "progress". For example CommonMark being standardized is progress that has happened in the last 10 years. You don't want to be tied to the old forever. At some point you have to introduce new technologies, and the way to do that is once there's wide agreement on them and multiple implementations. (just like there's wide agreement on HTML)

I think there is / can be progress in dynamic sites too, so you don't have to stick to static!

I’d quibble over at least CommonMark and Python.

CommonMark is not good for flawless longevity: it’s not as bad as non-CommonMark Markdown, but it’s still not fully settled and implementations vary in what HTML they will allow through and almost no one actually stops at CommonMark but implements other extensions which will normally break the meaning of at least some legitimate content.

Python is risky, even if you don’t use anything outside the standard library: consider what happened with Python 2.

I use the CommonMark reference implementation, which I assume stops at CommonMark :) I just download their tarball, and it's pure C, tiny, and very easy to use e.g. from shell or Python: https://github.com/oilshell/oil/blob/master/soil/deps-tar.sh...

Nothing happened to Python 2! I still use it, and it's trivial to build and run anywhere, even if distros drop it. Just download the tarball and run ./configure and make, which I've done hundreds of times for various purposes.

(The same is not true for node.js, Go, or Rust, which as mentioned is one reason I don't use them for my site. If people stopped making binaries for them, I'd be lost.)

Ironically, if you want something that will last 30 years without maintenance and upgrades, Python 2 is better than Python 3. (I use both; Python 3 is very good for many/most things.) There are memes about "security" mostly based on misunderstandings IMO.

As you point out, the real problem is PIP packages, but that's why my site doesn't depend on any. Or if it does, I again make sure I can download a tarball, and not rely on transitive dependencies and flaky solvers.

The Python 2 stdlib is definitely good enough for making a website. It's not good enough for bigger apps, but it's great for simple website automation.

----

The higher level point is that you can always "plan for the future" at the expense of the present. IMO avoiding things like CommonMark and Python will just make your site worse right now, which defeats the purpose of preserving it in the future. So there has to be a balance against extreme conservatism, and there has to be a way of making progress (new standards) while not succumbing to bad fashions. Likewise I think Oil looks like a "retro" project to some, but it does a lot of new things, and that is the whole point.

> (The same is not true for node.js, Go, or Rust, which as mentioned is one reason I don't use them for my site. If people stopped making binaries for them, I'd be lost.)

Thankfully, this no longer appears to be the case for Rust, thanks to mrustc [0], a compiler that can build a working Rust 1.54.0 toolchain (released 2021-06-29) from source. It requires only a C++14-compatible compiler and some other common tools; I've just verified that its build script works with no problems on my Ubuntu machine. To be safe, you'd want to specify exact dependency versions for everything (or better yet, vendor them locally), since the big crates all have different policies for when they break backward compatibility with older compiler versions.

[0] https://github.com/thepowersgang/mrustc

Yes definitely, it's great that progress has been made on this front. There has to be a middle ground between "web sites rotting after 5 years" and "POSIX forever", etc. The former is the status quo, and the latter doesn't produce very much.

I looked back to some early wiki pages of research for Oil, and it is surprising how many links are broken, and a few of them were on niche/valuable subjects