Hacker News Reader

Comment by bombcar | Hacker News Reader

Kiwix is great - I have a collection of various things from their library https://library.kiwix.org/?lang=eng downloaded for when I'm on a plane or the internet is otherwise unavailable.

That and the TeXlive PDF manuals can get me through anything.

View on HN

I second Kiwix. I found out about it not too long ago on the topic of portable Wikipedia readers. It really stands out as the best software part of such a solution.

I third Kiwix. Immensely useful when I was deployed without internet.

That sounds interesting, what was the context?

My guess is military.

Library or school in a remote village. There are computers (usually old computers), there might even be a LAN of some sort, but no internet (or very slow internet).

In those cases having local access to Wikipedia (and not necessarily just en; Kiwix has archives for all the languages) can be a great learning resource and reference.

Do you know 23B1? If not, probably you posted a reply to the wrong comment.

Hi sorry for the delayed reply, yes I was military and in the early days of Iraq/Afghanistan we didn't have much access to the internet so I brought it with me.

I used kiwix Wikipedia for a Polish friend in the Uk who couldn’t afford reliable internet access and was using public library computers. I found the English edition with images was too large for him, but the Polish edition was fine. Ideally I’d have liked a simple update system (Git like?) which he could have run at the library occasionally.

Yep, you can download StackOverflow for offline use too

Does it actually work? I installed the app and tried to download wikipedia two or three times, each time it just failed. Eventually I gave up.

I downloaded the files directly from the library if I recall correctly.

Personally I downloaded the larger files (>2GB) from a torrent file using my torrent manager. Much more reliable than over HTTP. You have checksums, it's resumeable, etc.

Given the sheer size of Wikipedia dumps - it's around 90 Gb with images! - I would recommend downloading them outside of the app.

https://download.kiwix.org/zim/

Yes it does.

I've downloaded the entirety of wikivoyage for example.

I wish one could create new articles in Kiwix’s zim files. Right now, Kiwix is basically a Wikipedia reader. Editing features would be very nice for local wikis to develop, and later on — maybe — to have such local article editions merged into the main Wikipedia, perhaps similar to how git works.

The .zim file format is heavily optimized for compactness and ease of serving. For starters, it doesn't even store the original MediaWiki markup, but rather pre-rendered basic HTML. Images only have the thumbnail version (the one that's shown inline when reading the article), there's no full-size to zoom in. And, of course, no edit history. Multiple articles then get bundled into clusters of ~1Mb each, and each cluster compressed using ZSTD.

https://wiki.openzim.org/wiki/ZIM_file_format

This all lets you squeeze English Wikipedia into 90 Gb. But it also makes it much more difficult to edit in-place, and, of course, no MediaWiki means that it cannot possibly work like git pull requests.

I totally understand the reason why it is made for read-only consumption. However, we live in a moment where storage is significant cheaper, and so is processing. There could have a compromise, though I do not see any indication of such. SQLite could very well be used here.

Reply on news.ycombinator.com