Hacker News Reader

Comment by zxcvbn4038 | Hacker News Reader

I was hoping for some discussion of how this was done, it looks like there is some sort of software for hosting a specially encoded version of Wikipedia, and the overall goal is to provide access to a Wikipedia in areas without good internet access, but details are sparse and spread out. I’d like to know more!

View on HN

According to links I found on the page, it looks like the software being used is provided by Kiwix [1], and the file format used to archive Wikipedia (and other sites Kiwix archives) is ZIM [2].

I agree, it's an interesting project. I would like to see more specifically about how many users they can serve at once from a Pi Zero. Unless they just stuck Cloudflare in front of it?

[1] https://www.kiwix.org/en/

[2] https://openzim.org

It's down at the moment with an error about capacity, so presumably not behind a CDN.

Hi, Yes this was a very simple project. Basically I had a raspberry pi zero w , installed apache and setup a virtual host that forwards all request to a another back end. Basically the Kiwix software that serves the wikipedia Zim file runs on port 8080. The raspberry pi has a 32 gig card and not enough space to host the 82 gigs of the wikipedia zim file. I had a spare external drive that I formatted in ext4. Originally the drive was formatted as extended FAT32 but the raspberry pi showed a lot of high CPU when accessing the Fat32 partition. I formatted it to ext4 and mounted the volume and downloaded the wikipedia zim file. Then I created an index file for Kiwix server and, started the Kiwix server and restarted apache had wikipedia running on a raspberry pi zero w. It did crash, I was thinking of adding cloud front as I have a free AWS account but I haven't got to it yet :)

Very cool! Depending on the volume you are expecting but I would consider Cloudflare over Cloudfront. I don't think you are doing anything incompatible with Cloudflare's free tier and absolutely worst case is they drop you if you are attracting enough traffic to impact their other customers. With Cloudfront you might end up with an unexpected bill if you breach the limits of the free tier.

Reply on news.ycombinator.com