Hugo and IPFS: how this blog works (and scales to serve 5k% spikes instantly)
withblue.ink
withblue.ink
* The "5k% spikes" being discussed here are on a tiny base load. His peak is 6K page views in a day. That's 4 pages a MINUTE. A raspberry pi could handle that. Anything can handle that. There's no load here.
* It's not clear what IPFS is getting him, but it's certainly not performance. Any performance improvements are coming from using the Cloudflare caching proxy, which is basically just using Cloudflare as a CDN. You can get that with a normal website behind Cloudflare with a lot less hassle.
It's fine to check out the implementation details for interest's sake, but let's just be clear that it all could have been done with a $5 VPS.
* That's 4 pages a MINUTE.
That's assuming that those 6k views were distributed evenly throughout the day. The article mentions the Hacker News / Reddit effect, so I'm assuming there was, indeed, a sizable increase in traffic for a short duration that a raspberry pi wouldn't be able to handle.
I do wish it showed precisely what happened on that 6k view day, however
Sure a raspberry pie could replace this in heartbeat but it’s not about current throughput being good, it’s about making it work as intended, ensuring the design is theoretically scalable, and marching toward a fully distributed network.
I realize the author is talking mostly about performance but the think that is exciting about ipfs is it’s not some monolithic project trying to take over the world but rather a modular set of standards (IPLD, MULTICODE, libp2p) that generally do a very good job of integrating with legacy and have reasonable future proofed design.
Ipfs is just one application bring all these thing together is in a particular confirmation but many more interoperable variations are likely soon to come.
If CloudFlare is running IPFS naked, then IPFS seems to be scaling.
If CloudFlare is caching IPFS, then it is CloudFlare saving their butts.
Can we confirm which it is?
I'm trying to chat with CloudFlare about my own P2P protocol, which I've seen handle HackerNoon's 15M monthly users (I saw about 10K concurrent users per second at peak load), to run GUN ( https://github.com/amark/gun ).
Because I do think it is important to test all these protocols out at bigger/larger scales, that certainly makes it easier to then debug and then fix problems (assuming CF isn't caching).
> If CloudFlare is caching IPFS, then it is CloudFlare saving their butts.
They are caching IPFS for people who don't support IPFS. They are not caching IPFS for people who do support IPFS, and instead passing the user through to the IPFS servers.
Which means Cloudflare is indeed saving their butts, because as soon as IPFS has widespread support, this falls flat on its face. Same as traditional hosting would without a CDN. Their IPFS setup may be able to distribute the load enough to avoid an internet hug crippling it (that's part of IPFS's point), but this setup in no way demonstrates that.
If you're just hosting a static site, it can be done for free. You can use Github pages (or any similar free static site host) + Cloudflare for free. Getting Slackdotted, HNed, or reddit hugged will all survive just fine, for free.
So to recap:
* Use a static site, maybe with a static site generator like Hugo
* Use a static site host, which could be github pages (free), s3 (cheap), or a $5 VPS anywhere running apache or nginx
* Use a free CDN, likely Cloudflare, to handle the brunt of the bandwidth.
The IPFS is interesting but way more work currently than any of this is worth. And if people are using IPFS compatible browsers, the load is being transferred to the IPFS servers anyway, which is worse than just using a regular hosting method and Cloudflare CDN. If IPFS actually had decent adoption, the method described would fall to a crippling internet hug.
The goal of IPFS is to distribute the load (for people with IPFS browsers), and it's possible the article's loadout would do that, but given he's seeing 160GB of transfer from his servers, it certainly doesn't seem to be doing that very effectively currently. And this setup does not in any way demonstrate IPFS is ready to do that, because Cloudflare is saving his servers' butts.
And yes, it's a chicken/egg scenario -- how do you prove that IPFS adoption would negate the need for CDNs, without widespread IPFS adoption? But currently this isn't some amazing Slashdot proof solution. They needed to setup 3 servers and a whole bunch of software when a single server + free CDN would've accomplished the same thing, faster and cheaper. It's presented as though CDN+IPFS is why this works so great, when really it's just CDN and IPFS just makes it more complicated.
If the goal is to learn, try out new tech, break dependence on CDNs, then this is great, but a single note in the article that this is a learning opportunity and it can be done easier without IPFS would've saved the author a lot of back and forth in the comments here.
This very naturally answers the repeated questions from people about the why of IPFS.
Yes you COULD serve this via 1 or 5 of the other 20-30 incredibly individually complex stacks with CDN functionality as well. But why would you if there was a common CDN protocol?
What would the upsides be if your ISP could offer you an opt in localish IPFS instance?
IPFS is a functional expression of the long term academic research into creating an internet oriented around content blocks vs server socket oriented as it is today -
https://en.wikipedia.org/wiki/Content_centric_networking
We, of course, need both so its not an either or.
We are backing our way into this world via javascript SRI -
https://en.wikipedia.org/wiki/Subresource_Integrity
Look at that, we need blocks of code that we know is what we want but we want flexibility in where it comes from.
If you host it yourself you have easy answers for that but if you’re relying on generous strangers you are going to hit limits: how many people are going to mirror large amounts of content, deal with DMCA takedowns and other legal requirements, etc. before giving up? Performance could in theory be competitive but there’s no guarantee and it’s notoriously hard to deal with unreliable nodes — which again can be dealt with if you have skilled support staff available. You’ll get some volunteers for the right cause but that only gets you so far – that’s why there’s a a couple decades of these projects dreaming big and failing once they get popular.
Yeah, thanks, I'm ancient. Before I fart myself to sleep reminiscing the good old days, we got ~10k visits in a hour, on a ASP.NET website (admittedly with good caching) but on the absolute dirt-cheapest of shared hosting. It didn't skip a beat. I've done similar with PHP and Django. As long as you can avoid hitting the database every request, you can scale to the moon with very little work.
Hugo is static. Your toaster should be able to support 5k users.
I'm not sure if that's 160gb/vm or 160gb across all vms - but either way that's 10-30 usd/month in bandwidth?
Pretty steep compared to a (single) 5usd/month vm or free (free tier of cdn provider).
Interesing write-up, though. Nice to see cf support ipfs.
Instead of running a single static server, they now have a cluster of three servers plus Cloudflare to serve up a few static files to a fairly small number of people. And even then they have problems with caching and IPFS-related bandwidth. And almost nobody actually accesses the site via IPFS, making the whole thing rather pointless.
This article seems to be mostly about the benefits of IPFS, but I can only see the benefits of Cloudflare caching. What am I missing?
It appears to me that Cloudflare accepts IPFS as (another) source for it's CDN operations & any benefits of flat CPU utilization are likely from the CDN and not IPFS.
In fact, 3 VMs in 3 regions is complete overkill. If the traffic numbers are to be believed, one could simply post their Hugo site to GitHub Pages or Netlify with zero extra steps or dollars spent. No IPFS needed
That works great for static pages. Hugo and Jekyll are basically precompiling static pages.
For a more dynamic site? I know some people who do special handling for bot and spider traffic. The bots get not exactly static content but much less dynamic content. I could see rerouting all traffic, especially for everyone without session cookies, to that version during a big spike.
Those solutions behave a little bit like the eventual consistency you see on very large websites, where values are approximated or cached with a very short TTL.
As others have commented, the simplest way to get that on a small site is to pony up money for a CDN. Maybe not the cheapest, but certainly the simplest.
I agree that the benefit of this implementation (3x IPFS + Cloudflare) over the dedicated VM/VPS might not be obvious for the amount of traffic/visits this specific blog is having, but it's a good alternative to know :-)
On a side note... Wouldn't it be great if OpenStreetMap/Google maps/your_favorite_map_provider was hosted on or via IPFS and there was an easy interface between the IPFS network and the www? This way the distribution of the tiles would be peerless, the CDN would be huge and each user would have locally the tiles most frequently used and serve them at the same time. No more dependency on big companies/providers, immune to DDoS/blocking/restrictions and free for all ;-)
What I'm missing is where a collection of static files couldn't just be served up from an S3 bucket and Cloudfront or Cloudflare on the front end -- you arguably have the same caching performance if not better since Amazon and Cloudflare have real SLAs for getting your bits from your bucket to a user's browser.
IPFS seems like un-needed complexity when there are a huge amount of options available. If you personally don't like Amazon, you can use Github pages, or DigitalOcean, or Netlify, or Fastly, or the list goes on and on.
Does anyone have a use for IPFS that isn't already covered by existing hosting+cdn options?
See https://developers.cloudflare.com/distributed-web/ipfs-gatew...
Essentially it's a gimmicky system which is propped up by using Cloudflare, which ultimately fails once it actually gets user adoption, in a way the tradition setup (CDN + basic or even free hosting) would not. This only works because most users don't have IPFS support currently, and if they did the servers would grind to a halt just as they would with no CDN at all.
The addressing mechanism the sun out into IPLD is quite clever and IPFS is nearing the cusp of having a noticeable impact on the web when you consider your browser is potentially a node running in the background.
I have been struggling to get adoption with students working on web sites at the university I work at but when we get it up and working it is so great for 90% of the things we do.