Jekyll blog on Amazon S3 and CloudFront | Hacker News Reader

Jekyll blog on Amazon S3 and CloudFront

vvv.tobiassjosten.net

84 pointstobiassjosten51 comments

For those of you who would prefer to just save your blog (or other static website) in Dropbox instead of uploading to S3, my startup just left beta today: http://www.site44.com

(Sorry for the shameless self-promotion; it's an exciting day for us!)

Huh. Part of me says "get off my lawn" with your web publishing from your dropbox blah blah ...

But that's pretty cool ...

there is also http://scriptogr.am/ for blog on dropbox

I also use Jekyll for my own blog [1]. However, because I really love naked domains (easier to dictate over the phone, shorter), I cannot do CNAMEs so I am hosting my blog on a free Heroku instance.

My initial Heroku config evolved too. I am now serving my pages through an Nginx server, installed with a custom buildpack [2] that I forked and modified a little to suit my directory structure (I could also make it compile the site on deploy but was too lazy for that).

I am also putting CloudFlare in front of it. Basically an article can end up on Hacker News, Reddit, Slashdot and the traffic combined will probably not make it even blink, while the hosting costs are zero.

Whenever I see people bitch about Wordpress woes, setups of clusters, security issues, or about caching/editing/versioning/archiving plugins or about costly add-ons and restrictions, it makes me chuckle a little - but then again, this setup is not for mere mortals or optimal for large content websites - for technical people however is a freaking nirvana.

[1] http://bionicspirit.com

[2] https://github.com/alexandru/heroku-buildpack-nginx

With AppFog [1], you can get WordPress with the tantan-s3 plugin pre-installed for uploads so you can share nothing (but the database) and scale with a click. You also get Varnish caching with no additional work.

Full disclosure: I am an engineer at AppFog.

[1] https://appfog.com/

I'm sure your service is useful, however allowed-plugins and other restrictions is something I like to avoid, not to mention that 1-click scaling is 1-click too many.

Wordpress is only useful if you want to publish in multiple categories with articles created by other (non-technical) people, need designs made by other designers - basically for a publishing factory or for people that don't want the hassle of learning to use Git, Jekyll and a good text editor.

However Jekyll/Git/Heroku is simplicity in its purest form, especially for developers and with simplicity comes a lot of power ... like I've had 2 blogs before this one and I lost the content on both out of ignorance, by stopping payments to my hosting provider, then later regretted it.

My content right now is on my laptop, mirrored by both GitHub and Heroku and you get for free the history of all changes, while the articles themselves get edited in the most powerful text-editor ever created (Emacs). Scaling comes for free. Costs are zero. The domain is mine. I can publish however many ads I want (coughwordpress.comcought). It wouldn't lock me out if I were under 18 (coughblogger.comcough).

To me that's just awesome. On the other hand it's just a stupid blog. Which it should be. Which is why less is more.

I host my current Octopress/Jekyll blog with Appfog and gave it a thorough thrashing with ab [apache bench] and blitz.io, performance has been great. It's easy enough to simply run an `af update` instead of `git push heroku`, in that sense.

Where is your fork of tantan-s3? If I recall it hasn't been updated by the author in over 4 years. I have a fork on GitHub here but it's not great:

https://github.com/cmwelsh/wordpress-s3

There is a certificate error on that redirected naked domain for https://appfog.com

Thanks for pointing that out. We'll have it repaired right away.

Still broken, appfog.com still gives me a certificate for: redirector.zerigo.net

I looked at Jekyll for my recent blog rewrite[1] but I ended up writing my own[2] and hosting it on Heroku, for many of the same reason. Even without CloudFlare it's been able to handle HN homepage and subreddit hits with no perceptible performance issues.

[1]: http://bugsplat.info/2011-12-14-yet-another-not-very-static-... [2]: https://github.com/peterkeen/bugsplat.rb

Amen on the Nerd Nirvana! :)

Personally I prefer [1] using a subdomain over the bare domain. If your registrar doesn't support www redirection you could always use WWWizer [2].

[1] http://vvv.tobiassjosten.net/internet/using-www-for-your-dom... [2] http://wwwizer.com/

WWWizer is pretty cool, but I'd rather roll my own on Heroku. I have a Heroku "app" that Heroku thinks is PHP, but it just contains a .htaccess file with some RewriteRule directives to 301 http://example.com to http://www.example.com. I then do the real hosting elsewhere. :)

Heroku is great for precisely this reason. Dynos are basically customizable http legos.

That's awesome--I didn't know CloudFlare has a free layer. I'd thought about using S3 or Heroku for hosting static webpages, but I didn't feel like trusting a single Heroku dyno to a traffic surge and lost my interest in using S3 once I found out Google Cloud Storage now lets you host naked domains!

One very nice side-effect of hosting on GitHub is that it integrates very nicely with your repository GitHub pages. URLs for your repos are simply http://domain/reponame. For example (apologies for the self-promotion):

* my website, hosted on GitHub: http://crystae.net/

* toy cellular automata project, hosted on GitHub: http://crystae.net/automatra

Author's argument about Github Pages not allowing custom Jekyll plugins is true, but you can always build the site locally with whatever plugins you like and deploy the built copy (see Octopress[1] for a popular example of this).

1: http://octopress.org/

That's definitely an option and to be fair I didn't use any Jekyll plugins up until recently. But my main gripe is still the lacking redirection. I can't lose that ability for this blog and all its accumulated URL fixes over the years.

For new blogs I could definitely consider GitHub Pages. In fact I use it for one I started just a month ago.

i think the 1% may not very accurate? my blog is also rated as being in the top 1% and it doesn't use cloudfront or s3 or anything cool like that. it's just static pages at webfaction (who are nice guys, but still, it's a shared host).

not trying to diss the post, which was very interesting, just suspect of the pingdom stats. unless using static pages alone is sufficient to get you into the fastest 1% these days? i guess it's possible. maybe they're hosted nearby... [update: the traceroutes differ in where they enter the usa]

Nearby Pingdom? It's always possible that you're in the same data center. In theory, CloudFront pages are always hosted nearby.

If wordpress.com can exist offering to host people's blogs, I think someone could create a site that does these techniques in the backend and provides a nice content management front end.

You could probably just slap CloudFront on the existing WP site. Might have side-effects but it would surely bump the performance up.

There actually is a way to redirect requests on GitHub Pages. It's a bit hacky, but if you want to stick with GitHub's hosting, it might be worth doing.

You can manually create a new page "index.html" in the corresponding subdirectory for your redirect. If you want /post/oops-i-made-a-tpyo to redirect to your updated oops-i-made-a-typo link, you would create /post/oops-i-made-a-tpyo/index.html and make an HTML redirect:

  <meta http-equiv="refresh" content="0;url=/post/oops-i-made-a-typo/" />

You can see this at work on my blog [1] and the code for the redirects I made [2] when migrating from Tumblr.

[1] http://frankjwu.com/

[2] https://github.com/frankjwu/frankjwu.github.com/tree/master/...

I switched my Hakyll site from NFSN to Amazon S3 back in June. My basic observations here would be:

1. I don't see why you would bother with Amazon's CloudFront. CloudFlare is as easy to use and free, unlike CloudFront; it's worked well for me. 2. s3cmd is a good utility, but there are occasional cornercases. On my Debian testing system, use of '--guess-mime-type --mime-type=text/html' may have... unfortunate consequences for quite a few files you would not expect, such as CSS or image files.

My side project Cactus is a simpler alternative to Jekyll and has s3 deployments built in: https://github.com/koenbok/Cactus

The rate at which these generators pop up is a testament to how awesome they are. I really think you're on to something with the built-in S3 deployment. Any plans for extending it with CloudFront caching?

For anyone wondering why you might not want to use GH pages, and use S3/Cloudfront instead... The Git 1.8 release notes ended up with this today:

"You're over the rate limit. Serve this file from your own servers. Contact support@github.com if you have questions."

https://raw.github.com/git/git/master/Documentation/RelNotes...

I wonder what the costs are like. Just looking at the S3 page, it seems this would actually be rather cheap, which makes it an attractive option (especially for a student like me :P). However, I'm not entirely certain on how much resources a small blog like this would use, so I don't know exactly how expensive it would be.

I run my blog off an EC2 Micro instance running Apache, which Amazon conveniently offers a free year of service for. (And it's not expensive paying for it yourself either ($69/yr for a Small). I used to use HostGator, AWS is much nicer--apart from outgoing emails, but once you're set up you're good.)

Pingdom is kind of nifty for its comparisons. I got the author's site to be only in the top 97% by running the benchmark again. My own site varies from being in the top 90% to 96% (I did get a 99% once), the only real "optimization" I do is reading cached pages from "disk" (EBS is notoriously pretty slow). I could probably shave off an average 100ms with some really simple tricks. Under heavy load I'm certain I'd fall over on such a puny machine that isn't even using nginx+memcached.

According to the pricing page [1], a reserved Small is $69/yr plus $0.039 for any hour the instance is running, which could add up to another $341 for the year.

[1] http://aws.amazon.com/ec2/pricing/

Good catch, I just quickly glanced at the figure without remembering the hourly rates still apply. Going with the yearly-reserved Micro you're at $128 for the year, which is a bit more than many shared hosts but at least you have absolute power over the software.

Just for a Jekyll blog, you could just host it on a free Heroku instance with an Nginx server and put CloudFlare in front of it.

In a free Heroku instance you have a lot of juice actually, especially for an Nginx server, the only problem being that the instance will go in idle mode after an hour of inactivity and so unlucky visitors can get some latency on the first request, although it's not that awful.

On the other hand CloudFlare is a pretty good proxy that works like a CDN, so with the right caching headers set, CloudFlare will serve many requests from its own cache.

> Pingdom is kind of nifty for its comparisons. I got the author's site to be only in the top 97% by running the benchmark again. My own site varies from being in the top 90% to 96% (I did get a 99% once)

If you strive for geek cred, you can get

> Your website is faster than 100% of all tested websites

The trick is to 1. have a static website, obviously, and everything inlined (so basically a single index.html); 2. run the test twice — dns responses should cache during the first one; 3. host your website in Amsterdam which is where they have (at least one of) their testers:

> Tested from Amsterdam, Netherlands on October 16 at 19:04:21

Oh, and don't use SSL.

Check out Amazon's pricing pages ([1] and [2]) for details.

As for resources spent, my own blog has had just over 10k pageviews since Saturday when I launched on S3/CF and for that I'm paying $0.07 (S3) + $0.12 (CF) = a whooping $0.19! Including a lot of writing files back and forth to get everything up and running.

[1] http://aws.amazon.com/s3/pricing/ [2] http://aws.amazon.com/cloudfront/pricing/

This is what a bill for my blog with some hundreds or few thousands of hits looks like: http://pygm.us/s6mFV+OP. This is solely S3.

Aside from being unable to figure out a anti-hotlink bucket policy, S3 is great and basically free in most for most blogs.

The blog: http://pygm.us/q6Nv3lja.

Wow, that's awesome.

Of course it's still literally infinitely larger than my current income :P. (I'm abusing the word "infinitely" horribly, but so be it.)

I think I'm almost definitely going to go the S3 route when I get around to redoing my site and starting a blog. (One day...)

I do this for my little personal website.

The key downside to hosting directly on S3 is it only does static pages. So it'll be manual HTML or a static-html-generating-tool like Octopress - none of this fancy dynamic content.

If you're happy with this constraint, hosting on S3 works pretty well for me.

I'm actually very happy with that constraint: I think something statically generated would be perfect for my goals and much simpler to use, understand and customize than something like Wordpress. The only thing to miss would be comments, but I was planning to use something like Diqus for that anyhow. (I will start a blog, one of these days :P.)

Also, almost anything would be an improvement over my current setup which is using PHP on Apache to do what a static compiler would do much better. All I really want is a way to factor my website and reduce code duplication.

Good to hear it's working well for you.

Optimizing is often an after-thought on side projects, so it's cool to see you take a stab at it for your personal blog.

Glad you like it! Definitely nice when you get to take the time with these kind of projects.

Just to clarify, are you saying in your blog post that you can't point a bare domain to your cloudfront endpoint, and that you have to use something like wwwizer to use a bare domain with your cloudfront setup ?

No, you can specify multiple CNAME values for a single CloudFront distribution; afaik nothing stopping you from configuring 'foo.example.com', 'bar.example.com' and 'example.com' to all point to 'randomletters.cloudfront.net'

You cannot point naked domains (like example.com) to CloudFront, because naked domains can only be configured with A records, not with CNAMEs.

I see, thanks.

For developers who want a simple blogging engine. Try http://substancehq.com . All your blog's content will be served via varnishd which is an awesome cache.

Varnish would have been fun to play with for a static blog. For one thing you then could use ESI to not have to invalidate the cache as often.

But I disagree with one thing — the "git push → deploy dance" is a pro for Jekyll! ;)

I have found it to be fun the first few times, but as your blog grows I've found it to be a pain to deploy every time I make a small correction. Also, with markdown there are always corrections to be made for formatting purposes.

I was watching a demo of Jekyll and Octopress pushed to github at a user group meeting. As a demo, the presenter made the requisite small change, committed, and pushed.

5 MB of upload later on a clogged wifi link, it was up live. Admittedly, that's because the generation was done on his laptop and pushed up, not done on the server side. But 5mb and a ton of rendering time is going to suck if you're ever changing things for formatting, or edit anything.

I just noticed your domain is vvv.tobiassjosten.net.

Yup! I explain why in my first post: http://vvv.tobiassjosten.net/internet/using-www-for-your-dom...

Even clicking through to the link, it loaded faster than I could Alt-Tab to the next window.

Reply on news.ycombinator.com