Pinboard 2014 Expenses | Hacker News Reader

Pinboard 2014 Expenses

docs.google.com

213 points_kcn874 comments

He runs a really, really right ship. Off the top of my head I think the SaaS budget alone at my companies is larger than that.

That said, by the standards of physically extant businesses, it is ridiculous how far you can get on $1k or $2k a month.

I am actually jealous of the level of control. I need to work harder !

Sorry that was meant to comment on the top, not yourself.

Edit: And was barely worth commenting at all, let alone this comment only drawing attention to the information-free nature of my opinions.

Oh dear ...

But the OP has shamed me into realising I do need to upgrade my financial controls - "worrying" does not seem to count as a system!

They use Hetzner for dedicated servers, slicehost for VPS, 2 colocation services, and AWS cloud.

For a relatively small site, why do they need so many different hosting providers?

I run admin + staging stuff and my own email on the dedicated servers. The Hetzner box has some offsite backups, too. I want something that won't die at the same time as the website.

I use two colos for redundancy. One is in Sacramento, and it mitigates the earthquake risk of the one in Fremont.

The AWS costs consists of stuff I'm too lazy to go find and delete. Everyone has an unused EC2 instance or S3 bucket somewhere; that's what keeps Amazon in the black.

Interesting. Are colo prices generally that high in the US? May I ask what's included at 600$ for 2(-3?)U of rack-real estate there?

  * What level of support do you have (can you call someone at the 
       DC at 4am and have him hot-swap a failed HD for you)?
  * How many shared or own gigabit or 100mbit uplinks are included? 
       (if >1: can do bonding on your interfaces?)
  * With how many other servers do you share the same circuit-breaker?
  * I guess you have at least 1 UPS + 1 non-UPS power supplies?
  * exceptionally good peering?
  * network/power SLA?
  * 24/7 access to your server?

I worked for a DC in europe some years ago and i'm genuinely intrigued how the price in your spreadsheet for colo-hosting is so much more than what i was used to (~100-150$/U).

$600 is for an entire cabinet, 15A, 100 Mbps bandwidth. HE doesn't offer half-cabinets in that facility, or I would have gotten one.

Note that 15 A is nowhere near enough power to fill it to even 1/3 capacity.

  > Note that 15 A is nowhere near enough power to fill it to even 1/3 capacity.

Yes that's pretty standard, afaik. The dc I worked at had individual circuits even for quarter- and half-cabbinets (2x 16A each), but i don't know how common that is in general. (instead of just putting 4 quarter-cabinets on the same circuits and have 1 customer blow his and 3 others' power)

Sounds like Fremont 1...

Is Pinboard still maintained by just one person?*

(*)https://news.ycombinator.com/item?id=8069474

Yes. Sometimes not even one!

Yeah, I still haven't made up my mind as to whether I should trust all my bookmarks with a guy who seems to have a knack for disappearing alone into strange corners of the world. Even if he is occasionally protected by a company of tanks and a Hind helicopter :)

Just make daily backups. That goes for any bookmarking service you use.

Couldn't you build that into Pinboard (via Dropbox or similar).

Yeah, but then you're trusting me to do the backups for you. That's much less protection.

yep, this in a crontab somewhere is all you need:

  curl -k -o ~/backups/pinboard-backups/pinboard-$(date +\%y\%m\%d).json 'https://api.pinboard.in/v1/posts/all?&auth_token=username:APItokenhere&format=json'

I meant it as a defense against concern against the bus scenario.

The backups would run while the site is live and if it ever spontaneously disappears then users would have an up-to-date backup available.

Better than the other "cloud" options, which end up getting sold or shuttered every year or two.

Also, if you are a customer, you get to follow the hilarious twitter account.*

*This is not true; anyone can and should follow the hilarious twitter account.

If you get the urge to stop paying Amazon, it shouldn't be hard to figure out what you're paying them for.

http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/...

(Fantastic to see you back on HN, BTW)

This is an old post, but may provide some insight...

https://blog.pinboard.in/2012/05/a_cloud_of_my_own/

I'm sure there's a HN discussion from back then also.

Edit: Lots more on the topic...

https://duckduckgo.com/?q=site%3Ablog.pinboard.in+server

Deleted comment

At one point the FBI accidentally seized his servers. He's pretty paranoid about redundancy and multi-site online data storage after that.

https://blog.pinboard.in/2011/06/faq_about_the_recent_fbi_ra...

That is not correct; see my comment upthread.

My bad.

I wonder what other services would suit this small, "artisinal", paid-for internet service model?

There's some RSS readers and email providers. Photo hosting and blogging are too "bulk" and high-profile for this business model, probably.

I suspect the abuse/takedown request workload is tiny, which helps a lot in keeping the cost down.

There are a lot of niche opportunities in larger segments. The hard part is always getting attention / traction, getting those first thousand customers to pay, getting enough money to afford some marketing and support (if necessary).

Just a simple example: HomeAway's huge vacation rentals business. Within the larger super category ('for rent by owner'), there are long term rentals, and that's a niche compared to the much larger vacation rental market. Several years ago, you could have created a modest business around that smaller niche, charging $30 or $50 per year for a listing. 10,000 listings at $50 per year becomes a nice, likely quite profitable, small business.

Real estate listing concepts are great if you can get traction and listings, the costs of hosting and serving up the data and images might as well be zero.

Maybe my memory is poor but I seem to recall SmugMug being somewhat similar in its early days for photo hosting. Svbtle could have done similarly in the blogging space if they'd charged early on.

In terms of single/small group companies running profitable Internet services though, there are tons, not least the well known ones around here involving people like patio11 and Amy Hoy. I'd say the sky's the limit. Indeed, it might be more fun to think of what services could never suit the model..

I've read so many terrible budget spreadsheets that I first I assumed it was all in thousands. Nice work on keeping the costs down!

Those figures aren't in thousands? That's some good cash management!

Source: https://twitter.com/Pinboard/status/494238943894700032

Is it me, or does $60-70/month for DNS seem pretty high?

Seems high to me too (I use DNSMadeEasy). But I'm too lazy to shop around for alternatives right now. The current setup Just Works™ and is not a big proportion of my costs.

CloudFlare offers DNS for free, or $20/mo if you want to pay them.

https://www.cloudflare.com/dns

That doesn't sound right. On Business Account with Add on pack it should only cost about $360 / Year. Which is $30 /m.

And you cant compare DNSimple / Route53 as suggested on twitter with DNSMadeEasy. Both in features and more importantly SPEED!.

DNSMadeEasy is already one of the best out there. Most of the others that offer similar features and speed are multiple times more expensive.

As far as i know there are only two other alternatives / recommendation. Cloudflare, which is free. Their DNS are quite fast, but i think there are limitation with what you could do with it.

And if you grow larger, EdgeCast recently offers DNS services as well. Although it start at $50, its price on per million queries are one of the cheapest in terms of paid DNS services. And like its CDN network it is very fast.

Agree, he should switch to Amazon route53. Should only cost a few bucks a month for anycast DNS distributed all around the world. Route53 also has some really cool failover and HA solutions like health checks, and failover to s3 buckets.

How does a DNS service fail over to a file store?

Magic... No, but really, here is the AWS blog post explaining the setup: http://aws.amazon.com/blogs/aws/create-a-backup-website-usin...

They think it's a bit much too :) https://twitter.com/Pinboard/status/494319940045111296

20M req/month seems like a trivial amount that can be handled by a bunch of geographically-distributed VPSes (for failover and better query times) running any sane (resources-wise) DNS server software.

For comparison, we have a bare-metal E3113@3GHz machine that at the time happens to run only a full-fledged PowerDNS server (sqlite + pipe backends, query cache disabled due to split-horizon requirements, and passes queries to a pdns_recursor if can't answer directly). The server easily handles about 500M reqs/month, with negligibly low load average. Judging from top(1) output — pdns_server-instance has about 2-4% CPU consumption, most of which is probably due to disabled caches.

But there are probably cheaper options with specialized services.

Pinboard is one of a few services I happily pay for. Honestly, I'd probably pay a subscription price for it. I've got 1041 bookmarks, it's fast, searches and tags flawlessly, the unread functionality is super useful, and I've got some (unofficial?) android app that makes it a snap to use with my phone. Keep up the great work.

Start paying for archiving then. :)

Does Hetzner has any proper competitor somewhere, with similar power/price ratio? Especially interested in some on the US side of the pond. They seem to be a go-to company for everyone that wants to host stuff on his/her own.

There aren't very many trust-worthy hosts that come close to Hetzner in the US market.

I use WebNX, and love them. They have a particular west coast focus (first major location out of LA). Their prices are a bit higher than Hetzner, but they provide an amazing service, have a solid network, and the prices are great; plus they'll do any custom setup you need. They frequently have deals on WebhostingTalk.

Check out Versaweb: http://versaweb.com/dedicated.php

You can get a E5-1650v2 with 32gb of ram and 50tb of bandwidth for $139 (no setup). They're a quality host, and their network is very fast.

Reliable Servers / Constant.com have some great prices and deals from time to time. Their network is great. They're comparable with Servermania mentioned by another post (Servermania is out of Buffalo; Constant is out of New Jersey, the DuPont Fabros datacenter).

ReliableSite.net is pretty great: http://www.reliablesite.net/

They're out of the same datacenter as Constant.com mentioned previously. Fast network, good prices.

The only one of these that comes close to Hetzner pricing is Versaweb however.

I do not have a server with them, have never used them, am not associated with them, etc, but one I keep seeing coming up in the US is http://www.servermania.com/ - it's still not quite as a cheap as Hetzner though (of who I'm a happy user).

Looking at their VPS offers, digitalocean is about the same.

Do OVH count as one? They do have DC in Canada.

This is great information on just how little it takes to run a service like Pinboard, with hundreds of thousands of active users, on your own hardware. Just imagine how much more it would cost on Heroku or even AWS, just so you can avoid having to think about ops.

Cegłowski (@Pinboard) is utterly brilliant on Twitter, by the way:

"I toyed with using Heroku but found that it was faster and more reliable to put hundred dollar bills through a paper shredder"

It's by far the best "corporate" Twitter account I've ever encountered.

The site only has about 25k active users. But I'm pretty sure it could handle about 10x the traffic without changing this spreadsheet much.

Hmm. For comparison, historious has around 7k active users and I only pay $10/mo for a 1 GB RAM linode.

Good for you!

what happens if that fails

Stats on usage, users, etc. at http://blog.pinboard.in/

Oh for some reason I thought it was more. I think my point still stands, though.

How do you define active user?

still happy i paid for pinboard a while ago.

the "no-nonsense, speed first" policy and the fact that it's actually being executed are my favorite things about the service.

Maciej finally bought an iPhone! When I met him last year in Australia he uses his 15" laptop to check his tweets.

About how much were the AWS expenses before moving to your own servers?

I never ran the site on AWS. Here's a spreadsheet from 2011, when I was running the site on leased servers:

https://docs.google.com/spreadsheet/ccc?key=0Au7yiR802qQmdHU...

I had to change providers numerous times that year for various reasons, hence the confusing number of services.

slicehost? :)

wat https://twitter.com/Pinboard/status/494239839349268481

You've stumbled upon the greatness that is Maciej's sense of humor. Just click the follow button, and consider yourself lucky.

Don't confuse Pinboard with Pinterest, which is thousands times bigger.

Pinterest active monthly users: 60 million

Pinboard active monthly users: 24 thousand

By all means confuse Pinboard with Pinterest if you are depositing that sweet VC money.

Just don't confuse it with pinboard-dot-com which looks pretty sketchy :)

Pinboard dot com is run by a hilarious, crotchety Linux guy who has been around forever. He did me a huge favor by not selling that domain despite great temptation.

Looks like he recently moved to pinboard.jp

pinboard.com is currently not responding, and was last cached on 29 Jul at 1:47 GMT (roughly 5 hours ago).

It's working here (redirecting to .jp)

www.pinboard.com redirects to a transition site.

So you're seeking VC money now? :)

I'm building a bookmark site as well and I'm looking for seed users.

If you have a such need, please try https://linklet.io/

It's not officially released yet so there might have problems.

I clicked, ready to chastise you for blatantly advertising on a competitor's thread, and for launching yet another business without charging users money.

I'll still chastise you for the blatant self-promotion on a competitor's thread, but kudos for actually charging money!

Reply on news.ycombinator.com