How I Built a Side Project | Hacker News Reader

How I Built a Side Project

stayintech.com

255 pointslillukka94 comments

Open articleView on HN

I sent a short list of questions to 1222 recipients.

Great!

The open rate of the email was 45,8%

That's excellent.

...and 27 people replied.

Oh. Oh dear.

~ 96% of respondents told that they would - or already have - recommended the tool for a friend or colleague.

96% of the people who are enthusiastic enough about the service to respond to an email would recommend the service, but the 97.8% of people didn't respond is by far the more important number. If the people who'd tried the app were a random sample of the population then 2.2% might be enough to suggest there's a valid premium business model there, but the people who tried the service weren't randomly selected - they were a self-selected group of people who are interested enough in user testing to sign up for the service. If you can't get a significant response from a group who are (alledgedly) interested in your type of product then you clearly have a big messaging problem.

> but the 97.8% of people didn't respond is by far the more important number.

I think the more helpful baseline is likely the folks that opened the email (we can't infer a lot from the ones that didn't -- email services may have simply junk-mailed the message for the other 54.2%). 27 / 560 is 4.8% which is still not awesome, but it's really not terrible.

1 in 20 people who saw OP question about the service were compelled enough to provide a response. That's not bad.

> If you can't get a significant response from a group who are (allegedly) interested in your type of product then you clearly have a big messaging problem.

I really wouldn't be so down on it. What's the average read-to-recommend on the average unsolicited call to action message? How many people actually provide extra details when an application on our computers crashes and the OS wants to send a bug report?

How many actually send the bug report? That's just a button click.

Providing their email for updates is a good indicator the base pitch is interesting, true. But it's an almost mindless activity. How many times a month does the average technophile give away an email address?

Enough times, is how many. And enough times to likely forget more often than not what they'd signed up for a day ago, let alone continue to really, really care about it.

I'm sure OP could have built a more compelling email campaign that used the right communication strategies to elicit a greater response. But, I'll take 5% over 0.

>27 / 560 is 4.8% which is still not awesome, but it's really not terrible.

That is called click-to-open ratio and you're right it is a good metric. However 4.8% is not a good number. Typical healthy marketing emails see CTOR around 20%

> The open rate of the email was 45,8%

How was this number calculated? There is not really a way to reliably detect this. Methods like the tracking-image do not work on Gmail.

From MailChimp's website

http://kb.mailchimp.com/reports/about-open-and-click-rates

And: https://blog.mailchimp.com/how-gmails-image-caching-affects-...

I click on a lot of emails just to find the unsubscribe button, and sometimes open-archive is the fastest way to get things out of my inbox

The emails were sent via Mailchimp, the open rate was gotten from Mailchimp's reports (analytics).

Yep indeed it's faulty: https://blog.mailchimp.com/how-gmails-image-caching-affects-...

I wouldn't say its faulty. They clearly said unique open rate is even more accurate now. The only thing that suffered was seeing multiple opens from same user. Which I feel is far less interesting

These numbers are usually overstating I find, likely a lot less people actually read the email.

So, this just counts broken email clients that load external images in email (which, I suppose are a lot -- but they're still broken, IMNHO).

I choked at seeing the $250/mo. Heroku cost for a checklist app! Wow. You can set up a Linode VM or DigitalOcean droplet for $5 or $10 a month, run a few commands to get software installed and get it reasonably hardened against intrusion. Hardly takes any time at all.

And for that do you get a scalable app on managed servers with a wildcard SSL cert, backups, analytics, email, and a managed PostgreSQL instance?

I'm not a Heroku user, so I can't vouch for their service one way or the other, but they do provide a lot of value-add. The time it takes to set up and maintain those services by hand is not free. Unless you yourself are in the business of providing a service platform so that you can generate economies of scale, the time you would spend over the course of a year in maintaining backups and software updates and addressing zero-day security issues for your $10/month Linode would quickly add up to the $3000/year the Heroku services cost. And that's all assuming that you already understand all the sysadmin-level details that, let's face it, most developers don't have the time to keep up with. And honestly a full-time sysadmin can't keep up with everything for every subsystem. There's just too much.

The more I think about it, the better value Heroku appears to be.

How scalable does a simple checklist app need to be? Especially when that checklist app has next to no users?

I just don't think that managing your own SSL cert and backups are worth $250 a month for a tiny application that no one uses. Logging into your VPS once a month to install updates is beyond trivial. That might not be my strategy if I had hundreds of paying customers, but the author of the article didn't have hundreds of paying customers.

The moral of the story is that you should choose the right tool for the job. A $250/mo. Heroku app for a developer who wants to fiddle around with it for a tiny side project is not a good value as far as I'm concerned.

Agree with brandon272.

$250/mo is not a viable business plan.

As much as I'd like to say my time is more valuable, honestly I'd rather save my $250/mo and built out a solution that's cheaper.

For the naysayers, my relatively small apps receive over 300k reqs/day and it's hosted on a much, much cheaper infrastructure. Just remember to cache things where appropriate and you'd be surprised how much $30-40 in server costs can take you.

Adding an unnecessary $200/month to your burn rate is silly, but if it'll kill your business then there isn't a viable business model there in the first place.

He'd only have needed, say, 10 paying customers to cover his expensive hosting. The fact he couldn't get those first 10 customers is why he failed, not the hosting costs.

What sort of infrastructure?

Bare Metal. I typically use something like OVH's offerings.

Managing SSL, backups, and updates well and correctly take time. Not just the time to do them but the time to learn what needs to be done. A productive software developer is not one who worries about squeezing every dollar out of her service providers. If you already know how to handle all the other services or enjoy learning, go for it. But for some people it's best to focus on what you enjoy and are good at, and outsource the rest.

I agree that if this app is just a hobby, it's not worth having all the extra services. But given the services provided, $250/month, when weighed against a good software developer's time, is a good value.

> The more I think about it, the better value Heroku appears to be.

Not really.

Netlify for $9/mo to serve static content over Akamai's network. API Gateway + Lambda + Dynamo for pennies per month when your traffic is low. If you get linked to by HN or some other source of massive traffic, you may pay ~$100 or so, but it will scale up (and back down) with no extra effort.

This setup is basically the same price as the $10/mo Linode for most side projects and will outperform the $250/mo Heroku setup.

He had... at its maximum 13,000 people on the service at a single time from an HN post. Most of these people just touched the landing page and bounced away. Any VPS should be able to handle that since its static traffic.

At its height, 600 people were daily active users (at a most generous estimate). Unless those 600 people were all active at the same time of day, you'd be able to handle it easily.

People underestimate what good caching can give you. I had an application that would only be able to handle 20/req/second without caching. With aggressive caching, it could handle 2100/req/second so long as that was all reads; and that was true. It handled 40,000 people a day with the $49 dedicated server hovering at 2% utilization in CPU for Postgres, Elasticsearch, Redis, Nginx, and Django Python.

> the time you would spend over the course of a year in maintaining backups and software updates and addressing zero-day security issues

Backups are a once and forget it style process. A simple cronjob can backup to another disk or upload to S3.

Zero-day security issues? They're handled by a managed server host. All kernel level updates are.

> Zero-day security issues? They're handled by a managed server host. All kernel level updates are.

And that is how much per month? A quick check with a well known company shows £350/mo. That sounds expensive, but on the other hand I'm not sure I would be leaving my security in hands of a $10/mo shop :)

Kernel updates really ought to be an automated process at any company. At that point it doesn't really matter if you have 3 servers going for $350 or 49 servers going for $50 a pop.

Yes, but you're not paying someone to just type "apt-get install"; you're paying someone to [1] avoid eventual mess as a result of that and [2] fast recovery if shit hits the fan.

In other words, you don't pay them so much for what they do every day, but for what you don't really hope never happens.

*She

Sorry, it looks like I can't edit HN comments after 3 hours, but I had no idea of the gender of the person.

Yikes. Or break all that down and get it all for free elsewhere. For a mechanical engineer that doesn't know much about doing this on their own, I suppose it's not a raw deal. I think lots of people make this argument about all those other services "not being free" and while that's true, that seems more to me an excuse for not having learned how to do those things quickly. I can set all that stuff up in a very short amount of time. My cost for doing that is almost nothing. The barrier to entry for that kind of thing goes down extremely quickly, in fact, I'd say the cost is only "high" the first time you do it. Every time after that it costs much, much less.

> scalable app

Well no, but certainly an app that can handle 100K hits per 24 hours.

SSL is essentially free now, not everyone needs wildcard. Backups are easy, analytics are easy. Email is hard - but there are services.

I guess it's about tradeoffs. If you spend a few hours (really, half a day should take care of the above. I've done it!) you can save $250/mo.

If that isn't worth it for you it isn't worth it. 2000K+ a year is definitely worth it for me!

No need to scale when there are no customers. Worry about scaling when there is lots of growth.

So did I. He had waaaaay too many things. I have an app running on Heroku that handles his peak traffic, daily, for just $16. The only difference between mine and his is the SSL add-on...but that's $20.

He loaded up too many things.

1 web dyno, the free Heroku Dev DB, SendGrid free starter plan, Google Analytics, and the SSL would run him about $27 ($20 for SSL and $7 for 1 paid Web Dyno so the app doesn't go down every 16 hours).

That's all he needs. $250/mo is OVERKILL that he doesn't need.

Pretty sure this is not a Heroku issue.

P.S. As others have pointed out, this is a simple checklist app. He doesn't even need the SSL add-on until he verifies this is something people will pay for. It's not that serious bro. People can login with their personal info over a normal HTTP connection for now.

I don't really understand the costs involved here. You can get a pretty powerful DigitalOcean droplet for $24 a month, which I personally run multiple side projects on with no issues. This could have stayed up indefinitely to tinker with, while you work on other stuff!

That's a good point, but Heroku has a much lower barrier of entry for people who are just starting out. You don't have to worry about configuring and maintaining a Linux box. Some people are willing to pay a premium to avoid those headaches and get back to what they really want to be doing: application development.

I'd argue the barrier is just as high. I taught a ten week course in back end development and I think the students would have done equally well deploying on a VPS or Heroku.

The curriculum called for using Heroku but I regret following it now. Heroku bills itself as being easy for a beginner but go ahead and try to deploy any simple Rails or Node project using their guides. Half the time something goes wrong. Either you need extra dependencies or you have to do extra configuration that the setup instructions didn't mention. In the end you have to look up how to check the logs and even if you get that far a beginner has no clue what those logs are really saying. Even as an experienced senior developer, I couldn't get the demo project I was showing them deployed without a ton of hassle and 4 attempts.

So while it may seem like a VPS has a lot more moving parts, it's a better deal overall. Same level of confusion and complexity for students but in the end they at least know a bit about how a server works (which Heroku hides) and it's way cheaper even with SSL. I could have run that same project for $30 up front and $10 monthly.

Many pet projects you can run for Free on Heroku. Their docs are pretty straight forward, not sure what your hiccups were but I've deployed quite a few sites to Heroku without much if any issues.

The asset pipeline in rails use to cause some issues but I think that has all been resolved with the 12 factor gem.

Also setting up a VPS is in no way easier than Heroku. You have to install nginx, ruby, mysql etc. Then you have to setup routing. Then you have to learn all sorts of sysadmin stuff so you don't get hacked. Setting firewall permissions, mysql permissions. Then you want to deploy that app. So you have to setup capistrano or whatever. Next thing you know day wasted.

Heroku?

    git push heroku master

Instead of spending money on a vps for my last side project, I went the heroku route and it was even easier. I spent 7$ on a private github repo.. which synced to heroku. And a free CircleCI account that also synced up with everything. Then just pushing my branch meant, automated testing and with heroku pipelines, a PR meant automatic test env.

Yes, my days of sysadmin work just to have an app in production are over. Heroku pipelines are great too!

Using AWS Elastic Beanstalk you can deploy by zipping a folder of your code and uploading it using a webpage form. The barrier for entry on cloud infrastructure is really low.

That's not the "right way" but arguably nor is using a Heroku box with no idea what's actually running on it.

$225 premium!? I mean come on. The article focuses a lot on the learning aspect of these exercises, and setting up your own services should really be a part of it. Especially for a one man operation. Typing apt-get install mongo / mysql / rethinkdb after the droplet is really all there is to it.

My experience as a one man operation is that you always have to make trade offs between building something yourself (especially if you actually need to learn a new skill-set to do it) and using someone else's solution. Honestly, in general this is essentially always the debate even at my regular job.

For me, Heroku has always been the choice over AWS or Digital Ocean. I haven't had it block me from doing anything I've wanted to do and my monthly bill is around $10 (for multiple sites). For my projects, I could never make a solid argument that moving away from Heroku would actually change the product for the end user in any meaningful way. And with the greater amount of time and energy I would need to focus on a non-Heroku server-side, it would likely slow progress.

Honestly, I don't really care what you use if it fits your needs. The main point here is that the author shut down his project, because they were spending $250 a month, to operate a very basic setup. Pick your favorite host, but a car payment to do so, is ludicrous.

I use both. I have a side project deployed to Heroku and it's fine there, but it costs me about $45/month. The same project would often have a better response time (when heroku sleeps the dyno) if it were hosted on DO, and both would require no maintenance (I haven't edited a line of code in ~2 years). It's a django/postgres setup, nothing special, and would run on the smallest DO instance.

I'm just too lazy to switch it over, honestly. I don't think it's a difference between "building it myself" and "using someone else's solution." It's about an hour of setup, maybe 2 if I encounter headaches. I probably ought to do it, given it'll earn me $35-40 per month forever.

> Typing apt-get install mongo / mysql / rethinkdb after the droplet is really all there is to it.

It's clear that the author was overspending—even a small dyno with a resource-hungry language like Rails can do a lot of traffic—but there's a lot more than just installing a web server, DB. Firewalls? WAL-E backups? Monitoring (of any kind)? A correct TLS setup? DB security?

Setting those up—in a way that you can replicate if the box is hosed—is a whole 'nother side project unto itself. Learning Ansible, Salt, etc. is great, but if your goal is to build X thing as a side-project, you'll never get anything done in a timely manner if you start reinventing wheels.

Sure, but most (all) virtual setups have automatic snapshots, and backups for a few bucks a month. The rest you mentioned, you SHOULD learn if you're getting your hands into web services. What happens when something goes wrong, who is fixing it if you have no idea how any of it is running? I guess I'm old school. You learn the basics first, and build upon it. It's not reinventing any wheel.

Before I quit using DigitalOcean, I enabled backups and found they were very spotty - occurring roughly weekly, but not on a predictable schedule (8 days, 6 days, 4 days, seemingly random). Not something to rely on in place of a real database backup policy (more like an emergency backup of last resort).

Plus, they recommend against[1] enabling backups for I/O heavy VMs (so, databases) because of the copy-on-write implementation and its effect on I/O performance.

I ended up going to AWS because I was tired of having to string together a web of iptables rules (e.g. when you provision a new webserver VM you have to iterate over your Postgres, Redis, HAProxy, Elasticsearch, etc. VMs to poke holes in their firewalls), setup WAL-E and monitoring it / testing backups, setup 3rd party monitoring VM resources for issues (CPU/memory/disk space), having to manage package updates and security fixes, aggregating logs to a 3rd party with rsyslog, all that stuff.

Unless your app is a complete throwaway that you don't care about getting hacked or losing all your data, you quickly have to start worrying about this stuff and it becomes a huge burden to roll it yourself on VPS providers.

[1]: https://www.digitalocean.com/community/tutorials/understandi...

I'm not in any way pushing the DigitalOcean setup. It was just cheap, easy to use, and offered FreeBSD as a host. I don't get spammed by them, no reliability issues, so I really don't have a reason to move at this time. If a project of mine gets too big, I'll evaluate then. But they are good for iterating lots of new ideas quickly.

I'm a former Linode employee, so you don't have to sell me on the advantages of the VPS. :) I think we need to remember that people new to programming don't know about all of the available tools and services. Costs could have been reduced by using a VPS, but they also could have been reduced without becoming a *nix expert. For instance, the author talked about buying SSL certificates, something that comes free with a CloudFlare account. They could have also used a CDN to reduce the load on their dynos.

I've had a site running on Heroku that has 200-300 visitors a day for almost 2 years. I'm still on the free dyno package. I wonder if I am just below some tipping point where I would need to bump up to the non-free stuff. My site runs fine and there are never spikes of traffic, it's pretty constant all throughout a 24 hour period.

@detaro, couldn't reply directly to your comment but so far they haven't charged me. They sent me a note saying that eventually I would be charged but maybe they got enough backlash that they decided to grandfather in the old dynos?

Your last sentence answered your question. Scale when it's a problem.

Did they grandfather "old" free dynos, or can you live with the forced off-time?

I don't regret a single penny I send to Heroku every month. It's just worth it for smaller to medium-sized projects when you don't have in-house dev-ops. Especially so for side projects when time is scarce.

I've attempted a number of times to bootstrap up my apps on DO, AWS, docker stuff, etc - all in anger. The Heroku tax is less than the time spent configuring, maintaining and monitoring my own servers.

In economic terms, this is called opportunity cost. That's time I could spent building features or playing guitar.

The problem wasn't that they used Heroku.

It's that they didn't exercise the sort of thrift possible with a website that has no users, and I'd wager that they didn't really know how to since they were a tenderfoot. I made similar mistakes as a beginner, paying too much for too little for things I didn't know I didn't need.

Heroku saves a lot of time for the 1-2 dynos ($25-50/mo) you need to vet your idea. With free Cloudflare SSL termination and a $5/mo t2.micro Postgres database, what else do you need?

For comparison, my somewhat popular forum with 257 online users at the moment and 300k req/day runs on 2 dynos. $50/mo for the application servers + a deploy/config solution is nothing.

> Typing apt-get install mongo / mysql / rethinkdb after the droplet is really all there is to it.

This is one of the reasons why web security is as bad as it is.

Even so heroku is not the right choice for this, way too expensive.

That is not relevant to this discussion, but I agree security should be a priority when developing anything.

Curious what you think the right choice for this is? Appfog, EngineYard?

Of those three (Appfog, EY and Heroku) I'd probably pick appfog, they have the simplest pricing model and the least chance of surprising you with some insane bill.

Not to mention DigitalOcean has excellent tutorials for beginners looking to set up quickly, like this one https://www.digitalocean.com/community/tutorials/how-to-inst...

@overcast For some reason I couldn't reply to your last comment but yea, absolutely, $225/month is ridiculous.

If the "reply" link doesn't show up in the thread view, try going to the comment's permalink page (by clicking on the timestamp).

Spinning up a DO droplet with an Ubuntu image and installing dokku is pretty darn close to Heroku ease of use when weighed against the cost/performance as well. Just configure ufw and sshd and you're off to the races. It's not exactly a hardened setup, but most side projects don't have much user data to start (or ever?).

I have done the same for my side projects and used cost as excuse to force myself to learn basic devOps.

I like this kind of posts, a side project helps creating side projects. it was a matter of time until I figured out that to make money you can simply teach people how to make money. who wouldn't want to learn that :)

Agreed. I like reading these. Kinda disappointed to see the project abandoned though. Seemed like you were really on to something and like others are saying, you probably could have found cheaper hosting to keep it going. Also would +1 seeing the project open sourced if it's not continued.

This goes along with a good site I saw mentioned in comments yesterday, https://www.goodui.org/. If your site was still up, I'd sign up. You might even go cheaper with something like a shared hosting solution, like webfaction. it's cheap, it's easy to setup stacks, you'd have plenty of space, and you don't have to manage your server.

It's mildly frustrating that that site mentions "Press J for next K for previous" at the bottom of the page after I've scrolled through them all..

Thanks for that link. I found it quite enlightening. I know quite a few sites that have some horrible ui that could benefit from that site.

It sucks that you had to kill your project because of costs but honestly I don't blame you for going the Heroku route. It does it all for you with some "mininmal" configuration. I like linux and have done enough maintenance on various sites so I generally don't feel lost but I'd kill to have someone walk me through the process of setting up a production ready web app just once.

Dude! $250 doesn't make any sense. I've been running a MUCH more complex side project in Heroku with ~80k visitors per week for way, way below that, closer to what you would pay with DigitalOcean or the like. For a checklist app and your traffic you shouldn't need the dynos or the database upgrade.

How do you keep your costs so low?

Aside from the free nature of the app, I think what's missing was a consistent growth strategy. In your opinion 1/ how much would be a fair price for such a service 2/ which/who sales/marketing saas/specialists would have helped it attract customers 3/ what would be the next features appealing enought to convice users to try it, use it more frequently, and invite other, should this app continued to live

"To learn more about my users, I sent a short list of questions to 1222 recipients."

brilliant read, one thing I'd modify is ask some/all these questions on signup or login. Tweak enough questions and techniques to ask [0] so as not to discourage. This way you can gain more info as as people use/try/login.

[0] Must be numerous ways to gain info building it into actions of tool.

Author here, happy to get feedback!

Was pretty surprised that it cost you almost 250$ for a month to run this. Perhaps you could have tried hosting elsewhere. For eg. this would be easily acoomodated in AWS free tier I think. Could have given you a longer runway to iterate. Anyway, cool idea. Few hundred visitors a day is not bad!

I'm pretty surprised as well. It's basically a simple CRUD app. I think that a single digital ocean box could have done the trick.

True, I should have perhaps looked at cheaper options. Dynos were cheaper on most months, that example was the 'worst' month. The wildcard SSL certificate for subdomains was a bit expensive.

I really dig the idea behind the site. I am not a UX developer, but I am improving and the service would be invaluable. What can we do to help get the iteration back in place?

I assume the reason for not going with DO, Linode or AWS was you're not a devOps guy, which is understandable. We can help you with that.

Hey man if you sign up for Cloudflare you can get SSL for free. Plus a lot of other goodies. If you need help with setting up a Digital Ocean box with Phusion Passenger, Capistrano, Ruby, let me know.

Especially when the website does not work (I have an SSL error, see my other reply above).

It's no longer live on that domain. The checklist is nowadays here: https://stayintech.com/info/UX

I remember scanning your site when it was on the front page a few weeks ago. It just looked like a checklist to me. I wouldn't have thought you could charge for it. Perhaps the idea was just a bit too static to be able to turn into an app and charge for?

Would you consider open-sourcing your work now that it's no longer live?

Good idea!

"monthly costs of running the service" was your killing argument. Why didn't you migrate to a cheap hosting ? My single host at scaleway (3.59€ per month) can handle many toy projects and gives me all freedom.

I find Scaleway tricky when deploying apps since it is ARM - the price is kick ass, but I wasnt able to go around limitations with my Python apps. Do you ever run in to issues like that? How is uptime?

Scaleway seem to be out of stock now, for a few months already. They don't accept new customers

Or OVH, where ~$4/mo gets you a VPS with 2GB of RAM.

Please don't do that with drop shadows. The combination of the non-3d image and the 3d drop shadow makes for a really disconcerting distortion effect. Are the corners of the image curled up or not? A simple 2-3 px drop shadow on the right and bottom edges would look much better imo.

Would you sell it? Email me hirowatari [at] gmail

I get a "Secure Connection Failed" on Firefox/Chrome when trying to access the userium.com website: SSL received a record that exceeded the maximum permissible length. (Error code: ssl_error_rx_record_too_long)

It's no longer live on that domain. I should have maybe titled my post 'post-mortem of a side project'? :) The checklist is nowadays here: https://stayintech.com/info/UX and the Wayback machine holds a copy of the old website.

For all the people saying Heroku is too expensive: a basic single server, 10M row database and SSL is around $50 a month on Heroku: https://www.heroku.com/pricing

Personally, for the reduction in admin tasks (which I think people greatly underestimate), improved deploys and rollbacks, and improved uptime about $50 a month is completely worth it compared to a standard server.

I don't know why the monthly bill in the article is $250 though.

Iteration 3: Die

Here's another checklist style app, only built on PHP AppEngine. So far there hasn't been enough traffic though to make a reasonable estimate of cost.

http://www.fallinloveapp.com

Reply on news.ycombinator.com