2,840 karma · joined August 9, 2016
That was my case too. It was mostly when I installed the prometheus-operator that things would go south, even if the nodes themselves were perfectly healthy and underutilized.
Getting noticed is one of the hardest parts. Just having a great product is no longer enough.
For every Basecamp, there are probably tens, or even hundreds of similar competitors.
Can I name a few? Maybe.
What about the top 10? Probably not, and that's exactly the problem.
I have also had a fair share of issues with AWS services in the past (Athena, they took care of it a few months later), and GCP too. And I understand that no product is ever perfect.
Just the discussion on this post alone, is a goldmine for anyone trying to figure out what worked, and what not for others. This is why I love HN.
To clarify a point about my article, I think everyone should do what works for them. Just because my unconventional stack for a solo founder works great for me, doesn't mean it will do the same for you.
Your mileage may vary, and it all comes down to tradeoffs.
Also, I'm sorry if I haven't been able to reply to each question, this post got way more attention than what I expected.
But I'm happy to answer specific questions, feel free to ping me on Twitter: https://twitter.com/anthonynsimon
Cheers!
Using a tool "just because" it worked for others, seems to be what often drives many decisions in this industry. For example, would you go for the 2,200+ microservice architecture from Uber[1] just because they say it solved all their problems? They probably have their reasons, and they are unlikely to be the same as everyone else.
I know my stack as a solo founder is unconventional, but it's what has worked best for me. And I'm happy with my choice.
If serverless does the trick for you, then focus on that, and keep building the features that your customers actually care about.
Scala was great, but unfortunately it made it difficult to onboard new team members into the project, so development suffered. Also the rest of our stack is mostly Python, so we couldn't use a lot of the common tooling, and libraries we had built for other projects.
We migrated it within a month, and it's been running for almost a year without issues.
At some point the service even recorded 3k+ req/s at ~200ms 99p latency. Yeah, maybe not Google scale, but more than enough for 99% of the businesses out there.
This service handles about 2.5 billion requests per month from one AWS region. We're a small team, and that's only one of the services we support. Which is great because we don't need to spend all of our time optimizing it.
I do track my expenses on a sheet manually, using a ballpark figure. At the end of the month I compare it with my AWS bill. If something seems off, I investigate it further. But I don't try to predict exact figures, more like expected ranges.
I do miss the fixed-cost nature of DigitalOcean/Linode. I didn't have to worry about sudden charges, you pay $20/mo per node and that's it. Lots of bandwidth included.
But for the most part it's fine, as long no bot starts bombarding your services with useless requests, egress costs tend to scale with revenue in my case, as my product's pricing is based on usage.
I also have Cloudflare in front of my AWS stack, it has blocked a few small attacks already, but nothing serious yet.
That's what I was going for :) Now I realize that I should have highlighted that part more on the blog post.
To clarify, this has already helped me to ship features to my customers quickly. I actually spend little time working on the stack. Last time I touched the infrastructure components was probably a few weeks ago.
Panelbear was originally built and launched in a weekend ($5/mo DO instance with SQLite), and I've been working on it for the past 3 months. I have been releasing a new feature every week. That is, in addition to my full-time job, so most of the time we're talking about 2 hours per day tops.
I'd also like to clarify that I am not suggesting this stack should be used by other solo founder SaaS. I actually advice against it in my post :)
I just wanted to share my stack, as it has worked really well in my case, and I thought it'd be interesting material for discussion.
Yes, I understand I'm entering a crowded market, but Panelbear is still in the early stages, and I'm trying to understand what problems my customers have that I could solve / how much value can I bring to them.
A crowded market also means more options for you as a customer. Specially when it comes to privacy-friendly tools, not only analytics, I'd argue that there's not enough options yet.
I initially built it for myself, as I was not fully satisfied with the existing solutions. And I do intend to offer more features than what you currently see, but unfortunately my time is limited and it's going to take a while to bring Panelbear to the feature-level I'd like it to be :)
For many projects this is more than enough. However, in my case I would be paying 2-3x more if I was using Heroku.
With Kubernetes, in case I wanted to deploy new projects or even spin up a "testing" env, I can use the same stack/cluster, and not have my costs increase.
I do realize that this is not for everyone. But for me it just made sense, and enabled me to work on new features faster than before.
However, it just made sense for me due to the simplified operations, and re-using most of the learnings from my full-time job. It has enabled me to move faster in shipping customer-facing features.
If I wanted to start a new project, I can leverage the same stack and be up and running in minutes. Which means I can take advantage of my cumulative efforts for any future project.
It's definitely not for everyone as you said, but it might be a good solution for many :)
I still have a lot to learn, but IMO nothing beats struggling with something for a long time, and pushing along until you're no longer terrible at it.
I feel that most of what I know I learned by starting projects on topics I initially new nothing about, and sticking to it.
Regarding on the job learning, I have been working with Python/Scala for my full-time job over the past 3.5 years, and that helped too. Specially in terms of running multi-regional services, and the challenges that come with that.
We run several applications on Kubernetes using AWS (combined >2k req/s at peak time), and that was a big challenge for me from which I learned a lot. Specially when you're on-call, and one faulty node brings down the entire cluster, that fails over to another region and repeat :)
Since I have a full-time job too, my time on this project is pretty limited, usually no more than 2 hours per day.
That's why I tried to automate and save time as much as I could using Kubernetes operators, well supported open source projects, and most importantly, tools I already feel comfortable with.
I don't have hard rules in terms of how I distribute my time, but recently it may have looked like this:
- 60% of the time I'm trying to understand the problem I'm solving (talking to customers, researching, evaluating features).
- 20% working on features, and issues.
- 20% promoting: writing blog posts, documentation, reaching out on Twitter, HN, Indie Hackers. Helping out customers integrate their website / understand how something works.
Of course, there's tons of other smaller things here and there, but those are the big "buckets" of my time.
Once I know what to work on next, I tend to prioritize as follows:
1) At the start of each week I pick 1-3 topics to work on next (for example writing a new blog post, working on a new feature, or even just writing documentation which I am going to next).
2) Each day when I'm working on my project I pick the most impactful thing I could work on, and start with that. I allow myself to explore a topic in depth, sometimes without a clear goal or deadline. After all it's supposed to be fun if I want to stick to it for the long term.
3) I try to respond to customer emails as soon as I can, and understand why they're giving my product a try. That's my gauge for what to work on the following week.
I know it seems like a lot of tools, but most of them are set up once and forget. Except for the occasional round of dependency updates. But even for that I have automated dependency/container scanning to detect vulnerabilities, and get notified if updating an image is recommended.
Most of the stack consists of "platform" tools, such as the ingress-nginx, external-dns, or cert-manager. So it's stuff I take with me for every project. It's like a personal Heroku I've been building, but way cheaper and fully customizable.
I've been iterating on this stack over the years with various other projects, and also learned a lot from running Kubernetes in production at my full-time job, so it's not something I'd advise for someone who just wants to deploy their MVP for the first time.
I am not affiliated with them in any way, just a happy customer. I'm actually using it for my side-business SaaS http://panelbear.com/ and saved me days (weeks?) of development work. Happy to help out if you need it.
However, there's a few edge cases that they don't support yet, such as setting the billing cycle anchor to the beginning of the month, and keeping it even if the customer changes the subscription. But I guess it's not a showstopper for a lot of use cases.
So far I'm quite happy with the SaaS approach as I'm able to prioritize my effort on improving the product for customers who support it via paid plans.
As a one-man team it's important to reduce operational efforts. I like very much the idea of having monthly subscriptions with predictable cash flows. Let's see how it goes.
As the author of a few popular open source projects [1] [2], it has been a fun ride, and I have learned a lot.
However, the honeymoon period is over for me. The vast majority of the time it’s a lonely, thankless effort.
The problem is that after several years and thousands of people and businesses using the software every day, very few actually take part in improving the project. Or even saying "thank you" to be honest.
I’m happy to work on my project and fix issues, but please don’t abuse it. Being open source means that you can change the code if you want to, not to get a developer to work for you “out of love”.
That’s why for my most recent project [3], I have decided to no longer start with an “open core” product. Instead it's a SaaS with a generous free tier. Price goes up the more value a customer gets out of it. Yes, people have already given me crap about not being open-source, but I do feel that so far it's a more sustainable model.
For comparison, I have exactly the same MBP mid 2014 15" with 16GB.
As I type I have Slack, IntelliJ (with large Scala project), VSCode, Spotify and Chrome with plenty tabs open, and sitting comfortably at ~7 GB.
I'm Anthony and I wrote the article. Happy to hear your thoughts.
Feel free to shoot me a message if you have any questions/feedback.
Cheers
Panelbear already offers some of the additional features such as reports and dashboard filters is being released this week. It's not really years away what I'm talking about.
The intention is to gather interest/feedback, then build and repeat, not the other way around. This is how I know I'm investing my time on features that my customers actually want.
I failed miserably in past products by working for a year on a "feature-complete" service, only to realize there was no product-market fit. That's why I'm turning things around this time.
And thanks for sharing your thoughts, this is valuable feedback for me and I now know the landing pages need more work :)
This is pretty much how I got started on https://panelbear.com
Feeling super lucky as I just launched last month and already several hundred websites are actively using it.
There is a free plan if you just want to try it out.
As I just launched the MVP, I don't offer yet some of the more advanced features, but I will focus on them for the coming months.
Here are some ideas I've been bouncing around with some customers:
- Filter by any dimension on the dashboards.
- More performance metrics (like time to first paint and Apdex scores).
- Automated insights and anomaly detection (i.e. "XYZ page has a higher bounce rate than normal this week" or "Visitors coming from Y spend 50% more time on your website.").
As it's an early stage product, I'm still pivoting the direction constantly to find my niche.
Not yet at $500 MRR but I’m focusing on providing a lot of value in a simple package, and automating as much as possible to reduce daily operations and move fast.
The stack mainly consists of Python, React, Postgres, Clickhouse, and Redis running on Kubernetes (deployment configured via Terraform and Kustomize). Was running on DO + Linode and just moved to AWS.
Happy to answer any questions!
I'm Anthony and I'm building Panelbear - https://panelbear.com
Super excited to announce that Panelbear is now open for business to all users!
It's been invite-only since early September, and thanks to the amazing feedback from the first 50 customers I've been able to focus on the most important features for the initial launch.
Happy to answer any questions here.
Cheers!
Invites are usually sent out the same day!
So yes, Renault outsold Tesla :)
In Munich, an average <80m2 apartment 40+ minutes away from the city center costs around 640K EUR. And if you're looking at a decent detached house we're talking about a million at the very least.
Unfortunately doesn't add up when salaries in tech are 55-85k EUR, with net income somewhere around half of that.
But yes, outside of purchasing real estate, cost of living is pretty good relative to quality of life.