The boring technology behind a one-person Internet company (2018)
broadcast.listennotes.com
broadcast.listennotes.com
It hurts to see people continue to make mistakes over and over, so I’m working on a new website and series of engineering posts to help share my approach to a lot of these problems.
Any product I start building usually begins in Rails. React is great. Vue is great. It’s not necessary, good ol’ request/response is just fine. You don’t need a service mesh. You don’t need Kafka. You add that stuff later when it’s required... if it’s required. Rails can’t be beat for startups. I wouldn’t waste any time on a single page app, it’s a completely pointless endeavor unless you have proven traction, users, revenue etc... and can afford to do it correctly.
> so I’m working on a new website and series of engineering posts to help share my approach to a lot of these problems.
Do you have a mailing list or something to get on to get updated on this?
OP's approach is mighty fine for his own endeavors, but if I had such a worker hired, there would be a serious talk about priorities. If no agreement is found even 100x engineer would be let go.
Engineers, as practitioners of engineering, are professionals who invent, design, analyze, build, and test machines, systems, structures and materials TO FULFILL OBJECTIVES AND REQUIREMENTS WHILE CONSIDERING THE LIMITATIONS IMPOSED BY PRACTICALITY, REGULATION, SAFETY, AND COST.
Your own project on your own time? Go for it. Clients should get the best fit, most reliable option, not the flavour of the moment.
(That all said, RoR is not a great choice these days as it is a niche skill)
Most of these opinions are not grounded and you can not leverage the related tools to improve the work process.
How surprising is it that, given how few variables we are left with, these variables are heavily over-engineered?
Those restrictions you describe are classic hallmarks of a toxic workplace IMHO, and somewhere I would leave as soon as I had the chance. I guess I'm lucky to have that choice.
It can be difficult to recruit for now.
(Also no, I'm not in web development, I have just observed this)
edit: to be clear, I'm not claiming to be a 'real' engineer. I love writing whimsical, complicated stuff if it makes things easier for users, and make sure I'm writing maintainable code my colleagues are comfortable with with code reviews and removing as many flourishes as possible.
Remember: You will not retain talent if you want them to act like code monkeys.
I also love to experiment with new interesting stuff. But that's not engineering, it's learning.
And it s debateable if learning other people’s apis is fun
Its because
I feel way more productive today than in the old jQuery, knockout messes
PS - I need to build an api for an app with a React front-end. My choice of language is Python (because I'm super familiar and because of the great ORMs Python has).
It’s my go to tool when I need to get stuff done. Combined with react and typescript, developing full stack is really smooth.
I've spent a lot of time building CRUD APIs and GraphQL interfaces, and this is the simplest solution that has saved me so much development time. JSON RPC gets so much hate, but it's made my life so much simpler.
The main focus is to:
- easily share React components that use Redux state and actions between projects
- rapidly develop new server-side services and immediately use them in React components (by removing code duplication when it comes to creating action creators and action types)
- have type safety
- provide basic server-side and client-side components for basic user actions such as login, user profile, etc
- make it easy to write tests
And at some point in the future, I hope to make it generic enough so the libraries mentioned above can be swapped for other libraries, but that might be too much work :)
i like next.js now but for simplicity's sake i think parrent is right.
The current project I work on is angular1 based witch is outdated and it also sucks if the dev was just learning it at the time, the backend is done with a PHP framework that is today outdated/deprecated, this is not an old project(probably started when angular was the hot new thing)
I am sure your react project in 5 years if you will try to build it from scratch you will find issues with the npm dependencies, lots of deprecation warnings in your modules in react. I am all for the best tool for the job , use SPAs where it makes sense,
For example, many vanilla PHP applications written 10 years ago will still run just fine today, whereas upgrading from one version of Laravel to the next version can be a major trauma.
it happens in JS land too, find some js package made by a newb and run a linter on it, I see issues on code that is written by coders with a few years of experience where they do not use properly the array functions, do not use correctly the lamda, they copy paste same code in 3 or more places.
Do you think that all js,Ruby or Pthon dev would properly use SQL without an ORM? I dpn;t think so, I found recently bugs in a JS codebase where file upload would fail if the file had non US characters in it's name because some parameter was not url encoded, so all developers make mistakes and ORMs were not popular at that time to prevent this kind of mistakes for SQL.
pick your poison i guess.
Also it sucks when you search for help and you need to double check if the solution is for yii1 not 2, or angular1 not the others etc.
that means when selecting a framework, don't pick the newest, but look for something that promises stability, and then stick with it. i picked aurelia to once angular 1 was no longer usable. and although i mainly chose it for other reasons, it looks like it will be a stable choice.
It is what it is, I will have to learn and maintain others code like I always done but it would be better if we had some stability as I mentioned, some standard web framework built in JS and browsers that most developers would use and the devs that want new and shiny could ignore that and use whatever.
For web when I inherit an older project I need to spend a lot of time in finding the correct version of things, like for an older react project it was using an official jsx transpiler but today that package is deprecated you can't just npm install it, so I would have been forced to upgrade the entire build system to use the new thing , but at that moment I had to fix a small thing so I found a hack for the issue and postpone the fixing of that issue.
What I think Web needs is some standard framework, or a framework core maybe jsx or other template thing but something standard to be used in mst projects and for side projects we can toy with the shiny new frameworks and languages.
I want a DataGrid with sortable and re sizable columns, i need to search the web, find something that is compatible and also make sure the license is compatible
I want a basic Dropdown widget that can do basic things like ex have a flag+country name or have a list of fonts where each font is using it's family to be rendered. Also the scrollbars can't be themed for when elements overflow in all browsers so you are forced to find something that reimplements the scrollbars.
When I give example of Qt,Net or Java I include the full SDK , if you did not worked with this kind of SDKs maybe you worked with Android one, basically you have the language STL plus anything you need from file I/O, Networking, Widgets, to the serial ports,screen, and a lot of things, you don't have to package install a new thing for each feature.
All the stuff you mention can be solved with simple libraries, maybe you could push for a library that solves such problems to be widely adopted. But frameworks always bring a new set of problems of their own, which is why no one is ever going to agree on the subject of those.
There is no need that browsers select things that are agreed but 100% of people, it needs to satisfy the needs of a large majority that need to create boring and stabe websited or applications.
I am wondering if in a few years you can pull a project from Github, run npm, gulp (or whatever the REAMDE says) and it will just work.
But again, that's just my experience, it may differ for other companies and/or regions.
If you need a contractor to help with a Rails project drop me a line (my email's in my profile). Been working with Rails since 2.x days. Not in Europe tho I'm GMT-4.
But spring boot wouldn't exist, if rails didn't change the landscape of web development.
Node.js is missing a great ORM but other than that I don't see many things missing in Node.js anymore.
There is also Rails' admin panel but most projects don't need that.
Node is a language that you could compare to Ruby.
You can ostensibly do the exact same thing with Node that you could do with Ruby on Rails... but you’d be reinventing the wheel all over the place and duct taping lots of different modules together that all utilize different design patterns.
If you’re good at designing systems, you can do it just fine, albeit slower than I could with Rails. But it’s possible. I have nothing against Node but there’s a time and a place for all tools. Getting an MVP up is not one of them, imho.
My point being: one person's "boring" could be another person's anything from "clunky and archaic" to "faddy" to "efficient and sensible." Not even Rails, built upon Ruby's idiosyncratic blood magick, is an entirely uncontroversial choice.
Dedicate a given day of the month or week for all your meetings so scheduling is simpler.
Chase invoices on the same date every month, chase all invoices that are overdue by some threshold you have set. Just follow the rule, and chase using a template. Perhaps even code the first few chases into an automated system.
Try to avoid customized proposals, use templates for many things. I agree this one is hard.
Managing employees, you have to consider each employee is a force multiplier. The idea here is to invest a day to get a week or similar.
Overall, you need to just invest a limited time into each decision. After that time, go with what feels best even if it isn't perfect and move on. I find the same thing hard myself.
It's so hard sometimes. No one to really bounce ideas off of, every little thing you have to do yourself. Staying inspired to work is sometimes difficult, because you are literally doing everything yourself it just gets so overwhelming.
https://github.com/pushcx/barnacl.es https://github.com/lobsters/lobsters
The hardest part of building this business is to keep motivated for a relatively long period of time. I'm still early in this startup journey. This is only the 2nd year of me working full-time on Listen Notes.
I think it would be helpful to surround yourself with like-minded people (online or offline) -- we are social animals. Indie Hackers is pretty good: https://www.indiehackers.com/
I live in San Francisco and I used to work for companies, so at least I can often hang out with some friends/former coworkers who are doing startups or working in tiny startups.
In my coworker space, people from different companies rarely talk to each other...
You managed to pull me away from my previous preference, player.fm. Good job!
What you gain is the flexibility to control e.g. which extensions you want to run, and exactly how you want to set it up. How important that is really depends on what you want to do.
We just made a Twitter recently, but it has all the links: https://twitter.com/ih_worldwide
To make up for the missing social interaction I play football with friends, do some consultancy jobs on the side and I help where I can with my son’s football club.
This really is my dream job and I enjoy it thoroughly. I have no problems staying motivated and every day I have loads of inspiration.
The moment I lose motivation will be a sign for me to sell everything and start something new. I ran an online marketing company before this and that was really exhausting for an introvert like me. Selling that company was the best decision I ever made, apart from marrying my wife.
Did you quit work to start this, or did you manage to work around your day job?
I get that it’s not for everyone but I grew up on a farm so I kind of got used to the quietness early on I guess.
---
I only learn when I have a project in mind that I want to do. No matter if it’s work on or around my house, creating an app or designing something like a logo or interior. Then I just start. Walk into a wall. Stop. Look for learning material or inspiration. Start over.
Reiterate. Learn more. Reiterate.
I’m not easily frustrated by failure, starting over or general lack of progress. If I have an interesting goal I just keep going. It might not be the most efficient way, but it’s how I learn best.
---
There is no magic bullet. It's a combination of inspiration, motivation and perseverance. Example: I spent 1 year learning to built an app in Swift (together with a friend, one evening every two weeks), then we decided to ditch most of what we learned and start over in Flutter, knowing that it would take us another year and I just don't care. I'll get there eventually.
Especially when you think "this all works great, but would still work equally great with another human or two here."
I'll see if I can clean it up and finish it though; if you have any particular questions you'd like to see included (selfish: so I can feel like it's less braggy and more interesting), I'd love to hear them. Contact info is on profile if you don't want to derail this thread. :)
Sure, you can do it with friends etc. But at some point you just become menace asking them to think about your stuff so much.
One man show / sole proprietorship is great, but its bad for family life. It's 24x7. Atleast thats how I saw my dad, dining room was the company meeting room and we all are unpaid company "helps",assisting our Dad support the family. This is the main thing that holds me back from being an entrepreneur (sole proprietorship 2.0).
at another space i find people are more social. they organize events, coding workshops, even cooking classes, occasionally they cook meals. there is a fully stocked kitchen that everyone can use whenever they like. just pay for materials used. people often go out for meals together. etc...
when you are working, people will leave you alone, but if you know who else is there and what they do, you can talk to them and help each other. it helps that many are freelancers and not secretive startups hustling for investment money.
I was expecting something more along the lines of PHP + a single MySQL machine, plus all the accounting is done on a tablet made of actual stone.
This is not that.
But as I said, it depends on your history and what you work quickly with.
In the article the author says they chose Ansible because docker felt too heavyweight and unnecessary. It’s true that docker has a lot of stuff you don’t need in there, but trust me when I say Ansible is no picnic. In fact I find it easier to take what I need with Docker and Kubernetes and get running way quicker than I would ‘naked’ Ubuntu machines that I need to semi-manual previsioning with shell scripts or Ruby scripts. I got up and running with Kubernetes on DigitalOcean in less than a day, and Dockerfiles using familiar technologies I’ve used before take less than an hour to iron out.
You’ll get people who say that Docker is overkill and unnecessary and far too complicated for a solo project, but then you’ll get others who believe it’s a small file with 20 lines of declarations, sat alongside a 30 line yaml file that can provision all of your infrastructure instantly across any cloud provider you choose.
I've always wondered about how easy it would be to setup a SAAS that adheres to HIPPA.
That said.....had I not done "this", then I probably would have done something more lucrative and "easy". I see non-PII opportunities everywhere, and (mostly) only hang around because what I do now pays the bills.
All the things they mentioned:
Ubuntu, PostgreSQL, Elasticsearch cluster, Redis, RabbitMQ, Django / Python3, uWSGI, Nginx, Celery, Celery Beat, Supervisord, React + Redux + Webpack + ES, Amazon S3, Cloudfront, react media player, Ansible, Datadog, PagerDuty, Rollbar, Slack, PyCharm, MacBook Pro, Vagrant + VirtualBox, GitHub
WeWork, iTerm2, tmux, Notion, G Suite, MailChimp, Amazon SES, Gusto, Upwork, Google Ads Manager, Carbon Ads, BuySellAds, Cloudflare, Zapier, Trello, Medium, GoDaddy, Namecheap, Stripe, Google text-to-speech API, Stripe Atlas, Clerky, Quickbooks, 1password, Brex, Bonvoy Amex card, Capital One Spark
PostgreSQL, Elasticsearch cluster, Redis, RabbitMQ, Django / Python3, uWSGI, Nginx, Celery, Celery Beat, Supervisord
On the other hand, I disagree with his points about docker being overly complex. Docker images are simple. Kubernetes, once you get the hang of it, is great on GKE and gives you automatic SSL via Google or let’s encrypt, and load balancing and auto-scaling just works. It’s probably more expensive than managing your own servers, at this level, but maybe not since you could pack more services into fewer compute instances.
My production server stack is Apache + some Ruby CGI scripts, to serve static files and handle billing webhooks. I spend less than an hour per week on devops maintenance.
KISS is the #1 principle when scaling a solo operation.
Didn't ruby remove its CGI libraries from their standard library somewhat recently. I believe there were mostly helper libs, but I am curious how that works.
You don't need libraries. If you have reasonably low traffic CGI can get you really far.
And even after that, I would expect that CGI augmented with some careful caching (read: all static content and some dynamic content with sensible expiration policies) will get you even farther without having to dramatically change anything.
For the very few people who don't know mike or sidekiq, read this: https://www.indiehackers.com/interview/how-charging-money-fo...
- Ansible for provisioning
- Python/Django for website/api
- VueJS for frontend(where needed, some pages are simple Django templates)
- Celery for background work
- uWSGI and Nginx as servers with AWS Load balancer
- Elasticsearch for search
- Redis for caching
- Postgres with Postgis as main datastore
- Datadog for monitoring
- Cloudflare for DNS
Some differences as I am working with a team:
- We do use multiple branches and git tags for releases. Feature branches are also common as multiple devs maybe working on different features.
- We use Gitlab-CI a lot for testing and auto-deployment(ansible script can be called from our machine as well)
- Terraform for infrastructure provisioning. We have stopped provisioning any AWS service by console. Once the service is provisioned by terraform, ansible takes over.
I have tinkered with Docker, Hashicorp Packer but this setup has been dead simple to reason and scale reasonably well.
Sure I prefer static languages nowadays since I work in a much bigger company, however I think the benefits of static languages is overhyped.
- We use standard tools like flake8, isort, black for code linting.
- We are not following TDD, but we do write tests for all features. Pytest helps here
- Proper code reviews. I cannot stress this enough. As a Tech-Lead I have pushed for code reviews even when working on (supposedly) tight deadlines.
- We are also exploring to use type hints introduced in Python3.7.
PS Shout-out to Poetry(https://poetry.eustace.io/) to make our dependency management easier than ever. Updating third party libs is a lot easier now.
There are four popular typecheckers that I'm aware of (mypy [1], pyre [2], pyright [3], pytype [4]), which is a little confusing, but any of the four will catch a lot of type issues that typical statically typed language compilers would catch. It's not as robust as true static typing - there are sometimes false positives and false negatives - but I find it helpful.
[1] https://github.com/python/mypy
[2] https://github.com/facebook/pyre-check
The real reason to switch would be performance. The interpreted nature of Python that makes things like Django and PyUnit possible also makes it kinda slow. Some places choose to parcel out a few key services in a faster language (C, Rust, Go) and use bindings to call out as needed.
A lot of people get caught up on "x is slow, y is fast" and try to over-architect too early on, then end up focusing on the wrong parts of their product, and sooner or later the project falls apart.
> ... and it becomes not so bad to work with
O_o
How did you manage to put these two statements in a single sentence implying positivity :)
Why do people choose to explicitly write unit tests over and over again for basic things which could be caught by a compiler?
People are losing so much productivity (both short and long term) and being deceived into not having the type system "stand on their way" while they're typing out the initial prototype. This always bites back when you start first significant refactoring.
I will never understand the appeal of the dynamically typed language. Modern, statically typed languages with strong type systems, provide so much more productivity, reliability and confidence in the codebase that it is really irresponsible to ignore them nowadays.
Also, I am a long time user of wakatime. Awesome product :)
If you create a course on udemy/YouTube on how to setup a production full stack, there are many basic setup videos or tutorials, but none on production full stack with load balancing, replication, caching, etc
It's incredible the amount of knowledge required for a single person tho when you think about it eh? It's the full frontend (which I would have more trouble with) + databases + caches + search engine + metrics + deployment + source control + sysadmin all baked in a single person who is also trying to make it a business!
Kudos for the effort and making it happen, one day I might be joining the same journey with the same stack! Just gotta figure out what actually motivates me to build a business on top of =)
I always thought every engineer should know how to deploy a server, install deps, understand caching, etc and setup an app... turns out, that is apparently not even remotely expected at most companies. I guess the bigger the company the more narrow the skillset required.
In bigger companies traditionally they had a person / department more specialized per part of the stack, I think is a lot about the "devops" culture.
The slow progress was dictated by the extra communication to coordinate the specialists, but at the end of the day it ends up being worth it.
(All of this of course assumes you work in an org with lots of really good specialists).
iteration is slower at bigger companies but some of those teams are inventing wheels as they go. which requires slower movement than general web/mobile dev.
I am a generalist full stack developer who has built a SaaS business very similar to the way OP has. No team of me’s could’ve produced, say, TensorFlow.
Great teams and companies have generalists and specialists.
A 3 person engineering team doesn't want to hire a specialist DB admin who knows Postgres back to front but can't code. A company of 1000+ engineers might, because they might have problems that require that expertise.
But any engineer working in a company of 1000+ engineers is also subject to friction resulting from resource allocation inefficiencies, communication overhead, regulatory compliance, etc: the reduced output from those factors has little to do with the specialist engineers doing worse work, and more to do with them working at the size of company that can afford to hire specialists.
(You could make the case that buck passing is an example of resource allocation inefficiencies, as it is often accidentally or purposely enabled by company management, but nonetheless, that's not a generalist vs specialist tradeoff.)
I must also point out that what FANG employees do get in abundance is the ability to learn from highly scalable systems that others build, cutting edge tech others work on, experiment on company's dime, pursue an eng/mgmt tract more in line with their personal interests, and job stability.
It also affords them an opportunity to be part of a team that one day builds a groundbreaking tech and pushes the envelope forward not just for the company but the entire industry.
I currently run the remains of my companies as a lab that is spread out to a few datacenters and provides a UX where anyone can request a VM, launch a container, or drop a php/java war/RoR/django/etc onto a custom app server of varying security restrictions. You can request a service/vm/container by API, by chat, or any other host of events through my half-baked event controller and change mgmt database. In a lot of cases, changes are a two-way street. You can modify e.g. a bind zone file and that will reflect upwards in a CMDB or vice-versa and watch the zone file update automagically. The original idea was to allow mixing sysadmin strengths and still maintain a reliable complex system.
So now I have a platform that spans multiple datacenters, uses infra as code as you would expect (supporting another cloud provider is simply adding glue to their apis), has loadbalancing and SSO, and it's just literally sitting on the sidelines exhausting the remaining budget until I finally get tired and liquidate it all. The motivation of building a business on it is so tiny after years of failed attempts and seeing the shared cloud model completely destroy ROI on holding hardware. I can and have built e.g. fleet tracking services. I have gobs of storage, so I run an object store for giggles. But have no clue how to generate revenue from these ideas when the market is already saturated. My last ditch idea is to create a learning ground for the public. Training on how to build apps that scale, manage systems at scale, and give a real world environment to folks who may otherwise not work at an organization with more than 100 servers. shrug . until then I chop-chop away at my dayjob :)
One comment - he dismisses serverless as being overengineering. I think the correct POV, moreso for the single-man company, is that running a server to perform a task is the overengineered option.
One can see from the snapshot the servers are indeed severely overprovisioned and underutilized. Building an api with api-gateway + lambda is less work than running django in uwsgi behind self-managed nginx, and is guaranteed to be more cost-effective for unpredicted load.
Same logic applies to the db servers - why not hosted?
And last - the inf is a good reminder that prefixing your api routes with /v1, /v2 is always a good habit.
Of course, the overarching rule is always "if it works, don't touch it".
Nontrivial cost reduction would be switching to a different host instead of aws.
As a solo entrepreneur I can say time risk is a crucial thing to be mindful of. I'll take 10h +/-1h vs 5h +/- 15h any day.
Also, people seem to be dismissing the value of a rock-solid local end-to-end dev stack which with the latest iteration of technology I feel has become very complicated and expensive. The OP can run the same Ansible for his dev stack as well as his prod and his dev stack doesn't turn into an AWS bill. Most of the AWS managed tech & services is only reliably testable on AWS at a functional/integration level, which is a cost that a company this small shouldn't have to absorb.
As me how I know. When I was a solo dev doing my own thing, I'd spend way too much time working on things that really wouldn't affect the business but were "good engineering things" to do. If I spent more time working on things that would grow the business instead of wasting weeks writing fancy deployment scripts, maybe I'd still be doing my own thing now!
Timeframe has to be considered. If it only takes a week, I'd strongly suggest biting the bullet. A week's budget can be quickly spent troubleshooting issues, scaling servers up and down, installing software updates, hardening systems and whatnot.
It will most likely be much cheaper to run too. Which, in a single-man operation, may be worthwhile.
Also some workloads are really bad for lambda, because you can (total napkin math) run 4 cores at 100% load for 24h a day to do your processing, all at the cost of "1 instance".
Depends on the use case. I run a cron monitoring service on a similar nginx/uwsgi/django/postgres stack [1]. My service needs to handle lots of really small and simple HTTP requests, and almost every request needs to do a (small and quick) database write. I did napkin math – at the current usage levels, Lambda per-request fees alone would use up significant chunk of my current hosting budget.
[1] https://blog.healthchecks.io/2019/08/a-look-at-healthchecks-...
As a small business owner, there are two types of cost that I need to consider:
Time: the time I use to do A is the time I can't use to do B. Unfortunately I haven't used serverless so far in my professional career -- in this sense, I'm not full-stack enough :) It takes time for me to learn it, understand it, operate it, and experience various outage scenarios to gain the true learnings. It's more costly for me (probably not for others) to use serverless than the things that I already understand. I'd rather spend more time on other non-engineering things nowadays -- believe it or not, I spend 1/3 of my working hours replying emails :)
Money: the money I spend on A is the money I can't spend on B. I decided not to use api-gateway + lambda & hosted db servers, primarily because of $$$. I actually did the pricing calculation a few times last year. In addition, api-gateway + lambda also require some time for me to learn, which I should use to talk to users, marketing, building new product feature, thinking (yep, thinking also uses some time budget :)...
Thank you for this excellent write-up. Monetizing APIs is always a great topic; did you consider or will you consider using a 3rd party API management service such as RapidAPI or Apigee to keep track and charge for API usage?
The api was launched in Dec 2017 right here on HN: https://news.ycombinator.com/item?id=15825900
The only issue I've found about using serverless is the database. In most cases (Firebase, Fauna, Cosmos, DynamoDB) you have to couple your stack to the DBaaS provider which is not a great idea. AWS recently announced Amazon Aurora PostgreSQL Serverless but while it allows you to use regular Postgres tools/queries you are again tied to AWS.
You'd also see that 7 of his 8 worker boxes are almost at 100%.
If you discount building an AWS-specific deployment process that includes "pip install" from an AWS linux machine image, zipping the project, and putting it in S3.
It's a niche, just like all solutions that aren't a single Unix process on a single Unix box. Even CGI scripts are a niche. You pick the niche that you know.
I was all gung-ho about serverless for a while. I wanted to release a demo for my product and thought I'd cut through all the hassles of managing my own server.
I found it bewildering. It was a whole new skillset with new benefits, but also new considerations and headaches. When push came to shove and the clock to release my demo started ticking down, I just went back to a linux server.
I use the same linux distro at home and on the server, and there are about 3 technologies I need installed. On retrospect I think I made the right decision, but happy to have my mind changed.
For someone starting out you don't need to worry about learning Linux and how to configure Apache to serve SSL certificates in the right way, but that is important to know, unless you are happy to always rely on (and pay for) someone else to do that.
To me serverless is similar to Heroku, it's great for starting out but as you start to grow it's going to quickly become a lot cheaper to maintain you own systems. Except with serverless it's not so easy to self-host because you end up relying on all the tooling the vendor provides.
Since the author is reading/commenting here, and there was a large amount of space in the original article outlining tools/services he uses, can I humbly suggest they use a tool like "Grammarly" or similar to help with the word-choice(s)?
Some distracting use of plurals for terms - e.g. "traffics", "stuffs", etc - may have been avoided and other spelling and grammar aspects could have helped make this easier to read. That all being said - the 'essence' of the article is to be commended.
Alright. Just make a "boring" website now, it's "easy".
If it's one thing i really dislike within both the scientific and the technological sphere it's this arrogance disguised as common knowledge. Because it's not. Articles like this is nothing but bragging. The author, whoever it is, clearly has a very long time working in the field acquiring this knowledge. Be humble.
Microservices, great for companies with many teams. Not so much when it's three people scrambling to create something meaningful. Monolith all the way.
- different SLA and scaling requirements per component/endpoint
- different security domains per component/endpoint
- different rollout strategies/policies per component/endpoint (no need to restart your long-lived client TCP sessions in parallel to rolling out a business logic fix)
- ...
I've been happily developing and deploying microservices for small customers, in teams of 1 to 10 people. But I also don't work with customers who just need simple CRUD apps.
If the 'app' has enough different 'bits' I'd favour a SOA no matter the team size.
Somebody now wants to say that isolation doesn't necessitate distinct services; they're probably right, but the alternative is great discipline - why not make it easy?
It's a form of defensive programming isn't it? - if widget factory shouldn't be using a function from auth helpers, make it impossible!
If you can't trust your team to follow a process correctly, you're fucked in a way microservices can't save you from.
This post seems like a sea of trendy buzzwords to me. Maybe he thinks it's not trendy because it's not docker/serverless, but to me that stack is mainly a load of fads. React/Redux/Webpack are "standard", damn, 4 years ago you could have replaced that with a plethora of alternatives, as you probably will be able to in 4 years time.
The stack described in the OP is a fairly modern stack...
Just wanted to say it.
Boring tech would be a million dollar business running on Wordpress.
I thought not. It's not a story the HN Council would upvote.
https://en.wikipedia.org/wiki/Small_and_medium-sized_enterpr...
My dream is to find a good idea and work solo. It is a little harder as a mobile engineer but you're right as you mention in the blog. There are technology nowadays that help me with backend/dev tools.
Cheers.
I also like the different and varied sources of income that you have, from the transcription service to ads to your API. Seems like you've built a great platform that you can use to experiment with different revenue models.
One additional question - your product is called Listen NOTES. Are you planning on adding note taking functionality to it at some point? One thing that I've always wanted to do was to jot down some set of notes to myself (typically during one of my bike rides). I always imagined that being some kind of voice activated thing, but I'd like that note sync'd to the spot within the podcast that I was listening to (and perhaps transcribed as well). Any thoughts about building something like this?
Thanks again for building this service!
I really think the key thing here is familiarity. K8s is a bit different, but certainly in OP's position I (personally!) would be more comfortable with an image for each component. Perhaps a machine image rather than docker, if each component is going to be on its own machine as described, but something at least semi-reproducible for sure.
When I'm working on something alone, and particularly if on and off and not for several hours every day I need to be able to come back to it in a sort of self-documented state that doesn't leave me scared to touch anything lest it crumble.
Of course it's not for everyone but outright saying it's a waste of time hasn't spent much time with the ecosystem to know how easy it's become latey.
Learning system packaging and how to rebuild src packages is much much simpler than managing a container ecosystem
Think about this... if it was so easy to manage containers why does redhat still ship its operating system in packages ?
I am curious though, since I am using Google Cloud (App Engine in particular) for most of my company's modest backend needs: Would Google AE be able to handle all these backend requirements as well (but obviously without all the configuration and setup required)? Or asked another way: When is the point when you should move away from something easy and low-hassle such as GAE to something more advanced that requires a bit more manual configuration like setting up your own AWS servers?
Not trying to be critical, just honestly trying to learn from folks that know better than I.
In that spirit, you should use the App Engine that you already know for as long as it seems to be working well for you. When you run into a problem that can't be solved in App Engine, that is the time to ask for advice on how to solve your problem.
I'm primarily a front-end dev so I keep things pretty simple on the back end side.
Stack:
Meteor, managed hosting on Galaxy
MongoDB, hosted on Compose
React
GraphQL / Apollo APIBut he needs to be careful not to overdo it. It's fun and exciting to get all this tech up and running. But at some point, it becomes a drudge. And burnout is right around the corner.
I would say that if he can farm out as much as possible and focus on marketing and sales which are the drivers for most company's continued success. In a sense, he has by using the cloud but what he's doing is way too much for one person.
I used to be a laid back kind of guy and would get irritated when I was hurried and clearly there was no impending death. But now I understand that the limit lies within us. At some point, we all give up. There are those that take a long time and there are those that give up relatively fast but giving up is part of the process. So we are in a hurry to get as much done before we decide to stop what has not been successful. By putting so much burden upon your self you make it so much more likely that you will give up before you find success.
You have no clue what is "too much" for the author of this post.
To all the people thinking “I couldn’t possibly do this myself” just look at the mention of UpWork towards the bottom. Looks like the author has brought in contractor help at some point.
it teaches you that you don’t need all the fancy stuffs, but you may need some of them. That you don’t have to shoot for the unicorn project, that you don’t have to build an A+ people team, etc.
It’s intelligent and gives all the interesting details to not give any kind of false impression.
On https://www.listennotes.com/api/pricing/
There's a typo: "Instantly access to 771,769 podcasts"
Instantly -> Instant
Really enjoyed the post, thanks for sharing.
Access as a noun or access as a verb.
"Instant access to x podcasts"
or
"Instantly access x podcasts"?
"Instant access to 771,769 podcasts" -> access as a noun, not a typo
"Instantly access to 771,769 podcasts" -> access as a confusion, definitely a typo
I think it's a great way for start up to manage the business but as more people get hired the organizational complexity might be too much to bear.
A lot of projects can and do apply CI tooling to achieve this. Every commit to a branch triggers a set of declarative deployment pipelines, simple. IIRC buzzword is "GitOps" if you want to find out more.
A useful direction to unpack is to define architecture. You cannot re-architect a building after it has been built, you must only demolish or rebuild elsewhere. This is why all architecture changes must be additive in nature, otherwise you're pulling the foundation out from under the building.
Software should be modern pyramids.
Granted those assumptions, you've got to get pretty far up Maslow's Hierarchy of Needs to hit "rent a WeWork"
Yes, Listen Notes is making some money - not a lot, but enough to cover all the cost and bring in a bit profit as of today.
The basic idea is that Listen Notes should be free to 99% of users, while making some money from 1% of super users.
We run ads (obviously) on the website and we provide API: https://www.listennotes.com/api/pricing/ And I've been experimenting some paid features that are needed for PR/marketing/journalists to do their job, e.g., https://www.listennotes.com/datasets/
And today is special, because on Sep 16, 2017 (Exactly 2 years ago today), I started to work on Listen Notes full-time!
Just curious, is your company still one-person?
Here is the pricing page: https://www.listennotes.com/api/pricing/
Looks like it's an ad-driven revenue model, and it looks like it gets ~ 1m views/month: https://www.similarweb.com/website/listennotes.com
Assuming $1 per 1,000 impressions, we get $1,000 / month.
That's purely the website. I can't speculate on the API side of his business though.
I've tried to learn Rails (Ditched learning Rails because JS framework are all the rage). Tried learning Flask/Django because it was considered easy. Ditched it too cause internet people said it's slow.
I tried learning Go, Phoenix and jumping between what's considered cool in last few years.
And here I am, no confidence to do a basic simple app. It's been an interesting journey with no luck because of constant chasing of 'Exciting' frameworks/Tools.
Pick something that has been around a while (Ruby on Rails, Python with Django, Java Spring) and go with it.
Focus on performance once it becomes a problem. In python, you can rewrite parts in C that need a speed improvement.
Otherwise, focus on making a great product that solves a problem for your users.
I'd suggest minimizing, or even ignoring, JavaScript. A really basic CRUD app doesn't need any.
The differences between languages and frameworks won't matter very much unless you're joining a team with a preexisting stack or an edge case company/application that needs top-of-the-line everything.
Even if your app usage does scale beyond what your first version can deliver, you'll often get a bigger improvement by rewriting your app in the same stack using the new knowledge you've learned, than you would by rewriting the same logic in a newer stack.
- PostgreSQL database
- Nginx proxy in front of Django apps for UI and API servers (I use gunicorn instead of uWSGI though)
- Cron jobs which invoke django-admin commands to keep the PyPI mirror in sync
Perhaps the only place I'm any fancier than OP is that my deploy script is in Python, not shell, since any time I try to write a shell script with even slightly nontrivial logic it falls over and catches fire :)
Here's a trimmed down version of my web configs: https://wakatime.com/blog/23-how-to-scale-ssl-with-haproxy-a...
I am the only engineer but two non-technical friends (a designer and a lawyer[0]) make up the rest of the company.
[0] the original idea was the lawyer's. :)
thanks Ananth
Unless the web also uses the API service to get its data. Really great article
The frontend can even have some exponential cool down on requests when the backend is down and the users who are online in that one minute window will suffer just notice it being a bit slower.
He even mentioned his uptime in last years in the post.
Recently I rode the Figma API release hype train, and I made a plugin which is quite successful. Almost 10k active installs. It is basically a collection of the most popular viewports and their market share. I also have a server where I store the data. Every other day I run a script which updates the market share and that is about it. To summarize it:
- I have a client (react app written in typescript) which has a very simple "caching" mechanism (if you asked the server for new data more then a day ago, ask again) and which shows the data and nothing else. - And I have a server (express js) running on Heroku (where I also have the Postgres DB). The server has a basic REST API and very default admin interface. But most of the time the server is not doing anything.
It was something I was able to put together quite fast, but now reading all the comments I'm thinking whether it isn't over-engineered and whether I should go with serverless and possibly reduce the costs.
My own one-man startup Trunk[0] is very infrastructure heavy so I’m super thankful for technologies like Docker. You don’t have to go full-blown Kubernetes but there are various small wins you can still have. Need to upgrade a piece of your stack? Just build a new container a deploy it! Need to bring up all your services in development? `docker-compose up`! And with Docker Swarm, it’s super easy to scale up background job and application processes across multiple cloud instances. I can’t imagine doing this without Docker.
When getting a business off the ground, it’s extremely important you do customer development and not new technology development. When running a business however, new technology can become a competitive advantage and help you move faster. Just make sure it’s really doing that.
- golang, gopherjs, bootstrap, DO, and kubernetes.
I never found a way to promote it to groups of people instead of individuals. Even in a meetups/expo setting it's hard to get critical mass and momentum.
I've built Libr (https://librapp.com), a full social networking platform designed to fill the void Tumblr left behind.
The hosting is serverless for both the front end assets and back end no servers to maintain and infinitely scalable.
Libr has a front end progressive web app built with VueJS hosted on S3 with CloudFlare in front.
The back end API is hosted on Lambda. By using serverless framework things are portable. I could easily migrate to another serverless provider or even host my own.
Both the front and back end use the same programming language, TypeScript.
The database is Elasticsearch, just like WordPress.com (they don't use MySQL for most things).
Each month, I receive a nice little invoice from AWS for under $1 for the hosting. So far CloudFlare is free for me.
I built everything by myself, with no team. I have had input from Libr users on feature requests and UI improvements.
Super important part. Any details about how much it costs for single person company? :-)
https://individual-family.kaiserpermanente.org/healthinsuran...
I wish I could've known more practical info about starting a company before I quit my day job, e.g., how much for insurance, what company credit card to use, how to pay tax, where to find lawyers... Online articles / advices are mostly focused on big picture or very abstract concepts or fortune cookie type words :(
Earlier this year I gave a talk to some college students, which has some more practice info about starting a small internet company:
https://broadcast.listennotes.com/good-enough-engineering-to...
I do the same thing on Linux.
build.sh, deploy.sh.
But, I also use a run.sh which runs tasks that I may not remember to run if I come back to software after a while on a certain machine.
Like 'git pull' or 'bundle install'. Then ./dev-setup.sh or whatever for starting up tasks.
I use multiple machines and having these two simple commands saves me so much frustration.
You sit down to began work. You just run ./run.sh and it will setup a dev instance for you. No sudden alerts that your gem is out of date... or worse... no warnings at all and hard to diagnose sudden bugs that steals half a day from you.
set makeprg=./build.sh
set autowrite
I've found trying to figure out complicated build tools to mostly be a waste of time unless you really need them. It also encourages you to not do things that would require very costly builds.
i build SAP applications now because they make it easy to work with a reusable backend.
for the rest of my system, i use basic LXC containers. no docker, or other container management system.
deployment consists of setting up an instance of the backend, and generate the frontend as static files that can be hosted whereever.
saltstack to control it all. done.
OP said they are AWS. If they're using Amazon Transcribe, the cost for that is $1.44/hr.
A 150%+ profit margin is awesome! Well for that one piece of the service. Obviously at least some of the cost of the website and database, etc must be included, but still. Pretty amazing.
I'm not sure where you got the $4/hr price, but indeed kudos to OP for building a business around reselling cloud services for profit.
> ./provision
provision usage:
./provision environment role [hostname]
This script will provision the given role in the specified environment using Ansible
If a hostname is specified, provisioning will be limited to that specific serverThat's a proper full stack developer with a business mindset as well. Hope I'll improve my backend / devops skill to that level at some point.
For example https://newreleases.io/
- Go with embedded databases for web services
- Vue.js for frontend
- Nginx just as there are other sites on the server
- Prometheus/Grafana for monitoring
- Gitea for code hosting and some automation
- Bind9 for DNS
- Debian
I've had an idea that I've been working on part-time but it really needs some full-time love. At what point do you decide to go for it full-time?
One question: what all do you use contractors for? How has your experience been in managing them?
Thanks
1. Built some reusable ReactJs components. 2. Design / illustrations 3. Proofread website copy / blog posts 4. Built experimental app like this one https://itunes.apple.com/us/app/just-listen-simple-podcast-a...
(probably there were other random things... I got to look at the billing history on my upwork :)
It's very good stuff!
To quote a line from an Elixir talk I recently watched: "it feels like cheating".
It mostly worked really well though, kudos to whoever figured it out. And you could mock a dev setup right in PyCharm.
Thank you! Let us solve problems and not chase ultra competitive expertise for its own sake.
Also curious if you have a monitoring/recovery strategy for when you go camping or on honeymoon and need to be offline.
Do you understand what the web, api, and DB servers are doing?
If your interested in Python and want to start small I can recommend Flask. Flask is smaller and could be more user friendly than Django.
Here’s a great tutorial. You’ll build a blog with Bootstrap, Python, Flask and SQLAlchemy.[1]
[1]https://www.youtube.com/playlist?list=PL-osiE80TeTs4UjLw5MM6...
https://wakatime.com/blog/14-pirates-use-flask-the-navy-uses...
(I'm the author of this blog post)
I couldn't do all these things when I was in school :) I worked in companies for a few years and learned some engineering practices. Then I had basic skills to prototype my own side projects. Then after working on many silly side projects, I started Listen Notes.
And initially, Listen Notes was running on 3 tiny DigitalOcean servers ($5/month each?). I logged in each server to git pull to "deploy to production". Then I added things little by little, day by day. It's a process. The key is to get started. People say that showing up is 80% (or whatever percentage) of success. I think this is very true. Just get started and you'll figure out things along the way.
The first version of a web service can be as simple as flask app which you run in a screen session somewhere. Better to start somewhere than get overwhelmed and never do the thing at all.
You take a project like this one step at a time. Some bits of it are relatively easy - setting up a few postgres servers doesn't take much knowledge. ElasticSearch is a little more obscure, but for the most part, things like this are running a few commands, and setting up a few config files with the help of docs and google. Same for Redis, nginx... etc. Which isn't to diminish devops - you can dive deep into each of these configurations and develop pretty complex setups, but by the time you actually need to you hope to be making enough money to pay someone else to do it.
You won't get everything perfect all the time. You'll have to revisit parts of the stack and tweak them. But you can take it a day at a time and do what you need to do.
I wonder when developers are working for themselves / very early stage, if one automatically becomes more conservative? If someone else is paying for your time, it's nice to experiment and grow personally. When it's your own buck, the focus is on pragmatic shipping and getting revenue coming in.
When you work for someone and they pay you to learn something hot why not take the opportunity as it will help with the resume.
As a small business no one pays you to learn.
What is your monthly AWS spend?
Does the Google speech-to-text API work well for your needs? How often do you use it?
I'm still not exactly certain where I fall on this topic. What I do know is that I don't like frameworks that conceal the plumbing and leave you with declarative bits that require memorization. Lack of discoverability is a huge barrier to mastery. The best code invites you to determine why something is happening.
What I've been thinking about lately that this article drew out for me again was that it's a shame that so many tools and libraries come not ready for production by default.
It would be nice to have some curated base file sets where insecure defaults were overridden, and all of the metrics and logging logic were wired up. You get the turn-key solution but you still have code that is more discoverable, and you can veto a few choices without too much effort.
What AWS EC2 instance types are you using? Can you please expand about your AWS stack?
Also doesn't mean that something quite so simple will work in every case. Not everyone is wasting their employer's money to build their resume.
Sharing is caring, be secure not paranoid :)
Remember it is often easier to penetrate around the gates where everyone is looking (some app stack + OWASP top 10) to instead focus getting inside a network (your dev's laptop, vpn connection, social engineered access, malware to your CEO or sales team, Wi-Fi connection to intercept the VPN tunnels, etc). Or i'm looking for holes in Docker versions to root, Kubernetes flaws, virtual machine dependencies, how many microservices layers do I have to deal with, etc etc.