Hacker News Reader

Comment by devduderino | Hacker News Reader

I care because it usually goes like this: Product manager > "Niche sass app {x} will never need to support more than 10-20 users"

Two weeks after launch > "We have 10k users and counting, why didn't you architect this for scale?"

Always assume you underestimated the scope of the project.

View on HN

Interesting. It might be a by-product of having worked in larger companies, but my experience has pretty much always been: Product manager > "We need this to work for 10k r/s at launch". Actual launch throughput (and all time peak): 100 r/s

Now I always mostly just assume (capacity asked / 10) is more than enough!

As an employee of a large company, I can relate to this experience. The manager has grandiose views about the product, but has no data to back his claims.

I've literally been waffling back and forth between UUID and BigInt for the last week. The reason for this is I need to handle eventually distributed systems. Do I need UUIDs? Maybe not, but after much debt, I've decided that the storage requirements of such an index in memory is worth the ability to move from Postgres to Postgresql-xl.

> Do I need UUIDs? Maybe not, but after much debt, I've decided that the storage requirements of such an index in memory is worth the ability to move from Postgres to Postgresql-xl.

There is a major benefit to UUIDs besides server-side scaling concerns: you can generate them offline in mobile apps and other client-side applications.

With serial primary keys, you are often stuck between waiting for a server round trip before saving data locally, or inventing complicated solutions for referring to not-yet-uploaded data locally.

The problem is not with memory problem is that after checkpoint postgres will do full page writes for serial you will be touching way fewer pages compared to UUID so write amplification for UUID vs bigint can be 60X+

Can you go into a bit more detail? I've seen amplification referenced before. My estimation is that the database will be pretty much read heavy. I will need to split a given database across multiple instances.

https://blog.2ndquadrant.com/on-the-impact-of-full-page-writ... this is fairly detailed analysis of this issue

Thank you. I'm reading over it now and for the next few days (to really grok it). I do wonder how bad it will be for low write situations. At the same time I'm open to switching to BigInt and using Postgresql-xl.

How about simply using non-random UUIDs?

That would def. help

luid, check it out

Ok what can you write it in that you can't scale to 10K users by just adding web servers? The hard to scale part is storage layer but even crappy options like RDS will happily support 10K users.

Microsoft Access.

Works fine for something one team uses, works... not fine for something the entire corporation uses.

Well there are tools to help easily migrate Access to SQL Server, but this is moving fairly far from the original premise of this topic.

Does Access actually work well for 20 users?

Last time I built something in Access was on Access 1997. I believe it had about 100 concurrent users and over a gig of data. Those both exceed the hard limits of access at the time so I had a single read/write database on a share and multiple read only databases on shares. I had to split tables among multiple database files as well.

My direct boss was the CFO and he was cheap as hell. It's amazing the things we can come up with when we have to.

Anecdotally it doesn't: had to use a time tracking program that used an Access database on a shared network folder and would frequently freeze up or crash due to locking issues (with around that many users). I imagine it could be done better though.

Yep

> but even crappy options like RDS

Curious as to why you see RDS as a 'crappy' option??

Given the choice of building and maintaining a MySQL box in the corner of your bedroom and hosting it on RDS - I know which way I would go (and I have done both over the years).

If you are talking 'scaling' in terms of the OP's article, then RDS is almost a no brainer, and you can scale your instances (and add replicated instances etc.), hide it behind a VPN, set up firewalls to prevent DDoS attacks all in a matter of minutes, with a few mouse clicks.

The highest IOPS you can get on RDS is about a single consumer grade SSD.

Fair enough. But if the alternative is setting up a consumer grade PC on a consumer grade internet connection in a (non environmentally or power controlled) corner of a room at home, then surely we are looking at a similar situation?

Given that spooling up a RDS instance is quicker and cheaper than buying all the hardware and spending time installing and configuring an SQL server, then if time to market is a critical factor, you would choose RDS over a home grown solution, wouldn't you?

[Edit: I just realised that this is actually the point you were making, and I got sidetracked by my above question]

Have you heard of hosting? No one is running servers in their house. Nobody.

No one hosts MySQL out of their bedroom. C'mon. The alternative is hosting the DB yourself at a hosting provider which is what most do and it works fine.

Everywhere I've worked that tried to architect for massive scales at the beginning has ended up with an architecture that inhibits scaling. Using correct indexes and transactions can go a long way.

Usually the opposite: business thinks it's going to get lots of users, engineers code for such, business way undershoots estimates, stuck with over-engineered app.

I've never heard of a PM saying such a thing.

It's a good things to overestimate in a Big Corp, because the managers always penny pinch and will put the blame on you if things go to shit.

If you're a startup, go the other way.

I don't know how shitty must a system be to not able to handle 10k users with vertical scaling.

Reply on news.ycombinator.com