Hacker News Reader

Comment by _xnmw | Hacker News Reader

AWS is complexity-as-a-service which abstracts away so much that you pay a HUGE price when it's time to scale -- and it's a confusing matrix of tradeoffs. I find it easier to login to a server, "sudo apt-get install" and tail some logs, than to try to manage the monstrosity that is AWS. Products that go horizontal-cloud-first are easy to scale, but burn an order of magnitude more money, with far worse realtime performance. At the end of the day, AWS is just an abstraction on top of CPU, RAM and disk -- if your app makes bad use of underlying CPU, RAM and disk, then no amount of cloud magic is going to fix that, it'll just mask the problem and you'll end up paying an arm and a leg. AWS/GCloud are actually incentivized to make it hard to debug and optimize performance, they would want you to waste resources on their platform.

View on HN

Exactly, at this decade it's still a ripoff = huge margin for them, but market is there, one can't deny that. It'll even out eventually so they're on par with smaller shops and dedicated/colocation, eventually they'll have no choice but to cut margins and leverage scale to kick out smaller competition, they'll delay it, of course, as much as possible with lock-in dark techniques etc – but at the end there is no reason, in theory, why they should not be more attractive price wise with anything else you can come up with. At this decade-tick timeline they're happy taking huge margins and people just throw money without much of first principle etc thinking.

No:

- They’ll leverage GDPR to make it only possible to comply to all privacy standards (SOC, credit card management…) if you use AWS,

- Worse, they’ll provide special Linux repos with vulnerability fixes that only they provide. Log4j can make all your distributions obsolete in a single day, and only Amazon Linux users will have that patch, this quickly.

- Then insurances will stop covering people who deploy their software manually, because vulnerabilities are too prone to happen.

In the long run and for most higher global nationwide scale AWS/GCP will be cheaper from 1) general reduced price every few quarters/years 2)private contract. It’s important to realize those cloud providers does a whole bunch of work to reduce cost internally so customers can pay less (eventually)

AWS and competing clouds are a non-starter for anything resource intensive such as processing lots of data (needs tons of CPU/GPU) or serving/moving lots of it (needs tons of bandwidth).

The costs are bearable for early-stage startups that are mostly typical web applications with little resource requirements, or bigger startups that have so much VC money they are happy to burn it, but beyond these two scenarios clouds are very bad value for money.

Generally, application level decisions will have a bigger influence on cost than underlying compute storage for large data processing applications.

Tradeoffs that will dominate the costs

1. Are you using a fast language or a slow language? (10-100x cost difference)

2. Are you using an efficient storage format such as parquet? (10-100x cost difference)

3. Are you using a reasonably efficient query planner job processor? (1-10x cost difference).

4. Are you using a good algorithm for the data processing task? meaning a good efficient SQL, An efficient imperitive algorithm etc. (unbounded cost difference)

The above tradeoffs will have a cost difference of up to 10^5 ( or greater depending on point 4). Once you account for risk, utilization, capex, and people costs. The cost difference between different compute platforms is usually negligible to the above points.

What about the cost of optimizing the process? If compute can be obtained very cheap elsewhere, it may be cheaper to just use that with an inefficient process rather than spend time/money optimizing the process and then running it on expensive compute.

Depends,

Cloud providers cut compute/storage costs every few years. You can get long term contracts with them that have heavy discounts for guaranteed spend. Switching compute instance types is pretty cheap, and switching compute platforms is relatively low cost when the time comes. Meaning that investors can generally expect your compute costs to fall YoY all else being equal.

Building out your own compute generally means adding new headcount to deal with colo things such as vender management, technical details such as networking, capes/opex management, and finally hardware selection. The odds that the hardware you select perfectly matches the compute you need in 2 years is low. The odds that a cloud provider has made a cheaper compute option in 2 years are moderate. The odds that your app needs something that’s available trivially in a cloud provider are high.

Reply on news.ycombinator.com