Alternative clouds are booming as companies seek cheaper access to GPUs
techcrunch.com
techcrunch.com
We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us.
It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation.
More competition in this space can only be a good thing.
$40/month for a machine, doesn't get any more predictable than this.
So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.
Who, in their right mind, goes for a bare metal to AWS, when there are so many decent and time-tested options out there?
I'd argue that startups should have a good reason for not using AWS. The costs for their basic services is not that much compared to the cost of development.
What I started doing is just running my own "cloud" out of my house for personal projects. I have all of the things I need. There's some overhead in terms of maintenance and up-front setup cost in terms of time and equipment, but after that it's pretty smooth sailing.
If it's literally called a dedicated server, why would you suspect otherwise?
Source: I worked in the core EC2 dataplane for a couple years. PEs and leadership there would not be happy with misleading customers. We constantly thought of the customer experience there.
It didn’t mean ”I suspect that what they are getting from Amazon isn’t a dedicated server”.
Rather it meant ”I suspect that they’re getting a dedicated server from somewhere else that isn’t Amazon”.
The original comment was talking about a dedicated machine for personal projects with a fixed cost of $40/month.
Did you mean a dedicated VM or VPS?
(I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.)
(Edit: holy moly those prices are fantastic!)
https://www.hetzner.com/dedicated-rootserver/matrix-ex/
This is ~10x cheaper than the closest AWS option, and without the extra fees.
Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.
You can also evidence of this on their 10-Q :^)
This is pretty heads-to-heads with EC2, in terms of how it works behind the scenes.
It's not just "space in a rack and you deal with the servers yourself and you come to fix them if they break".
These companies know what they're doing.
(Also, at Hetzner you can even rent their network hardware if you want to make custom solutions.
In fact, I have no idea why people go for those. You pay a massive premium for the "privilege" of not owning the infrastructure, and being subject to opaque pricing and outages that are completely beyond your control.
And in terms of actually managing the stuff, now you have to pay staff to manage your cloud things too.
It makes no sense to me.
Not to mention unlimited free 1 Gbit egress/server
There's also middle ground in the form of Digital ocean's kubernetes which runs noticeably cheaper than big tech cloud offerings.
Good ol' bare metal is real nice, but it won't save you from application complexity, security requirements, and so on - you still need to manage it somewhat. If you're not a startup looking for market fit at least.
I recommend shopping at https://lowendbox.com/ and https://lowendtalk.com/categories/offers
Oh you can!
I've got several dedicated servers at OVH. My absolute cheapest one is an "ECO" / Kimsufi (Kimsufi is a company which spun out of OVH then, a few years later, back into OVH) which I pay... 5 EUR / month. 6 EUR / month with VAT (so 6.5 USD per month).
Sure, it's not beefy at that price: an Atom N2800 with 4 GB or RAM but it is a dedicated server with its own IPv4 IP (yup, there can be uses for that).
I mostly use it as a jump host / reverse-ssh-with-a-known-fixed-IP thinggy.
They've got great dedicated servers at very good price and they're not the only ones in that space.
These can be rebooted/reinstalled remotely and they're monitored: OVH shall deal with hardware failure, if any, for you (never had any so far).
My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening.
To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing how wildly volatile the pricing is, and c) starting to reach for services with transparent, fixed pricing
This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or cost analysis and then saying "OK, everyone off the cloud" once they got an insane cloud bill.
But cloud costs can be managed, and for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project. Also, the canonical rationale I see on HN for moving away from the cloud is "I can just rent a box for $X/month". If all you're using the cloud for is a dumb, static set of compute, I agree that you can probably do it cheaper on your own. I know hardly any companies (from small startups to large enterprises) who use the cloud that way.
Yea, this is the line that everyone uses. I've worked at these companies, and the reality just doesn't line up with that. You end up still needing your whole Ops team, they're just building cloud tooling instead of on-prem tooling.
As to whether or not the migration away is happening...major cloud providers are already seeing people leave, and profits starting to contract. It's very early. Like I said in the original post, it took us a long time to wind up the cloud and it's going to take a long time to come back to reality.
Something roughly cloud-shaped will probably always remain..there are some legit use cases especially for companies that have spiky load profiles. I don't mean we're literally going back to running servers out of our IT closets. I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs.
That I can definitely agree with, I just believe that's fully possible to do with cloud (for the most part, though there are certainly some head turners like the recent news that AWS was charging for forbidden attempts on private S3 buckets, which is bonkers) and cloud cost management tools.
https://x.com/michaeldell/status/1780672823167742135?s=46&t=...
1. As the saying goes, talk is cheap. It's one thing to ask "What do you plan to do?" vs. what you actually do. Look at revenue graphs for AWS, Azure and GCP over the past 5-10 years, right up until the end of 2023. They are definitely not shrinking.
2. I'd be more than a bit skeptical of the messenger, given that Dell obviously has a vested interest in telling people they need to buy more servers.
3. Even if you take what the surveyed CIOs say at face value, asking "Are you planning to move some workloads back to private cloud/on-prem from public cloud" is totally consistent with what I said. There was rush of "just put everything on the cloud" without thinking through it strategically. But just because you're pulling back on a few ill-thought-out cloud projects doesn't mean that overall industry-wide public cloud investment isn't still skyrocketing.
Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on your cable or ISP bill, resort fees in hotels, service charges on your restaurant bill, fees on car rentals, fees from your bank when you so much as breathe on your account, and of course sales taxes which for some reason are never listed on the tag in any store.
Additionally, every single business owner I know complains that whatever it is they're selling, (1) it isn't worth the brain damage to sell to customers looking for the lowest price, (2) as long as people comparison shop in a harebrained way, hook pricing (aka up front pricing that looks low and turns out high) is only rational. It's not like they're providing a bad service for the cost.
Yes, many deceptive business practices are rational acts on the part of the businesses. That doesn't mean they should be tolerated.
At small scales, annoying your customers is bad business. You only need to lose a few before it begins to hurt. Customer complaints are more likely to be a consideration in business decisions.
At larger scales, a business can begin to preferentially adopt practices intended to drive away some customers. Perhaps you don't want the "pathological" customers, to borrow one of Patrick McKenzie's terms. You can make more money with less effort by being more selective about your customers.
At extremely large scales, you largely stop thinking about groups of customers altogether. All of your decisions are driven by aggregates -- did all sales go up this quarter, or down? The effort required to do a deep dive into the behaviors and preferences of any individual market segment may not make sense anymore on a quarterly basis. At this scale, you might be able to afford to annoy tens of thousands of customers and still have a very nice graph next quarter.
So, when someone says a business practice shouldn't be "tolerated", that's a perfectly reasonable position, except it doesn't actually work for businesses operating at extremely large scales. It's too difficult for customers to organize a protest in a way that will influence that business's decision-making.
So much business has moved online in the last 20 years, while the US has leveled off at 80% urbanization over the same time period, along with more and more businesses congealing into BigCos, combined with the recent domination of private equity: lots and lots of things are now operating at a scale where customer concerns just aren't a part of the business model anymore.
Coffee shops, fast food, big-box retail, online retail, SaaS, PaaS: all of these can thrive while running on exorbitant pricing and abusive customer policies, because their volume of customers is so large that it's nearly impossible to be so bad that you'll piss off enough customers to impact your decision-making. (Unless you're Sony.)
It's actually very easy (in theory). You (vote for someone who will) ban fraudulent and anti-competitive behavior, sue the offenders and have them pay huge fines. The fact that this very rarely works out is a failure of the political system.
The problem in my mind is that the largest companies have too much efficiencies of scale to compete with on price.
When competition can't undercut on price, it is hard to argue that customers aren't being served by monopoly mega corps.
Chokepoint Capitalism is a pretty okay book that kicks the legs out from under both assumptions.
If people specifically want to break them up to drive down prices, I don't think that will always be successful.
I'm not really sure what you mean. The EU has managed to put together much better consumer protections in a lot of areas. There are large businesses there.
The sales tax rate is different in different places. It costs a different amount to ship to Florida than Alaska and picking it up at the factory is free (even if nobody does). The advertised price is if you have your own modem, renting one from the cable company is more.
None of these are inherently wrong. You should be paying more if you're having it shipped to a remote location with high shipping costs, and the cost of that shouldn't be dumped on every other customer. But it's kind of a loophole if you want the advertised price to be lower than what people are actually going to be paying in practice.
There is no reason you can't show final price including tax on the label in a physical shop. There is no reason why a restaurant should be able to charge a 20% service charge instead of increasing regular prices by 20%. If you are buying a concert ticket or airline ticket the displayed price should include all mandatory fees. They can upsell you on additional services, but they can't suddenly notice in the last checkout step that your price is higher because the website you are using is charging a fee; that fee was known to them at the beginning of the transaction and should have to be disclosed at that point in time at the latest. If you want to go even further you can also dictate that shipping and handling fees are only allowed to include reasonable costs of actual shipping and handling.
All of these are normal common-sense regulations in most first-world countries.
Sure there is. When the price label is affixed by the factory/warehouse then you would have to track where everything is going and be unable to share inventory. Also, if the sales tax rate changes then all the labels become wrong. These would ultimately increase costs for consumers.
Adding sales tax is also not at all misleading because the customer is not going to be surprised by it and there isn't going to be a competing merchant across the street who can avoid charging it.
There are also business customers with their own sales tax ID and they can buy things without paying sales tax when they're being incorporated into a product where they collect the sales tax themselves.
> There is no reason why a restaurant should be able to charge a 20% service charge instead of increasing regular prices by 20%.
This is actually true. A mandatory undisclosed fee is BS. But it doesn't help much, because if they want to do it then they just make it "optional" where the way to avoid it is more of an inconvenience than paying the fee.
> They can upsell you on additional services, but they can't suddenly notice in the last checkout step that your price is higher because the website you are using is charging a fee; that fee was known to them at the beginning of the transaction and should have to be disclosed at that point in time at the latest.
The last step is where you disclose your address. Before that they may not even know which country you're in, much less the city/state, and there are a thousand legitimate reasons to have different prices or fees in different jurisdictions.
> If you want to go even further you can also dictate that shipping and handling fees are only allowed to include reasonable costs of actual shipping and handling.
That doesn't really help, they're typically charging the actual cost. They just don't include it in the advertised price because it makes you inclined to make the purchase online instead of saving $10 by picking it up for the same price but no shipping charge the next time you go to the competing local store.
There's a reason Amazon's major competitive advantage is free two day shipping, derived from having the scale to achieve low shipping costs themselves.
Price tags are pretty much universally handled at the stores. There are some goods where the price tag is attached, but that's more the exception and not the rule.
This is a solvable problem. So much so that if you've traveled in most nations you'll see that all prices include tax. Not including tax is a particularly weird aspect of US culture that simply doesn't exist in other nations, even those with a large amount of imported goods.
Advertising isn't. If the same ad is viewable by people in different states, what price are they supposed to put on it? Shouldn't this be the same price as what they show in the stores, or else people will have trouble knowing if the advertising was deceptive?
> This is a solvable problem. So much so that if you've traveled in most nations you'll see that all prices include tax.
Don't most of those other nations have a uniform tax rate?
Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.
California has a 7.25% sales tax. Oregon has a 0% sales tax. A 7.25% difference is larger than the entire net margin on many products.
> Sales are mostly about moving items, so they are already losing huge margins at that point anyway.
It seems like we've now gone from "there is no reason they can't do this" to "the merchant can just eat the sales tax and go out of business"?
> Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.
Which is basically what they do now. You see the final price at the register before you pay. You also know that there is going to be sales tax, no one is being misled.
Putting a different price on the tags than there is on the ad would just confuse people and make it harder to tell when you're not getting the advertised price because you have to do fractional arithmetic on every soup can and candy bar to see if it matches.
I was referring to city/county taxes, not state taxes. Tor example, a local city to me in the US had a total of 14% sales tax, while outside the city was 10%.
Anyway, a sale’s motivation is to move products off the shelf and make room for more profitable products; they’re already losing money.
California borders with Oregon. Radio waves cross state lines. People see highway billboards and then drive into another state. People use the internet and you don't know where they are.
> Anyway, a sale’s motivation is to move products off the shelf and make room for more profitable products; they’re already losing money.
A sale's purpose is frequently to bring customers into the store to buy other products, or get customers to try a product for the first time to drive repeat business.
There is also plenty of advertising that isn't discounts. Maybe your product is great and you want to inform the customer that they can get something better than the competition for the same price. Maybe you're just publishing a comprehensive price list for all your products.
And in all cases, "losing money" is not a binary result. Losing a small amount, or not making as much as you wanted to, is very different from losing so much that you go out of business.
This statement has always been BS. If you set the tax rate to a percentage of revenue which exceeds the business's margins, all businesses go out of business. "Maybe grocery stores shouldn't be in business" is so absurd that the statement seems designed to deliberately shut down reasoned debate by stunning people into silence or making them so angry they respond rashly.
And in this context it doesn't even apply. The issue here is that customers in Oregon are going to pay a lower price than customers in California because Oregon has lower sales tax and the tax is part of the price they pay. The business can't eat the tax in California so their only alternative would be to advertise the higher price. But if they have to advertise the higher price then many of them will just charge the higher price everywhere so all you're doing is screwing customers in the jurisdiction with lower taxes out of paying lower prices. This also reduces competition by making it harder for customers to compare prices, because a store that only operates in a state with lower taxes would then seem to have lower prices than one that operates in multiple states and has to advertise the highest price, even if it didn't charge you the highest price where you actually are.
> This isn’t a problem in the EU where countries border other countries and sales tax varies from 6-19-24%
Or it is a problem and you're just eating the cost of getting screwed by the law while defending it.
Unless I misunderstand, isn't this EXACTLY what the US does to ensure certain businesses are or aren't profitable in their smaller jurisdictions? This is how they entice businesses to come to their cities/states and they even compete on giving bigger tax breaks. I assume they also do the exact opposite to keep out businesses they don't want.
> The issue here is that customers in Oregon are going to pay a lower price than customers in California because Oregon has lower sales tax and the tax is part of the price they pay.
They're already doing this ... this just makes it so you can do simple addition in your head while you walk around the store instead of throwing in some multiplication -- and if you live in a state where certain kinds of items have different tax rates, knowing what those tax rates are.
This isn't rocket science, I don't understand why you are making it seem so complicated.
Traditionally this is called fines rather than taxes. The ostensible purpose of general sales tax is not to bankrupt every grocery.
> this just makes it so you can do simple addition in your head while you walk around the store instead of throwing in some multiplication
Why do you have to do any math in your head at all? The register will do it for you at the end.
You could as easily say that the marketing department not knowing the features of the product justifies them making whatever claim they want in an advertisement. Why should a company be permitted to advertise something they don't know to be true, let alone something they know to be false?
The cheapest customers are always the most expensive to work with. It’s a sad reality.
I worked at several companies who for some foolish reason saw the cheapo customers as some untapped market and when they raced to the bottom they lost every time.
When it comes to healthcare, they absolutely are. The US spends more than double for the same outcomes as other developed countries.
And no, this isn't because of the cost to develop new and novel drugs (which aren't used in 99.9% of routine health care)
No more junk fees in CA.
- https://www.sfgate.com/food/article/sf-restaurants-junk-fees...
What do you mean by this? How would “people change” to get out of resort fees or confusing pricing systems?
People are capable of change.
Pay +10% for “priority” rides in Lyft, supposed to arrive in 1-5 minutes, whereas “regular” is 7-15. Car shows up in 18 minutes. Priority payment does not get refunded.
Honestly what’s even the point in caring anymore. Living in America is about getting grifted until you can hopefully figure out your own grift. The irony of posting this thought on this website is intentional.
The American government spends something approaching a trillion dollars annually on Medicaid, "a government program that provides health insurance for adults and children with limited income and resources."[1]
Seperately, Obamacare[2] created a private health insurance market where low income people can obtain free or heavily discounted private health insurance coverage, according to their income.
[1] https://en.wikipedia.org/wiki/Medicaid [2] https://en.wikipedia.org/wiki/Affordable_Care_Act
I'm a healthy individual, not even 30 yet, never smoked, never broken a bone, and never even had stitches in my life. And when I last checked their prices, I was making only $45k/yr. For coverage with Obamacare, I would be paying $350/month for the "catastrophic" plan. Which includes no prescription copay, no dental, no vision, and only kicks in after I've spent $100,000 in one year, and it takes 6 months to take effect after signing up. It's only there for serious issues like losing a leg or cancer.
I've bought my insurance on the ACA marketplace since it opened ten years ago. There's no difference I can see between the plans offered on the health insurance marketplace and those offered directly from the websites of the same insurers that offer coverage in my state (Blue Cross, CVS Aetna, United, Ambetter, etc).
The very highest deductible "catastrophic" plan offered on the marketplace in California for a 29 year old has a $9450/year deductible, which is also the maximum out-of-pocket expense for the year if you have this plan. A $100K deductible plan does not exist, and when you enroll during the annual enrollment period or after a qualifying life event, plans take effect the day you make your first payment, not months later.
I'm 10 years your senior and pay less than $350 per month with a lower deductible than the plan quoted above, with no government subsidies.
I'm also in Georgia which may have different regulations regarding the deductibles. I remember looking at the plans around 2 years ago and realizing that there was no way I could afford the premium, let alone the yearly deductible.
I just took another quick look at a non-Healthcare.gov site. Insurance for me would be $313/month with a $9,100 deductible. But it does not cover doctor visits, generic drugs, or specialist visits until after I pay the full $9,100.
Why would I want to pay $313/month for essentially no coverage until I spend 20% of my income towards a deductible before I see any benefits?
I'm on healthcare.gov looking at Georgia's plans this year for someone with $45K of income. You have options starting at $129/month. Many of these sub-$200 plans get you doctors visits for $40-60, prescription drugs for under $25 each, mental health treatment for under $60 per visit. This is all without hitting your deductible at all, they're day 1 prices.
If you paid the cash prices for many of these doctors, specialists, therapists, they'd be many times higher than the insurance negotiated costs. Look under the "covered costs" estimates for things like mental health treatment, diabetes maintenance, broken bone treatment, etc and you'll see that the estimated annual cost for the insured is often half or less the plan's deductible -- which tells you that hitting the deductible is not when the savings start. I don't think my wife or I have ever hit our out-of-pocket maximums in a year, yet the insurance has saved us more than it's cost in most years.
You're going to start interacting with the healthcare system a lot more than you have in your 20s once you're in your 30s. We all do at that age. And if you have even the worst ACA plan, you'll start to understand what it's doing for you regardless of the deductible.
(You also get their negotiated rates when you go to the doctor, I assume.)
https://d3ul0st9g52g6o.cloudfront.net/2024/GA/sbc/2024_58081...
Why perpetuate crude and inaccurate stereotypes that smear and disparage America in general? America is spending a huge amount of money every year precisely on providing healthcare for the poor.
Besides which, it looks like Medicaid does indeed operate in Texas, which is discoverable in less than 10 seconds of Internet searching: https://www.hhs.texas.gov/services/health/medicaid-chip/abou...
And, seperately, Obamacare's marketplace works nationally, including in Texas: "More Texans than ever before enrolled in ACA health plans in 2024"[1]
[1] https://www.texastribune.org/2024/01/24/texas-aca-health-ins...
From your own article (I knew this stat would be in there):
> Currently, Texas leads the nation in the number of uninsured residents with nearly 5 million people living here without health insurance coverage, nearly double the national average.
Why do you think so many are uninsured if it is so simple for them to get health coverage?
That is categorically false, and testifes to some sort of deep seated bias against our own country profoundly embedded in your worldview -- one which extends all the way to glib counterfactual promotion of verifiably false information about the country and its supposed moral shortcomings.
2) Texas has a much larger population than 49 of the other 50 states -- about 30 million, higher than any state but California. Of course the number of uninsured residents will be higher than the national average. You didn't control for uninsured per capita.
Why are so many uninsured if it's possible (I didn't say easy) for them to get coverage? Probably, a large part of the reason is people who go around promoting the (completely false) common trope that "poor Americans just can't get health coverage, the government does nothing for them."
As to point 2) I thought about including the info in my previous comment, but it wasn't in your own source material and I'm a bit lazy. So here, I'll do it in this one.
> Texas is still the state with the highest percentage of uninsured residents, at nearly 17 percent, according to the most recent U.S. Census Bureau survey released Thursday. [0]
[0]https://www.texastribune.org/2023/09/14/census-bureau-texas-...
Speaking as a Texan (also pretty sure I'm American) who lives with some disability, I can tell you that I've looked at the systems and it's a real fear of mine that I'll end up as one of these statistics.
I’m pretty sure there is a moral to this story..
High numbers of people who have illegally immigrated and are worried about getting into programs subsidized by the government which might result in questions about residency status leading to deportation?
High rates of misinformation surrounding the costs of insurance and availability of welfare programs in the state?
I just looked up getting a plan in Texas. 40 year old male non-smoker in Texas earning $45k/yr can get insurance for $128/mo. They're eligible for an HSA, so they can put tax-free savings that roll over every year into an investment account to help cover the $7,400 deductible. PCP and preventative care visits are free. Generic drugs are $10. Urgent care out of pocket is $160. Other plans have slightly higher premiums but much lower deductibles, some have different co-pays.
Where Medicaid is expanded, the income requirements for Medicaid are far from sane. If you make over 100% to 138% of the federal poverty line, which is $15,060/year for an individual, you are not eligible for coverage. For example, someone who makes $16k to $21k a year, depending on where they live, is ineligible for coverage despite making poverty wages.
[1] https://www.kff.org/affordable-care-act/issue-brief/status-o...
What’s funny is I’m probably one of the people whose paycheck is dependent on me not understanding that to paraphrase Upton Sinclair. When I started at my current employer, one of the devs straight up said the government should be doing what we do and we shouldn’t have a business.
Hospitals are obligated to provide care regardless of your ability to pay in the moment, and of course they often take care of indigent and foreign patients, with the understanding that they won't be able to recoup their costs.
If you don't have your identification on you, then they simply give you a number to call and they usually ask to respond within a few business days because after that, it's harder to get insurance approval.
This all changes if you're admitted to the hospital and need surgery, because I think there's pre-approval required from your insurance carrier. Though since I worked in the ER, my memory is hazy on that.
If they believe you were deceptive and simply refused to be identified, then that is technically illegal, so they will put up your picture and will alert the police if they see you again. But if you never intend to visit the hospital again, it doesn't really matter. No one is going to hunt you down unless you're doing it on such a large scale that it can't be ignored.
This all said, I want to say that I still think it's an unethical thing to do.
They ignored all the threats, the department in charge of threatening seemingly didn’t talk to the department in charge of graduation, and to this day (5 years later) they still receive near-daily letters in the mail requesting payment. I can’t say I find my friend’s actions unethical in the slightest.
Ah! The UC system…
Aren't inmates technically supposed to get healthcare? Could be a last-ditch option I suppose if you can't get something covered any other way. Personally I'm trying to stay employed to keep my coverage, but sometimes you can't outrun the layoffs.
But I think we need to up our game in this cat and mouse game a bit. For example, in aggregators -- like Expedia or Google Flights, etc -- why not try to capture some of these fees in the price? I can search for hotels with parking but what about sorting hotels by price and including the parking price? It's hard to compare when I see a $120 hotel that has a $50 valet vs a $150 hotel that includes parking. But that's the thing I'm actually after a lot of times. Or similar with flights and baggage fees. We should be able to collect a lot of this type of information and properly present it to the users and try to make these types of dark patterns ineffective (still will be cat and mouse and this is only a specific type of pattern, but still, I think there are things we can do)
No wonder Americans have no idea how the economy is doing.
and then pay % of your refund as a fee to CPA, so they help you maximize your refund.
It’s a real shame banks received 2-5% of most transaction for what costs them pennies. Sure, there are benefits, but their ask isn’t covering it.
I think lots of people go along with these as-you-go services because they’d rather deal with an unexpected bill, than having their servers shut off.
This should apply to everything -- cloud compute, healthcare, phone bills, internet services, everything.
If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront.
F that, if I paid for insurance I shouldn't have to pay anything out of pocket
> If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront.
F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should be legally mandated to be within 5% of the actual cost.
That's not how insurance works, bud. For one, you likely don't pay for insurance, its heavily subsidized by your employer, and they will determine which policies to offer you. For two, you share the risk with everyone else who has a policy with your provider.
I could see an argument for insurance companies being legislated to force them to cover more previously uncovered services, as the ACA did.
> F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should be legally mandated to be within 5% of the actual cost.
Sure, you are welcome to contact your insurance BEFORE you obtain services and find out what they will cover. In fact, its incumbent upon you to do that, and not expect HCP's to do that for you.
Had experience with that too. Insurance company will not give you a straight answer as to what’s covered or not- as for example the anesthesiologist may be out of network and you’ll never know until it’s too late. The health care provider won’t give you that info either. So you end up rolling the dice and hoping it all is covered at the end of the day.
Are you sure you don't make this up? Any insurance that I've ever had always let you search if a particular provider is in their network. I just verified on the website of my current insurance. Dozen of anesthesiologists nearby are listed. In fact they often go out of their way to notify you if your doctors leave their network.
See for example https://www.reddit.com/r/personalfinance/comments/ho49d5/ane... - first hit of many for “anesthesiologist out of network”
Especially if you have an emergency surgery it’s not like you’re picking your anesthesiologist from an approved list before entering the OR, even if you took the time to make sure the hospital you visited is in network.
Finally look at the fine print for your insurance company’s website. You’ll see multiple disclaimers noting that the information may not be accurate - for example from blue cross blue shield:
> Blue Cross and Blue Shield Licensees have made reasonable efforts to ensure that the list of providers displayed is up to date and accurate. Please call the provider before making an appointment to verify that the provider continues to be part of the network.
Then when you call the provider to confirm, they shunt you back to your insurance provider. It’s hilarious if it didn’t have significant financial consequences.
Oddly enough, I had the exact opposite experience recently, doc's billing dept said specific medical appliance would not be covered so I had to pay out of pocket, only to find out later it was reimbursed by insurance and I got refunded for my out of pocket cost.
That is truly funny. I have literally spent an hour across multiple agents to quote a single service out of pocket. I’ve done it multiple times across several providers. Every time it’s the same painful procedure where nobody can ever give you an answer.
I struggled to find the words to describe the way you’re treated when you ask for a quote. This is the best I’ve come up with: It’s like shopping at a luxury store with no price tags- if you dare ask the question “how much does this cost?” perhaps you don’t belong here.
Maybe I'm naive, but I don't think that's intentional. I think it's just a byproduct of trying to build something that works for everyone and every use case.
As you make your target market bigger and bigger, you continually hit edge case after edge case that you try and solve with "just one more" rule or option. Eventually the system becomes so complex that no layman can understand it.
The whales on AWS aren't overspending because AWS uses the granularity of their billing to make sure they aren't. That's how they keep the whales happy.
For the little guys, I'm not even sure it's worth AWS's time to nickel and dime them. AWS hands out thousands to ten of thousands of dollars in credits like candy. I'm pretty sure it's more important to them to lock in whales when they're still minnows than to bilk an extra $500 a year out of a 10 person start up.
AWS has one of the nicer to comprehend billing systems.
I don't think that's AWS explicit business model, but I think they're perfectly fine with it happening.
Complicated cost calculations are only a part of the issue. You (or your team) also have fault in that you did not take the time to understand the costs associated with your decisions and not utilize AWS cost management capabilities (that is assuming you did get a surprise bill in lieu of an alert saying you hit a budget threshold).
And that is in part due to the shift from having dedicated ops teams, to having programmers take on more infrastructure tasks. This isn't unique or novel--incorrectly configured buckets, committing access keys, poor IAM setup, etc happen so frequently due to devs who have no real practical experience managing production infrastructure having unchecked access to AWS.
This is patently not true. AWS is very transparent with the pricing and offers a cost calculator. Specifically for EFS, you could have used the EFS service cost calculator before setting off with your endeavours:
https://calculator.aws/#/createCalculator/EFS
Other services are available at https://calculator.aws/
Any work on any cloud platform has an imperative step: the cost analysis. Once constituents of the solution are finalised (step 1), it is compulsory to proceed to step 2: estimate the charges it is going to cost and decide whether it is affordable or not. Skipping the step 2 and blaming the cloud platform provider business model is not fair and is akin to overspending on a credit card and blaming the credit card issuer for allowing you to do it.
Cloud is not dissimilar from UNIX/Linux – both give one a large variety of elaborate footguns to shoot oneself with, and if they worked as root on a software development project on a UNIX/Linux box and accidentally wiped the root file system out, who is to blame? Since cloud charges are usage based («pay as you go»), the cost estimate is a requisite that can't be avoided unless the budget is unlimited.
2 vCPU + 8 GB memory: $69.18
10 GB Hyperdisk Balanced: $0.80
3060 provisioned IOPS: $15.30
155 MB/s provisioned throughput: $6.20
Total: $91.48
But just look at the SSD pricing. That's a ten gigabyte SSD provisioned for 155MB/s, for $22/month. You can go just outright buy a 256gb NVME at significantly higher bandwidth on Amazon for like $25, flat. The n4 tier instances removed the ability to use their cheaper general-purpose SSDs; you have to use hyperdisks.
I'd be surprised if we don't see the big cloud providers start struggling over the next ten years. I think they engineered planetary-scale systems that are just way too expensive and complex to justify the cost they're charging; ZIRP phenomena.
The t2d, t2a and h3 instance types have vCPU = core, and all other instance types have vCPU = thread.
But, to be clear: We're not.
Then if you are going to have remember that cloud networking is pretty beefy and if you want k3s to do distributed storage you will need some pretty beefy network hardware.
There are a lot of things hidden in the cloud costs that people forget about.
The one thing running your own stuff does allow you to do is make choices and trade offs. If this switch goes down and we have 6 hours of downtime to replace it what is that worth etc.
For instance, the cost for a pair of redundant symmetric gigabit fiber is in the thousands a month and may require tens of thousands of construction costs. These quickly add up, and the upfront costs can quickly reach six figures.
You can always save by going on-prem, assuming you have no uptime requirements. But the moment you sign an SLA, those savings go out the window.
For that matter, have you checked the price, in qty 1, of a server that will absolutely destroy anything reasonable from a major cloud vendor in terms of IOPS to stick behind that switch or router? Even if you believe the numbers on the website of a major server vendor and forget to ask for a discount, it’s still quite reasonable in comparison to a major cloud.
I have no doubt there are great use case for cloud, and that at the proper scale you can negotiate, and I understand that startups might be faster moving with the cloud. But I feel like the highest value cloud provides is 1. replacing capex with opex, and 2. making scaling easier with the direction: cloud is pay first, get questions from direction later. "on-prem" is negotiate with the direction until the service degrades and then scramble to integrate the new server under the pressure.
Don't discount the value-add of your skills. Giving any of my family members a stack of bare metal servers would serve no end-purpose or web requests. Approximately zero of said servers would even end up plugged in at all, much less operationalized. My family all works in tech or similarly demanding fields. That is, the cost of "the ops team / guy" can be significant for small and medium enterprises.
Despite this, the AWS pricing margins have grown to the point of excess, and are no longer competitive, even against other major cloud players. This is strange to me because the tooling is all interoperable (e.g., Terraform/Vaggi-form), with no fatal friction or lock-in. It is a rare use-case that the latest CPU is worth paying a penny extra for compared to a 3 or 4 generation old chip. The only winner is the cloud provider because it improves their COGS ratio.
TL;DR: the top tier clouds are priced on the order of luxury goods, on par with a Bugatti or G6 jet. That is, unless you ruthlessly track and prune each expense, which costs you time and attention which could have been spent growing the business or at least non-overhead tasks. The eventuality of managing your own DIY fleet of machines is a total headache, as any given computer may work flawlessly for the next ten years or only the next 10 minutes. When it goes south, you're back to being a monkey plugging in cables and scratching your head. This sad activity is only a few people's cup of tea.
Obligatory reference to Warren G's Regulators:
"You've gotta be handy with the steel, if you know what I mean."
https://youtube.com/watch?v=hms5vmekId4
Edit: Sorry for the rambling comment, I suppose it is a more complex topic than I realized prior to crafting the words above.
That is if you rent rack space. If you rent bare metal and the bare metal has a failing disk, you file a request to replace said disk, and the data center engineer will be plugging cables, not you.
That means you have to worry about hardware failures, but you don't have the inconvenience of having to be physically present.
They’re already struggling!
In the past, the three big clouds would deploy cutting-edge CPUs at scale ahead of general availability for ordinary rackmount servers.
Now?
The AMD EPYC 9004 series processors were announced over a year ago in March 2023, but are still trickling out as “preview” in selected regions in Azure. Similarly, Intel Xeon fourth-gen CPUs haven’t even been announced by Azure, but Intel is already shipping fifth-generation CPUs!
I suspect that up until a couple of years ago, the usage of public cloud was increasing at such a pace that the providers were buying a truckload of CPUs every six months, so they were keeping up with the latest tech.
They must have had new signups dry up as soon as interest rates went up, and they’re now milking their existing kit instead of expanding with new generation servers.
"On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month."
Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~720 hours in a month (roughly), and that means CoreWeave would cost $1,720.80 per month, Azure is $2,448 per month, and Google Cloud is $2,642.40 per month.
Why are all of these numbers reported in the article off? Some slightly--Azure and Google Cloud are close, but CoreWeave is off by about 30%. I won't go further into the numbers as to why the author came up with these results, but I'm just wondering if this article was written by AI, which would explain why basic multiplication is incorrect.
That's $1483.20 a month, whereas the article says $1200 and should say $1720 if they'd got the maths right.
I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.
Only problem might be everytime it spins up to download the large model might be wasteful as far as getting charged for network/bw usage— wonder if it would be more cost-efficient to have persistent storage or just see how much time and bw it is to download on every cold start…
As a small tech startup, an easy button for compute needs is perfectly sensible, as it allows focusing on the product. If the startup grows switching becomes more expensive, so AWS gets its money as long as the amounts are not seen as the main cost driver. That stage, I think (with no hard data), is the AWS sweetspot. The company is paying a lot for AWS, but does not yet want to do a full analysis, hire dedicated cloud cost optimization staff and deal with friction of switching.
If the startup grows stable and profitable it will likely do a proper cost analysis and make AWS bills saner, maybe with a mix of on-prem, AWS and non-AWS cloud services. But that requires a stable period, both in time and in functionality, which is not something that an unprofitable startup has.
I think with the end of ZIRP and tighter access to VC funds the number of startups that can afford losing a lot of money to go through an explosive growth period will shrink, and so will the AWS profits.
If you're doing a lot of model training, buying GPUs or long-term reservations of GPUs is a no-brainer. But when it comes to inference, latency matters and it gets trickier talking between e.g. your AWS infra and your GPUs somewhere else.
It seems lots of providers can give you enough to get by doing inference in a company's earliest stages. But what if I need hundreds or thousands of A100s during peak usage? Is anyone doing this successfully with a non-hyperscaler?
I understand that's exactly what those provider _don't_ want, because it means they can't lock us in. But particularly when comparing an inference provider to GCP, where we already run everything in Triton on GKE, I don't want to rewrite my code just to see what their hardware layer is like.
Another complication is we often run multiple tightly-integrated models for a single application, where having them on the same GPU is critical. This is tricky or impossible in some inference-provider-frameworks.
There's too many options for running the latest LLM, and far fewer for running a bespoke set of fine-tuned models on GPUs.
I mean locking customers long term is probably better, but a certain % of product still vould be sold for a spot price? This made me wonder what is the optimum split on that - obviously depends on your market placement, but isnt nvidia basically a monopoly now?
Have you seen their pricing? They are robbing everyone and selling out quickly. I don't think "auctions" are going to add much to thee bottom line.
Nvidia effectively created this AI world by building tooling and software for something we didn't know existed up until not too long ago. AI was created on the shoulders of the work that was done.
AMD wasn't taking the risk on the unknown, for whatever reason.
Water under the bridge. The ship is certainly busy course correcting now. Give it some time.
I'm sure people said that too and look at where we are at today.
Core performance has always been a thing.
Upstarts are not easily going to be able to pursue that, so NVidia has a strong interest in supporting them.
It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.
My understanding is that your reserved cloud starts at 64 GPUs (8 pods) and with quite a long reservation time frame.
Also we expect to have a bunch of capacity come online of the on-demand cloud this year. We're getting the GPUs as quickly as we can and racking them as quickly as possible, but we have to wait just like everyone else for the GPUs to roll off the fab. :)
The bit that freaked me out was 2 customers were 60% of revenue.
Those two customers can change your policies in ways you don't want to, because you can't afford to piss them off. And if something happens to one of them you're fucked.
This doesn't make sense for training models, where a training run is on the scale of days & weeks.
I wished that the techcrunch article mentioned other companies, like sfcompute, which offer hourly compute instead of yearly contracts.
The price is also $2.49/hr which does not seem predatory at all.
They are providing H100s, A100s, L40s for very cheap. They also do not charge for the network usage. I Highly recommend them as the price per flop is unbeatable anyplace, and they have over 4000 gpus to use at a time.
It has been amazing to watch this industry explode, and we believe it is great for consumers. The same instances on Amazon versus these alternative providers are 3x more expensive.
NVIDIA and many hardware providers are leaning into this trend. As clouds become more and more vertically integrated, AMD, NVIDIA, and others will benefit from spreading their hardware to more clouds.
Knowing that these models will not be running in 3 easily controlled clouds may also benefit us in the long run as each provider will have different levels of comfort with models of varying capabilities.
That might be the plan, but I doubt it's going to work for GPUs like it did for AWS.
There's no moat here, other than the hardware. There's no value-add that the provider can add for free and lock you in.
Couldn't you have said that about cloud services like AWS before they existed, that they can rent servers to people but there won't be any lock in?
And equally, just because that's the case with GPU providers right now, is there a reason one or more of them wouldn't be able to develop software which runs on the GPUs to do some of the things people are currently renting GPUs for, or middle-ground software which make it easier to do other stuff on them than just renting plain GPU access, and turning it into optional services (with lock in) on top of the raw hardware rental?
As an example idea: it's good that NVIDIA decided to let as many people use CUDA as possible (and I suspect if they hadn't then they wouldn't have seen nearly as much success), but if they or anyone else releases an equivalent to "CUDA v2" tomorrow, but instead of allowing anyone to download it instead put it behind a billing page with AWS-style pricing that covers both software and GPU, would it not succeed if the software did make thinks easier for people just like AWS does (in some ways)?
edit: I just realised I ignored the "for free" bit of your comment - but it wasn't free for Amazon or Google to build their cloud software either.
It doesn't seem to really be the case for the big cloud providers, either, as most of what they offer is fairly commoditized -- I've been using GCP for years to host Kubernetes clusters, Postgres databases, Redis instances, standard Linux VMs, etc., and while there'd be a cost associated with migrating out of GCP, I don't feel locked into it in any way, as the tools I'm using there are the same tools I'd be using regardless of where they were hosted.
The market for IaaS and for SaaS are very different from each other, and the walled-garden approach doesn't seem to have significant traction in the IaaS world, thankfully.
https://github.com/skypilot-org/skypilot
Open source CLI to deploy multiple gpu vm’s on all major cloud providers, with an option to use spot pricing with 1 cheap vm used as a controller to always make sure you have the most inexpensive deployment available with failover and load balancing.
It’s like beating the cloud providers at their own game I wouldn’t be surprised if they banned it.
So my suggestion.. dump all your work @ Core-weave.. It's cheaper than buying the hardware yourself let alone the cost of managing it.
They've been running for years already. They can also offer these lower end gpus at that price cause the higher end ones offset things.
And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections.
If one of these clouds offered fair pricing to SMBs, it could be a great bottoms up growth strategy.
(*) Not LambdaLabs afaik, but they rarely have on demand capacity anyway, and you can only get reasonable price with 3 year reservation (which is, surprise surprise, more than the hardware cost).
Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute, Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute.
As far as I can tell none of these providers give you a GPU originating in Azure, AWS, or GCP.
If you're reserving thousands of GPUs from the same hyperscaler, even if they're the only cloud you run on, you're not paying the price shown in the calculator. If you have other suppliers, you'll get an even better deal. Then you resell that reserved compute as on-demand compute, somewhere between your costs and what your customers would pay a hyperscaler directly.
Interesting. GPU-only providers targeting the AI market only need to implement a fraction of the services that AWS does. They don't even need to be geographically distributed. What does it matter if your GPU cluster is on the other side of the planet?
Pretty soon, these specialists will build object storage and all of the other "costs" that the legacy hyperscalers already incur.
Then, to migrate away you need all sorts of devops folks and the ability to deal with incompatibilities.
Uncertainty about pricing and the hardware bottleneck is a real problem for our users.
I just raised this point in our blog today.
Let me know how your 400G deployment goes with your vendors VRF implementation.
Nvidia love OCI I thought I read