Hacker News Reader

Comment by CodesInChaos | Hacker News Reader

Sparse block storage is one of the cloud features I miss most. What I'd expect:

* You can create a volume, but pay only for in-use blocks.

* You create a volume bigger than what you'll ever need => no resizing required (At least 16 TiB, but possibly even something crazy like an exabyte)

* There is some kind of "trim" operation that marks a block as free

* Ideally it's possible to choose the maximum block index (size of block-volume) independently from the maximum number of in-use blocks (for cost control)

View on HN

Sparse block storage is achievable with s3backer and similar tools which treat bucket/object storage as arbitrarily large volumes.

Practically, sizing thin-provisioned filesystems too large wastes a lot of space on filesystem metadata.

All operating systems and clouds that I've worked with support online volume and filesystem resizing which is as close to the semantics of thin-provisioning as I've ever needed. I wasn't actually aware that StatefulSets don't resize volumes as smoothly as desired but I haven't had a problem manually deleting pods to get the filesystem on a PVC to grow (the way it works for Deployments).

Great feature, but cloud providers are incentivised to make it difficult for people to save money. This would instantly vaporise a big chunk of revenue, because people would not have to pay for pre-provisioned storage.

Maybe one of the lower tier providers should use it as a differentiator.

Why add a low margin feature to your product/service?

Because it draws in customers that will then also use your higher margin features.

Amazon historically has gone after 'low margin' businesses. The cloud, at 25% margins (for AWS) is is 'low margin' compared to traditional software sales.

> Great feature, but cloud providers are incentivised to make it difficult for people to save money.

Every business is incentivized this way. Competition is what pushes prices down, of which there is plenty in the cloud industry (though maybe still not as much as we would like).

AWS EBS volumes used by a PVC can be expanded without downtime by increasing the request on the PVC. The only prerequisite is that the StorageClass has allowVolumeExpansion=true

Edit: Now that I actually read TFA I see it mentions most CSI drivers already provide this functionality. They provide a workaround similar to what I’m sure most people use, and I agree this functionality seems like it could be handled by StatefulSet.

Yep, the cloud long term is a terrible investment. Due to its sticky nature innovation in technology is seldom passed down to consumers and instead is pocketed by the provider.

I guarantee you internally cloud providers over provision their storage.

AWS has elastic filesystem, but the perf is crap from what I understand.

I haven't used it, so your mileage is a total unknown, but our "big data" folks like it.

AWS won't change to your desired model, they mint money on underutilized resources (and hidden I/O charges/network)

It's not at the block level, but Amazon's EFS checks most of these boxes.

i read a story on HN recently about someone having left a `nohup yes > /some/efs/mount/file.txt` type command running, and not noticing until the bill came due...

since then, I'm totally on board with pre-provisioned storage. at the very least, some way to put an upper bound on volume size (maybe efs has that? not sure.)

Oh wow that's horrific. That will write gigabytes in a few seconds on an SSD.

Unsurprisingly, AWS EFS does not support NFS quotas.

For production software, you should always have an upper limit on everything.

There is always a point where you can tell in advance that it doesn't make sense to keep going. It is never sensible to literally go on forever with anything -- it can only break stuff in annoying ways when it runs into real world limitations (like financial ones in that case.)

I sound aggressive about this because it's such a common mistake. It always goes something like,

"Why does this list have to be unbounded?"

"Well, we don't want to give the user an error because it's full."

"Okay, but does it really need to support 35,246,953 instances?"

"Sure, why not?"

"How long would the main interaction with the system take if you stress it to that level?"

"Oh, I don't know, at that level it might well take 20 minutes."

"And the clients usually timeout after?..."

"5 seconds."

"Would the user rather wait for 20 minutes and then get a response that might be outdated by that time, or get an error right away?"

"They may well prefer the error at that point."

"So let's go backwards from that. Will it ever make sense to support more than 250,000 instances?"

"That corresponds to the five second timeout and then some. I guess that's fine in practise..."

It's not that hard!

100% agreed.

Often limits need to be addressed at the product level. At some point at my $work we started pushing hard enough back at product to say that we're placing a hard limit on every entity. What the limit is can be negotiated, re-evaluated, and changed -- but changes to it need to be intentional and done with consideration to operational impact.

My personal pet peeve is when i'm dealing with a client library that doesn't expose some sort of timeout.

I'd also add unbounded queues to the list above as a subtle place where the lack of a limit can really cause production issues. Everything may look like it's working fine until you realize that you've got a 4gb process with a giant buffer that'll take forever to drain.

Clear and realistic examples.

Terrifying. Please link the story if you can find it!

luckily it was recent and unique -- easy for google to find :)

here it is: https://news.ycombinator.com/item?id=32175328

> Amazon's EFS

No snapshots. No support for ACLs. Poor performance. High cost.

Where is such block storage available? Seems fantastical...?

Ceph's RBD basically works like this (but it's a self-hosted software package, not IaaS), so you still need to manage the storage pool yourself.

"Where is such block storage available? Seems fantastical...?"

SAN vendors have been selling devices that deliver thin provisioned network block devices you can snapshot, clone, resize dynamically and replicate in a cluster for about 20 years now. Third parties can do all of this in cloud environments as well. See Netapp Cloud Volumes. You could roll your own using Ceph, or even a ghetto version with a subset of these capabilities with thin provisioned LVM+iSCSI, DRBD etc.

You should be able to point and click a thin provisioned, dynamically expandable, snapshot-able, clone-able, replicate-able pay as you go network block device with whatever degree of performance, availability and capacity you wish to pay for. The fact that you can't is a function of the business model and mentality of IaaS cloud operators; they don't want your old fashioned 'stateful' workloads. Neither does kubernetes. So you're swimming against the current.

Reply on news.ycombinator.com