since then, I'm totally on board with pre-provisioned storage. at the very least, some way to put an upper bound on volume size (maybe efs has that? not sure.)
since then, I'm totally on board with pre-provisioned storage. at the very least, some way to put an upper bound on volume size (maybe efs has that? not sure.)
There is always a point where you can tell in advance that it doesn't make sense to keep going. It is never sensible to literally go on forever with anything -- it can only break stuff in annoying ways when it runs into real world limitations (like financial ones in that case.)
I sound aggressive about this because it's such a common mistake. It always goes something like,
"Why does this list have to be unbounded?"
"Well, we don't want to give the user an error because it's full."
"Okay, but does it really need to support 35,246,953 instances?"
"Sure, why not?"
"How long would the main interaction with the system take if you stress it to that level?"
"Oh, I don't know, at that level it might well take 20 minutes."
"And the clients usually timeout after?..."
"5 seconds."
"Would the user rather wait for 20 minutes and then get a response that might be outdated by that time, or get an error right away?"
"They may well prefer the error at that point."
"So let's go backwards from that. Will it ever make sense to support more than 250,000 instances?"
"That corresponds to the five second timeout and then some. I guess that's fine in practise..."
It's not that hard!
Often limits need to be addressed at the product level. At some point at my $work we started pushing hard enough back at product to say that we're placing a hard limit on every entity. What the limit is can be negotiated, re-evaluated, and changed -- but changes to it need to be intentional and done with consideration to operational impact.
My personal pet peeve is when i'm dealing with a client library that doesn't expose some sort of timeout.
I'd also add unbounded queues to the list above as a subtle place where the lack of a limit can really cause production issues. Everything may look like it's working fine until you realize that you've got a 4gb process with a giant buffer that'll take forever to drain.
Unsurprisingly, AWS EFS does not support NFS quotas.
here it is: https://news.ycombinator.com/item?id=32175328