eerikkivistik

As far as I can tell we are in violent agreement. When you do complicated things you have to tune, when you don't you don't.

Sure, a few examples that happen with modern JVMs (even 25). Essentially anything addressing a large amount of memory (think > 100gb). This can be databases, streaming solutions and so on, where you start tuning the GC just to have a smooth experience. Then there is the case of memory fragmentation of the heap, so you might want to switch out your GC. What else, basically any system that uses a lot of temporary memory is borked, because Java has a nasty habit of not returning unused heap to the OS by default. All in all, my point is, if you are doing anything that actually utilizes the hardware near its limits you will be tuning by quite a bit.

Perhaps you work on simple CRUD applications? In my experience the 90/10 are reversed.

I know a company already operating like this in the fintech space. I foresee a front page headline about their demise in their future.

The only reliable way to get things fixed with these large companies, is to have a direct point of contact to them. I have one. You can find my email in my profile, ping me and I will try to get a human in the loop for you.

I once sent this as a reference to a government agency I was consulting, to illustrate to them, how they operated.

It reminds me quite a bit of collision engines for 2D physics/games. Could probably find some additional clever optimisations for the lookup/overlap (better than kd-trees) if you dive into those. Not that it matters too much. Very cool.

There used to be a joint online project to compute these tables in a SETI like distributed system. Everyone who contributed their CPU cycles, could use the tables. And yeah, around 2005-2008.

The latency and bandwidth really isn't there for HPC.

Sure, so the use case I have requires elastic storage and elastic compute. So CephFS really isn't a good fit in the cloud environment for that case. It would get prohibitively expensive.

I've had to test out various networked filesystems this year for a few use cases (satellite/geo) on a multi petabyte scale. Some of my thoughts:

* JuiceFS - Works well, for high performance it has limited use cases where privacy concerns matter. There is the open source version, which is slower. The metadata backend selection really matters if you are tuning for latency.

* Lustre - Heavily optimised for latency. Gets very expensive if you need more bandwidth, as it is tiered and tied to volume sizes. Managed solutions available pretty much everywhere.

* EFS - Surprisingly good these days, still insanely expensive. Useful for small amounts of data (few terabytes).

* FlexFS - An interesting beast. It murders on bandwidth/cost. But slightly loses on latency sensitive operations. Great if you have petabyte scale data and need to parallel process it. But struggles when you have tooling that does many small unbuffered writes.

So the argument is essentially "8 billion people dying is a problem, that is worse than whatever the result of longevity is". I'm not sure that it is.

It seems to ping-pong back and forth. Our monitoring is basically spamming us with DNS down, DNS up every 5 minutes or so.

Agreed, let's upvote this thread higher to get some eyes on this.

No idea. Never actually seen a failure like this in practise.

Whois isn't even showing the domains and name servers. Not to mention DNS records. Its like the domains don't exist anymore.

Down for us as well.

While I do love Postgres and use it daily on AWS and Google Cloud, I will add that the managed Postgres on Google Cloud is a mess in some areas. For example they use some EOL extensions outdated for 10+ years (a specific example is GEOS) and refuse to update it and give no control for you to upgrade it either.

I might just take you up on that.

Thanks for clearing that up. Few followup questions:

You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct?

How long does data, that is no longer being accessed sit in the cache and count towards billing?

As for availability, are you in the process or do you have plans to also support Google Cloud?

Can you elaborate on a few things with regards to your pricing:

* What does "$0.05 / gigabyte transferred" mean exactly. Transferred outside of AWS or accessed as in read and written data?

* "$0.20/GiB-mo of high-speed cache" – how is the high-speed cache amount computed?

FlexFS kicks ass. I benchmarked it for our data storage and processing layers in value.space (satellite data processing and analysis) and we will most likely migrate to FlexFS in the near future.

Out of curiosity, why did you choose EFS, it's insanely expensive at even modest scales?

Not sure. Perhaps a further investigation by the authorities will reveal the reason.

You absolutely can and it is true.

> So, the bridge was under stress and moving(breaking off?) slowly by few mm each year causing the eventual collapse?

This is what it looks like from the data. The key here is "diverging paths", which means different parts of the bridge were moving in different directions, putting strain on it.

That is the goal.

So this is done via InSAR (https://en.wikipedia.org/wiki/Interferometric_synthetic-aper...), leveraging ESA satellite data. Not sure on the buzzword bingo there.

It is an after the fact survey done on historical satellite data. More data can be found here: https://newsletter.value.space/p/bridge-collapse-germany-aty...

I was gonna say... When your system is large enough to run into this specific performance bottleneck, pop a bottle and celebrate, you are making enough money to solve that problem.

While knowing this information is useful, most services fail in different domains and problems way before you reach that point. I'm not sure people really comprehend how hard you can hit a single machine before you need to distribute a workload.

Regarding Baldur's Gate 3 on M1 Pro, can confirm, the performance is just amazing.