Hacker News Reader

Comment by twa927 | Hacker News Reader

Does max. RAM usage need to be specified upfront, or what's billed is the live-allocated amount?

"You are billed only for the CPU and memory allocated while a request is active on a container instance, rounded up to the nearest 100 milliseconds."

https://cloud.google.com/run/pricing

You can configure the amount allocated: https://cloud.google.com/run/docs/configuring/memory-limits

Ok thanks. I'm looking for a way to run a service with memory peaks - 50pct is 200MB and 99pct is 2GB. It looks like no current serverless solution would handle it without overpaying by allocating 2GB for all executions.

Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvious how much memory is actually being used without having language/runtime specific signals. The mitigation in Cloud Run is both concurrency, and that you're only billed while a request is active.

(Disclosure: Google Cloud PM)

> Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvious how much memory is actually being used without having language/runtime specific signals.

I'm not sure what do you mean here, Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right? Yes, Java can always use large heaps but we're running Python and C++ where a process' memory usage directly relates to what a program allocates (even PyPy with GC has this property).

> The mitigation in Cloud Run is both concurrency, and that you're only billed while a request is active.

When there are memory peaks, larger deployments without container-level concurreny look better. For my example 16GB of RAM allows running 8 containers to get a chance for a 2GB task to complete, but on average 90% of the memory will be wasted. On a single 16GB server I can run 48 tasks with 40% wasted and a high chance of the 2GB tasks finishing. Yes, in this scenario I must handle tasks killed due to OOM but the difference in throughput is so large that it's worth it.

Cloud Run is built upon Knative so it runs inside k8s (which uses CRI API of a backend, Docker or any other alternative) which handles OOM by inner resource manager, not cgroups.

You can use Cloud Run with GCP-managed infrastructure, and also inside your own GKE cluster.

The GCP-managed infrastructure exposes the Knative API, but doesn't actually run in GKE/k8s.

(I work on GCP but not much/on Cloud Run. Above is correct to my knowledge, but I'm not an expert.)

Good to know, thank you.

> Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right?

Cloud Run uses gVisor as its container runtime.

Disclaimer: I work on GCP but not much on/with Cloud Run.

Yes, it needs to be specified upfront.

The default is 128M and you can specify up to 2G

Reply on news.ycombinator.com