Cloud Computing Without Containers
blog.cloudflare.com
blog.cloudflare.com
Note that the core point here is multi-tenancy. With Isolates, we can host 10,000+ tenants on one machine -- something that's totally unrealistic with containers or VMs.
Why does this matter? Two things:
1) It means we can run your code in many more places, because the cost of each additional location is so low. So, your code can run close to the end user rather than in a central location.
2) If your Worker makes requests to a third-party API that is also implemented as a Cloudflare Worker, this request doesn't even leave the machine. The other worker runs locally. The idea that you can have a stack of cloud services that depend on each other without incurring any latency or bandwidth costs will, I think, be game-changing.
I have yet to use this specific product and after reading the blog post I'm actually really frustrated that I didn't fully understand the things that I could do with Workers as I had a proof-of-concept app recently that would have been nice to test with since we've had issues with cold start delays on Lambda in similar projects in the past. And that second point that you make, above, ... holy cow; that's going to change the way I design systems and applications for customers.
Personally speaking, I am a fan of CloudFlare like some of the more obsessed are fans of Apple[0]. Every time your company comes out with a new announcement, it's solves some problem I'm experiencing on projects at that moment.
I could write paragraph after paragraph of all of the grief that using your services has eliminated from mine and my customer's lives, but I am certain you get that enough, already. There is no way my customer's appreciate your free tier as much as I, the developer, do. And your UX is awesome. Whichever one of your team mates designed the API button with instructions next to each option in the control panel as a way to onboard people to the API ... it's those stupid little things -- because that's there, when I'm setting up a client with a unique configuration for the first time, I start from a baseline, set it manually, test, and click into that link to see what commands I'm putting into the script should I have another client with similar needs. I probably wouldn't have thought to even look to see if you had an API were it not for those links.
I'm excited to try out Workers next time I have a Lambda need that fits well. Keep up the good work -- it's taking a ton of grief out of my life.
[0] A gentleman from your company sent me a T-Shirt for providing feedback for the Argo tunnel (called something different back then in beta) product -- you guys even do the damn T-Shirts right ... subtle logo on the shoulder and front, no words, and really soft cotton. It's my favorite "free vendor t-shirt" shirt.
edit: A sentence ... got away from me and I had a dangling bullet
Yes, and that is critical to our use case.
> Are requests to these workers http requests?
Currently, yes.
> Is there any backend storage or file access (or blob) from workers?
Currently there is Workers KV (key-value storage), and we have some cooler stuff in the works.
I would actually argue with #1. The truth is, this appears to be a better tech even if it isn't globally distributed.
Lightweight serverless functions with no cold-start are a really great primitive to build on. We figured them out in this way to enable global distribution, but that's a bonus, not necessary to really love the concept.
You have to create a process for each tenant, but that should be much faster and take much less memory than V8 JITting JavaScript or WebAssembly (Linux process creation is very well optimized).
You can try static linking, dynamic linking or even just forking plus an ad-hoc executable loader in userspace.
V8 isolates are the same separation as browser tabs, and thus are also designed to be initialized extremely quickly. I’d be surprised if spawning a new process is faster, and it’s not going to be faster if your goal is to run JS/WASM.
V8 is one of the most well tested and secured pieces of software in existance, and that it has a much smaller surface area than the Linux process isolation you're referring to.
Are you also stunned every time you open your browser? Because it's the same thing.
More recently, Chrome has introduced Site Isolation, which actually allows iframes to run in separate processes. On desktop, this is enabled by default as of Chrome 67. On Android, it is not enabled yet, because it has been shown to be too expensive. The Chrome team wants to fix this, but it's not clear (to me, at least) whether they'll be able to overcome the inherent barriers here.
Chrome started working on Site Isolation before Spectre, but Spectre accelerated interest in it. My take is that Spectre is probably not the main reason that the Chrome security people (who are awesome, BTW) want to do it, but it provided a great excuse to rally support behind it.
For Workers -- much like for Android Chrome -- process-per-tenant is still too costly to be feasible. So, we need to pursue other approaches, as I've described elsewhere in this thread.
When will KV be out of beta? It's hard to commit to a platform like CF Workers when it's obviously still not intended for mainstream usage and there is no public timeline.
I'm happy to give you beta access for you to experiment. That said, we take the distinction between a beta and GA very seriously. We don't want you relying on software we're not, at the very least, using in production ourselves.
That day will come very soon! We're moving several projects into KV (allowing us to move them out of centralized data centers), and we expect KV to track GA in the next few months.
Do you have plans to deploy this tech at cell towers in metro areas and support 5G-enabled applications in the future?
Doesn't this undermine your assumption that external timers can't be used in a timing side channel attack because "the network is extremely noisy"[1]? Consider a third-party API implemented as a Cloudflare Worker that just returns Date.now(). How much noise will there be if that runs locally?
EDIT to clarify: I’m assuming they can do 10,000 tenants per server. If they did 1 tenant per 1 TinkerBoard and charged $2/mo for it flat no hourly fee that would be an interesting business model IMO and it achieves hard isolation between tenacts.
ETA: It looks like it is complicated, but might work. See https://nixos.wiki/wiki/NixOS_on_ARM/Scaleway_C1
[1] https://security.googleblog.com/2018/07/mitigating-spectre-w...
wasm really has quite a lot of interesting and unexpected applications! (though of course non-wasm tech can accomplish the same thing, but it’s nice that we are getting close to some form of universal binary format)
For existing container glue -- you could probably do this with runc (though unfortunately we are also written in Go) and the architecture is such that runc has no long-running process (only your container continues running after we've set it up). I believe LXC can also produce similar tenancy (and they're written in C which avoids quite a few of the Go problems).
Docker has a nice and evolving API and OS zones provide an amazing core technology.
My point was that "Linux containers" (so tools like LXC, or runc, or others) are more than capable of being used in this way. Docker has a variety of architectural and other problems that result in it not being as friendly to this kind of use case.
So when you run "docker containers" on Joyent they just untar the image and run it in an LX-branded zone. There is no Go or other problems and they sure as hell don't run Docker as root in their control plane.
I'm relatively sure 10ms is not the worst case cold start time of lambda.
> Cold starts are a symptom of something else (likely shipping function data)
Which is all part of spinning up a containerised function in lambda.
About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.
What do you mean by "use" in this context? Does every program that links to glibc "use" the memory for glibc? (The answer is no, because the page cache is shared. But if you just looked at memory usage you might be fooled into thinking it is copied for every program.)
Containers use a drastically tiny amount of in-kernel memory. We are talking fewer than a couple of kilobytes in the worst case.
It appears to me that the main worry was with using Kubernetes, and they've applied this to containers as a whole. I agree that in this sort of usecase Kubernetes really doesn't make sense (using container images at all makes no sense in this case), but the core kernel primitives and simple container runtimes available are more than suitable.
There is an argument that having no userland-kernel context switches has a performance improvement (and it does) but I would like to see data before accepting at face value that "context switches are expensive" is an appropriate thing to optimise for.
After all they are still doing context switches, it's just in userspace. And if v8 is multi-threaded they've just reinvented the N-to-M scheduling model that is very well known to be fundamentally broken because of various well known pathologies.
My point is that the article in question makes claims that I'm not sure are correct, and I wonder whether the engineers behind this solution decided against containers because of their initial experience (this isn't a dig against them, this is a very common trait of almost everyone -- you don't want to waste your time if the first impression you have is negative). Several of the statements (especially about memory) appear to be based on misunderstandings of how containers would actually operate in such a scenario (or based on testing with sub-optimal configuration).
Maybe I'm completely wrong that containers would work under this kind of load in this scenario, but reading the article I didn't find many arguments I would expect to see if someone had really tried to make it work with containers and found the flaws. Instead it reads (at least to me) to be more of a "at first glance this doesn't appear to work" -- which is a fine thing to base product decisions on, but it isn't really okay to then spread (what appears to me to be) misinformation unless you have done significant testing to justify it.
So I disagree that my argument is moot -- and I would like to know how much cheaper the userland context switches are in V8 (I just looked and it appears that V8 does have a multi-threaded core now -- but hopefully CloudFlare doesn't use that with their userland threading+isolation model...).
With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data.
With Workers, we're seeing an order of magnitude better.
There is no one, simple reason for this difference -- rather, it's a large number of factors working together. If I enumerated every one of them, you probably wouldn't find any of them compelling on its own.
Sure, if we could convince all our customers to write tiny C programs, with just the right constraints, maybe they could be as efficient in terms of RAM usage. Maybe. In Sandstorm, a raw C/C++/Rust app could indeed fit in a couple megabytes. But there's still context switching overhead. And no one wants to write C, they want to write JavaScript. ¯\_(ツ)_/¯
Great! :D
> With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data.
I hate to keep harping on this, but what do you mean by "used" here? Are you saying that the RSS was 100MB per container, or that the sum of private mappings used was 100MB (/proc/$pid/smaps)? Did you use a filesystem like overlayfs that facilitates page-cache sharing by allowing read-only inode sharing between different containers, or a driver like {btrfs,devicemapper,aufs} that didn't? (Sandstorm was kicking around a while ago, so this might've been before overlayfs was in mainstream use.)
I know I probably look like an asshole, but I actually don't think I understand what you mean by used -- because saying that a container uses 100MB (which I take to mean that each new container spawned costs 100MB of real mmeory) simply doesn't sound right. It implies you could run less than 80 containers on an 8GB machine, and I doubt that Sandstorm programs were this big. It's like someone telling you that they spent $1500 on lunch -- I'm actually confused what the word "spent" means here.
I'm sure that you'd see less RSS memory usage by only having one process, but it is very possible that this memory usage benefit is not actually real -- I could be completely wrong but I'm just having trouble understanding how putting the same code inside a single program could make such a difference (given that the page cache already shares the memory for the V8 process). If you were to run 10k V8 processes your RSS would increase by 10k but your real memory usage would only increase very slightly because of the minor kernel memory cost of "struct task_struct".
As for context switches, yeah okay that's a cost you pay by having more than one process on a machine. But you do still have context switch overhead (though obviously it's much smaller) if you're running more than one program's state in a single process -- you have to switch context to a different set of protected variables right?
> And no one wants to write C, they want to write JavaScript.
My point about page caches is that the code for V8 is in the page cache and thus running more V8 processes doesn't take up any more real memory than just running one. So that cost of running JavaScript on top should be similar (though probably slightly larger because V8 stores other program information in memory -- but definitely not 10x larger).
"Application containers" get a bad rap because Docker's model of a bunch of tar archives with separate root filesystems isn't actually the best way of getting density (nor is it what a lot of people really want).
Does it take 500ms every time you type 'ls' in a console? That's starting a process.
> About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.
It's not the claim. I wouldn't object to someone saying "we charge X, lambda charge Y". But the post seems to confound implementation decisions in lamda with fundamental properties of containers and processes, which are simply not correct.
Launching a process is different than launching a language runtime or interpreter. They generally are not optimized for initial start time in the way an Isolate is. The duration of a context switch varies, but when dealing with tens of thousands of requests per second per machine they are very significant.
A context switch is O(10000) cycles and 10k is totally doable.
I agree that the Isolate architecture is more efficient, I just object to the straw man comparison. (And would love some real data.)
Your account's Lambda functions won't share a kernel with anyone else's
This is definitely possible with containers. Docker has historically had density problems for a variety of reasons (its architecture as well as Go being awful for systems programming in this context). But you can definitely get this density with a simple container runtime that doesn't have large long running processes.
Saying you can't run 10k containers is like saying you can't run 10k processes on Linux.
I'm sure that it might be somewhat faster to have it supported in V8 directly, but I don't want people to think such density is impossible with containers. (And with page-cache sharing you'd be able to not have to load 10k V8 programs into memory as well.)
Also things like page cache sharing would make a huge difference to the real memory cost of having many processes. When you run 10k programs you don't have 10k copies of glibc.
Sure.
This looks to me like just another form of vendor lock-in.
Can you say more about the assumed security model here? We built gVisor [1] (and historically used nacl or ptrace) precisely because things like Isolates aren't (historically?!) trustworthy enough for side-by-side. Plus the whole V8-only ;).
Edit: But let me say, I still think this is awesome (and love seeing more people jumping into the "just run code" space).
[1] https://cloud.google.com/blog/products/gcp/open-sourcing-gvi...
I have not seen something like this demonstrated in a Javascript/high-level-language context yet.
Even if you ignore side channels, there's a lot that can go wrong in the software only case. I'm pretty sure that site isolation as a feature predated Meltdown and Spectre (though the default changed).
This is tricky to answer concisely because we've implemented a huge number and variety of defense-in-depth measures. (Plus, I'm in transit right now and can't type much -- maybe I'll edit to add more details later.)
First, we believe Chrome has done a pretty good job hardening V8 over the years, and we get a lot of comfort knowing there's a $15,000 bug bounty for V8 breakouts. We update V8 continuously, tracking the version shipping in Chrome.
That said, we obviously don't simply rely on V8 for everything. We've modeled what a V8 breakout is likely to look like, and added a variety of mitigations.
Presumably, a V8 breakout bug is likely to allow an attacker to run arbitrary native code within the Workers runtime process. That's bad, but they will quickly run into some barriers to weaponizing this. For example, we obviously run with ASLR, so an attacker looking for other isolate's data would be flying blind and would likely raise segfaults. Any segfaults in production raise an alert and are investigated. Similarly, we run a tight seccomp fitler that denies all filesystem and network access, and if an attacker ever tries to invoke such syscalls, it will raise an alert and be investigated.
It's worth noting that we do not allow eval() (or any other mechanism of "code generation from strings"), hence all code we execute has to have been uploaded through our code deployment pipeline. This implies we have a copy of all code. When a segfault raises an alert, we immediately look at the code that caused it. Anyone using a zero-day against us is very likely to burn their zero-day long before they manage to pull off a useful attack.
You're probably also wondering about Spectre. Here's a previous comment of mine on that topic: https://news.ycombinator.com/item?id=18280156
Again, this is just a couple of the things we're doing... there's really too much to list in a HN comment. I hope to find time to write this up more formally in the future. :)
This is interesting, because preventing code generation from strings goes beyond `eval()` and `new Function()` - it also includes things like the output www.jsfuck.com can produce (which can indirectly call things like `eval()` and `new Function()` without ever mentioning `eval` or `new Function` in their source). How do you prevent things like that?
You can entirely remove eval such that even if code contained a very obfuscated form of `eval("foo")` it would get "eval is not defined".
Since they're not just sanitizing their input, but rather are forbidding things at the runtime level, things like jsfuck.com are totally irrelevant and should have no bearing on their security.
https://github.com/v8/v8/blob/53d3f5ba2a7ee77244bdee882da9da...
Line 9097, SetAllowCodeGenerationFromStrings()
Given the comfort in the bug bounty, and that bugs are inevitable, it seems like cloudflare is willing to risk some exposure to upcoming exploits. How do you feel about this in an era of sophisticated nation states that I’m sure would have a keen interest in being “inside” cloudflare?
I actually think our approach to Spectre -- focusing on preventing the observation of the side channel, rather than blocking speculation -- is likely to be more robust against Spectre variants that haven't been revealed yet. Case in point, we developed some of our core mitigations before we knew about Spectre at all.
Spectre affects containers and VMs too.
Pretty sure if you run k8s or VMs your company's IP isn't exposed to random engineers at the provider.
Do you trust the model enough to get Workers and Workers KV PCI-DSS Level 1 certified?
That is, totally different use case... going with V8-only in return for some better non-functional metrics is a trade off Cloudflare made for a serverless platform, but not one that Google (or anyone else, probably) would make for a container-based platform?
I think Cloudflare workers are a cool implementation of domain-specific scripting. I’ve used it and it’s great. And with their scale I’m sure they’ll advance the state of the art in various ways. But there’s no radically new insight here.
My question is why would I want to run an unmodified application on this? Because of existing business logic in the form of shared libraries that I'd like to leverage, unmodified?
Complete straw-man here, but this is what I envision:
Ignorant businessmen buys a new silver bullet from a (potentially ignorant or malicious) salesperson or idea broker then wants you to "make it work" on the "serverless cloud".
Most B2B software provides no reasonable method of determining if their product(s) actually work. So changing the amalgamation of your boss's silver bullets (your business application) requires the rebuilding of a hard won set of folk knowledge. This task is terrifying, unproductive, potentially impossible (non-permissive licenses), and something that most developers in this situation would like to avoid.
Of course there are decent ways to avoid and/or address situations like the one described, but this seems close to what I've witnessed in industry.
There are monolothic applications and overly restrictive central IT processes where the “folk knowledge” you describe is basically a job-security obfuscation tactic for developers to wield political control and act as credit-mongering constraints on what other developer teams might want to do, under the guise of it all being for the safety or stability of the monoliths (which are already unstable and unsafe).
Particularly in cases where multiplicity of different runtime environments is critical for innovation, this central IT bureaucracy can be totally killer.
The worst is when outside consultants come in and just reinforce what the central IT people are already stonewalling with.
For this reason, it can actually be great when new silver bullets come in from consultants who are able to break that hegemony. Sure, the silver bullets are dipshitty business crap, but if it at least offers a hook for using better modern tooling, you can successfully not die from the central IT crap.
Then by showing the results of the new tools, you can undermine the “folk knowledge” gate keeper people, and force them to be honest about the declining business value of protecting the so-called stable legacy monoliths.
The other drawback is you have less layers of protection if operating in process with language-based and/or tactical mitigations. The MMU can isolate what we call known unknowns where something goes wrong (eg bitflip or RAM decay on protection mechanism), the app goes rogue, and it's still contained a bit. That's why the strongest approach from high-assurance security is still separation kernels: microkernels designed specifically for security that fit into L1 caches, are often mathematically verified in some way, support individual apps running in own address space optionally on a language runtime, and support VM's for legacy apps. Muen is an example of that approach for x86, seL4 an example for ARM (mainly ARM), and INTEGRITY-178B a commercial example for PPC (beware marketing hype):
https://www.ghs.com/products/safety_critical/integrity-do-17...
Just because one abstracts-away the sandbox from the users doesn't mean that the sandbox can't be used as a layer of protection. The wiser approach would be to use an additional layer, though it's abstracted away from the user's point of view.
As far as I can tell, the wasm community is still primarily focused on running in the browser. There is an explicit goal of supporting "non-web" targets, but no consensus on how to expose system APIs, let alone a full implementation.
Which removes many of the advantages.
But perhaps web assembly is easy enough to decompile?
Not that long ago Cloudflare was bitten by a nasty bug where their own parser mixed up HTTP responses from different customers https://bugs.chromium.org/p/project-zero/issues/detail?id=11...
Cloudflare’s response to this incident did not inspire a lot of confidence. What I am trying to say is that I won’t trust their workers implementation unless a competent external security researcher audits it.
Edit: Related discussion https://news.ycombinator.com/item?id=18417257
Several observations recorded in the Project Zero report about CF's response were problematic:
"I had a call with cloudflare, and explained that I was baffled why they were not sharing their notification with me."
"They gave several excuses that didn't make sense, then asked to speak to me on the phone to explain. They assured me it was on the way and they just needed my PGP key. I provided it to them, then heard no further response."
"I can already see that the (huge) list does not contain many pages that still have cached content from before the bug was fixed."
"Cloudflare did finally send me a draft. It contains an excellent postmortem, but severely downplays the risk to customers."
Granted, I have not been keeping up with what has changed at CF since then so maybe things are better. But the decisions taken wrt Workers doesn't inspire a lot of confidence. Even a Google Cloud engineer mentioned that Isolates probably aren't trustworthy enough to run untrusted code side-by-side https://news.ycombinator.com/item?id=18417257
If the Unix process abstraction is no longer fit for all purposes, why don't we get around to de-layering the operating system so that the Unix process abstraction is just another 'Virtual Machine' sat on top of something more fundamental?
Is it time for the return of the pure hypervisor?
Perhaps I'm misunderstanding you, but the Unix process abstraction was never meant to isolate processes from each other. It was designed to allow multiple processes to run on the same machine.
On the other hand, the Unix OS was indeed designed to isolate itself from processes running on other Unix OS'es, which is exactly why we use the entire OS as the container of processes that need to be isolated.
Some of the density advantages depended on Paravirtualization, which is quite difficult to make safe against Meltdown: https://xenbits.xen.org/xsa/advisory-254.html
I recognize this is just a little marketing (which is fine), but I'm not sure what this is supposed to mean any more.. containers have identical efficiency to running on a raw VM, and VMs only trap to emulation in very few circumstances (e.g. IO) any time in the past decade. Running pure compute load in a container, guest VM or host VM should have almost identical performance.
But they are running multiple client's services in the same process, so that's how they're saving - less context switching, smaller memory footprint, etc.
I'm interested in how isolated the client's applications actually are...
As the article says, faster spinup (since v8 is already running) seems to be the customer's greatest advantage only, saving an order of magnitude of memory is certainly a big deal for the operator.
I think since you can compile pretty Much anything to webassembly, porting over some existing tools might even be viable.
(Disclosure: I'm the tech lead for Workers.)
>> Unlike essentially every other cloud computing platform I know of, it doesn’t use containers or virtual machines.
This is light in comparison to their posts about QUIC, anycast geolocation, etc.
They surely have done a lot of interesting work, but they'd lead you to believe they're hosting half the internet, and invented every advancement to CDNs in the last decade.
I know that the article briefly addressed security, but in my head, I can't even work out how they manage to securely manage multiple tenants within the same process. This isn't a knock on them, it's a lack of knowledge on my part, and I'm wondering how that would even work.
Isolates in Dart can be run based on different dependency graphs (i.e. you can have an Isolate for user A's project, and a separate on for user B's, within the same process). However, things like the current working directory are shared by default. There's an IOOverrides class that lets you change this contextually, so theoretically you could hack something together that patches the entry point code to use it. I'm not sure how that could work in Node, though. What if someone were trying to overwrite another user's files?
Overall, though, the #1 thing that I can't reconcile in my head is that if all the isolates are running in the same process, wouldn't they all have the same permissions? What is there to prevent someone from touching another person's files, or, if they run in the same process as the server, more sensitive data? AFAIK you can't run a Worker/Isolate as a different user within the same process?
Not only that, but what about fork(2) bombs, etc.? The only thing I can think of is for Isolates to run under the context of some user who not only has no home directory, but also cannot spawn processes, or read/write/delete files.
Lastly, the only other thing I can't work out is how whether the Isolates run in the same process as the server (and if so, how do they do this securely), or how they communicate with the separate process, if technically any Isolate could console.log (maybe via TCP sockets?).
The whole CloudFlare workers concept is super cool... I just don't understand how it works. I feel like I'm missing some key here.
Guessing we'll see a competitive product from Fastly in the near future.
> This might mean Isolate-based Serverless is only for newer, more modern, applications in the immediate future.
It sounds like you're implying unless your app is written with Node or Go / Rust then it must be legacy. That's not really fair to say.
Plenty of modern apps are built with Python, Ruby and Elixir. I don't know if they are all webassembly compile targets, but in either case, you should probably rewrite that to be less "better write everything in Node or you're a dinosaur!".
Note: I work at Cloudflare
No other changes to config.
It'd be huge for p2p/decentralized apps. As of now, you can only decentralize data - this allows decentralized compute as well.
Really really interested to see where this could lead.
Not really: There are many nice managed languages that use JS as compile target. You can use ClojureScript, Scala, Purescript, Fable (F#), Haste (Haskell) etc.
Not sure what progress has been made on it.
It specifically covers isolates vs. contexts, which should help contextualize it.
I'm not particularly convinced Javascript is that much better than any other language as the language goes, but the implementation is far ahead of almost anything else. I know of other implementations of this idea in other languages ("Safe" in Perl [1], various attempts at sandboxing in Python which have generally failed, Safe Haskell [2], many others), but without the browser use case driving them forward, they tend to be poorly tested and even decay over time. It tends to be a use case popular enough to get some stab at support, but not popular enough to quite result in a robust, real-world-tested library support. The browser is an exceptional use case.
Or WebAssembly, for which many languages now have a backend.
IIRC it won't quite work because the plumbing required for WASM on the host side isn't done there yet [0]. The plumbing is mostly [1]. In a project I did on the side to reduce Go WASM size and startup time [2], I built just enough of this plumbing to solve my needs. Go initially targets only JS [3], so sadly us with non-web use cases might have to wait.
0 - https://github.com/go-interpreter/wagon/issues/69 1 - https://github.com/golang/go/blob/master/misc/wasm/wasm_exec... 2 - https://github.com/cretz/go-wasm-bake 3 - https://github.com/neelance/go/pull/7#issuecomment-377298992
FTA “In an Isolate universe you have to either write your code in Javascript (we use a lot of TypeScript), or a language which targets WebAssembly like Go or Rust.”
So if want to write serverless functions in Go or Python you go somewhere else. But if you, as most other do, write them in Javascript, then this is a relevant platform to consider.
And what's available is the same as a Service Worker in Chrome. So you can use http fetch but there is no raw socket etc. Does it support WebRTC (not sure if there is a way to actually use that on the server but maybe there is some weird use case for communicating between Workers).
Also wondering what options there are for files or databases etc.
It sounds like the design is for them to handle individual requests, but just out of curiousity, how long can the workers keep running?
1MB max size, max 5-50ms CPU time, max 128MB RAM: https://developers.cloudflare.com/workers/writing-workers/re...
That said I've got a worker script running a pass-through websocket to GraphQL origin server and it keeps the connection open.
I'm doing authentication within the worker too. Long-term I can see ability to run absolutely everything there: microservices, GraphQL server (Apollo has a beta), auth, static SPA files, SSR etc. $0.50 per million calls ($5/mth minimum). And they intend to be within 10ms ping of 99% of the global population.
If they can workout how to do databases on the edge (they're working on it), then I can see a future where you could build scalable apps that serve millions of users for maybe less than $100/mth across the entire stack.
It's a mystery to me why we need millions of instructions just to put "hello world" in a pipe though. I'm thinking of learning low level programming, but I'm too scared of what I might find.
Surely there were others before like Azure Service Fabric ?
Ultimately most of us want to put files on a server and start a process. There’s just an endless variety of constantly re-invented ways to do that. Choose your scaffolding.
What they're describing is actually running code from multiple people on the same process.
By "choose your scaffolding", I'm just saying "how much of the OS kernel would you like to re-invent?"
That isn't a wholly cynical position. Doing so might introduce new capabilities or cost efficiencies, which seems to be Cloudflare's pitch. But very often we get excited by the possibilities and lose sight of the tons of additional stuff now required to achieve quite simple things. In the worst case (e.g. VMware) we end up re-inventing hardware abstraction, scheduling, networking, memory management, and filesystems i.e. most of the damn kernel. Docker is a less egregious offender but still deserving of the "too much scaffolding" label.
In the very best outcome, the capability gets stripped down to the core enhancement and pushed back into the expectations for a base OS, which is for example why practically every OS now comes with a hypervisor and a container mechanism.
Lambda, Google/Azure functions, arguably Google App Engine etc.
Seems like a very pragmatic system. Get some great benefits for a reasonably well defined and common use case (JavaScript apps that don't need to run external binaries), without doing anything crazy just to support apps that don't quite fit in the architecture. It might not work for everything, but for many apps this will be a very simple and low overhead solution. Kudos!
- Process isolation
- Close to hardware
- No cold start
But it also has some advantages like:
- Any language, including compiled languages
- Local testing, backed by a standard
I am still experimenting with the scaling potential; my biggest concern is memory requirements. Currently I'm writing the project's blog using the project, and then am going to do some performance tests.
It's hosted on a 5 dollar digital ocean instance at bigcgi.com.
[edited for formatting]
And how doesn't that put you to square 1 with isolation? Cloudflare is running multi-tenant loads; you ideally want to have fine grained control on resource utilization and access controls. With CGI, you pretty much have to reinvent Docker to get the same level of isolation...
I'm not sure I follow how this is superior to other solutions except for that CGI is easier to test locally.
As for resource utilization/constraints, it runs on freebsd and uses rctl to limit memory usage and number of processes. One thing I want to find out is how to balance user resource constraints with the total load of the shared server.
Still, I do wonder how much of the problem with containers is container overhead. To me a big difference is about what costs can be amortized. With CGI, pretty much every cost is paid at every request. With long running daemons, the per request overhead is removed. With Cloudflare's functions, the scheduling, request, and startup overhead are (theoretically) removed, leaving mostly just execution.
With just CGI there's some win since you can drop the scheduling overhead, but I think you would probably lose that advantage as soon as you hit the upper limit where forking per request stops scaling. With FastCGI you can enable better scaling but since you now have to manage long running processes, you are officially in the scheduling business, though you could probably beat container scheduler like Kubernetes in terms of raw overhead.
I don't know. Maybe this is a good idea, but I'll admit my first impression was mostly "isn't this just CGI as it always was?"
I haven't used this, just read the same article you have, but the claim seems reasonable.
We've deliberately incurred the context switching costs of using multiple processes for decades because the hardware-enforced isolation of address spaces via the MMU is desirable. Of course you can gain efficiency by throwing away the multiprocess model and running separate security domains in the same address space of a single process, we're not breaking new ground here by going backwards.
What property of this application allows it to share CPU time and memory more efficiently than the OS can?
Will there be logs or graphs to help investigate such issues?
That is a heck of a statement to make.
Compare that to Linode/Hetzner/OVH/etc. where you get half-CPU for that money (i.e. 360 hours CPU time per month).
And yeah, sure. Not apples-to-apples. FaaS, infinite scalability, yadda, yadda. But computing http responses is still computing http responses. Is this FaaS magic dust really worth 1400+% premium?
Also, it is super-common (especially on low end) to offer burstable CPU VMs [1,2]. So you are not getting charged for full CPU, unless you specifically ask for it.
[1] https://www.hetzner.com/cloud?country=us (bottom of the page, toggle between "default" and "dedicated CPU")
[2] https://aws.amazon.com/ec2/instance-types/t3/ (product details, "Baseline Performance/vCPU" column)
If requests only require <10ms of CPU anyway, $5 per 10 million requests is pretty reasonable to me.
I re-read the article and it is indeed cpu time not runtime for Cloudflare Workers. Sweet!
The part of the article that clarifies this:
> This is not meant to be a referendum on AWS billing, but it’s worth a quick mention as the economics are interesting. Lambdas are billed based on how long they run for. That billing is rounded up to the nearest 100 milliseconds, meaning people are overpaying for an average of 50 milliseconds every execution. Worse, they bill you for the entire time the Lambda is running, even if it’s just waiting for an external request to complete. As external requests can take hundreds or thousands of ms, you can end up paying ridiculous amounts at scale.
> Isolates have such a small memory footprint that we, at least, can afford to only bill you while your code is actually executing.
> In our case, due to the lower overhead, Workers end up being around 3x cheaper per CPU-cycle. A Worker offering 50 milliseconds of CPU is $0.50 per million requests, the equivalent Lambda is $1.84 per million. I believe lowering costs by 3x is a strong enough motivator that it alone will motivate companies to make the switch to Isolate-based providers.
I understand how async io works. I'm used to AWS Lambda charging for request start to end time regardless of cpu usage.
Having anything on someone else computer it's not "The Network is the Computer" it's simply someone else computer. Also it can't scale, no matter how much you work on it, no matter what wonderful tech you create/integrate. Many now talk about edge computing because of that, and I fear it can succeed for a certain amount of time but still it can't scale.
We are people, not puppet, we need to be autonomous and social, not a herd of sheep with very few shepherds.
Try breaking https://contained.af/
https://www.twistlock.com/labs-blog/escaping-docker-containe...
A good example is HTTP middleware in your web framework of choice: there are things it does that can’t be done safely or correctly on the client, but that you may want to do at the edge before it hits your backend entirely.
It's based on node.js right now. It's a bit clunky, but we have many success stories of customers running it locally, on their CI and deploying to infrastructure.
Currently the open source version is not the same as the production version (due to the distributed nature of our platform.) We're working on fixing that very soon and going all-in open source.