Serverless Docker Beta
zeit.co
zeit.co
- "sub-second cold boot (full round trip) for most workloads"
- HTTP/2.0 and websocket support
- Tune CPU and memory usage, which means even smoother scaling
And all that for any service you can fit in a Docker container - which is also how you get near-perfect dev/prod parity, an often overlooked issue with other serverless deployment techniques.
On top of all that, ZEIT has one of the best developer experiences out there today. Highly recommend trying it out.
And for the perpetual serverless haters out there: this is not a product for you, FANG developer. I think people underestimate how well serverless fits for the majority of web applications in the world today. Most small-medium businesses would be better off exploring serverless than standing up their own k8s cluster to run effectively one or two websites.
That said, I look forward to the company (or side project) where "serverless" can save me from also assuming the "devops" role.
Interestingly, as Firebase evolved during our use, nearly all of our external-instance use cases were obsoleted by more powerful native serverless support, esp. around functions.
All of which is the best of both worlds for serverless: an easy escape hatch to custom instances, and an ever-decreasing need for that escape hatch.
* video encoding;
* ETL processes;
* other analytical workloads;
* long-running websocket connections with Twitter/Facebook/etc APIs.
From my perspective, serverless solves the "boring" part of making REST APIs easier to manage, which were already very easy to manage.
How would serverless be applies to, say, a python script that streams Twitter tweets using websockets?
[0] https://aws.amazon.com/sqs/ [1] https://aws.amazon.com/kinesis/
I guess you could argue where to draw the "serverless" line, at functions or containers, but Zeit is calling this container service "serverless" so I think Fargate would fall into the same category. I think it would make sense for Zeit to eventually support long-running containers too (looks like the current max is 30 minutes, I'm not sure how they chose that number)
Either it works with web-hooks that you could lead to a Lambda via API-Gateway.
Or it needs to pull the data, then you could trigger Lambda via CloudWatch intervals.
I think this is the issue/risk with serverless. You either get locked into one of the big 3, or you end up doing all of the ops work to run your own stateful systems. As some of the people above you said, managing and scaling the stateless HTTP components is not the hard/expensive part of the job.
1) you don’t have to manage infrastructure
2) you don’t have to think about infrastructure (what size cluster do i need to buy?)
3) you pay for what you use at a granular level. (GB stored, queries made, function invocations, etc)
4) scale to zero (when not in use, you don’t pay for much of anything)
But the primary utility of serverless is an attempt at solving Amazon's problem of being commoditized by containers.
Can we please call it "stateless" instead of "serverless"?
The user experience (as an end user using the website) is pretty terrible IMO. It often takes multiple seconds for various areas of the site to load (bound by the network). It's also super Javascript heavy and just doesn't feel good even on a fast desktop workstation.
Authentication is also a nightmare from a UX point of view. Every time I access the site it makes me click through an intermediate auth0 login screen.
I really hope this doesn't become a common trend to run web apps like this. It would make the web nearly unusable.
I'm not a "serverless" hater either. I want technology to move forward to make my life as a developer better. I don't care what tech it is, but right now I just don't get that impression from serverless set ups (both from the developer and end user POV).
The only one that seems like it could even be related is the page loads being slower. But I'd be surprised if the issue were the serverless aspect of the architecture and not just that this software is apparently shoddy overall.
Renders server-side, loads quick, minimal JS.
In the mid-90’s it was common to run httpd via inetd. That lasted until HTTP 1.1 came along. I see people running websites out of Lambda now and just get a sense of deja vu. We realised this start-on-demand style didn’t scale for highly interactive use cases 20+ years ago!
The very list of benefits on ZEIT, mentions 4 things: first two of them are clear over-engineering bloat (plus premature optimization), and second two were actually created only because of serverless. They were (and nothing more was mentioned in section benefits ;) :
* Clusters or federations of clusters * Build nodes or build farms * Container registries and authentication * Container image storage, garbage collection and distributed caching
I don't know why everybody can't see fakeness of argument'you need clusters, farms and hundreds of servers'. You don't. Actually you do only (contrary to your statement), if you're FANG.
Why? Because look at real world HUGE examples. E.G. stackoverflow (and no, your company/startup/whatever, will not reach their level of traffic) can do everything on literally dozen of servers, while they admitted that in some scenarios 1 web server was enough. Source: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...
Our 10x..100x smaller companies would perfectly do on 2..4. There is no need for whole over-engineering.
The ultra funny thing is ZEIT selling 'deployment self-heal' as old known (windows anybody) and ridiculed recipe: it will work after restart. Right. It's better to shut off car engine, go out, go in, and start again. This is XXI engineering :)
[0] https://www.thestreet.com/story/13230576/1/what-are-fang-sto...
They are on the forefront of serverless.
Everything is deployed as immutable infrastructure.
When I get to the image, it is in the middle of everything and I don't really have an idea what is going on. Even watching it multiple times, I am not sure where it starts, ends, what the individual steps are.
Or is it because I just don't know enough about this stuff?
- Normal video with playback controls?
- Add a clearly distinguishable "this is where it starts" screen.
- Is there maybe a way to control playback of a gif via JS/HTML? Add a "restart" button.
- Add a timeline into the .gif that shows where I am (similar to usual video player).
Also, nobody seems to make use of these features, but GIFs do not have to loop, nor do they have to loop by going back to the very beginning of the animation as well. In particular, I find it extremely ineffective where there are GIFs that only show a "final product/scene" for like 1 frame before looping back to the beginning.
I don't think that's true.
Twitter and imgur do convert gifs to mp4, and you can see controls there, but that's because they're not gifs.
Do you have a citation or example of controls on actual gifs in any commonly used web browser?
Here's an example with a pause/play button:
https://rawgit.com/nbedos/termtosvg/develop/examples/awesome...
- Which one is more accessible?
- Which one is more "compressed" and mobile ready?
Not sure this is "standard" but imgur has their gifv "format" which will serve up the correct underlying format to the device
https://help.imgur.com/hc/en-us/articles/208606616-What-is-G...
Specifically, to be covered you have to download a binary release from https://github.com/cisco/openh264/releases during the installation to be covered by Cisco's MPEG LA patent license, but they provide builds for most major platforms.
We picked video/gif as the delivery format to give people a real-life impression of performance. (I took the videos myself while using the platform on a regular production plan.)
For more complex topics I find animations less helpful.
I only do this because its convenient using shareX and supported at the same levels images (no embedding needed), just traditional `img` tags. Also, it works natively with markdown so its convenient
The only suggestion is to download "Gif scrubber" or "Play this gif" or "Videoplayback Controller" off google chrome extensions
Complexity is hardly ever in the solution, but mostly in the problem. Single solutions to complex problems often ignore/forget important parts of the problem and they come back to bite you, hard.
- customers who insist every package has to be installed in some special place because /opt is ‘reserved’
- have to have non-standard ports for everything because it might slow down attackers
- have to have an Apache proxy in front of everything, always - even internal components. ‘Cos.
- won’t invest in trusted SSL certificates for internal services.
- every sql query has to be wrapped in a stored procedure - no exceptions.
... list goes on
- You have 20-30 specialized applications which aren't package based stored in an application share mounted at /app with the convention /app/name/version or something but then this tool hardcodes itself to /opt/name.
- Using nonstandard ports doesn't make you more secure against someone who's specifically targeting you, but it absolutely stops automated attacks and logspam. What do you mean you can't change the port?!?
- Because there's no such thing as a secure internal network. All sites go through the hardened proxy.
- Invest? Nobody is going to pay for certs for internal services and who wants to set up a reverse proxy If you can't use an internal CA that's just poor form.
- That's a new one for me. I assume because they have a single shared database for all of their apps.
That of course does increase (dba) administration overhead, but it seems on it's face simpler and far more "programmer efficient" than storing every single query as a stored procedure.
You know, unit test your stuff?
But on the other end I'm constantly fighting the complexity of existing environment and preventing to much new complexity being added. It seems that every time a new tool/product is introduced which makes things simpeler, it just end up being a Hydra and total complexity is only increased.
I still have to find the holy grail if there is one, some middle ground? Or maybe it's just human nature to always make complexity where there is none since "It can't be that simple, can it?".
tomorrow new devs, picking up a 5yr old project: this is downloading a centOS build from 7yrs ago! which only run on docker from 5yrs ago! and nobody know how to build a new base image with the libraries the old one have! lets rebuild it in <whatever snake oil they have in 5yrs from now>
the problem is not serverless or not, its clueless people and corporations with code/process rot. and the only lesson from the cycle, is to never trust people who proclaim one solution is the holy grail for all problems.
I went from right clicking and deploying from visual studio to SSH and configuring dockerfiles, docker compose, even nginx.conf to loadbalance.
You do get more bang for the buck with such setup but its too much work on infrastructure and less time for development.
edit: add kubernetes to it, although AKS and GKE being free lessens the burden its still too much for a software dev like me.
I‘m a developer too and I feel the burden too. Can you tell what you mean here?
Careful with filling your brain with domain-specific knowledge.
:)
Being a FaaS proponent, I still think this could be a good idea, because of the kinda standardized workings of Docker. I mean, AWS also uses Docker for local testing.
I like to think of docker like git. It solves some problems with distributing data but it doesn't solve the problem of developers writing code, writing tests, pipelines, etc. Nor even problems with 3rd party APIs etc. Obviously this isn't a perfect analogy but my point is docker isn't a magical silver bullet.
I've worked in places where their solution to everything was "docker" and it honestly caused more complexity problems than it solved. That's not to say I don't like docker; in fact I've been a big advocate for containerisation long before docker was a thing. However like with any tech, the key is using the right tool for the right job instead of attacking every screw with the same hammer.
You use dat or MaidSAFE to write client side apps. The back end is end to end encrypted, secure, automatically rebalanced, uncensorable, permissionless, and people install your app and use a cryptocurrency to pay for resources.
And of course you use public key cryptography to maintain your app and upgrades propagate without needing to select a domain or host for them.
Today you'd have to open up your cloud DB provider to the world since Zeit can't provide a list of IPs to whitelist. This is a showstopper for me unfortunately.
Functions makes life easier camp. Serverless development and deployments have nicer properties which make them easier to reason about and eliminate entire classes of errors.
Functions make edge computing possible camp. Serverless functions can be deployed in datacenters around the globe close to your users, offloading compute from your core and improving latency.
Now let's talk about DB semantics. You and many other people in the first camp probably want your business logic on strongly-consistent SQL transactions. That's good, it's the right semantics for the job. But it's incompatible with the edge model where the functions are decoupled from the central datastore.
So I think that you're asking for something the community isn't mature enough to provide yet. The momentum is towards unification where we need stratification (with respect to coupling).
It bugs me how many systems have opt-in correctness.
1. Most applications don't need 100% correctness. Few people are writing nuclear reactor control systems, or bank account management software
2. This model frees you to up to compute at the edge more.
Is there a chance that you consider something valid that eventually wasn't? Yes. But in a single actor-per interaction system (e.g. an single actor mutating something, multiple people seeing effects, etc.), typical to the web, this is OK.
People know about CAP theorem but hang onto the C with dear life. Let it go. It makes everything easier. Honest.
We like and use CosmosDB because it fits this criteria. We anticipate that Google Spanner, CockroachDB and similar databases will become the go-tos in combination with ZEIT Now.
We use existing technologies. Anyone can use any cloud Database service. We've datacenters on San Fransisco and in Belgium. So, based on those users can choose where they need to deploy their DBs.
Usually we recommend to configure databases via env variables. Users can also use our [now secrets](https://zeit.co/docs/getting-started/secrets) service as well to avoid hard-coding secrets.
Spanned databases are great, but most of the time performance is not there (It's getting there, but it needs to be there for a year before people start to care)
Transactional consistency and data normalisation - pffft.
SQL is still doing very well running things behind the scenes.
The problem is that it's possible, and very likely, there are exploits in the wild for your database server -- that are known but you failed to update for a day, or are 0 day exploits -- which are exploitable without having an authenticated account. Those issues can't be exploited if you firewall your database server to known IPs, but once you make it world accessible, all bets are off.
https://devcenter.heroku.com/articles/connecting-to-heroku-p...
1. You can't force MySQL to only accept connections over SSL. You can only enable SSL, and set specific accounts to only allow SSL logins. This means that any sort of "unauthenticated attack" on MySQL will work -- if you can exploit MySQL without a valid login, enabling SSL for users won't help you.
2. Amazon RDS supports using SSL connections, and will issue your MySQL server an SSL cert from their certificate authority, so your client can validate the server. It does not, however, support client SSL certificates, for the server to validate the client. Which means the only thing SSL connection is doing for you is encrypting the connection -- it's not in any way validating the client, and anyone can download the RDS region's CA certificate and then connect/exploit your MySQL connection normally.
You're suggesting that from the time a user makes a request to the function, the function should load up, and then create a client VPN connection to the database server, and then create a Database connection? Pretty sure that'd end up pushing towards the same cold start time. Also, no managed database provider is going to offer a VPN connection to it, so you're now definitely maintaining your own database server, when part of the point of getting into serverless is to not worry about servers.
https://github.com/zpnk/deploy.now/issues/27
Hopefully someone from Zeit reading this can get my fix merged, it seems to be quite a popular service
> Serverless means never having to "try turning it off and back on again"
> Serverless models completely remove this category of issues, ensuring that no request goes unserviced during the recycling, upgrading or scaling of an application, even when it encounters runtime errors.
> How Does Now Ensure This?
> Your deployment instances are constantly recycling and rotating. Because of the request-driven nature of scheduling execution, combined with limits such as maximum execution length, you avoid many common operational errors completely.
Somehow this sounds very expensive to me (like restarting Windows 2000 every hour just to avoid a BSoD, except that here it’s not that time consuming a process) and seems to leave aside caching, state management and other related requirements on the wayside for someone else to handle or recover from.
Or it’s likely that I’ve understood this wrong and that this can actually scale well for large, distributed apps of any kind. Sounds like magic if it’s that way.
> I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say "Yeah it works but you're leaking memory everywhere. Perhaps we should fix that." I’ll just restart Apache every 10 requests.
The crucial difference is that the activation cost of booting up a full OS (like Windows 2000) is massive (typically involving a chain-reaction of CPU and IO-intensive init services), whereas that is not the case for our serverless infrastructure.
What you do point out is one of the most important engineering challenges we had to solve (and continue to be focused on).
Configuration state is the bane of server management. Ideally you want to maintain state in a system specifically designed for it - i.e. a database, or version control software.
Looking forward to using your product!
1. AWS Fargate
2. Joyent Triton (which was as simple as DOCKER_HOST=<joyent-ip> docker run back in 2015)
3. hyper.sh (available since 2016 I think?)
4. OpenFaaS cloud (though you have to go through a couple more steps, you can bring a custom docker image as your 'function').
5. A metric ton of hosted kubernetes offerings
Looking at Now, it looks like you are billed by the 'plan' that you choose, and that decides how many deployment instances you are limited to. What does a deployment instance mean for something that is 'serverless'?
EDIT: Whoops, I see that there are 'on demand' prices for deployment instances too--now I just need to figure out how deployment instances map to serverless.
Here's my question, being relatively ignorant of Docker's internals: _is it possible_ to improve that docker create/docker start time from 300-400 ms (all in) to <100ms? 300-400ms is kind of a lot of latency for a cold boot still, and people still do things like keepalive pings to keep functions warm, so it would be pretty great to bring that down some more.
I take the view that there is a "warmth hierarchy" (an idea I spitballed with other folks in Knative before Matt Moore took it and filled it out a lot).
You have a request or message. You need to process it. The hierarchy runs CPU > Memory > Disk > Cold.
CPU-warm: immediately schedulable by the OS. Order of nanoseconds.
Memory-warm: in-memory but can't be immediately scheduled. Our reference case is processes that have been cgroup frozen. Order of tens of milliseconds.
Disk-warm: the necessary layers are all present on the node which is assigned to launch the process. Order of hundreds of milliseconds.
Cold start: nothing is present on the node which is assigned to launch the process. It will need to fetch some or all layers from a registry. Order of seconds to minutes.
Some of these problems can't be solved by Kubernetes (it doesn't know how to freeze a process), some of them can't be solved by Docker (it doesn't know about other nodes that might have warmer disk caches). Then there's the business of deciding where to route traffic, how to prewarm caches and based on what predictive approach, avoiding siloisation of processes due to feedback loops.
As these are knocked down they will pay dividends for everyone.
The main things are optimizations around pre-allocating network namespaces, we've built some experimental code to skirt around this https://github.com/fnproject/fn/blob/master/api/agent/driver... (which can be configured & tested) and we're testing approaches like this to attempt to get start times down to as low as we can. <100ms still seems like a lofty goal at this point, but this is becoming a pain point for various FaaS platforms that are container based and some optimization work should come out of all this. A few layers up, it's a little easier to speed things when throughput increases for a given function, but devs do seem to want the 0->1 case to be almost as fast as the 'hot' case so we FaaS people need to answer this better in general, it isn't something we want users to have to think about!
What are the other fundamental differences?
Different pricing, different ways of conceptualizing but from a user perspective that's a pretty good comparison.
Can we use "stateless" instead of "severless"? That seems like a better term.
Currently I have to make sure to set min instances to 1 on the 'production' version of the API and set min instances = 0 to older versions.
Will have to try before knowing for sure but I seems like switching to serverless docker would mean I don't have to make the distinction between 'old production' and 'new production' anymore and keep servicing requests to very old versions of my API without an expensive (7 seconds) cold boot.
Nice!
So, you don't scale up and down as you deploy. Once you call the `now alias` it'll take care of the scaling.
A few questions: How would one coordinate between multiple nodes running an app? (scenario in which the nodes cooperate to find each other. is there some sort of discovery available?) For the docker case, does it support health probes? (how you you know if the app is healthy?)
Because you give us your code under the constraint that it will expose data over a port, we have a much better safety baseline than stock schedulers, that operate under the assumption that the process describe by `CMD` is a black box.
We will describe this in more detail in coming weeks. One of our priorities is to minimize the cognitive load on the developer necessary to run a healthy service at scale.
> A new slot configuration property which defines the resource allocation in terms of CPU and Memory, defaulting to c.125-m512 (.125 of a vCPU and 512MB of memory)
Sure sounds like needing to be concerned about the hardware. That feels like a leaky abstraction that the serverless design pattern claims or appears to take care of, but seems like it doesn't in practice. Is "serverless" the right level of abstraction? I'm not sure.
For an simple web app, you might not need special CPU and Memory requirements. But for a video encoding/decoding serverless app may needs different CPU/Memory requirements (and GPU).
That's what this `slot` configuration property does.
The default is good enough for all the general purpose apps. But if you need more, you can control it.
This development was inevitable, especially since most Functions-as-a-service infrastructure was really docker (or some other) containers running in the background, spinning up on demand using a pool of servers. Azure Container Instances launched last year, and now Google Cloud Functions has serverless containers in alpha, along with the serverless addon for GKE/Kubernetes, and there should be a good bit of announcements this year as other providers follow.
The max time limits still seem pointless to me, there should be an option for unlimited uptime so the full spectrum of ephemeral function to entire app can be managed and deployed using a single flow.
Why would I want to switch to this? What are pros/cons of having Docker in there?
Can we call it "stateless"? Because there's theoretically no server state between requests?
1. Once your docker container is built, am I paying to store it on Zeit?
2. How is a Dockerfile versioned? Can I update it? Or do I need to redeploy.
3. Is pricing for containers granular by time or per request to a container?
4. How can these Dockerfiles talk to each other? Is there an API or method to fetch specific url's for each container?
Looks like a great idea, am curious to try it out sometime.
No. You don't need to pay to store containers.
> 2. How is a Dockerfile versioned? Can I update it? Or do I need to redeploy
Once the Dockerfile is built, you'll get a URL for that. It's a immutable URL and you don't need to re-deploy again. It'll sleep if there are no HTTP requests. If there are it'll start again.
> 3. Is pricing for containers granular by time or per request to a container?
It's based on the container running time. But we'll have a another blog post on this soon with more info.
> 4. How can these Dockerfiles talk to each other? Is there an API or method to fetch specific url's for each container?
There's no internal API to communicate with each other. But every docker container has a URL. You can use our [alias](https://zeit.co/docs/features/aliases) feature and communicate with each other via HTTP.
Not trying to shill, not everything should be serverless, and I should work to be more platform agnostic, but AWS has been great to and for me, AWS marketing material [2]
[1] https://medium.com/@amiram_26122/the-hidden-costs-of-serverl...
Maybe it simply leads to smaller teams and you save in employee costs.
[One of the differences is performance. You'll find that we can build and execute much more quickly :)]
There are many examples here: https://github.com/zeit/now-examples
Care to elaborate? Did you write your own container builder (for example like Google did with kaniko) that you use instead of docker? If not, I fail to see how you can build faster than me locally.
Yes. We build the container based on the Dockerfile.
I guess since the Zeit solution is based on Docker something like this should also be possible...
So where do your functions execute?
…or maybe that's “Cloud”.
Is there a clear explanation of what “serverless” means, and how it's different from “Cloud”, suitable for semi-technical non-developers?
I think "stateless" is a better term.
Serverless - "Re reimplemented/packaged a code runtime, and jerked off about marketing a bunch"
My Butt - "Run your shit on someone else's computer"
TBF, if the 'serverless' shit was sold as "run functions without needing to fiddle with an os", that's a decent selling point.
It's not server-less, tho.