Azure Container Instances
azure.microsoft.com
azure.microsoft.com
This is a surprisingly unique product. AWS ECS and GKE both require some form of management of the underlying VM. A lot of that management is abstracted away, but not in the same way this is.
That being said, pricing [2] seems odd. At $0.0000125/GB_Second and $0.001/Core_Second, lets say you want to replicate an Azure A3 instance (4core/7gb/$130). That would cost... over $10,000/mo. Is my math right on this? It can't be.
For one-off jobs, maybe this makes sense, but as the backbone of a Kubernetes pool or something I'm not so sure.
[2] https://azure.microsoft.com/en-us/pricing/details/container-...
GAE is great, don't get me wrong. But it's so much more than just Dockerized app hosting, and it's not so great at doing just that. If hyper.sh has any half decent onboarding flow thought out, it should be a clear winner.
You should try the custom runtime again! It's come a long way since Beta.
Good point about the "getting it" with their docs. I suppose there's always that ethereal point with new concepts... It's just frustrating when it takes so long to get there. And, somehow, once you're there, it becomes incredibly hard to empathize with people who don't "get it" yet. Consequently, docs suffer.
As the old joke goes: Once you understand monads, you immediately lose the ability to explain them to others.
Anyway, good to hear they've improved and stabilized.
They also have open sourced their tech [1], so for certain companies they probably are compelling.
Perhaps they should provide the kind of ease-of-use on top of the stuff they already provide that will cut open a steady market for them, like a Heroku but with this underlying tech and thus more dynamic on scaling and flexible on tech stack.
[1] https://github.com/hyperhq/runv is one example
The vision is to merge PaaS into IaaS, e.g. a microservice-native cloud. I want to ask more details of the upper-layer features you imagine. Thanks!
- I start researching how to deploy this, but I don't care about optimizing for a lot of scale. I DO care a lot about iteration speed, integration with image registry + code repository + CI/testing services (ideally out of the box), and not making it hard to maintain/deploy/expand for the rest of the team (if an associate engineer can be brought up to speed on the infra and become fairly independent, that'd be wonderful).
- I probably care about burstable, pay-for-usage infra. But it'd be great if that was somewhat abstracted away. Perhaps even based on requests-per-minute, and I set up limits about how far it should scale. (This would immediately put this far and above Heroku -- good, cost-efficient auto scaling)
- I really care about not having to maintain my own DB cluster myself. And similarly for key/value storage and block-file-storage (like s3). And I'd absolutely love to not have to myself tweak and implement something common like cron or logging.
- And I'd probably care about integration with error-detection services and performance monitoring services.
- It would be absolutely amazing if there were default stacks/recipes with load balancers, so the common cases were easy (a common case like a standard RDBMS-backed JSON REST Api using cron and background jobs, or a static website (which basically covers all SPA-like frontends, if you add in a flexible reverse-proxy URL-rewriting solution))
Then, I could pick it up and be fine depending on it, since costs would scale down to my level (plus some premium I guess, but hey I'm even okay with Heroku for some things). But I could probably continue to stick with it for a long time. Inertia in ops means that if you get green field projects early and sustain them, my guess is that they stick around longer than they really even should -- but that's just an unverified guess on my part, idk.
ps -- the thing that really strongly drew me to hyper.sh was that I could abstract away the whole cluster behind `hyper`, like `docker` on my own computer. That was an amazing selling point, coming at after the popcorn machine of container management solutions with their very own intricate towers of complexity. It's what I like about sandstorm.io in part -- abstracting away a lot of complexity about hosting apps. But there's other complexity that could perhaps be eased or simplified, since running a software project is not simply the same as running an app on a computer, networking, load balancing, data security, backups and so on add up to a lot. So it's not enough that `hyper` abstracts away the data center, because there's scope for some more simplification for a certain slice of market: smaller teams who can't afford a full ops team, or teams with a small ops team who want to tackle a monstrous tech project.
Part of the reason we chose to open source is that we want the community to innovate. We are continuing to build the feature set, however I need to say that the workflow varies from app to app. Therefore, by providing the base building block and allowing others to create different solutions, we could enable more options to the market.
I could totally see even seeding a possible 'marketplace' with your own services. Like perhaps Func as a service, powered by Hyper. And others also benefiting from the underlying infra being so simple. Why wouldn't a platform-provider use an infra service that's flexibly scalable on perf+price? They probably have the skillset on hand for that too and don't need any more finesse than already exists, beyond stability.
Anyway, that's just my own ramblings. Hope you're having fun with Hyper! It sure looks like a lot of love/tears went into it.
"30 Create Requests * 300 seconds * 1 Core * $0.0000125 / GB-s = $0.1125"
0.0025 create + (0.0000125 1 core/second + 0.0000125 1GB memory/second) * 86400 seconds/day * 30 days/mo = $64.8025/mo
> You are charged $0.000013 for every Core-s used
So that's about 3x as much to not manage the VMs yourself in a cluster. For even a moderate cluster, that seems like it would rapidly stop making sense.
I'm one of cofounders of Hyper. Thanks for mentioning us!
I think both ECS and GKE are valid options their segment. Things like ACI and Hyper.sh is more of a container-native IaaS. The on-demand event-driven compute infra is a perfect use case, but it also works for long-running workload (k8s pool), where the difference is that developers only need to consume the k8s api, not the software itself.
> An Azure Container Instance is a single container that starts in seconds and is billed by the second. ACI offer highly versatile sizing, allowing you to select the exact amount of memory separate from the exact count of vCPUs...
az container create -g aci_grp --name nginx --image library/nginx --ip-address public –cpu 2 --memory 10
Because, previously, “serverless” seemed to mean not needing to deal with anything lower-level infrastructure than function calls (that is, a higher level of abstraction than even a classical PaaS like GAE managed runtimes), while container hosting, dynamically scalable or not, is somewhere between classic IaaS and classic application-language PaaS.
1. Invisible infrastructure
2. Microbilling
3. Event-based programming model
ACI offers 1 and 2 without forcing 3, thanks to containers.
We deploy these tasks across Kubernetes clusters on AWS, GCP, and Azure.
Since these tasks are schedule irregularly and are short lived, we had to do a lot of work to dynamically scale the nodes up a head of their demand and down after, and we typically have to pay for at least 10 minutes of usage no matter how quickly the job finishes.
This "pay-by-the-second" will be a huge win for us. Most of our tasks deal with S3/Redshift or GCS/BigQuery, do we can't immediately use this. But as we onboard more clients working with Azure Storage/Data Lake/Data Warehouse I see some big operational gains for us.
Here's hoping we see similar developments across the other major cloud providers. Very impressed with Azure's development in the last 3 years!
On the downsides, they're small and they have one data center, and they're not Microsoft. But their tech is open source.
While I appreciate the competition from GCE and Azure, what I really want is a tool that will run in any one of their clouds, but offers the same ease-of-management, and lets me go from one cloud to another or to a private cloud without breaking a sweat. I want the competition to be 70% on price and 20% on added-management-value and 10% on bundled services.
Terraform is basically this tool, but I want an even easier interface, terraform still feels somewhat too specific to me -- I don't want to even have to write config or specify some "aws" adapter that will make my config work on some provider. I want instant, multi-cloud (possibly) heroku, using only the network, hard-drive-space, ram, and lxc "primitives".
Someone (maybe me if I ever find time) just needs to get to work making F/OSS versions of all the bundled tech (ex. blob storage, cloud function runners, dynamically configurable DNS resolvers, simple alerting, etc) that runs in a container, and then the question just becomes "where can I get the cheapest most performant VPS that will host my containers".
but really this whole business of writing chef recipes and provisioning harnesses is really the same kind stuff. it seems important because you can't run without it, and thats what your whole day is...but really its pretty secondary to what you're actually trying to accomplish (run a service)
its interesting to think about what that world might look like...someone is going to make something like that stick at some point. so...why are people provisioning their own containers/vms instead of using the higher level services right now?
Compare this: http://www.ec2instances.info/?cost_duration=monthly
To VPS hosting from providers like: https://www.packet.net/bare-metal/ https://cc.delimiter.com/cart/dedicatedcore-vps/
The value provided by the services being managed is large, but honestly, for a lot of well-built infrastructure pieces, there is a lot of trust already for the services to not go down. Most startups/lifestyle companies/small businesses/whatever couldn't bring down a Postgres instance on a reasonably-provisioned machine if they tried (and the app is written with at least a smidgen of thought towards performance)
With respect, from what I can tell, most of the features of cloud66 are already included in Gitlab, and when I convert my current infrastructure to Kubernetes I can even start taking advantage of the auto-deploy-to-kubernetes features.
$49 / Month is a bit too much for me, but I don't think I'm the target anyway!
I'm also the kind that likes to run my own infrastructure and spend time on things I maybe shouldn't be spending time on :)
This sounds very similar to some ideas I've been stewing on over the past few months. One of them being a multi-cloud terraform-like tool which abstracts away the low level details of which provider an instance is provisioned onto (whichever is currently cheapest). It could also automatically determine how firewall/security groups/networking should be configured.
If you happen to create two instances in two different providers that need to communicate privately over say port 443, the security groups would be updated automatically to account for this, bridging the two providers.
One major thing to consider in doing this is the bandwidth. If you look at the fine print in these providers, the outgoing bandwidth is where they really get you. So if your backup server is in a different provider than your database, you might see some hefty data transfer fees while performing a daily backup.
Just curious, what's your background? Are you an infrastructure type, or are you a general developer who just wants a tool like you've described? Sometimes I can't tell if this is something people really want or if I've just drank too much of the infrastructure automation koolaid.
Sorry for the delay -- I've definitely had this at the back of my mind too, but with every advance made by cloud providers, I wonder if I could ever possibly get into that fight, especially since I'm not an expert at any of the cloud computing platforms -- Hashicorp seems to be doing amazing work in this area also, field seems to be full.
My background is basically that I really enjoy computers/programming and like to be self sufficient so I prefer a shallow (working) knowledge of all parts of the stack. I maintain a few VPSes and use them to host n-tier (where n is usually 2) projects. I'm always interested in the best way to do stuff and I use new projects to try stuff out.
Ops is particularly interesting to me because I've gone through a lot of iterations of how I did ops for my apps -- from SSH-in-and-do-stuff -> use-fabric-to-ssh-in-and-do-stuff -> dockerize & ship containers -> dockerized + systemd -> dockerize & registry (thanks to gitlab) + systemd + ansible. I've also used dokku and of course it blew my mind (I never used/wanted to pay for heroku), so while I stayed relatively low level, I was cognizant of what was happening a few levels up.
I saw kubernetes a few years ago and wasn't sure whether I wanted to go with it, but I've taken a fresh look over the last week and think it might be time to give it a try.
Side (controversial?) note - I think the time where it's acceptable for junior/mid-level developers to only know one part of the stack is dead. The only thing keeping these roles possible are the fungibility requirements for large corporations. Smaller, hungrier (almost literally) corporations can't afford to have some developer that is only an expert at writing python but never works on frontend code or deals with the database.
> I wonder if I could ever possibly get into that fight
I wonder this too. I've been using AWS for 4+ years at my day job(s). The main pain point for almost all users, including myself is the cost. I think a product/tool that helps companies save on cloud costs would be invaluable. The difficulty is convincing crusty old ops people that the product is something worth trying.
Anyway, after my current employer runs out of money, I'm going to do a startup of some sort. While one of my many idea was like the one we outlined above, I'm actually starting to shy away from the infra/op verticals lately.
My email's in my profile, if you ever want to bounce ideas.
Email incoming!
[1] https://github.com/coreos/tectonic-installer [2] https://coreos.com/tectonic
Right now my plan is to start with bare kubernetes just to cement understanding and then add rancher, before looking at any other higher level system (I mean there's stuff like helm out there, but it just seems to be too many levels above the ground for me at this point). I'm also using ansible right now (and relatively new to that too, but it's basically just fabric on steroids so I'm not too stressed about it), so I haven't started using terraform yet (but want to).
If you really want to draw a comparison with AWS product, it's pretty much a serverless EC2 ECS.
I'll give you an example - When I first discovered Lambda I had been working with containers for 6-8 months already and when I saw lambda it hit me that they just have container images capable of running python and node projects that this spin this up in and run the code we ask them to.
My mind immediately then went to our data pipeline that I'd like to run on a nightly schedule. This is typically done using a tool like Airflow or Luigi but I thought I could probably even just build the whole thing in lambda tasks that trigger via cron and SQS.
The reason we ultimately did not do that with Lambda was because it limited the amount of A. time and B. Compute resources one lambda task could use and we did not want to try to optimize for that when working with increasingly large amounts of data.
With something like this (ACI) I could actually do exactly this and even if its triggered by something like Airflow or Luigi I could run those on a much smaller instance and save the larger compute problems to being done on ACI in small spurts
(Disclosure: I'm a product manager on Azure Functions.)
Would be interesting to see a mashup of Azure Functions and ACI. In other words, use a event-driven trigger to spin up a container instance so you wouldn't be limited by the "serverless programming model".
You can try for free for up to 1 hour (no credit card required!) at https://functions.azure.com/try.
(I'm a product manager on Azure Functions)
The difference between it and other serverless offerings like Lambda and Functions is that you can run your jobs for upto 3 hours (after which we will restart it, not kill it, so if it is idempotent it will work ). It supports PowerShell/Python right now. Another core feature is being able to run scripts on your own VM from the Cloud called the Hybrid Worker. This has support both for Windows/Linux machines. https://docs.microsoft.com/en-us/azure/automation/automation... (Disclosure: I am a member of the Azure Automation team)
As far as I understand, with services like AWS ECS you need to provision the infrastructure first and pay for its uptime, whereas this allows more ephemeral containers to be run with minimal setup, and you only pay for the compute time used
Would only be useful for short lived jobs, but a really nice idea none the less.
The container security model will almost certainly improve in the future, but for now I'm only ok with other people in my same company sharing the kernel, not incentivised attackers.
[edit]
I'm going to unfud my comment. Some further reading makes me think maybe they spin up something like kvm containers and use a minimal distribution such that they can get to "seconds". If it were me, I'd have pre-running instances of the base image that were ready for a customer to attach and own.
Each container has hypervisor level isolation. We are not relying on kernel level isolation for security isolation between different user's containers.
Does anybody know if they mentioned anywhere what they use? LXC or Jails? Or some homegrown stuff?
Unikernels and a whole zoo of other types of tiny operating systems would be enabled by this.
I'm not a fan of containers - I feel they are reimplementing much of the operating system infrastructure within the OS at the price of high and unnecessary complexity.
It's frustrating that cloud computing has so many benefits, but at another level we must wait and hope that Amazon Google and Microsoft are willing to implement new architectures such as microsecond level boot and teardown.
I'm guessing it's powered heavily by kubernetes, so maybe that answers the question, but I'd be interested to know more about the details.
Thanks for showing up and weighing in on this! k8s connector looks really cool, is this a totally unique thing or are there anything comparable for ECS? I've never heard of a Kubernetes cluster with virtual nodes! Sounds like you could use this connector and potentially save yourself from ever needing to configure autoscaling in the Kube cluster.
I'm really curious how things are going at Microsoft for this incredibly productive team of people, from Deis, who have put out so much great software that has kept my attention. Hope that everything is great!
> The core second price at the top is incorrect. The per-second prices for cores and GBs are the same: $0.0000125. We are getting that fixed now.
A container has all the dependencies contained and has a private set of files and registry.
So no installation and no-side effects.
Also it prevents "server bloat", where a web or AD-LDAP server gets incidental utility roles attached, like running scripts or being a bastion/jump-box. You can't really RDP to the container, so you can't update DNS from there.
Is the API support for this coming?
https://github.com/Azure/aci-connector-k8s/blob/master/synch...
https://github.com/Azure/aci-connector-k8s/blob/master/aci.t...
Docs/SDK updates should roll out in the next 1-2 weeks.
https://github.com/Azure/azure-rest-api-specs/tree/current/s...
Please send us feedback.
I use containers for this, because it's a really easy way to really quickly get access to compute instances (which vary in size from like 4gb ram to 100gb+ of ram depending on what the user submitted) really quickly and for really cheaply because I only pay for the time it actually runs, rather than renting out an EC2/VM instance all the time (which would either be underpowered, or end up costing too much), or build some frankenstein-ian thing to dynamically provision and spin up EC2 instances on demand, set them up and then shut them down/terminate them once they finish.
Lots of weird issues.
You might want to go into some details about your issues, because a comment that vague is not helpful.