Cortex: API platform for machine learning engineers

Maintainer here. There are a bunch of feature-level differences between Cortex and SageMaker (spot instance support on Cortex is one that comes up a lot), but the real core differences are:

1. Cortex is for ML engineering, SageMaker is for data science.

Cortex is designed specifically for the needs of production-scale ML applications. Deploying a model as an API with Cortex will be familiar to anyone who has used Serverless or Beanstalk before--you write your API in a Python script (Cortex provides a Predictor class for this), you configure your deployment with a YAML file, and you deploy with the Cortex CLI. You can see a GIF of a deployment on the repo: https://github.com/cortexlabs/cortex

In order to deploy the API, Cortex automates all the necessary cloud infrastructure. It spins up a k8s cluster, containerizes/deploys your API, configures autoscaling, implements a load balancer, streams logs, tracks predictions, handles rolling updates, implements different deployment strategies, cleans up, etc.

Because Cortex is just focused on serving models, not the data science/model training side, it can do all of the above automatically in ways that are familiar to software engineers (e.g. YAML/a CLI instead of notebooks/a Python SDK). This solves a lot of production challenges with ML, particularly around reliable deployment processes and collaboration.

Additionally, decoupling serving from training allows Cortex to be agnostic as to how you develop your model. You can use SageMaker, download a pre-trained model, whatever you like, so long as it can generate predictions.

2. Cortex is free and open source.

The obvious benefit here is cost--you don't pay the ~40% premium for EC2 instances that SageMaker charges. Additionally, we believe that there are fundamental benefits to open source infrastructure, in terms of transparency and control.