Terraform is really good at describing how infrastructure should be provisioned (VMs, load balancers, dns entries, networking, etc). Provisioning software on a VM and keeping it in a consistent state, however, is not something it's very good at. Userdata is very difficult to do anything complex with (limited size payloads, optimized for uploading a single shell script), and the provisioner system is explicitly described as a "last resort". This makes Terraform not so good at describing how software should be provisioned.

There is a bit of a movement, however, behind using it to deploy software by pairing it with Packer. You use Packer to create an e.g. AMI whose sole job it is to run your software (like a Docker container) then use Terraform to launch a bunch of EC2 instances that have juuuust enough resources to effectively run your software. That'd allow you to eliminate k8s from your stack, though it remains to be seen which stack would be more cost-efficient to run on.