The way you usually solve this is by having sane defaults, and giving users different mechanisms for configuration based on how complex their configuration needs are. This can take a tiered approach.
As an example, simpler and straight forward things (such as disable egress traffic from S3 if the bill exceeds X) can be in the UI itself. Then, for customers who need more control, an option to configure via json or yaml similar to cloud formation. For anyone who needs even more giving an option to call a customer defined lambda function would give them the ability to at any metrics and take appropriate action.