holoway

This is fabulous. It captures the feeling of being in the cathedral quite well - it fully transported me back to that spot. Great work!

It works great. If you think of it as 'dropping all the reasons we chose IaC', then yes - that's obviously dumb. If you think of it as 'getting all those benefits, plus faster feedback loops, AI agents, and an easier programming model' then.. not so much.

Yeah, in all sorts of ways. You can look at it in a Grid of components. You can look at it in a Map, seeing all the relationships. You can look at it via an API. You can have an AI Agent summarize it for you. It's super transparent.

Right - obviously, if you need the actual code deployed to run your test, there is not much anyone can do about that. But let me tell you how you would set that up, from scratch, in System Initiative (assuming you have a working deployment at all).

I assume the use case here is 'I want to deploy the application on every pull request to net-new infrastructure, then run my test suite, and destroy the test infrastructure once the PR is merged or the code is updated'.

You would fire up the AI Agent and ask it to discover an existing deployment of the application. Probably give it a hint or the boundaries you care about (stop at the network layer, for example - you probably don't want to deploy a net new VPC, subnets, or internet gateways). Once that's done, you'll have a model of the infrastructure for your application in System Initiative.

Then you'll turn that into a repeatable template component, by either asking the AI to do it for you, or selecting the related infrastructure in our Web UI and hitting 'T'. You'll add some attributes like 'version' to the template, and plumb them through to right spot in the code we generate for you.

Then you're going to call that code from our GitHub action on every PR, setting the name and the version number from the branch and the artifact version, naming the change set after the PR as well. You'll let the action apply the change set itself, which will then create the infrastructure.

The next step will be to run your tests against the infrastructure.

On merge you'll have another GitHub action that opens a change set and deletes the infrastructure you just created, so you don't waste any cash.

Notice what I didn't tell you to do - figure out how to create new state files, build new CI/CD pipelines, or anything else. Just started from the actual truth of what you already have, used our digital twins to make a repeatable template out of it, then told the platform to do it over and over again with an external API.

Hope that helps it make sense.

Nope. Terraform/OpenTofu state has several big differences.

The first is that Terraform/Tofu can drift. This is why people suffer when a change gets made outside of IaC, and the statefile no longer tracks. That's because IaC tools are by design unidirectional - change should only ever flow from the IaC to the Infrastructure. In SI, this is fine - the resource state can update, and then you can decide if it was beneficial (at which point we just update the component side of the equation, and you're done) or not (at which point you would decide what action to take to revert the change.)

The second is how it gets generated. In Terraform/Tofu, it's a side effect of the 'apply' phase - basically a compile time artifact. In System Initiative it's the heart of the system - the code you write is operating on that model, not generating that model. This makes programming it much simpler. You can change the model through our Web UI, you can change it through an API, you can change it with an AI Agent, the resource can change because the underlying cloud provider changes it, and it all just works.

For what it's worth, that means hitting the 'apply' button in System Initiative. It's a totally viable workflow - it's not either or, it's 'and'.

It's absolutely AWS focused today - but one upside of the approach is that building the models is straightforward, because we can build a pipeline that goes from the upstream's specification, augments it with documentation and validation, etc. We'll certainly be expanding coverage.

That's cool. For what it's worth, the software is all open source, precisely because it's critical in this way. I realize that's like telling you that you can take care of this puppy yourself if you want. :)

Even if you don't move away from IaC, you can still get benefits from the approach by having SI discover the results, and then do analysis.

You should check out the site again today - I think it will help at least at a high level of what it's like to use System Initiative today.

We didn't recreate the AWS API. Rather than think about it as the API calls, imagine it this way. You have a real resource, say an EC2 instance. It has tons of properties, like 'ImageId', 'InstanceType', or 'InstanceId'. Over the lifetime of that EC2 instance, some of those properties might change, usually because someone takes action on that instance - say to start, stop, or restart it. That gets reflected in the 'state' of the resource. If that resource changes, you can look at the state of it and update the resource (in what is a very straightforward operation most of the time.)

The 'digital twin' (what we call a component) is taking that exact same representation that AWS has, and making a mirror of it. Imagine it like a linked copy. Now, on that copy, you can set properties, propose actions, validate your input, apply your policy, etc. You can compare it to the (constantly evolving, perhaps) state of the real resource.

So we track the changes you make to the component, make sure they make sense, and then let you review everything you (or an AI agent) are proposing. Then when it comes time to actually apply those changes to the world, we do that for you directly.

A few other upsides of this approach. One is that we don't care how a change happens. If you change something outside of System Initiative, that's fine - the resource can update, and then you can look at the delta and decide if it's beneficial or not. Because we track changes over time, we can do things like replay those changes into open change sets - basically making sure any proposed changes you are making are always up to date with the real world.

And yes, there is no more IaC under the hood.

However! Folks with big IaC deployments can still use all the discovery and troubleshooting goodness, and then make the change however they want. System Initiative is fine either way.

Nope - I mean we make a 1:1 model of the real resource, and then let you propose changes to that data model. Rather than thinking of it like code in a file, think of it like having a live database that does bi-directional sync. The speedup in validating the change happens because we can run it on the data model, rather than on 'real' infrastructure.

Then we track the changes you make to that hypothetical model, and when you like it, apply the specific actions needed to make the real infrastructure conform. All the policy checking, pipeline processing, state file management, etc. is all streamlined.

Ryan can give you more details about his own experience. (I'm the CEO of System Initiative) But a lot of it comes from switching to a model where you work with an AI agent alongside digital twins of the infrastructure.

In particular, debugging speed improves because you can ask the agent questions like:

`I have a website running on ec2 that is not working. Make a plan to discover all the infrastructure components that could have an impact on why I can't reach it from a web browser, then troubleshoot the issue.`

And it will discover infrastructure, evaluate the configuration, and see if it can find the issue. Then it can make the fix in a simulation, humans can review it, and you're done. It handles all the audit trails, review, state, etc for you under the hood - so the actual closing of the troubleshooting loop happens much faster as well.

We're being intentionally pragmatic here. If you're building a digital twin of, say, an F1 car - the complexity of the simulator has to be very high. It's more like building a mock of physics than just the car.

With Infrasturcture, it turns out that what you need to know is "did I make a valid configuration", or "does this set of things work together". It's less about making a mock of the results, and more about simulating that the results would have the effect you think they will. So we can't tell you "will your application work on this size of instance" (although if you know that, you could encode that!) - but we can tell you if the options your setting are correct, if the AMI exists in the region, etc etc.

1. I'm sympathetic to your doubt - I had the exact same doubts, and part of why it's taken us so long to build is we refused to sacrifice what was good about IaC to get "simplicity". If this was ClickOps, they should be allergic to it. But it isn't. Under the hood is a very powerful new primitive - a reactive hypergraph of functions. The UI is there because it's a fast way to compose things together, and to radiate more information than you can get from an editor alone. But everything you do is tracked, it's fully open and extensible by writing code, and it's much easier to use to communicate with your team members who might not know all the intricacies of IaC.

2. There is how it is better today, and there is how it can be better in the future. Focusing on just today - frequently the kind of review that needs to be done is by an external subject matter expert. Being able to bring those people in to a change set, show them what the change you are proposing is, and have them inspect and alter it with you in real time is great. An example here is one of our early users wanted to use ECS, but had never used the service before. So they put things together in SI, asked someone who had that expertise to look it over - they could see the architecture, they could change properties, add a few missing things. It was much more straightforward than a back and forth in a PR.

But that's not to say that, in the future, there isn't more to do. We need to have more functionality around who needs to review things, build more specialized views for the review (there's no reason you should be stuck doing a review only in a single view of the architecture), use the snapshots we have of the entire graph to build more insightful ways of communicating what's changed (and what actions will happen when you apply.)

Think multiplayer and powerful review and approval semantics.

Thanks for the feedback! I appreciate it.

To your first question, adding what we call management components is high on the to-do list. They serve this role, in addition to letting you describe workflow across the other infrastructure they manage.

You add support directly in System Initiative, by writing the schema and functions in TypeScript. You can see how in the docs.

The resources can be periodically refeshed, and the data flows through all your open change sets automatically - kind of like an automatic rebase. This is also an area where we’re doing work - both to figure out the right intervals and to show drift more clearly.

The trick is that we’re not replacing programming with the UI - we’re just making it easier to compose things together. Plenty of code in SI - just not the way you write IaC today.

Yep! The application of Change Sets is a similar moment to record the "why", alongside all the information about what was done. We have more to do here, along with adding things like RBAC that requires review before change sets can be applied based on ownership of resources, etc. But this is the plan!

This worked in a previous version of SI (we've built like 4 different versions of it over the years) - but we haven't brough it back in to the current version yet. But we absolutely will!

You can't import terraform state right now. High on the road map (https://docs.systeminit.com/roadmap) is discovery of existing resources, starting with the import of individual things, and eventually building the component backwards from discovering the resources. We think that'll be better than importing terraform state, because it will essentially be a rolling reflection of the output, rather than trying to back into it from terraform.

But we're 100% open to importing it from the state file if that's what folks need.

Thanks! Let us know if you need anything.

It's a good question! Having a history of how things transformed over time matters. We keep track of that in a couple of ways - one is the graph snapshots themselves, and the other is a raw history log. It's really just another view of a snapshot of the graph at a particular time, and a delta between that graph and the current.

We have lots of planned work coming here - but we have a very rich dataset to do it from, and we're stoked to get there.

Rest assured, we'll have more competitive content over time. :)

All marketing is marketing that will land on flat ears. In the end, you'll have to try System Initiative, see if it is a fit for your use case today, and if it isn't, if it's worth paying attention to tomorrow. I wouldn't (and you shouldn't either) make a technology decision based on what anyone says on their website or blog. :)

Today the obvious drawbacks:

* Terraform has tons of coverage in their provider ecosystem, and we're not close to that yet.

* We have some enterprise features still to add.

* There is some work to be done around huge infrastructures, both in how to provide easy ways to visualize them and how we scale the underlying graphs.

https://docs.systeminit.com/roadmap/

We have plans for all these things, but it's early days. My advice (not just for SI) - you should always build representative prototypes if you want to understand what a technology might do for you. Your circumstances matter, and your problems are likely unique.

I mean, we didn't start there - we wound up there :)

But yeah, you're not wrong that it's got a lot of inspiration from things like mobx and rxjs.

Think about it this way - how do you update the IaC from multiple places? How do you visualize drift? IaC grows like code grows - it gets factored differently, it gets abstracted, etc. It's easy enough to imagine how you would update a single declaration, but thinking about how to make the "code" reactive will break your brain.

So what we do instead is have a reactive data model, and shift the code part to a reactive graph of functions.

I get it. System Initiative is 100% open source (https://github.com/systeminit/si) - if we go out of business, or you are dissatisfied with the product we offer, you can just take your workspace and keep using it.

We don't make a free distribution of System Initiative - but we expect that someone will eventually, and you could use that.

It's not a bad summary, but it is a shallow one. :)

You should love it because it's a more intuitive and more powerful way to build this kind of infrastructure automation. What's happening under the hood isn't just infrastructure as code with a UI - it's a full reactive model of how things work. That's what makes the UI possible, but it's also what brings about so much power - the code that drives those models is also fully exposed and versioned.

So when you have something like a policy to write, you think about what resources you need, use them as inputs to the function, and then store the results. Check out what an early user had to say about it: https://matthewsanabria.dev/posts/take-the-system-initiative...

We'll find out if you love it or not. :)

It's actually pretty good - usually the reason it's not accurate is because enough data isn't being fed to the simulator. That's one of the things that was great about doing it in SI - it wasn't hard to get the data in to the simulator.

But if I was AWS, I would also say you should check your IAM against the real world, because if you don't, it's pretty easy to wreck you environment. ;)

Just as soon as enough people use SI that we can have a conference.