Hacker News Reader

Comment by orf | Hacker News Reader

Not to be too negative, but:

> When modeling AWS IAM policy in System Initiative, we realized that AWS provides a sophisticated Policy Simulator. So we modeled it, connected our IAM Policies and resources to it, and had a new, real time interface to test the validity of IAM policy. It took less than an hour from start to finish.

Clicking the link takes you to the docs on policy simulator, which seems to show it’s quite limited and isn’t representative of actual, deployed IAM rules:

> Important:

> The policy simulator results can differ from your live AWS environment. We recommend that you check your policies against your live AWS environment after testing using the policy simulator to confirm that you have the desired results.

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_poli...

View on HN

It's actually pretty good - usually the reason it's not accurate is because enough data isn't being fed to the simulator. That's one of the things that was great about doing it in SI - it wasn't hard to get the data in to the simulator.

But if I was AWS, I would also say you should check your IAM against the real world, because if you don't, it's pretty easy to wreck you environment. ;)

For other resources that don't have a simulator provided by AWS (e.g. an EC2 instance) how do you ensure that your simulation is accurate? How do you keep the simulations up to date?

You write a function. As an example, here's one way to check if you have a valid AWS credential

``` async function main(component: Input): Promise < Output > { const authCheck = await siExec.waitUntilEnd("aws", [ "sts", "get-caller-identity", "--region", "us-east-1" ]);

    if (authCheck.exitCode === 0) {
        return {
            result: "success",
            message: 'Credentials are Valid'
        };
    }

    return {
        result: "failure",
        message: 'Credentials are invalid. Please check the credentials set on the secret/credentials prop!'
    };

} ```

Checking the validity of an STS token requires that you have a bona fide token first. Thus it’s not a “digital twin”; it is the real McCoy. And STS tokens are free.

There is no digital twin I’m aware of that is capable of simulating the real behavior of an EC2 instance. There are just too many variables to consider. To test instance launch and runtime behavior to a meaningful degree of certainty, you have to launch one first. And that means accepting the costs of doing that.

(I notice, too, that you appear to be executing the AWS CLI to do this. I’m not sure if that’s bad or not, but it smells a little fishy.)

It’s terribly slow, given that it’s starting an entire Python process, configuring boto3, etc. that’s 2 seconds on my machine, just to run —help. And it’s all to make a single HTTP request (80ms)

Not sure if this is just pseudocode though.

You can use fetch if you want to call the API directly. It is just an example.

Yeah I figured! I like the design, it’s pretty cool.

We're being intentionally pragmatic here. If you're building a digital twin of, say, an F1 car - the complexity of the simulator has to be very high. It's more like building a mock of physics than just the car.

With Infrasturcture, it turns out that what you need to know is "did I make a valid configuration", or "does this set of things work together". It's less about making a mock of the results, and more about simulating that the results would have the effect you think they will. So we can't tell you "will your application work on this size of instance" (although if you know that, you could encode that!) - but we can tell you if the options your setting are correct, if the AMI exists in the region, etc etc.

If you need any help, you can reach out to me at pravanjan@palette69.design

Reply on news.ycombinator.com