Hacker News Reader

You're basically coming for my job as an SRE/Platarch but great documentation you've covered a lot of scenarios and in a pretty thorough manner.

You should figure out a way to import current infra state and control it from your dashboard and build off of that. That'd be really interesting. Like, take over a tfstate file or whatever pulumi/cdk uses. And definitely just build off of aws/gcp/azure/oracle without IAC like terraformer does.

Release engineer here. I believe you have hit on some of the ideas we've certainly discussed way down the road (or sooner than you'd think?). For example, can we inform the AI about your existing infrastructure patterns, tooling, configs, etc. and then ask it to "suggest best practices", "compare to security and compliance policies", "improve" it, "analyse optimisations", "identify unused/overprovisioned pieces", etc. etc.!

I'll throw you a big bone, I just cut our nearly $million/month aws infra costs down 45% by switching from x86 to arm64. So factor that in for sure. I was hoping for 15-25%.

Fantastic, we've already begun switching RDS instances to Arm64 (simple, easy, and effective), but still haven't cracked (yet) building for Arm64 compute containers in a safe and effective way. I love it!

What about arm64 makes building containers in a safe and effective way harder compared to x86?

The long history and dominant ecosystem on x86 makes containerization more seamless today. ARM64 support is evolving quickly but still has maturity gaps that can make safe and efficient container deployment more challenging. Careful testing and validation is required.

Exactly right but also in our case, we support single and multi-tenant first/second, and even third party support. That is, we need to support any workload from any customer or customer's customer at any moment now or in the future. We can easily say "if you have an x86 cluster, we guarantee 100% compatibility." But if we switch to ARM, we'd have to make exceptions and do double the testing on any existing workloads, as well as all the complications with building twice and doing releases and notes for each arch... It really does spin out of control quickly.

Wow. Is that a general web app or some highly specialized computing?

I would like to know that as well because last time I've read about this scenario, the x86 version of the app (IIRC golang app) was running almost twice as fast, so pound for pound wasn't worth the switch.

Speaking personally, the feeling is that for general compute like web servers and apis it is acceptable and will show a decrease in costs for the same performance. That is my understanding, and I am eager to try it.

I will check it out. Thanks for sharing.

What is a Platarch?

Platform Architect :)

As mentioned platform architect. I work on the research, design and construction of PaaS systems (which usually include SaaS as well, the backend of what lambdas or "functions" etc run on).

PaaS being Platform as a Service (think GKE/EKS/A...KS?)

How did you get into that originally ?

Not the person you asked, but I needed a place to run my shitty python scripts and the minimal requirement in my org was 'stand up an ec2 and get the OS managed by (insert other team)'

I complained about it and designed and built an MVP for a k8s SaaS in a week or so and pitched it to leadership and got some buy-in.

"Hello database, why are you slow?" /s

Serious feedback:

* Support Digital Ocean (currently only AWS supported)

* Even read-only access is scary... it's not clear if that means you have access to my sensitive user data or just the infra metadata.

Normally I would just include relevant info about my AWS infra in my prompt... how is this better than that?

Release engineer here.

- Support Digital Ocean: We are always looking to add more integrations! - Read-only access to user data: You are welcome to check on the policy and/or adjust it. We default in the recommended case to Read-Only, but you can improve on that with a permissions boundary - How is it better than gathering output from AWS, copy/pasting it into a browser, reading the answer and copy/pasting back into terminal? Well, it's all in one place and it allows a tighter loop between gathering, analysing, and giving responses in one place. It gives lay people better access to data they might not know how to or be able to access, and it gives advanced users smoother, tighter loops on data and access they might already have. Hopefully that gives you some ideas.

"Hello AWS, why is there this subnet and who added it?"

I'm seriously curious how is asking a chatbot better than looking at a well designed diagram...

You're very lucky if there's an up to date diagram and you got confidence in it's correctness.

Like a hand-made diagram? That's a blast from the past...

I mean a diagram generated directly from your actual infra using the same APIs as the chatbot

Are there existing tools today that can generate a diagram frim the output of the AWS API?

There's this: https://docs.aws.amazon.com/solutions/latest/workload-discov...

And also CloudMapper, CloudCraft, Hava, Lucid, Hyperglance, Cloudviz, and Cloudockit.

So, a resounding yes.

Not to say there couldn't be better tooling for infra diagram generation. I don't know anything open source and cross-provider, I wish someone worked on that. Hiding the grand overview from the user behind a bot with a fuzzy interface just seems counter productive.

FYI you may want to add something to the homepage to clarify - I was confused initially, because I clicked through to the Launch YC post, and then I clicked the URL at the top and saw the homepage, which talked about Environments as a Service rather than 'talking to your infra'.

Release engineer here, we see this as complementary to environments. When you spin up your infrastructure and environments using our service, how do you interact with them? How do you see what is happening? How can you (easily, as a lay person) figure out how to diagnose, maybe even fix, problems? That is the idea, hope you like it.

Done

I like the idea, but how do you handle hallucinations? E.g. when the user asks about their AWS bill, how can they be sure the numbers they get are accurate?

Would be good if any user interface that uses these kinds of llm solutions always include the raw data from the backing services when asked for. Like how in chatgpt you can open and inspect its interaction with a plugin.

Release engineer here. You can do a bunch of cool tricks like go into `/expert` mode and view all the details. Join our slack workspace (https://release-ai.slack.com) and we have tonnes more tricks and techniques to share!

Release engineer here. That's an excellent question, and we worry about it all the time. The AI "seems" authoritative, but it can't even add 1+1 sometimes :crying-emoji:. We've tried to engineer the prompts and tooling so that it will say "I don't know" if it doesn't know. But we've still seen it say some crazy things, like "Your cluster is fine" when it clearly wasn't. :tounge-sticking-out-emoji: I guess the only real answer is you have to trust but verify.

Thanks for the answer. Yeah, that's pretty much what I expected would be the case. Speaking as another dev in the AI space, it seems like reliability and consistency are the hardest issues when it comes to making AI genuinely useful in production vs. just a neat toy, and there's no stock solution.

Tommy, CEO here. We also have some ideas on reporting hallucinations and feeding wrong answers back into the prompts automatically to help reduce instances of hallucinations. We have a few other ideas and would welcome any ideas folks have to help with this problem.

After thinking about it for a bit, I have an idea that might help. The writeup is probably too long for an HN comment, though. Could I email you?

Of course. tommy@release.com

You need to engineer a system when the AI state something it has to give a command that should support what it says and explain how the command shows that it is true. At this point the command should be really executed and its output or error fed yo the AI so that it can confirm the statements or correct it.

I am crazy how they think a system with no feedback loop can be always accurate. Only perfect mathematics can work like this, any -like system need to have a feedback loop.

Excellent idea, we do internally feed the answers back to the system to improve its own inputs and outputs. The funniest part of some of this experience has been to find cases where even humans were hallucinating: "Hey, I thought this was shutdown?!" or "I can't find the bucket!" Even on a bad day, the humans are still ahead though.

How about applying good old fashioned bean counting?

> But we've still seen it say some crazy things, like "Your cluster is fine" when it clearly wasn't. :tounge-sticking-out-emoji:

It’s difficult to take you seriously when you write like this about show-stopping bugs.

I was referring to problems we found during initial development, but I appreciate that I didn't clarify that well.

I’m specifically referring to things like this:

> :tounge-sticking-out-emoji:

If you want to use emoji text like that when communicating about positive things, that’s one thing. Some people will find it grating, but that’s up to you. But if you use it to talk about negative things like bugs, that will piss people off. There’s a time and a place for emojis, and communicating bad news is definitely not one of them. It gives the very strong impression you don’t take bugs seriously.

I apologise for the poor choice on my part. I'll take everything you said to heart and learn from this experience.

"how much money am i wasting per minute on this huge disk just to achieve high iops?"

Relase AI Bot: "Try switching to gp3, you will save money and likely see a performance boost!"

Nice work! I'm working on a similar standalone DevOps AI Agent (OpsTower.ai). This post shows how the agent is structured and how it performs against a 40 question evaluation dataset: https://www.opstower.ai/2023-evaluating-ai-agents/

That's an impressive article and a lot of good work put into it!

Can i not use it without paying?

16:20:50 You: tell me about my infra

16:20:55 Error: Unexpected Error: err=Exception('got 400 from Release API for https://api.release.com/accounts/x/cloud_integrations'), type(err)=<class 'Exception'> Trace Id: 3f448f09-9cdd-489a-b59d-34f1fb03e3e3

16:21:10 You: what kind of databases am i using?

16:21:15 Error: Unexpected Error: err=Exception('got 400 from Release API for https://api.release.com/accounts/x/cloud_integrations'), type(err)=<class 'Exception'> Trace Id: 8d173a36-fd0a-42f6-b62d-0a4721423301

You can, sorry about that! This should be fixed now.

If you have any other problems please hop in the slack or shoot us an email! https://join.slack.com/t/release-ai/shared_invite/zt-20dxgp2...

Hey awesome work! How are you thinking about convincing CTOs/Heads of engineering to get their companies using this? I'm assuming those are the people you're talking to since devs can't just pick up and use this at work - lmk if I'm wrong.

Is it start with read-only, become SOC2 compliant and add more functionality from there or is it more nuanced than that?

Looks cool! A bit hesitant to try it out, though. Can you talk a bit about your privacy and security practices?

Do you (or do you plan to) use customer data (or prompts/results) for training? Do you ever read any of the customer's AWS data beyond what's strictly necessary for the functionality of the tool? What data do you retain?

I looked at your privacy policy but it's pretty generic.

I do the infra from our company. I _want_ them to train on our infra and everyone else's infra and to tell me what actually works and will solve my problem, not what my account manager recommends because AWS approve it. That's basically the value add of the tool.

Love this.

Exactly.

It would be interesting if just like an AI coding assistant, this would be a DevOPs/SRE/Infra assistant, whereby you can also just do your regular CI/CD/deployments/whatever - and have it make recommendations on caveats based on other architectures, or such.

e.g.

"describe examples for an architecture to accomplish X" and it spits out some examples - then you can choose, and save these "Infra-Prompts" and then later say

"generate a new sand-box based on X and connect it to such-and-such and notify USERA when their environment is available, include a status page, and alert cron and a costing line in the daily CFO report" etc

Couldn't agree more. Imagine if it could look at all the other 30-50 person companies who are running a SAAS and say "teams like yours use ECS + Fargate instead of Kubernetes" (as an example)

So is there already a "stack crawler" - or how does it know their infra?

I don't really know honestly, I'm just a proponent of "information sharing" in these spaces. I work with a few software providers that have a "give-back" clause for modifications, and it's a wildly powerful thing. I'd like to see it in this space too.

If I was implementing it, I would add some sort of survey a few days after the recommendations/queries are made and ask "Did this work, why/why not?", and attempt to train off of that. If you wanted to verify the accuracy, you could implement a test that ensures that things don't change wildly (e.g. if you tell it you're a small business running a SAAS on AWS that it doesn't recommend switching to Azure, or using EKS if you're already on ECS).

We need an AI as Infrastructure (TM) Conference, mayhaps.

-

It would be great to see AIAI Talks from folks on how well they leveraged AI for streamlining making the world a better, more equitable and connected safe-space place!

But how will it learn "what works" just based on different company's infra? I don't see how that would help an AI learn without things being labelled, and the answer can't be "lower latency means better". I'd actually be more trusting of the results of the current approach (GPT-4), because at least it's trained on every blog, forum response, and book ever.

> But how will it learn "what works" just based on different company's infra?

The same way as the current data was trained. I'm not expecting magic, I'm expecting help.

> because at least it's trained on every blog, forum response, and book ever.

Random blog posts that people write as examples as they're learning how to structure their infrastructure is not what I want to mirror our infrastructure on. I don't blindly trust the AWS documentation, I don't blindly trust a blog post, so why will I blindly trust an AI tool?

... You seriously just tell people to deploy a stack with your IAM role that "has the least permissions possible" good luck lol. Another thin wrapper around open-ai that doesn't even do the heavy lifting (deploying a secure and trusted authenticated role to access resources that can be easily audited etc)

For what it’s worth, literally every vendor that operates in its customers’ AWS accounts does this. Create a cross account role, trust the vendor’s account, and give it read only permissions (although don’t use the built in “read only” role since that includes access to things like S3 objects.

At my current job I can trigger a pipeline to deploy a complex set of cloud formation stacks but my account doesn’t have access to logs of those deployments. Devops!

Release engineer here. You can pretty easily audit the policy we use (Read-Only) and you can also add a permissions boundary if you want. We would love to get any feedback and improvements you can offer if you are inclined. We have a slack workspace users can join, check it out. https://release-ai.slack.com

Congratulations on the launch. I own and run a DevOps consulting company and being able to ask AWS billing questions or create CloudWatch notifications in human language is gonna be a game changer. Looking forward to using and following Releases progress. The .com domain name must have cost a fortune. lol

I like the “talk to kubernetes” demo I can see that being useful and perhaps a new normal for command line interactions with tools. I wonder if it can help with security hardening recommendations? You might be able to do that in tandem with aqua security or similar.

I like the concept, but would love to hear if it's a feature or some long-term product vision.

It looks like the main business is Environment as a Service.

Do you see ReleaseAI as a new product? Is it related to Env as a service?

Or maybe you are considering to pivot?

Tommy, CEO here. We believe it is incredibly complimentary to our core service. What we've found in building Release through our support channels is we spend a tremendous amount of time consulting with the DevOps teams of our customers answering questions like what can be answered with ReleaseAI. It helps us help ourselves and our customers. In the long run we see this being integrated into our core product in really interesting ways as well. There will always be a standalone version.

Really curious to know what led you to this project? When ChatGPT4 released did Bella start choking in your head with all of these possibilities?

What is Bella a reference to?

We spend a ton of time working directly with DevOps teams of our customers, helping them in using Release, but also just a lot of questions related to their infrastructure and AWS/GCP. We have some crazy good Integration Engineers that field a lot of this and we thought maybe AI could help free them up and scale That’s what sent us down this path.

Sorry, Dave, I cannot allow you to do that.

Thanks for all of the feedback. I would love to get everyones ideas on what they would want Release AI to do for them. Please reply in the comments.

I've worked with Dave before. He will in fact implement ask the things in a very short period of time. We used to talk about him as the master of spinning developer plates.

does your entire business depend on the OpenAI API? what happens if they close it down?

Release engineer here. No, this tool merely complements our core offerings and also has some use as a standalone tool. We can currently use any of the AI API offerings that are compatible; OpenAI is the one that is most common though. If they shut it down, a lot of people would have to migrate to a different provider but our tool could still work.

cool thanks for the reply

Can't wait to try it!

Weekend project!

Love it.

Funny, while I was reading the site, I was wondering if you could build crons on it - and that was the last example :-)

Some cool things I'd like to try would be:

"Create an alert whenever [EVENT] happens and contact [whomever]"

"Give me a system summary every morning at 9:15 AM called "Stand-up Status"

"Summarize all events for [OBJECT] each [TIME PERIOD] and send a report daily at X"

"Give the CFO a daily spend report each morning"

-

Can it do things like this?

Release engineer here.

With the exception of the first example (I'm not sure how good it would be at event monitoring like that, though we're absolutely going to try it out!), I think all of these should work. We've tested the daily spend report already.

Give it a shot and let us know how it works for you!

For the first one, maybe an Intrusion Detection

"Alert me anytime someon attempts to login to [SENSITIVE SYSTEM]

-

Can this do SPLUNK like log analytics?

"Give me a table of all activity of [TYPE] in [THESE LOGS/SYSTEMS]" (for whatever metric youre tracking)

"Give me a cron of uptime every hour for [system, site, cluster, whatever]"

"create a status page for critical systems A B C X Y Z"

--

I dont have any AWS infra to throw this at right now - but I do love this

As a Dir. Of DevOps in my career - these were very common questions thrown at me on the regular from PMs, C-suite, engineering etc...

So if this were a self-service query portal for teams with permissions/roles on what sort of questions could be asked from other teams that would be cool.

this is read only? It cant deploy/launch/buy services can it?

Since this is currently talking to AWS and k8s directly, unless you are setup in a way that would let AWS know about the intrusion detection, then this is likely out of scope for now.

Similar to the logs/systems access. If AWS or k8s can read the logs, there is a chance we can crunch them, if they are in a separate logging platform, we would currently be unable to fetch that information. Great ideas for future features though!

It is currently read only yes.

re:splunk (and opensearch, databricks, etc), we're already doing those with louie.ai and running early self-hosted + saas cohorts. Your questions are very much the type we'd interested in exploring with you! Feel free to signup on our early access program on the site or reach out directly (leo@graphistry.com).

I cannot tell how much this company can change your world in this space. Efficiency is paramount in all we do in life and work.

Let's get Release wider on the map.

Show HN: Release AI – Talk to Your Infrastructure