871 karma · joined September 2, 2016
My personal opinion is that AI and agents are being misrepresented… The amount of setup, guidance and testing that’s required to create smarter version of a form is insane.
At the moment my small test is: Compressed instructions (to fit within the 8k limit) 9 different types of policies to guide the agent (json) 3 actual documents outlining domain knowledge (json) 8 Topics (hint harvesting, guide rails, and the pieces of information prepared as adaptive cards for the user) 3 Tools (to allow for connectors)
The whole thing is as robust as I can make it but it still feels like a house of cards and I expect some random hiccup will cause a failure.
I've been going though my personal projects feature by feature. So far I've had good success, and as I'm doing it step by step I'm checking what's being created. 90% of the time it's correct and when bugs occur I can work through them and identify the issue, and then explain it to the agent to fix.
I don't think you could ever just set an agent off to create something by itself, unless you have a very detailed comprehensive technical document for it to follow along outlining the big picture and all details within - even then I think the context window wouldn't be enough and it may start tripping up.
The projects I've tried to date: - A love2D game (success) - Buildroot linux for an SBC with above game embedded (success, but with several issues related to the framebuffer, other drivers etc. Fixing this took about an hour of my time and burnt through all of the available thinking model tokens in two sessions. - A few offline web projects (ongoing, success when going feature by feature) - A micro controller project (ongoing)
"CRW's first-generation global monitoring products were operational at NOAA until April 30, 2020, when they were officially retired, and succeeded by CRW's next-generation operational daily monitoring products."
MS really needs to distinguish between them all.