This is cruft [1]. No one who is capable of using this needs it--it's a line in the prompt at most.
[1] https://knowyourmeme.com/memes/thinking-quickly-dave-constru...
[1] https://knowyourmeme.com/memes/thinking-quickly-dave-constru...
But it's very useful and skills is a good way to share promots, so whatever.
Try the skill or the prompt. It will likely improve your experience interacting with agents
- Create a skill to perform some common task
- Watch the agent use it
- Wait until the agent makes a mistake
- Ask it to update the skill to warn it against making that mistake again
I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly
One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescriptive. Often, something briefer and human-written actually works better
Another: I generally let the LLM propose edits to the skill, but I review them carefully and often modify them, because I find it has a tendency to solve the current problem at the price of worsening the solution to a previous one
Yep! I have a canned prompt that basically boils down to "scroll up, read through and give me the top `n` things that were difficult..." and more often than not a small skill or change to `agents.md` comes out of that.
Why do you think it can answer that? It has zero clue.
But in practice it does produce fairly reasonable output fairly often, and if you've been there watching and correcting it you can probably validate the result quite easily. "Re-read everything before" and similar are definitely anthropomorphizing, but that doesn't mean they're ineffective.
Incredibly so. All too often a “hey, you shouldn’t have done that here” turns into “NEVER X”, without it actually understanding or making an effort to understand why the correction was made, though I would say usually the context to do so should suffice. I have a rule for it to reason this through and use nuanced language and would say the frontier GPTs (5.4 to 5.6 Sol) get it right around half of the time (very rough estimate). The conclusions are really dumb sometimes.
I’d be interested in an expertly crafted skill here, maybe there are things about it you can still improve. This is an area in which I still find LLMs to be quite lacking.
It also made me appreciate the complexity of the nuances and levels of indirection of what I want to teach it: “Upload to the remote host, which in this case is this but might be another one in another use case, only do this on the local system and this on the remote, do this to X unless Y …, prefer doing Z unless another rule overrides it …”
I sometimes feel like there could be a more expressive way to structure to structure these rules (bring the snark!).
But also development instructions focusing on how and what to research before implementing (some of the stuff I do ends up heavily influenced by papers and publications while the naïve implementation is often bad, old, or gets stuck.
It makes it kind of funny publishing skills online because by the time somebody knows they want the skill, they already have it.
[1] https://microsoft.github.io/SkillOpt/docs/guideline.html