Hacker News Reader

Comment by iainmerrick | Hacker News Reader

I can't quite tell what's being compared there -- just looks like several different LLMs?

To be clear, I'm suggesting that any specific format for "skills.md" is a red herring, and all you need to do is provide the LLM with good clear documentation.

A useful comparison would be between: a) make a carefully organised .skills/ folder, b) put the same info anywhere and just link to it from your top-level doc, c) just dump everything directly in the top-level doc.

My guess is that it's probably a good idea to break stuff out into separate sections, to avoid polluting the context with stuff you don't need; but the specific way you do that very likely isn't important at all. So (a) and (b) would perform about the same.

View on HN

Your skepticism is valid. Vercel ran a study where they said that skills underperform putting a docs index in AGENTS.md[0].

My guess is that the standardization is going to make its way into how the models are trained and Skills are eventually going to pull out ahead.

0: https://vercel.com/blog/agents-md-outperforms-skills-in-our-...

Agents add a docs index in context for skills, so this is an issue of finding that the current specific implementation of skills in Claude Code is suboptimal.

Their reasoning about it is also flawed. E.g. "No decision point. With AGENTS.md, there's no moment where the agent must decide "should I look this up?" The information is already present." - but this is exactly the case for skills too. The difference is just where in the context the information is, and how it is structured.

Having looked at their article, ironically I think the reason it works is that they likely force more information into context by giving the agent less information to work with:

Instead of having a description, which might convince the agent a given skill isn't relevant, their index is basically a list of vague filenames, forcing the agent to make a guess, and potentialy reading the wrong thing.

This is basically exactly what skills were added to avoid. But it will break if the description isn't precise enough. And it's perfectly possible that current tooling isn't aggressive enough about pruning detail that might tempt the agent to ignore relevant files.

The current tooling isn't aggressive enough in that it's not the first thing that the agent checks for when it is prompted, at least for claude code. Way more often than not, i remind the agent that the skill exists before it does anything. It's very rare that it will pick a skill unprompted. Which to me kind of defeats the purpose of skills, I mean if I have to tell the thing to go look somewhere, I'll just make any old document folder in any format and tell it to look there.

I had good success with this after tuning my triggers + main agent prompt.

I explicitly tell it about the skills and that it should load them when the context feels correct.

```prompt.md

Company Name codebase ...

# Skills

Use the company specific skills that look like `company-*`. Load them once per conversation if they seem relevant to what you are working on.

```

```SKILL.md

---

description: Company TypeScript libraries and conventions

trigger: Writing or reading TypeScript in Company services ---

# company-ts

```

I agree, but this is at least partly down to how well the descriptions are composed, because that is pretty much the only difference between a skill and what Vercel does. It might well be there's a need for changes to surrounding prompting from the tools as well, of course.

If the end user is not crafting skills “correctly” and they aren’t getting picked up as a result, then that tells me the ecosystem around skills detection and the documentation surrounding it needs work. Ultimately the interface presented to the end user is “put these skills in, Claude will look and decide if it needs them”, but no additional tooling warning or guidance if it doesn’t work as expected. It’s just silent failure. Pretty garbage UX

Exactly, many people seem to not understand that frontmatter’s description field needs to be longer „when?” Instead of shorter „what” - this is the only entry-point into the skill.

(not an expert)

I see this with Cursor all the time with tools. Cursor will stop editing files in the editor and use the command line to echo edits into a file. It's so frustrating.

For some reason I made my own skills framework.

The LLM instructed to run the initialization script as the first thing, before reasoning about the use request (this proved tricky to achieve). The scripts greps the content matter out of the skill files, along with the file path.

I have no clue if this outperforms an embedded index.

> To be clear, I'm suggesting that any specific format for "skills.md" is a red herring, and all you need to do is provide the LLM with good clear documentation.

Agent Skills isn't a spec for how information is presented to the model, its a spec whose consumer is the model harness, which might present information made available to it in the format to the model in different ways for different harnesses, or even in the same harness for different models or tasks, considering things like the number and size of the skill(s) available, the size of the model context, the purpose of the harness (is it for a narrow purpose agent where some of the skills are central to that purpose?), and user preference settings.

The site itself has two different main styles of integration for harnesses described ("tool based" and "filesystem based"), but those are more of a starting point for implementers that an exhaustive listing.

The idea is that skill authors don't need to know or care how the harness is presenting the information to the model.

Reply on news.ycombinator.com