I paid people to try and follow my README | Hacker News Reader

I paid people to try and follow my README

shkspr.mobi

426 pointsedent205 comments

> But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.

I sometimes write readmes for myself, so that I can remember the exact steps to generate a data report, etc.

It's surprising how much they become incomprehensible after just a couple of weeks; when everything's in our head it's all clear, fluid and self-explanatory; but once we have forgotten the context, nothing makes sense anymore.

Most people don't think linguistically, we have way more conceptual thinking. What makes it difficult is we rarely realize we are thinking conceptually when writing because these concepts occur automatically and we don't notice it.

In school this lead to a lot of difficulties for me, one in writing those comments out for other people to understand, but the other seemed to be that when reading the average statements used in education for teaching I could map the same strings of language to multiple and sometimes conflicting statements because of the inexactness of the language used.

It turns out explaining concepts while leaving little room for different interpretation is hard.

> explaining concepts while leaving little room for different interpretation is hard

To play devil's advocate: explaining concepts while leaving little room for different interpretation is also pointless. If you don't care about the interlocutor's interpretation, then why are you even talking to them? If the task is really deterministic, then automate it.

Because the Junior eventually grows up to become a Senior, and they will be responsible for maintaining said automation.

Also, if it’s truly interpretive it leads to a “bus factor” of one, where somebody going on vacation or whatever represents a real timeline risk.

School, and specifically learning is not deterministic.

While we can automate away most things, human learning doesn't seem to be one of them.

Yea, that's what I was trying to get at. It is not deterministic because the interpretation is the point. It is how the learner integrates the material. If you don't interpret, then you're not learning, you're just memorizing.

A teacher should therefore embrace the fuzzy nature of explaining concepts.

With humans it is a mix of both. Learning just the algorithm is hard, you need some amount of initial data to conceptualize, and this amount of data can vary pretty greatly between people.

You have to learn the rules before you know which ones can be bent and which ones can be broken.

The problem I have is that when I'm asked to follow a procedure that you know just while reading it was created by someone with less experience and better/faster/cheaper ways are available to get to the same result. Deviation will however get you into trouble because the procedure is what is vetted and approved. Deviations from procedure could open one up to liability if things later do not work as expected.

To me that indicates a cutoff point though.

If you absolutely know you can do it better. Then the procedure needs updating.

But the procedure might be what it is because of externalities relying on artifacts of the way this is done. Which to you is an "unknown unknown".

As someone who does think linguistically, I often end up at odds with people who are expecting me to imply things in my text, when my text is written specifically to convey exactly what I mean and nothing further.

Yes 100%.

There is no undercurrent or symbolism, just read the email as written please.

I became aphantasic at ~15 and spent years after interfacing with non tech people over the phone and email.

In English it's so damn hard to be precise compared to languages like Portuguese.

Also, LLMs blather so fucking much context is impossible to track wtf they are even writing about.

* Fast screenshotting and arrows/drawijg should be first class on all OSes.

I've never heard of becoming aphantasic, how did that happen?

I haven't either, but two things spring to mind (because I'm not OP). 1. Head trauma. 2. Adult onset diabetes.

It can probably happen.

There are studied cases involving brain trauma like strokes which introduce aphantasia.

https://pmc.ncbi.nlm.nih.gov/articles/PMC7071355/

> While the loss of mental imagery following brain lesions was first described more than a century ago, the key cerebral areas involved remain elusive. Here we report neuropsychological data from an architect (PL518) who lost his ability for visual imagery following a bilateral posterior cerebral artery (PCA) stroke. We compare his profile to three other patients with bilateral PCA stroke and another architect with a large PCA lesion confined to the right hemisphere. We also compare structural images of their lesions, aiming to delineate cerebral areas selectively lesioned in acquired aphantasia. When comparing the neuropsychological profile and structural magnetic resonance imaging (MRI) for the aphantasic architect PL518 to patients with either a comparable background (an architect) or bilateral PCA lesions, we find: (1) there is a large overlap of cognitive deficits between patients, with the very notable exception of aphantasia which only occurs in PL518, and (2) there is large overlap of the patients’ lesions. The only areas of selective lesion in PL518 is a small patch in the left fusiform gyrus as well as part of the right lingual gyrus. We suggest that these areas, and perhaps in particular the region in the left fusiform gyrus, play an important role in the cerebral network involved in visual imagery.

I’m curious about your statement that you “became” aphantasic. Any specific triggering event, or how did you realize this?

AFAIK aphantasia hasn’t been super widely known about until the last decade or two (obv I have no idea how old you are now)

I learned about it in high school, and that was 80s/90s but not by name per se. I took the ASVAB and there were questions like "fold a piece of paper in half twice, punch holes in a pattern, and then pick which image would be the result when the paper is unfolded" that I thought were ridiculously easy. I was told that they were difficult for people that cannot form mental images and these help test that ability.

> It turns out explaining concepts while leaving little room for different interpretation is hard.

This pops up as one of the annoying aspects of LLM writing, at least for coding models. They are always saying things like "this test only establishes this, not that." Much of the time, what they say is either obvious, or you get to thinking "maybe the test should establish the other thing as well."

My boss gives me shit regularly for not remembering things. He doesn’t seem to understand that when you manage environments in all three clouds, storage arrays in four different countries from different manufacturers, four on prem virtual clusters, Active Directory, entra, etc that you can’t remember everything all the time if you haven’t touched it in a while.

I started a daily journal when my team’s workload got to be so much that we can’t remember everything. It helps, but even going back to it weeks later there were things I did not write down because I assumed I’d remember them later.

I’ve since gotten better at being more comprehensive, and trying to think in the “how to make a sandwich” way of instruction. I’m not being condescending to my future self, I know my future self has too much shit to mange to remember it all.

You should take this up with someone in the org you trust

So long as you know where to look and can quickly check, not remembering is OK. If your boss wants you to remember everything of the top of your head, he's being a knobhead.

"All three clouds" is such a weird phrase.

There is no "cloud". There is other peoples hard drives.

But let's pretend the "clouds" you use are "clouds". Ok. What makes the other hosting providers not "clouds"? Bit of a no true Scotsman.

This useless and obnoxious criticism actually ties into TFA a bit: when GP was writing for this audience, they wrote in a way that wasn't overly precise, but specific enough that we'd understand their point.

This comment actually ties into the GP's point about how when you get further from the context, not being overly precise will lead us to eventually not understanding that part of the point.

That's why having a long shell history is so great, you can just scroll back further to get more context, because it really captured everything.

Transcript

Your comment needs some documentation! ;oP

I find that writing this documentation is both a "muscle" and gets better with experience.

e.g. as you both write documentation and see yourself or others use it, you start to get a feel for what people tend to understand and how to communicate it.

You can also do "dog fooding" where one person or group writes the docs and then other people follow them. If you iterate on this quickly, you can get to really good docs in a short amount of time.

My new test is to ask an LLM questions about the documentation and see if it gets correct answers. I wonder if this will become a common QA step for the docs at some point, it's very useful for finding gaps in the docs or things that are unclear.

The one thing I have found that really helps here, for self-directed documentation, is to write it for a modified version of yourself.

You may still be the audience. But you will be four or five years older, shit will have gone on in your life, you will have more to remember, you will be tired, you will have less patience, you will resent being forced to do archaeology on yourself, and have a dim view of the irresponsible young scamp who thinks he has an excellent memory that you are right now.

Write for that person and your documentation will be better.

(As you may be able to tell, I am now that person. And I fear there are two more cycles of this to go)

I just went and looked at the first equivalent documentation that I wrote to myself in 2009 explaining how to automate mounting disks via fstab. It's very complete. I was tripped up on documentation that I wrote to myself in the past year only a week ago because my understanding of *nix systems now causes blindspots when writing to future-me.

>I hadn't actually explained what the software would do.

Half the posts I see lately are like, "Gleam 2.0. What we learned" and then you go to the homepage and it's "Gleam is a Tribble for your Fork! (Scroll down) See if you qualify for Gleam Enterprise!"

I hate marketing websites especially for developer products.

“You no longer have to florp, now you can vorp!”

Big numbers, random charts. 10x 100x 200x!

Who is this for? Just make the docs the home page.

The best explanation I have heard is that these pages exist for the finance team that has to go try to understand what the product is that they are being asked to authorize a purchase for.

So what comes across as extremely vague, unclear, and perhaps obfuscatory, is actually somewhat properly targeted to someone who needs to decide "is this an appropriate class of purchase for a dev team." They don't need to decide if it's the best technology for the purpose, just that it is a technology for that purpose.

This is especially clear on every single one of the AWS technology top hits. Clearly the product is so vaguely described that an actual user gleams zero usable information on whether it will solve the task they have at hand, or what the capabilities are.

Pricing:

Individual - free Pro - $20/mo Business - $50/mo (mysteriously the same as Pro) Enterprise - contact us

I was confused by your mention of "Gleam", but I suppose it is just a generic name, not referring to Gleam [0], the functional programming language, as the latter is an example of a project that focuses on concise and minimalist documentation (btw, its README is just a reference to the website).

[0] https://gleam.run/

(Updated the text as the downvotes indicated people misunderstood what I wrote. I agree about OP's observation)

The generated confusion ironically supports OP's point

It’s never irony

It’s a bit funny how normalized this is. Surely being more clear would provide some market advantage?

Or perhaps not, if folks who need a Tribble for their Fork achieve instant enlightenment upon reading the marketing copy.

I recently had this experience with hotjar [0]. I _already use hotjar_ and still struggled to understand the page lol

If somebody from hotjar/cintentsquare sees this, I’d prefer if your page was more straightforward about the name change. Maybe try “we’ve rebranded to content square”

[0] https://contentsquare.com/hotjar/

It also happens with marketing emails, or (worse) waiting lists: "great news, we're live!" -- Who the heck are you again?

Now imagine living in San Francisco where you see this shit on every bus ad and billboard. Except these days they’re all “Tribbling. With agents. gleam.ai”

That's true.

But, I also see a lot of posts that have comments like, "How am I supposed to know what Gleam does when I don't know what a Tribble is? The blog doesn't explain anything! How can someone write an article and not explain these terms that they use so much?"

And the linked blog article is hosted in the Daily Tribble News section of www.tribbleworld.com.

And cudos to OP, because I'm not into Mastodon but after reading the first section only I understand what his app does.

finally someone mentioned this

I wholeheartedly hate that useless infantile names are so normalised in software industry, even libraries are as such, which at some point makes it just insane

"Gleam Is Now on Tangled": https://news.ycombinator.com/item?id=48959143

This is very close to what you call usability testing in the field of UX design.

The Nielson Norman Group has a good introduction to this: https://www.nngroup.com/articles/usability-testing-101/

Is this interesting to people on HN?

I majored in a mix between coding and design.

Yes very much so and thanks for the link.

I think I read somewhere (and in any case it matches my experience) that the first user will catch 50% of the problems and that with 5 users you can catch close to 90%. So human testing is not just invaluable, it's pretty cheap for the benefits it brings.

Its not just QA and bug hunting.

You want to hire your target audience, which may be vastly different from yourself, and suddenly you discover that the language is throwing them off, the color scheme brings different meanings, and they just don't understand the flow which felt completely natural for you.

Probably you learned that from them! The first user is 31%, from their testing.

https://www.nngroup.com/articles/why-you-only-need-to-test-w...

https://www.nngroup.com/articles/how-many-test-users/

Yes, very probably, thanks!

At every UX job I've had we generally ran usability tests with 6 participants. This was before the online usability testing tools proliferated and we had a lab in our office (one way mirror with observation room and everything). Every UX researcher I worked with said based on the available research and personal experience it is as you said: more than 5-6 participants would hit diminishing returns pretty hard and wasn't really worth it.

It's more like the first user catches all of the critical bugs and a handful of users catch all of the high impact bugs. Cause that's kinda the definition of the impact of critical and high impact bugs.

I was also going to post "this sounds like usability/user research".

But I guess we all re-discover things when we need to.

The biggest issue I've seen with tech docs is often expert users of a given tool or product are enlisted to write the docs, because of their expertise.

But then ironically they end up writing those docs for an audience that shares their level of expertise, rather than for the intended audience.

So you end up with lots of assumptions or leaps of logic in the docs that the intended audience can't follow.

I'd wager this is interesting, seems there are a lot of users here building things on the side (or main projects) that need clear UX if they're anything like me, they don't have that UX background to know the best practices. I know I just clicked your link anyway...

There's the famous quip that Steve Jobs hated focus groups. What people miss is that he (and Apple overall) was heavy on UX testing.

I love this. Most readmes are plain bad. I think the most egregious is when a readme doesn't state what the project does. I get that not every project is aimed at the public, but if you go through the bother of creating a readme file, why not go the extra 10 centimeters by writing the most basic information? Other issues: * outdated (and thereby wrong) information * using un-introduced abbreviations (bonus points for abbreviations that have common meanings, e.g.: EG, NB, IE, ETC)

> go the extra 10 cm

Going to steal this

Could have used extra 1 centimeter to format the comment's list properly. :)

> using un-introduced abbreviations (bonus points for abbreviations that have common meanings, e.g.: EG, NB, IE, ETC)

Every company should give their new employees a list of in-company invented words and abbreviations, so that you don't search for them online and then feel like an idiot for not being able to find them. Especially when the older employees use them as if they are common knowledge.

There are a number of (newish) AI projects on GitHub whose README make zero sense. I would read it three times but still have no idea what it does.

If your project is aimed at average developers yet someone with professional software engineering experience like me cannot understand it, sorry I'm not going to use it.

> I hadn't actually explained what the software would do.

I cannot tell you how many READMEs I've read that follow the pattern: "<uninformative-name> is a <buzzword> <buzzword> written in <language>." I only have some semblance of what it does after using/seeing a demo; too often one that isnt available through the README.

The other variant is "Foo is an alternative to bar". While it could work in the niche you are in and it is totally fine to not target people outside of it, it could be really hard to get into.

Yeah. It was infuriating trying to figure out Minecraft Java mods because they are all like that.

> My jokes aren't funny and are actively confusing.

I used to write technical documents in prose style, sometimes with meandering stories. I guess I picked it up from my early blogging days. I realized I hated reading some of them back. So I tried to keep it cut and dried. I do sometimes sprinkle a bit of colorful wording just to add a bit of humanity but only if it doesn't get in the way of the main message.

Being short and concise is usually the better way.

I tend to be too verbose in writing as I want to explain the context in more detail, but short, accurate, concise is best. Many developers fail at that too, though. Many projects do not have working examples. That annoys me the most. It sends a message of "I don't care about new users learning how to use my project".

> Being short and concise is usually the better way.

The golang docs are like this. As a novice, you are looking for detailed prose, but as you progress, you come to appreciate the terseness.

Same for me. What helped is the realization that I was trying to cater to everyone in the same document. Now I try to follow the organization outlined in https://diataxis.fr/ I’m still very bad at documentation in general but I’m less dissatisfied when I come back a few months later.

I had diataxis in mind when I wrote my comment. It's an important and excellent guideline on picking the style based on purpose of the doc.

I started using diataxis for all my docs a while ago and I've never gone back to any other kind of documentation framework. In addition, all docs that do not follow this framework makes me really sweaty.

>"I don't care about new users learning how to use my project"

I published something the other day with minimal instructions, and felt briefly conflicted.

But I figured, if you want to run it, you'll find a way! (It probably doesn't even work on other operating systems, but porting it would take what, 20 seconds of Codexing?) It was true before AI, and it's definitely true now.

My intended audience is people who want to get their hands dirty. Though I suppose these days, that's the machine's job...

isnt this what footnotes are for?

> My jokes aren't funny and are actively confusing.

Well, I found that sentence in the post was very funny ;-)

When deciding on a style for documentation, I typically draw the line between tutorials and references. The linear top-down flow of an introductory guide lends itself well to inserting additional context throughout it even if not completely on-topic, while in an API or hardware reference you generally want to keep each section reasonably self-contained, trivially searchable for (minimizing false hits by carefully choosing keywords) and readable independently of the others. I have found the literate programming approach [1] of writing entire tutorials as code to work pretty well for this purpose, which I've used to great effect in some of my pet projects [2].

[1] https://en.wikipedia.org/wiki/Literate_programming

[2] https://github.com/spicyjpeg/ps1-bare-metal

I'm glad the author mentioned this particular learning. Even if you do enjoy reading your own jokes, many other people will find them at best annoying and at worst confusing. When you add in people from other language/culture backgrounds the risk/reward of jokes gets even worse!

There was a famous conflict over rms's joke about the abort() function in the glibc manual[0], which said:

> Proposed Federal censorship regulations may prohibit us from giving you information about the possibility of calling this function. We would be required to say that this is not an acceptable way of terminating a program.

I think that joke illustrates nicely what I mean: it would only have made sense to people in USA, and would have just confused others. Even those who understood it would - IMHO - most likely not appreciate it being in the glibc manual. People don't read manuals to be entertained - they read them to find out as quickly as possible how to get their work done.

[0] https://lwn.net/Articles/770966/

It depends. Sometimes having a short joke or interesting wording in otherwise terse text can help with what is otherwise a bit of a slog, but agreed that it can be confusing.

>People don't read manuals to be entertained

Tell this to my younger self, when he used to read manual because of boredom.

OK, you got me there - I actually used to do this, too!

> "They were the ones who caught the mistakes that no spell chequer could."

I thought that maybe "spell chequer" was the valid British term, which would be interesting, so I searched but it isn't. It turns out that the joke here is that "chequer" is a valid British word, so a word-based spell checker won't flag "spell chequer", so it's self-referential. I see why people found his jokes actively confusing.

Agreed, and yet I liked it.

This used to be called "I hired QA people to identify gaps in my project", but hey now it has become paying people to follow readme

This sounds exactly like UX usability testing. Sit down with someone and watch them work through whatever process you’ve created.

It’s normal to compensate them for their time.

Normally though you don’t modify it after each participant. But for something very niche like following a README (as opposed to an e-commerce flow targeted to the general population) it might be fine, if less rigorous.

Yes, this is right! It's closest to usability testing in UX design.

As I already mentioned in another comment, I recommend this post for people who want to go a bit deeper: https://www.nngroup.com/articles/usability-testing-101/

There's a zeroth step missing from both the Quickstart and Full Set Up: install php-fpm, configure it, and configure your http server to serve using it as a backend.

It seems to be taken for granted — that's something you'd already have in place if you're already serving applications in PHP, but would have to figure out on your own if you haven't served anything in PHP so far in your life.

It's a tricky problem. How far back in the stack do you go? The README assumes that you know how to use git to check out the files - or that you can easily save them from the repository. Should it include that as a step in the tutorial?

As I say in the linked article, it depends on what sort of user you have. For a "getting started with Raspberry Pi" document, you might well want to include how to insert an SD card etc.

I'll have a think about the best way to help people figure out if they're running PHP. Thanks for the feedback!

It can't hurt to make a sentence or two about assumptions.

Like "This manual assumes that you have a Linux/BSD, a C compiler, GNU Make, and a text editor".

Installation instructions usually (should) have a “prerequisites” section. You don’t have to explain how to install the prerequisites, but they should be listed.

They are - https://gitlab.com/edent/activity-bot/-/blob/main/README.md?...

From the root comment, it appears to be incomplete.

No need for Adobe Creative Cloud is a nice touch

Yeah, he's taking the piss instead of listing prerequisites.

Tells something about the project, IMO.

Tells me he's having fun

> How far back in the stack do you go?

My rule of thumb for my READMEs: there should be a list of commands, that when executed in order and in a clean machine, result in the software doing something useful. Yes, this includes `git clone`.

If there's something the user might already have, like the webserver, I add a comment "skip this if you already have a web server". If there are any shortcuts that make it not production-ready, it's time to break out the ALL CAPS.

Limiting the operations to simple commands also helps me keep honest about the instructions (no hidden assumptions), and forces the software to be minimally testable.

Why stop at `git clone`? Why not include `apt install git` and equivalents for all OSs?

Whenever I write documentation, my first step is to explain how silicon can be used as a transistor.

Yeah well I produce home grown silicon in super novae.

'If you wish to make an apple pie from scratch, you must first invent the universe.'

I used to think it was a struggle to walk the user through introductory EM physics.

But, it turns out that was a walk in the park compared to explaining how to acquire and isolate the dopants, not to mention building up the pure silicon wafers.

If Windows/MacOS doesn't ship git by default, then yes, that should be included. On Linux, the people who are running Linux From Scratch can probably infer what the problem is.

I don't know about these days, but at some point neither Debian or Ubuntu Server shipped with Git. You can still find tutorials that start with apt-get update and apt-get install git-core

In my company that comes be default. Also, `git clone` helpfully includes a canonical path to the repository, in case you found the README laying around somewhere.

Otherwise, yes, I would include apt install for the dependencies, which is also incredibly valuable to make explicit. The only tricky part is what package manager to reference.

Generally it seems like good READMEs assume you have a compatible OS ready to go, but will give you a summary of all commands to get a working setup from there.

Feedback and issues you need to troubleshoot with your projects is a good indicator of your audience level, and from my experience it’s helpful to understand that documentation is always under development just like the code

from experience in ENT support where i was sending instructions & quick fix scripts to technically capable persons, you will learn very fast when you’ve missed the mark with your documentation / instructions. tons of times i had ready made solutions that i thought “just copy and paste and go what could possibly go wrong?” and was caught off guard how often a little too much knowledge lends to confusion. i am not blaming the users here it is my fault that i didn’t explain things like “no don’t change this date in the fix that is a special date when the issue could have earliest occurred and it’s there to avoid grabbing more than we need to parse”, but i didn’t tell that so of course people changed it to all sorts of dates thinking they had to

such feedback and issues also got me way better about writing code that avoided chances for such mistakes as i didn’t want users to have to read a novel to understand what to do; it’s a fine balance between what to solve with documentation and what to solve with code

It's really not that tricky at all, every install document I ever wrote has a "prerequisites" section telling you the prerequisites.

Even as someone working with PHP I would prefer if the project provides we with some guidelines for setup. Especially if it requires some specific version or extension. Ideally the whole dev environment should be containerized. Then you would again require people to understand and use that layer, but depending on the projects complexity definitely something to consider to make it easier to work with

Humans are great, but AI can really help here too. Let me explain :)

I write docs that AI agents have to follow to play a game through an API, then spin up 20 sub-agents each with their own identities / properties etc. and watch where they fail. They get stuck in the same places as humans.. or they will point out the "obvious" steps not explicitly mentioned.

now where the AI tests start to fall apart is that an agent doesn’t tell you the doc is confusing, it just does something wrong with full confidence. A person on a call says “wait, what?” and that’s worth the 25 euros :)

Most README files should include a screenshot.

Even if it is a command-line tool, a screenshot helps provide a better understanding of what to expect.

Turns out a picture is worth a thousand words.

Documentation is so important. I have been treating documentation in the same way I handle code. I found mkdocs works pretty well and integrated with a github job that updates the docs when I commit changes to my main branch. It also allows contributors to correct errors or add helpful instructions to documents. I think I have not paid enough attention to my biases though and like the idea of hiring someone to go through the process. I think my instructions are sound but maybe not so much for a user who is not as familiar as I am. I may not be doing things the optimal way but I have used a lot of documentation over the years and feel what I have done addresses gripes I have had with "Big Tech" provided docs.

mkdocs still uses "web fonts" for icons. This hack was required for compatibility with Internet Explorer, but is just cargo-culting these days. It's an accessibility issue in that users who disable web fonts (for readability) don't see icons, they just see unicode placeholders everywhere ("tofu").

I am using Material for mkdocs and I don't see any difference when turning off web fonts.

You mean https://squidfunk.github.io/mkdocs-material/ ? Try opening that site, and press the Down or PgDown keys on your keyboard.

I see what you mean. My docs don't do that so not sure what is going on there.

Most documentation reads like you should already know what you’re doing, which makes sense because it was written by someone who already knows how to do the process.

I think good technical writing requires the same skills as good product ownership, that is empathy for the user and their perspective. Often technical writing is an after thought and not someone’s whole role and it really shows.

Good article

I know there are some great README's (and other documents) around, that document best-practices or templates for great README's. I found one that looked very useful and would've sworn I starred the repo to find it again in time of need. Alas, can't find it. Anyone has some good resources to point to, to add to this thread?

Update: Found some related HN threads (omitted link-rotted submissions).

- I'd like to review your README https://news.ycombinator.com/item?id=26842191 (91 comments)

- Readme.so – Easiest Way to Create a Readme https://news.ycombinator.com/item?id=27006740 (65 comments)

- Readme Driven Development https://news.ycombinator.com/item?id=1627246 (57 comments)

Theres a GitHub repo called awsome readmes

Thanks for the tip. A couple I found with a quick search:

https://github.com/matiassingers/awesome-readme

https://github.com/yowainwright/awesome-readmes

We most times forget that not everyone can read our minds or see exactly what we see in our systems. When it comes to communicating ideas, there's always the requirement to actually communicate what it is the readers needs to know.

I oftentimes find myself spending more time rewriting readmes than writing code.

Treat the readme like a journey/walkthrough of your product, follow an order, and keep it simple to understand.

Yes. The amount of projects that don't just run is outstanding. Luckily docker container projects are inherently better at this is in terms of dependencies, but there are still often weird assumptions or medical incantations to get them during.

This is an underrated comment. If your project can run in Docker, providing a Dockerfile in addition to the README is incredibly valuable.

I agree. I have provided Dockerfiles in work projects, and it was successful. However, I found it difficult to run tasks using just the Dockerfile. The containers became too large. And I had to write too long arguments for volume mounts and environment variables. In the end, I had to include the steps for starting and running the containers in the README.

(I am deeply concerned about this issue. So I implemented a little tool called Lask, and it tried to solve this kind problems. https://github.com/lask-task-runner/lask )

I forgot to mention it but a docker compose is even better to mitigate incantations. It seems every project I use that has a docker file also has some custom script around it that boils down to mounts and variables

I've done this for internal dev tools! It's amazing how many assumptions you have about everything.

I've great success with friction logs: https://mikebifulco.com/posts/how-stripe-uses-friction-logs

If you are a platform team, going through this with your internal customers is both driving adoption and making your tools better. Highly recommended!

LLMs are also good for this; point them at your repo and ask them to do a fresh install and note all friction. One time when the lack of context really helps.

Hey - so cool to see someone else does this! Thanks for sharing my link, too. Glad you found it useful!

> My jokes aren't funny and are actively confusing.

2real4me

In all seriousness tho, I don't really put jokes in readmes or code comments. Jokes should be tied to a moment where they make sense, not just be present in perpetuum. Slack is great for jokes, or maybe even a notion design doc comment, alongside the actual feedback.

But sticking jokes in your readme just feels like "I have you here for other reasons, now you have to listen to me be funny".

When I was testing my books' instructions, I would operate from a fresh state and only allow copy paste. Every single command and line of code had to be expressed and in the correct order.

This was generally really a good way to go about it, because it requires everything to be correct with no room for adjustment.

Still people would miss things, but it always came from skipping instructions (sometimes completely.) Maybe 10 support inquiries total.

To a certain extent, you can think of writing documentation like writing code. Leverage your coding skills to improve your explanatory writing.

Each phrase or sentence is an operation that changes the state. The state is the mind of the reader. For it to work, you have to understand the starting state, and then construct a valid sequence that modifies the state step by step until it reaches the desired state. Every step has preconditions and postconditions. You can't leave important values uninitialized. You can't refer to symbols that haven't been defined. You can't just sit down and blurt out whatever comes to mind; you have to "play computer" (or "play reader") in your head to model the effects of what you're writing. You need to be aware of which "platform" you're targeting (developers, users) and understand quirks of each variation of that platform. Some of your operations might fail, and you may need a way to detect and/or recover.

Obviously don't take it too far and reduce writing to this. But I think it's helpful for getting into a mindset where you are thinking about communication in an end-to-end, closed-loop way. Your mind needs to be engaged and stay engaged with the question of what the experience is like for the reader. It's very easy to default to an open-loop mode where you just have a random string of thoughts about the subject, let your brain translate them into words, write that down, and call it done. There's a big difference between expressing thoughts and communicating ideas effectively.

Thinking about it this way could also maybe help with motivation. It's satisfying to write computer code and really nail it and have it do its job effectively, right? You can get a similar feeling of satisfaction from good writing.

these are really good heuristics, thanks! I am going to adopt this in my writing/editing processes.

"They were the ones who caught the mistakes that no spell chequer could."

Nice, good article lol. I appreciate a joke or two in a readme but totally understand the annoyance, I think forgetting to actually say what the software does is super common as well though. Often trying to figure out if something found on github will actually solve a problem only to be met with a list of install commands.

The 'look awesome project with funky name, here's how to install off you go' without a word to what one earth it actually does is amazingly common. Plenty of things posted here I bounce away from after hitting the linked page, finding something someone is obviously very proud of, but not having a clue what it's actually about :p

Besides emojis, I find it too long. READMEs should be succinct and be like a switchboard to other docs (much like LLMS.txt tries to be for agents).

Also, it features an FAQ. FAQs are problematic (as in not often effective): https://passo.uno/what-the-faq/

Edit: Clarification

The link says "My opinion is that FAQs pose a problem only when there’s no strategy around their usage." Calling them "problematic" based on that seems like an exaggeration

I've read some very bad READMEs. I mean really bad, like not even tell me what the program is for bad. I remember one that forgot to tell me what its own name was.

But really this is basic usability testing. I still run into issue at work where its like "Oh, you need to install C# runtime" or some other thing so it can even start. We have since fixed the build chain to check for that.

I know its hard to step out of your own context but that is what documentation is.

There are over 600 definitions of the work "run" and over 400 for the word "set" and you really expect that some end user in another industry cares what your context is. It's our job to care about their context because they are paying us to solve their problem.

Does the author aware the blog is unreadable on Firefox mobile? I might have something wrong on my phone too, but I don't have an issue with anything else.. https://imgur.com/a/ctriVnD

Did you muck with the system fonts? On my Android FF it looks just fine, and yellow not black.

That's the "drunk" theme the site has. Scroll all the way to the right in the switcher, and select "reset".

You have pressed the "Drunk Mode" button on the theme switcher - probably on a previous visit.

Change the theme at the top and normality will be restored.

I half agree.

I build apps with Claude Code, and I test them by having the AI click through the UI with Playwright. That's enough to check that things work as specified and nothing is broken.

But I think the purpose is different when an AI tries it and when a person tries it. To put it in extreme terms, AI is for UI and people are for UX. People find UI problems too, though: in my music player, songs got blocked from playing in Safari on a real iPad, and I only found it by using it myself. The things found in this article, like the jokes that didn't land or not knowing what the tool even does, are on the side only people can find.

(I wrote this in Japanese and used AI to translate it.)

Not related to a README, but very much related to how programmers / creators speak (by which I mean type).

My way in to WordPress support back in 2004 was decoding answers to others from Photomatt and others.

A user would ask a question about WordPress and, for example, Photomatt would answer. His answer was always correct. Technically correct. But it didn't land for the question asker. They would reply with .. 'What?'

I would then replay with "What Matt has said is right, and this is what he means, this is the answer"

I gave them the information they needed in words they could understand.

It was not Matt's fault, it was not the user's fault.

It was translating in a way.

In general, software is still Beta if a program requires a readme file to install and use.

It is under 5 minutes to write a shell/bat/make/cmake script for each platform to configure library requirements, import OS specific data, and enable GPU/NPU features. Then run through the application build, installation package with stripped performance build, and or a few regression tests.

Lets say you have 4k downloads a month on a small project, and it takes 1 hour for each admin to read/configure. You just saved about 5 years of your users lives reading your document. =3

Many devs love to shit on the guy from the 'I just want the exe. Why is there no exe'? meme. On some level I get it. And in some cases it's not even reasonable to provide a single binary or whatever. But on the other hand, github is being used to deliver end-user software. It is no longer just a dev's workspace/playground. That and the amount of makefiles and python scripts I've seen fail due to some dependency nonsense fills me with rage. It's never Just™ typing 'make' unless the code is trivial. I can wade through the bullshit. I shouldn't have to, and many others aren't even able to.

A lot of mature applications just gave up chasing the bleeding edge, and now only distribute Flatpack/snap/AppImage which is similar to classic exe/DLL ecosystems. A kind of silent tragedy for efficient shared library paradigms.

The hard truth is OSS is often chaotic as there is no authority saying a given API is "good-enough" to version freeze, and it limits how much individuals can integrate without upstream source-tree contiguous-integration at the mercy of 150k devs whims. =3

Given the amount of dependency bullshit I've had to deal with over the years, I'm glad the idea of using shared libraries for everything is dying. I like the core idea of it, but the implementation has never actually Just™ Worked™ in my experience outside of either trivial or curated examples. I guess all those mature applications discovered the same thing.

Flatpacks are great. I've barely ever had a problem with them, and the problems I did have were solvable with a cursory web search. Yes, they take up more space. That's a worthwhile trade-off in exchange for actually working.

>A kind of silent tragedy for efficient shared library paradigms.

This only tells me you have no idea how distributions work. The distributions themselves decide what kind of artifact they want to get out of building the software. The developer of the application cannot know that by definition because there is an unbounded quantity of distributions out there.

The only thing the application developer can do is provide a solution that works everywhere or provide a solution that helps with customized builds. There is no solution that both works everywhere and is customized everywhere.

>This only tells me you have no idea how distributions work

Packagers do try to coordinate distro release cycle mess, but ultimately long term support for complex programs is tied to the OS version constellation in upstream builds. Thus, the users are going to experience the Beta stage of that code... even though people often assume a polished release-candidate isn't just a euphemism for public Beta.

Thus, the assets needed to maintain a projects function are likely to stop working within 3 years or less. Indeed, some parts of the ecosystem are worse than others.

Low attention code creates 2 paradoxes:

1. new versions may not be supported on newer OS releases due to API mutation, and old dependency injection

2. versions may not be back-ported due to old releases missing API features, and new dependencies

I think it is hilarious people make up silly stories like it is not a fundamental constraint/liability in FOSS. The situation is not anecdotal, and reaches deep into the kernel driver areas. This is why we can't have nice things, as a billion conflicting use-cases is just a chaotic ecosystem. =3

I'm so old that I remember when a software deliverable included documentation, and the product delivery was incomplete without documentation.

This used to be standard onboarding practice everywhere I’ve worked.

Point new starter at the readme and get them to fix any issues (hopefully very few!).

IDEO.org's design kit has a lot of helpful "design research" tools to solicit person-to-person feedback. Perhaps helpful to people reading this https://www.designkit.org/methods/interview.html

I find it surprising/annoying/frustrating that many README's start of with how to download/install the software, without ever telling you what it is designed to do. (Perhaps you must have some inkling of what it does to be motivated to read the README.)

Excellent article! Thank you for the reminder that I need to be spending more time with users… or hirelings :)

Same thing happened to me. The install command in my project's README was broken for one release. I didn't see it because it worked on my computer. I only found it when I tried on a new machine. Now we run that exact command automatically before every release.

What I love bout this is how closely it hews to the principles of design research. Same principles, applied deeper in the experience as LLMs make things more accessible than no code, or CMS before that.

All people creating, need to speak to other people. Loved reading this.

>My jokes aren't funny and are actively confusing.

You didn't have to murder me like that!

Also - formalize what can be. There may be e.g. opportunities to write scripts that can be tested and linted. I realize this isn't always possible, but I do think it should be ruled out rather than ignored.

Understandability depends on the audience.

E.g. as someone who doesn't "Fediverse", the intro section leaves me having no idea what an ActivityBot is or what an ActivityBot account is.

The trend currently is telling your user to run this script in their terminal, addition point if you tell them to run with sudo.

See: Claude code installation.

This is one of those things LLMs are really good at.

Create a new container or sandbox, copy in the git repo or point it at your staging docs, and see what happens.

LLMs are good at reading READMEs. They are absolutely horrendous at writing them.

I try to actually write my own readme a still, but it’s obviously a rapidly vanishing art.

I think the problem is that this doesn't simulate for people who can't or won't read the manual

Actually, that's a good point. I often run into issues that would have been solved by more careful reading.

Paying people to record themselves using your software without reading the instructions might actually be very informative.

I imagine the ideal is a product that requires no manual. That's not always doable, but sometimes it is. Sometimes you can get very close!

Software X has a feature.

Me: "hi support, feature is not available??"

Support: "it doesn't work in that situation. Here's a link to our documentation with two dozen bullet points about where and when that feature doesn't work".

It's as if you selected a cell in Excel, and the copy-paste buttons were missing, and you raised a support case with Microsoft and they said "copy-paste does not work if your spreadsheet was opened from a NAS share. It's documented here that it doesn't work: <link>".

That is documentation as a defense, instead of the company putting in the effort to make the thing work, they put in documentation that it doesn't work so they can blame the user who "won't read the documentation".

> I hadn't actually explained what the software would do.

This is my #1 annoyance when looking at a trending repo

I hate READMEs with a gazillion emojis, too much noise.

Indeed, these make me back out of a project immediately if I don’t have a strong need to use it, and it takes extra cognitive effort to ignore the emojis and focus on the actually meaningful text.

This one wasn't as bad as the majority that have the emoji in each header too. At least these emoji did seem to properly represent the item they're next to, not an immediate rocket ship emoji in the header following by irrelevant ones of various sizes littered throughout the text.

Cannot agree more. Looks kinda childish too.

It was nice when emoji were used sparingly and with purpose and intention.

As my childhood English teacher said - too much of anything, good for nothing.

That's interesting. In my testing, about half the participants liked them, one didn't, and the rest didn't express a strong preference.

I kept them because they make me smile.

To me it’s similar to what you wrote about the jokes you removed: the emojis are distracting and not actually fun. This use of emojis comes across as a tired meme. It’s better if the project makes people smile due to its intrinsic qualities like working well and doing what they want.

>I kept them because they make me smile.

But isn't the whole point to make your users smile? No one but you cares if YOU smile.

I needed to read your sentence a few times to figure out you don't hate README files. I too dislike emojis overuse in README files.

To me an emoji-filled README is a good sign both the README and the project were generated by an LLM.

Immediate reaction - LLM generated. Hard to imagine a real human picking emojis to match feature description for a long list.

It was popular before LLMs. Probably popularized by Notion.

Somewhat related question: what is it about READMEs that AI agents love to dump the most useless, hard to contextualise & comprehend, irrelevant rubbish into them that makes understanding a project and onboarding so hard?

It feels like like the AI agents can't help themselves sometimes, and the judgement exercised around what's included and omitted is baffling.

But maybe READMEs have always been this bad, and AI agents have raised the baseline?

That sounds pretty similar to what I get from LLMs regardless of what they're writing for. Any time I get more than a paragraph out from an LLM, it tends to have of a lot of irrelevant details or unnecessary hedging, and I have to either wade through it to find the actual answer or try to convince it to make it more direct.

As someone on the spectrum, this does happen to me with people sometimes too; I struggle when asking a question and getting an answer that doesn't fit the "shape" of what I expect (e.g. asking a yes or no question and getting a relatively long sentence in response that doesn't contain either "yes" or "no" in it, which means I need to do the equivalent of applying it as a diff to my mental model and seeing if there are conflicts). This happens less frequently with other humans though, and on average the amount of effort I need try to figure out what they're saying is a lot lower. This doesn't make it less frustrating when I get that kind of output from an LLM, but it doesn't surprise me all that much that it happens.

I'm sure I do this all the time to people too, though. One of the biggest lessons I've learned in the past several years is that I communicate in ways that I'd probably have trouble understanding in reverse a lot more frequently than I realized, even I still do the things I find confusing from others less often than most people I interact with. I'm sure a lot of people might find this comment to be pretty much exactly like what I'm complaining about even though I feel fairly confident at least in this moment that my point is clear.

> As someone on the spectrum, this does happen to me with people sometimes too; I struggle when asking a question and getting an answer that doesn't fit the "shape" of what I expect (e.g. asking a yes or no question and getting a relatively long sentence in response that doesn't contain either "yes" or "no" in it, which means I need to do the equivalent of applying it as a diff to my mental model and seeing if there are conflicts). This happens less frequently with other humans though, and on average the amount of effort I need try to figure out what they're saying is a lot lower. This doesn't make it less frustrating when I get that kind of output from an LLM, but it doesn't surprise me all that much that it happens.

I've have people in my life where I've got situation going to the point I got sick of it and just started responding 'That didn't answer my question. Do you remember what my question was?'. Turns out many people seem to invent out of whole cloth what your question is rather than actually responding to it. Seems like a whole lot of wasted effort. I mean, it's one thing to ask a yes or no question and getting 'it depends on X, Y, Z, blah blah blah', or just 'I'm not sure/don't know'. There have been far to many cases where I ask something and the response is functionally 'well you see, it all began in 1969, back when dinosaurs ruled the earth'.

LLMs seem to atleast actually respond to your question. The question might be phrased in a way which doesn't match what you intended, and the answer will be verbose and require you to wade through prose to actually find it, or it might just be wrong, but it will at least be an answer to the question you gave it.

My personal experience is that people who respond to my like this do tend to understand what I've asked but aren't particularly direct in the way they describe things, which both causes them to give answers hard for me to understand and would cause them to probably not do a great job trying to phrase my question in their own words (and very few people are memorizing the conversation so that they can recite parts of it back later on).

Some of the noise comes from writing for two audiences(humans and agents) in the same file. There are established conventions for separating the two, and research suggests that the “repository overview” documents agents tend to favor aren’t even helpful to agents themselves. https://arxiv.org/abs/2602.11988

>But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.

For God's sake, this is not "biases," wtf?? That's straight up just not reading/following the document you're supposed to be testing/reviewing. When I test my documents I actually follow them exactly, step by step, and I always catch these sorts of mistakes. Always. I always copy/paste commands because I know that is what the customer will do, so I have to make sure that works flawlessly.

Dude, if you actually follow your own document, you don't have to pay people to do it for you. Also, you can make sure the person you are paying doesn't ignore the document like you apparently do.

I stopped reading, I'm not interested in whatever else this dude has to say. I'm literally flabbergasted.

Maybe it's because my documents have always gone to real paying customers who have to get through this install, and not some hobby project I'm super proud of or whatever and I don't think I'm super clever? Idk.

So usability testing?

Does anyone still write README files by hand now?

> My jokes aren't funny and are actively confusing.

:’)

And did it work?

Probably not, because almost no one reads the readme file unless there is a installation problem. =3

Coolest thing I've read in months.

Great idea! I don't understand this heartfelt statement:

    I took notes by hand (fuck feeding the machine) and spent several hours being told what incorrect assumptions I'd made

What does it refer to?

Given the agents.md I’m assuming the readme was just spit out by an LLM and the goal was to not read as LLM text. Problem is, it reads like you prompted it to be written in more simple language.

I just find it a bit misleading that you’re saying you want to talk to real people all while trying to clean up AI slop.

I've made my AI policy very clear at https://gitlab.com/edent/activity-bot/#what-is-your-ai-polic...

None of the code or documentation was written by or assisted by AI.

Originally had replied “explain the agents.md then” but decided I should actually check it myself… lol fair enough

I will say, if I did stumble on your project, just seeing that file there would be enough to make me skip past and not spend time reading. I look for those before investing any time reading a readme.

Thanks for getting the joke :-)

I did wonder about not having an Agents file because, just like you, I consider it a sign of poor quality. But I needed a way to discourage AI use and, in the end, none of my user research subjects mentioned it.

Writing good documentation is difficult. From those who say "the source code explains everything", I think 80% are too lazy to write documentation in the first place.

Having said that, I found consistently that when a project has working examples, ideally documented a bit, aka explained, they tend to work much better than those projects that have no examples. Working examples often also help get into a project quickly and check out how it works. It helps to learn too.

READMEs are not useless, of course, but the quality varies a lot. I also know of folks who use AI slop spam to improve it, but while it may improve a little bit, it generates a lot of horribly to read text that makes no sense. I am noticing this with the ruby core dev team - they (almost) all suddenly have perfect language skills but it is more like an advanced babelfish translator. What they piece together here makes no sense. Claude in particular is now famous for this slop content. And I don't understand what it is used: real people read any of this AI slop? Because I just skip it or filter it away these days.

>From those who say "the source code explains everything"

Because theoretically you should be able to describe not only your entire application logic, but upper and lower bounds of inputs as well. Good code would describe this inherently.

Documentation is only useful when a) the application is not source available, so you have no choice, b) you want to save a human developer time for them to understand your code, or c) you want to use documentation as a cache hit for agent use (less token spend)

Almost sounds like having a QA team is a good thing!

Now we have new ways to solve this problem: send Haiku or Luna to do task using computer use, but without access to code. If it can't complete it, or takes too long, docs are bad.

> My jokes aren't funny and are actively confusing.

Confession time.

I used to be prone to giving things slightly silly names, particularly unrecoverable structured exceptions. Examples include PancakeLandingException, ReallyBadException, CataclysmicException, ApocalypticDeathException, and the like.

And then years and years ago I used to work for a company called Redgate and I started a tradition of slightly silly messages when early access builds of our .NET products would expire, all based on Monty Python sketches and quotes. So, obviously, the dead parrot sketch featured in there.

So far, so harmless, but this did come to a head somewhat spectacularly and in a couple of different ways.

Firstly, in 2009 one of my colleagues used a modified quote from The Life of Brian as an expiry message on a build. Somebody who appeared to be some sort of religious zealot complained loudly to the company. We were both in LA at the time, with a couple of other colleagues, attending build 2009, so we woke up to a chain of something like 50 panicked emails in our inboxes with people expressing differing levels of outrage and/or amusement whilst discussing various grovelling apology options... until someone figured out that it was actually an elaborate troll that we'd swallowed hook, line, and sinker. I can't remember the name of the person who caught us out but, hats off, well played, sir, well played. It did unfortunately mean we became a bit more cautious and business-like with our early access build expiry messages.

Secondly, and this one needs a bit of context setting... I've always been a fan of descriptive and explicit error messages: there should be enough information in any error message that most of the time the user can figure out what's wrong and fix their own problem OR at least so that if they get in touch with support, then support can quickly figure out the problem and get back to them with a solution. I'm not a fan of unhelpful, information poor, obfuscatory, or cryptic error messages.

But when I wanted to cause an application to exit because there'd been an error related to tampering with our licensing code I played somewhat against type. I wanted error messages that would uniquely identify what had happened, making it easy for us to figure out, whilst giving the user no clue (because I wanted to make it very slightly harder for hackers/crackers - but let's be real: this would never have actually stopped anyone). So I used successive lines of dialogue from a scene in House where House is trying to guess who Wilson's girlfriend is. I have no clear recollection of why I chose this dialogue to reproduce, but... I did.

Anyway, this did lead to some slightly confused support requests coming in from users mostly trying to use the tools legitimately in slightly unusual scenarios, but nothing that was overly burdensome. That was until early 2011, when Greg Young - he of event sourcing fame - posted the following gist because he'd encountered an error that said, "Because I wanna ask you about your girlfriend. I must know who she is, or you would've told me her name.": https://gist.github.com/gregoryyoung/871736.

Not at all creepy, right? And, of course, it went viral on twitter. Cue another massive email thread although, this time round, people just thought it was funny. However, we did decide to make the error messages a bit more boring and, in the end, I just gave them numbers.

Mostly I'm just glad the error Greg got wasn't the final line of dialogue in the exchange between House and Wilson: "Yo mamma." That would have been bad.

The 'fake error messages to thwart crackers' thing is pretty relatable - I think back in my day I went with 'early-exit trap (#23)' or the likes for this type of error condition. It does help that we didn't have any phone support - numbers would potentially be painful that way! Another common one I see is dictionary words though even something neutral like 'anteater' or 'weasel' or 'high-jig-south' can both be frustrating to see as a user as well as when trying to deal with support for the inevitable case an integrity check fails on real systems...

"I took notes by hand (fuck feeding the machine)"

Closed. You are on a machine right now as am I, lets not pretend we don't like them for clout.

I tried reading after that line but I couldn't take it seriously anymore.

Nix.

I know, I know: It's complicated. But have you heard of AI agents?

But I just onboarded 4 interns on a project where all they had to do was

  1. Install the Nix package manager
  2. Install direnv, enter the project repo, and `direnv allow`
  3. Toolchain, git hooks, MCP servers, in-repo issue tracker, everything is available

Our project manager requested information that was available in the issue tracker. I told her, she could get all her answers by asking our agent, and it'd automatically reference the issue tracker. I figured I'd just need to show her how to install the Nix package manager. But no, she already had it because another project by another team depended on it.

Putting wrong information is README is so outdated when you have programmatic setup of your entire toolchain.

> I know someone is going to say "why not just ask an LLM to simulate a range of users?" The answer is very simple - I want to speak to real people. People are brilliant! They can make you laugh, you can see their cat when it wanders on to the call, they bring a unique perspective to the problem, and they're really happy when you give them a €25 voucher. Some will gladly do it for free and make you happy!

So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).

Agents and people catch different things. My agent reviewers find real bugs, like pinch-zoom breaking while a finger is on the joystick. But none of them said "this is boring". I did, on my phone, when I walked my character up to a house and nothing happened.

Yes, an agent wont tell you if something is boring or funny, at least not unless you ask (and i don't think its response would have much value) but it will tell you if your instructions can be followed or not - which was the main thing the author wanted to test against, at least as far as i understood.

My point wasn't that you can use agents for all feedback, but for the given case of testing your readme file's instructions (the article's title even mentions it is about the readme file) you can certainly use agents for that.

I respectfully disagree. LLMs will never provide you with the Lizardman Constant, and that's only one category of response they'd miss.

Reply on news.ycombinator.com