Hacker News Reader

Comment by noxer | Hacker News Reader

I wish people would find a way to include any code present in a README in some kind of automated test.

Nothing more frustrating than if you copy something from there but it turns out the README wasn't updated since the very first day and all the "hello world" code or install instructions are completely outdated

Maybe not include any code is the better solution, instead add an example folder.

View on HN

That's included in Rust![1]

Code examples from the documentation automatically become "documentation tests" to make sure your examples are still up to date when updating your code.

[1] https://doc.rust-lang.org/rustdoc/documentation-tests.html

The README.md isn't part of the rust code, so it's not checked by this unless you use tools to generate your README from doc comments like https://github.com/livioribeiro/cargo-readme.

Actually, it's possible and even frequently used, for example in lazy-static, through the `doc_comment` crate.

    #[cfg(doctest)]
    #[macro_use]
    extern crate doc_comment;
    
    #[cfg(doctest)]
    doctest!("../README.md");

Now, the readme examples are tested like everything else.

https://github.com/rust-lang-nursery/lazy-static.rs/blob/mas...

I like this about elixir, in the documentation you can use exdoc to make sure that the code examples actually work, and provide code samples that are tested. It makes reading documentation for projects quite nice.

I've been using https://github.com/brodie/cram for this. It's a neat little shell testing tool that can be told to check that every 4-space indented markdown code block output what it says it outputs, so I just cram my README.md.

An example of this in action: https://github.com/liskin/liscopridge/blame/68a656b7beb10a5c..., https://github.com/liskin/liscopridge/blob/68a656b7beb10a5cd...

I've written this tool to test example code in markdown documents using user-specified shell commands: https://github.com/anko/txm I have had great success using it in my own projects, so perhaps others might find it useful too.

One thing golang did right is the go playground. When I put code in my README, I also include a playground link.

Example: https://github.com/kstenerud/go-concise-encoding#library-usa...

I did that! I hope you are not the person who suffered that from me (did you use Picnic CSS a few years back?). So for a newer project I put some setup code that will look for all code snippets with a specific comment and run that with the code after the comment. For generating the website documentation that test bit can be stripped (though I kept it for now).

Example: https://github.com/franciscop/server/blob/master/docs/docume...

This is great advice and I wish it was the standard. I had this problem on one of my projects so I used a githook to basically paste in actual compiled example files from the test suite on committing. It does mean the actual readme is more like a template and you can't edit the README file directly anymore but all the examples always work now!

I personally like at least one small example embedded in the README since it's zero effort, an example folder is great for _more_, but annoying to get started, what should I look at in the example folder first?

You'll like this https://kodare.net/2020/10/23/100_percent_coverage_docs.html it's my journey to 100% coverage for all code samples in ALL the documentation for iommi.

I do something similar: my README contains the output of --help, and I have a CI step that ensures that all the lines of the actual help output are present in the README. Caught me a couple times forgetting to update new flags and version changes.

I've spent hours putting that functionality into a unit test. Worth every minute!

This is brilliant!

It’s one of those things that’s so easy to forget or not even think about but once you see it or think about it, it’s essential.

Generate it with an m4 include.

Reply on news.ycombinator.com