Introducing the FreeBSD Test Suite | Hacker News Reader

Introducing the FreeBSD Test Suite

julipedia.meroh.net

41 pointsprofquail8 comments

For background, this is essentially the NetBSD test infrastructure ported to FreeBSD. But it needs a lot more tests to be fully useful. Getting decent test coverage for an OS (userspace and kernel) is pretty hard...

Not exactly. NetBSD is still using the deprecated ATF tools in its test suite while in FreeBSD I went straight to their replacement, Kyua.

Also, because FreeBSD uses Kyua, its test infrastructure can be composed of test programs of various kinds (including legacy test programs and TAP-compliant test programs) and not only ATF-based test programs. This is something that NetBSD cannot do yet.

Note that NetBSD does have Kyua in the base system (optionally), but moving the continuous testing systems over from the ATF tools is a stuck project. The reason is that Kyua needs some more work to make that a reasonable migration for the people that run such machines. In particular, the generation of reports needs much improvement as well as the collection of results from various hosts. See http://wiki.netbsd.org/kyua/import/ for some details.

However, because FreeBSD needn't be tied to the deprecated tools, and because I am the one setting things up from scratch, I am using this project as a playground for the new framework. Doing this deployment in a major project quickly surfaces all kind of big issues that need addressing -- and is certainly a boost in willpower to fix them :-)

(Yes: an eventual goal is to also move NetBSD over to Kyua but that will have to wait until Kyua proves itself "good enough" on FreeBSD's land. Homogenizing the two systems will have the nice result of simplifying the sharing of tests!)

Yes I saw your talk at Eurobsdcon, need to talk to you sometime about adding tests written in Lua to Kyua...

I didn't see it in article, but Kyua is supposed to be pronounced "queue-ay" (QA). Now you know.

Edit: Ha! I see Julio adjusted his stance on this:

The name is Kyua, which is just a play on the pronounciation of the Q.A. acronym. Originally, my intention was to pronounce Kyua as Q.A., but in reality this never happened. Today, just read the name as your instinct would: "Kyu-ah".[1]

Oh well. Now we all know the backstory, and we can all talk like learned experts to others.

[1] http://wiki.netbsd.org/kyua/

Yes I did :) I found myself and everybody else never reading it like Q.A... so, well, just use the more natural pronunciation!

Reminds me of the scene from the Tom Hanks movie The Wonders where the band is originally cute with the spelling, Oneders, and an emcee introduces them as the Oh-Nee-Ders.

For the opposite effect, does anyone else find themselves pronouncing gnu (the animal) with a hard G, a la GNU?

guh-new vs. new (referring to wildebeest) is something I remember from days before rms' GNU even existed.

Could anyone give some backstory to Joe Web Developer over here? What does it mean to test an OS? Are these unit tests, or is it more like installing the FreeBSD on hardware or a VM and verifying behavior?

Reply on news.ycombinator.com