Hacker News Reader

Comment by greggman | Hacker News Reader

Specs are important but I'd argue a conformance test is equally or even more important.

With a conformance test people can test their implementations and when ambiguities arrise new tests can be added or old tests fixed. Without, conformance tests different interpretations of a spec lead to divergent behavior. Once that behavior is out there long enough it becomes difficult to fix as people are depended on it and/or its quirks.

View on HN

I'm confused. How are you supposed to have a conformance test without first having a spec? To what would the implementations be conforming, if not a spec?

Sorry, What I meant to really say is specs are not sufficient. Sure specs are important. But even with specs, without comprehensive conformance tests implementations will diverge.

I work on the WebGL spec and tests based off the OpenGL ES spec and tests. Even though the OpenGL ES spec is long and detailed, every edge case for which there is no test is broken or different in one driver or another.

It's entirely possible to build a conformance test suite around an existing 'black box' of code and accept that whatever it does, even stuff that might be considered a bug, is what any compliant implementation should also do.

(You could also fix some of the more egregious bugs, and then expect implementations to follow your new behaviour)

In an ideal world, the spec would be the test. A spec is just a human readable test. A test is just a machine readable spec.

> Specs are important but I'd argue a conformance test is equally or even more important.

I agree, but the article mentions tests prominently. They didn't miss that out.

Here are the tests we wrote for HTML-to-Markdown conversion. Could be useful when switched the other way around :-)

https://github.com/bergie/to-markdown/blob/master/test/tests...

Also worth looking at the tests for Text::Markdown which is the canonical version of Markdown on CPAN.

https://metacpan.org/source/BOBTFISH/Text-Markdown-1.000031/...

> Without, conformance tests different interpretations of a spec lead to divergent behavior.

I think you mean either:

> Without conformance tests incorrect interpretations of a spec lead to divergent behavior

> Without conformance tests different interpretations of an ambiguous spec lead to divergent behavior.

Conformance tests should test specific behavior defined in the spec. Tests will never (or rarely) prove spec conformance but spec conformance should always prove test conformance.

I have yet to encounter a spec that was not in some way ambiguous.

1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).

Overloading is a feature of natural language though, and therefore completely unavoidable in writing a spec.

> 1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).

What is it specifying—or, perhaps a better question, what would an implementation look like? It might be more precise to say "I have yet to encounter an implemented spec that was not in some way ambiguous."

> 1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).

It is only unambiguous if you specify the numbering system you are using ;)

</pedantic>

Is there actually a notation and + operator for which 1 + 1 = 2 is ambiguous? Only being semi-rhetorical, I'd be interested to know if there is one. (Modular arithmetic is not considered ambiguous, at least not in the math neck of the woods.)

1 + 1 = 10b is what he's saying.

Yes, that's what I meant - even "1 + 1 = 2" can be ambiguous if no context is provided.

Not really answering your question, but more to the point of the discussion in my opinion, a() + b() could be ambiguous if a() and b() are functions with side effects and the evaluation order of the +-operator is not specified.

1 + 1 != 2 if the 1s are magnitudes of vectors - if the directions are opposite, then they can be 0. In fact, 0 <= 1 + 1 <= 2 in that system.

That's not how addition works. You're either adding the vectors or you're adding the magnitudes. You don't get to sleight of hand by writing one and adding the other.

Hee hee, your interpretation of his sentence is introducing ambiguities :)

Reply on news.ycombinator.com