Comment by eesmith | Hacker News Reader

Comment by eesmith | Hacker News Reader

Parent Full thread

We have literally no idea what the 'sanity test' involved actually did, and the common use in computing has little connection to any such use in psychology.

https://en.wikipedia.org/wiki/Sanity_check informs me:

"A sanity check or sanity test is a basic test to quickly evaluate whether a claim or the result of a calculation can possibly be true." ...

"In computer science, a sanity test is a very brief run-through of the functionality of a computer program, system, calculation, or other analysis, to assure that part of the system or methodology works roughly as expected. This is often prior to a more exhaustive round of testing. " ...

"In software development, a sanity test (a form of software testing which offers "quick, broad, and shallow testing"[1]) evaluates the result of a subset of application functionality to determine whether it is possible and reasonable to proceed with further testing of the entire application."

and further comments that "sanity test" for some people is interchangeable with "smoke test".

It also adds:

"The Association for Computing Machinery,[8] and software projects such as Android,[9] MediaWiki[10] and Twitter,[11] discourage use of the phrase sanity check in favour of other terms such as confidence test, coherence check, or simply test, as part of a wider attempt to avoid ableist language and increase inclusivity. "

I could not find a definition for "sanity test" in psychology. I know about the cognitive test that Trump made famous with ‘Person, woman, man, camera, TV‘.

I cannot cop to "smoke test" being interchangeable with "sanity test," in any way. Smoke tests are obvious crashes, often before any real input. I had a friend on the build team of Windows NT 5.0 (until it was called Windows 2000) and their version of smoke test was "can it boot up to the login screen," and that's similar to working in electronics, where powering up your machine or circuit is all you do and you hope it doesn't release the magic smoke, before you actually do anything with it.

On that alone, I am very doubtful on that article.

This is also distinguished from what I have taken to calling a "pre-flight checklist" at the start of large programs, making sure databases are connected, requisite tables are present with the correct columns and datatypes, that specific files or directories exist, that files may be of a certain "freshness," and so on.

I'll stand by it: sanity test is a lot more descriptive and useful than "quick test." Those footnotes are only kicking the can down the road of justifiability. But let's examine them according to the aforementioned The PC Replacement is Less Useful criterion.

Recommending just "test" is ... well, double-plus ungood, in the sense that we now have no idea of the qualities of this test and how it could be distinguished from some other test. Hard pass.

"Confidence test" only implies a statistical likeliness, like a confidence interval. It does not imply the This Ought Not to Happen of a sanity test.

"Coherence test" is interesting but ... coherent against what?

If someone wants to sell me on a replacement, they are free to try, but the replacement must be at least as good at communicating what it does as the term sanity test. If this fails, then it will be scoffed at, and should be.

We do not know if what was described as a "sanity test" actually meets your definition of a sanity test, or was used for some other purpose like a smoke test.

A quick look using Google Scholar finds people using "sanity test" for smoke testing, like

"Typical behavior is to allocate any special resource requirements it needs, map the device into virtual address space, initialize the device and perform a brief “sanity test” to ensure that the device appears to be working correctly" at https://onlinelibrary.wiley.com/doi/pdf/10.1002/%28SICI%2910...

or

"The minimum essential test cases that need to be executed to evaluate the essential functionality are known as Sanity Test Cases" https://uksim.info/icaiet2014/CD/data/7910a048.pdf

or

"Sanity test is a brief run-through of the functionality of the software system to assure that the system works as expected." https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6006830

But it's also used for full testing, including human intervention, like

"There will always be a role for manual testing. For one, it is the only real way to sanity-test your automation itself." - https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

and

"Sanity test cases which checks basic functionality and are run for pre-system acceptance and when product goes thru major change. These test cases deliver a very high project value to both engineering dept and to customers." https://ijaer.com/admin/upload/06%20Apr_2012_Sheo%20Kumar.pd...

as well this example which distinguishes between shallow-and-wide smoke tests with deep-and-narrow sanity tests.

"a smoke test generally consists of a collection of tests that can be applied to a newly created or repaired computer program. This is a “shallow and wide” approach to the application." while "Sanity testing will be performed whenever cursory testing is sufficient to prove that the system is functioning according to specifications. A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep. It will normally include a set of core tests of basic GUI functionality to demonstrate connectivity to the database, application servers, printers, etc." - http://archives.christuniversity.in/disk0/00/00/48/68/01/the...

Going with PC Replacement question, since we don't know the actual goal of what was called "sanity test", we can't ourselves come up with a better name.

Which is why we should go with the OP's assessment that "quick" is an appropriate term for what they are doing. Since you don't know what that is, it doesn't matter if you buy a new term or not.

>as well this example which distinguishes between shallow-and-wide smoke tests with deep-and-narrow sanity tests.

I think this even is contradictory for what a lot of people think a "sanity test" means. A "test" may check that something is correct, where a sanity test would be a lower fidelity test that simply checks that something is not obviously wrong.

Oh, I agree. I included it as an example of an outlier. My overall point is that "sanity test" has a range of meanings, so without knowing what the original goal of the now-named "fast" tests, it's hard to really know if "fast" is indeed a poor substitute name.

>Recommending just "test" is ... well, double-plus ungood, in the sense that we now have no idea of the qualities of this test and how it could be distinguished from some other test. Hard pass.

Describe the difference between a test and a "sanity test?"

Reply on news.ycombinator.com