Hacker News Reader

Comment by Homunculiheaded | Hacker News Reader

One thing that this article fails to point out is how often librarians were wrong before Google. In 1986 there was a study[0] that showed that across the board reference librarians were only correct about 55% of the time.

I think most people today would consider a query answering system that had an accuracy of 55% to be an interesting curiosity, but certainly not ready for real-world application.

It's funny how frequently we measure machine learning performance of an "easy for humans" task and fail to compare it to human accuracy on the same data. We just assume humans would do perfectly on it. I'm sure there are a few MNIST digits that I would get wrong.

[0] P.Hernon, C McClure "Unobtrusive Reference Testing: The 55 Percent Rule," Library Journal, 111 April 15,1986

View on HN

It's the pre-internet sources and Government employees fault and not the Reference Librarian as a profession. This is out of the US Library system only.

Your "Across the Board" = "Government documents and central reference areas of 26 U.S. libraries" Not your academic or local libraries that most of us think of as Librarians.

I was a System's Librarian and contend that Librarians are needed more now more then ever due to the Internet but that is a different tale. One-Third the Librarians were let go in my state during the economic down turn).

Reference Librarians should be quoting sources and I bet you a million dollars that a Reference Librarian gives you the source and it is the source that is wrong.

Suggesting that reference librarianship is "easy for humans" is sort of... ridiculously wrong. There's a reason it requires a master's degree.

But why were people asking reference librarians questions?

When I spoke to librarians in the mid-90s, they'd just point me towards sources that could have an answer. It was up to me, the reader, to ultimately find the answer.

Depends on where you were and who you were asking.

At several large research university libraries, I found the reference librarians could be downright obsessive in tracking down some questions. It would depend on workloads and other factors, I suppose.

This was mostly for pleasure or K-12 assignments. The sort of questions that were very fact based.

"Do snakes poop?"

"How far away is Mars?"

That sort of thing.

I was specifically thinking of uni-level questions.

At the uni level, aren't they more supposed to help you find the appropriate database rather than find the answer for you?

I might ask a librarian how to access back copies of a journal, but I wouldn't expect him or her to do my lit review for me...

You can get pretty far with being right only 51% of the time. With 55% you have an edge.

How often are less than 55% of the results of a Google search relevant? Count ads.

That' s not comparable at all. 55% of answers were correct, not 50% of the possible material in the library was relevant.

If a search for something like "crane hook design principles" returns a page with "buy crane hook design principles for less" at the top, it is wrong. And wronger than the wrong of a librarian because it's not even trying to be correct, just thrown out there for the edge case. We just happen to write it off as "that's just Google".

Is that what it shows you? I just searched for that exact thing. The first two were ISO standards, the third wikipedia, and the fourth a paper on crane hook stress analysis.

Not as good as a directed search by a professional, sure (and I haven't actually looked at any of those papers, so who knows how relevant they actually are), but my experience seems a lot better than what you're implying.

If you followed either of the links to ISO, you would have found that they are offerings to sell the standards. The technical information is behind a paywall. That's ISO's business model. The link to Wikipedia is about cranes in general.

If a librarian said "the answer will cost you $181 (CHF 178,00)" or "here is a book about cranes" we would probably agree these high quality agree even if we might disagree over their being wrong. Nevermind that ISO standards are not necessarily a good set of design principles because minimum requirements are not the same as design tradeoffs.

I don't think that's entirely fair. Surely the most useful measure of "wrong" is whether the user comes away with the correct answer?

Adverts on a webpage might definitely make it less likely that a user comes away with that answer, but I don't think they'd invalidate a user's successful result any more than gossiping with the librarian would invalidate their help if they successfully find an answer.

The advertisements are noise. SEO also produces noise. At some point the signal is lost. Google's business model is built on maximizing the amount of noise a user will tolerate whenever the user in hope that the user stops searching and starts shopping. If you search for a book, Amazon will appear before the Wikipedia page.

Nobody contested your argument, specific knowledge of niches s still fairly scarce on the net, at least if looking for gratis information. OTOh a well sorted library supplies books that cost a fortune, so google couldn't be replace that, at least not yet.

Reply on news.ycombinator.com