Hacker News Reader

Comment by josevalim | Hacker News Reader

I have a fun anecdote. About 5-6 years ago, Elixir completely disappeared from the top 100 after spending some time in the top 50. People reached out to me and then I reached out to TIOBE to understand why and the reason given was "bad presence on Amazon".

After further investigation, the root cause seemed to be that we finally had enough published Elixir books. At the time, if you searched for "xyz programming" on Amazon and only found a few results, Amazon would pad those results with non-relevant entries. However, because Elixir reached about 20-30 books, we were no longer padded, so we suddenly got worse rankings than every other language with only a handful of books. This happened on every Amazon domain they searched on, so it compounded and effectively kicked us out of the top 100 altogether. This all happened at a time Elixir language activity had already reached top 25 on GitHub PRs/stars.

View on HN

First of all kudos for making elixir.

And secondly, Like you are saying of "xyz programming", then to my understanding let's say I searched "elixir programming" on amazon, and then earlier there were not much books so it was (padded?) but once it reached 20-30 books, it wasn't padded but then how does it have an impact on search ranking. I still can't comprehend how having more books can have a negative impact on a popularity index and if such an index like TIOBE is doing so, then its clearly messed up.

My understanding (which may be wrong) from the exchange is that they literally search for "elixir programming" on several websites, including Amazon. So it is very sensitive to whatever changes those websites do to their own search engines. I can no longer reproduce the behaviour from back then but it is very understandable that websites like Amazon are optimizing their search results for sales and other key metrics rather than term precision.

Your understanding is correct. Their methodology is really that silly and susceptible to wild swings.

Amazon (US) is right there, we can try it:

  "python programming" : 6526
  "uxntal programming" : 2
  "elixir programming" : 2085
  "kotlin programming" : 390

I tried a couple of very new/niche languages like granule/futhark/carbon/jasmin but got either no results, or only obviously unrelated junk. For the languages above I quickly scanned the top result and they looked relevant.

> or only obviously unrelated junk

Does Tiobe detect that it's junk/padding or they just scrap the number and take it at face value?

The latter. That's precisely what they do. I emailed them and verified this.

That's pretty funny, since the search turned up downright confounding results, like books about programming with an author named "Jasmin". For "carbon programming" I got a ton of books about C, can't guess why, but it's surely not good data.

Maybe I'll make a language called "Introduction to" or "Linear" and shoot to the top of the index.

I think he's saying that before, when you searched for "elixir books" you would get:

- Elixir for Dummies

- Elixir for Beginners

- Elixir Programming with Phoenix

- General programming book

- Some other general programming book

- Etc

And this list would end up being a full page.

After you got a few Elixir specific books, you only had those, but the page was shorter. So ranking was lower.

Amazon is one of the search engines TIOBE uses.

It seems like Amazon showed other unrelated things if search for Elixir Programming to bolster the search results.

So you got more result than books existed. Maybe 50,100 or even more results.

After a certain threshold Amazon stopped doing that, so you get less results.

Less results, lower TIOBE position

Reply on news.ycombinator.com