Hacker News Reader

Comment by minimaxir | Hacker News Reader

Er, how is the "rank" defined? # of new repositories in that language during those years? # of total repositories in that language during those years? Or some weighted average?

I really am not fond of the trend of "arbitrary rankings" a lot of startups have been using recently as content marketing to create statistical analyses that cannot be questioned.

I could reverse-engineer the ranking chart using the GitHub Archive on BigQuery to check it, but I have no idea how to actually determine the statistic for ranking.

View on HN

Rank is number of repositories with that language created in a given year, so e.g., the languages with the most repositories created would be ranked 1, the second most 2, etc.

Source: I ran the query.

So I ran this BigQuery:

   SELECT repository_language, COUNT(*) as num_repositories
   FROM [githubarchive:year.2014]
   WHERE type="CreateEvent"
   AND IS_EXPLICITLY_DEFINED(repository_language)
   GROUP BY repository_language
   ORDER BY num_repositories DESC
   LIMIT 10

which returns the top languages in 2014 by # of repositories created, per your statistic.

The output is:

JavaScript 1273811

Java 930359

Ruby 769712

Python 630549

PHP 601101

C 473113

CSS 445501

C++ 355996

C# 183349

Objective-C 165102

Which doesn't completely match up with the order in the chart for 2014. (Positions for Python and PHP are reversed)

From the original post:

> The rank represents languages used in public & private repositories

The delta is due to private repository usage influencing the rank.

It looks like a simple count of the language(s) detected by the linguist project -- https://github.com/github/linguist

Agreed, does not make much sense if it isn't a weighted average. A simple count could very well be littered with Hello-world repos (I've found a lot of those, in various languages) by people who are 'trying' github on.

Good idea, maybe something like the number of code lines. That would value bigger projects higher then small projects. Which actually make sense to me. Because small projects are probably written in a specific language for a reason: batch files, css, html, examples, tests. Whereas I am interested in what language people choose for bigger projects, where several languages could used but still one has to be chosen.

It would also make sense to weight in the date a project started. Newer (big) projects would then weight higher then older (big) projects.

So I think taking in account the number of lines and age together is a good way to really visualize trends.

This would inflate the popularity of a language in proportion to its verbosity, but that could be approximately normalized by using line count ratios from the Great Language Shootout, which explicitly tracks how many lines various languages use to solve the same problem. Precision is pointless--the line ratios vary from problem to problem--but a rough approximation would be useful as long as you could get the statistics with and without it.

I think number of contributors would be a good metric, even though it penalizes lone projects, but I suspect that would be relatively even across languages compared to other metrics. For example, LoC is proportional to verbosity. Commit count is emphasizes languages that tend to have more bugs.

The problem is that one language might require less lines of code to achieve something of the same size/complexity of another language, if it's terser.

IIRC the guy who wrote this article http://redmonk.com/dberkholz/2014/05/02/github-language-tren... about GitHub language trends scraped the numbers from the search page, e.g. the following search yields all repos created in 2014 and the language breakdown in the sidebar: https://github.com/search?utf8=%E2%9C%93&q=created%3A%22%3E%...

Reply on news.ycombinator.com