Programming Language Popularity on GitHub and Stack Overflow | Hacker News Reader

Programming Language Popularity on GitHub and Stack Overflow

langpop.corger.nl

79 pointsstesch50 comments

Programming Language Popularity on Github and Stackoverflow.

Okay.

There are a lot of funny bits. For example the 'language' 'IO' is tagged quite a lot on stackoverflow. Or was it 'Input / Output'?

Common Lisp has fewer tags on stackoverflow - maybe because many questions are just tagged 'Lisp'.

'J' has very little lines changed. That could be a good sign, since it is based on APL, which is famous for its one line programs.

Perl6 is at the bottom. Okay.

CSS is now a programming language.

XML is now a programming language.

Self has some tags on Stackoverflow. But mostly in the context of other programming languages, not the programming language Self.

Etc.

This is flawed in so many details.

>There are a lot of funny bits. For example the 'language' 'IO' is tagged quite a lot on stackoverflow. Or was it 'Input / Output'?

I got so fed up with "IO" tag being used for Io (http://iolanguage.org/) questions on Stackoverflow that I created the iolanguage tag and then re-tagged all the Io questions!

I did this a few years ago (2010 i think) so luckily there wasn't many questions to change :)

Io question count currently stands at 44 and not the 7,126 as shown on the chart.

ref: http://stackoverflow.com/tags/iolanguage

XML

With XSLT, yes.

http://fxsl.sourceforge.net/articles/FuncProg/Functional%20P...

XSLT is listed separately

"CSS is now a programming language. XML is now a programming language."

Seconded. Could we please stop call CSS a language. If not, add PDF to the list.

Interestingly, PDFs actually do contain an embedded programming language in them, a subset of PostScript. [1] PostScript, meanwhile, is actually Turing complete with conditionals and loop structures inside of it. [2] I'd argue that makes PDFs a whole lot more programming language-y than CSS or XML.

[1]: http://en.wikipedia.org/wiki/PDF#Technical_foundations

[2]: http://en.wikipedia.org/wiki/PostScript#The_language

"PDFscript" (as I like to call it) is not Turing complete. It's more similar to SVG in terms of capability. From the PDF spec:

"To simplify the processing of content streams, PDF does not include common programming language features such as procedures, variable, and control constructs."

    > Could we please stop call CSS a language

If you're going to be pedantic, be pedantic right. CSS is indeed a language, even if it's arguably[1] not a programming one.

[1] http://stackoverflow.com/questions/2497146/is-css-turing-com...

I feel that these popularity indices are pretty flawed and most often used as arguments under a confirmation bias. Defining "Popularity" is no good measure in my opinion. This graph just shows that with many lines changed many questions arise, it does not say whether people enjoy doing so. I would be much more interested in spots like "Logos" in the bottom right - 160 Mio Lines changed, only 38 questions of StackOverflow. So what does that mean? Is everything about the language crystal clear? Beat that, JavaScript.

Github is a reliable source but I am not sure whether it is representative. At least it appears to be very popular with web folk. Other sources are missing such as bitbucket, especially since it allows for free of charge private repositories. Like this, the only thing this says is that github use correlates with the lines of JavaScript. Not sure whether this is a good measure for popularity. Much like a kid in kindergarten, who shows up every day and talks to everyone. Yes, his presence correlates with the presence of others, but what if he is just a jerk and nobody really likes him?

Fair points.

And of course basing the github measure on lines of code gives quite an unfair advantage to verbose languages like Java.

Perhaps they could look at compression ratios when (say) gzipping a decent sample of each language, and use these to weight the LOC metric?

You are absolutely right, I didn't think about this when I wrote my comment. LOC is in itself a very poor metric, therefore "Changed LOC" doesn't improve any interpretation. Including the number and sizes of repos would be better, since LOC pushes verbose languages. Java and C# code usually features a lot of empty or hollow lines (in C# for example, one opens the curly brace for a function in a new line).

I mean, yeah, most of the comments indicate that we agree that this kind of graph or "false statistic" is flawed.

But can we find any value in it? How do we interpret this graph, despite its basic problems? What good stuff can we do with it?

A scale for cross-language comparisons seems a hard ask because everyone is implicitly interested in answering different questions. Which languages do people enjoy using the most? Which have the most code "out there" in some setting or other (open source, commerce, in deployment, scripting, ...)? Which have the most engaged and vocal communities? Which have the biggest pool of skilled workers? which will I get the most career benefit from learning?

Perhaps the useful information is more in the correlations between different related metrics, than in drawing up ranked lists (surprise surprise, Java > Haskell!). This graph helps visualise the correlation between these two, which seems significant but far from perfect. Outliers like SQL can then be identified, which point to problems with the metrics (e.g. with SQL, presumably github fails to spot lines of SQL embedded in other langauges).

Or they could do away with LOC as a metric entirely and use number of repos instead?

That would probably overvalue scripting languages, as they are more likely to be used for smaller languages.

Also a lot of repos contain multiple languages (web languages more often than not do)

I think language popularity falls into two different categories: (1) languages other people will pay you to work in, and (2) languages that you would work in of your own choice.

For #1, I would measure the frequency that a language appears in job ads.

For #2, I think the trending repositories in Github would be a better metric than commit lines.

Good point. If anything, the number of commit lines has as much to do with the verbosity of the language as it does its popularity.

Hold on a second, so a programming language that is very verbose, and needs lots of assistance to be workable is considered popular?

Sure. For some definitions of popular.

It would be interesting to see the same data broken down by year. Lua, Clojure, and Go are at about the same position, but I wonder which one is gaining traction. Probably Go.

So I have seen many of these kinds of graphs and associated complaints that come with them. Much of the criticism revolves around the usefulness of ``popularity'' as a proxy for other things that we actually care about in programming languages having to do with their immediate usefulness to us. These are all good to discuss, but I think there is a more pressing issue, having to do with the granularity of the sample. I work in an area of scientific computing, and have 1-2 languages at any given time that I work with often, and have familiarity with 3-4 more. However, I have friends that work in slightly different areas of scientific computing that have a different set of languages. Integrating information from these people, I have a rough set of the languages used in my local neighborhood. And I will say there is actually quite a bit of diversity in the neighborhood, even though its all within scientific computing. So who is to say what lies in the realm of friend of a friend of a friend? I don't have a feel for this distribution.

This is where I think integrating these kinds of graphs with a filtration over distances on some network would be highly useful. It is not clear which network(s) would be the best, but there are some clear advantages to going with induced link through github collaborations. What I am suggesting might be more helpful for niche domains than it would be for the ``median'' programmer (if such a thing exists), but who knows, maybe there is some usefulness there too?

Dart, CoffeeScript and TypeScript all have +50M lines of code.

It is weird that Dart has 50M LOC when Go only has 100M LOC. My analysis of Dart's popularity suggests that it is actually a lot less than 2x less popular than Go. I though Go was actually 10x more popular that Dart at least.

I wonder how much double counting and so forth (as a result of forks) is present in this analysis. Maybe there is a bias as to what is released publicly to Github versus used in production.

This is only tangentially related, but I remember seeing a site a few years ago that ranked languages by crowdsourcing about 100 subjective comparisons between different pairs of languages. It made statements like:

    - Programs written in this language tend to work
      correctly the first time they are accepted by the
      compiler
    - The type system in this language helps me to be more
      productive
    - Code written in this language is easy to read and
      understand at a glance
    - I would choose this language to write a desktop GUI
      application in

and asked people to choose which of a pair of languages each statement applied more to. The data could be ranked by language or question, and I found it really useful and discovered a few languages I wouldn't have heard of otherwise. But for the life of me, I can't find the site anymore. Does this ring a bell to anyone?

Yes. http://hammerprinciple.com/therighttool

That's it! Thanks a million :)

I was surprised to see how highly-ranked some of the languages were (like Fortran and R) that I had previously discounted as mostly abandoned in the "github crowd".

Obviously Fortran is used a lot in HPC and numerical routines, but I always figured those were closed-source and rarely updated, with development focusing on their wrappers.

> Obviously Fortran is used a lot in HPC and numerical routines, but I always figured those were closed-source and rarely updated, with development focusing on their wrappers.

There are quite a few that are open-source, and often under very permissible licenses, even public domain, because they were developed under university research programs.

I remember finding at least half a dozen, if not more, open high-performance libraries that implemented a direct method for solving sparse linear systems. I'm fairly sure that applies to most everything else.

My suspicion is that most of the "lines changed" of Fortran were in BLAS, LAPACK etc bindings for Python, R, Julia.

Perhaps even just the same commits replicated in lots of people's forks of (say) scipy -- do they control for this? what about rebases of the same commit?

Github gets lot of undeserved credit. Most of the code in the world doesn't even know what is Github.

Then again, not that much code is aware of version control systems (mostly the code of VCS'es themselves and related tools)

He meant "coders". As in: "most coders don't know or use GitHub".

thanks, typo

Undeserved? Wait what?

I think s/he is implying that github's importance is exaggerated by the people who use and love it, and that the number of people not using github dwards the number of people using it.

Github mus-identitifies QT translation files as "TypeScript", assigning all commits on some very active QT projects (like Bitcoin) to that language, which massively inflates the typescript usage on metrics like this.

Github uses linguist to identify the codes programming language and unfortunately it's far from perfect - https://github.com/github/linguist

There are known issues with distinguishing code between certain languages. For eg. There are issues between Perl & Prolog (and sometimes even Puppet). And up to very recently [1] most Rebol code was identified as R :)

[1] That is until this patch was merged in (although the repos language stats only get updated after a new push happens) - https://github.com/github/linguist/pull/1005

Not my data or article but for those who want to poke around with it http://langpop.corger.nl/formatted_data is the relevant URL.

Lines of code changed is confounded by syntactic properties of the languages: For instance, languages that make a lot of use of indentation, like Lisp, will have more changed lines of code but that doesn't necessarily reflect more activity. In Lisp, you often have huge diffs even for minor changes because these minor changes can lead to a change in indentation of many otherwise unaffected lines of code.

Nice work! GitHub should overrate the languages of the cool boys, but underrate expressive languages because less LOC. stack overflow on the other side should overrate day to day work languages but also overrate complicated languages that generate more questions.

At the end of the day there's a clear correlation so I think this graph is a reliable source of information. Is there a way to remove logarithmic scale?.

Intuitively it feels quite accurate. C#, Java, C++, Python, JS cript at the very top. Scala above F#. New entrants not too down in the bottom.

Deleted comment

Well, no. Nothing about Javascript makes it an "error prone jungle".

It has some strange coercion behavior and some floating point caveats, that programmers can learn and avoid in a week or so of learning the language. After that, it's fairly streamlined and easy to use, and it surely does not necessitate relentless rewriting to fix bugs much more than any other similar language.

Completely ignoring errors relating to function calls with wrong argument counts does increase the probability of hidden bugs. That's not quite easy to avoid completely. Every time a function signature changes, all callers become buggy (without even a runtime error to signify this).

I did a analysis of programming languages using github data before: http://exceedhl.github.io/blog/2013/05/07/programming-langua...

I was shocked to see Pascal so high on the list. Pascal was the first compiled language I ever used. But I don't even know what development environment you would use for it now.

Flag: The data's from Feb 2013, i.e. over a year old, which should be reflected in the title.

This is the first accurate programming language popularity chart I've seen.

I see Puppet but not Chef. That's something that should be fixed. Then again, this was done in 2013. I bet a fair amount has changed since then.

Also, SLOC for ranking on Github? We don't use that to assess "productivity" for a reason.

It might not measure productivity, but it sure gives a hint about how much a language is used.

Sure, a language can be 1/5 as terse as another. But not 1/100 as terse.

Unlike Puppet, Chef doesn't have it's own language. All chef cookbooks are going to count as ruby code.

Obvisouly Java is most popular because it is the best programming language. It got OOP and is on the super fast JVM, nothing can beat it.

Reply on news.ycombinator.com