Hacker News Reader

Comment by brabel | Hacker News Reader

This blog post assumes Python is more productive than other languages. People like to make this claim about not only Python, but many other dynamic programming languages, specially Ruby, PHP and Lisp... but there's very little evidence to support that... programmers tend to be most productive in whatever language they know best. If they know both Python and Java equally well, I would bet they would be almost exactly as productive in either.

View on HN

I used to work at a C++ for prod, Python for prototype place. Time and time again, a 1-2kloc Python prototype, the bulk of which was written in a week, would take months to translate to C++, at times longer than the whole process of research+prototype.

I think it was 2 things: one, it is genuinely harder to write a valid C++ program. The language imposes non-trivial constraints on the programmer, for better and worse, and you have to navigate them for the thing to even compile. But two, C++ invites pedantry. I heard endless discussions about "ooh, template meta-programming", "struct or class", "but is this idempotent", "you should be using move semantics", "why on earth is this a unique_ptr".

I'm not saying C++ is worse, or that its trade-offs are not worth it, but for sure, from my experience, translating an algorithm into code is done far more quickly in Python than in C++. YMMV

Ah and also, this was definitely not a question of deficient C++ coders. The hiring standards for C++ were so high we were always short of devs. Meanwhile, Python prototypes were usually written by part-time ex-academia Python dabblers.

I think a large part of the difference is the prototype vs prod. I have written prototypes in C++ in a few weeks that still took months to go to production. It is just that production-ready code require significant more ceremony, more config files, more complex error handling, code reviews and of course feature creep.

I still agree that Python can be much more efficient to write than C++. I just wish it wasn't so unbearably slow.

I would mostly agree, though not entirely. Continuing with the anecdata, at times I had enough waiting and put my prototype into prod, with significant hardening. I can't say I always got it right, but usually 3-5 days of very focused poking made the thing at least reliably run at the desired cadence, with rare but loud and graceful failures. For sure though, there is a tax for doing it all "properly".

Even then though, I have the impression that Python just makes it easier to write it all more quickly. It just doesn't invite you (as much) down rabbit holes of doing things even more properly.

> But two, C++ invites pedantry. I heard endless discussions about "ooh, template meta-programming", "struct or class", "but is this idempotent", "you should be using move semantics", "why on earth is this a unique_ptr".

Interestingly, I've consistently had the opposite experience. In C++, when I ask, "how do I do X" there will be a few different options, but pretty much any of them will do. As long as you avoid things that are definitely UB, you're fine. In Python, I'll find endless religious arguments about which is the most "Pythonic" method. Whenever I try to just hack out Python code to just get things done, I always feel like I'm being judged, because usually the quick-and-dirty method is far from the most Pythonic.

I think it all depends on who does your code reviews. I agree with GP that c++ has religious adherents to the one-true-undocumented-in-their-head standard who will make writing it a nightmare. In python the philosophy is supposed to be - there is an obvious way to do things and that’s the right way. However this does fail from time to time.

Ruby on the other hand embraces many different syntaxes to do the same thing, but they’re all accepted as correct in the traditional view.

But two, C++ invites pedantry. I heard endless discussions about "ooh, template meta-programming", "struct or class", "but is this idempotent", "you should be using move semantics", "why on earth is this a unique_ptr".

I wonder what the result would have been in eg. Go or Rust, where that level of conversation doesn't exist. True, others would have taken place instead, but at a higher level nearer the design, where they should have been happening anyway.

> Time and time again, a 1-2kloc Python prototype, the bulk of which was written in a week, would take months to translate to C++, at times longer than the whole process of research+prototype.

I've heard reports like this for many years.

My pet theory is that the discrepancy here is ~10% static types and ~90% memory management. Having a GC (or refcounting or whatever) fully baked into the language so that you don't have to think at all about how values are passed around and stored is a monumental productivity boost. Possibly the biggest win in software engineering productivity in the history of the field.

Absolutely. I recently translated Microsoft's example code for using the Windows crypto API from C++ to Python, and huge chunks of the code melted away as it was mainly concerned with freeing buffers. Exceptions also helped.

> Python prototypes were usually written by part-time ex-academia Python dabblers

This seems like a monumental level of operational waste. I'd love to hear more about this particular setup

Well, part-time programmers. Full-time employees, doing mostly semi-academic stuff.

There should be one-- and preferably only one --obvious way to do it.

-PEP 20 -- The Zen of Python

Although that way may not be obvious at first unless you're Dutch.

—The next line

This ZoP line is by far the most misused cliché of all things Python. How is it determine that “prototyping in C++ and rewriting in C++” is not the obvious “one” way? Because you don’t like it? A Zen is intentionally self contradicting to allow introspection, not judge others :)

Hold my beer.

-- Django

Amateurs.

-- Perl

Slow kids

-- assembler

Pot, kettle, black.

-- fpga

Back at you.

-hdl

I wonder how your C++ would've gone if you had a C++ library of Python-like data types. Something like a 'variant' type that could hold arbitrary integers, strings, and dicts or lists of those (or anything else you commonly used).

You would initially trade some of the C++ performance, but the code shouldn't be more complicated or difficult to write or compile than Python. Then, you'd only need to change the performance sensitive parts to use real C++ arrays or structs.

This is something I do except the slow parts are in Python and the fast parts are in C++ with pybdind11 connecting the two worlds

I thought the C++ tendency was to optimize performance sensitive parts with C or assembly.

C++ is C. We used to optimize with assembly but nowadays with a processor's superpipelined architecture and the compiler's global register optimizations and so forth your inlined assembly is more likely to throw a monkey wrench into the whole works and make everything run slower. It's a better use of your time to look for a better algorithm.

Not really. C++ is not C, nor faster than C. e.g. TCO in C++ is awful due to ctor/dtor (could be mitigated by some compiler hints if available).

Isn't that what Python is? Except C and not C++.

No, because you wouldn't have to pay the virtual machine overhead (among others). Another possibility would be using Cython - write the prototype in pure Python, then Cythonize the parts which need to run more quickly.

There are some new competitors for Cython. The article mentions Numba, which I've been pleased with, and mypyc, which I haven't tried yet.

I've been programming for nearly 40 years now and I've used all the popular languages in that time (popular for microcomputers) and I've mastered a few. I just started Python for the first time last year. Am I a Python expert? No. But I can tell you right now I can get a project done faster in Python than using any language in which I'm an expert. It's that productive.

Does that mean Python has all the performance you would ever need? No - we know it doesn't, but we also know that you rarely need that kind of performance and when you do it's usually localized to a very narrow portion of your code. Take that portion of code and implement in C and optimize to your heart's content. Even with that you'll still get your project done much faster.

Python is the tool of choice for those developers who just want to get stuff done.

I've used Python for 15 years and I'm far more productive with Clojure. Don't generalize.

I agree with you on Clojure, but I'm not going to be able to get my team to adopt Clojure - unless you're going to tell me how you convinced your team to adopt Clojure! :)

What features of clojure do you feel make you more productive?

I’m primarily a python user, but I spent time writing code in Ocaml to understand the potential benefits, but… (I would love to have my mind changed)… it feels like many of the features people touted about Ocaml have already made their way into python…?

1. Immutability / referential transparency seem more like nice-to-haves for a codebase, rather than the real reason people tout FP…?

2. Sum/product types and pattern matching are being added soon to python

3. Mypy is starting to gradually enable python typing, although I assume it’s still a work in progress

4. Python allows us to use map/reduce, and to pass functions around as arguments…?

I want to understand the potential benefits of FP more, but my experience with Ocaml hasn’t shown me great improvements yet. Open to having my mind changed.

In Ocaml, these things are all first class citizens and well integrated features.

In python, most of these "features" are done half right and duct-taped on at the last second as an afterthought.

Presenting a specific experience refuting an earlier generalization is reasonable here, I think.

I mean it is not just mentioning a specific experience to refute a generalization when there is also this: "Python is the tool of choice for those developers who just want to get stuff done."

Admittedly that last bit was a bit inflammatory, but I've gotten tired of the constant arguing over languages at where I work. I'm on a bit of a rogue team that's just adopted Python and are getting stuff done. Other people are just now starting to pay attention because we're getting so much done and now work is headed our way as a result. Wait until they find out we're not using any of the languages they're bickering about and are using lowly Python! Still, as awesome as my team is I think they'd draw the line at Clojure.

I love Clojure. As with Python, Clojure comes with batteries included. It's dynamic typing and REPL makes development really quick and the JVM runtime allows to achieve better performance than Python in many cases. You can even go as far as compiling to native executable with GraalVM.

I spent 7 years working with C#, I like C# and I think what they’ve done in the past few years is amazing.

I’m much more productive in Python. Not as in “I feel” more productive, but as in measurably more productive.

Where Python falls short of something like .Net and C# is that I probably wouldn’t have been more productive in Python if I didn’t have 7 years experience with a rather strict environment, and as far as using Python on major projects, well let’s just say that there is a reason people use TypeScript instead of JavaScript and Python doesn’t fall into that category.

But for most programming and for most minor systems or services, Python is just wildly good.

Same here. I do game development in C# and Python is my goto for sketching POCs for complex mechanics, despite never formally learning the language. It could easily be the difference between spending the whole day vs two hours on the same issue.

> there is a reason people use TypeScript instead of JavaScript and Python doesn’t fall into that category.

I thought the dynamic vs static typing debate was pointless until I joined a large Python project. Now I pretty much consider anyone that can keep up with one superhuman.

I think part of the problem is that many people who are used to static type analysis get lazy with naming and interface design. Toss one of them into your team and they'll "prove" the need for static type analysis.

Possibly. IME it’s quick Python POCs that didn’t bother with good practices that turn into critical production services without anybody going back and applying good coding standards.

That's a theme, for sure. "We don't have enough time to refactor!" is generally less true than "We need to slow down to go faster."

I also spent many years working with C# before working with Python professionally. I've also spent a lot of time with Lua, though not for paid work.

The second point matches my experience with dynamic languages as well. To use them well really takes a certain level of discipline, but it pays off. It's why I'm a bit skeptical of Python as a beginners language, which it sometimes has been touted as.

About large projects, presently I am working with Python on a somewhat large project (currently 53k sloc, I guess large is relative). Its been great, though we are pretty strict about almost absolutely everything being type-hinted.

53k SLOC is tiny project. 5 senior developers at 1k SLOC per day will generate 53k SLOC in two weeks, just one sprint.

How many people are writing 1k SLOC/day? And doing so consistently? If you're banging out that much code, I'd be severely concerned for the quality of the code itself as well as overall design.

Senior developers (really seniors, not 3 year seniors) can trivially produce such amount of code, because they wrote almost same code dozen times already.

My point isn't about who _can_ do it but who _does_. If anyone on my team was writing anywhere near 1k LOC/day, I'd think it was a serious problem. If the whole team was pumping out 1k LOC/day/person, I have to imagine I'd be getting as far away from the entire team/org as possible.

Maybe not the best example, but I used to work with a guy who had pretty large scripts that he was writing for one of our projects. It turns out it was mostly copy-paste, so it sort of got the job done -- except he fixed bugs in one place but not in the other. The whole thing was a huge mess to understand and maintain. And he was a senior engineer at the time (now principal, to my great amazement).

Developers at fixed-price contracts. Fast work + low number of bugs = good margin. It's exhausting, but, IMHO, it's better to work hard for 6 months and then relax for few months than to slowly push project for 2 years with red eyes and headache.

Why are they re-writing the same code instead of re-using it? Either convert to a library, or copy&tweak the existing code.

Why are they writing the same code again instead of writing different code? Why aren't they learning about new APIs for topics they aren't so experienced in? What makes them "senior developers" and not "senior transcriptionists"?

Who is tasked with digging into 10 year old code, written by an ex-employee, to find a subtle bug? Who is identifying and fixing performance issues?

How are these "senior developers" able to write 1KLoC/day plus documentation and developer tests? I find test code and documentation each take as much time as writing the code in the first place.

Why aren't some of these senior developers producing negative LoC? https://www.folklore.org/StoryView.py?story=Negative_2000_Li...

Because previous code is owned by previous client.

For example, 1M+ LoC of code, including build system, CI, test cases, built-in documentation, in 6 months is about 7kLoC per day, divided by 5 developers it's about 1400LoC per day per developer.

I completed 30+ projects already, I have 20+ years of experience. Few years ago I was able to close up to 20 tickets per two week sprint, when I worked with same level senior developers and dedicated PM, product owner, and QA team, at large outsource company at fixed-price projects.

It's easy when tickets are properly sized and described by PM, when PO is responsive and easy to reach, and when QA covers your back for complex test cases. 6 month project + 1-2 months to recover after, and then another such project.

Are your fixed-price estimates based on LoC?! Else, why aren't you embedding that knowledge in a corporate library ("contractors tools") which you license to all your future customers? Twenty years of playing one song over and over for years, even done well, doesn't make someone a concert musician.

Those numbers are ridiculous, and I say this having 25+ years of professional experience.

The typical industry numbers are under 100 LoC/day.

Eg, slide 20 of https://www.slideshare.net/ddskier/calculating-the-cost-of-m... ("A world-class developer (e.g. Facebook or Google senior engineer) will write 50 LOC per day")

"Improving Speed and Productivity of Software Development: A Global Survey of Software Developers" at https://uweb.engr.arizona.edu/~ece473/readings/9-Improving%2... (Fig 6) has Lines-of-Code per Total Man Months at about 1,750, so about 81 lines per day (assuming 21.62 work days per month).

"A Practical Approach to Software Metrics" at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=819938&... says "Many industry rules of thumb describe programmer productivity in terms of lines of code, for example 350 [noncomment source statements] per engineering-month of effort." That's 16 lines per days.

Going the other way, in the COCOMO estimator at https://strs.grc.nasa.gov/repository/forms/cocomo-calculatio... your 1 MLoC for an "organic" project estimates 3390 person months. Not the 30 months you mentioned.

That said, it's really easy to distort LoC measurement. And I mean beyond the "put in 1,000 lines of the form 'a=1'" cheat.

For example, LoC traditionally refers to source code, not test, CI, etc. that you use. Why did you use that non-standard definition?

Test code, for example, may contain a lot of data records and autogenerated code. This doesn't require the same work as a source line of code.

Consider SQLite. https://sqlite.org/testing.html says the library is 143.4 KSLOC, while the test suite is 91911.0 KSLOC - nearly 100 million lines of test code! These tests were not all written by hand. At that ratio of test code to source code you would have about 1,400 lines of source code.

And it's possible to mis-measure source code. A few years ago I added about 400,000 lines of code to my project in a few weeks. These were auto-generated when I replaced a very confusing set of C preprocessor directives with a homebrew template system to compile specialized versions of functions across my parameter space.

I then added some Cython projects, which generates C files about 50x larger than the original pyx files.

Counting auto-generated code makes LoC a worthless measure.

You also wrote you have a QA team. I assume their test cases are in your repo, and counted in your 1M+ LoC count, but you didn't include their time.

The post-release defect rate per KLoC is about 7.47 (average) and 4.3 (median). See An Overview of Software Defect Density: A Scoping Study at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6462687... .

If you write 1KLoC/day then you're likely generating 2 post-release defect bugs per day. That's in addition to bugs your QA team catches.

Your "up to 20 tickets per two week sprint" simply isn't enough to catch up with the number of bugs you likely generate.

Your numbers are so far outside of industry standards and academic findings that, with 20+ years of experience, you must know they are exceptional and difficult for anyone else to accept on your simple say-so.

You numbers are right. I just checked my stats on current project: 101 LoC per day. On this project I have ) incompetent PM, so ticket descriptions are vague thus I need to manage ticket queue by myself, ) hard to reach product owner, so I must figure lot of things at my own, *) no dedicated QA team, so I need to write test scenarios in addition to automatic test cases and then perform them manually, etc.

> Are your fixed-price estimates based on LoC?!

I don't now. I was not a part of sales.

> Else, why aren't you embedding that knowledge in a corporate library ("contractors tools") which you license to all your future customers?

Because each customer has different requirements, different goals, different language and framework. Nobody wanted to pay for a library.

> If you write 1KLoC/day then you're likely generating 2 post-release defect bugs per day. That's in addition to bugs your QA team catches.

In practice, I had about 1 serious bug slipped per 1-2 weeks. Professional QA team known a lot of corner cases to test already. Professional developers know them too, even when writing in a different language using different framework. Moreover, we know how to prepare good CI, efficient automated tests, how to write efficient documentation, who is responsible for what, etc.

> Your numbers are so far outside of industry standards and academic findings that, with 20+ years of experience, you must know they are exceptional and difficult for anyone else to accept on your simple say-so.

How much I will be paid if you will accept my story? :-)

101 LoC per day * 10 days = 1,000 LoC, so "1 serious bug slipped per 1-2 weeks" isn't that far off the median rate.

> Because each customer has different requirements, different goals, different language and framework

Sure, but that doesn't match your earlier statement that the senior developers 'wrote almost same code dozen times already'.

That is, I don't see how "almost the same code" is able to handle different requirements, etc.

How much would you pay me to accept your story? ;)

Python, like all dynamic languages, works best to bash out a new project into production ASAP, get a bonus and move on, while poor folks maintaining after you are trying to untangle it.

I think there's a lot of "Python's the greatest thing in the world" coming from ML/AI and it always reminds me of this. I wish I knew who said it.

"People who know one language think it's the greatest in the world. People who know more than one think they all suck."

> "People who know one language think it's the greatest in the world. People who know more than one think they all suck."

The Blub paradox: http://www.paulgraham.com/avg.html

(BTW, the TL;DR is people that know more than one language admit they all fall short one way or another, but there are some languages that are more productive than others).

You're saying you have less problems untangling C/C++/Fortran/etc. code left from previous folks who wanted to ship something and move on?

Python’s “one right way to do it” is specifically designed to avoid this. This isn’t Perl.

Python has multiple different ways to do things even in the standard library lol

Current Python doesn't reflect this philosophy though. You have lots of options for strings and string formatting. There is pattern matching and if statements. The standard library often isn't the best option for stuff (like HTTP requests) so you use another library. Package management and deployment is far from solved, with lots of different tools.

Pet peeve: there is no right way to do type annotations. Two major projects aren't even compatible: https://www.cs.rpi.edu/~milanova/docs/dls2020.pdf

> You have lots of options for strings and string formatting.

“Strings and string formatting” is a very broad domain.

For most specific tasks, there is one low-impedance approach and it takes very little (but not zero) reflection and/or experience to find it.

Most new Python features directly address specific tasks for which there are currently multiple relatively high-impedance approaches taken because there is not one obviously correct way.

Python's “one obviously correct way” is not “one possible way” (the latter approach is closer to Go.)

There's a great deal of evidence that supports that, it's just not scientifically organized. If you just look at the amount of high quality high polished projects out there for the dynamic programming languages it absolutely dwarfs those of static languages. And then add to that the fact that most of those are by solo devs in their free time.

It's not just that they are more popular, Java reigned supreme for years when Ruby and PHP and later Node.js outpaced it's communities by miles. And you can't say Java developers are not open source oriented, because they 100% are.

And the argument that programmers are simply most productive in the language they know best is trivially untrue. When I first learned of Ruby, I could dream in C#, I was absolutely fluent, knew the standard library by heart. I switched to Ruby and only looked back whenever I needed tight performance. And this is not just an anecdote, the whole Ruby on Rails movement was basically Java developers fleeing to greener pastures.

I don't know a single highly experienced multi-lingual developer who does not reach for a dynamic language when they need to deliver something quick and easy.

The only exception I know off is using Go for small networked services, which is quite comfortable and intuitive for a static language. Also outside that niche Go quickly loses its productivity edge.

> I don't know a single highly experienced multi-lingual developer who does not reach for a dynamic language when they need to deliver something quick and easy.

Hi, highly experienced, multi-lingual developer here (Python, Ruby, PHP, JavaScript, Java, Go, C#, F#, OCaml, Elixir/Erlang, ReasonML, SQL, Smalltalk, Objective-C, Swift, leaving off quite a few prior Web 2.0 ones for brevity) nice to meet you. As you can see, I've done just about all of it: every paradigm, every syntax. It all honestly blurs together after a certain point, and it just becomes easier and easier to pick up a new language the more you learn.

Anyways, these days, I never reach for a dynamic language when I need to deliver something quick and easy. The difference is just too marginal. Not worth the downsides (and the downsides are immense!).

Because, as my experience has taught me, invariably one of those quick and easy ones will turn into something that becomes business critical, lives on for years after you are gone, and will be much more difficult for other, typically more junior, developers to update or enhance your code. Static typing is not only marginally less productive these days with all the great tools and IDE's out there (not to mention Go which has one of the least obtrusive static type checkers I've seen, or, even better if the org allows you, a language with a HM type system), but for a marginal improvement in productivity, you pay a heavy long term cost. Not worth it.

Dynamic languages are great for rapid prototypes. After that, convert it to a static language. Your junior devs that join after you, who aren't familiar with the entire ecosystem your work has will thank you.

If I'm prototyping something like a log file analyzer or web scraper I use a dynamic language because it gives me a debugger REPL. I can interactively develop code with immediate results and copy & paste straight from the terminal into the editor.

When I want to develop further, I put a debugger statement right where I want to pick up, where all the data is available, and develop from execution.

Some static languages have REPLs but few are as good as dynamic languages for mutating existing code in the middle of a debugging session.

If you don't write much code which involves exploration of data or unknown APIs, then this mode of development may not be as useful to you, but it's a significant productivity advantage for me and the reason I reach for a dynamic language. Ruby is my go-to, with binding.pry as my debugger REPL.

> After that, convert it to a static language

I don't think you can have a hard and fast rule like this. It will depend on the situation. In many instances, a "prototype" built in Python or similar is perfectly fine.

In addition, one can make a horrendous mess in statically typed languages as well. One of the absolute worst projects I ever worked on was written in a popular statically typed language. This is of course an anecdote and not to say statically typed languages are worse, but they're not automatically better either.

Just a note too that I'm a big fan of, e.g., Rust & TypeScript, and I often use type hints in Python, so I'm not anti-static typing.

Way easier to refactor a that statically typed mess.

There is a floor for how bad you can write static typed code.

There is no floor to how bad of JavaScript or Python or Ruby you can write. The madness can descend to the inner most circles of coding hell. Only Perl exceeds it.

Well which one do you tend to reach for then? Purely out of curiosity.

Go in the general case, Java if complex domain modeling is required (or requires "enterprise grade" B.S - SOAP, WSDLs, or other crazy XML specified madness like Adobe extension stuff, etc), and after that: anything functional if I am allowed.

Explained:

Go's type system is generally less rigid. It strikes a good balance of strict enough. A lot of Go's converts aren't from "systems" languages like it targeted originally, but rather former Python/Ruby/PHP/JavaScript backend devs. I love the performance and low level levers I can pull with Go (although to be fair Java is quite fast enough). But finally, Go is easy to learn (26 language keywords?), the standard library is mostly great, and the worst developers I've seen still write mostly maintainable code that builds fast, which is what I optimize the most for these days.

Java, for all its warts and legacy cruft, these days you can write fairly good java, utilizing modern libraries. I love most things from Codahale, who in turn I think pushed orgs like Spring to write better libraries, so now everything's pretty good. Plus all the legacy stuff comes in handy when you have to deal with arcane government or financial systems, something I have to interface with frequently.

But if I had my choice, I'd use something where you can express functional programming concepts intuitively, without fighting the language or having to do it at a heavy performance cost, like Rust or better yet, just a full fledged FP language like OCaml (whom I understand heavily inspired Rust)

> I don't know a single highly experienced multi-lingual developer who does not reach for a dynamic language when they need to deliver something quick and easy.

Maybe this shines light on your social circles more than actual language impact on proficiency?

The "our language is so much more productive" myth exists in many languages, including the one I currently use. But when you dig deeper, there are almost always other social or technical factors that explain the productivity gap. That kind of gap exists even between people using the same tool.

The reality of it is that assessing what makes a developer productive is incredibly hard, and people doing so to claim their language of choice is better rely on anecdata and couldn't explain what "methodology" means to save their life.

> I don't know a single highly experienced multi-lingual developer who does not reach for a dynamic language when they need to deliver something quick and easy.

This argument is not addressing the claim for general purpose software, only for "quick and easy" software.

To people in this thread: please stop to think before responding to the "wrong point". I think we can all agree that dynamic, scripting languages are more adequate for "quick and easy" software, but this is not what the point of the discussion is. The point of the discussion is, or should be in my opinion, whether it's true that for general purpose software, written by teams, that is not trivial to write by oneself in 5 minutes, that scripting languages are more productive than statically typed, compiled languages.

Where do you draw the line? Is GitHub quick and easy software? Is Shopify? Were LinkedIn, Twitter? Are things built on Tensorflow?

I think you misunderstand the scripting language revolution of the mid 2000's. It's not that we suddenly realized scripting languages were the best for quick and easy projects. We realized scripting languages were suitable for a whole lot more than some data processing. We could much more effectively build huge scalable software platforms.

It got so bad that the sentiment flipped, and people like Joel Spolsky had to go out of their way writing blog posts that you could in fact build successful modern web platforms in C#. And then he went building the world's most popular project management tool in Node.js anyway.

> I don't know a single highly experienced multi-lingual developer who does not reach for a dynamic language when they need to deliver something quick and easy.

If you're picking from a popular language, the language itself rarely matters for this at all. Every popular language does roughly the same things. Writing speed (length of keywords, for example) is a non-issue. The ecosystem is by far the most important thing.

It doesn't matter how efficient I am in Go if I'm missing a crucial library that I'll now have to write myself.

JavaScript is a classic example where, even if you are pretty fast at writing your code, you will be dramatically slowed down by: 1) needing to add a new package every 5 min to do something trivial, 2) looking through five half-dead libraries to find one that seems maintained and usable, and 3) finding out that some of the libraries you chose are buggy.

So for a quick prototype, I'd weight a language's qualities as follows:

- ecosystem: 90%

- static analysis/tooling: 5%

- stdlib: 3%

- syntax: 2%

And for a long-term, complex project, I'd weight them closer to:

- ecosystem: 50%

- static analysis/tooling: 46%

- stdlib: 3%

- syntax: 1%

Python was Googles language of choice for a while, so people jumped at it to work at Google.

Popularity and expansive use may have nothing to do with quality, and a lot to do with financial influence on peoples agency.

Business wants people templating out directories of performant code, not generating syntactic art for the ages.

The classic, now quiet dated citation, is "An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl" by Lutz Prechelt.

Recent-ish commentary (July 2021) about it at https://renato.athaydes.com/posts/revisiting-prechelt-paper-... , HN commentary at https://news.ycombinator.com/item?id=28108806 .

Google Scholar gives about 476 paper which cite Prechelt's work. I have not followed other work in that field.

Norvig had a response to Prechelt as well: http://www.norvig.com/java-lisp.html

Mentioned and used as the Lisp baseline in the link I gave. ;)

> Mentioned and used as the Lisp baseline in the link I gave. ;)

Just saw that, nice. I like Norvig because he really cuts to the chase.

I doubt it, though if anyone has data I’d love to see it. At the moment I’m quite comfortable in both javascript and rust. But I find javascript (/ typescript) noticeably more productive for small to medium projects. I can get more done with less work. For a variety of reasons it seems to take more work to write good rust code than good javascript code.

This isn’t a knock on rust - I just think rust trades off programmer productivity for correctness and performance. And it shows, on both sides.

> But I find javascript (/ typescript) noticeably more productive for small to medium projects.

This seems like a pretty apples-to-oranges comparison. Rust is intended to do things you couldn't/wouldn't use JavaScript for.

If you're talking about a small-to-medium project where you don't care much about mishandling memory, of course JavaScript is going to be more productive. Rust is forcing you to tell the compiler a lot of things that JavaScript assumes you don't care about (and is, most of the time, correct).

> I doubt it, though if anyone has data I’d love to see it.

I recall a study being quoted in my university courses, which described the amount of code needed to get certain things done between different languages, which showed that Python, Ruby and others are on the less verbose side, the reasoning being that on average they'd also be more productive.

This seems to coincide with my personal experience, where JavaScript with React was much easier to work with in smaller projects, whereas using TypeScript with React lead to much slower development because of all the typing that needed to be handled, especially in cases of union types. Now, one can say that it's worth the effort to be more confident in your code doing what you want it to both now and after X months, much like you could sometimes prefer the type systems of Java or .NET over Python, Ruby or PHP, but in my eyes those tradeoffs always come with slower development velocity.

The sad thing, however, is that DuckDuckGo (and possibly other search engines) failed to return the original study or anything like it, only resulting in low quality blog content:

  - https://duckduckgo.com/?t=ffab&q=programming+language+comparison+amount+of+code
  - https://duckduckgo.com/?q=programming+language+comparison+verbosity
  - https://duckduckgo.com/?q=programming+language+lines+of+code+java+ruby
  - http://libgen.is/scimag/?q=programming+language+verbosity
  - http://libgen.is/scimag/?q=programming+language+lines+of+code

Anyone have any better search queries for this? Any idea why the search result quality is generally so low? Any ideas which study it might have been referencing?

The research on this topic has been discussed on HN multiple times. It is indeed hard to look up previous discussions, but the conclusion is always inconclusive... one thing everyone seems to agree nowadays is that measuring LOC is a bad proxy for programmer productivity, specially when you take code maintenance into consideration (which very few research papers do), and that types do have a small, but measurable effect on improving quality (though whether that's at the cost of productivity is still unclear as far as I know - yes, programmers need to spend a bit more time to declare their types, but without them they have to spend more time every time they need to call anything).

Google Scholar is a better method to find published papers.

The first result for "program language comparison", at least when I view https://scholar.google.com/scholar?q=programming+language+co... is: Prechelt, Lutz. "An empirical comparison of seven programming languages." Computer 33.10 (2000): 23-29.

There's all sorts of work which cite that paper, like "An empirical investigation of the effects of type systems and code completion on api usability using typescript and javascript in ms visual studio".

For fun, here are some of those citation which themselves use the phrase "An empirical study" or similar in the title:

"An empirical study on the impact of static typing on software maintainability"

"An empirical study of the influence of static type systems on the usability of undocumented software"

"Do developers benefit from generic types? An empirical comparison of generic and raw types in Java"

"An empirical study on C++ concurrency constructs"

"An empirical study on the factors affecting software development productivity"

"An empirical study to revisit productivity across different programming languages"

I think different languages do have sweet-spots, and it often has to do with library availablity (for me at least).

I consider myself as someone who knows "far too much about C++", but recently when I had to spider some webpages and read some values out of them to populate a database, I did that in Python because it's a handful of lines due to high quality libraries that all work nicely together. I honestly wouldn't know how to do that in a short amount of time in C++.

I admit that Python is more productive than C++, but the reason is mostly to do with memory management... a language with a GC, like Java or JavaScript, would use a very similar level of abstraction as Python and therefore, should "cost" about the same to write.

Python and other dynamic languages don't use static types and that may give them a (very) small advantage in productivity as well, but only for very small programs... as program size increases, in my experience, statically typed languages take the lead in productivity... as OP is about writing general software, not just small scripts, I really have a hard time agreeing that Python is either the best tool for the job, or the most productive tool at all.

It's more than that. For example, converting a string to uppercase in C++:

    #include <string>
    #include <cctype>
    #include <algorithm>
    #include <iostream>
     
    int main()
    {
        std::string s("hello");
        std::transform(s.begin(), s.end(), s.begin(),
                       [](unsigned char c) { return std::toupper(c); });
        std::cout << s;
    }

Same thing in Python:

    print('hello'.upper())

That takes advantage of the fact that Python puts a bunch of relatively obscure functionality in scope by default... thus increasing the chance that programs will use the wrong method or rely on stuff they don't really intend to. Case manipulation isn't something most modern programs should even be doing at all, since it's almost impossible to internationalize it.

In fact it's really mostly name space management. Python doesn't require you to pull in "upper" because it starts with a relatively big name space. It doesn't require you declare the main program because you're relying on the fact that Python executes a file on loading it (and that reliance is arguably bad Python style). All of the "std::" stuff is a name space management choice.

... and the rest has nothing to do with static typing either. The way the code is indented on multiple lines is a stylistic convention. The transform primitive requiring you to specify bounds is a library choice, related to the language's apparent lack of a universal way to map over a container. I'm not sure why you need the lambda. The rewrite in place is memory model stuff inherited from C.

In Haskell, which is fully compiled and is about the most rigid static typed language you could ever imagine:

   import Data.Char(toUpper)
   main = putStrLn (toUpper <$> "hello")

If you'd asked for something that didn't require oddball functionality, then that could probably also have been a one-liner.

C++ is forcing a style on you that encourages you to avoid a copy. You could, of course, have a library like:

    std::string ToUpper(const std::string& str) {
      std::string as_lower(str);
      absl::c_transform(str, as_lower, std::toupper);
      return as_lower;
    }

to enable:

    int main() {
      std::cout << ToUpper("hello");
      return 0;
    }

but this would be no good! You have chosen an API that forces your users to be gratuitously inefficient. Maybe that is OK for your personal project, but an API like that has no business in the standard library.

You just explained why C++ will never be as productive as Python.

No one is stopping you from writing your C++ inefficiently if you choose to. I am skeptical that spending a bit of time thinking about ownership/lifetimes is really such a big hit to productivity.

> Python and other dynamic languages don't use static types and that may give them a (very) small advantage in productivity as well, but only for very small programs... as program size increases, in my experience, statically typed languages take the lead in productivity... as OP is about writing general software, not just small scripts, I really have a hard time agreeing that Python is either the best tool for the job, or the most productive tool at all.

As a counter point to this - Python has a large number of C-based libraries with excellent bindings. To take Numpy has an example, you can the static-types and speed of Numpy to do all your maths, whilst Python acts as the manager. Using Numpy "feels" like normal Python, you don't really feel like you're working at native-level, but you get all the advantages of native speeds and memory usage.

And to your other point, I've worked on large C++-based projects which were a complete mess, especially when you had to make any changes to existing code. I'm not sure the language has as much an affect here as just using the correct design principles.

I've done big desktop program in python. For some parts, we just used modules written in C, because code was easily 20 times faster than python, I was even surprised at how fast simple blits could be on modern hardware without any graphics card acceleration. But python version was several times faster to write.

> but there's very little evidence to support that

Not if you don't go looking for it: http://www.norvig.com/java-lisp.html

ETA: And don't get me started on Design Patterns: https://norvig.com/design-patterns/design-patterns.pdf

> programmers tend to be most productive in whatever language they know best

This is true, but most software is built by multiple people (for corporations) and will eventually be maintained by multiple other people. Whether an individual is "most productive" in a language is rarely important.

I'm a C++ programmer and a python programmer. I assure you your conclusion is incorrect. Python with type checking is by far more productive. Many people who know python and another language can testify to this general truth.

C++ is probably the least productive non-joke language ever created: it has deterministic destruction which requires (manual) ownership tracking, it has an extremely slow compiler with horrible error messages, it has massive performance differences between debug and optimized builds, it's focus on performance brings lots and lots of extraneous concepts to its libraries when just seeking to get something working (std::allocator, std::string_traits and many other similar examples).

Comparing Python to Java, C#, Haskell, Erlang, Go, maybe even C or Rust, would show a much smaller performance difference.

> requires (manual) ownership tracking

Have you tried RAII?

Would you like to know more? https://stackoverflow.com/questions/2321511/what-is-meant-by...

RAII is exactly what I mean by "manual ownership tracking and deterministic destruction". RAII is better than completely manual memory management (C style malloc/free), but it still requires you to design your program such that every piece of memory is owned by some pointer with the correct properties. You have to choose between copying, bare pointers/references, unique_ptr and shared_ptr whenever two pieces of data are related to each other, or whenever you pass a piece of data to another function.

Rust is a little better since at least the ownership concept is known to the compiler and automatically enforced.

(Tracing/Copying/Compacting) GC is much easier since this entire concept goes away. You always pass references to data, and the physical storage is "owned" by the GC itself. It's also much faster for certain workflow patterns, though it always consumes more memory than deterministic destruction schemes.

I do a lot of Rust programming, I personally find Rust more productive right now.

Granted, people come from different backgrounds, and "the right tool for the right job" always applies. But it's undeniable that generally speaking, ie. not considering any specific cases or requirements, some languages are definitely more productive than others.

Sometimes you just want to throw entirely differently shaped data into the same array or dictionary and be done with it instead of creating a "proper" type-safe design which most likely means writing lots of pointless boilerplate code. IMHO that's the whole point of a scripting language like Python, to write quick'n'dirty scripts, not "real programs".

Reply on news.ycombinator.com