A Comparison of Programming Languages in Economics [pdf] | Hacker News Reader

A Comparison of Programming Languages in Economics [pdf]

economics.sas.upenn.edu

36 pointstodd820 comments

As having both, an economics and computer science background, I often feel like economists focus on the wrong metrics when evaluating programming languages.

The importance of raw computing speed of a programming language pales in comparison with the importance of the applied methods, as well as the time saved in development. For example, while a speed-up by a factor of two sounds good on paper, the choice of search strategy in a constrained optimization problem can change the running time by several orders of magnitude. I am currently working on a model where the seemingly unimportant choice of price normalization (i.e. which price in the model is set to 1.0 to determine the price level) can make a difference between 2 seconds or 200 seconds of computation time.

Meh, they dismiss java & pypy very quickly but they don't understand that these languages require warm-up. If you benchmark runs in less than 2 seconds half of the time measured is starting the runtime, profiling & compiling code. I think that interesting computational problems in economics have a much longer running time.

For this kind of algorithm, basically working with arrays of doubles, java should be within 20% of c++ and normally it is just as fast.

Would you have any reference regarding the performance of java/c++ with respect to manipulation of arrays of doubles? Just curious to know where that 20% comes from.

The 20% comes from personal experience, maybe I should have said 'close' but that can also mean anything. Basically this is an area where the hotspot heavy duty compiler (C2) does the same optimizations as gcc or llvm. I would say that I would expect gcc & llvm to be within 20% of each other. The 20% is basically different tradeoffs made in the optimization pipeline (different peephole opts, different auto-vectorisation, register allocation algorithm, ...).

Java starts to seriously lags behind C++ when you need good control over the memory layout of your data. In java you can't have an array of objects, you get an array with references to objects. There are no elegant ways to solve that problem, the language just lacks the proper tools to do it elegantly. Memory layout is of course vital for cache efficiency, so that can give C & C++ a big advantage over java. Since this benchmarks uses arrays of doubles, this isn't an issue.

I see your point now, thanks for the explanation. I mostly do scientific calculations, so effectively manipulations of array of doubles, and I currently mostly use C#. I was under the impression that the performance is similar to C++ (GPU aside), for that the comparison with Java had got my attention.

Even though C# is in many ways like Java, it allows (via unsafe blocks and pointers) to better perform memory management. Not sure how widely it is used in practice.

Does Java have any way of auto vectorization? On modern CPUs SIMD can speed up numerical computation several times easily. However auto vectorization is hard in general, so compilers don't necessarily do a good job and manually optimized code is common. C++ has TMP and libraries such as Eigen use TMP to provide a nice interface to SIMD optimized code. I don't know whether Java has equivalent of compiler intrinsics for SIMD instructions, without which Java code would be far slower for many numerical computation. I believe C# has plans to add an abstraction for SIMD and Mono can already emit SIMD instructions.

Intel recently contributed improved auto-vectorisation for artithmetic to the hotspot compiler. Auto-vectorisation of move instructions has been in hotspot for a few years.

>> If you benchmark runs in less than 2 seconds half of the time measured is starting the runtime, profiling & compiling code.<<

Please show that it takes 1 whole second (measured after the Java program starts and before it terminates) for the JVM to profile and compile the 100 lines of code.

That would suggest the JVM was pure junk.

>>half of the time measured is starting the runtime, profiling & compiling code<<

Here are 10 cold-run times reported by their program:

    Elapsed time is 3.122233018 seconds.
    Elapsed time is 2.970587912 seconds.
    Elapsed time is 3.110304658 seconds.
    Elapsed time is 2.886701204 seconds.
    Elapsed time is 2.966404493 seconds.
    Elapsed time is 2.905329605 seconds.
    Elapsed time is 2.968715703 seconds.
    Elapsed time is 2.922929706 seconds.
    Elapsed time is 2.906730019 seconds.
    Elapsed time is 2.919119506 seconds.

Here are the stats reported by JMH after 200 samples:

    Result : 2.950 ±(99.9%) 0.015 s/op
      Statistics: (min, avg, max) = (2.932, 2.950, 2.984), stdev = 0.017
      Confidence interval (99.9%): [2.935, 2.965]

There is a problem with their representation of Java, and it has nothing to do with "require warm-up".

The problem is that their Java program is just a single static method.

Their Java program is not written as Java.

https://news.ycombinator.com/item?id=7978410

Lots of discussion from a year ago.

I would be happy to use languages like J/K or Julia for our actuarial projection simulations instead of C++. However, despite quicker development times, a factor of 2 in the runtime for our stochastic projections would be terrible. Has anyone experience with those languages in an HPC environment and can compare it to a C++ implementation?

Dang, someone wanting to replace C++ with J... I'd love to say you could do this, since I am a big fan of J, but if you need to squeeze the last bit of efficiency out of a machine on a problem that can't be parallelized, C++ is probably a pretty good idea. If the "expensive in write time" logic isn't expensive in calculation time, you could wrap the expensive pieces in some verbs which call out to C.

I'm often surprised though; a naive implementation of wavelets in J was quite comparable in runtime performance to a naive implementation of wavelets in C.

Try numba in python. Its usually fortran fast and often faster than Julia.

github: https://github.com/jesusfv/Comparison-Programming-Languages-...

"Java's issues with floating point arithmetic in high-performance scientific computation"

Can someone explain this?

Unlike C++, Java ensures that floating point calculations always yield the exact same result, regardless of the underlying platform. This can degrate performance.

I am not sure if they meant that thought, in my opinion this is an advantage of Java.

Floating point calculations are only guaranteed to be platform independent when you use the 'strictfp' modifier.

One of the issues is, or used to be, data locality. Arrays can be all over the place. Lots of cache stalls that way. There are tricks to deal with it, but I've never seen it in open source code. Might as well just use blas at that point.

Reply on news.ycombinator.com