Castor: A C++ library to code “à la Matlab” | Hacker News Reader

Castor: A C++ library to code “à la Matlab”

github.com

I don’t see the justification for the end user to do their programming in C++ versus Python+Numpy or Julia.

Python is already the de facto replacement for Matlab in academia and engineering.

And Julia is up and coming, offering many of the benefits of Python+Numpy but designed with performance in mind.

Why should I bother with compilers and make files and deal locating memory and all the rest of C++?

Because that’s what I’ll have to deal with as soon as I take one step outside of Castor’s (admittedly friendly syntax) and with to do other things in the same language.

Why? Speed and integration.

How much of a speed up versus Python/Julia are we talking about? I didn’t see any direct comparison.

I’m not sure I understand what you mean by integration.

It's linear algebra will be the same speed as python/Julia (they both call to BLAS/LaPack). It's diffeq solvers will almost certainly be worse than Julia (Julia is SOTA there). Not sure about the rest of the functionality.

> It's diffeq solvers will almost certainly be worse than Julia (Julia is SOTA there)

Worse in what sense? Speed? Almost certainly not, since you would be implementing the solution directly in C++, so it will be as fast as you need(since you can use inline ASM). In Julia the really good diffeq libraries are not built in and this is not a diffeq solving library, so you would have to implement them yourself if you were just using this(so you can use almost exactly the same algorithms, and with more control over memory management in C++ you can maybe have it be a bit faster).

I also imagine that with C++17 or higher tooling, alongside something like CINT, it starts to feel quite productive.

Have you seen DifferentialEquations.jl? It's an incredibly fast suite of solvers that pretty consistently is faster than Sundials/CVODE/lsoda or any of the other C++ solver. (for a bunch of benchmarks vs pretty much everything else, see https://github.com/SciML/SciMLBenchmarks.jl)

Indeed, that particular implementation is very fast. One of the goals of the C++ language is to leave no space for a lower level language than C++. A motivated person could just re-implement those algorithms in C++, in the absolute worst case using inline Assembly and at the very worst exactly match their performance. One of the nice things about Julia is you get good performance without going through a huge amount of effort. However you could always do as good or better in C++(with maybe more effort).

Julia is pretty much the same but a few levels higher. Julia gives you some really powerful tools like @code_native (basically godbolt), inline assembly (or @code_llvm) something that , powerful macros, and better code reuse. If you look at the size of Julia libraries compared to similar C++ alternatives, the Julia ones are often about as fast (or faster), but typically around 10x less code. (my favorite example is that Julia's reader for Arrow files is more complete than the C++ one while being smaller than the makefiles for the C++ one (https://twitter.com/MoseGiordano/status/1489255661728796679)

> How much of a speed up versus Python/Julia are we talking about? I didn’t see any direct comparison.

These things may not matter to your application, but performance for me is more than just linear algebra. With C++ it’s straightforward to manage the working set so I can keep data with computational locality together, manage prefetching at the like. I can run the algos on multiple cores (no GIL, for example). Etc etc.

There is plenty of performance comparison between C++ and Python on the web, but alone the statistics mean nothing. Is your speed of development more important than runtime? For many people that would make Python faster. In my case I don’t use C++ as if it is C with other stuff bolted on so my development is just as fast in either language, so I choose the one which gives me more expressive power.

It’s not like I’m dissing Julia or Python, though my go to for little explorations is Common Lisp. They are just designed for different points in the solution space.

> I’m not sure I understand what you mean by integration.

Sorry, I didn’t mean mathematical integration, I meant integration with OpenCV, various robot hardware etc, as well as deeper integration with the hardware.

> They are just designed for different points in the solution space. I guess my problem is that I don't understand which point of the solution space Castor is trying to go for.

> There is plenty of performance comparison between C++ and Python on the web Sorry if I wasn't clear myself. I didn't mean for general python vs c++ comparison. I meant Castor vs the things it is trying to position itself as an alternative to (e.g. numpy or julia).

From my personal experience: system languages are really, really good at interacting with operating systems. When you have things deployed on a server that machine generally has one task, and it makes sense to optimize as many layers as possible. For example on a windows system I used MS's `FILE_FLAG_SEQUENTIAL_SCAN` flag to triple read performance; you could of course do the same in python by dipping into ctypes but it doesn't make for a pleasant development experience.

The big trade-off is you're specifying the task more precisely and as a result of that extra labor you get better performance for the life of the tool. SQLITE has a good story of putting together dozens of small optimizations, each <1% performance change, and at the end the library's throughput doubled in pretty much all tests. It's not worth it for research, but it is everywhere else.

> Python is already the de facto replacement for Matlab in academia and engineering.

Maybe in some disciplines, but in control engineering, basically everybody still uses Matlab. The python control systems libraries are a cheap joke compared to what Matlab offers.

The benefit of Julia vs c++ is questionable. Every time you run your program, you have to wait for it compile anyway with Julia. With c++, I pay this cost once, not every run. They keep claiming time-to-first-plot is better. When I tried it a couple weeks ago, it still took __minutes__ to produce a single, simple plot. JIT makes Julia not only frustrating to use, but worse, makes it unsuitable for integration into real time systems. So for a lot of applications, I have to rewrite the algorithm anyway whether I use Julia or python. So something like castor (especially with it's integrated plotting!) could save me time from re-writing.

As for the compilers and makefiles? Well, Julia's module/include/import system is worse. At least with a makefile, all the state is in contained within the local build directory, not some global cache julia magically creates.

How were you making the plot? On my computer, I see time julia -e "using Plots; plot(exp)" taking 13 seconds (and I haven't done anything fancy). If it's taking significantly longer than that, something is very wrong.

Generating a similar plot in Python takes less than half of a second, including startup time. While 13 seconds isn't "minutes" it's still egregiously slow.

Yeah. It's the main pain point of Julia. For me, it's not really a problem since for interactive use, I mainly use a repl which stays open for hours, and for production use cases, you can make a system image (which essentially caches all the compilation ahead of time).

Also, this is a major focus of current Julia development.

13s seems horribly slow to draw a plot.

I just pasted the code from the docs

    julia> using Gadfly
    julia> plot(y=[1,2,3])

I just timed this again. It took 53 seconds. So __minutes__ was a bit of an exaggeration. Using Plots instead of Gadlfy takes 20 seconds. Either way, it's an eternity. I'd rather just compile c++.

Tangent - what kind of "control engineering" are you using Matlab for? I call myself a "controls engineer" sometimes because I PID tune the occasional servo motor or write some PLC-like code but for the most part we use traditional coding (C#, C++ or similar) techniques.

Are you talking about much larger systems or more complex stuff that requires real controls theory?

The biggest advantage is the ability to deploy the code as static binaries to be used in other applications. Not needed if you're just scripting and plotting but a huge deal if a scientists fancy numerical code needs to be used in production once you're done prototyping.

Usually say people that don't do anything serious besides plotting some basic graphs :)

If you're already a C++ programmer this kind of thing is great since a lot of us already prototype in Python or Matlab so a glidepath to C++ is helpful. Otherwise I agree, prob go with Python.

Even less justification when you can use GNU Octave, which has the same language, can use the same scripts, has the same functions, and has Octave Forge to download plug-ins for those functions you could miss from MATLAB.

Wonder if Matlab's C++ auto-generated code is competitive in performance.

One thing is that this library has built in support for H-matrices(last I checked Matlab did not), so it can be arbitrarily more efficient if your problem decomposes in a nice way(most matrices in practice have large low rank sections, so this may be a very large class of problems).

It's really good.

I don't have any links or docs but I've used it quite a bit. With a recent of MATLAB, the codegen is good enough that my workflow has become "prototype in MATLAB" -> codegen -> profile then hand optimize. That is, for things which lend themselves well to MATLAB.

That sound fun, what kind of problems do you use it for? I can imagine inner-loop type math stuff but would be leery of larger chunks of code.

Most of the things I'm interested in are signal processing oriented. Things where it's very easy to structure the solution to the problem as dataflow/boxes and arrows. Essentially, "things that lend themselves well to MATLAB" ;-)

(to be more concrete, because I personally hate vague answers when I want specifics) I've done a lot of array processing sound stuff, microphone array processing and calibration, as well as computational music stuff with this framework. Also built a mostly vision-based pipeline this way, calculating how "awake" a mouse is from a video feed for medical applications.

> I can imagine inner-loop type math stuff but would be leery of larger chunks of code.

100%. I'd never structure an application this way, but it's pretty great if you structure your larger code to be modular enough and then have certain processing elements done via codegen from MATLAB. What I love is getting bit-for-bit accuracy with the math from MATLAB. Of course, you then do things to get more performance by sacrificing accuracy, but it's great to start off from a place of knowing with certainty that the math is right.

Thanks for posting this project but it will be helpful if the link to directed to the original JOSS paper because it seems that nobody is referring it in the comments [1].

In Hennessey's and Patterson's Turing Award Lecture, "A New Golden Age for Computer Architecture", they lamented about the difficulty of mapping the the general purpose programming language (Python, Java, C, Fortran) onto the Domain-specific Architecture (DSA) and suggested that the best way is to utilize Domain-specific Language (DSL). They've pointed to the significant rise of DSL and specifically mentioned Matlab, TensorFlow, P4 and Halide as popular examples [2]. The fact that Matlab is by far the most popular DSL compared to the other three is somewhat very interesting fact to note.

From the Castor's JOSS paper when referring to Matlab and Julia, "Despite the many advantages that these languages have and their high popularity, many codes are still developed natively in Fortran, C, and C++, for practical or historical reasons. Even if there are tools to automatically generate C/C++ code from a high-level language (as Matlab Coder), this work is often done manually by specialists." This precisely my sentiment of the current data science languages and my thought is that why not a language that has best of both world, native compiled language and a seamlessly integrated data science DSL that's much more than a library?

Enter D language, the only language that for several years now has BLAS library faster than the one implemented in Fortran [3], support C language in its latest compiler [4] and has been touted as the better C++ language by its authors [5]. What it really needs now is a seamlessly integrated DSL with Matlab syntax familiarity to win over the type A data scientist [6]. The DSL is better be implemented as CTFE that would enable higher level of integration with the D language [7].

As for another popular DSL mentioned in the Turing Lecture, TensorFlow, D has already has one minimalist TensorFlow (TF) alternative library and it's already faster than TF [8]. For the other two DSL, namely P4 and Halide, it will be probably happen sooner rather than later that the D alternatives will emerge and personally interested to explore P4 written in D language.

[1] Castor: A C++ library to code “à la Matlab”:

https://www.theoj.org/joss-papers/joss.03965/10.21105.joss.0...

[2] A New Golden Age for Computer Architecture:

https://cacm.acm.org/magazines/2019/2/234352-a-new-golden-ag...

[3] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen:

http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

[4] Adding ANSI C11 C compiler to D so it can import and compile C files directly:

https://news.ycombinator.com/item?id=27102584

[5] Origins of the D Programming Language:

https://dl.acm.org/doi/pdf/10.1145/3386323

[6] There are two types of data scientists — and two types of problems to solve:

https://medium.com/@jamesdensmore/there-are-two-types-of-dat...