Hacker News Reader

Comment by ktta | Hacker News Reader

I don't have much experience with any kind of stock market analysis/ HFT programming.

I've seen people use python (or R) a lot in online tutorials and courses (there's one on coursera[1] check it out). It's understandable that python is preferred since it's an easy language to get started with, but I've never really seen C/C++ used by people online (Although I've heard about the use of FPGAs by the hardcore guys)

Is the only reason C++ isn't recommended because it is difficult? The performance difference seems rather large to me. And especially if you're doing lots of number crunching, it would really benefit using C/C++ wouldn't it?

Do people use them but I don't hear about them because anyone above 'tutorial level' doesn't share their code or talk about it?

Or are the R and Python Computational libraries close to the performance of C/C++ that it's a viable compromise?

[1]: https://www.coursera.org/learn/computational-investing

View on HN

I don't think any retailer can trade on the scale and speed that would require C/C++ over Python. With lower capital and significantly higher transaction costs, retail traders (mostly) need to have holding periods for at least hours, so speed isn't really an issue. It's more about finding longer term anomalies to deploy maybe at most a few dozen positions, as opposed to reacting to billions of quotes over any trading day.

I think Python is a great tool for finance, particularly with libraries like Pandas, Numpy, etc. Most of the work in those libraries is C/Fortan/etc anyway. Great community. For more 'academic' style work/exploration, I think R might have an edge, but I personally find it more difficult to write and maintain high quality code in R.

I'd like to just throw a quick plug in for an open source project I'm working on [1]. It's an event-driven trading library written in Python. We're pretty close to having live trade execution ready in an alpha state.

The motivation is that you can put together a pretty simply strategy in ~40 lines of code. The codebase is designed to be pretty flexible & modular. Mike from http://quantstart.com has some great resources & many of his recent articles use the same library.

[1]: https://github.com/mhallsmoore/qstrader

Given the network latency of 43ms between, say, Houston, and NYC, you are completely correct.

Source: https://ipnetwork.bgtmo.ip.att.net/pws/network_delay.html

HFT is only a small part of financial data analysis. Everybody I know who works with mathematical modelling in finance use tools like SAS, Matlab, R and even Excel to build and run their models. If you're working at something like a pension fund and thinking years into the future then shaving a few milliseconds or even minutes here and there isn't just isn't a priority.

People use Python for finance and time series because of how easy it to read in the data. There is a library called pandas which allows you to read in a txt, csv, or excel file easily and do any type of array manipulation you want. The pandas library is written in C++, so it's basically a python wrapper.

Conversely I don't think C++ has this same type of library for reading in data in a single line.

While C++ is faster than Python overall, for most financial simulations Python is definitely fast enough.

Pandas was started as a skunkworks project inside a big hedge fund (AQR) by Wes McKinney (who after a few years at Cloudera is now at another hedge fund, Two Sigma). So it's no coincidence that pandas is very well adapted to financial / investment analysis.

The Pandas library is not written in C at all actually. You can view the source here: https://github.com/pandas-dev/pandas/tree/master/pandas

It's mostly Python with a bit of Cython, and pull requests that are not pure Python are more likely to be rejected. There's basically zero C++ in Pandas itself.

Speaking only for R, most of the performance-critical functions included in packages these days are just wrappers to a C++ implementation. It isn't until you need to implement something custom that performance starts to be an issue.

That combined with it's ease-of-use working with tabular data, and the myriad of packages available make it a very popular choice.

If Rcpp were not a thing, R might have died from being slow. Since it is a thing, though, people can easily write packages that are certainly fast enough for what they want to do, and that keeps the language useful.

The main bottle neck in data analysis is not running the code but thinking about your data, visualizing it in different ways, and looking for interesting corner cases your model is not good at predicting.

In that sense it is much faster to do these analyses in Python or R than in C++.

>Or are the R and Python Computational libraries close to the performance of C/C++ that it's a viable compromise?

That's a big part of it. Python's data analysis tooling is generally written on top of Numpy, which is insanely optimised code that an average C/C++ developer couldn't compete with in terms of perf. Numpy will completely smoke naive C code to do common data manipulation tasks. So it's a double win, really. You get very good perf out of the box, and you get a high level dynamic typed language which lets you focus on high level logic and iterate quickly.

The other part is the ecosystem. For a variety of reasons, Python has become the premier language for Big Data™, and over the past 5 years has accreted a huge collection of libraries for analytics, visualization, ML, distributed compute, etc. A single developer can now click these libraries together to achieve stuff that could only be done with huge teams before. You can build a distributed training cluster for a deep learning algorithm and deploy it to Amazon in maybe 2k lines of code. C++? I don't even know where to begin.

This. Python's stength as a language IMO comes from it's ability to interface with more efficient code. You can write the meat of your library at near C levels of efficiency, and have python treat it like a "black box" essentially. That seems to be how NN code works often as well.

Given that the R interpreter is written in C and admits trivial FFI bindings, as demonstrated by libraries like glmnet or gbm calling out to it, I don't see how this is an inherent advantage of Python.

While it doesn't have an inherent advantage, it has the mindshare and momentum of a community that has these tools now.

R could be just as capable as Python, but I think Python has largely won the race to be the most popular language for data analysis which in turn encourage more developers to commit to it, cementing Python's advantage.

R still has solid lead in statistics and a good mindshare amongst academics.

> R could be just as capable as Python, but I think Python has largely won the race to be the most popular language for data analysis which in turn encourage more developers to commit to it, cementing Python's advantage.

Your comparing Apples and Oranges. R is a domain specific language and will never be a general purpose language.

It is not true that Python won any race in statistics. http://www.kdnuggets.com/2015/05/r-vs-python-data-science.ht...

Let alone in industry investment coming from Microsoft and other major players.

R is above Python in Statistics in momentum and numbers. Python is a good choice but Python is still playing catch up to R due to the speed at which R is developing. R with data.table and Hadleyverse (https://www.r-bloggers.com/welcome-to-the-hadleyverse/) and RStudio the momentum has been clearly on the side of R.

R just 5 years ago was a fraction of the users it has today.

Python and R are both good choices with equal speed but the difference is that R is a domain specific language that has a lot of positive ecco system.

R is a LISP. I would disagree heavily with it being domain-specific. It is as capable and Turing complete as any language. The only argument you can create is about performance and the judiciousness of putting stats functions in the base library, as opposed to Common Lisp which ships with even less. Not only "will" it be a general purpose programming language, it already is.

Worth noting that there's a difference between analysis/research and a trading platform etc.

I'd guess that low effort threshold to try and backtest ideas is very attractive even if you're going to have to rewrite fast/better/stronger in other languages to trade in real time?

It mostly depends on what you want to do. If you want to try to beat the speed of the internet like some of the HFT stories you hear, you're going to want to cut as much fat out as you can. If you are more interested in doing some analysis of historic data where you don't care about speed so much, it's a lot easier to play around with things in a language like R or Python.

I hear that R/Python's role in the project is initial modelling and testing. You train trading algorithms in R or Python, then deploy them in a C/C++ algorithm for speed.

NumPy is C++, Fortran, and sometimes CUDA. Depending on usage.

> It's understandable that python is preferred since it's an easy language to get started with

Python is a good choice but R is amazingly good language that doesn't deserve to be pushed back negatively.

It's not easier to learn Python then R. They both have plus and minuses but knowing both if I was to teach someone statistical programming I absolutely would teach R over Python.

Numpy for Python provides wrappers around C arrays, so (apparently) comes close in terms of performance.

Kdb is used in many large banks.

Reply on news.ycombinator.com