Hacker News Reader

Comment by jarym | Hacker News Reader

Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the market.

I.e. places with defined risk where you will know if you’re wrong if it goes against you by x% while you expect a y% gain if you’re right AND y>x is worth more than the number of times you’re wrong.

The types of Algos that work well for this are edge identification ones - I know this because I am (not as well as I’d like) successfully doing it.

LSTMs haven’t performed so well for me in this task but non-NN algos have. CNNs however were promising but didn’t match what I’d come up with - still searching for the holy grail that’ll make me rich!

View on HN

.. because you buy at the price, and sell at the price (spread and fees ignored for now).

Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit.

Predicting price change and predicting price are semantically equivalent, although a specific algorithm might be better at one than the other.

Predicting price means you’re predicting one variable with no idea of hot likely you are to be wrong and how wrong you’re likely to be and says nothing of where your expectations are for price to go after.

It is semantically different to say: if price goes to Y then you have odds that it will then go to Target 1 and then slightly lower odds it goes to Target 2.

Do you long now, or short now, or neither? If you go long, you’ve predicted price goes higher. Regardless of what you think about the entire path. Mathematically nothing else makes sense.

Given less than 100% certainty, traders don't want to predict price, they want to predict future distribution of price over some time period.

Source: hedge fund trader

True. That’s still considered a prediction of price among my trading colleagues.

You want to predict price, but a price prediction is useless without an accurate estimate of the error of your prediction.

Replying to myself instead of all three replies so far:

People, prediction is a general term. Many predictors come with accuracy estimates (and outside of finance, often prediction bounds). But even if it was only one number - if you have good prediction of the expected price change, that could be sufficient to trade as it encompasses, by definition, the sun of probability of different outcomes times their magnitude.

Either E[price] or E[log price] is a single predicted value you can successfully trade with as long as you are far from your margins, and depending of course on your utility functions.

But as I mentioned, in most fields, when you talk of a “predictor”, that’s not a single number but also accuracy estimates or even a full fledged probability distribution of future events.

Right, but that's not what this article (and several others I've seen) are doing - they are measuring the performance of some AI by putting on some unnecessary (and quite likely impossible) constraints on what it is supposed to be outputting.

All I'm pointing out is that measuring any stock trading algo by treating it as a regression problem for the exact next time step is a naive approach - that's not the same thing as what human traders are doing.

Anyway, if you are trading then I wish you lots of success.

> Why does everyone naively try to predict price? No ‘traders’ are interested in predicting it - what traders do is identify good locations to enter or exit the market.

I agree.

I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that).

Back then, in the indian market, you could see that most of the stocks, although skyrocketing upwards, all followed the slow vs fast moving averages to buy and sell! Back then, they weren't looking at RSI, stochastics, support lines, etc, etc. It was crazily predictable...but over time it was really interesting to see it become more haphazard and like western stocks. That is, the fundamentals came into play and as you say, the traders began to use other metrics to buy and sell.

How much did you make?

> I've built many systems in this area, but it wasn't until I started working in the Indian market (>10 yrs ago) that it became abundantly clear that trying to calculate the long/shorts signals using historical (/time series) data was a waste of time. (And yet my primary role was to provide tools that did exactly that).

I'm currently working on building similar tools in my area of work for the Indian market and would really appreciate if you could shed some more light into the things you learned from your experience in working in this domain.

Better, why do people think they can predict long term dynamics of what is basically a chaotic system? I think finding some small local dynamics is fine, but applying a neural network to try say something about global long term dynamics complete garbage - ie long term weather simulation, stock markets etc.

Chaotic systems can be deterministic, just that you will never be able to accurately measure all the variables to make a long term prediction accurately.

In the weather example, people know the equations that approximate how it works. Why value does a neural network bring? Knowing the equations is better understanding.

There was a Quanta Magazine article talking about predicting the evolution of a flame-front using ML. The ML algorithm remained accurate for eight Lyapunov intervals; eight times longer than the previous SOTA.

https://www.quantamagazine.org/machine-learnings-amazing-abi...

The only real argument I think you can make is that it might be more efficient to have a neural network quickly spit out an approximate solution instead of solving the actual equations.

But if you have the time having the actual equations is more valuable?

Assuming that "having the actual equations" is possible, which in market predictions it most likely is not.

Is this what you are talking about? https://www.dukascopy.com/fxcomm/fx-article-contest/?Automat...

Can you share some pointers to more reading material on algos you consider more successful?

Can you point to any good sources for this? A search for 'edge identification stocks' yielded mostly irrelevant results.

For example this blog post (not mine): http://jon.io/machine-beats-human-using-machine-learning-in-...

Talks about the Mean Shift algorithm described here: https://en.wikipedia.org/wiki/Mean_shift

Reply on news.ycombinator.com