Using a Keras Long Short-Term Memory Model to Predict Stock Prices

This is pure nonsense. This isn’t even the right way to begin thinking about this as a forecasting task — the target series should be log-normal returns, not raw asset price. The performance of this model is laughably bad, which is probably why he spends zero time evaluating its effectiveness. You could trivially get better forecasts than this by naively repeating the last-observed price.

This isn’t ML. It’s cargo-cult performance of words and ideas that ML people use.

I'm not a purist believer in the efficient market hypothesis. BUT, I doubt there's much alpha to be gained simply from looking at price data which is widely and publicly available. Also keep in mind that markets are dynamic feedback loops so even if this model had an edge, the act of publishing this article would work against you to neutralize that edge in the future.

There's a good reason the most successful 'quantitative' trading fund, Rennaisance Technologies is so secretive and subjects its employees to a lifetime NDA/non-compete.

If there is edge in the market (and I think there are anomalies that can be traded on), I would assume it to come from correlating proprietary datasets with price. Or datasets that have a high barrier to access/analyze. One example that I'd bet still works in some industries would be counting trucks from a supplier to estimate product demand before the companies announces earnings ;)

All these LSTM Stock Market tutorials seem to be a variation of this guys tutorial which he did a while ago:

https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Se...

Except Jakob explains the folly of the method.

Whereas the author of this piece says the following:

From the plot we can see that the real stock price went up while our model also predicted that the price of the stock will go up. This clearly shows how powerful LSTMs are for analyzing time series and sequential data.

The problem with these types of systems are it's difficult to do backtesting on a single continuous data stream... I write about that here:

https://blog.projectpiglet.com/2018/01/perils-of-backtesting...

I recommend doing something similar to the original post. Neural networks tend to produce valid looking output for a stock price easily, a random walk does too. You have to find correlations, causations, and then review the results carefully.

For reference, I wrote my own financial advisor (not directly utilizing deep learning, that functions relatively well):

http://projectpiglet.com/

It works relatively well when checking the "causality" which also has it's limits: https://blog.projectpiglet.com/2018/01/causality-in-cryptoma...

EDIT: Added some evidence:

* 2016 stocks: https://imgur.com/a/j8YWR

* Early 2018 crypto: https://blog.projectpiglet.com/2018/01/30-weekly-returns-usi...

* Early 2018 stocks: https://twitter.com/AustinGWalters/status/976347632439209985

* All 2018 on Robinhood only: https://imgur.com/a/2CxEFqI

* Also (I am lettergram on HN), created: https://hnprofile.com and https://redditprofile.com

I preliminarily did the same thing as in this article a while back (with crypto data, since it's easily accessible), just as a way to learn keras.

Somehow my network always learned to output a delay of the input no matter how hard I tried to shape it. I've searched through literature briefly and some examples. Some blog posts even claimed they had good time series prediction when I clearly saw they were having the same problem.

What I mean to say is, be critical of what you find online, and be critical of your results. This is definitely not an easy problem, and some argue that it's unsolvable all together :)

Do not minmax scale the data. New data outside ranges seen before will be chopped off. Do make relative (high/low), so patterns you find generalize to other domains. Don't fit complex LSTM models on tiny datasets. Somehow this is the shiny thing all starters jump on, while the most simple models are the most proven and robust. Don't just share such a model without giving a disclaimer ("I never tried this out on real data, and would not invest my own money in this solution"). This is real money, real economies, and you could horribly crash these. That is also why you don't write with authority about how to build a bridge (while you never crossed a bridge you've build before). The only reason for the "clearly showing how powerful LSTM models are" is it responding to test set samples it got wrong, and correcting with a lag. How about some evaluation, preferably on another stock than trained on?

These types of experiments pop up from time to time. You have to compare the performance relative to the single lag error. So the baseline is to use the previous time step to predict the current time step. Ultimately, it's not worthwhile to attempt to predict stock timeseries the SNR is far too low. It's more valuable to attempt to predict a trend, e.g., +1 is there will be upward movement in time window x, -1 if downward, 0 is within some epsilon of the current price.

11:15, restate my assumptions: 1. Mathematics is the language of nature. 2. Everything around us can be represented and understood through numbers. 3. If you graph these numbers, patterns emerge. Therefore: There are patterns everywhere in nature.

This doesn't take away from the exercise, but I notice the model seems to "predict" the movement just after it occurs.

It just lags the current price. If there is no alpha in the price signal, then the rational thing to do is to just stay at the current price, which is what the algo seems to have learned to do.

Pretty underwhelming. From the very limited experiment demonstrated, the LSTM seems to be playing catch up rather than actually predicting trends.

You can use whatever prediction technique you like, but if your model is wrong, then so will the prediction. Predicting stock prices requires considering as many factors as you can gather that goes into setting the stock price, and how the factors correlate with each other.

> previous price of a stock is crucial in predicting its future price

This is a poor and incorrect model. The stock price has plenty other variables and many of which are unknown. What's used here is just a single variable.

This is complete rubbish. Zero evaluation compared to any sane baseline, zero thinking about representation (predicting raw price?!). I'll just say it, ML was frankly better when fewer people knew it and there was less money in claiming to be an expert.

Signal/noise on the market is extremely low, and determining whether you’re predicting something or just data snooping your way to a function that looks like it predicts something is a serious competitive advantage in itself (... because all you need then is novel functions to try).

Working out a function that passes a split test is inevitable and easy tbh. That function then making money is highly unlikely. This is especially true if your data sources are public.

My favourite part of this is that Francois Chollet (the author of Keras) specifically warns against this exact use of LSTMs in his book (Deep Learning with Python)

The author uses stock market as a toy example. He is not actually attempting to do serious forecasting.

Only comparing the prediction accuracy against simpler models like ARIMA or VARMAX (as the author suggest) can tell you if the model has any use.

Again, as always, and every time -- if a stock market price-predicting scheme worked, it would not be published on the internet for free. It would not be for sale. It would be in constant use to make money for the authors.

I'm skeptical of any system that proves it can predict stock prices vaguely.

I would be less skeptical if it was more humble, and specific in its constraints. For example, could it predict the % probability a stock would up X% if it had N consecutive negative days? Or, how many minutes/hours after earnings are announced is the after-hours price usually indicative of what the price will be the next day?

You can't predict everything, but you might be able to have a probabilistic prediction of specific scenarios. though even then I'll still be skeptical.. daily stock movements are totally random.

I saw some guys present who did this with currency trading. They had to retrain the model every night and it didn't trade for the first four hours after the market opened.

Total noob question in this space (algotrading with DL algorithms).

What DL is good at, is to automate tasks that are easy for humans (e.g. telling cat from dog, understand a sentence from sound wave, translation, etc) but hard for machines without DL.

Now, if a task is even hard/borderline impossible for humans to achieve (e.g. predicting stock market with higher accuracy and consistency), why would we believe DL could do a better job than humans?

These predictions and models work until they don't. And when they don't, they blow up in spectacular fashion.

Look at Long Term Capital Management. It had big names behind it and the model/strategy made a lot of money, until I believe the Russian bond crisis caused the markets to act irrationally. Then everything they had invested in quickly went down the toilet, and it was big enough to almost cause a financial crisis.

I wonder if someone could comment on how AI is used at big investment banks? I assume its much more than technical analysis (like this model). I imagine that NLP might be helpful for quickly ingesting a news feed and then making buy/sell decisions based on the news in less time than humans can read and react?

I don’t know what’s more embarrassing. This article, or the fact that it’s #1 on HN right now.

Is this graph at the end a one-step-ahead prediction? Because if it is, then it looks very much like a 50/50 success/failure ratio. Which is what I would expect, because otherwise you could extract a signal from it.

I cannot fault your enthusiam but I can fault your method :)

Read Tsay's Financial Time Series Analysis for a better idea how to forecast financial data.

The biggest problem you will face is stationarity: the statistical properties of the data is not constant over time. For example, the mean and std dev is not constant over time. Using returns instead of raw prices helps to make better financial forecasts.

Two methods to explore:

1. You are better off predicting stock prices by predicting future returns and then forecasting is the current price plus predicted future return.

2. You could use your neural model to predict absolute size of returns using realized volatility.

No, this is a bad example of prediction; if you bet money on the direction of prediction, you'll lose much. Drawing prices and making it looking as similar is not correct prediction. At least you should show >50% correct direction prediction if it is considered to be "well processed" machine learning.

From my experience, LSTM or other recurrent neural network models only work "well" at forecasting bounded and periodic or oscillating time series. Might work for something like seasonal sale data, but would fail spectacularly with unbounded and chaotic time series like stock prices.

I used to run a technical analysis site for exchange traded funds (ETFs).

Skipping to the "results" of this experiment, I am seeing nothing. It looks like a slow, short period moving average (SMA)... which does nothing in terms of predicting anything. It just averages out the chaos over time.

> From the plot we can see that the real stock price went up while our model also predicted that the price of the stock will go up. This clearly shows how powerful LSTMs are for analyzing time series and sequential data.

Is laughter or tears the right reaction here?

I know nothing about ML and not very much about finance, but I find it incredibly unlikely that some random guy on Medium has an algorithm that can be used to accurately predict the market. Why isn't he hilariously rich?

I believe a simple moving average would achieve the same result and that mixing training and test data for validation invalidates your model results...

On a positive note, nice showcase of how simple implementation with Keras is.

Seriously, how on earth is this the news #1 of the frontpage of HN? Isn't this community supposed to know better?

Before clicking I thought "I bet they do stacked LSTM to predict the _next_ time step" :|

Let's take it easy on the author... LSTM's are cool especially when you first grok them, and stock data is the most abundant time series data out there.

Think the important thing is model performance and tuning hyper parameters which is missing from this tutorial.

Without that you are just regurgitating some model that does something.

Simple linear regression would work just fine for something rudimentary like this. This using neural nets stuff for everything lately is getting ridiculous.

Regardless of whether or not this would make anyone money, it's a really nice introduction to forecasting time series using LSTMs. Thanks for the post!

Shame on me for looking. But this is a account that post only from fritz.ai -- might be nonsense spam.

Well, I'm only on mobile. But the problem with stock prices is often that you don't have good data, unless you are working for the big four or whatever. And even then, they are apparently still working on integrating external data in real time.

Imagine having the whole Twitter stream, maybe categorized by some network in an automatic way.

Also prequential evaluation.

Feed it 3 cycles of a perfect sine wave. What does it predict?

It's tough to make predictions, especially about the future.

-- Yogi Berra

Predicting stock prices with deep learning is the "Killing Baby Hitler" of Time Travel. Even though they know it won't work out, everyone tries it their first time.

Isn't the concensus these days that every bit of information is squeezed out so that the stock price always reflects all that is predictable?

In addition, major factors are external and trying to predict further price only based on past price is bound to disappoint.

Oblig XKCD https://xkcd.com/1570/

Seriously tho', articles like this need to come with a big banner at the top saying warning: do not try this at home.

wonder how effective those models will become when everyone starts believing in those models to do trades.

Why on earth there is supposed to be even a correlation between so-called past and future (bullshitting as a business aside)?

It is like predicting Trump by Nickson or, even better - predicting a next Trump.

We'll know if it works when he gets to be a trillionaire.