Stock Price Prediction with LSTMs

Predicting for only the last year is good, but there are two even better ways:

- An expanding window where you train on the first year of data and predict the second. Then train on the first two years and predict the third, etc.

- A rolling window where you train on years 1 and 2 and predict 3. Then train on 2 and 3 and predict 4, etc.

You need to show that your predictions work for any time period, not just the past year.

You'll see things like the global recession recovery and other major outliers skewing your data.

It's actually using just one day's data (one point) to predict the next day's data, let alone predicting 5 days.

Training with trends and patterns is like a technical trader looking for patterns. When things go right they attribute to the pattern, but when it’s wrong, they attribute it to unforeseen events. The unforeseen events is something if can be predicted, you will be rich lol

it is; by scaling his single variable before making a test-train split, he's allowed his model to know how the future is distributed.

Ditto. There’s no way this is projecting almost a year into the future and maintaining accuracy. It’s being trained on the test somehow or it’s doing something different from what I’m interpreting.

for one, the author scaled their mean-OLHC values _prior_ to making their train-test split, meaning data from the test set has leaked into the training set. Time series prediction is a bit easier if you understand how the future values are distributed!

That's because it's a one-step behind prediction. (If you go to the linked notebook and look carefully at the graph its evident.) If you could shift the prediction one step forward, you'd be rich!

It's because the test dataset involves randomly choosing samples spread throughout the time series.

Imagine dropping 10% of the pixels randomly spaced out in a historical graph and then try to fill them in. It's trivial because you can just average the previous and next sample and have an extremely high accuracy.

I'm the farthest thing from an expert but here are some factors to consider. You're going to train on the past data. Typically at least to begin with you pick some time period and train on part of the data and keep back some to test on. And yes, there's definitely a risk of overfitting. So some of the real data will be in sample and some out of sample for testing. If you want to get more sophisticated there is one of a number of resampling techniques in which you systematically choose different subsets of the data to train on and then look at the variation of the predictions. That can give you some idea of the error you'll get. Not sure how that works on data with high dimensionality but if you're doing an autoregression/autocorrelation (i.e. predicting tomorrows price based on it's past behavior) you'll only have the single time series. The first thing you'll likely notice if you try and apply that sort of model to stocks is that stocks are at least as highly correlated and likely much more so to the market than they are to their own past behavior.