lr-price-prediction-tsla-20210718.1.csv

Hey Zack, I came across your article about using linear regression to model closing price with EMA10. It seems that you were using EMA10 calculated using price on the day of to predict closing price of the same day. Did I understand that correctly?
I know this is a toy example, but wanted to make sure I understood your method! Thanks!

That's correct. IMO, there's a strong case for shifting data in some cases though for this article I felt the EMA--given it accounts for the previous n-many periods (10, in this case)--didn't need such a shift.

Implementing a function to find an optimal lag period to maximize correlation could 100% be a better approach in some cases. Maybe even the EMA case--I didn't run a comparison test.

If one were to use a feature that was an observed value for a single period like "open price greater than 30% above the 52-week low," finding a shift might be useful.

Here's a link to an article that does an OK job offering an example of such a case: https://wtm695450085.medium.com/example-of-the-use-of-shift-for-linear-regression-in-python-how-to-find-optimal-correlation-shift-d86980e82148

The article you linked to about correlation shift is an interesting read!

My concern with using EMA at time t to predict the closing price at time t is that when you get EMA at time t, the actual closing price would have become available at time t. This essentially turns the problem into correlational inference instead of prediction. Trading based on predicted price at time t when the actual price of time t wouldn't provide us an edge.

I agree with your concern here. The article was casual in nature but this is definitely a fundamental enough issue it should be addressed. I will definitely plan on updating the article. Out of curiosity, have you done any experimentation using a 1-period shift for such cases?

Hi. Coud you please share the code for generating this table and the one for the previous graph, of the linear regression with the the predicted values, real values and residuals?
Thanks!

can you please share the source code for generating the table and graphs?