Baselight

S&P500–11year History

Stocks from 2010 through 2020

@kaggle.shawnseamons_sp500_11year_history

About this Dataset

S&P500–11year History

Everyone has looked at the stock market so I wasn't expecting to get excited about looking for patterns. But it's a good practice set to develop skills and learn new models. After working through a deep learning model, trying to predict the best-performing stocks of the next day, my most sophisticated model regressed toward the mean. I mean the exact mean. I was working with 446 stocks, training a model on the prior 30 days, and making a prediction on the next day. My model returned what it thought would be the best performing ten stocks, that I would then compare against the actual best performers. I ran the train/predict over 200 days, using six dense layers of 1,000 nodes on 100 epochs with EarlyStopping. When I calculated the mean of the predicted best 10's actual score, it was 223.04. I learned from this exercise that the yahoo data isn't predictive at the day-to-day scale and my best model was just flipping a coin. The Random Walk Hypothesis is the best explanation.

That's when I turned to another technique to try and make sense of the data. I had an intuitive hunch that when a stock starts to dive, there is an emotionally charged reaction, an irrational over-reaction, which will lead to a predictable rebound. Because my Keras model failed to produce results, I decided to spend some time answering the question: Is there a rebound after a dive? This isn't a complicated modeling technique. Instead of using data to predict an event, we choose an identifiable event and follow its performance forward. I chose the day's worst dive, but you could look for stocks that have consecutive days of growth, the day's best performer, stocks that lose 20% of their value over a 5 trading-day period, etc. Pick an event that you can define and describe and then look for it in the historical data. Look at your subset's performance against the rest of the market (ideally over the exact time window) and see if you can beat my 10X market growth!

I called my subset Unicorns because there were only 45 out of 1.3M records. They don't happen that often and nearly half lost value. But as a group, they showed a dependable rebound 10 and 20 days after the event. The most recent Unicorn was flagged on 4/14/21. Discovery (DISCA) was wrapped up in the Archegos bubble where this guy lost $20B in two days. Because it lost over half its value in the bubble, its relatively mild decline three weeks later put it in the Unicorn group. I'm not advocating a risky investment in a particular stock. Certainly, this one is unique because of its participation in an acute bubble. But the historical peers of this stock make it worth watching and doing some follow-up.

I have to give a shout-out to Bryan B. who continued to press at each dead end. The rebound isn't significant among all stocks, and he prompted the interest in comparing relative volatility. Special thanks to Ludo A. for playing mentor while I learned to code in Python. Happy pattern hunting!

Share link

Anyone who has the link will be able to view this.