Our stock sentiment analysis system has been working like a charm now for a little over 3 days. There was some confusion on the Yahoo News API which caused a temporary outage. But the data has moved around enough such that we can now start showing some data science on the collected data and see if we cannot make some educated choices in what we expect the stock price to do in the upcoming weeks.
Tableau is a free program that you can use to make stellar visualizations of your data. While the product is significantly more complex and robust, we are just going to use some of the basic features.
Creating a Visualization in Tableau
In the free version of Tableau, we can only hook our data source up to an export of our MariaDB that is keeping a record of all our stock sentiment data. For the last 3 days, during the times I have my computer open, every 10 minutes our system was fetching, scraping, analyzing and recording the latest news for 5 major tickers. Now, we want to be able to see that data in a way that we can see the change of price as it relates to sentiment.
To do this, we export our tone table to a CSV file and then import that CSV into Tableau. It imports without much effort.
Next we create a new “worksheet” which is a single visualization. For our columns we select the Recordtime column. We also right click on it and assign it as a dimension. We drag the price, anger, joy, fear, and disgust columns over to the row section. We make sure they are marked as dimensions as well. We should now see 5 graphs stacked. Finally we add a filter on the ticker symbol so we can see each ticker seperately.
We can copy this sheet and make a new one and switch the ticker. We do this for each of our ticker symbols. Finally we create a dashboard. Here we drag each sheet into the dashboard to see the 5 tickers simultaneously.
Initial Data Analysis
In this chart I think the most blaring thing is the anger vector shows clear correlation to the stock price. Once the price drops somewhat the anger value goes through the roof. Disgust and joy are less of a proper fit for the data but they do show some response to stock price.
I will next try to run some covariance matrix calculations to see what I can see about the similarity of our features. Keeping in mind this is not a lot of data to start from.