Data Science Bootcamp - Week 4

Part of the General Assembly Data Science Bootcamp Series

Why bother with data visualization?

While summary statistics (eg. sum, mean, stdev) are important concepts in the study of data science, they are not enough when you want to see a more complete understanding of your data.

Take for example the summary statistics data in the table above. All the summary statistics indicate that all the data sets are identical. Or are they? You’ll see in the charts below, that could be further from the truth. They are all totally different, if not for the visualization with the charts, we will be none the wiser!

This is known as Anscombe’s Quartet, which was first constructed by the statistician Anscombe in 1973. He wanted to demonstrate the need to graphing the data before analyzing it, and the effect of outliers on these statistical properties.

The above example highlights the shortcomings of summary statistics alone. It also shows the effects of outliers on these summary data. We are visual beings, and given the table above, it would not have given the impact and understanding that the charts would have easily conveyed.

Week 4 - Python visualizations, aka throw away visualizations

In conjunction with exploratory data analysis and slicing and dicing your data, and in this case we use the most popular language and library for data scientists - Python and Pandas, we will also run the data through a series of visualizations. We like to see patterns at a high level, and evaluate early on if we can continue, because part of data science is ensuring we have enough data and if the quality of the data is good enough.

The charts above were generated by a popular python based library called Seaborn which is based on yet another python visualization library called matplotlib. Both these libraries enable the data scientist to easily create many different types of visualizations straight from their Jupyter notebooks.

When working with your visualization, specially in your Jupyter notebooks, the preference is to create many, and we were encouraged to plot as many charts as we can, the idea being these will be treated as throw away charts. These visualizations were created for the sole purpose of finding pattern at the early stages of the data science end-to-end process.

Capstone Project proposal due this coming Monday

I still have the whole weekend to complete my Capstone project proposal, as well as the Unit 2 assignment. This will be a busy weekend. I will still be updating the proposal draft below, but I have decided that my project will be using the Formula 1 Racing dataset.

Formula 1 Dataset

Ever since the first season of Drive to Survive, I’ve been captivated by the drama and excitement that is Formula 1. I’ve been consuming this public API in some of my past blog posts (DynamoDB and Single-Table Design, Simple GraphQL consumer with Apollo Client) and I thought it was fitting to continue this trend and explore the insights and predictions that can be gleaned from it:

• predict the winner of a race
• predict the pole sitter in the qualifying race
• predict who will be the fastest lap
• predict which team will be fastest pit stop
• explore the effect of the weather on the outcome of the rece
• who wins the constructor at the end of the year
• who is the last place in the next race

Resources

• Anscombe’s Quartet
• Anscombe’s Quartet Jupyter Notebook
• Pandas - Open source data analysis and manipulation tool
• Seaborn - Statistical Data Visualizations