Memorial Day marks the ringing in of the summer season. If you are like me, there’s no better way to celebrate the beginning of summer than a perfectly toasted bratwurst or hotdog. And while I am more than happy to enjoy the end-product, I have very little interest in rolling up my sleeves and “seeing how the sausage is made”. This is how we treat data analytics. We are happy to digest the insightful dashboards, intriguing graphics, and actionable takeaways. However, very few of us are interested in rolling up our sleeves and doing the necessary dirty work to prepare the data to allow for useful insights.
Many companies starting their data analytics journey make the mistake of skipping the data cleaning process all together. None of us want to see how the sausage is made, we just want the bratwurst to magically appear. But as we have seen over, and over, insightful analytics cannot be achieved with poor data quality. A 2017 Harvard Business Review article titled Only 3% of Companies’ Data Meets Basic Quality Standards outlined the data quality crisis best, stating that, “47% of newly-created data records have at least one critical (e.g., work-impacting) error”. If you are struggling with data quality issues, know that you are not alone. In this article, we will examine why data quality is so critical, why companies struggle, and finally what can be done about it.