I once had a client who noticed an oddity in their data. They were witnessing a surge in activity from people in Kansas.
This caused confusion and excitement. Was the topic going viral in Kansas? Was there a trend about to explode across the USA? They created sub-groups in the community for the influx of Kansans, the PR and marketing folks began running more ads in Kansas, and they sent a few people on the ground to investigate why their popularity was exploding in Kansas.
What nobody seemed to notice is no-one was actually joining the Kansas groups. The number of orders from Kansas wasn’t increasing. Our survey results didn’t seem to show a surge in Kansans flooding the community.
So I looked a little deeper at the tool used to collect the data and noticed the surge resembled a cliff-edge rather than a bell-curve. One day few people were visiting from Kansas and the next it was by far the most popular source of visitors. This timed exactly with an update to the data tool itself. The release notes for the update noted an ‘improved geolocating’ system.
So I called the vendor and solved the mystery in ten minutes. When the tool couldn’t determine the address of visitors, it simply assigned them to the middle point of the country. Which happened to be Lebanon, Kansas.
It’s easy to get excited about an anomaly in the data – especially one that shows a sudden change in behavior. Before you do though, I suggest looking to check if the methodology for collecting the data has changed.