Your Big Data Assumptions May Be Flat-Out Wrong

Posted on June 21, 2018 I Written By

Anne Zieger is veteran healthcare branding and communications expert with more than 25 years of industry experience. and her commentaries have appeared in dozens of international business publications, including Forbes, Business Week and Information Week. She has also worked extensively healthcare and health IT organizations, including several Fortune 500 companies. She can be reached at @ziegerhealth or www.ziegerhealthcare.com.

It’s an article of faith in healthcare circles that leveraging big data stores can improve patient care. But what if this cherished assumption is flat-out wrong?

A new study published in the Proceedings of the National Academy of Sciences suggests that big data number-crunching might actually undermine providers’ ability to improve patient health.

To conduct the study, researchers from UC Berkeley, Drexel University and the University of Groningen compared data collected on hundreds of people, including both individuals with psychiatric disorders and healthy individuals. They found that group results didn’t capture some wide variations in symptoms from person to person.

Researchers concluded that big data analyses are a poor substitute for working with individuals, noting that these analyses are “worryingly imprecise” and that the variance between individuals is four times larger than those captured by big data. In other words, it concludes that big data analyses minimize differences between patients dramatically.

The authors said that it doesn’t work to generalize conclusions about individuals, whose emotions, behavior and physiology can vary greatly.

“Diseases, mental disorders, emotions, and behaviors are expressed within individual people, over time,” said study lead author Aaron Fisher, an assistant professor of psychology at UC Berkeley in a prepared statement. “A snapshot of many people at one moment in time can’t capture these phenomena.”

At this point, you’re probably thinking that this is terrible news. But Fisher believes that there are practical ways to address the problem. “Modern technologies allow us to collect many observations per person relatively easily, and modern computing makes the analysis of these data [points]  possible in ways that were not possible in the past,” Fisher said.

I don’t know about you, but I doubt that gathering loads of individual patient data will be as easy as Fisher suggests. Our current methods for documenting patient encounters in EHRs already impose significant burdens on physicians. Asking them to do more probably won’t fly, at least for the near term.

Not only that, there’s the question of how to work with this new data. We’d all like to see patients get highly individualized care, but current systems used by providers probably aren’t up to the task just yet.

I guess the bottom line here is that while Fisher et al are on to something, it will probably be a long time before healthcare organizations get there. In the meantime, it’s good to see that researchers are challenging our assumptions and keeping us on our toes.