If you asked me to describe the rising philosophy of the day, I’d say it is data-ism. We now have the ability to gather huge amounts of data. This ability seems to carry with it certain cultural assumptions — that everything that can be measured should be measured; that data is a transparent and reliable lens that allows us to filter out emotionalism and ideology; that data will help us do remarkable things — like foretell the future.
Over the next year, I’m hoping to get a better grip on some of the questions raised by the data revolution: In what situations should we rely on intuitive pattern recognition and in which situations should we ignore intuition and follow the data? What kinds of events are predictable using statistical analysis and what sorts of events are not?
I confess I enter this in a skeptical frame of mind, believing that we tend to get carried away in our desire to reduce everything to the quantifiable. But at the outset let me celebrate two things data does really well.
First, it’s really good at exposing when our intuitive view of reality is wrong. For example, nearly every person who runs for political office has an intuitive sense that they can powerfully influence their odds of winning the election if they can just raise and spend more money. But this is largely wrong.
After the 2006 election, Sean Trende constructed a graph comparing the incumbent (在任者的) campaign spending advantages with their eventual margins of victory. There was barely any relationship between more spending and a bigger victory.
Likewise, many teachers have an intuitive sense that different students have different learning styles: some are verbal and some visual; some are linear, some are holistic (整体的). Teachers imagine they will improve outcomes if they tailor their presentations to each student. But there’s no evidence to support this either.
Second, data can illuminate patterns of behavior we haven’t yet noticed. For example, I’ve always assumed people who frequently use words like “I,” “me,” and “mine” are probably more self-centered than people who don’t. But as lames Pennebaker of the University of Texas notes in his book, The Secret Life of Pronouns, when people are feeling confident, they are focused on the task at hand, not on themselves. High-status, confident people use fewer “I” words, not more.
Our brains often don’t notice subtle verbal patterns, but Pennebaker’s computers can. Younger writers use more negative and past-tense words than older writers who use more positive and future-tense words. In sum, the data revolution is giving us wonderful ways to understand the present and the past. Will it transform our ability to predict and make decisions about the future? We’ll see.
