We see what we want to see, we hear what we want to hear, and we present to others what we want them to see.
That in its essence is cherry-picking, or more broadly speaking the confirmation bias. In other words, we seek information and evidence that validates our hypotheses and represents our beliefs, tending to subconsciously overlook, or consciously ignore, all other, possibly opposing, data.
Going with the cherry-picking metaphor, we selectively pluck the fruits that lie within our vision and appear the healthiest to us, intentionally or unintentionally ignoring blemished fruits of equally good quality, or located higher up in the tree.
Cherry-picking is thus one particular kind of the wider framing problem in journalism, where media focuses attention on certain events, or even particular aspects of certain events, based on their own biases, and presents them within a predetermined field of meaning.
Cherry-picking in the news
“To err is human”, and cherry-picking is one such natural human source of error. It is a cognitive bias that bears evidence in numerous psychological studies (visit the Wikipedia page on confirmation bias for an easy read describing a few of these). Innate as it is to being human, there can be no field that is resistant to this tendency. Nevertheless, the association of journalism with truth and objectivity places it at a particular risk with regard to cherry-picked reporting.
More specifically, there are three ways in which cherry-picking can bias a news story:
- Selecting and presenting evidence that promotes your claims while underrepresenting contradictory evidence
- Presenting a misrepresented interpretation of ambiguous information in a way that supports your claim
- Rationalizing any contradictory evidence in a way that still seems to favour your claim
Does this happen often in the media? Unfortunately, yes. Here is an excellent example from a CNN news clip from 29 September 2014, where the channel reporters asked Reza Aslan, an Iranian-American writer and religious scholar for his opinion on the provocative remarks of comedian Bill Maher on his show, against Muslims. The entire video clip (inserted below) is 9 mins long, but the section between 3:00 – 7:00 minutes is an absolute must-watch! It is interesting to see how despite Aslan’s repeated attempts to break the frame of Islam and violence, the reporters keep returning to it, and repeatedly rationalizing it against Aslan’s remarks.
Data journalism and cherry-picking
Data journalism, despite its illusions of being driven by hard-core facts, is equally vulnerable to cherry-picking. For one, we might still only search for data that represents our hypothesis. This is more so the case given that almost all our online browsing is carried out through custom-driven search engines like Google. We simply type in what we are looking for, and do not need to be confronted with counter-evidence at all. The other issue is that we could misrepresent the data, favouring and presenting more information that supports our case, and under-representing data that illustrates the other side of the story.
One example of cherry-picking, particularly with regard to data-driven journalism, is the widely argued subject of climate change. There have been articles that claim it’s a fact and that humans are partly responsible, and there have been articles denouncing it. Of late though, there is growing consensus that climate change is a very real problem and a threatening one. Irrespective, the debate on the various effects of climate change continues to exist. In an article posted on 19 March 2014, blogger Roger Pielke Jr., reporting for FiveThirtyEight, wrote about how the increasing costs of disasters in recent times are just the result of increasing worldwide wealth, and have nothing to do with climate change. However, in a counter article published by The Guardian, reporter Dana Nuccitelli elaborates on how Pielke’s article is a misrepresentation on the following accounts:
- Pielke speaks of disasters as one category. However, while the increasing costs of earthquakes (one type of disaster) can be accounted for by increasing global GDP, the increased frequency and costs of storms and floods (other disaster categories) cannot be fully explained by the wealth factor.
- Pielke’s posts mention nothing about the studies that have found the exact opposite effect evidencing human-caused climate change to be a potential attribute of increased damage caused by hurricanes.
- Pielke’s research does not account for engineering and building law improvements that have contributed to a decline in hurricane damage, thereby lowering the total damage cost estimates and underestimating this measure.
The above example is thus an illustration of how cherry-picking can characterize the more fact-based data-driven journalism as well.
So what’s the solution?
“I know that most men—not only those considered clever, but even those who are very clever, and capable of understanding most difficult scientific, mathematical, or philosophic problems—can very seldom discern even the simplest and most obvious truth if it be such as to oblige them to admit the falsity of conclusions they have formed, perhaps with much difficulty—conclusions of which they are proud, which they have taught to others, and on which they have built their lives.”
–“What is Art?”, Tolstoy
In my opinion, the only way we can refrain from cherry-picking is by first openly acknowledging that however unbiased and objective we attempt to be, by virtue of our cognitive biases, we will always be prone to such a fallacy. Once we accept our weakness, we can tackle it better. In an article published in 2013 in the Journal of Mass Media Ethics, academician Sue Ellen Christian proposes at least two interesting journalistic best practices, among others, that can reduce such cognitive biases in particular:
- Think about how we think; i.e. explore our cognitive prejudices and tendencies
- Counterargue and consider the opposite side of the story
Further, in reference to the field of data journalism in particular, online journalist and blogger Paul Bradshaw mentions two starting-points: “either you have a question that needs data, or a dataset that needs questioning.” Even though cherry-picking could characterize both these routes, maybe the latter path can better withstand the cognitive bias and be a more objective way of arriving at your story.