This fact-checking report is the culmination of all the knowledge and analytical experience we have gained over the last few weeks, critiquing different data-driven news articles in our blog posts. To gain first-hand experience with the fact-checking and reporting practices commonly employed by journalists on a day-to-day basis, we chose one news medium and analyzed five different stories they published over the past few months. These were not just any stories, but stories which we had a reason to doubt, or question at the least. Misleading titles, generalizing claims, interpretations drawn from incomplete facts, or just the fact that it all sounded too good to be true; these were some of the first hunches that urged us to launch a more detailed investigation of the story being told.
Taking on the Univers
We picked a news medium very close to home; the student magazine of Tilburg University: Univers. The Univers is a cross-media platform that publishes a print magazine once every 3 weeks, as well as runs a website and blog, offering students news of recent events both on and off-campus, in addition to opinions, and articles on sports and culture. Since its target group consists of both Dutch and international students, the Univers publishes articles in Dutch and English. Being international students at Tilburg ourselves, we chose to look at the English language articles published by the Univers in recent months, and picked five articles in particular to take a closer look at:
- Starbucks sends employees to college
- The Chinese are (not) coming!
- TiU scores better in student questionnaire
- Erasmus students less likely to be unemployed
- English language may harm Dutch science
To remain consistent in our fact-checking process, we prepared a comprehensive set of steps beforehand, that we followed for all the stories we were investigating. In addition to locating all the original sources and cross-checking all the numbers, quotes and names we came across, we also took a deeper look at the frame and the context that formed the articles. To clarify all the queries we had remaining after tallying these details, we approached the International Editor of the Univers, Jozien Wijkhuis, directly in a face-to-face interview, and confronted her with all our questions and doubts. This was a valuable part of the process, since it not only made some details about the stories clearer, it also gave us a better insight into how journalists working for the Univers go about writing their stories.
Our comprehensive fact-checking process
- Read the article and highlight every fact that could be questioned or needs to be cross-checked
- Write down own questions about facts that appear incomplete or misleading and need to be investigated further
- Find the original sources, whether or not they are mentioned explicitly in the article
- Read the content of the original sources and check:
- whether the facts match the article
- if the article contains the same message as original source
- Check for a Dutch version of the same article on Univers if possible
- Check articles from different news source on the same topic
- Write out a preliminary draft of the findings so far
- Identify remaining issues to be clarified with journalist (e.g., facts that do not match or cannot be located, misleading title, any statement of opinions)
- Interview the journalist and pose all the questions face-to-face
- Include the journalist’s responses in the analysis
Frames and misleading titles
An advantage of going through five different articles from the same medium, and also frequently the same author, was that we could see a repeated pattern in the issues that were raised. For one, the titles were often a bit misleading. In fact, that was the first thing that led us to bring some of these articles under scrutiny, for example, The Chinese are (not) coming!, or English language may harm Dutch science. On researching these articles further, we found that it was not the Netherlands that the Chinese were not coming to, but some US universities and mainly in the field of science and engineering, and that English language being detrimental to the humanities, and not science, was actually just the personal, unproven opinion of four university lecturers from Amsterdam. While catchy headlines draw in the readers, reliability of wording is an important element of a trustworthy news story.
Another consistent issue we noticed across the articles was the rather weak presentation of the original sources. In some cases, the writer(s) left out the use of quotation marks or phrases such as “according to” to delineate people’s personal opinions. These statements could then be incorrectly interpreted by the reader as the opinion of the writer, or a general fact of the matter. For example, in the Chinese are (not) coming article, the journalist provided explanations as to why the Chinese students are not coming to the US in the form of fact, that was not supported. However, the explanation provided were the opinions of experts from another article. In other articles, the writers forgot to mention the location where an original study or material was first published. Further, facts do not hold much meaning without a context, and are consequently always interpreted within certain frames. Frames are extremely important as it shows the article in a certain light, a context. However, the Chinese student article also it lacked a clear purpose and frame as the relevance of a study for the US to students in the Netherlands was not touched on at all. As a result, the article the appeared to be a random with no distinct purpose. Questioning the author led to the discovery that the purpose of the article was bringing awareness to the global development of higher education, and the frame was students and higher education. A rather broad and assuming stance, since it did leave us, the targeted student readers in doubt.
What are facts without depth?
The thing that disappointed us the most however, was the overall lack of depth across all the articles we looked at. Indeed, as Wijkhuis mentioned herself, the articles, particularly ones like these are limited to a third of the magazine page, or are meant for online posts, are often written within a short, pressing time frame, and are limited to being (superficial) summaries of the subject being covered. It seemed to us to be very often the case, that the journalist created a stripped down version of another news story, taking very little time to research the details provided there, and fact-checking for themselves the verity as well as the implications of what they were reporting. The case of the Starbucks story was a particularly clear example of this, where though the journalist mentioned having checked the Arizona State University and Starbucks sites for information about the employee study program, she missed the important distinction this program made between freshmen and seniors. The finer details of the program made it clear that the claim that Starbucks provided free education to all its employees was indeed too optimistic, something that Wijkhuis herself admitted when confronted with this information in the interview.
Another explanation for this the lack of depth was found while discussing the article about Tilburg University’s questionnaire. Here, when confronted with the more detailed and objective Dutch counterpart article on the same subject, the author admitted to leaving out some details because they did not seem applicable for international students, particularly because these international students are only here for a few months. This touches upon two issues. First, the international students often do not have full overview of the stories being reported. Second, those studying full-time and following an entire Bachelor’s or Master’s program, like us, are not taken into account as an audience!
Nevertheless, we do want to clarify that despite these deeper doubts about titles, frames and details, the facts and numbers were indeed presented accurately, with at least a primary effort at locating the original source and checking the numbers, even though this original source was left out of mention in the article sometimes.
And to end this piece, here’s what we think…
Although we recognize that time and page space are recurring restrictions when compiling a news story, whether they hold priority over objective and reliable reporting remains a topic of debate. Summaries of articles might be a suitable manner of online reporting when offered as short updates. However, as university students, trained to be critical and exacting, we expect the reporters of the magazine of an international research-driven university to come up with more objective and better-researched pieces, offering multiple viewpoints and some real arguments and opinions.