Evaluating Scientific Information (if you're not an expert)

While fake news is a real concern, it is not a new problem; nor is it just a problem with news (fake research or bad science can be just as dangerous).

It has always been important to evaluate the source of information - is it accurate? is it biased? is it up-to-date? is it trustworthy?

If an article has been peer-reviewed, some of those questions - at least, about how good its scientific methods may be - are answered for you. When researching healthcare, it's best to look for evidence-based medicine. But even with those, you still need to think critically about each source - the PROVEN test is one good tool for that.

The following questions will help you think critically during the source evaluation process:

What's the Purpose? 

•  Can you find out, or figure out, why the source was created, and why it was published  in this particular type of source (book, article, website, blog, etc.)?
• If it is a news story or press release about scientific information, why is it being published now? (to educate, inform, persuade, sell, entertain?) If it was written by researchers, describing a scientific experiment, why did they think it was important to investigate that topic?  
• Who is the intended audience—the general public, experts or other researchers, policy-makers? 

Is it Relevant?

• Does this source meet your information needs? Is it directly on-topic, or does it focus on a side issue?
• Is this source match the requirements for your assignment, is it appropriate for how you plan to use it?
•  Is it too general or too specific? Is it too basic or too advanced? Or is it Just Right?

Is it Objective?

• If it is a news story or press release about scientific research, does it give the reader a direct link to the research report - and if so, does it report in it accurately?
• Do the authors, publishers, or sponsors have a particular political, ideological, cultural, or religious point of view? And if so, do they acknowledge this point of view, or try to disguise it? Is there a statement about potential conflicts of interest by the researchers?
• If this discusses more than just one research study, does it leave out important facts or perspectives on this topic? Does it offer multiple points of view? Does it critique other perspectives respectfully?

Is it Verified?

• Do the authors support their information with factual evidence? If it gives statistical data, are they statistically relevant?
• Does this item cite or link to other sources - and if it does, can you verify the credibility of those sources? (Perhaps by finding the original source of the information.) Does it only cite or link to other sources by the same author(s)?
• Can you verify the information in other credible sources?
• Does the source contradict itself, include false statements, or misrepresent other sources?
• If it is in a scholarly or professional journal, was it peer-reviewed? 

Are the Authors Expert?

• Does the source provide contact information for the authors, publishers, and/or sponsors? 
• What makes the authors, publishers, or sponsors of the source authorities on the topic? Do they have related education, or personal or professional experience? Are they affiliated with an educational institution or respected organization?
• Is their expertise acknowledged by other authorities on the topic? Do other sources cite this source?

Is it New?

• When was the information in the source first published or posted? 
• Does this topic require the most up-to-date information, or could information found in older sources still be useful and valid? 
• Are newer sources available that would add important information to your understanding of the topic?
• Is there information that the authors should have known about, but not mentioned in this piece?

Evidence-Based Medicine takes peer review several steps further. The Centre for Evidence-Based Medicine defines EBM as: 

 "the conscientious, explicit, and judicious development and use of current best evidence in making decisions about the care of individual patients". 

In other words, it is treatment advice based on scientifically-proven research, with the best results for most people, that is as up to date as possible while still being thoroughly checked. EBM relies heavily on systematic reviews and meta-analyses to determine the best evidence.

Title: "Science News Cycle" - originally published 5/16/2009

Image credit: Jorge Cham (2009) http://phdcomics.com?comics?archive_print.pho?comicid=1174