14 Seven Pillars: Evaluate

Information is published in a variety of formats, each with its own special considerations when it comes to evaluation. Consider the following formats.

Social Media

Social media is a quickly rising star in the landscape of information gathering. Facebook updates, tweets, wikis, and blogs have made information creators of us all and have a strong influence not just on how we communicate with each other but also on how we learn about current events or discover items of interest. Anyone can create or contribute to social media and nothing that’s said is checked for accuracy before it’s posted for the world to see. So do people really use social media for research? Currently, the main use for social media like tweets and Facebook posts is as primary sources that are treated as the objects under study rather than sources of information on a topic. But now that the Modern Language Association has a recommended way to cite a tweet, social media may, in fact, be gaining credibility as a resource.

News Articles

These days, social media will generally be among the first to cover a big news story, with news media writing an article or report after more information has been gathered. News articles are written by journalists who either report on an event they have witnessed firsthand, or after making contact with those more directly involved. The focus is on information that is of immediate interest to the public and these articles are written in a way that a general audience will be able to understand. These articles go through a fact-checking process, but when a story is big and the goal is to inform readers of urgent or timely information, inaccuracies may occur. In research, news articles are often best treated as primary sources, especially if they were published immediately after a current event.

Magazine Articles

While news articles and social media tend to concentrate on what happened, how it happened, who it happened to, and where it happened, magazine articles are more about understanding why something happened, usually with the benefit of at least a little hindsight. Writers of magazine articles also fall into the journalist category and rely heavily on investigation and interviews for research. Fact-checking in magazine articles tends to be more accurate because magazines publish less frequently than news outlets and have more time to get facts right. Depending on the focus of the magazine, articles may cover current events or just items of general interest to the intended audience. The language may be more emotional or dramatic than the factual tone of news articles, but the articles are written at a similar reading level so as to appeal to the widest audience possible. A magazine article is considered a popular source rather than a scholarly one, which gives it less weight in a research context but doesn’t take away the value completely.

Scholarly Articles

Scholarly articles are written by and for experts in a field and generally describe formal research studies or experiments conducted to provide new insight on a topic rather than reporting current events or items of general interest. You may have heard the term “peer review” in relation to scholarly articles. This means that before an article is published, it undergoes a review process in order to confirm that the information is accurate and the research it discusses is valid. This process adds a level of credibility to the article that you would not find in a magazine or news article. Scholarly articles tend to be long and feature specialized language that is not easily understood by someone who does not already have some level of expertise on the topic. Though they may not be as easy to use, they carry a lot of weight in a research context, especially if you are working in a field related to science or technology. These sources will give you information to build on in your own original research.

Books

Books have been a staple of the research process since Gutenberg invented the printing press because a topic can be covered in more depth in a book than in most other types of sources. Also, the conventional wisdom for books is that anyone can write one, but only the best ones get published. This is becoming less true as books are published in a wider variety of formats and via a wider variety of venues than in previous eras, which is something to be aware of when using a book for research purposes. For now, the editing process for formally published books is still in place and research in the humanities, which includes topics such as literature and history, continues to be published primarily in this format.

Choosing Materials

When choosing a source for your research, what criteria do you usually use? Gauging whether the source relates to your topic at all is probably one. How high up it appears on the results list when you search may be another. Beyond that, you may base your decision at least partly on how easy it is to access.

These are all important criteria, to varying degrees, but there are other criteria you may want to keep in mind when deciding if a source will be useful to your research.

Quality

Scholarly journals and books are traditionally considered to be higher quality information sources because they have gone through a more thorough editing process that ensures the quality of their content. Generally, you also pay more to access these sources or may have to rely on a library or university to pay for access for you. Information on the Internet can also be of a high quality but there is less of a quality assurance process in place for much of that information. In the current climate, the highest quality information even on the Internet often requires a subscription or other form of payment for access.

Clues to a source’s level of quality are closely related to thinking about how the source was produced, including what format it was published in and whether it is likely to have gone through a formal editing process prior to publication.

Accuracy

A source is accurate if the information it contains is correct. Sometimes it’s easy to tell when a piece of information is simply wrong, especially if you have some prior knowledge of the subject. But if you’re less familiar with the subject, inaccuracies can be harder to detect, especially when they come in subtler forms such as exaggerations or inconsistencies.

To determine whether a source is accurate, you need to look more deeply at the content of the source, including where the information in the source comes from and what evidence the author uses to support their views and conclusions. It also helps to compare your source against another source. A reader of Our Virginiamay not have reason to believe the information the author cites from the Sons of Confederate Veterans website is inaccurate, but if they compared the book against another source, the inconsistencies might become more apparent.

Relevance

Relevance has to do with deciding whether the source actually relates to your topic and, if it does, how closely it relates. Some sources may be an exact match; for others, you may need to consider a particular angle or context before you can tell whether the source applies to your topic. When searching for relevant sources, you should keep an open mind—but not too open. Don’t pick something that’s not really related just because it’s on the first page or two of results or because it sounds good.

You can assess the relevance of a source by comparing it against your research topic or research question. Keep in mind that the source may not need to match on all points, but it should match on enough points to be usable for your research beyond simply satisfying a requirement for an assignment.

Bias

An example of bias is when someone expresses a view that is one-sided without much consideration for information that might negate what they believe. Bias is most prevalent in sources that cover controversial issues where the author may attempt to persuade their readers to one side of the issue without giving fair consideration to the other side of things. If the research topic you are using has ever been the cause of heated debate, you will need to be especially watchful for any bias in the sources you find.

Bias can be difficult to detect, particularly when we are looking at persuasive sources that we want to agree with. If you want to believe something is true, chances are you’ll side with your own internal bias without consideration for whether a source exhibits bias. When deciding whether there is bias in a source, look for dramatic language and images, poorly supported evidence against an opposing viewpoint, or a strong leaning in one direction.

Reputation

Is the author of the source you have found a professor at a university or a self-published blogger? If the author is a professor, are they respected in their field or is their work heavily challenged? What about the publication itself? Is it held in high regard or relatively unknown? Digging a little deeper to find out what you can about the reputation of both the author and the publication can go a long way toward deciding whether a source is valuable.

You can investigate the reputation of an author by looking at any biographical information that is available as part of the source. Looking to see what else the author has published and whether this information has positive reviews is also important in establishing whether the author has a good reputation. The reputation of a publication can also be investigated through reviews, word-of-mouth by professionals in the field, or online databases that keep track of statistics related to a journal’s credibility.

Credibility

Credibility has to do with the believability or trustworthiness of a source based on evidence such as information about the author, the reputation of the publication, and how well-formatted the source is. How likely would you be to use a source that was written by someone with no expertise on a topic or a source that appeared in a publication that was known for featuring low quality information? What if the source was riddled with spelling and formatting errors? Looking at sources like these should inspire more caution.

Objectively, credibility can be determined by taking into account all of the other criteria discussed for evaluating a source. Knowing that some types of sources, such as scholarly journals, are generally considered more credible than others, such as self-published websites, may also help. Subjectively, deciding whether a source is credible may come down to a gut feeling. If something about a source doesn’t sit well with you, you may decide to pass it over.

The chapter about “SIFT: Four Moves” provides an excellent method for evaluating sources and claims you find online. An alternative evaluation method is the CRAAP test which was developed by librarians at California State University at Chico. It gives you a good, overall set of elements to look for when evaluating a resource.

Currency

One of the most important and interesting steps to take as you begin researching a subject is selecting the resources that will help you build your thesis and support your assertions. Certain topics require you to pay special attention to how current your resource is—because they are time sensitive, because they have evolved so much over the years, or because new research comes out on the topic so frequently. When evaluating the currency of an article, consider the following

  • When was the item written, and how frequently does the publication it is in come out?
  • Is there evidence of newly added or updated information in the item?
  • If the information is dated, is it still suitable for your topic?
  • How frequently does information change about your topic?

Relevance

Understanding what resources are most applicable to your subject and why they are applicable can help you focus and refine your thesis. Many topics are broad and searching for information on them produces a wide range of resources. Narrowing your topic and focusing on resources specific to your needs can help reduce the piles of information and help you focus in on what is truly important to read and reference. When determining relevance consider the following:

  • Does the item contain information relevant to your argument or thesis?
  • Read the article’s introduction, thesis, and conclusion.
  • Scan main headings and identify article keywords.
  • For book resources, start with the index or table of contents—how wide a scope does the item have? Will you use part or all of this resource?
  • Does the information presented support or refute your ideas?
  • If the information refutes your ideas, how will this change your argument?
  • Does the material provide you with current information?
  • What is the material’s intended audience?

Authority

Understanding more about your information’s source helps you determine when, how, and where to use that information. Is your author an expert on the subject? Do they have some personal stake in the argument they are making? What is the author or information producer’s background? When determining the authority of your source, consider the following

  • What are the author’s credentials?
  • What is the author’s level of education, experience, and/or occupation?
  • What qualifies the author to write about this topic?
  • What affiliations does the author have? Could these affiliations affect their position?
  • What organization or body published the information? Is it authoritative? Does it have an explicit position or bias?

Accuracy

Determining where information comes from, if evidence supports the information, and if the information has been reviewed or refereed can help you decide how and whether to use a source. When determining the accuracy of a source, consider the following:

  • Is the source well-documented? Does it include footnotes, citations or a bibliography?
  • Is information in the source presented as fact, opinion or propaganda? Are biases clear?
  • Can you verify information from referenced information in the source?
  • Is the information written clearly and free of typographical and grammatical mistakes? Does the source look to be edited before publication? A clean, well-presented paper does not always indicate accuracy, but usually at least means more eyes have been on the information.

Purpose

Knowing why information was created is a key to evaluation. Understanding the reason or purpose of the information, if the information has clear intentions, or if the information is fact, opinion or propaganda will help you decide how and why to use information

  • Is the author’s purpose to inform, sell, persuade, or entertain?
  • Does the source have an obvious bias or prejudice?
  • Is the article presented from multiple points of view?
  • Does the author omit important facts or data that might disprove their argument?
  • Is the author’s language informal, joking, emotional, or impassioned?
  • Is the information clearly supported by evidence?

 

License

Icon for the Creative Commons Attribution 4.0 International License

Tackling Wicked Problems by Cathie LeBlanc is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book