36 Evaluating for Relevancy
Relevant sources are those that pertain to your research question. You’ll be able to figure that out fairly quickly by reading or skimming particular parts of sources and maybe jotting down little tables that help you keep track. We’ll show you how below, including what questions to ask as you review specific sources.
The timeliness or currency of a source is a key criteria in determining a source’s relevance to your research project.
For instance, if your research question is about the life sciences, consider only the most recent sources relevant because the life sciences change so quickly. There is a good chance that anything but the most recent sources may be out of date. So aim for sources no more than 5 years old; in this discipline, most sources older than 5 years are likely out of date. (Another discipline that calls for even newer sources is computer security, where the “expiration” of sources is even sooner.)
But suppose your research question is about the Edo Period in Japan (1603-1868) or about Robert Falcon Scott, who explored the Antarctic from 1901-1913. In these cases, an item from 1918 might be just as useful as an item from 2018 (although new information may have been found in the 100 year gap).
These sample research topics also give you two more clues about how to treat the timeliness or currency of sources as you consider relevance:
- Because of how long ago they lived or occurred, it would be unusual for many sources on Robert Scott (historical figure) or the Edo Period (historical period) to have been published very recently. So, unlike sources for the biosciences, the currency for these topics is much less significant.
- Note: Of course, research occurs in trends, which means that some topics, even in academic environments, are more likely to be published given an specific topic. For example research on color therapy might have been a topic that was researched a lot in the 1980s, but you’ll likely have a hard time finding much that has been published recently. If you do research on topic like this, be prepared to find few sources, or only sources that are older or too old for your purposes. In situations like these, speak to your professor or a librarian for advice about another topic that might be related to your interest that has more current documented sources.
- Primary sources might be considered especially relevant to all three research questions. Bioscience journal articles that provide research findings for the first time count as primary sources. And primary sources (such as Scott’s diaries and expedition photographs, as well as paintings, literature, clothing, and household items from the Edo Period) go a long way to explain faraway people and times. (See Primary, Secondary, & Tertiary Sources.)
Currency
When determining the timeliness of a source, see the section “Where to Look”, which includes where to look on websites, articles, and books for information about a source’s currency.
To be considered relevant to your research question, a source wouldn’t necessarily have to cover all of your main concepts, but finding sources that do is ideal. Otherwise, you just have to make do with what you’ve got. Don’t forget that each source would have to pass the currency test, too, if currency is important to your research question. So it’s wise to record your decisions about the sources’ currency on your tables, too. Take a look at this sample table:
| Currency Okay | Current Research | Historical Background | |
| Source A title | X | X | |
| Source B title | X | ||
| Source C title | X | X | X |
Having multiple sources means that you will be able to use each one for the various information needs you have. One source may provide sufficient historical background information while another might have the latest research on the issue. It is not often that all sources will serve all needs, and believe it or not, this has a lot to do with the information cycle, format, and type of information available to you.
Recall that a newspaper or magazine article might have the most current information on a topic/issue, but the book source might have more accurate or reliable information. Your scholarly journal article source might be just right in terms of currency, accuracy, and reliability, but it might not meet the mark if the article focuses too much on a narrow aspect of your topic/issue and not enough on the aspect you need to focus on. Or, you might find that as you progress through the research process, you might be better off focusing on the aspect that has been written about. Again, your ability to evaluate a source in not only determining the credibility, currency, reliability, authority, and objectivity of the source, but also whether or not it meets your information need, or the appropriateness or relevance of your source to your research investigation.
Time-Saving Tips
Instead of thinking you have to read the entire source in order to figure out whether it is relevant, read or skim only parts of each source. Looking at the appropriate sections will give you enough information to make an educated guess about relevancy.
What should you be looking for as you read and skim? Go back to your research question and parse out its main concepts. (Identify main concepts in your research question in order to search precisely.)
For instance, suppose your research question is: How does having diverse members in a group increase the group’s critical thinking?
What are this question’s main concepts? Our answer is: group diversity and critical thinking.
When judging which sources are relevant to the main concepts, assess whether each source you’ve found pertains to at least one of these concepts. Jot down in a little table like the one below to keep track of which sources address each main concept.
| Currency Okay | Group Diversity | Critical Thinking | |
| Source A title | X | X | X |
| Source B title | X | ||
| Source C title | X | X |
To be considered relevant to your research question, a source wouldn’t necessarily have to cover all of your main concepts, but finding sources that do is the ideal. Otherwise, you just have to make do with what you’ve got. Don’t forget that each source would have to pass the currency test, too, if currency is important to your research question. So it’s wise to record your decisions about the sources’ currency on your tables, too.
Most sources won’t cover all the criteria you are looking for; ideally, a source will cover most or at least some of the criteria.
EXAMPLE: Source’s Main Concepts and Currency
Research Question: How does having diverse members in a group increase the group’s capacity for critical thinking?
The table in this hypothetical example indicates that both Sources A and C are relevant because each pertains to at least one main concept from the research question. Currency doesn’t seem to matter much to this research question, so all three sources were marked current. But since currency is all that Source B has to offer, it is not relevant for this project.
If you do make little tables for relevance, it’s probably a good idea to hang on to them. You might find them helpful later in your research process.
Where to Look in Websites, Articles, and Books
The information below shows where to look and what questions to ask yourself to assess three kinds of relevancy to your research question. Whatever you do, don’t stop evaluating a source after looking only at a website’s name or the title of another source.
Save time by looking in particular sections of the source for information that will help you figure out whether the source is relevant to your research project. Much of our advice below comes from “Speedy Reading” in The Craft of Research, second edition, by Wayne Booth, Gregory Colomb, and Joseph Williams, 2003, pp. 108-109.
On an open website, check the name of the website and its articles for clues that it contains material relevant to your research question. For the timeliness and currency of a source, skim any dates, datelines, “What’s New” pages, and press releases to see whether any website content works with the time considerations you need. Look at the site’s publication lists. How current are these publications? Was the last publication posted from 2010? If so, what does this tell you about the organization that produces this information? Is the organization still viable? Why did it stop posting current publications on a particular topic? Page creation or revision dates that you find can also help but are not the best indicators for a source.
Skim any site map and index on the website for keywords related to your research question. Try the keywords of your research question in the search box. Do you see enough content about your keywords to make you think parts of the website could be helpful?
Remember to be flexible with your terms; sometimes a synonym for a main concept might yield more results.
For an article source, think about the title. Does it have anything to do with your research question? Again if timeliness/currency is an important factor, consider when the source was published. Is the publication date within your parameters? Also skim the abstract to see whether the article works with the time considerations you need. For instance, if there is a time period in your research question, does the article address the same time period? Or does it address an earlier period and the timeline actually reflects when the article was published, which might be well after the time period of interest to you?
Look at the abstract and section headings in the article to locate the problem or question that the article addresses, its discussions and conclusion, and the outline of the article’s argument for its main claim. What insights do these sections provide on your research question? Is there additional information about the context of the research being conducted?
Do the article’s introduction and conclusion sections help you answer your research question and/or offer a description of the situation surrounding your question so you can explain in your final product why the question is important? Check whether the bibliography contains keywords related to your research question. Do the sources cited in the bibliography pertain to your research question? How current are the sources cited?
For a book source, check whether the title indicates the book could be about your research question. Consider whether time should have an impact on what sources can be considered relevant. If so, is the publication date or copyright date (usually listed in the library catalog or on the back of the book’s title page) too early or late for any time constraints in your research question? Maybe it is just right. Also skim some of the preface and introduction to see whether the book works with the time considerations you need.
For help responding to your research question, skim the book’s table of contents, which you may consider as an outline or skeleton of the book’s content, and any summary chapters to locate the problem or question that the book addresses, its solution, and the broad outline of the book’s argument for its main claim. Does the book give you information about what others have written about your research question? Does it offer a description of the situation surrounding (in other words, the context) your research question? Look for your keywords in the bibliography. Do the sources cited pertain to your research question? Skim the index for topics with the most page references. Do the topics with the most page references pertain to your research question? Again, be flexible and consider synonyms for your main concepts when looking up words in the index or bibliography.
Connecting the Dots beyond the Title
Instructions: Now you can practice evaluating for relevance beyond the title.
- Your research question:
How does “prospect theory” in behavioral economics help explain medical doctors’ decisions to favor surgery or radiation to cure cancer in patients? - Judging by the title, the following source seems relevant to your research question:
“Cancer Treatment Prescription–Advancing Prospect Theory beyond Economics,” in Journal of The American Medical Association Oncology, June, 2016. - Read the abstract of the article below. Then decide whether this source is relevant to your research questions above. That is, might the article help you meet any of your project’s information needs about the research question? If there is at least one need it meets, then the article is likely relevant to your research investigation.
- Answer the question below the abstract to indicate the source is relevant. Then compare your answer with our feedback.
To review, your information needs are:
- To learn more background information.
- To respond to your research question.
- To convince your audience that your answer is correct or, at least, the most reasonable answer.
- To describe the situation surrounding (context) your research question for your audience and explain why it’s important.
- To report what others have said about the research question, including any opposing responses to your research question or claim.
- Abstract
“Cancer Treatment Prescription–Advancing Prospect Theory beyond Economics,” in Journal of The American Medical Association Oncology, June, 2016 (Note to students: This article and abstract are fictitious.)
Cancer Treatment is complex. We expect oncologists to make treatment decisions according to definitive standards of care. Finding out that prospect theory demonstrates that they react very much like most other people when deciding to recommend surgery or chemotherapy for their patients indicates that more self-reflection on oncologists’ part could help patients make better decisions. (Prospect theory describes how people choose between alternatives that have risk when the probability of different outcomes is unknown.)
Objective To show whether prospect theory applies to how oncologists framed their recommendations for surgery or chemotherapy for patients in good condition and bad condition.
Design, Settings, and Participants Records of 100 U.S. oncologists were examined for the years 2014 and 2015, which documented patient conditions and the way oncologists framed their recommendations regarding surgery or chemotherapy. Thus, a quasi-experimental ex post facto design was used for the study.
Main Outcomes and Measures This study explored the relationship between the way in which the oncologists “framed” the choice of surgery or chemotherapy as they made recommendations to patients, to patients’ conditions, and the choice actually made. Those results were compared to what prospect theory would predict for this situation.
Results Physicians seemed to present their recommendation of surgery or chemotherapy in a loss frame (e.g., “This is likely to happen to you if you don’t have this procedure”) when patients’ conditions were poor and in a gain frame (e.g., “By having this procedure, you can probably dramatically cut your chances of reoccurrence”) when their conditions were less poor. These results are what prospect theory would have predicted.
Conclusions and Relevance This study opens up the possibility that, as described by prospect theory, a person’s choice of framing behavior is not limited to how we naturally act for ourselves but includes how we act for other people, as the oncologists were acting on behalf of their patients. More research is necessary to confirm this line of evidence and determine whether oncologists’ decision making and framing is the most effective and entirely according to the best standards of care.
