1.4 Evaluating and Citing Sources
Learning Objectives
By the end of this section, you will be able to:
- Understand how to identify appropriate credible sources of information when researching a topic.
- Differentiate between primary and secondary sources.
- Differentiate betwwen popular and scholarly sources.
- Evaluate sources of information for their strengths and weaknesses using the CRAAP test.
- Differentiate between pseudoscience or junk “bad” science and real science.
- Describe the dangers of “bad” science.
Using Credible Sources
Since this is a biology class, we will be focusing on questions that can be answered scientifically. Remember that in the scientific process, observations lead to questions. A scientific question is one that can be answered by using the process of science (testing hypotheses, making observations about the natural world, designing experiments).
Sometimes you will directly make observations yourself about the natural world that lead you to ask scientific questions; other times you might hear or read something that leads you to ask a question. Regardless of how you make your initial observation, you will want to do research about your topic before you start setting up an experiment. When you’re learning about a topic, it’s important to use credible sources of information.
Types of Sources
Whether conducting research in the social sciences, humanities (especially history), arts, or natural sciences, the ability to distinguish between primary and secondary source material is essential. Basically, this distinction illustrates the degree to which the author of a piece is removed from the actual event being described. This means whether the author is reporting information first hand (or is first to record these immediately following an event), or conveying the experiences and opinions of others—that is, second hand. In biology, the distinction would be between the person (or people) who conducted the research and someone who didn’t actually do the research, but is merely reporting on it.
Primary sources
These are contemporary accounts of an event, written by someone who experienced or witnessed the event in question. In general, these original documents (i.e., they are not about another document or account) are often diaries, letters, memoirs, journals, speeches, manuscripts, interviews, photographs, audio or video recordings, or original literary or theatrical works.
In science, a “primary source” or the “primary literature” refers to the original publication of a scientist’s new data, results, and conclusions. These articles are written for other experts in a specific scientific field.
You’ve probably done a writing assignment or other project during which you have participated in a peer review process. During this process, your project was critiqued and evaluated by people of similar competence to yourself (your peers). This gave you feedback on which to improve your work. Scientific articles typically go through a peer review process before they are published in an academic journal. In this case, the peers who are reviewing the article are other experts in the specific field about which the paper is written. This allows other scientists to critique experimental design, data, and conclusions before that information is published in an academic journal. Often, the scientists who did the experiment and who are trying to publish it are required to do additional work or edit their paper before it is published. The goal of the scientific peer review process is to ensure that published primary articles contain the best possible science.
Secondary sources
The function of a secondary source is to interpret the primary source. A secondary source can be described as at least one step removed from the event or phenomenon under review. Secondary source materials interpret, assign value to, conjecture upon, and draw conclusions about the events reported in primary sources. These are usually in the form of published works such as magazine articles or books, but may include radio or television documentaries, or conference proceedings. These texts are often referred to as popular sources because they are written for a general audience. Popular sources are intended to inform, entertain, or persuade. They are often written in everyday language and may not include a list of references.
In science, it is often extremely difficult to read and understand primary articles unless you are an expert in that specific scientific field. Secondary sources are typically easier to read and can give you the important information from a primary source, but only if the secondary source has interpreted the information correctly! It is always better to go to the primary source if possible because otherwise you are relying on someone else’s interpretation of the information. However, it is always better to use a source that you can read and understand rather than a source that you can’t. For this reason, it is very important to be able to identify credible secondary sources.
Popular vs. Scholarly Sources
| POPULAR | SCHOLARLY |
| Broad range of topics, presented in shorter articles | Specific, narrowly focused topics in lengthy, in-depth articles |
| Articles offer overview of subject matter; interpretation, rather than original research; sometimes contain feature articles and reports on current social issues and public opinion | Articles often contain previously unpublished research and detail new developments in field |
| Intended to attract a general readership without any particular expertise or advanced education | Intended for specialist readership of researchers, academics, students and professionals |
| Written by staff (not always attributed) or freelance writers using general, popular language | Written by identified specialists and researchers in subject area, usually employing technical, subject-specific language and jargon |
| Edited and approved for publication in-house (not peer-reviewed) | Critically evaluated by peers (fellow scholars) in field for content, scholarly soundness, and academic value |
| Articles rarely contain references or footnotes and follow no specific format | Well-researched, documented articles nearly always follow standard format:
abstract, introduction, literature review, methodology, results, conclusion, bibliography/references |
| Designed to attract eye of potential newsstand customers: usually filled with photographs or illustrations, printed on glossier paper | Sober design: mostly text with some tables or graphs accompanying articles; usually little or no photography; negligible, if any, advertising; rarely printed on high-gloss paper |
| Each issue begins with page number ‘1’ | Page numbers of issues within a volume (year) are usually consecutive (i.e., first page of succeeding issue is number following last page number of previous issue) |
| Presented to entertain, promote point of view, and/or sell products | Intended to present researchers’ opinions and findings based on original research |
| Examples: Newsweek, Rolling Stone, Vogue | Examples: Science, Nature, Journal of Microbial and Biochemical Technology |
Evaluating Credibility
When you write a scientific paper (or any paper, really), you want to back up your statements with credible sources. You will need to identify credible sources to help you research scientific topics inorder to develop interesting scientific questions. You also need sources to help you form a well-educated hypothesis that is not just based on your guess about what will happen. A credible source is one that is trustworthy from which the information can be believed. Credible sources are written by people who are experts in the field (or at least are very knowledgeable) about the subject that they are commenting on.
We will be using a variation of the CRAAP test to help you determine whether or not sources that you find are credible or not. The CRAAP Test was created by Sarah Blakeslee, of the University of California at Chico’s Meriam Library.
Use the table below to help you evaluate the credibility of your sources.
Credibility Table
| Factors to consider | Least reliable
(0 points) |
Possibly reliable
(1 point) |
Most reliable
(2 points) |
| Currency | No date of publication or revision given | Outdated for this particular topic | Recently published or revised |
| Reliable source | Unreliable website, no additional info available | Possibly reliable | Official government or organization, institutional sites, academic journals |
| Author | No author is given / the author is not qualified to write about this topic | Author is educated on topic or is staff of an organization assumed to be knowledgeable on this specific topic | Specifically identified expert in this field with degrees / credentials in this subject |
| Accuracy | No review process and information is not supported by evidence from cited sources | The information may have been reviewed or edited by someone knowledgeable in the field. It mentions but does not directly cite other sources | The information has been peer reviewed and is supported by evidence from cited credible sources |
| Purpose | Obviously biased or trying to sell you something | Sponsored source; may present unbalanced information | Balanced, neutral, presents all sides of the issue fully |
In general, do not use a source if it doesn’t pass the CRAAP test! For our purposes, do not use any sources that score less than 6 points using the credibility table.
Several examples are given below for sources that you might come across if you were researching the topic of vaccine safety.
Example 1:
CDC (Centers for Disease Control and Prevention). Aug 28, 2015. Vaccine Safety [Internet]. [cited May 12, 2016]. Available from: http://www.cdc.gov/vaccinesafety/index.html
| Score | Discussion – why did you give that score? | |
| Currency | 2 | Aug 28 2015 is recent and shows that this information is updated frequently. |
| Reliable source | 2 | I looked at the “about this organization” and learned that the CDC is a major government organization that works to protect Americans from health, safety, and security threats. They are a division of the US department of health and human services. |
| Author | 1 | A specific author was not identified, but the page states that the content is from the CDC, which suggests that it was written by a knowledgeable staff member. |
| Accuracy | 1.5 | No information is given about the review process, but it was probably edited by staff at the CDC. There is a list of citations and links to primary scientific articles supporting the information. |
| Purpose | 2 | The point of view does not appear to be biased because it seems to be presenting factual information. Admittedly, it only presents the pro-vaccine side of the argument. There are no ads on the page or other information trying to change the reader’s viewpoint. |
| Credibility Score | 8.5/10 | This seems like an excellent source to use for research. It’s readable and I could look at the primary articles if I wanted to check them out. |
Example 2:
Stop Mandatory Vaccination. N.d.. The Dangers of Vaccines and Vaccinations [Internet]. [cited May 12, 2016]. Available from: http://www.stopmandatoryvaccination.com/vaccine-dangers/
| Score | Discussion – why did you give that score? | |
| Currency | 1 | The copyright is given as 2015, but there is no date for this specific article. It does reference something that took place in 2015, so it is likely written after that. |
| Reliable source | 0 | The “About” page states that the organization was started by Larry Cook using a GoFundMe platform |
| Author | 0 | Larry Cook has been devoted to the natural lifestyle for 25 years, but doesn’t appear to have any degrees or specific expertise on this topic. Other contributing authors include Landee Martin, who has a Bachelor’s of Science in Psychology (which isn’t related to vaccine safety), and Brittney Kara, who is a mother who has studied holistic living for the last 17 years. None of the individuals specifically identified on the website appear to be experts in the field. |
| Accuracy | 0.5 | It seems unlikely that there is any review process. There are links to several sources, but none of them appear to be primary scientific articles. Several are links to interviews. |
| Purpose | 0 | This source is extremely biased. Even the name of the website is biased. There is a link to donate to the webpage. There are at least 10 ads for anti-vaccine books and websites. |
| Credibility Score | 1.5/10 | I would not want to use this source to research this topic. It’s extremely biased and doesn’t seem to offer much evidence for its assertions. |
Pseudoscience and Other Misuses of Science
Pseudoscience is a claim, belief, or practice that is presented as scientific but does not adhere to the standards and methods of science. True science is based on repeated evidence-gathering and testing of falsifiable hypotheses. Pseudoscience does not adhere to these criteria. In addition to phrenology, the study of the fomration of the skull as indicative of mental faculties and traits of character, some other examples of pseudoscience include astrology, extrasensory perception (ESP), reflexology, reincarnation, and Scientology.
Characteristics of Pseudoscience
Whether a field is actually science or just pseudoscience is not always clear. However, pseudoscience generally exhibits certain common characteristics. Indicators of pseudoscience include:
Real Science |
Pseudoscience |
You know it’s real if it…
FACT: The idea of a scientist as “know-it-all” is false!
|
Beware if it…
|
Persistence of Pseudoscience
Despite failing to meet scientific standards, many pseudosciences survive. Some pseudosciences remain very popular with large numbers of believers. A good example is astrology and zodiac signs.
Astrology is the study of the movements and relative positions of celestial objects as a means for divining information about human affairs and terrestrial events. Many ancient cultures attached importance to astronomical events, and some developed elaborate systems for predicting terrestrial events from celestial observations. Throughout most of its history in the West, astrology was considered a scholarly tradition and was common in academic circles. With the advent of modern Western science, astrology was called into question. It was challenged on both theoretical and experimental grounds, and it was eventually shown to have no scientific validity or explanatory power.
Today, astrology is considered a pseudoscience, yet it continues to have many devotees. Most people know their astrological sign, and many people are familiar with the personality traits supposedly associated with their sign. Astrological readings and horoscopes are readily available online and in print media, and a lot of people read them, even if only occasionally. About a third of all adult Americans actually believe that astrology is scientific. Studies suggest that the persistent popularity of pseudosciences such as astrology reflects a high level of scientific illiteracy. It seems that many Americans do not have an accurate understanding of scientific principles and methodology. They are not convinced by scientific arguments against their beliefs.
Dangers of Pseudoscience
Belief in astrology is unlikely to cause a person harm, but belief in some other pseudosciences might — especially in health care-related areas. Treatments that seem scientific but are not may be ineffective, expensive, and even dangerous to patients. Seeking out pseudoscientific treatments may also delay or preclude patients from seeking scientifically-based medical treatments that have been tested and found safe and effective.
Junk Science or “Bad Science”
As discussed above, good science practices include following the scientific method, have a testable hypothesis, good experimental design, including large sample sizes and replication, results that are statistically analyzed, and interpretation and analysis subject to peer review. If a researcher misses these steps and/or tries to manipulate data, they are following junk science.
An example of this is found with the autism-vaccine debate. You may have heard that certain vaccines put the health of young children at risk. This persistent idea is not supported by scientific evidence or accepted by the vast majority of experts in the field. It stems largely from an elaborate medical research fraud that was reported in a 1998 article published in the respected British medical journal, The Lancet. The main author of the article was a British physician named Andrew Wakefield. In the article, Dr. Wakefield and his colleagues described case histories of 12 children, most of whom were reported to have developed autism soon after the administration of the MMR (measles, mumps, rubella) vaccine.
Several subsequent peer-reviewed studies failed to show any association between the MMR vaccine and autism. It also later emerged that Wakefield had received research funding from a group of people who were suing vaccine manufacturers. In 2004, ten of Wakefield’s 12 coauthors formally retracted the conclusions in their paper. In 2010, editors of The Lancet retracted the entire paper. That same year, Wakefield was charged with deliberate falsification of research and barred from practicing medicine in the United Kingdom. Unfortunately, by then, the damage had already been done. Parents afraid that their children would develop autism had refrained from having them vaccinated. British MMR vaccination rates fell from nearly 100 per cent to 80 per cent in the years following the study. The consensus of medical experts today is that Wakefield’s fraud put hundreds of thousands of children at risk because of the lower vaccination rates and also diverted research efforts and funding away from finding the true cause of autism.
Correlation-Causation Fallacy
Many statistical tests used in scientific research calculate correlations between variables. Correlation refers to how closely related two data sets are, which may be a useful starting point for further investigation. It can be either a positive correlation, that is when one variable increases so does the other, or it can be a negative correlation. With a negative correlation, as one variable increases, the other variable decreases. Correlation, however, is also one of the most misused types of evidence, primarily because of the logical fallacy that correlation implies causation. In reality, just because two variables are correlated does not necessarily mean that either variable causes the other.
The example illutrated in the graph below can be used to demonstrate the correlation-causation fallacy. Assume a study found that both per capita consumption of mozzarella cheese and the number of Civil Engineering doctorates awarded are correlated; that is, rates of both events increase together. If correlation really did imply causation, then you might conclude that the eating mozzarella cheesse increases your chances of earning adoctorate in Civil Engineering.
An actual example of the correlation-causation fallacy occurred during the latter half of the 20th century. Numerous studies showed that women taking hormone replacement therapy (HRT) to treat menopausal symptoms also had a lower-than-average incidence of coronary heart disease (CHD). This correlation was misinterpreted as evidence that HRT protects women against CHD. Subsequent studies that controlled other factors related to CHD disproved this presumed causal connection. The studies found that women taking HRT were more likely to come from higher socio-economic groups, with better-than-average diets and exercise regimens. Rather than HRT causing lower CHD incidence, these studies concluded that HRT and lower CHD were both effects of higher socio-economic status and related lifestyle factors.
Citing Sources
One of the goals for any class is to help students become better scholars. And, one of the important skills of scholarship is proper citation of resources used. Citations demonstrate your “credentials” as a scholar, and provide a resource to your readers of good reference material.
Why do you have to cite your sources?
No research paper is complete without a list of the sources that you used in your writing. Scholars are very careful to keep accurate records of the resources they’ve used, and of the ideas and concepts they’ve quoted or used from others. This record keeping is generally presented in the form of citations.
A citation is a description of a book, article, URL, etc. that provides enough information so that others can locate the source you used themselves. It allows you to credit the authors of the sources you use and clarify which ideas belong to you and which belong to other sources. And providing a citation or reference will allow others to find and use these sources as well. Most research papers have a list of citations or cited references and there are special formatting guidelines for different types of research. There are many “proper” formating styles for academic work because each discipline has its own rules. In general, you should check with your instructors about which format to use in a specific class. Whichever “official” format you use, make sure that you use it consistently throughout your paper.
Plagiarism
Plagiarism is presenting the words or ideas of someone else as your own without proper acknowledgment of the source. When you work on a research paper you will probably find supporting material for your paper from works by others. It’s okay to quote people and use their ideas, but you do need to correctly credit them. Even when you summarize or paraphrase information found in books, articles, or Web pages, you must acknowledge the original author. To avoid plagiarism, include a reference to any material you use that provides a fact not commonly known, or whenever you use information from another author. In short, if you didn’t collect the data or reach the conclusion on your own, cite it!
These are all examples of plagiarism:
- Buying or using a term paper written by someone else.
- Cutting and pasting passages from the Web, a book, or an article and insert them into your paper without citing them.
- Using generative Artificial Intelligence (gAI) without acknowledging and citing it.
- Using the words or ideas of another person without citing them.
- Paraphrasing that person’s words without citing them.
Tips for Avoiding Plagiarism:
- First, use your own ideas—it should be your paper and your ideas should be the focus.
- Use the ideas of others sparingly—only to support or reinforce your own argument.
- When taking notes, include complete citation information for each item you use.
- Use quotation marks when directly stating another person’s words. Quotes are not frequently used in scientific writing unless you are directly quoting someone’s spoken words.
Glossary
academic journal: a publication that contains articles written by experts in a specific field of study for the purpose of sharing research with the academic community (also known as a scholarly journal). Articles in academic journals are usually, but not always, peer reviewed.
citation: a description of a book, article, URL, etc. that provides enough information so that others can locate the source you used themselves
correlation: how closely related two data sets
negative correlation: as one variable increases, the other variable decreases
positive correlation: as one variable increases, the other variable increases
credible source: a source that has been evaluated for authority (written by a trustworthy author or organization), accuracy, bias, currency, and relevancy in the context of the research being conducted
junk or “bad” science: ideas presented as scientific fact that are untested or unproven using the scientific method or that have used manipulated data or skipped important steps of the scientific method
logical fallacy: a deceptive or false argument that is flawed in reasoning and can be disproven
correlation-causation fallacy: a logical fallacy that occurs when someone assumes a cause-and-effect relationship between two things based on a correlation, rather than performing an experiment
plagiarism: presenting the words or ideas of someone else as your own without proper acknowledgment of the source
primary source: refers to the original publication of a scientist’s new data, results, and conclusions. These articles are written for other experts in a specific scientific field
peer review: a process through which experts in the specific scientific field about which a paper is written review an article and critique experimental design, data, and conclusions before that information is published in an academic journal.
pseudoscience: a claim, belief, or practice that is presented as scientific but does not adhere to the standards and methods of science
secondary source: at least one step removed from the event or phenomenon under review. Secondary source materials interpret, assign value to, conjecture upon, and draw conclusions about the events reported in primary sources
falsifiable: able to be disproven by experimental results
hypothesis: a suggested explanation for an event, which can be tested
hypothesis-based science: a form of science that begins with a specific explanation that is then tested
scientific method: a method of research with defined steps that include experiments and careful observation
variable: a part of an experiment that can vary or change
References
Bartee, L., Shriner, W., and Creech C. (n.d.) Principles of Biology. Pressbooks. Retrieved from https://openoregon.pressbooks.pub/mhccmajorsbio/chapter/using-credible-sources/
Clark, M.A., Douglas, M., and Choi, J. 2018. Biology 2e. OpenStax. Retrieved from https://openstax.org/books/biology-2e/pages/1-introduction
Introductory Biology: Ecology, Evolution, and Biodiversity Copyright © 2023 by amuir13 and Erica Kosal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, except where otherwise noted.
Molnar, C., & Gair, J. 2015. Concepts of Biology – 1st Canadian Edition. BCcampus. Retrieved from https://opentextbc.ca/biology/
TED-Ed. 2017, July 6. How to spot a misleading graph – Lea Gaslowitz. YouTube. https://www.youtube.com/watch?v=E91bGT9BjYk&feature=youtu.be
Wakefield, A.J., Murch, S.H., Anthony, A., Linnell, J., Casson, D.M., Malik, M., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet, 351: 637–41.
Wikipedia contributors. (2020, June 18). Andrew Wakefield. Wikipedia. https://en.wikipedia.org/w/index.php?title=Andrew_Wakefield&oldid=963243135