features of a scholarly research article

What is a Scholarly Article: What is a scholarly article

Determineif a source is scholarly, determine if a source is scholarly, what is a scholarly source.

Scholarly sources (also referred to as academic, peer-reviewed, or refereed sources) are written by experts in a particular field and serve to keep others interested in that field up to date on the most recent research, findings, and news. These resources will provide the most substantial information for your research and papers.

What is peer-review?

When a source has been peer-reviewed, it has undergone the review and scrutiny of a review board of colleagues in the author’s field. They evaluate this source as part of the body of research for a particular discipline and make recommendations regarding its publication in a journal, revisions prior to publication, or, in some cases, reject its publication.

Why use scholarly sources?

Scholarly sources’ authority and credibility improve the quality of your own paper or research project.

How can I tell if a source is scholarly?

The following characteristics can help you differentiate scholarly sources from those that are not. Be sure to look at the criteria in each category when making your determination, rather than basing your decision on only one piece of information.

• Are author names provided?
• Are the authors’ credentials provided?
• Are the credentials relevant to the information provided?
• Who is the publisher of the information?
• Is the publisher an academic institution, scholarly, or professional organization?
• Is their purpose for publishing this information evident?
• Who is the intended audience of this source?
• Is the language geared toward those with knowledge of a specific discipline rather than the general public?
• Why is the information being provided?
• Are sources cited?
• Are there charts, graphs, tables, and bibliographies included?
• Are research claims documented?
• Are conclusions based on evidence provided?
• How long is the source?

Currency/Timeliness

• Is the date of publication evident?

Additional Tips for Specific Scholarly Source Types

Each resource type below will also have unique criteria that can be applied to it to determine if it is scholarly.

• Books published by a University Press are likely to be scholarly.
• Professional organizations and the U.S. Government Printing Office can also be indicators that a book is scholarly.
• Book reviews can provide clues as to if a source is scholarly and highlight the intended audience. See our  Find Reviews  guide to locate reviews on titles of interest.
• Are the author’s professional affiliations provided?
• Who is the publisher?
• How frequently is the periodical published?
• How many and what kinds of advertisements are present? For example, is the advertising clearly geared towards readers in a specific discipline or occupation?
• For more information about different periodical types, see our  Selecting Sources  guide.
• What is the domain of the page (for example: .gov, .edu, etc.)?
• Who is publishing or sponsoring the page?
• Is contact information for the author/publisher provided?
• How recently was the page updated?
• Is the information biased? Scholarly materials published online should not have any evidence of bias.

Is My Source Scholarly? (Accessible View)

Step 1: Source

The article is most likely scholarly if:

• You found the article in a library database or Google Scholar
• The journal the article appears in is peer-reviewed

Move to Step 2: Authors

Step 2: Authors

The source is most likely scholarly if:

• The authors’ credentials are provided
• The authors are affiliated with a university or other research institute

Move to Step 3: Content

Step 3: Content

• The source is longer than 10 pages
• Has a works cited or bibliography
• It does not attempt to persuade or bias the reader
• It attempts to persuade or bias the reader, but treats the topic objectively, the information is well-supported, and it includes a works cited or bibliography

If the article meets the criteria in Steps 1-3 it is most likely scholarly.

Common Characteristics of a Scholarly Article

Common characteristics of scholarly (research) articles.

Articles in scholarly journals may also be called research journals, peer reviewed journals, or refereed journals. These types of articles share many common features, including:

• articles always provide the name of the author or multiple authors
• author(s) always have academic credentials (e.g. biologist, chemist, anthropologist, lawyer)
• articles often have a sober, serious look
• articles may contain many graphs and charts; few glossy pages or color pictures
• author(s) write in the language of the discipline (e.g. biology, chemistry, anthropology, law, etc.)
• authors write for other scholars, and emerging scholars
• authors always cite their sources in footnotes, bibliographies, notes, etc.
• often (but not always) associated with universities or professional organizations

Types of Scholarly Articles

Peer Review in 3 Minutes

North Carolina State University (NCSU) Libraries (3:15)

• What do peer reviewers do?  How are they similar to or different from editors?
• Who are the primary customers of scholarly journals?
• Do databases only include peer-reviewed articles?  How do you know?

Is my source scholarly

Is My Source Scholarly?: INFOGRAPHIC

This infographic is part of the Illinois Library's Determine if a source is scholarly.

"Is my source scholarly" by Illinois Library  https://www.library.illinois.edu/ugl/howdoi/scholarly/

Anatomy of a Scholarly Article: Interactive Tutorial

Typical Sections of a Peer-Reviewed Research Article

Typical sections of peer-reviewed research articles.

Research articles in many disciplines are organized into standard sections. Although these sections may vary by discipline, common sections include:

• Introduction
• Materials and Methods

It's not hard to spot these sections; just look for bold headings in the article, as shown in these illustrations:

• Last Updated: Oct 22, 2020 11:31 AM
• URL: https://libguides.mccd.edu/WhatisaScholarlyArticle

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

Definition and Introduction

Journal article analysis assignments require you to summarize and critically assess the quality of an empirical research study published in a scholarly [a.k.a., academic, peer-reviewed] journal. The article may be assigned by the professor, chosen from course readings listed in the syllabus, or you must locate an article on your own, usually with the requirement that you search using a reputable library database, such as, JSTOR or ProQuest . The article chosen is expected to relate to the overall discipline of the course, specific course content, or key concepts discussed in class. In some cases, the purpose of the assignment is to analyze an article that is part of the literature review for a future research project.

Analysis of an article can be assigned to students individually or as part of a small group project. The final product is usually in the form of a short paper [typically 1- 6 double-spaced pages] that addresses key questions the professor uses to guide your analysis or that assesses specific parts of a scholarly research study [e.g., the research problem, methodology, discussion, conclusions or findings]. The analysis paper may be shared on a digital course management platform and/or presented to the class for the purpose of promoting a wider discussion about the topic of the study. Although assigned in any level of undergraduate and graduate coursework in the social and behavioral sciences, professors frequently include this assignment in upper division courses to help students learn how to effectively identify, read, and analyze empirical research within their major.

Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students make the most of Scholarly Articles." Library Management 33 (2012): 525-535.

Benefits of Journal Article Analysis Assignments

Analyzing and synthesizing a scholarly journal article is intended to help students obtain the reading and critical thinking skills needed to develop and write their own research papers. This assignment also supports workplace skills where you could be asked to summarize a report or other type of document and report it, for example, during a staff meeting or for a presentation.

There are two broadly defined ways that analyzing a scholarly journal article supports student learning:

Improve Reading Skills

Conducting research requires an ability to review, evaluate, and synthesize prior research studies. Reading prior research requires an understanding of the academic writing style , the type of epistemological beliefs or practices underpinning the research design, and the specific vocabulary and technical terminology [i.e., jargon] used within a discipline. Reading scholarly articles is important because academic writing is unfamiliar to most students; they have had limited exposure to using peer-reviewed journal articles prior to entering college or students have yet to gain exposure to the specific academic writing style of their disciplinary major. Learning how to read scholarly articles also requires careful and deliberate concentration on how authors use specific language and phrasing to convey their research, the problem it addresses, its relationship to prior research, its significance, its limitations, and how authors connect methods of data gathering to the results so as to develop recommended solutions derived from the overall research process.

Improve Comprehension Skills

In addition to knowing how to read scholarly journals articles, students must learn how to effectively interpret what the scholar(s) are trying to convey. Academic writing can be dense, multi-layered, and non-linear in how information is presented. In addition, scholarly articles contain footnotes or endnotes, references to sources, multiple appendices, and, in some cases, non-textual elements [e.g., graphs, charts] that can break-up the reader’s experience with the narrative flow of the study. Analyzing articles helps students practice comprehending these elements of writing, critiquing the arguments being made, reflecting upon the significance of the research, and how it relates to building new knowledge and understanding or applying new approaches to practice. Comprehending scholarly writing also involves thinking critically about where you fit within the overall dialogue among scholars concerning the research problem, finding possible gaps in the research that require further analysis, or identifying where the author(s) has failed to examine fully any specific elements of the study.

In addition, journal article analysis assignments are used by professors to strengthen discipline-specific information literacy skills, either alone or in relation to other tasks, such as, giving a class presentation or participating in a group project. These benefits can include the ability to:

• Effectively paraphrase text, which leads to a more thorough understanding of the overall study;
• Identify and describe strengths and weaknesses of the study and their implications;
• Relate the article to other course readings and in relation to particular research concepts or ideas discussed during class;
• Think critically about the research and summarize complex ideas contained within;
• Plan, organize, and write an effective inquiry-based paper that investigates a research study, evaluates evidence, expounds on the author’s main ideas, and presents an argument concerning the significance and impact of the research in a clear and concise manner;
• Model the type of source summary and critique you should do for any college-level research paper; and,
• Increase interest and engagement with the research problem of the study as well as with the discipline.

Kershaw, Trina C., Jennifer Fugate, and Aminda J. O'Hare. "Teaching Undergraduates to Understand Published Research through Structured Practice in Identifying Key Research Concepts." Scholarship of Teaching and Learning in Psychology . Advance online publication, 2020; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students make the most of Scholarly Articles." Library Management 33 (2012): 525-535; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946.

Structure and Organization

A journal article analysis paper should be written in paragraph format and include an instruction to the study, your analysis of the research, and a conclusion that provides an overall assessment of the author's work, along with an explanation of what you believe is the study's overall impact and significance. Unless the purpose of the assignment is to examine foundational studies published many years ago, you should select articles that have been published relatively recently [e.g., within the past few years].

Since the research has been completed, reference to the study in your paper should be written in the past tense, with your analysis stated in the present tense [e.g., “The author portrayed access to health care services in rural areas as primarily a problem of having reliable transportation. However, I believe the author is overgeneralizing this issue because...”].

Introduction Section

The first section of a journal analysis paper should describe the topic of the article and highlight the author’s main points. This includes describing the research problem and theoretical framework, the rationale for the research, the methods of data gathering and analysis, the key findings, and the author’s final conclusions and recommendations. The narrative should focus on the act of describing rather than analyzing. Think of the introduction as a more comprehensive and detailed descriptive abstract of the study.

Possible questions to help guide your writing of the introduction section may include:

• Who are the authors and what credentials do they hold that contributes to the validity of the study?
• What was the research problem being investigated?
• What type of research design was used to investigate the research problem?
• What theoretical idea(s) and/or research questions were used to address the problem?
• What was the source of the data or information used as evidence for analysis?
• What methods were applied to investigate this evidence?
• What were the author's overall conclusions and key findings?

Critical Analysis Section

The second section of a journal analysis paper should describe the strengths and weaknesses of the study and analyze its significance and impact. This section is where you shift the narrative from describing to analyzing. Think critically about the research in relation to other course readings, what has been discussed in class, or based on your own life experiences. If you are struggling to identify any weaknesses, explain why you believe this to be true. However, no study is perfect, regardless of how laudable its design may be. Given this, think about the repercussions of the choices made by the author(s) and how you might have conducted the study differently. Examples can include contemplating the choice of what sources were included or excluded in support of examining the research problem, the choice of the method used to analyze the data, or the choice to highlight specific recommended courses of action and/or implications for practice over others. Another strategy is to place yourself within the research study itself by thinking reflectively about what may be missing if you had been a participant in the study or if the recommended courses of action specifically targeted you or your community.

Possible questions to help guide your writing of the analysis section may include:

Introduction

• Did the author clearly state the problem being investigated?
• What was your reaction to and perspective on the research problem?
• Was the study’s objective clearly stated? Did the author clearly explain why the study was necessary?
• How well did the introduction frame the scope of the study?
• Did the introduction conclude with a clear purpose statement?

Literature Review

• Did the literature review lay a foundation for understanding the significance of the research problem?
• Did the literature review provide enough background information to understand the problem in relation to relevant contexts [e.g., historical, economic, social, cultural, etc.].
• Did literature review effectively place the study within the domain of prior research? Is anything missing?
• Was the literature review organized by conceptual categories or did the author simply list and describe sources?
• Did the author accurately explain how the data or information were collected?
• Was the data used sufficient in supporting the study of the research problem?
• Was there another methodological approach that could have been more illuminating?
• Give your overall evaluation of the methods used in this article. How much trust would you put in generating relevant findings?

Results and Discussion

• Were the results clearly presented?
• Did you feel that the results support the theoretical and interpretive claims of the author? Why?
• What did the author(s) do especially well in describing or analyzing their results?
• Was the author's evaluation of the findings clearly stated?
• How well did the discussion of the results relate to what is already known about the research problem?
• Was the discussion of the results free of repetition and redundancies?
• What interpretations did the authors make that you think are in incomplete, unwarranted, or overstated?
• Did the conclusion effectively capture the main points of study?
• Did the conclusion address the research questions posed? Do they seem reasonable?
• Were the author’s conclusions consistent with the evidence and arguments presented?
• Has the author explained how the research added new knowledge or understanding?

Overall Writing Style

• If the article included tables, figures, or other non-textual elements, did they contribute to understanding the study?
• Were ideas developed and related in a logical sequence?
• Were transitions between sections of the article smooth and easy to follow?

Overall Evaluation Section

The final section of a journal analysis paper should bring your thoughts together into a coherent assessment of the value of the research study . This section is where the narrative flow transitions from analyzing specific elements of the article to critically evaluating the overall study. Explain what you view as the significance of the research in relation to the overall course content and any relevant discussions that occurred during class. Think about how the article contributes to understanding the overall research problem, how it fits within existing literature on the topic, how it relates to the course, and what it means to you as a student researcher. In some cases, your professor will also ask you to describe your experiences writing the journal article analysis paper as part of a reflective learning exercise.

Possible questions to help guide your writing of the conclusion and evaluation section may include:

• Was the structure of the article clear and well organized?
• Was the topic of current or enduring interest to you?
• What were the main weaknesses of the article? [this does not refer to limitations stated by the author, but what you believe are potential flaws]
• Was any of the information in the article unclear or ambiguous?
• What did you learn from the research? If nothing stood out to you, explain why.
• Assess the originality of the research. Did you believe it contributed new understanding of the research problem?
• Were you persuaded by the author’s arguments?
• If the author made any final recommendations, will they be impactful if applied to practice?
• In what ways could future research build off of this study?
• What implications does the study have for daily life?
• Was the use of non-textual elements, footnotes or endnotes, and/or appendices helpful in understanding the research?
• What lingering questions do you have after analyzing the article?

NOTE: Avoid using quotes. One of the main purposes of writing an article analysis paper is to learn how to effectively paraphrase and use your own words to summarize a scholarly research study and to explain what the research means to you. Using and citing a direct quote from the article should only be done to help emphasize a key point or to underscore an important concept or idea.

Business: The Article Analysis . Fred Meijer Center for Writing, Grand Valley State University; Bachiochi, Peter et al. "Using Empirical Article Analysis to Assess Research Methods Courses." Teaching of Psychology 38 (2011): 5-9; Brosowsky, Nicholaus P. et al. “Teaching Undergraduate Students to Read Empirical Articles: An Evaluation and Revision of the QALMRI Method.” PsyArXi Preprints , 2020; Holster, Kristin. “Article Evaluation Assignment”. TRAILS: Teaching Resources and Innovations Library for Sociology . Washington DC: American Sociological Association, 2016; Kershaw, Trina C., Jennifer Fugate, and Aminda J. O'Hare. "Teaching Undergraduates to Understand Published Research through Structured Practice in Identifying Key Research Concepts." Scholarship of Teaching and Learning in Psychology . Advance online publication, 2020; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Reviewer's Guide . SAGE Reviewer Gateway, SAGE Journals; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946; Gyuris, Emma, and Laura Castell. "To Tell Them or Show Them? How to Improve Science Students’ Skills of Critical Reading." International Journal of Innovation in Science and Mathematics Education 21 (2013): 70-80; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students Make the Most of Scholarly Articles." Library Management 33 (2012): 525-535.

Writing Tip

Not All Scholarly Journal Articles Can Be Critically Analyzed

There are a variety of articles published in scholarly journals that do not fit within the guidelines of an article analysis assignment. This is because the work cannot be empirically examined or it does not generate new knowledge in a way which can be critically analyzed.

If you are required to locate a research study on your own, avoid selecting these types of journal articles:

• Theoretical essays which discuss concepts, assumptions, and propositions, but report no empirical research;
• Statistical or methodological papers that may analyze data, but the bulk of the work is devoted to refining a new measurement, statistical technique, or modeling procedure;
• Articles that review, analyze, critique, and synthesize prior research, but do not report any original research;
• Brief essays devoted to research methods and findings;
• Articles written by scholars in popular magazines or industry trade journals;
• Academic commentary that discusses research trends or emerging concepts and ideas, but does not contain citations to sources; and
• Pre-print articles that have been posted online, but may undergo further editing and revision by the journal's editorial staff before final publication. An indication that an article is a pre-print is that it has no volume, issue, or page numbers assigned to it.

Journal Analysis Assignment - Myers . Writing@CSU, Colorado State University; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36.

• << Previous: Annotated Bibliography
• Next: Giving an Oral Presentation >>
• Last Updated: Jun 3, 2024 9:44 AM
• URL: https://libguides.usc.edu/writingguide/assignments

Evaluating Information

• Understanding Primary and Secondary Sources
• Exploring and Evaluating Popular, Trade, and Scholarly Sources

Reading a Scholarly Article

Common components of original research articles, while you read, reading strategies, reading for citations, further reading, learning objectives.

This page was created to help you:

Identify the different parts of a scholarly article

Efficiently analyze and evaluate scholarly articles for usefulness

This page will focus on reading scholarly articles — published reports on original research in the social sciences, humanities, and STEM fields. Reading and understanding this type of article can be challenging. This guide will help you develop these skills, which can be learned and improved upon with practice.

We will go over:

There are many different types of articles that may be found in scholarly journals and other academic publications. For more, see:

• Types of Information Sources

Reading a scholarly article isn’t like reading a novel, website, or newspaper article. It’s likely you won’t read and absorb it from beginning to end, all at once.

Instead, think of scholarly reading as inquiry, i.e., asking a series of questions as you do your research or read for class. Your reading should be guided by your class topic or your own research question or thesis.

For example, as you read, you might ask yourself:

• What questions does it help to answer, or what topics does it address?
• Are these relevant or useful to me?
• Does the article offer a helpful framework for understanding my topic or question (theoretical framework)?
• Do the authors use interesting or innovative methods to conduct their research that might be relevant to me?
• Does the article contain references I might consult for further information?

In Practice

Scanning and skimming are essential when reading scholarly articles, especially at the beginning stages of your research or when you have a lot of material in front of you.

Many scholarly articles are organized to help you scan and skim efficiently. The next time you need to read an article, practice scanning the following sections (where available) and skim their contents:

• The abstract: This summary provides a birds’ eye view of the article contents.
• The introduction:  What is the topic(s) of the research article? What is its main idea or question?
• The list of keywords or descriptors
• Methods: How did the author(s) go about answering their question/collecting their data?
• Section headings:  Stop and skim those sections you may find relevant.
• Figures:  Offer lots of information in quick visual format.
• The conclusion:  What are the findings and/or conclusions of this article?

Mark Up Your Text

Read with purpose.

• Scanning and skimming with a pen in hand can help to focus your reading.
• Use color for quick reference. Try highlighters or some sticky notes. Use different colors to represent different topics.
• Write in the margins, putting down thoughts and questions about the content as you read.
• Use digital markup features available in eBook platforms or third-party solutions, like Adobe Reader or Hypothes.is.

Categorize Information

Create your own informal system of organization. It doesn’t have to be complicated — start basic, and be sure it works for you.

• Jot down a few of your own keywords for each article. These keywords may correspond with important topics being addressed in class or in your research paper.  
• Write keywords on print copies or use the built-in note taking features in reference management tools like Zotero and EndNote.  
• Your keywords and system of organization may grow more complex the deeper you get into your reading.

Highlight words, terms, phrases, acronyms, etc. that are unfamiliar to you. You can highlight on the text or make a list in a notetaking program.

• Decide if the term is essential to your understanding of the article or if you can look it up later and keep scanning.

You may scan an article and discover that it isn’t what you thought it was about. Before you close the tab or delete that PDF, consider scanning the article one more time, specifically to look for citations that might be more on-target for your topic.  

You don’t need to look at every citation in the bibliography — you can look to the literature review to identify the core references that relate to your topic. Literature reviews are typically organized by subtopic within a research question or thesis. Find the paragraph or two that are closely aligned with your topic, make note of the author names, then locate those citations in the bibliography or footnote.

See the Find Articles page for what to do next:

• Find Articles

See the Citation Searching page for more on following a citation trail:

• Citation Searching
• Taking notes effectively. [blog post] Raul Pacheco-Vega, PhD
• How to read an academic paper. [video] UBCiSchool. 2013
• How to (seriously) read a scientific paper. (2016, March 21). Science | AAAS.
• How to read a paper. S. Keshav. 2007. SIGCOMM Comput. Commun. Rev. 37, 3 (July 2007), 83–84.

This guide was designed to help you:

• << Previous: Exploring and Evaluating Popular, Trade, and Scholarly Sources
• Last Updated: Feb 16, 2024 3:55 PM
• URL: https://libguides.brown.edu/evaluate

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

• Research Guides

Reading for Research: Social Sciences

Structure of a research article.

• Structural Read

Guide Acknowledgements

How to Read a Scholarly Article from the Howard Tilton Memorial Library at Tulane University

Strategic Reading for Research   from the Howard Tilton Memorial Library at Tulane University

Bridging the Gap between Faculty Expectation and the Student Experience: Teaching Students toAnnotate and Synthesize Sources

Librarian for Sociology, Environmental Sociology, MHS and Public Policy Studies

Academic writing has features that vary only slightly across the different disciplines. Knowing these elements and the purpose of each serves help you to read and understand academic texts efficiently and effectively, and then apply what you read to your paper or project.

Social Science (and Science) original research articles generally follow IMRD: Introduction- Methods-Results-Discussion

Introduction

• Introduces topic of article
• Presents the "Research Gap"/Statement of Problem article will address
• How research presented in the article will solve the problem presented in research gap.
• Literature Review. presenting and evaluating previous scholarship on a topic.  Sometimes, this is separate section of the article. 

​Method & Results

• How research was done, including analysis and measurements.  
• Sometimes labeled as "Research Design"
• What answers were found
• Interpretation of Results (What Does It Mean? Why is it important?)
• Implications for the Field, how the study contributes to the existing field of knowledge
• Suggestions for further research
• Sometimes called Conclusion

You might also see IBC: Introduction - Body - Conclusion

• Identify the subject
• State the thesis 
• Describe why thesis is important to the field (this may be in the form of a literature review or general prose)

Body  

• Presents Evidence/Counter Evidence
• Integrate other writings (i.e. evidence) to support argument 
• Discuss why others may disagree (counter-evidence) and why argument is still valid
• Summary of argument
• Evaluation of argument by pointing out its implications and/or limitations 
• Anticipate and address possible counter-claims
• Suggest future directions of research
• Next: Structural Read >>
• Last Updated: Jan 19, 2024 10:44 AM
• URL: https://researchguides.library.vanderbilt.edu/readingforresearch

• Hirsh Health Sciences
• Webster Veterinary

Guide to Scholarly Articles

• What is a Scholarly Article?
• Scholarly vs. Popular vs. Trade Articles

Types of Scholarly Articles

Qualitative, quantitative, and mixed-methods articles, why does this matter.

• Anatomy of Scholarly Articles
• Tips for Reading Scholarly Articles

Scholarly articles come in many different formats each with their own function in the scholarly conversation. The following are a few of the major types of scholarly articles you are likely to encounter as you become a part of the conversation. Identifying the different types of scholarly articles and knowing their function will help you become a better researcher.

Original/Empirical Studies

• Note: Empirical studies can be subdivided into qualitative studies, quantitative studies, or mixed methods studies. See below for more information  
• Usefulness for research:  Empirical studies are useful because they provide current original research on a topic which may contain a hypothesis or interpretation to advance or to disprove. 

Literature Reviews

• Distinguishing characteristic:  Literature reviews survey and analyze a clearly delaminated body of scholarly literature.  
• Usefulness for research: Literature reviews are useful as a way to quickly get up to date on a particular topic of research.

Theoretical Articles

• Distinguishing characteristic:  Theoretical articles draw on existing scholarship to improve upon or offer a new theoretical perspective on a given topic.
• Usefulness for research:  Theoretical articles are useful because they provide a theoretical framework you can apply to your own research.

Methodological Articles

• Distinguishing characteristic:  Methodological articles draw on existing scholarship to improve or offer new methodologies for exploring a given topic.
• Usefulness for research:  Methodological articles are useful because they provide a methodologies you can apply to your own research.

Case Studies

• Distinguishing characteristic:  Case studies focus on individual examples or instances of a phenomenon to illustrate a research problem or a a solution to a research problem.
• Usefulness for research:  Case studies are useful because they provide information about a research problem or data for analysis.

Book Reviews

• Distinguishing characteristic:  Book reviews provide summaries and evaluations of individual books.
• Usefulness for research:  Book reviews are useful because they provide summaries and evaluations of individual books relevant to your research.

Adapted from the Publication manual of the American Psychological Association : the official guide to APA style. (Sixth edition.). (2013). American Psychological Association.

Qualitative articles  ask "why" questions where as  quantitative  articles  ask "how many/how much?" questions. These approaches are are not mutually exclusive. In fact, many articles combine the two in a  mixed-methods  approach. 

We can think of these different kinds of scholarly articles as different tools designed for different tasks. What research task do you need to accomplish? Do you need to get up to date on a give topic? Find a literature review. Do you need to find a hypothesis to test or to extend? Find an empirical study. Do you need to explore methodologies? Find a methodological article.

• << Previous: Scholarly vs. Popular vs. Trade Articles
• Next: Anatomy of Scholarly Articles >>
• Last Updated: Aug 23, 2023 8:53 AM
• URL: https://researchguides.library.tufts.edu/scholarly-articles

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Supplements
• French Abstracts
• Portuguese Abstracts
• Spanish Abstracts
• Author Guidelines
• Submission Site
• Open Access
• About International Journal for Quality in Health Care
• About the International Society for Quality in Health Care
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Contact ISQua
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Primacy of the research question, structure of the paper, writing a research article: advice to beginners.

• Article contents
• Figures & tables
• Supplementary Data

Thomas V. Perneger, Patricia M. Hudelson, Writing a research article: advice to beginners, International Journal for Quality in Health Care , Volume 16, Issue 3, June 2004, Pages 191–192, https://doi.org/10.1093/intqhc/mzh053

• Permissions Icon Permissions

Writing research papers does not come naturally to most of us. The typical research paper is a highly codified rhetorical form [ 1 , 2 ]. Knowledge of the rules—some explicit, others implied—goes a long way toward writing a paper that will get accepted in a peer-reviewed journal.

A good research paper addresses a specific research question. The research question—or study objective or main research hypothesis—is the central organizing principle of the paper. Whatever relates to the research question belongs in the paper; the rest doesn’t. This is perhaps obvious when the paper reports on a well planned research project. However, in applied domains such as quality improvement, some papers are written based on projects that were undertaken for operational reasons, and not with the primary aim of producing new knowledge. In such cases, authors should define the main research question a posteriori and design the paper around it.

Generally, only one main research question should be addressed in a paper (secondary but related questions are allowed). If a project allows you to explore several distinct research questions, write several papers. For instance, if you measured the impact of obtaining written consent on patient satisfaction at a specialized clinic using a newly developed questionnaire, you may want to write one paper on the questionnaire development and validation, and another on the impact of the intervention. The idea is not to split results into ‘least publishable units’, a practice that is rightly decried, but rather into ‘optimally publishable units’.

What is a good research question? The key attributes are: (i) specificity; (ii) originality or novelty; and (iii) general relevance to a broad scientific community. The research question should be precise and not merely identify a general area of inquiry. It can often (but not always) be expressed in terms of a possible association between X and Y in a population Z, for example ‘we examined whether providing patients about to be discharged from the hospital with written information about their medications would improve their compliance with the treatment 1 month later’. A study does not necessarily have to break completely new ground, but it should extend previous knowledge in a useful way, or alternatively refute existing knowledge. Finally, the question should be of interest to others who work in the same scientific area. The latter requirement is more challenging for those who work in applied science than for basic scientists. While it may safely be assumed that the human genome is the same worldwide, whether the results of a local quality improvement project have wider relevance requires careful consideration and argument.

Once the research question is clearly defined, writing the paper becomes considerably easier. The paper will ask the question, then answer it. The key to successful scientific writing is getting the structure of the paper right. The basic structure of a typical research paper is the sequence of Introduction, Methods, Results, and Discussion (sometimes abbreviated as IMRAD). Each section addresses a different objective. The authors state: (i) the problem they intend to address—in other terms, the research question—in the Introduction; (ii) what they did to answer the question in the Methods section; (iii) what they observed in the Results section; and (iv) what they think the results mean in the Discussion.

In turn, each basic section addresses several topics, and may be divided into subsections (Table 1 ). In the Introduction, the authors should explain the rationale and background to the study. What is the research question, and why is it important to ask it? While it is neither necessary nor desirable to provide a full-blown review of the literature as a prelude to the study, it is helpful to situate the study within some larger field of enquiry. The research question should always be spelled out, and not merely left for the reader to guess.

Typical structure of a research paper

The Methods section should provide the readers with sufficient detail about the study methods to be able to reproduce the study if so desired. Thus, this section should be specific, concrete, technical, and fairly detailed. The study setting, the sampling strategy used, instruments, data collection methods, and analysis strategies should be described. In the case of qualitative research studies, it is also useful to tell the reader which research tradition the study utilizes and to link the choice of methodological strategies with the research goals [ 3 ].

The Results section is typically fairly straightforward and factual. All results that relate to the research question should be given in detail, including simple counts and percentages. Resist the temptation to demonstrate analytic ability and the richness of the dataset by providing numerous tables of non-essential results.

The Discussion section allows the most freedom. This is why the Discussion is the most difficult to write, and is often the weakest part of a paper. Structured Discussion sections have been proposed by some journal editors [ 4 ]. While strict adherence to such rules may not be necessary, following a plan such as that proposed in Table 1 may help the novice writer stay on track.

References should be used wisely. Key assertions should be referenced, as well as the methods and instruments used. However, unless the paper is a comprehensive review of a topic, there is no need to be exhaustive. Also, references to unpublished work, to documents in the grey literature (technical reports), or to any source that the reader will have difficulty finding or understanding should be avoided.

Having the structure of the paper in place is a good start. However, there are many details that have to be attended to while writing. An obvious recommendation is to read, and follow, the instructions to authors published by the journal (typically found on the journal’s website). Another concerns non-native writers of English: do have a native speaker edit the manuscript. A paper usually goes through several drafts before it is submitted. When revising a paper, it is useful to keep an eye out for the most common mistakes (Table 2 ). If you avoid all those, your paper should be in good shape.

Common mistakes seen in manuscripts submitted to this journal

Huth EJ . How to Write and Publish Papers in the Medical Sciences , 2nd edition. Baltimore, MD: Williams & Wilkins, 1990 .

Browner WS . Publishing and Presenting Clinical Research . Baltimore, MD: Lippincott, Williams & Wilkins, 1999 .

Devers KJ , Frankel RM. Getting qualitative research published. Educ Health 2001 ; 14 : 109 –117.

Docherty M , Smith R. The case for structuring the discussion of scientific papers. Br Med J 1999 ; 318 : 1224 –1225.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1464-3677
• Print ISSN 1353-4505
• Copyright © 2024 International Society for Quality in Health Care and Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Finding Scholarly Articles: Home

What's a Scholarly Article?

Your professor has specified that you are to use scholarly (or primary research or peer-reviewed or refereed or academic) articles only in your paper. What does that mean?

Scholarly or primary research articles are peer-reviewed , which means that they have gone through the process of being read by reviewers or referees  before being accepted for publication. When a scholar submits an article to a scholarly journal, the manuscript is sent to experts in that field to read and decide if the research is valid and the article should be published. Typically the reviewers indicate to the journal editors whether they think the article should be accepted, sent back for revisions, or rejected.

To decide whether an article is a primary research article, look for the following:

• The author’s (or authors') credentials and academic affiliation(s) should be given;
• There should be an abstract summarizing the research;
• The methods and materials used should be given, often in a separate section;
• There are citations within the text or footnotes referencing sources used;
• Results of the research are given;
• There should be discussion   and  conclusion ;
• With a bibliography or list of references at the end.

Caution: even though a journal may be peer-reviewed, not all the items in it will be. For instance, there might be editorials, book reviews, news reports, etc. Check for the parts of the article to be sure.   

You can limit your search results to primary research, peer-reviewed or refereed articles in many databases. To search for scholarly articles in  HOLLIS , type your keywords in the box at the top, and select  Catalog&Articles  from the choices that appear next.   On the search results screen, look for the  Show Only section on the right and click on  Peer-reviewed articles . (Make sure to  login in with your HarvardKey to get full-text of the articles that Harvard has purchased.)

Many of the databases that Harvard offers have similar features to limit to peer-reviewed or scholarly articles.  For example in Academic Search Premier , click on the box for Scholarly (Peer Reviewed) Journals  on the search screen.

Review articles are another great way to find scholarly primary research articles.   Review articles are not considered "primary research", but they pull together primary research articles on a topic, summarize and analyze them.  In Google Scholar , click on Review Articles  at the left of the search results screen. Ask your professor whether review articles can be cited for an assignment.

A note about Google searching.  A regular Google search turns up a broad variety of results, which can include scholarly articles but Google results also contain commercial and popular sources which may be misleading, outdated, etc.  Use Google Scholar  through the Harvard Library instead.

About Wikipedia .  W ikipedia is not considered scholarly, and should not be cited, but it frequently includes references to scholarly articles. Before using those references for an assignment, double check by finding them in Hollis or a more specific subject  database .

Still not sure about a source? Consult the course syllabus for guidance, contact your professor or teaching fellow, or use the Ask A Librarian service.

• Last Updated: Oct 3, 2023 3:37 PM
• URL: https://guides.library.harvard.edu/FindingScholarlyArticles

Harvard University Digital Accessibility Policy

Communication Research Strategies

• Comm Research Strategies
• The research process
• Defining your topic and crafting your research question
• Identifying search terms from your question
• Broaden or narrow your search
• Research methods
• Find background information
• Find journalism data and polls
• Find journal articles
• Find books and ebooks
• Find U.S. news articles
• Find international news articles
• Interlibrary loan
• How to distinguish between types of journal articles
• Components of a scholarly article, and things to consider when reading one
• Critically evaluating articles & other resources
• Literature reviews (opens a new guide) This link opens in a new window
• Annotated bibliographies
• Writing tools
• Citing sources (opens a new guide) This link opens in a new window
• Associated Press Style Manual
• Understanding & Avoiding Plagiarism (opens a new guide) This link opens in a new window

Contact me for research assistance

Components of a scholarly article

• Experimentation
• Results, analysis, & discussion
• Limitations
• Conclusion(s)

“What is this article about?”

Abstract : Most articles start with a paragraph called the “abstract”, which very briefly summarizes the whole article.

Introduction : This section introduces the research question under consideration in the article, and discusses what the article contributes to existing knowledge within the field.

Modified from the Department of Sociology, SUNY Brockport: https://www.brockport.edu/academics/sociology/journal.html

“What do we already know about this topic and what is left to discover?”

Literature review : the authors will review the existing research and theory on the topic, either as part of the introduction, or after the introduction under its own subtitle. The review of literature is meant to discuss previous work on the topic, point out what questions remain, and relate the research presented in the rest of the article to the existing literature.

“How did the author do the research?  Why this way and not this other way?”

Methodology : There is always some discussion of the methods used by the authors to conduct the study being reported.  If these is any sort of control group being used as part of the methodology, it will be described here.

"What specifically did the authors do?"

Experimentation : A description of the experimentation in which the authors engaged.  This section will report any data produced by the experiment, be it numerical data, qualitative response data, etc.  Data comes in many different forms.

“What did the authors find?  What did their findings reveal or say in response to their research question?”

Results, analysis, & discussion : A section or multiple sections will be devoted to analyzing the experiment that was conducted, as well as its results.  These sections then discuss the results, and what the analysis of these results revealed.

"What limitations does this research have, and how might they affect the conclusions that may be drawn from this research?"

Limitations: A description of any limitations of the experimentation in which the authors engaged, and how these limitations may affect the conclusions that may be drawn from this experimentation.

“What does it all mean and why is it important?”

Discussion and Conclusion : Articles typically end by discussing what the results mean and how the study contributes to existing knowledge. Answers to research questions are addressed in this section, as is whether any hypotheses were supported or not. The conclusion typically places the research in a larger context and discusses where future research on the topic may be headed.

Things to consider when reading a scholarly article / Shortcuts to reading journal articles

• Things to consider when reading a scholarly article
• Shortcuts to reading journal articles

Owen Lin. (2012, May 23). Deep Thoughts. Retrieved from https://www.flickr.com/photos/owen-pics/8406656015/. Used under the Creative Commons License.

• Know your research question or argument.  Though your question/argument may change or evolve as you delve deeper into the research process, you will want to have a solid idea of your research focus.  
• You don't have to read the entire article in order.  Start with the abstract which will give you a general summary of the article. If the abstract seems relevant then move to the conclusion or discussion section of the article to gain a better understanding of the article's main claims. At this point if the article does not seem relevant or useful then discard it. However, if the article does seem useful then spend as much time as necessary reading the article.  
• Read critically.  What is the author's argument? You will need to use your judgment when evaluating each source of information. Further research may be necessary if you find the author to be biased or you do not believe the validity of their argument.  
• Read the reference section.  Reading the references or works cited may lead you to other useful resources. You might also get a better understanding of the major players in the area you are researching.  
• Take notes.  How you do this is up to you. Make sure you keep your research question and argument in mind so you can be more efficient when taking notes. Created by Rachel Arteaga, for CSU Chico: http://libguides.csuchico.edu/c.php?g=462359&p=3163509

• Read the abstract first : Titles don’t always give much information. The abstract should give you just enough information to let you know the basics of the article. From this you will know whether you should read on or look elsewhere for your project. Some journals print a list of keywords pertaining to the article as well. These are further clues about the article.  
• Read the introduction and discussion/conclusion next : These sections will give you the main argument of the article, which should be helpful in determining its relevance to you and your project. You’ll also get a glimpse of the findings of the research being reported.  
• Read about the methods next : If what you’ve read so far interests you, get a sense of how the research was done. Is it a qualitative or quantitative project? What data are the study based on?  
• Read the Analysis and results next : If you decide that you are committed to this article, you should read in more detail about this research.

Created by the Department of Sociology, SUNY Brockport: https://www.brockport.edu/academics/sociology/journal.html

What makes information peer-reviewed vs. scholarly vs. non-scholarly? Which type of source should I use?

• What makes information peer-reviewed vs. scholarly vs. non-scholarly?
• Which type of source should I use?

There is a nuanced distinction between peer-review and scholarship, which typically doesn't matter when evaluating sources for possible citation in your own work.  Peer-review is a process through which editors of a journal have other experts in the field evaluate articles submitted to the journal for possible publication.  Different journals have different ways of defining an expert in the field.  Scholarly works, by contrast have an editorial process, but this process does not involve expert peer-reviewers.  Rather, one or more editors, who are themselves often highly decorated scholars in a field, evaluate submissions for possible publication.  This editorial process can be more economically driven than a peer-review process, with a greater emphasis on marketing and selling the published material, but as a general rule this distinction is trivial with regard to evaluating information for possible citation in your own work.

What is perhaps a more salient way of thinking about the peer-review / scholarship distinction is to recognize that while peer-reviewed information is typically highly authoritative, and is generally considered "good" information, the absence of a peer-review process doesn't automatically make information "bad."  More specifically, the only thing the absence of a peer-review process means is that information published in this manner is not peer-reviewed.  Nothing more.  Information that falls into this category is sometimes referred to as "non-scholarly" information -- but again, that doesn't mean this information is somehow necessarily problematic.

Where does that leave you in terms of deciding what type of information to use in producing your own work?  That is a highly individual decision that you must make.  The Which type of source should I use?  tab in this box offers further guidance on answering this question, though it is important to be aware that many WSU instructors will only consider peer-reviewed sources to be acceptable in the coursework you turn in .  You can ask your instructor for his or her thoughts on the types of sources s/he will accept in student work.

Image:  Martin Grater. (2017, Nov. 1). Deep Thought. Retrieved from https://www.flickr.com/photos/152721954@N05/24304490568/. Used under the Creative Commons License.

Your topic and research question or thesis statement will guide you on which resources are best.  Sources can be defined as primary, secondary and tertiary levels away from an event or original idea. Researchers may want to start with tertiary or secondary source for background information. Learning more about a topic will help most researchers make better use of primary sources.

While articles from scholarly journals are often the most prominent of the sources you will consider incorporating into your coursework, they are not the only sources available to you.  Which sources are most appropriate to your research is a direct consequence of they type of research question you decide to address.  In other words, while most university-level papers will require you to reference scholarly sources, not all will.  A student in an English course writing a paper analyzing Bob Dylan's lyrics, for example, may find an interview with Dylan published in Rolling Stone magazine a useful source to cite alongside other scholarly works of literary criticism.

The WSU University Libraries' What Sources Should I Use? handout, as well as the other sub-tabs under the  Evaluating information  section of this guide (which is indeed the section you are currently viewing) offer further guidance on understanding and identifying scholarly resources, and comparing them against different criteria to evaluate if they will be of value to your research.  How many non-scholarly works (if any) you are at liberty to cite alongside scholarly ones is often a question to ask of your professor.  Some may not want you to cite any, whereas others may be ok with some non-scholarly works cited alongside scholarly ones.

Image:  Brett Woods. (2006, Jan. 6). Deep Thoughts. Retrieved from https://www.flickr.com/photos/brettanicus/87653641/. Used under the Creative Commons License.

• << Previous: How to distinguish between types of journal articles
• Next: Critically evaluating articles & other resources >>
• Last Updated: Jul 8, 2024 10:12 AM
• URL: https://libraries.wichita.edu/comm-research-strategies

• Interlibrary Loan and Scan & Deliver
• Course Reserves
• Purchase Request
• Collection Development & Maintenance
• Current Negotiations
• Ask a Librarian
• Instructor Support
• Library How-To
• Research Guides
• Research Support
• Study Rooms
• Research Rooms
• Partner Spaces
• Loanable Equipment
• Print, Scan, Copy
• 3D Printers
• Poster Printing
• OSULP Leadership
• Strategic Plan

AEC 406: Projects (Ecampus)

• Citations & References
• Characteristics of Scholarly Articles

Journal Information

Author and affiliation, specialized vocabulary, research sponsors, introduction, literature review, methodology.

• Getting Journal Articles
• Borrowing Books
• Annotated Bibliograpahy

One of the first things you want to do is look at the journal citation . The information about the article, such as the journal title, volume, issue, and pages is usually on the first page of the article. You will need this information to cite the article correctly in your paper. Sometimes you can tell that the article is scholarly just by the name of the journal, but not always. Scholarly articles are generally long so looking at the number of pages is one indicator that it is scholarly.

You may also want to find out more about the journal in which the article was published. For example, you may want to know if the article is in a peer-reviewed journal. 

One of the first things to look for is the author or authors. In a research article, the authors will list their affiliation, usually with a university or research institution. In this example, the author's affiliation is clearly shown on the first page of the article. In a research article, you will never have an anonymous author or need to look for the author's name or affiliation.

An abstract is a summary of the main article. An abstract will include information about why the research study was done, what the methodology was and something about the findings of the author(s). The abstract is always at the beginning of the article and will either be labeled "abstract" or will be set apart from the rest of the article by a different font or margins.

The abstract should tell you what the research study is about, how the research was done (methodology), who the research sample was, what the authors found and why this is important to the field.

In many articles there are key words or terms that describe the content of the article. This is another good place to look for additional search terms.

Some research is sponsored by an institution, such as the U.S. government or a non-profit or private company. If the research project is sponsored by someone, you will want to investigate who that is and if that could impact how the research was done. For example, a pharmaceutical company that sponsors research on a particular medication may mean that the research is not entirely bias-free.

Most articles will start with an introductory section, which may be labeled introduction. This section introduces the research study, the thesis statement and why the research being conducted is important.

Questions to ask while you read:

• What is the thesis? What are the authors trying to prove or disprove?
• What is the contribution that the authors are making to the field?

The literature review section of an article is a summary or analysis of all the research the author read before doing his/her own research. This section may be part of the introduction or in a section called Background. It provides the background on who has done related research, what that research has or has not uncovered and how the current research contributes to the conversation on the topic. When you read the lit review ask:

• Does the review of the literature logically lead up to the research questions?
• Do the authors review articles relevant to their research study?
• Do the authors show where there are gaps in the literature?

The lit review is also a good place to find other sources you may want to read on this topic to help you get the bigger picture.

The methodology section or methods section tells you how the author(s) went about doing their research. It should let you know a) what method they used to gather data (survey, interviews, experiments, etc.), why they chose this method, and what the limitations are to this method.

The methodology section should be detailed enough that another researcher could replicate the study described. When you read the methodology or methods section:

• What kind of research method did the authors use? Is it an appropriate method for the type of study they are conducting?
• How did the authors get their tests subjects? What criteria did they use?
• What are the contexts of the study that may have affected the results (e.g. environmental conditions, lab conditions, timing questions, etc.)
• Is the sample size representative of the larger population (i.e., was it big enough?)
• Are the data collection instruments and procedures likely to have measured all the important characteristics with reasonable accuracy?
• Does the data analysis appear to have been done with care, and were appropriate analytical techniques used? 

A good researcher will always let you know about the limitations of his or her research.

Research articles are full of data . The data should be complete and directly support the conclusions the authors' draw about their research question.

Tables, graphs, and charts are good indicators that this is a research article. The tables should represent the data in a clear and readable manner.

The results section in a scholarly article is where the author(s) talk about what they found in their research study. Most scholarly articles will have a section labeled results or findings.

The discussion section is where the author(s) write about what they found and what they think it means. The authors may also draw some conclusions about the research and what significance it has in this section. This section will also tell you what some of the issues were with the research or using a specific population for a research study.

The final section is usually called the conclusion or recommendations. Here is where the authors summarize what they found, why they think their research is significant and, if appropriate, make recommendations about future actions or future research that needs to be conducted. In some cases, the conclusion is part of the discussion section.

• << Previous: Step 2
• Next: Getting Journal Articles >>
• Last Updated: Aug 27, 2024 2:08 PM
• URL: https://guides.library.oregonstate.edu/AEC406

Contact Info

121 The Valley Library Corvallis OR 97331–4501

Phone: 541-737-3331

Services for Persons with Disabilities

In the Valley Library

• Oregon State University Press
• Special Collections and Archives Research Center
• Undergrad Research & Writing Studio
• Graduate Student Commons
• Tutoring Services
• Northwest Art Collection

Digital Projects

• Oregon Explorer
• Oregon Digital
• ScholarsArchive@OSU
• Digital Publishing Initiatives
• Atlas of the Pacific Northwest
• Marilyn Potts Guin Library  
• Cascades Campus Library
• McDowell Library of Vet Medicine

What is a Scholarly Article and How Do I Find One?

• The Basics of Academic/Scholarly Resources
• Database Essentials
• How To Read a Scholarly Article
• Library 101: Useful Information

Us the red button below to schedule an appointment, or contact me via email to schedule an appointment at a different time.

All About Scholarly Resources

• The Basics of Scholarly Resources
• Anatomy of a Scholarly Article
• Popular Resources
• What is a Scholarly Article Chart

At some point in your academic career you will probably be asked to locate and use a scholarly or academic resource.  As an academic library, Meriam Library collects mostly scholarly resources. What does this mean? Scholarly resources have the following features:

1. They are  written by experts -  look for an author's credentials or affiliations.

2. They are written for other experts or people in academia. Think of each scholarly work as a voice in an ongoing conversation to which you will add your voice when you write a paper. 

3. They use scholarly language with technical, discipline specific vocabulary.

4.   They provide verifiable and reliable evidence for claims. Even if the resource is a general history/overview it will contain well researched information that the reader can verify.

5. They may be peer reviewed. Many journals go through an editorial process where other experts review and assess the information. 

How do you know if a journal is peer reviewed? Some databases will let you check a box to limit to peer reviewed articles. You can also look at the journal's website which will explain the editorial process including whether or not the journal is peer reviewed. 

For more comprehensive journal publication information see below:

• Ulrich's Periodicals Directory Call Number: Z6941 U5 2015 vol.1 - 4 Use this directory to look up information about periodicals including whether the publication is peer-reviewed.

Scholarly articles are published in scholarly journals. Most of these journals are discipline specific. For example if you study microbiology you might want to look in the Journal of Bacteriology. If you study Shakespeare you might look at Shakespeare Quarterly. Scholarly articles have certain things in common.

What to look for :

• Bibliographic information  (author, title, publisher, date, volume and issue number)
• Author credentials and affiliations  (what and where of  expertise ) 
• An  abstract  stating a summary of the article
• Science and social science articles will most likely have an  introduction, methodology  (how research was conducted),  results, discussion , and  conclusion
• Notes, references, or works cited ; This information is provided so readers know where the information was obtained, can verify sources, and/or use information for their own research. 

Sometimes you will encounter popular resources. Not only will you need to learn how to differentiate scholarly and popular resources but you will need to know when it is appropriate to use a popular resource. Here is some information about popular resources: 

Features of popular resources such as magazines :

Glossy pictures Written for general audiences most likely by non-experts such as journalists Easy to locate and purchase Catchy headlines and titles Articles tend to be short Lots of color images and advertisements Little or no reference to where the information was obtained Examples: Time, Oprah, National Geographic, People, Sports Illustrated

• More Information Use this chart for more information about scholarly vs. popular resources.

Click the image or use the below PDF for more information.

• What is a Scholarly Article?
• What is a Scholarly Article? (COVID Impact 2020)
• Next: Database Essentials >>
• Last Updated: Oct 14, 2020 9:25 AM
• URL: https://libguides.csuchico.edu/scholarly

Meriam Library | CSU, Chico

• EMU Library
• Research Guides
• Library & Research Help

Peer Review & Scholarly Sources

Why use scholarly sources.

• Scholarly Sources & Peer Review
• Types of Articles

Because of the level of authority and credibility evident in scholarly sources they contribute a great deal to the overall quality of your papers. Use of scholarly sources is an expected attribute of academic course work.

In all disciplines, knowledge is built by responding to the ideas and discoveries of those who came before us. Scholarly journal articles are unique in that they require authors to document and make verifiable the sources of the facts, ideas, and methods they used to arrive at their insights and conclusions. Scholarly articles also strive to identify and discuss the merits of alternative explanations and viewpoints for the positions they espouse. This makes it easier to assess the truth, as well as the strengths and weaknesses, of the claims made in a paper. This is the case for those with knowledge of a subject (for example, your professor), as well as for those just beginning to learn about a subject (for example, you).

As you know, anyone can say just about anything in articles posted on the web. While you might agree with the conclusions of a paper found on the web, you are often not given the chain of evidence you need to assess the truth of those conclusions. Likewise, articles published in popular magazines, while they provide information and opinions, are not required to document evidence that either supports or negates their conclusions. Scholarly journal articles, unlike web-based or popular magazine articles, are designed and structured to provide the elements necessary to most thoroughly evaluate the validity and truth of an author's position.

• << Previous: Types of Articles
• Last Updated: Jul 24, 2024 1:01 PM
• URL: https://guides.emich.edu/scholarly

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Scholarly Sources: What are They and Where can You Find Them?

• 3 minute read
• 46.7K views

Table of Contents

As part of your research journey, it is not only important to conduct high-quality research, but also to add credibility to the research articles you want to publish. Distinguishing your work from the millions of other research publications is challenging but citing good quality scholarly sources is one way to get there!

What is a scholarly source?

A scholarly source is an article or publication written by a subject-matter expert who ensures that the information it contains is updated, accurate, and of utmost quality. Usually, this information is derived from their findings, theories, analyses, and insights from years of research. Scholarly sources can include journal articles, books, conference publications, and even websites.

How are scholarly sources different from popular sources?

As opposed to scholarly sources, popular sources may not be written by subject matter experts. These include newspapers, magazines, book reviews, and editorials, which are non-technical and appeal to the common public. Although they are easier to understand, they may not always contain reliable information.

Scholarly sources undergo a rigorous peer-review process before they are published for the verification of facts or for the identification of conflicts of interest. They are, therefore, much more reliable.

What counts as a scholarly source?

Here are certain characteristics that can help identify scholarly sources:

1. Accuracy

Information in scholarly sources is factually and grammatically accurate. Additionally, a bibliography or a list of references generally accompanies scholarly publications to verify and acknowledge the sources that they are derived from.

2. Author details

You will find the names of all the authors along with their respective qualifications and institutional affiliations in a scholarly source, which adds to its credibility.

3. Lack of bias

Scholarly sources are based on facts and not opinions, with the information free of any confirmation or political bias. The authors appeal to the reader’s sense of logic and not emotion while writing the article.

4. Intended audience

Scholarly sources are largely meant for scholars, researchers, faculty, academicians, and other experts in the field. Since they are not written for the average reader, the language used can be highly technical.

5. Timelines

Sources used for the scholarly publication are usually quite recent, i.e., they will have been published three to five years before the date of the scholarly publication itself.

6. Publisher

Publishers for scholarly sources generally include university presses, professional associations, academic institutions, and commercial publishers.

7. Peer-review

Scholarly sources are generally peer-reviewed, i.e., they have been reviewed and verified by review boards comprising specialists in the field. However, you should note that in some cases, articles may not be peer-reviewed and still be considered scholarly.

The purpose of scholarly sources is to communicate highly academic, research-based ideas.

Why are scholarly sources important?

1. provide credibility.

Citing scholarly sources is the best way to establish credibility of your research publication.

2. Lend authority

Since scholarly sources contain verified expert knowledge, citing them lends authority to your research submission.

3. Ensure impartiality

Scholarly sources prevent confirmation bias. Since they are derived from years of research on a subject, they offer authentic, unbiased views of the research topic, which is highly preferable.

Where can you find scholarly sources?

Popular search engines which contain scholarly sources include Science Direct, Google Scholar, Microsoft Academic, WorldWideScience, ResearchGate, PubMed, JSTOR, Academic Search Premier, and OneSearch. These websites provide thousands of scholarly sources based on various subject areas. You might be able to find scholarly sources in your university library as well!

If, however, you are still confused about finding an authentic scholarly source, seek out Elsevier Author Services . We have experts here to guide you throughout your research journey, with useful advice on how to get your research published!

Writing an Effective Cover Letter for Manuscript Resubmission

Differentiating between the abstract and the introduction of a research paper

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

TSL 3251 - Applied Linguistics for ESL (Del Prado)

• Intruduction
• Using Primo
• Using Subject Databases
• Using Effective Keywords

Anatomy of a Research Article

• Identiying Primary Sources
• APA Style Guide

In class, we discussed not just how to locate research articles in the LLBA, but also how to read them strategically by identifying their main components. As I noted in class, this will vary by field and there are times you can identify the field based just on how an article is structured. Being aware of this is an important research skill and part of being a strategic reader.

Remember that, when conducting your literature review, you don't need to read every article you find in depth. Rather, your scanning it for relevance to see which ones fit the scope of your project. You can do this by reading the abstract and introduction. If they look promising, scroll down to the results, discussion, and conclusion to see how well it fits with your project.

For example, common features of Social Science articles are:

• Author credentials
• Introduction
• Literature review
• overview of the study being conducted or analyzed
• Works Cited

Being able to identify these components and noting that they each serve a specific purpose can make a 30-40 page research study seem a lot less intimidating.

This can also give you insight into the norms of the field you're working in and how to structure your own study. 

***Pro Tip:

In class, I showed you how to scan articles for terms like "more research is needed" or "further research is needed." These terms usually appear in the discussion section or conclusion of an article and they're useful because the authors are telling you what to do next. Once you start gather articles for your lit review, scan them for those terms and you'll gain insight into where to go with your own research because they're literally telling you what needs to be done next.

• << Previous: Using Effective Keywords
• Next: Identiying Primary Sources >>
• Last Updated: Sep 12, 2024 4:35 PM
• URL: https://guides.ucf.edu/linguistics

Peer Reviewed

GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation

Article metrics.

CrossRef Citations

Altmetric Score

PDF Downloads

Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society’s evidence base, particularly in politically divisive domains, is a growing concern.

Swedish School of Library and Information Science, University of Borås, Sweden

Department of Arts and Cultural Sciences, Lund University, Sweden

Division of Environmental Communication, Swedish University of Agricultural Sciences, Sweden

Research Questions

• Where are questionable publications produced with generative pre-trained transformers (GPTs) that can be found via Google Scholar published or deposited?
• What are the main characteristics of these publications in relation to predominant subject categories?
• How are these publications spread in the research infrastructure for scholarly communication?
• How is the role of the scholarly communication infrastructure challenged in maintaining public trust in science and evidence through inappropriate use of generative AI?

research note Summary

• A sample of scientific papers with signs of GPT-use found on Google Scholar was retrieved, downloaded, and analyzed using a combination of qualitative coding and descriptive statistics. All papers contained at least one of two common phrases returned by conversational agents that use large language models (LLM) like OpenAI’s ChatGPT. Google Search was then used to determine the extent to which copies of questionable, GPT-fabricated papers were available in various repositories, archives, citation databases, and social media platforms.
• Roughly two-thirds of the retrieved papers were found to have been produced, at least in part, through undisclosed, potentially deceptive use of GPT. The majority (57%) of these questionable papers dealt with policy-relevant subjects (i.e., environment, health, computing), susceptible to influence operations. Most were available in several copies on different domains (e.g., social media, archives, and repositories).
• Two main risks arise from the increasingly common use of GPT to (mass-)produce fake, scientific publications. First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the scientific record. A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools and is also optimized to be retrieved by publicly available academic search engines, particularly Google Scholar. However small, this possibility and awareness of it risks undermining the basis for trust in scientific knowledge and poses serious societal risks.

Implications

The use of ChatGPT to generate text for academic papers has raised concerns about research integrity. Discussion of this phenomenon is ongoing in editorials, commentaries, opinion pieces, and on social media (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now several lists of papers suspected of GPT misuse, and new papers are constantly being added. 1 See for example Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . While many legitimate uses of GPT for research and academic writing exist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its undeclared use—beyond proofreading—has potentially far-reaching implications for both science and society, but especially for their relationship. It, therefore, seems important to extend the discussion to one of the most accessible and well-known intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated  (Simon et al., 2023).

Google Scholar, https://scholar.google.com , is an easy-to-use academic search engine. It is available for free, and its index is extensive (Gusenbauer & Haddaway, 2020). It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023). However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases. Instead, Google Scholar uses automated crawlers, like Google’s web search engine (Martín-Martín et al., 2021), and the inclusion criteria are based on primarily technical standards, allowing any individual author—with or without scientific affiliation—to upload papers to be indexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017). A large part of Google Scholar’s index consists of publications from established scientific journals or other forms of quality-controlled, scholarly literature. However, the index also contains a large amount of gray literature, including student papers, working papers, reports, preprint servers, and academic networking sites, as well as material from so-called “questionable” academic journals, including paper mills. The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.

To understand the occurrence of ChatGPT (co-)authored work in Google Scholar’s index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024). The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs. Most of these GPT-fabricated papers were found in non-indexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings. 2 Indexed journals mean scholarly journals indexed by abstract and citation databases such as Scopus and Web of Science, where the indexation implies journals with high scientific quality. Non-indexed journals are journals that fall outside of this indexation. More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations. To avoid increasing the visibility of these publications, we abstained from referencing them in this research note. However, we have made the data available in the Harvard Dataverse repository.

The publications were related to three issue areas—health (14.5%), environment (19.5%) and computing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-related papers, and “analysis,” “sustainable,” and “global” for environment-related papers. In several cases, the papers had titles that strung together general keywords and buzzwords, thus alluding to very broad and current research. These terms included “biology,” “telehealth,” “climate policy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and design did not include a detailed analysis of which parts of the articles included fabricated text, our dataset did contain the surrounding sentences for each occurrence of the suspicious phrases that formed the basis for our search and subsequent selection. Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion. This was confirmed during the joint coding, where we read and discussed all articles. It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.

Evidence hacking and backfiring effects

Generative pre-trained transformers (GPTs) can be used to produce texts that mimic scientific writing. These texts, when made available online—as we demonstrate—leak into the databases of academic search engines and other parts of the research infrastructure for scholarly communication. This development exacerbates problems that were already present with less sophisticated text generators (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the public release of ChatGPT in 2022, together with the way Google Scholar works, has increased the likelihood of lay people (e.g., media, politicians, patients, students) coming across questionable (or even entirely GPT-fabricated) papers and other problematic research findings. Previous research has emphasized that the ability to determine the value and status of scientific publications for lay people is at stake when misleading articles are passed off as reputable (Haider & Åström, 2017) and that systematic literature reviews risk being compromised (Dadkhah et al., 2017). It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023). Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking —the strategic and coordinated malicious manipulation of society’s evidence base.

The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific. Due to the large number of archives, repositories, mirror sites, and shadow libraries to which they spread, there is a clear risk that GPT-fabricated, questionable papers will reach audiences even after a possible retraction. There are considerable technical difficulties involved in identifying and tracing computer-fabricated papers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to mention preventing and curbing their spread and uptake.

However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them. To illustrate this mechanism, climate deniers frequently question established scientific consensus by pointing to other, supposedly scientific, studies that support their claims. Usually, these are poorly executed, not peer-reviewed, based on obsolete data, or even fraudulent (Dunlap & Brulle, 2020). A similar strategy is successful in the alternative epistemic world of the global anti-vaccination movement (Carrion, 2018) and the persistence of flawed and questionable publications in the scientific record already poses significant problems for health research, policy, and lawmakers, and thus for society as a whole (Littell et al., 2024). Considering that a person’s support for “doing your own research” is associated with increased mistrust in scientific institutions (Chinn & Hasell, 2023), it will be of utmost importance to anticipate and consider such backfiring effects already when designing a technical solution, when suggesting industry or legal regulation, and in the planning of educational measures.

Recommendations

Solutions should be based on simultaneous considerations of technical, educational, and regulatory approaches, as well as incentives, including social ones, across the entire research infrastructure. Paying attention to how these approaches and incentives relate to each other can help identify points and mechanisms for disruption. Recognizing fraudulent academic papers must happen alongside understanding how they reach their audiences and what reasons there might be for some of these papers successfully “sticking around.” A possible way to mitigate some of the risks associated with GPT-fabricated scholarly texts finding their way into academic search engine results would be to provide filtering options for facets such as indexed journals, gray literature, peer-review, and similar on the interface of publicly available academic search engines. Furthermore, evaluation tools for indexed journals 3 Such as LiU Journal CheckUp, https://ep.liu.se/JournalCheckup/default.aspx?lang=eng . could be integrated into the graphical user interfaces and the crawlers of these academic search engines. To enable accountability, it is important that the index (database) of such a search engine is populated according to criteria that are transparent, open to scrutiny, and appropriate to the workings of  science and other forms of academic research. Moreover, considering that Google Scholar has no real competitor, there is a strong case for establishing a freely accessible, non-specialized academic search engine that is not run for commercial reasons but for reasons of public interest. Such measures, together with educational initiatives aimed particularly at policymakers, science communicators, journalists, and other media workers, will be crucial to reducing the possibilities for and effects of malicious manipulation or evidence hacking. It is important not to present this as a technical problem that exists only because of AI text generators but to relate it to the wider concerns in which it is embedded. These range from a largely dysfunctional scholarly publishing system (Haider & Åström, 2017) and academia’s “publish or perish” paradigm to Google’s near-monopoly and ideological battles over the control of information and ultimately knowledge. Any intervention is likely to have systemic effects; these effects need to be considered and assessed in advance and, ideally, followed up on.

Our study focused on a selection of papers that were easily recognizable as fraudulent. We used this relatively small sample as a magnifying glass to examine, delineate, and understand a problem that goes beyond the scope of the sample itself, which however points towards larger concerns that require further investigation. The work of ongoing whistleblowing initiatives 4 Such as Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg. There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022). Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon. Our findings underline that the risk of fake scientific papers being used to maliciously manipulate evidence (see Dadkhah et al., 2017) must be taken seriously. Manipulation may involve undeclared automatic summaries of texts, inclusion in literature reviews, explicit scientific claims, or the concealment of errors in studies so that they are difficult to detect in peer review. However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science. Society’s methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders. To address this multifaceted problem, we first need to understand why it exists and proliferates.

Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page. Non-indexed journals dominate.

Most questionable papers we found were in non-indexed journals or were working papers, but we did also find some in established journals, publications, conferences, and repositories. We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1). Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases). Table 1 divides these papers into categories. Health and environment papers made up around 34% (47) of the sample. Of these, 66% were present in non-indexed journals.

Finding 2: GPT-fabricated, questionable papers are disseminated online, permeating the research infrastructure for scholarly communication, often in multiple copies. Applied topics with practical implications dominate.

The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs. The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.  Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media. It would be difficult, or impossible, to remove them from the scientific record.

As apparent from Table 2, GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.

A word rain visualization (Centre for Digital Humanities Uppsala, 2023), which combines word prominences through TF-IDF 5 Term frequency–inverse document frequency , a method for measuring the significance of a word in a document compared to its frequency across all documents in a collection. scores with semantic similarity of the full texts of our sample of GPT-generated articles that fall into the “Environment” and “Health” categories, reflects the two categories in question. However, as can be seen in Figure 1, it also reveals overlap and sub-areas. The y-axis shows word prominences through word positions and font sizes, while the x-axis indicates semantic similarity. In addition to a certain amount of overlap, this reveals sub-areas, which are best described as two distinct events within the word rain. The event on the left bundles terms related to the development and management of health and healthcare with “challenges,” “impact,” and “potential of artificial intelligence”emerging as semantically related terms. Terms related to research infrastructures, environmental, epistemic, and technological concepts are arranged further down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A second distinct event further to the right bundles terms associated with fish farming and aquatic medicinal plants, highlighting the presence of an aquaculture cluster.  Here, the prominence of groups of terms such as “used,” “model,” “-based,” and “traditional” suggests the presence of applied research on these topics. The two events making up the word rain visualization, are linked by a less dominant but overlapping cluster of terms related to “energy” and “water.”

The bar chart of the terms in the paper subset (see Figure 2) complements the word rain visualization by depicting the most prominent terms in the full texts along the y-axis. Here, word prominences across health and environment papers are arranged descendingly, where values outside parentheses are TF-IDF values (relative frequencies) and values inside parentheses are raw term frequencies (absolute frequencies).

Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.

Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking (Tripodi et al., 2023) and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.

We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers. The papers’ bibliographic information was automatically added to a spreadsheet and downloaded into Zotero. 6 An open-source reference manager, https://zotero.org .

We employed multiple coding (Barbour, 2001) to classify the papers based on their content. First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool’s role was acknowledged. Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis—health, environment, computing, and others—and then determining which subjects were most affected by this issue (see Table 1). Out of the 227 retrieved papers, 88 papers were written with legitimate and/or declared use of GPTs (i.e., false positives, which were excluded from further analysis), and 139 papers were written with undeclared and/or fraudulent use (i.e., true positives, which were included in further analysis). The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other’s interpretation of the data simultaneously in a shared spreadsheet file. This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001). Redoing the category coding later based on our established coding schedule, we achieved an intercoder reliability (Cohen’s kappa) of 0.806 after eradicating obvious differences.

The ranking algorithm of Google Scholar prioritizes highly cited and older publications (Martín-Martín et al., 2016). Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications. Only the query “as of my last knowledge update” had more than two search engine result pages. On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.

To understand how the papers making fraudulent use of generative AI were disseminated online, we programmatically searched for the paper titles (with exact string matching) in Google Search from our local IP address (see Appendix B) using the googlesearch – python library(Vikramaditya, 2020). We manually verified each search result to filter out false positives—results that were not related to the paper—and then compiled the most prominent URLs by field. This enabled the identification of other platforms through which the papers had been spread. We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.

We used descriptive statistics to count the prevalence of the number of GPT-fabricated papers across topics and venues and top domains by subject. The pandas software library for the Python programming language (The pandas development team, 2024) was used for this part of the analysis. Based on the multiple coding, paper occurrences were counted in relation to their categories, divided into indexed journals, non-indexed journals, student papers, and working papers. The schemes, subdomains, and subdirectories of the URL strings were filtered out while top-level domains and second-level domains were kept, which led to normalizing domain names. This, in turn, allowed the counting of domain frequencies in the environment and health categories. To distinguish word prominences and meanings in the environment and health-related GPT-fabricated questionable papers, a semantically-aware word cloud visualization was produced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text versions of the papers. Font size and y-axis positions indicate word prominences through TF-IDF scores for the environment and health papers (also visualized in a separate bar chart with raw term frequencies in parentheses), and words are positioned along the x-axis to reflect semantic similarity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a manually produced list including terms such as “https,” “volume,” or “years.”

• Artificial Intelligence
• / Search engines

Cite this Essay

Haider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024). GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School (HKS) Misinformation Review . https://doi.org/10.37016/mr-2020-156

• / Appendix B

Bibliography

Antkare, I. (2020). Ike Antkare, his publications, and those of his disciples. In M. Biagioli & A. Lippman (Eds.), Gaming the metrics (pp. 177–200). The MIT Press. https://doi.org/10.7551/mitpress/11087.003.0018

Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? BMJ , 322 (7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the opportunities and challenges of ChatGPT in academic writing: A roundtable discussion. Nuclear Medicine and Molecular Imaging , 57 (4), 165–167. https://doi.org/10.1007/s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology , 72 (12), 1461–1476. https://doi.org/10.1002/asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals . arXiv. https://doi.org/10.48550/arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science , 27 (3), 310–324. https://doi.org/10.1177/0963662517728024

Centre for Digital Humanities Uppsala (2023). CDHUppsala/word-rain [Computer software]. https://github.com/CDHUppsala/word-rain

Chinn, S., & Hasell, A. (2023). Support for “doing your own research” is associated with COVID-19 misperceptions and scientific mistrust. Harvard Kennedy School (HSK) Misinformation Review, 4 (3). https://doi.org/10.37016/mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple access to Google Scholar authors and citation using Python (1.5.0) [Computer software]. https://doi.org/10.5281/zenodo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable papers in citation databases as an issue for literature review. Journal of Cell Communication and Signaling , 11 (2), 181–185. https://doi.org/10.1007/s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake papers in the era of artificial intelligence. Diagnosis , 10 (4), 390–397. https://doi.org/10.1515/dx-2023-0090

DeGeurin, M. (2024, March 19). AI-generated nonsense is leaking into scientific journals. Popular Science. https://www.popsci.com/technology/ai-generated-text-scientific-journals/

Dunlap, R. E., & Brulle, R. J. (2020). Sources and amplifiers of climate change denial. In D.C. Holmes & L. M. Richardson (Eds.), Research handbook on communicating climate change (pp. 49–61). Edward Elgar Publishing. https://doi.org/10.4337/9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://aclanthology.org/W17-0237

Google Scholar Help. (n.d.). Inclusion guidelines for webmasters . https://scholar.google.com/intl/en/scholar/inclusion.html

Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022). AI-enabled image fraud in scientific publications. Patterns , 3 (7), 100511. https://doi.org/10.1016/j.patter.2022.100511

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods , 11 (2), 181–217.   https://doi.org/10.1002/jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon’s “Sting” in science. Journal of the Association for Information Science and Technology , 68 (2), 450–467. https://doi.org/10.1002/asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research , 13 (4), 1148–1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801/

Jones, N. (2024). How journals are fighting back against a wave of questionable images. Nature , 626 (8000), 697–698. https://doi.org/10.1038/d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology , 307 (2), e230171. https://doi.org/10.1148/radiol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the scientific record on abortion and mental health outcomes. BMJ , 384 , e076518. https://doi.org/10.1136/bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74 (5), 570–581. https://doi.org/10.1002/asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoulders of an academic search engine giant. Scientometrics , 107 , 1477–1487. https://doi.org/10.1007/s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A multidisciplinary comparison of coverage via citations. Scientometrics , 126 (1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4 (5). https://doi.org/10.37016/mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts. Information Visualization , 23 (3), 217–238. https://doi.org/10.1177/14738716241236188

Swedish Research Council. (2017). Good research practice. Vetenskapsrådet.

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing . Scientific American. https://www.scientificamerican.com/article/chatbots-have-thoroughly-infiltrated-scientific-publishing/

Subbaraman, N. (2024, May 14). Flood of fake science forces multiple journal closures: Wiley to shutter 19 more journals, some tainted by fraud. The Wall Street Journal . https://www.wsj.com/science/academic-studies-research-paper-mills-journals-publishing-f5a3d4bc

The pandas development team. (2024). pandas-dev/pandas: Pandas (v2.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.10957263

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313. https://doi.org/10.1126/science.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own research’: Affordance activation and disinformation spread. Information, Communication & Society , 27 (6), 1212–1228. https://doi.org/10.1080/1369118X.2023.2245869

Vikramaditya, N. (2020). Nv7-GitHub/googlesearch [Computer software]. https://github.com/Nv7-GitHub/googlesearch

This research has been supported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the research program Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

Competing Interests

The authors declare no competing interests.

The research described in this article was carried out under Swedish legislation. According to the relevant EU and Swedish legislation (2003:460) on the ethical review of research involving humans (“Ethical Review Act”), the research reported on here is not subject to authorization by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All data needed to replicate this study are available at the Harvard Dataverse: https://doi.org/10.7910/DVN/WUVD8X

Acknowledgements

The authors wish to thank two anonymous reviewers for their valuable comments on the article manuscript as well as the editorial group of Harvard Kennedy School (HKS) Misinformation Review for their thoughtful feedback and input.