literature review research conduct

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 9, 2024 11:05 AM
• URL: https://libguides.usc.edu/writingguide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 13 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

$29.99 / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

$199.00 per year

only $3.90 per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

How I fled bombed Aleppo to continue my career in science

Career Feature 08 MAY 24

Illuminating ‘the ugly side of science’: fresh incentives for reporting negative results

Hunger on campus: why US PhD students are fighting over food

Career Feature 03 MAY 24

Japan can embrace open science — but flexible approaches are key

Correspondence 07 MAY 24

US funders to tighten oversight of controversial ‘gain of function’ research

News 07 MAY 24

France’s research mega-campus faces leadership crisis

News 03 MAY 24

Mount Etna’s spectacular smoke rings and more — April’s best science images

Plagiarism in peer-review reports could be the ‘tip of the iceberg’

Nature Index 01 MAY 24

Recruitment of Principal Investigators by the School of Life Sciences, Peking University

The School of Life Sciences at Peking University is actively seeking talents, with an emphasis on bioinformatics/computational biology/AI/RNA biology.

Beijing (CN)

School of Life Sciences, Peking University

2024 Recruitment notice Shenzhen Institute of Synthetic Biology: Shenzhen, China

The wide-ranging expertise drawing from technical, engineering or science professions...

Shenzhen,China

Shenzhen Institute of Synthetic Biology

Head of Operational Excellence

In this key position, you’ll be responsible for ensuring efficiency and quality in journal workflows through continuous improvement and innovation.

United States (US) - Remote

American Physical Society

Rowland Fellowship

The Rowland Institute at Harvard seeks outstanding early-career experimentalists in all fields of science and engineering.

Cambridge, Massachusetts

Rowland Institute at Harvard

Postdoctoral Fellowship: Chemical and Cell Biology

The 2-year fellowship within a project that will combine biochemical, cell biological and chemical genetic approaches to elucidate migrasome biology

Umeå, Sweden

Umeå University

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: May 2, 2024 10:39 AM
• URL: https://libguides.northwestern.edu/literaturereviews

News alert: UC Berkeley has announced its next university librarian

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Conducting a literature review: why do a literature review, why do a literature review.

• How To Find "The Literature"
• Found it -- Now What?

Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You identify:

• core research in the field
• experts in the subject area
• methodology you may want to use (or avoid)
• gaps in knowledge -- or where your research would fit in

It Also Helps You:

• Publish and share your findings
• Justify requests for grants and other funding
• Identify best practices to inform practice
• Set wider context for a program evaluation
• Compile information to support community organizing

Great brief overview, from NCSU

Want To Know More?

• Next: How To Find "The Literature" >>
• Last Updated: Apr 25, 2024 1:10 PM
• URL: https://guides.lib.berkeley.edu/litreview

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 
• How to write a good literature review 
• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

• Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 
• Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 
• Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 
• Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 
• Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 
• Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. 

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• Life Sciences Papers: 9 Tips for Authors Writing in Biological Sciences
• What is an Argumentative Essay? How to Write It (With Examples)

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students, why traditional editorial process needs an upgrade.

Conduct a literature review

What is a literature review.

A literature review is a summary of the published work in a field of study. This can be a section of a larger paper or article, or can be the focus of an entire paper. Literature reviews show that you have examined the breadth of knowledge and can justify your thesis or research questions. They are also valuable tools for other researchers who need to find a summary of that field of knowledge.

Unlike an annotated bibliography, which is a list of sources with short descriptions, a literature review synthesizes sources into a summary that has a thesis or statement of purpose—stated or implied—at its core.

How do I write a literature review?

Step 1: define your research scope.

• What is the specific research question that your literature review helps to define?
• Are there a maximum or minimum number of sources that your review should include?

Ask us if you have questions about refining your topic, search methods, writing tips, or citation management.

Step 2: Identify the literature

Start by searching broadly. Literature for your review will typically be acquired through scholarly books, journal articles, and/or dissertations. Develop an understanding of what is out there, what terms are accurate and helpful, etc., and keep track of all of it with citation management tools . If you need help figuring out key terms and where to search, ask us .

Use citation searching to track how scholars interact with, and build upon, previous research:

• Mine the references cited section of each relevant source for additional key sources
• Use Google Scholar or Scopus to find other sources that have cited a particular work

Step 3: Critically analyze the literature

Key to your literature review is a critical analysis of the literature collected around your topic. The analysis will explore relationships, major themes, and any critical gaps in the research expressed in the work. Read and summarize each source with an eye toward analyzing authority, currency, coverage, methodology, and relationship to other works. The University of Toronto's Writing Center provides a comprehensive list of questions you can use to analyze your sources.

Step 4: Categorize your resources

Divide the available resources that pertain to your research into categories reflecting their roles in addressing your research question. Possible ways to categorize resources include organization by:

• methodology
• theoretical/philosophical approach

Regardless of the division, each category should be accompanied by thorough discussions and explanations of strengths and weaknesses, value to the overall survey, and comparisons with similar sources. You may have enough resources when:

• You've used multiple databases and other resources (web portals, repositories, etc.) to get a variety of perspectives on the research topic.
• The same citations are showing up in a variety of databases.

Additional resources

Undergraduate student resources.

• Literature Review Handout (University of North Carolina at Chapel Hill)
• Learn how to write a review of literature (University of Wisconsin-Madison)

Graduate student and faculty resources

• Information Research Strategies (University of Arizona)
• Literature Reviews: An Overview for Graduate Students (NC State University)
• Oliver, P. (2012). Succeeding with Your Literature Review: A Handbook for Students [ebook]
• Machi, L. A. & McEvoy, B. T. (2016). The Literature Review: Six Steps to Success

Graustein, J. S. (2012). How to Write an Exceptional Thesis or Dissertation: A Step-by-Step Guide from Proposal to Successful Defense [ebook]

Thomas, R. M. & Brubaker, D. L. (2008). Theses and Dissertations: A Guide to Planning, Research, and Writing

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

What is a literature review, traditional (narrative) literature review, integrative literature review, systematic reviews, meta-analysis, scoping review.

• Developing a Research Question
• Selection Criteria
• Database Search
• Documenting Your Search
• Organize Key Findings
• Reference Management

Ask Us! Health Sciences Library

The health sciences library.

Call toll-free:  (844) 352-7399 E-mail:   Ask Us More contact information

Related Guides

• Systematic Reviews by Roy Brown Last Updated Oct 17, 2023 513 views this year
• Write a Literature Review by John Glover Last Updated Oct 16, 2023 2721 views this year

A literature review provides an overview of what's been written about a specific topic. There are many different types of literature reviews. They vary in terms of comprehensiveness, types of study included, and purpose. 

The other pages in this guide will cover some basic steps to consider when conducting a traditional health sciences literature review. See below for a quick look at some of the more popular types of literature reviews.

For additional information on a variety of review methods, the following article provides an excellent overview.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x. Review. PubMed PMID: 19490148.

• Next: Developing a Research Question >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

In order to help minimize spread of the coronavirus and protect our campus community, Cowles Library is adjusting our services, hours, and building access. Read more...

• Research, Study, Learning
• Archives & Special Collections

• Cowles Library
• Find Journal Articles
• Find Articles in Related Disciplines
• Find Streaming Video
• Conducting a Literature Review
• Organizations, Associations, Societies
• For Faculty

What is a Literature Review?

Description.

A literature review, also called a review article or review of literature, surveys the existing research on a topic. The term "literature" in this context refers to published research or scholarship in a particular discipline, rather than "fiction" (like American Literature) or an individual work of literature. In general, literature reviews are most common in the sciences and social sciences.

Literature reviews may be written as standalone works, or as part of a scholarly article or research paper. In either case, the purpose of the review is to summarize and synthesize the key scholarly work that has already been done on the topic at hand. The literature review may also include some analysis and interpretation. A literature review is  not  a summary of every piece of scholarly research on a topic.

Why are literature reviews useful?

Literature reviews can be very helpful for newer researchers or those unfamiliar with a field by synthesizing the existing research on a given topic, providing the reader with connections and relationships among previous scholarship. Reviews can also be useful to veteran researchers by identifying potentials gaps in the research or steering future research questions toward unexplored areas. If a literature review is part of a scholarly article, it should include an explanation of how the current article adds to the conversation. (From: https://researchguides.drake.edu/englit/criticism)

How is a literature review different from a research article?

Research articles: "are empirical articles that describe one or several related studies on a specific, quantitative, testable research question....they are typically organized into four text sections: Introduction, Methods, Results, Discussion." Source: https://psych.uw.edu/storage/writing_center/litrev.pdf)

Steps for Writing a Literature Review

1. Identify and define the topic that you will be reviewing.

The topic, which is commonly a research question (or problem) of some kind, needs to be identified and defined as clearly as possible.  You need to have an idea of what you will be reviewing in order to effectively search for references and to write a coherent summary of the research on it.  At this stage it can be helpful to write down a description of the research question, area, or topic that you will be reviewing, as well as to identify any keywords that you will be using to search for relevant research.

2. Conduct a Literature Search

Use a range of keywords to search databases such as PsycINFO and any others that may contain relevant articles.  You should focus on peer-reviewed, scholarly articles . In SuperSearch and most databases, you may find it helpful to select the Advanced Search mode and include "literature review" or "review of the literature" in addition to your other search terms.  Published books may also be helpful, but keep in mind that peer-reviewed articles are widely considered to be the “gold standard” of scientific research.  Read through titles and abstracts, select and obtain articles (that is, download, copy, or print them out), and save your searches as needed. Most of the databases you will need are linked to from the Cowles Library Psychology Research guide .

3. Read through the research that you have found and take notes.

Absorb as much information as you can.  Read through the articles and books that you have found, and as you do, take notes.  The notes should include anything that will be helpful in advancing your own thinking about the topic and in helping you write the literature review (such as key points, ideas, or even page numbers that index key information).  Some references may turn out to be more helpful than others; you may notice patterns or striking contrasts between different sources; and some sources may refer to yet other sources of potential interest.  This is often the most time-consuming part of the review process.  However, it is also where you get to learn about the topic in great detail. You may want to use a Citation Manager to help you keep track of the citations you have found. 

4. Organize your notes and thoughts; create an outline.

At this stage, you are close to writing the review itself.  However, it is often helpful to first reflect on all the reading that you have done.  What patterns stand out?  Do the different sources converge on a consensus?  Or not?  What unresolved questions still remain?  You should look over your notes (it may also be helpful to reorganize them), and as you do, to think about how you will present this research in your literature review.  Are you going to summarize or critically evaluate?  Are you going to use a chronological or other type of organizational structure?  It can also be helpful to create an outline of how your literature review will be structured.

5. Write the literature review itself and edit and revise as needed.

The final stage involves writing.  When writing, keep in mind that literature reviews are generally characterized by a  summary style  in which prior research is described sufficiently to explain critical findings but does not include a high level of detail (if readers want to learn about all the specific details of a study, then they can look up the references that you cite and read the original articles themselves).  However, the degree of emphasis that is given to individual studies may vary (more or less detail may be warranted depending on how critical or unique a given study was).   After you have written a first draft, you should read it carefully and then edit and revise as needed.  You may need to repeat this process more than once.  It may be helpful to have another person read through your draft(s) and provide feedback.

6. Incorporate the literature review into your research paper draft. (note: this step is only if you are using the literature review to write a research paper. Many times the literature review is an end unto itself).

After the literature review is complete, you should incorporate it into your research paper (if you are writing the review as one component of a larger paper).  Depending on the stage at which your paper is at, this may involve merging your literature review into a partially complete Introduction section, writing the rest of the paper around the literature review, or other processes.

These steps were taken from: https://psychology.ucsd.edu/undergraduate-program/undergraduate-resources/academic-writing-resources/writing-research-papers/writing-lit-review.html#6.-Incorporate-the-literature-r

• << Previous: Find Streaming Video
• Next: Organizations, Associations, Societies >>
• Last Updated: Feb 29, 2024 4:09 PM
• URL: https://researchguides.drake.edu/psychology

• 2507 University Avenue
• Des Moines, IA 50311
• (515) 271-2111

Trouble finding something? Try searching , or check out the Get Help page.

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Browse Econ Literature

• Working papers
• Software components
• Book chapters
• JEL classification

More features

• Subscribe to new research

RePEc Biblio

Author registration.

• Economics Virtual Seminar Calendar NEW!

How to Undertake an Impactful Literature Review: Understanding Review Approaches and Guidelines for High-impact Systematic Literature Reviews

• Author & abstract
• Related works & more

Corrections

• Amrita Chakraborty
• Arpan Kumar Kar

Suggested Citation

Download full text from publisher.

Follow serials, authors, keywords & more

Public profiles for Economics researchers

Various research rankings in Economics

RePEc Genealogy

Who was a student of whom, using RePEc

Curated articles & papers on economics topics

Upload your paper to be listed on RePEc and IDEAS

New papers by email

Subscribe to new additions to RePEc

EconAcademics

Blog aggregator for economics research

Cases of plagiarism in Economics

About RePEc

Initiative for open bibliographies in Economics

News about RePEc

Questions about IDEAS and RePEc

RePEc volunteers

Participating archives

Publishers indexing in RePEc

Privacy statement

Found an error or omission?

Opportunities to help RePEc

Get papers listed

Have your research listed on RePEc

Open a RePEc archive

Have your institution's/publisher's output listed on RePEc

Get RePEc data

Use data assembled by RePEc

Top 12 Psychology Books That Are A Must-Read In 2024

Master the art of research: topics and tips for your psychology paper, nmc showed 37,000 medicos with disturbed mental well-being.

• The Pros and Cons of Online Therapy
• The 5 Anger Languages in Relationship

Your cart is empty

• by Psychologs Magazine
• May 14, 2024
• 6 minutes read

Writing a psychology research paper is a journey through the depths of human behaviour and mental processes. Therefore, choosing psychology research topics is crucial for a compelling paper. Before diving into your research proposal, a thorough investigation of journal databases like PsychInfo and ERIC can unveil a wealth of relevant studies. The importance of selecting the right topics for psychology research cannot be overstated. It sets the tone for the entire research paper, ensuring your work is grounded in innovative inquiry. A roadmap can be constructed by outlining your ideas of how to go about writing the paper in detail.

Topic Selection

It’s always best to identify general areas of interest in psychology such as cognition and social behavior or mental health. This broad area will help you to narrow down to more specific topics. Identifying gaps in existing research helps to identify areas of interest. This can provide a good foundation for your study and ensure a meaningful contribution to the field of mental health. 

It’s time to narrow down your broad interests to focus on a specific and manageable topic. Structure and organization become essential at this stage to follow. Just remember that the more a research question intrigues you the more it will help you to follow through with all the research phases. 

Also, when you are narrowing down your topic, it is important to select one which is not only in trend currently, but also well-researched. This will help you during the literature review phase of your research process. Lastly, it’s always best to keep in mind guidelines or criteria set by your research guide and educational institution. Your topic should meet academic standards and procedures.

Read More: 6 Reasons You Should Study Psychology

The Review of Literature 

When you begin conducting your literature review, you must refer to online databases like PsycInfo, Social Sciences Citation Index (SSCI), and Medline to search for relevant research literature.

Make sure to read the abstract, introduction, method, results, and discussion sections of each article. Discussion sections are also important since they highlight gaps in existing research. An Excel sheet must be made where you write down the summaries and key points of each research for your literature review. 

In your literature review, you must describe, compare and evaluate the research studies. The goal of evaluation is to identify gaps and inconsistencies in the literature, thus providing suggestions for solving the research problem. These tips help you develop a thorough literature review. It highlights significant studies as well as sets a foundation for your research paper.

Read More: The Depths of Indian Psychology: Traditions and Modern Insights

Adhering to APA Format

The general structure of the APA format includes the title page, abstract, introduction, method, results, discussion, and references.

The title page should have a running head, page number, title of the paper, author’s name, and institutional affiliation. The title should be concise including the key variables or the study’s focus. The abstract should be on a new page, limited to 120 words. It should summarize the research topic, methods, results, and significance. 

Your introduction should start with an exploration of your research question, including a brief history of the same. The introduction will also include a summary of the previous findings in the form of a literature review. Make sure to address gaps in the current research and how the current research will impact the field.

The method section describes in detail how the research was performed. It includes a description of the participants, the study design, the materials that were used, and the procedure of the study. It is essential to provide enough information for another researcher to duplicate your research.

The results section should describe the data that was collected through descriptive statistics as well as inferential statistics which is your t-test, ANOVA and other appropriate tests for your study. Your discussion has to interpret the results and address your research topic. It is essential to discuss the limitations and scope of future research surrounding your research. 

When referring to current research for your review literature, make sure to compile a list of all the sources . It’s also important to cite them throughout the paper wherever direct quotations are used. Refer to the APA format for in-text citations .

Read More; Do You Know About the Basic Principles of Psychology?

Analyzing Your Data

Descriptive statistics in research provide a summary of your data sets. Calculate the mean, median and mode of your data set. Thereafter, you calculate the dispersion of your data which includes variance, standard deviation and range to understand the spread of your data. 

Creating graphs and charts like histograms, bar charts and scatter plots helps visually present the data. Identify the skewness of your data to understand the asymmetry of your data distribution. Identify the techniques you will analyze your data with like t-tests, regression, or ANOVA. This analysis should help answer your research question. 

These tips help ensure that your data analysis is communicated which makes your findings understandable to other researchers and the psychology community. 

Read more: 10 Job Roles in the Field of Psychology

The importance of selecting engaging and significant topics, conducting comprehensive literature reviews, and adhering to the standards of APA format thus far has been highlighted. Researchers ensure their work helps in the understanding of human behaviour. The journey from topic selection to data presentation has its challenges, however, research papers tend to influence the scholarly community and societal perceptions of psychological phenomena. We encourage aspiring researchers to continually seek out new avenues for inquiry and adopt a critical yet open-minded stance toward their findings. The pursuit of knowledge in psychology research helps formulate interventions and policies that can help human well-being. 

1. What are the top topics for conducting Research in psychology? 

To select a psychology research topic, consider areas such as psychotropic medications for mental health, disorders like anxiety , depression , and bipolar disorder , schizoaffective disorder, accessibility of mental health services, the impact of systemic racism on mental health, and mental health concerns within the LGBTQIA+ community.

2. What are some essential tips for writing an effective psychology research paper? 

When writing a psychology research paper, ensure to support your arguments with solid evidence from scientific studies. Adhere to the appropriate academic format.

3. Where can I get extra help with APA style for my research?

You can find support online and offline to improve your understanding of APA style. The APA Publication Manual is a key resource for guidance on formatting, citations and other important aspects. 

• Chiang, I. A., Jhangiani, R. S., & Price, P. C. (2015, October 13). Generating good research questions . Pressbooks. https://opentextbc.ca/researchmethods/chapter/generating-good-research-questions/
• Paré, G., & Kitsiou, S. (2017, February 27). Chapter 9 Methods for literature reviews . Handbook of eHealth Evaluation: An Evidence-based Approach – NCBI Bookshelf. https://www.ncbi.nlm.nih.gov/books/NBK481583/
• Ratan, S. K., Anand, T., & Ratan, J. (2019). Formulation of research question – Stepwise approach. Journal of Indian Association of Pediatric Surgeons , 24 (1), 15. https://doi.org/10.4103/jiaps.jiaps_76_18
• Chiang, I. A., Jhangiani, R. S., & Price, P. C. (2015, October 13). American Psychological Association (APA) style . Pressbooks. https://opentextbc.ca/researchmethods/chapter/american-psychological-association-apa-style/
• Chiang, I. A., Jhangiani, R. S., & Price, P. C. (2015, October 13). Conducting your analyses . Pressbooks. https://opentextbc.ca/researchmethods/chapter/conducting-your-analyses/
• Chiang, I. A., Jhangiani, R. S., & Price, P. C. (2015, October 13). Expressing your results . Pressbooks. https://opentextbc.ca/researchmethods/chapter/expressing-your-results/
• San José State University Writing Center, & Tablas-Mejia, I. (n.d.). Conclusion section for research papers. Fall 2021 . https://www.sjsu.edu/writingcenter/docs/handouts/Conclusion%20Section%20for%20Research%20Papers.pdf
• P, B. (2024, March 27). Learn how to write a psychology research paper here . https://www.sharkpapers.com/blog/research-paper-writing-guides/psychology-research-paper

Share This Post:

Leave feedback about this cancel reply.

• Rating 5 4 3 2 1

Related Post

Top 12 Psychology Books That Are A Must-Read

What is the Importance of Studying Behaviourism?

6 Reasons You Should Study Psychology

Psychology behind Innovation

How to Prepare for UGC NET?

Bridging the gap: a systematic analysis of circular economy, supply chain management, and digitization for sustainability and resilience

• Published: 13 May 2024

Cite this article

• Bhawna   ORCID: orcid.org/0000-0003-3032-0104 1 , 2 ,
• Parminder Singh Kang 2 , 3 &
• Sanjeev Kumar Sharma 1  

The primary objective of this research paper is to conduct a comprehensive and systematic literature review (SLR) focusing on Sustainable Supply Chain Management (SSCM) practices that promote Circular Economy (CE), sustainability, and resilience through adopting emerging digital technologies. A SLR of 130 research articles published between 1991 and 2023 was used to analyze emerging trends in CE, supply chain management (SCM), and digitalization. This study meticulously examined research publication patterns, the intricate themes explored, influential scholars, leading countries, and substantial scientific contributions that have shaped this multifaceted domain. This paper contributed to the collective understanding of how SSCM practices, driven by the principles of CE and empowered by the adoption of digital technologies, foster sustainability, resilience, and innovation within contemporary SCs. The research findings presented herein are primarily based on an analysis of the current literature from only Scopus and Web of Science (WoS) databases, which may restrict the generalizability of implementing these results. Based on this study, organizations and practitioners can assess the maturity of their SCM practices, gauge the resilience and digitalization levels of their SCs, and align them with academic literature trends. This enables practitioners to bridge the gap between scholarly advancements and real-world SCM implementation. Through its systematic review, the study provides a structured literature review that offers a collective understanding of SSCM practices driven by CE principles and empowered by digital technologies. This understanding enables sustainability, resilience, and innovation within contemporary SCs, benefiting academicians and practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

Data can be provided upon request.

Alamelu R, Jayanthi M, Dinesh S, Nalini R, Shobhana N, Amudha R (2023) Sustainable supply chain and circular economy ingenuities in small manufacturing firms-a stimulus for sustainable development. Mater Today: Proc 92:17–23

Google Scholar  

Bag S, Pretorius JHC (2022) Relationships between industry 4.0, sustainable manufacturing and circular economy: proposal of a research framework. Int J Organ Anal 30(4):864–898. https://doi.org/10.1108/IJOA-04-2020-2120

Article   Google Scholar  

Bag S, Wood LC, Xu L, Dhamija P, Kayikci Y (2020) Big data analytics as an operational excellence approach to enhance sustainable supply chain performance. Resour Conserv Recycl 153:104559. https://doi.org/10.1016/j.resconrec.2019.104559

Birkel H, Hohenstein N-O, Hähner S (2023) How have digital technologies facilitated supply chain resilience in the COVID-19 pandemic? An exploratory case study. Comput Ind Eng 183:109538. https://doi.org/10.1016/j.cie.2023.109538

Branke J, Farid SS, Shah N (2016) Industry 4.0: a vision for personalized medicine supply chains? Cell Gene Therapy Insights 2(2):263–270. https://doi.org/10.18609/cgti.2016.027

Carter CR, Rogers DS (2008) A framework of sustainable supply chain management: moving toward new theory. Int J Phys Distribution Logistics Manage 38(5):360–387

Centobelli P, Cerchione R, Esposito E, Passaro R, Shashi (2021) Determinants of the transition towards circular economy in SMEs: a sustainable supply chain management perspective. Int J Prod Econ 242:108297. https://doi.org/10.1016/j.ijpe.2021.108297

Cerqueira-Streit J, Endo G, Guarnieri P, Batista L (2021) Sustainable supply Chain Management in the Route for a circular economy: an integrative literature review. Logistics 5(4):81. https://doi.org/10.3390/logistics5040081

Chadegani AA, Salehi H, Yunus MM, Farhadi H, Fooladi M, Farhadi M, Ebrahim NA (2013) A comparison between two Main Academic Literature collections: web of Science and Scopus databases. https://doi.org/10.48550/ARXIV.1305.0377

Chari A, Niedenzu D, Despeisse M, Machado CG, Azevedo JD, Boavida-Dias R, Johansson B (2022) Dynamic capabilities for circular manufacturing supply chains—exploring the role of industry 4.0 and resilience. Bus Strategy Environ 31(5):2500–2517. https://doi.org/10.1002/bse.3040

Chauhan C, Singh A (2019) A review of industry 4.0 in supply chain management studies. J Manuf Technol Manage 31(5):863–886

Dentoni D, Pinkse J, Lubberink R (2021) Linking Sustainable Business models to Socio-Ecological Resilience through Cross-sector partnerships: a Complex Adaptive systems View. Bus Soc 60(5):1216–1252. https://doi.org/10.1177/0007650320935015

Donthu N, Kumar S, Pattnaik D (2020) Forty-five years of Journal of Business Research: a bibliometric analysis. J Bus Res 109:1–14

Edwin Cheng TC, Kamble SS, Belhadi A, Ndubisi NO, Lai K, Kharat MG (2022) Linkages between big data analytics, circular economy, sustainable supply chain flexibility, and sustainable performance in manufacturing firms. Int J Prod Res 60(22):6908–6922. https://doi.org/10.1080/00207543.2021.1906971

El Baz J, Tiwari S, Akenroye T, Cherrafi A, Derrouiche R (2022) A framework of sustainability drivers and externalities for industry 4.0 technologies using the best-worst method. J Clean Prod 344:130909. https://doi.org/10.1016/j.jclepro.2022.130909

Esmaeilian B, Sarkis J, Lewis K, Behdad S (2020) Blockchain for the future of sustainable supply chain management in industry 4.0. Resour Conserv Recycl 163:105064. https://doi.org/10.1016/j.resconrec.2020.105064

Ferasso M, Beliaeva T, Kraus S, Clauss T, Ribeiro-Soriano D (2020) Circular economy business models: the state of research and avenues ahead. Bus Strategy Environ 29(8):3006–3024. https://doi.org/10.1002/bse.2554

Fisher O, Watson N, Porcu L, Bacon D, Rigley M, Gomes RL (2018) Cloud manufacturing as a sustainable process manufacturing route. J Manuf Syst 47:53–68. https://doi.org/10.1016/j.jmsy.2018.03.005

Gaur V, Gaiha A (2020) Building a transparent supply chain blockchain can enhance trust, efficiency, and speed. Harvard Business Rev 98(3):94–103

Ghisellini P, Cialani C, Ulgiati S (2016) A review on circular economy: the expected transition to a balanced interplay of environmental and economic systems. J Clean Prod 114:11–32. https://doi.org/10.1016/j.jclepro.2015.09.007

Golicic SL, Smith CD (2013) A meta-analysis of environmentally sustainable supply chain management practices and firm performance. J Supply Chain Manage 49(2):78–95

Govindan K, Hasanagic M (2018) A systematic review on drivers, barriers, and practices towards circular economy: a supply chain perspective. Int J Prod Res 56(1–2):278–311. https://doi.org/10.1080/00207543.2017.1402141

Graham G, Hardaker G (2000) Supply-chain management across the Internet. Int J Phys Distrib Logist Manag 30(3/4):286–295. https://doi.org/10.1108/09600030010326055

Hendry LC, Stevenson M, MacBryde J, Ball P, Sayed M, Liu L (2019) Local food supply chain resilience to constitutional change: the Brexit effect. Int J Oper Prod Manage 39(3):429–453. https://doi.org/10.1108/IJOPM-03-2018-0184

Hofmann E, Rüsch M (2017) Industry 4.0 and the current status as well as future prospects on logistics. Comput Ind 89:23–34. https://doi.org/10.1016/j.compind.2017.04.002

Ingemarsdotter E, Jamsin E, Kortuem G, Balkenende R (2019) Circular strategies enabled by the internet of things—A framework and analysis of current practice. Sustainability 11(20):5689

Jabbarzadeh A, Fahimnia B, Sabouhi F (2018) Resilient and sustainable supply chain design: sustainability analysis under disruption risks. Int J Prod Res 56(17):5945–5968. https://doi.org/10.1080/00207543.2018.1461950

Lopes de Sousa Jabbour AB, Jabbour CJ, Godinho Filho M, Roubaud D (2018) Industry 4.0 and the circular economy: a proposed research agenda and original roadmap for sustainable operations. Ann Oper Res 270:273–286

Kähkönen A-K, Lintukangas K, Hallikas J (2018) Sustainable supply management practices: making a difference in a firm’s sustainability performance. Supply Chain Management: Int J 23(6):518–530

Karmaker CL, Aziz A, Ahmed R, Misbauddin T, Moktadir MA (2023) Impact of industry 4.0 technologies on sustainable supply chain performance: the mediating role of green supply chain management practices and circular economy. J Clean Prod 419:138249

Lu HE, Potter A, Sanchez Rodrigues V, Walker H (2018) Exploring sustainable supply chain management: a social network perspective. Supply Chain Management: Int J 23(4):257–277

Luthra S, Mangla SK (2018) Evaluating challenges to industry 4.0 initiatives for supply chain sustainability in emerging economies. Process Saf Environ Prot 117:168–179

MacArthur E (2013) Towards the circular economy. J Ind Ecol 2(1):23–44

MacArthur E (2017) A new textiles economy: redesigning fashion’s future. Ellen MacArthur Foundation 1–150

Merli R, Preziosi M, Acampora A (2018) How do scholars approach the circular economy? A systematic literature review. J Clean Prod 178:703–722. https://doi.org/10.1016/j.jclepro.2017.12.112

Morali O, Searcy C (2013) A review of sustainable supply Chain Management practices in Canada. J Bus Ethics 117(3):635–658. https://doi.org/10.1007/s10551-012-1539-4

Nandi S, Sarkis J, Hervani AA, Helms MM (2021) Redesigning Supply chains using blockchain-enabled circular economy and COVID-19 experiences. Sustainable Prod Consum 27:10–22. https://doi.org/10.1016/j.spc.2020.10.019

Obeidat SM, Abdalla S, Al Bakri AAK (2023) Integrating green human resource management and circular economy to enhance sustainable performance: an empirical study from the Qatari service sector. Empl Relations: Int J 45(2):535–563. https://doi.org/10.1108/ER-01-2022-0041

Pagell M, Wu Z (2009) Building a more complete theory of sustainable supply chain management using case studies of 10 exemplars. J Supply Chain Manage 45(2):37–56

Pettit TJ, Fiksel J, Croxton KL (2010) Ensuring supply chain resilience: development of a conceptual framework. J Bus Logistics 31(1):1–21. https://doi.org/10.1002/j.2158-1592.2010.tb00125.x

Ren Y, Li R, Wu K-J, Tseng M-L (2023) Discovering the systematic interlinkages among the circular economy, supply chain, industry 4.0, and technology transfer: a bibliometric analysis. Clean Responsible Consum 9:100123. https://doi.org/10.1016/j.clrc.2023.100123

Schmidt CVH, Kindermann B, Behlau CF, Flatten TC (2021) Understanding the effect of market orientation on circular economy practices: the mediating role of closed-loop orientation in German SMEs. Bus Strategy Environ 30(8):4171–4187. https://doi.org/10.1002/bse.2863

Shin N, Park S (2021) Supply chain leadership driven strategic resilience capabilities management: a leader-member exchange perspective. J Bus Res 122:1–13. https://doi.org/10.1016/j.jbusres.2020.08.056

Singh G, Singh S, Daultani Y, Chouhan M (2023) Measuring the influence of digital twins on the sustainability of manufacturing supply chain: a mediating role of supply chain resilience and performance. Comput Ind Eng 186:109711

Stock T, Seliger G (2016) Opportunities of Sustainable Manufacturing in Industry 4.0. Procedia CIRP 40:536–541. https://doi.org/10.1016/j.procir.2016.01.129

Talla A, McIlwaine S (2022) Industry 4.0 and the circular economy: using design-stage digital technology to reduce construction waste. Smart Sustainable Built Environ. https://doi.org/10.1108/SASBE-03-2022-0050

Tavera Romero CA, Castro DF, Ortiz JH, Khalaf OI, Vargas MA (2021) Synergy between circular economy and industry 4.0: a literature review. Sustainability 13(8):4331. https://doi.org/10.3390/su13084331

Tortorella G, Fogliatto FS, Gao S, Chan T-K (2022) Contributions of industry 4.0 to supply chain resilience. Int J Logistics Manage 33(2):547–566

Yadav G, Luthra S, Jakhar SK, Mangla SK, Rai DP (2020) A framework to overcome sustainable supply chain challenges through solution measures of industry 4.0 and circular economy: an automotive case. J Clean Prod 254:120112. https://doi.org/10.1016/j.jclepro.2020.120112

Download references

This research paper is part of a funded research project under Mitacs, Canada (Funding Ref. FR106245).

Author information

Authors and affiliations.

University Institute of Applied Management Sciences (UIAMS), Panjab University, Chandigarh, India

Bhawna & Sanjeev Kumar Sharma

Department of Decision Sciences, School of Business, MacEwan University, Edmonton, AB, Canada

Bhawna & Parminder Singh Kang

School of Engineering and Sustainable Development, Faculty of Technology, De Montfort University, Leicester, UK

Parminder Singh Kang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Bhawna .

Ethics declarations

Conflict of interest.

There are no relevant financial or non-financial competing interests to report.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Bhawna, Kang, P.S. & Sharma, S.K. Bridging the gap: a systematic analysis of circular economy, supply chain management, and digitization for sustainability and resilience. Oper Manag Res (2024). https://doi.org/10.1007/s12063-024-00490-4

Download citation

Received : 12 October 2023

Revised : 20 March 2024

Accepted : 16 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1007/s12063-024-00490-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sustainable supply chain management
• Circular economy
• Supply chain sustainability
• Supply chain digitalization
• Research trends
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 13.5.2024 in Vol 26 (2024)

Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review

Authors of this article:

• Line Farah 1, 2 * , PharmD   ; 
• Isabelle Borget 2, 3, 4 , PharmB, PhD   ; 
• Nicolas Martelli 2, 5 , PharmD, PhD   ; 
• Alexandre Vallee 6 * , MD, PhD  

1 Innovation Center for Medical Devices Department, Foch Hospital, Suresnes, France

2 Groupe de Recherche et d'accueil en Droit et Economie de la Santé Department, University Paris-Saclay, Orsay, France

3 Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, Villejuif, France

4 Oncostat U1018, Inserm, Équipe Labellisée Ligue Contre le Cancer, University Paris-Saclay, Villejuif, France

5 Pharmacy Department, Georges Pompidou European Hospital, Paris, France

6 Department of Epidemiology and Public Health, Foch Hospital, Suresnes, France

*these authors contributed equally

Corresponding Author:

Line Farah, PharmD

Innovation Center for Medical Devices Department

Foch Hospital

40 rue Worth

Suresnes, 92150

Phone: 33 952329655

Email: [email protected]

Background: Artificial intelligence (AI)–based medical devices have garnered attention due to their ability to revolutionize medicine. Their health technology assessment framework is lacking.

Objective: This study aims to analyze the suitability of each health technology assessment (HTA) domain for the assessment of AI-based medical devices.

Methods: We conducted a scoping literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched databases (PubMed, Embase, and Cochrane Library), gray literature, and HTA agency websites.

Results: A total of 10.1% (78/775) of the references were included. Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process. When it comes to implementing specialized HTA for AI-based medical devices, several practical challenges and potential barriers could be highlighted and should be taken into account (AI technological evolution timeline, data requirements, complexity and transparency, clinical validation and safety requirements, regulatory and ethical considerations, and economic evaluation).

Conclusions: The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into the effectiveness, cost-effectiveness, and societal impact of AI-based medical devices, guiding their responsible implementation and maximizing their benefits for patients and health care systems.

Introduction

Artificial intelligence (AI) has emerged as a transformative technology with vast potential across various sectors, including health care [ 1 , 2 ]. In this field, AI-based medical devices have garnered significant attention due to their ability to revolutionize diagnosis, treatment, and patient monitoring [ 3 ]. These devices use advanced algorithms and machine learning techniques to analyze complex medical data sets, thereby providing valuable insights and support to health care professionals in their decision-making. To meet the ever-increasing demand for integration of AI-based medical devices into clinical practice, their efficient evaluation through adapted health technology assessment (HTA) is now crucial [ 4 ]. Several frameworks have been published showing an adaptation of items for reporting clinical trials related to AI-based medical devices ( Table 1 ). However, there is still a lack of a much needed adaptation of the standard HTA framework to better suit the assessment of AI-based health care technologies.

a HTA: health technology assessment.

b CONSORT-AI: Consolidated Standards of Reporting Trials–Artificial Intelligence.

c CONSORT: Consolidated Standards of Reporting Trials.

d SPIRIT-AI: Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence.

e SPIRIT: Standard Protocol Items: Recommendations for Interventional Trials.

f TRIPOD-AI: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis-Artificial Intelligence.

g TRIPOD: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.

An HTA involves a systematic evaluation of a medical technology’s clinical, economic, ethical, and social implications to determine its overall value and impact on health care delivery [ 9 ]. The European Network for Health Technology Assessment has designed an HTA Core Model that provides a methodological framework for production and sharing of HTA information [ 5 ]. It evaluates the following nine domains: (1) health problem and current use of technology, (2) description and technical characteristics of the technology, (3) safety, (4) clinical effectiveness, (5) costs and economic evaluation, (6) ethical analysis, (7) organizational aspects, (8) patients and social aspects, and (9) legal aspects ( Multimedia Appendix 1 ).

A full understanding of the capabilities, limitations, and specificities of AI-based medical devices is paramount to complete these HTA domain assessments and, thereby, inform evidence-based decision-making and allow for policy development and the responsible integration of these technologies into health care systems [ 10 , 11 ]. Much uncertainty remains with regard to the reliability of AI-based medical devices, data issues, and regulatory processes, resulting in multiple challenges faced by HTA agencies assessing new technologies and delivering their approval [ 4 ]. Nevertheless, AI-based medical devices require strict regulations and specific legislations [ 12 - 14 ]. Over the last few decades, the evaluation of AI-based medical devices through an HTA process has received growing interest, as shown by the increase from 1 published article in 1990 to 94 in December 2023, with 484 articles in total during the period 1990-2023 ( Table 2 ).

The objective of this review was to critically assess the comprehensive suitability of the current HTA process for AI-based medical devices. By evaluating the performance and capabilities of AI-based medical devices across multiple dimensions, this review aimed to inform health care professionals, policy makers, and researchers about the challenges and opportunities associated with these technologies. Ultimately, this review sought to facilitate evidence-based decision-making; promote responsible implementation; and maximize the potential benefits of AI-based medical devices in improving health care quality, accessibility, and outcomes.

To this end, we analyzed the suitability of each HTA domain for the assessment of AI-based medical devices and proposed an adapted HTA framework.

Search Strategies

A scoping literature review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology [ 15 , 16 ]. All articles related to HTA methods for AI-based medical devices were selected. Data extraction focused on assessment criteria, methodological evaluations, and results.

The search terms and strategy outlined in Textbox 1 were used for the scoping review.

Database and search strategy

• “technology assessment, biomedical” [MeSH term] OR (“technology” [all fields] AND “assessment” [all fields] AND “biomedical” [all fields]) OR “biomedical technology assessment” [all fields] OR (“health” [all fields] AND “technology” [all fields] AND “assessment” [all fields]) OR “health technology assessment” [all fields]
• “equipment and supplies” [MeSH term] OR (“equipment” [all fields] AND “supplies” [all fields]) OR “equipment and supplies” [all fields] OR (“medical” [all fields] AND “device” [all fields]) OR “medical device” [all fields]
• “artificial intelligence” [MeSH term] OR (“artificial” [all fields] AND “intelligence” [all fields]) OR “artificial intelligence” [all fields]
• (“artificial intelligence” OR “artificial intelligence-based medical device”) AND “technology assessment”
• (technology assessment) and (artificial intelligence)

We searched multiple databases: PubMed, Embase, and the Cochrane Library. Additional articles were retrieved manually from the gray literature and from HTA agency websites (Bundesinstitut für Arzneimittel und Medizinprodukte [Federal Institute for Drugs and Medical Devices in Germany], Haute Autorité de Santé [French Health Authority], National Health Service, National Institute for Health and Care Excellence, National Institute of Public Health, International Network of Agencies for Health Technology Assessment, and the European Network for Health Technology Assessment).

Study Selection Process

The studies were selected by 2 reviewers (LF and AV). After removing duplicates, both reviewers independently screened the abstracts to select eligible articles and then analyzed full-text reports for eligibility. A third party (NM) resolved the possible discrepancies highlighted during the selection process should a consensus not be reached. An extraction database was used to list the selected articles meeting the inclusion criteria to ensure that all eligible articles were included.

The inclusion and exclusion criteria were related to (1) language (this review was limited to the English and French languages), (2) article type (reviews and primary research were included, whereas other article types were excluded, such as abstracts, commentaries, editorials, letters to the editor, case reports, case series, animal studies, phase-1 and phase-2 studies, pilot studies, duplicate studies, irrelevant studies, studies with the wrong aim, and studies available in abstract form only), (3) type of technology (only AI-based medical devices were eligible), and (4) type of evaluation (HTA articles were included).

Data Extraction

In total, 2 analysts (LF and AV) independently extracted data items from the selected articles. A third party was involved to resolve any discrepancies highlighted during the selection process. The following data were extracted from each article: (1) general characteristics of the studies (authors, journal, and publication date), (2) study objective, and (3) HTA assessment domain for AI-based medical devices related to the article.

Methodological Quality Appraisal

Neither the methodological quality nor the risk of bias of the included articles were assessed, consistent with scoping review guidance [ 15 ].

Application Example of the HTA of an AI-Based Medical Device

Concerning each HTA domain, we added a case study of the HTA of several AI-based medical devices in diabetes to illustrate our recommendations.

Scoping Review Results

The literature search resulted in 775 citations summarized in the PRISMA flowchart ( Figure 1 ). After removal of 32.6% (253/775) of duplicates, we carried out an initial screening of the remaining 522 publications, resulting in 129 (24.7%) potentially relevant full-text papers. After further screening, we assessed 77 reports for eligibility, based on which we excluded 11 (14%) for not referring to AI-based medical devices and 2 (3%) for not being HTAs. We then included a further 14 records from 59 potential citations available from HTA agency websites and gray literature.

This gave a total of 78 included articles covering one or multiple HTA domains, the distribution of which is provided at the bottom of Figure 1 .

We then summarized the data collection for each selected article in Multimedia Appendix 2 [ 1 , 2 , 4 , 9 , 10 , 17 - 89 ].

Suitability of Each HTA Domain for the Assessment of AI-Based Medical Devices

As domain 1, health problem and current use of technology, is suitable for any type of medical technology and systematically addressed these technologies, we focused on the suitability of the HTA domains (from 1 to 9).

Domains 1 and 2 (Health Problem, Current Use, Description, and Technical Characteristics of the Technology)

Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process [ 17 ]. These devices often rely on accessing and analyzing diverse health care data sources, including electronic health records, medical images, genetic data, and wearable device data. Therefore, it is essential to evaluate their ability to seamlessly integrate and exchange data with existing health care systems [ 18 ].

First, interoperability refers to the ability of AI-based medical devices to interact and communicate with other health care technologies and systems [ 10 ]. This includes the ability to access and use data from different sources, such as laboratory systems, imaging archives, and patient health records. The assessment should consider whether the devices adhere to relevant data standards and protocols, ensuring efficient data exchange and compatibility with existing health care infrastructures [ 19 ].

Second, data integration involves the ability of AI-based medical devices to aggregate and analyze data from multiple sources to provide comprehensive and accurate insights [ 20 ]. The HTA should assess whether the devices can handle different data types, formats, and resolutions and whether they can effectively integrate and harmonize data from disparate sources.

In addition, data privacy and security considerations are crucial when evaluating AI-based medical devices [ 21 ]. The assessment should examine whether the devices comply with relevant data protection regulations, use appropriate data anonymization and encryption techniques, and have robust security measures in place to protect patient information [ 22 ].

Therefore, a comprehensive HTA should address both the interoperability and data integration capabilities of AI-based medical devices, ensuring that they can seamlessly interact with existing health care systems and integrate data from multiple sources and that they adhere to data privacy and security standards. By evaluating all these aspects, the HTA could determine the devices’ feasibility, scalability, and potential impact on health care delivery.

Domain 3 (Safety) and Domain 4 (Clinical Effectiveness)

The assessment of clinical effectiveness and the impact on patient outcomes is a crucial aspect when evaluating AI-based medical devices through a comprehensive HTA [ 4 ]. While accuracy and performance metrics are important, it is essential to determine how these devices translate into tangible benefits for patients and health care delivery [ 23 ].

Clinical utility refers to the extent to which the AI-based medical device improves clinical decision-making, patient outcomes, and health care processes [ 24 ]. The HTA should examine whether the device provides actionable and reliable information that potentially helps health care professionals make more accurate diagnoses or improve treatment plans or monitoring strategies [ 25 , 26 ]. It should also evaluate the potential impact of the device on patient outcomes by assessing, for example, improved survival rates, reduced complications, or enhanced quality of life [ 27 ].

To assess clinical utility, the HTA should consider the device’s performance in relevant clinical scenarios and its ability to address specific clinical questions or challenges. This may involve evaluating the device’s performance against established clinical guidelines or expert opinions as well as considering the device’s potential to fill gaps in clinical practice or enhance existing diagnostic or treatment methods.

Furthermore, the HTA should examine the broader impact of AI-based medical devices on health care systems and resource allocation [ 21 ]. This includes evaluating the devices’ potential to optimize resource use, reduce health care costs, or improve workflow efficiency. Economic evaluations such as cost-effectiveness analyses can provide insights into the value for money and long-term cost savings associated with the adoption of these devices [ 28 , 29 ].

A comprehensive HTA should thoroughly assess the clinical effectiveness and impact on patient outcomes of AI-based medical devices, examining their performance in relevant clinical scenarios, their alignment with clinical guidelines, and their potential to improve health care processes and resource allocation. By evaluating these aspects, the HTA can provide a holistic understanding of the AI devices’ effectiveness and their potential to positively transform health care delivery [ 21 ].

AI-based medical devices must undergo rigorous clinical validation to assess their performance and reliability in real-world health care settings [ 30 , 31 ]. Clinical validation involves evaluating the device’s accuracy, sensitivity, specificity, and overall diagnostic or prognostic performance. In addition, the devices should be tested across diverse patient populations and compared against gold-standard reference methods or expert opinions [ 32 ].

Furthermore, the use of real-world evidence (RWE) is crucial for a comprehensive HTA [ 33 ]. RWE involves gathering data from routine clinical practice, electronic health records, and other sources to evaluate the device’s effectiveness and safety in real-world settings [ 34 ]. These data can help assess the device’s performance in a broader patient population and identify any potential limitations or biases that may arise in specific clinical scenarios.

Robust clinical validation studies and the integration of RWE provide critical evidence for the evaluation of AI-based medical devices in an HTA process [ 4 ]. These studies should include a sufficient sample size, appropriate study design, and statistical analysis to ensure the validity and generalizability of the results [ 35 - 38 ]. By considering clinical validation and RWE adapted to AI-based medical devices, HTA can provide valuable insights into their clinical utility and impact in real-world health care settings, facilitating evidence-based decision-making for their adoption and use.

A lack of safety evaluation was highlighted in a systematic review, showing that only 9% of AI-based medical device studies evaluated safety criteria [ 4 ]. However, safety is crucial for the confidence in and adoption of AI-based technologies for both patients and health care professionals [ 39 ]. While these devices have the potential to improve diagnosis and treatment outcomes, their integration into clinical practice must be accompanied by robust safety evaluations [ 2 , 40 ]. Identifying and mitigating potential risks, such as algorithmic bias, data privacy breaches, and algorithm failures, is essential to protect patient well-being and maintain trust in these innovative technologies [ 41 ].

To ensure the safety of AI-based medical devices, it is crucial to establish standardized evaluation frameworks and guidelines [ 39 ]. These frameworks should encompass rigorous testing methodologies, validation procedures, and continuous monitoring of AI performance. Collaboration among stakeholders, including manufacturers, regulatory agencies, health care providers, and researchers, is essential to develop and implement comprehensive safety evaluation protocols. Addressing the safety gap in AI-based medical devices not only ensures patient welfare but also instills confidence in health care professionals to embrace and use these technologies effectively [ 40 ]. By prioritizing safety evaluation, we can unleash the full potential of AI-based medical devices, leading to transformative advancements in health care delivery [ 1 ].

Domain 5 (Costs and Economic Evaluation)

Costs and economic evaluation play a crucial role in the comprehensive assessment of health technology adoption, particularly in the context of the HTA of AI-based medical devices [ 4 , 9 , 42 ]. While AI technologies show huge potential to improve health care delivery, reduce time to diagnosis, and save money, their economic impact requires careful consideration to ensure their successful integration and sustainability [ 23 ].

When assessing AI-based medical devices through the HTA process, economic evaluations involve analyzing the costs and benefits associated with the adoption and use of the technology [ 4 ]. These evaluations go beyond the upfront costs of acquiring the AI-based medical device; they encompass various factors, such as training, infrastructure modifications, maintenance, and ongoing operational costs [ 43 ]. Cost analysis also includes improved patient outcomes, reduced hospital readmissions, and shortened hospital stays. By conducting economic evaluations, decision makers can gauge the cost-effectiveness and cost utility to weigh the affordability of AI-based medical devices for their particular use against the potential mid- and long-term cost savings considered [ 44 ].

HTA evaluates the suitability of AI-based medical devices by considering their potential economic benefits, risks, and ethical implications [ 45 , 46 ]. By incorporating economic evaluation into the HTA process, decision makers can make evidence-based choices about the allocation of limited health care resources and prioritize interventions that provide the greatest value for money [ 4 , 47 ].

Finally, considering cost savings and conducting economic evaluations as part of the HTA process helps facilitate the adoption of AI-based medical devices [ 37 ]. It provides decision makers with the necessary information to determine the financial feasibility and potential return on investment associated with implementing these technologies. HTA ensures that AI-based medical devices are suitable for integration into the health care system, fostering confidence in their effectiveness, efficiency, and cost utility [ 4 , 43 , 48 ].

Domain 6 (Ethical Analysis) and Domain 8 (Patients and Social Aspects)

Ethical and societal implications are critical aspects that must be considered when evaluating the suitability of AI-based medical devices for a comprehensive HTA [ 45 ]. The integration of AI technologies into health care raises important ethical concerns that need to be addressed to ensure responsible and equitable use [ 49 ].

Data privacy and patient consent are primary ethical considerations [ 50 ]. The HTA should evaluate whether AI-based medical devices adhere to strict data protection regulations, maintain patient confidentiality, and obtain appropriate informed consent for data use. It is essential to assess whether the devices have mechanisms in place to handle sensitive patient information securely and protect it against unauthorized access or data breaches.

Algorithmic fairness and bias are additional ethical concerns [ 51 ]. AI algorithms can inadvertently perpetuate biases present in the data used for training, resulting in unequal treatment or access to health care resources [ 52 ]. The HTA should assess whether the devices have been evaluated for fairness and bias and consider the steps taken to mitigate any identified biases [ 53 ].

Moreover, the HTA should examine the impact of AI-based medical devices on health care disparities and access to care [ 54 ]. It is crucial to assess whether the devices have the potential to exacerbate existing inequalities or whether they can contribute to reducing disparities by improving health care access, particularly for underserved populations.

Accountability and transparency in AI decision-making processes are also important ethical considerations [ 55 ]. The HTA should evaluate whether the AI devices provide clear explanations for their outputs and ensure that health care professionals and patients can understand and challenge the device’s recommendations [ 21 ].

A comprehensive HTA should thoroughly examine the ethical and societal implications of AI-based medical devices, ensuring that they prioritize patient privacy, fairness, and equitable access to care; fostering trust; and ensuring that these devices align with societal values and goals.

To ensure the adoption of AI-based medical devices through a truthfully AI concept, explainability, interpretability, and transparency are important considerations for their comprehensive HTA [ 56 ]. Indeed, these devices often use complex algorithms and machine and deep learning techniques, resulting in their operating as “black boxes” that reach decisions and recommendations that are challenging to understand [ 55 ]. However, explainability and transparency are essential to ensure trust, accountability, and acceptance of AI-based technologies in health care. First, Explainability refers to the ability to understand and interpret the reasoning behind the device’s outputs [ 56 ]. It involves providing clinicians and users with transparent explanations of how the AI algorithm processes input data and generates results. This enables health care professionals to trust the device’s recommendations and make informed decisions based on the provided information. The HTA should evaluate the extent to which AI-based medical devices can provide interpretable and understandable explanations of their decision-making processes [ 17 , 46 ]. Second, Transparency involves disclosing important information about the AI-based medical devices, including the data used for training, algorithmic methodologies used, and potential limitations or biases [ 57 ]. Transparency promotes trust and allows stakeholders to assess the device’s reliability and potential risks. The HTA should assess whether the device manufacturers provide clear documentation and information to health care professionals and patients about the device’s capabilities, limitations, and potential errors [ 21 , 58 , 59 ]. Transparency is closely linked to regulatory considerations [ 60 , 61 ]. The HTA should consider whether the device complies with relevant regulatory standards and whether the manufacturers have provided the necessary documentation and evidence to support their ethical claims [ 21 , 62 , 63 ].

A comprehensive HTA process should address concerns regarding the explainability and transparency of AI-based medical devices, trust, accountability, and ethical implications related to the use of AI in health care [ 45 , 46 ].

Domain 7 (Organizational Aspects)

Organizational aspects play a crucial role in the effective integration of AI-based medical devices into health care systems [ 64 ]. To ensure consistency, transparency, and comparability in the evaluation process, there is a growing need for a robust methodological framework that provides standardized guidelines for assessing the organizational impact of the implementation of AI-based medical devices [ 65 ]. According to a descriptive analysis led in German hospitals, the main barriers to AI-based medical device adoption were lack of resources (staff, knowledge, and financial) [ 66 ].

Clear indicators are needed to measure the organizational readiness for and impact of AI-based medical devices [ 4 , 10 , 67 ]. Several criteria have been highlighted, such as (1) health care workplace readiness and stakeholder acceptance, (2) AI-based medical device organization alignment assessment, and (3) business plan (financing and investments) [ 64 ].

Domain 9 (Legal Aspects)

The roles and responsibilities of health care professionals are also impacted by these AI solutions [ 68 ]. It is necessary to evaluate whether they complement or replace health care professionals’ expertise and whether additional training, supervision, or support is required for their optimal use [ 69 , 70 ]. As AI technologies could evolve and become more prevalent in health care, it is crucial to ensure that these devices comply with existing legal frameworks and regulations [ 49 , 71 ]. HTA plays a pivotal role in evaluating the legal implications of AI-based medical devices by assessing factors such as data privacy, security, liability, and regulatory compliance [ 72 ].

One key legal aspect to consider is data privacy and protection [ 73 , 74 ]. AI-based medical devices often rely on vast amounts of patient data for training and decision-making. Therefore, it is essential to evaluate whether these devices adhere to relevant data protection laws, such as the General Data Protection Regulation in the European Union [ 75 , 76 ]. HTA examines the measures taken by AI-based medical device manufacturers to safeguard patient privacy, including data anonymization, encryption, and secure data storage practices [ 37 ]. These issues are highlighted in the Artificial Intelligence Act published in June 2023 by the European Commission [ 77 , 78 ].

Cybersecurity is another critical consideration [ 21 , 79 ]. As AI-based medical devices handle sensitive patient data and make critical health care decisions, it is crucial to assess the security measures implemented to prevent unauthorized access, data breaches, or tampering [ 80 ]. HTA evaluates the robustness of the security protocols implemented by device manufacturers and their compliance with industry standards and regulations [ 21 , 81 ].

Liability is also a significant legal aspect to be addressed in the context of AI-based medical devices [ 82 , 83 ]. When errors or adverse events occur due to the use of these devices, determining liability can be complex [ 84 ]. HTA examines the legal frameworks and liability guidelines pertaining to AI technologies, including whether clear guidelines exist regarding the responsibility of manufacturers, health care providers, and users in case of device malfunctions or errors [ 85 ]. Assessing liability aspects within the HTA process helps establish accountability and ensures that legal frameworks adequately address potential risks [ 86 ].

Regulatory compliance is a crucial consideration when assessing the suitability of AI-based medical devices for HTA [ 87 ]. Depending on the jurisdiction, AI devices need to undergo regulatory approval processes before being introduced into the market [ 88 ]. HTA examines whether AI-based medical device manufacturers have obtained the necessary regulatory approvals, such as clearance from the relevant health authorities or certification from regulatory bodies such as the Food and Drug Administration (FDA) in the United States or notified bodies in Europe [ 89 ]. This evaluation ensures that AI-based medical devices comply with existing regulations and are fit for clinical use [ 61 ].

Recommendations to Adapt the 9 HTA Domains for AI-Based Medical Devices

Taking into account the previous considerations, some recommendations could be proposed to adapt and personalize these standard HTA domains to AI-based medical devices’ specificities. Therefore, we suggest 4 main recommendations by domain in Table 3 .

a ML: machine learning.

b DL: deep learning.

c DMS: diabetes management system.

d FDA: Food and Drug Administration.

The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions [ 90 - 92 ]. It promotes consistency in the assessment methodologies, reporting formats, and presentation of findings, enabling decision makers to make informed choices based on reliable and comparable evidence. This standardization contributes to the overall credibility and acceptance of HTA outcomes related to AI-based medical devices [ 93 ]. In addition, a methodological framework would address the need for appropriate expertise and skills to conduct the HTA of AI-based medical devices [ 94 - 99 ]. It would outline the qualifications and competencies required for the individuals involved in the assessment, including knowledge of AI technologies [ 46 ]. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices ( Figure 2 ).

Evaluating the trade-offs and weighing different features of AI-based medical devices is indeed a complex and important task, especially in domains such as health care where the impact on human lives is significant. The acceptability of AI-based medical devices should be assessed on a case-by-case basis considering various factors, including, for instance, performance, accuracy, cost, explainability, and the specific context in which they are being used.

Trade-offs between accuracy and cost are common in AI. It may be acceptable for AI to increase follow-up care costs if it significantly improves accuracy and patient outcomes. For example, if an AI system can detect diseases at an earlier stage, it might lead to more effective treatment and, ultimately, lower overall health care costs in the long run.

The balance between explainability and performance is a critical consideration. While explainable algorithms are preferred for safety and transparency reasons, there may be cases in which a highly complex, unexplainable algorithm outperforms explainable ones. In such cases, the trade-off between transparency and performance should be carefully evaluated based on the specific use case and potential risks.

The minimum performance for adding this technology to the tools available to human clinicians depends on the specific task and the level of trust that patients, health care providers, and regulators have in the technology. Some key factors to consider include the following: (1) The complexity of the task—AI-based medical devices should excel in tasks that are well defined and data driven, but they may not replace human clinicians in tasks requiring complex decision-making, empathy, or ethical considerations. (2) Safety and reliability—AI-based medical devices should demonstrate a high level of safety and reliability, ideally surpassing the performance of human clinicians in terms of avoiding errors. (3) Ethical considerations—AI-based medical devices should adhere to ethical standards, including patient privacy, informed consent, and unbiased decision-making, which are often considered even more important than performance metrics. (4) Regulatory approval—regulatory bodies often establish performance thresholds for AI-based medical devices. Compliance with these thresholds is essential for market acceptance.

In general, AI should aim to complement and enhance the capabilities of human clinicians rather than completely replacing them. The specific threshold for acceptable performance will vary across applications and contexts, and it should be determined through a combination of rigorous testing, peer-reviewed studies, and input from health care professionals and patients.

It is important to note that ethical considerations, patient safety, and the potential for bias should always be at the forefront of these discussions, and AI-based medical devices should not be adopted solely for the sake of automation or cost reduction if they compromise these critical aspects of health care.

Use Case of the Application of the Aforementioned HTA Recommendations for AI: AI-Based Medical Devices in Pathways for Patients With Diabetes

Concerning the HTA domain 3 on the safety of an AI-based medical device, the potential risk of patient injury of an insulin delivery AI-based medical device system should be taken into account in the risk management process of algorithm development [ 96 ]. In the case of an evolutive deep learning–based medical device without continuous ongoing safety assessment, a wrong dosage administration due to an AI error, for instance, could provoke a serious adverse event such as an acid ketosis coma for a patient with diabetes.

A risk management plan should be available for users and regularly updated with the impact of the AI changes on the safety plan.

In relation to the HTA domain 4, which focuses on the effectiveness of an AI-based medical device, the FDA in the United States has proposed a regulatory framework for modifications to evolutive AI- and machine learning–based software as a medical device with modification guidance focused on the risk to users or patients resulting from the AI changes [ 100 ]. They have asked for an Algorithm Change Protocol with specific methods in place to achieve and control the risks of the anticipated types of modifications related to performance, use, or inputs. A continuous clinical effectiveness assessment could be an interesting approach for an AI-based medical device detection system for diabetes foot ulcer for patients needing adaptative treatment modifications to prevent ulcer development [ 101 ].

Concerning the HTA domain 5, to evaluate a diabetic retinopathy screening AI-based medical device, clinical effectiveness is not sufficient. The cost-effectiveness of different types of AI diabetic retinopathy screening should be compared with no screening and ophthalmologist screening. A recent study demonstrated that AI-based screening was the most cost-effective, not only saving costs but also improving the quality of life of patients with diabetes [ 44 ]. In this case, the long-term assessment of the economic impact of AI introduction in diabetic retinopathy screening highlighted the added value of this technology.

The HTA domain 7 on organizational impact has to assess how the AI-based medical device can be effectively integrated into the health care pathway and prevent wasteful spending. More thorough attention must be paid to the following aspects: (1) evaluating needs and determining the added value of the implementation of the AI-based medical device; (2) assessing workplace preparedness, including stakeholder acceptance of the introduction of the AI-based medical device and involvement; and (3) analyzing the alignment between AI technology and organizational structure [ 64 ]. Decision makers and technology advocates need to address the complexities of AI more comprehensively and understand the systemic challenges that its adoption poses to health care organizations and systems. As an example, consider an AI tool used for diagnosing diabetic retinopathy in a primary care setting, such as by a family physician or nurse [ 102 ]. In theory, this could lead to shorter waiting times for patients. However, if the health care organization faces challenges such as a shortage of specialized staff (eg, ophthalmologists), insufficient organization of care pathways, and lack of specialized facilities for proper management and follow-up after diagnosis, the introduction of AI might adversely affect the quality of care and patient experience. In such a scenario, the AI application might merely transfer the delay from primary to secondary care, failing to address the fundamental issue.

Finally, concerning the last domains (6, 8, and 9) about ethical, patient, social, and legal aspects, AI-based continuous glucose monitoring and insulin pumps should also be assessed on ethical and legal aspects [ 103 ]. While citizen juries have generally shown support for AI in research and treatment, concerns remain. The risks of data theft and privacy breaches necessitate careful consideration of ethical and legal issues for patients. Although AI can aid in decision-making, it cannot wholly substitute a physician’s expertise. Effective regulations and systems designed to ensure safety, reduce bias, and enhance transparency are essential.

By implementing these recommendations, stakeholders can foster the responsible integration, regulation, and evaluation of AI-based medical devices. These measures can enhance the evidence base, address ethical concerns, and maximize the potential benefits of these AI technologies in improving health care outcomes while protecting patient safety, privacy, and equity.

Practical Challenges and Potential Barriers to Implementing HTAs Specific for AI-Based Medical Devices

When it comes to implementing specialized HTAs for AI-based medical devices, several practical challenges and potential barriers could be encountered and should be taken into account: First, AI technological evolution timeline—AI technologies evolve at a much faster pace compared to traditional medical devices, making it challenging for HTA frameworks to keep up with the latest developments and assess their long-term impact effectively. AI-based medical devices have a short product lifetime, between 12 and 18 months, in contrast to drug products. This shorter life cycle highlights the need for evolutive and fast-track HTA processes for AI-based medical devices. Second, data requirements and quality—AI systems rely heavily on large data sets for training and validation. Ensuring the availability of high-quality, representative data is a significant challenge. There is also the issue of data privacy and security, which must be addressed. The availability and quality of data and evidence required for conducting HTAs on AI-based medical devices present a complex and evolving landscape. Assessing the feasibility and challenges associated with gathering such data is crucial for robust evaluations. First, the availability of data can vary significantly depending on the AI-based medical device. While some devices may have access to vast amounts of high-quality real-world patient data, others might face limitations due to the novelty of the technology or issues related to data privacy. Second, the quality of data is paramount as inaccurate or biased data can lead to flawed assessments. Ensuring data accuracy, representativeness, and relevance is a constant challenge in AI-based medical device evaluations [ 104 ]. The rapid pace of AI development can result in limited long-term data, making it difficult to assess the device’s sustained performance and safety [ 105 ]. Balancing the need for robust evidence with the dynamic nature of AI technologies is a significant challenge that HTA organizations must address to provide valuable insights for informed decision-making in health care. Third, complexity and transparency—the complex algorithms used in AI-based medical devices can be difficult to understand and assess, leading to issues with transparency and explainability [ 17 ]. This complexity can pose a challenge for regulators and assessors in HTA processes. Fourth, clinical validation and safety requirements—generating robust clinical evidence to demonstrate the safety, efficacy, and effectiveness of AI-based medical devices can be challenging. This includes proving that these devices perform consistently across diverse patient populations. Moreover, to account for a life cycle that would include updates that may improve performance, one solution to investigate could be the proposition of the FDA in the United States in 2019 to experiment with a “dynamic certification” [ 100 ]. It allows for the re-evaluation of the AI-based medical device in case of a substantial modification of the indication of the medical device or the way to deliver the diagnosis, for instance. Fifth, regulatory and ethical considerations—adapting existing regulatory frameworks to accommodate AI-based medical devices, addressing ethical concerns such as bias, and ensuring equitable access are critical challenges [ 106 ]. Sixth, economic evaluation—determining the cost-effectiveness of AI-based medical devices, especially when benefits might be indirect or long term, poses a unique challenge for HTA [ 93 ]. Seventh, stakeholder engagement and trust—building trust among health care providers, patients, and policy makers regarding the reliability, trustworthiness, and usefulness of AI-based medical devices is crucial but challenging [ 107 ]. Eighth, integration into health care systems and interoperability—the integration of AI-based medical devices into existing health care workflows and systems can be complex and resource intensive [ 18 ]. Ninth, global and local applicability—ensuring that AI-based medical devices are effective and appropriate for use in different global and local contexts, considering varying health care systems and population needs, is another significant barrier.

Practically, one suggestion to implement such frameworks could be to implement these recommendations in the future European clinical joint assessment guidelines. As they are being currently discussed in a European project on a common HTA process for connected medical devices that includes AI-based medical devices, it could be the opportunity to tackle these challenges at the European level [ 108 ].

Addressing these challenges requires a concerted effort from regulators, health care providers, technology developers, and other stakeholders in the health care ecosystem.

Conclusions

AI-based medical devices have the potential to transform health care delivery, but the suitability of the current comprehensive HTA requires careful adaptation of the evaluation across the 9 dimensions. While these AI devices show promise in improving accuracy, safety, and efficiency, there is a need for robust clinical validation, integration into workflows, economic evaluation, and addressing of ethical and legal implications. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into their effectiveness, cost-effectiveness, and societal impact, guiding their responsible implementation and maximizing their benefits for patients and health care systems.

Data Availability

The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions

LF and AV contributed to the conceptualization. LF and AV contributed to the methodology. LF, AV, NM, and IB contributed to the article selection. LF and AV contributed to the writing and original draft preparation. NM and IB contributed to the writing and review and editing. LF and AV contributed to the validation. The authors have read and agreed to the published version of the manuscript.

Conflicts of Interest

None declared.

The 9 health technology assessment (HTA) domains for the assessment of artificial intelligence–based medical devices adapted from the HTA Core Model proposed by the European Network for Health Technology Assessment.

Summary results of the scoping review and general characteristics of the studies included in the scoping review.

• Bajwa J, Munir U, Nori A, Williams B. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthc J. Jul 2021;8(2):e188-e194. [ CrossRef ] [ Medline ]
• Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in Healthcare. Cambridge, MA. Academic Press; 2020.
• Park CW, Seo SW, Kang N, Ko B, Choi BW, Park CM, et al. Artificial intelligence in health care: current applications and issues. J Korean Med Sci. Nov 02, 2020;35(42):e379. [ CrossRef ] [ Medline ]
• Farah L, Davaze-Schneider J, Martin T, Nguyen P, Borget I, Martelli N. Are current clinical studies on artificial intelligence-based medical devices comprehensive enough to support a full health technology assessment? A systematic review. Artif Intell Med. Jun 2023;140:102547. [ CrossRef ] [ Medline ]
• HTA core model. European Network for Health Technology Assessment. URL: https://www.eunethta.eu/hta-core-model/ [accessed 2021-10-20]
• Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK, SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. Sep 2020;26(9):1364-1374. [ CrossRef ] [ Medline ]
• Rivera SC, Liu X, Chan AW, Denniston AK, Calvert MJ, SPIRIT-AICONSORT-AI Working Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. BMJ. Sep 09, 2020;370:m3210. [ CrossRef ] [ Medline ]
• Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. Jul 09, 2021;11(7):e048008. [ CrossRef ] [ Medline ]
• Chen Y. Health technology assessment and economic evaluation: is it applicable for the traditional medicine? Integr Med Res. Mar 2022;11(1):100756. [ CrossRef ] [ Medline ]
• He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. Jan 2019;25(1):30-36. [ CrossRef ] [ Medline ]
• Giordano C, Brennan M, Mohamed B, Rashidi P, Modave F, Tighe P. Accessing artificial intelligence for clinical decision-making. Front Digit Health. Jun 25, 2021;3:645232. [ CrossRef ] [ Medline ]
• Vayena E, Blasimme A, Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. Nov 6, 2018;15(11):e1002689. [ CrossRef ] [ Medline ]
• Dzobo K, Adotey S, Thomford NE, Dzobo W. Integrating artificial and human intelligence: a partnership for responsible innovation in biomedical engineering and medicine. OMICS. May 2020;24(5):247-263. [ CrossRef ] [ Medline ]
• Zawati M, Lang M. What's in the box?: uncertain accountability of machine learning applications in healthcare. Am J Bioeth. Nov 2020;20(11):37-40. [ CrossRef ] [ Medline ]
• Peters MD, Godfrey C, McInerney P, Khalil H, Larsen P, Marnie C, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI Evid Synth. Apr 01, 2022;20(4):953-968. [ CrossRef ] [ Medline ]
• Pollock D, Peters MD, Khalil H, McInerney P, Alexander L, Tricco AC, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evid Synth. Mar 01, 2023;21(3):520-532. [ CrossRef ] [ Medline ]
• Farah L, Murris JM, Borget I, Guilloux A, Martelli NM, Katsahian SI. Assessment of performance, interpretability, and explainability in artificial intelligence–based health technologies: what healthcare stakeholders need to know. Mayo Clin Proc Digit Health. Jun 2023;1(2):120-138. [ CrossRef ]
• Lehne M, Sass J, Essenwanger A, Schepers J, Thun S. Why digital medicine depends on interoperability. NPJ Digit Med. Aug 20, 2019;2:79. [ CrossRef ] [ Medline ]
• Esmaeilzadeh P. Use of AI-based tools for healthcare purposes: a survey study from consumers' perspectives. BMC Med Inform Decis Mak. Jul 22, 2020;20(1):170. [ CrossRef ] [ Medline ]
• Zia A, Aziz M, Popa I, Khan SA, Hamedani AF, Asif AR. Artificial intelligence-based medical data mining. J Pers Med. Aug 24, 2022;12(9):1359. [ CrossRef ] [ Medline ]
• Ming J, He Y, Yang Y, Hu M, Zhao X, Liu J, et al. Health technology assessment of medical devices: current landscape, challenges, and a way forward. Cost Eff Resour Alloc. Oct 05, 2022;20(1):54. [ CrossRef ] [ Medline ]
• Filkins BL, Kim JY, Roberts B, Armstrong W, Miller MA, Hultner ML, et al. Privacy and security in the era of digital health: what should translational researchers know and do about it? Am J Transl Res. Mar 15, 2016;8(3):1560-1580. [ Medline ]
• Johnson KB, Wei WQ, Weeraratne D, Frisse ME, Misulis K, Rhee K, et al. Precision medicine, AI, and the future of personalized health care. Clin Transl Sci. Jan 2021;14(1):86-93. [ CrossRef ] [ Medline ]
• Sloane EJ, Silva RJ. Artificial intelligence in medical devices and clinical decision support systems. In: Clinical Engineering Handbook (Second Edition). Cambridge, MA. Academic Press; 2020.
• Steuten LM. Early stage health technology assessment for precision biomarkers in oral health and systems medicine. OMICS. Jan 2016;20(1):30-35. [ CrossRef ] [ Medline ]
• Javaid M, Haleem A, Pratap Singh R, Suman R, Rab S. Significance of machine learning in healthcare: features, pillars and applications. Int J Intell Networks. 2022;3:58-73. [ CrossRef ]
• Rowland SP, Fitzgerald JE, Holme T, Powell J, McGregor A. What is the clinical value of mHealth for patients? NPJ Digit Med. Jan 13, 2020;3:4. [ CrossRef ] [ Medline ]
• Voets MM, Veltman J, Slump CH, Siesling S, Koffijberg H. Systematic review of health economic evaluations focused on artificial intelligence in healthcare: the tortoise and the cheetah. Value Health. Mar 2022;25(3):340-349. [ CrossRef ] [ Medline ]
• Kirisits A, Redekop WK. The economic evaluation of medical devices: challenges ahead. Appl Health Econ Health Policy. Feb 2013;11(1):15-26. [ CrossRef ] [ Medline ]
• Park SH, Choi J, Byeon JS. Key principles of clinical validation, device approval, and insurance coverage decisions of artificial intelligence. Korean J Radiol. Mar 2021;22(3):442-453. [ CrossRef ] [ Medline ]
• Tsopra R, Fernandez X, Luchinat C, Alberghina L, Lehrach H, Vanoni M, et al. A framework for validating AI in precision medicine: considerations from the European ITFoC consortium. BMC Med Inform Decis Mak. Oct 02, 2021;21(1):274. [ CrossRef ] [ Medline ]
• Bolboacă SD. Medical diagnostic tests: a review of test anatomy, phases, and statistical treatment of data. Comput Math Methods Med. May 28, 2019;2019:1891569. [ CrossRef ] [ Medline ]
• Hogervorst MA, Pontén J, Vreman RA, Mantel-Teeuwisse AK, Goettsch WG. Real world data in health technology assessment of complex health technologies. Front Pharmacol. Feb 10, 2022;13:837302. [ CrossRef ] [ Medline ]
• Simon GE, Bindman AB, Dreyer NA, Platt R, Watanabe JH, Horberg M, et al. When can we trust real-world data to evaluate new medical treatments? Clin Pharmacol Ther. Jan 2022;111(1):24-29. [ CrossRef ] [ Medline ]
• Pongiglione B, Torbica A, Blommestein H, de Groot S, Ciani O, Walker S, et al. Do existing real-world data sources generate suitable evidence for the HTA of medical devices in Europe? Mapping and critical appraisal. Int J Technol Assess Health Care. Apr 26, 2021;37(1):e62. [ CrossRef ] [ Medline ]
• Pongiglione B, Torbica A. How real can we get in generating real world evidence? Exploring the opportunities of routinely collected administrative data for evaluation of medical devices. Health Econ. Sep 2022;31 Suppl 1(Suppl 1):25-43. [ CrossRef ] [ Medline ]
• Zemplényi A, Tachkov K, Balkanyi L, Németh B, Petykó ZI, Petrova G, et al. Recommendations to overcome barriers to the use of artificial intelligence-driven evidence in health technology assessment. Front Public Health. Apr 26, 2023;11:1088121. [ CrossRef ] [ Medline ]
• Daubner-Bendes R, Kovács S, Niewada M, Huic M, Drummond M, Ciani O, et al. Quo Vadis HTA for medical devices in central and eastern Europe? Recommendations to address methodological challenges. Front Public Health. Jan 08, 2021;8:612410. [ CrossRef ] [ Medline ]
• Larson DB, Harvey H, Rubin DL, Irani N, Tse JR, Langlotz CP. Regulatory frameworks for development and evaluation of artificial intelligence-based diagnostic imaging algorithms: summary and recommendations. J Am Coll Radiol. Mar 2021;18(3 Pt A):413-424. [ CrossRef ] [ Medline ]
• Choudhury A, Asan O. Role of artificial intelligence in patient safety outcomes: systematic literature review. JMIR Med Inform. Jul 24, 2020;8(7):e18599. [ CrossRef ] [ Medline ]
• Belenguer L. AI bias: exploring discriminatory algorithmic decision-making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry. AI Ethics. 2022;2(4):771-787. [ CrossRef ] [ Medline ]
• Binder L, Ghadban M, Sit C, Barnard K. Health technology assessment process for oncology drugs: impact of CADTH changes on public payer reimbursement recommendations. Curr Oncol. Mar 01, 2022;29(3):1514-1526. [ CrossRef ] [ Medline ]
• Wolff J, Pauling J, Keck A, Baumbach J. The economic impact of artificial intelligence in health care: systematic review. J Med Internet Res. Feb 20, 2020;22(2):e16866. [ CrossRef ] [ Medline ]
• Abràmoff MD, Roehrenbeck C, Trujillo S, Goldstein J, Graves AS, Repka MX, et al. A reimbursement framework for artificial intelligence in healthcare. NPJ Digit Med. Jun 09, 2022;5(1):72. [ CrossRef ] [ Medline ]
• Bélisle-Pipon JC, Couture V, Roy MC, Ganache I, Goetghebeur M, Cohen IG. What makes artificial intelligence exceptional in health technology assessment? Front Artif Intell. Nov 02, 2021;4:736697. [ CrossRef ] [ Medline ]
• Alami H, Lehoux P, Auclair Y, de Guise M, Gagnon MP, Shaw J, et al. Artificial intelligence and health technology assessment: anticipating a new level of complexity. J Med Internet Res. Jul 07, 2020;22(7):e17707. [ CrossRef ] [ Medline ]
• Love-Koh J, Peel A, Rejon-Parrilla JC, Ennis K, Lovett R, Manca A, et al. The future of precision medicine: potential impacts for health technology assessment. Pharmacoeconomics. Dec 2018;36(12):1439-1451. [ CrossRef ] [ Medline ]
• Gomez Rossi J, Rojas-Perilla N, Krois J, Schwendicke F. Cost-effectiveness of artificial intelligence as a decision-support system applied to the detection and grading of melanoma, dental caries, and diabetic retinopathy. JAMA Netw Open. Mar 01, 2022;5(3):e220269. [ CrossRef ] [ Medline ]
• Naik N, Hameed BM, Shetty DK, Swain D, Shah M, Paul R, et al. Legal and ethical consideration in artificial intelligence in healthcare: who takes responsibility? Front Surg. Mar 14, 2022;9:862322. [ CrossRef ] [ Medline ]
• Haynes CL, Cook GA, Jones MA. Legal and ethical considerations in processing patient-identifiable data without patient consent: lessons learnt from developing a disease register. J Med Ethics. May 2007;33(5):302-307. [ CrossRef ] [ Medline ]
• McCradden MD, Joshi S, Mazwi M, Anderson JA. Ethical limitations of algorithmic fairness solutions in health care machine learning. Lancet Digit Health. May 2020;2(5):e221-e223. [ CrossRef ] [ Medline ]
• Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern Med. Nov 01, 2018;178(11):1544-1547. [ CrossRef ] [ Medline ]
• Fletcher RR, Nakeshimana A, Olubeko O. Addressing fairness, bias, and appropriate use of artificial intelligence and machine learning in global health. Front Artif Intell. Apr 15, 2021;3:561802. [ CrossRef ] [ Medline ]
• Tachkov K, Zemplenyi A, Kamusheva M, Dimitrova M, Siirtola P, Pontén J, et al. Barriers to use artificial intelligence methodologies in health technology assessment in central and east European countries. Front Public Health. Jul 14, 2022;10:921226. [ CrossRef ] [ Medline ]
• Durán JM, Jongsma KR. Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J Med Ethics. Mar 18, 2021. (forthcoming). [ CrossRef ] [ Medline ]
• Amann J, Blasimme A, Vayena E, Frey D, Madai VI, Precise4Q consortium. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. Nov 30, 2020;20(1):310. [ CrossRef ] [ Medline ]
• Kiseleva A, Kotzinos D, De Hert P. Transparency of AI in healthcare as a multilayered system of accountabilities: between legal requirements and technical limitations. Front Artif Intell. May 30, 2022;5:879603. [ CrossRef ] [ Medline ]
• Baltaxe E, Hsieh HW, Roca J, Cano I. The assessment of medical device software supporting health care services for chronic patients in a tertiary hospital: overarching study. J Med Internet Res. Jan 04, 2023;25:e40976. [ CrossRef ] [ Medline ]
• Garfield S, Polisena J, S Spinner D, Postulka A, Y Lu C, Tiwana SK, et al. Health technology assessment for molecular diagnostics: practices, challenges, and recommendations from the medical devices and diagnostics special interest group. Value Health. 2016;19(5):577-587. [ CrossRef ] [ Medline ]
• Fraser AG, Biasin E, Bijnens B, Bruining N, Caiani EG, Cobbaert K, et al. Artificial intelligence in medical device software and high-risk medical devices - a review of definitions, expert recommendations and regulatory initiatives. Expert Rev Med Devices. Jun 2023;20(6):467-491. [ CrossRef ] [ Medline ]
• Beckers R, Kwade Z, Zanca F. The EU medical device regulation: implications for artificial intelligence-based medical device software in medical physics. Phys Med. Mar 2021;83:1-8. [ CrossRef ] [ Medline ]
• Melvin T, Torre M. New medical device regulations: the regulator's view. EFORT Open Rev. Jun 03, 2019;4(6):351-356. [ CrossRef ] [ Medline ]
• Fleetcroft C, McCulloch P, Campbell B. IDEAL as a guide to designing clinical device studies consistent with the new European Medical Device Regulation. BMJ Surg Interv Health Technol. Mar 04, 2021;3(1):e000066. [ CrossRef ] [ Medline ]
• Alami H, Lehoux P, Denis JL, Motulsky A, Petitgand C, Savoldelli M, et al. Organizational readiness for artificial intelligence in health care: insights for decision-making and practice. J Health Organ Manag. Dec 03, 2020. (forthcoming). [ CrossRef ] [ Medline ]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo RM. Methodological frameworks and dimensions to be taken into consideration in digital health technology assessment: protocol for a scoping review. JMIR Res Protoc. Oct 11, 2022;11(10):e39905. [ CrossRef ] [ Medline ]
• Weinert L, Müller J, Svensson L, Heinze O. Perspective of information technology decision makers on factors influencing adoption and implementation of artificial intelligence technologies in 40 German hospitals: descriptive analysis. JMIR Med Inform. Jun 15, 2022;10(6):e34678. [ CrossRef ] [ Medline ]
• de Hond AA, Leeuwenberg AM, Hooft L, Kant IM, Nijman SW, van Os HJ, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit Med. Jan 10, 2022;5(1):2. [ CrossRef ] [ Medline ]
• Ahuja AS. The impact of artificial intelligence in medicine on the future role of the physician. PeerJ. 2019;7:e7702. [ CrossRef ] [ Medline ]
• Widrig D, Tag B. HTA and its legal issues: a framework for identifying legal issues in health technology assessment. Int J Technol Assess Health Care. Dec 2014;30(6):587-594. [ CrossRef ] [ Medline ]
• Vella Bonanno P, Bucsics A, Simoens S, Martin AP, Oortwijn W, Gulbinovič J, et al. Proposal for a regulation on health technology assessment in Europe - opinions of policy makers, payers and academics from the field of HTA. Expert Rev Pharmacoecon Outcomes Res. Jun 2019;19(3):251-261. [ CrossRef ] [ Medline ]
• McKee M, Wouters OJ. The challenges of regulating artificial intelligence in healthcare comment on "clinical decision support and new regulatory frameworks for medical devices: are we ready for it? - a viewpoint paper". Int J Health Policy Manag. 2023;12:7261. [ CrossRef ] [ Medline ]
• Hordern V. Data protection compliance in the age of digital health. Eur J Health Law. Jun 2016;23(3):248-264. [ CrossRef ] [ Medline ]
• Stanberry B. The legal and ethical aspects of telemedicine. 2: data protection, security and European law. J Telemed Telecare. 1998;4(1):18-24. [ CrossRef ] [ Medline ]
• Poullet Y. Legal aspects of data protection in medical informatics. Stud Health Technol Inform. 1991;1:138-160. [ Medline ]
• Dove ES, Chen J. To what extent does the EU general data protection regulation (GDPR) apply to citizen scientist-led health research with mobile devices? J Law Med Ethics. Mar 2020;48(1_suppl):187-195. [ CrossRef ] [ Medline ]
• Marovic B, Curcin V. Impact of the European general data protection regulation (GDPR) on health data management in a European Union candidate country: a case study of Serbia. JMIR Med Inform. Apr 17, 2020;8(4):e14604. [ CrossRef ] [ Medline ]
• EU Artificial Intelligence Act. URL: https://artificialintelligenceact.eu/ [accessed 2023-07-24]
• Proposal for a regulation of the European parliament and of the council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts. European Union. 2021. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206 [accessed 2023-07-24]
• Camara C, Peris-Lopez P, Tapiador JE. Security and privacy issues in implantable medical devices: a comprehensive survey. J Biomed Inform. Jun 2015;55:272-289. [ CrossRef ] [ Medline ]
• Migliore A, Ratti M, Cerbo M, Jefferson T. Health Technology Assessment: managing the introduction and use of medical devices in clinical practice in Italy. Expert Rev Med Devices. May 2009;6(3):251-257. [ CrossRef ] [ Medline ]
• Pisapia A, Banfi G, Tomaiuolo R. The novelties of the regulation on health technology assessment, a key achievement for the European Union health policies. Clin Chem Lab Med. Jul 26, 2022;60(8):1160-1163. [ CrossRef ] [ Medline ]
• Maliha G, Gerke S, Cohen IG, Parikh RB. Artificial intelligence and liability in medicine: balancing safety and innovation. Milbank Q. Sep 2021;99(3):629-647. [ CrossRef ] [ Medline ]
• Jassar S, Adams SJ, Zarzeczny A, Burbridge BE. The future of artificial intelligence in medicine: medical-legal considerations for health leaders. Healthc Manage Forum. May 2022;35(3):185-189. [ CrossRef ] [ Medline ]
• Samore MH, Evans RS, Lassen A, Gould P, Lloyd J, Gardner RM, et al. Surveillance of medical device-related hazards and adverse events in hospitalized patients. JAMA. Jan 21, 2004;291(3):325-334. [ CrossRef ] [ Medline ]
• Bleher H, Braun M. Diffused responsibility: attributions of responsibility in the use of AI-driven clinical decision support systems. AI Ethics. 2022;2(4):747-761. [ CrossRef ] [ Medline ]
• Street J, Stafinski T, Lopes E, Menon D. Defining the role of the public in Health Technology Assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care. Apr 2020;36(2):87-95. [ CrossRef ] [ Medline ]
• Massella M, Dri DA, Gramaglia D. Regulatory considerations on the use of machine learning based tools in clinical trials. Health Technol (Berl). 2022;12(6):1085-1096. [ CrossRef ] [ Medline ]
• Marcus HJ, Payne CJ, Hughes-Hallett A, Marcus AP, Yang GZ, Darzi A, et al. Regulatory approval of new medical devices: cross sectional study. BMJ. May 20, 2016;353:i2587. [ CrossRef ] [ Medline ]
• Milam ME, Koo CW. The current status and future of FDA-approved artificial intelligence tools in chest radiology in the United States. Clin Radiol. Feb 2023;78(2):115-122. [ CrossRef ] [ Medline ]
• Vreman RA, Mantel-Teeuwisse AK, Hövels AM, Leufkens HG, Goettsch WG. Differences in health technology assessment recommendations among European jurisdictions: the role of practice variations. Value Health. Jan 2020;23(1):10-16. [ CrossRef ] [ Medline ]
• Allen N, Walker SR, Liberti L, Salek S. Health technology assessment (HTA) case studies: factors influencing divergent HTA reimbursement recommendations in Australia, Canada, England, and Scotland. Value Health. Mar 2017;20(3):320-328. [ CrossRef ] [ Medline ]
• Agboola F, Wright AC, Herron-Smith S, Mathur D, Rind D. Evaluation of diversity of clinical trials informing health technology assessments in the United States: a 5-year analysis of institute for clinical and economic review assessments. Value Health. Sep 2023;26(9):1345-1352. [ CrossRef ] [ Medline ]
• Hendrix N, Veenstra DL, Cheng M, Anderson NC, Verguet S. Assessing the economic value of clinical artificial intelligence: challenges and opportunities. Value Health. Mar 2022;25(3):331-339. [ CrossRef ] [ Medline ]
• Nielsen CP, Lauritsen SW, Kristensen FB, Bistrup ML, Cecchetti A, Turk E. Involving stakeholders and developing a policy for stakeholder involvement in the European network for health technology assessment, EUnetHTA. Int J Technol Assess Health Care. Dec 2009;25 Suppl 2:84-91. [ CrossRef ] [ Medline ]
• Scott AM, Wale JL, HTAi PatientCitizen Involvement in HTA Interest Group‚ Patient InvolvementEducation Working Group. Patient advocate perspectives on involvement in HTA: an international snapshot. Res Involv Engagem. Jan 10, 2017;3:2. [ CrossRef ] [ Medline ]
• Gagnon MP, Desmartis M, Gagnon J, St-Pierre M, Gauvin FP, Rhainds M, et al. Introducing the patient's perspective in hospital health technology assessment (HTA): the views of HTA producers, hospital managers and patients. Health Expect. Dec 2014;17(6):888-900. [ CrossRef ] [ Medline ]
• Gagnon M, Desmartis M, Gagnon J, St-Pierre M, Rhainds M, Coulombe M, et al. Framework for user involvement in health technology assessment at the local level: views of health managers, user representatives, and clinicians. Int J Technol Assess Health Care. Jan 2015;31(1-2):68-77. [ CrossRef ] [ Medline ]
• Angelis A, Kanavos P. Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment. Pharmacoeconomics. May 2016;34(5):435-446. [ CrossRef ] [ Medline ]
• Wale JL, Thomas S, Hamerlijnck D, Hollander R. Patients and public are important stakeholders in health technology assessment but the level of involvement is low - a call to action. Res Involv Engagem. Jan 05, 2021;7(1):1. [ CrossRef ] [ Medline ]
• Artificial intelligence and machine learning (AI/ML)-enabled medical devices. U.S. Food & Drug Administration. Oct 19, 2023. URL: https://tinyurl.com/m8ba5aw4 [accessed 2023-11-25]
• Cassidy B, Hoon Yap M, Pappachan JM, Ahmad N, Haycocks S, O'Shea C, et al. Artificial intelligence for automated detection of diabetic foot ulcers: a real-world proof-of-concept clinical evaluation. Diabetes Res Clin Pract. Nov 2023;205:110951. [ CrossRef ] [ Medline ]
• Mackenzie SC, Sainsbury CA, Wake DJ. Diabetes and artificial intelligence beyond the closed loop: a review of the landscape, promise and challenges. Diabetologia. Feb 2024;67(2):223-235. [ CrossRef ] [ Medline ]
• Sridhar GR, Lakshmi G. Ethical issues of artificial intelligence in diabetes mellitus. Med Res Arch. Aug 2023;11(8):1-8. [ CrossRef ]
• Zhang L, Zhang L. Artificial intelligence for remote sensing data analysis: a review of challenges and opportunities. IEEE Geosci Remote Sens Mag. Apr 13, 2022;10(2):270-294. [ FREE Full text ] [ CrossRef ]
• Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf. Mar 2019;28(3):231-237. [ CrossRef ] [ Medline ]
• Beil M, Proft I, van Heerden D, Sviri S, van Heerden PV. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med Exp. Dec 10, 2019;7(1):70. [ CrossRef ] [ Medline ]
• Bærøe K, Miyata-Sturm A, Henden E. How to achieve trustworthy artificial intelligence for health. Bull World Health Organ. Apr 01, 2020;98(4):257-262. [ CrossRef ] [ Medline ]
• Julian E, Gianfrate F, Sola-Morales O, Mol P, Bergmann JF, Salmonson T, et al. How can a joint European health technology assessment provide an 'additional benefit' over the current standard of national assessments? : insights generated from a multi-stakeholder survey in hematology/oncology. Health Econ Rev. Jun 02, 2022;12(1):30. [ CrossRef ] [ Medline ]

Abbreviations

Edited by Y Zhuang; submitted 02.08.23; peer-reviewed by A de Hond, E Vashishtha, I Cano, N Hendrix; comments to author 28.10.23; revised version received 17.12.23; accepted 28.12.23; published 13.05.24.

©Line Farah, Isabelle Borget, Nicolas Martelli, Alexandre Vallee. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.