example of a research plan paper

How to write a research plan: Step-by-step guide

Last updated

30 January 2024

Reviewed by

Short on time? Get an AI generated summary of this article instead

Today’s businesses and institutions rely on data and analytics to inform their product and service decisions. These metrics influence how organizations stay competitive and inspire innovation. However, gathering data and insights requires carefully constructed research, and every research project needs a roadmap. This is where a research plan comes into play.

Read this step-by-step guide for writing a detailed research plan that can apply to any project, whether it’s scientific, educational, or business-related.

• What is a research plan?

A research plan is a documented overview of a project in its entirety, from end to end. It details the research efforts, participants, and methods needed, along with any anticipated results. It also outlines the project’s goals and mission, creating layers of steps to achieve those goals within a specified timeline.

Without a research plan, you and your team are flying blind, potentially wasting time and resources to pursue research without structured guidance.

The principal investigator, or PI, is responsible for facilitating the research oversight. They will create the research plan and inform team members and stakeholders of every detail relating to the project. The PI will also use the research plan to inform decision-making throughout the project.

• Why do you need a research plan?

Create a research plan before starting any official research to maximize every effort in pursuing and collecting the research data. Crucially, the plan will model the activities needed at each phase of the research project .

Like any roadmap, a research plan serves as a valuable tool providing direction for those involved in the project—both internally and externally. It will keep you and your immediate team organized and task-focused while also providing necessary definitions and timelines so you can execute your project initiatives with full understanding and transparency.

External stakeholders appreciate a working research plan because it’s a great communication tool, documenting progress and changing dynamics as they arise. Any participants of your planned research sessions will be informed about the purpose of your study, while the exercises will be based on the key messaging outlined in the official plan.

Here are some of the benefits of creating a research plan document for every project:

Project organization and structure

Well-informed participants

All stakeholders and teams align in support of the project

Clearly defined project definitions and purposes

Distractions are eliminated, prioritizing task focus

Timely management of individual task schedules and roles

Costly reworks are avoided

• What should a research plan include?

The different aspects of your research plan will depend on the nature of the project. However, most official research plan documents will include the core elements below. Each aims to define the problem statement , devising an official plan for seeking a solution.

Specific project goals and individual objectives

Ideal strategies or methods for reaching those goals

Required resources

Descriptions of the target audience, sample sizes , demographics, and scopes

Key performance indicators (KPIs)

Project background

Research and testing support

Preliminary studies and progress reporting mechanisms

Cost estimates and change order processes

Depending on the research project’s size and scope, your research plan could be brief—perhaps only a few pages of documented plans. Alternatively, it could be a fully comprehensive report. Either way, it’s an essential first step in dictating your project’s facilitation in the most efficient and effective way.

• How to write a research plan for your project

When you start writing your research plan, aim to be detailed about each step, requirement, and idea. The more time you spend curating your research plan, the more precise your research execution efforts will be.

Account for every potential scenario, and be sure to address each and every aspect of the research.

Consider following this flow to develop a great research plan for your project:

Define your project’s purpose

Start by defining your project’s purpose. Identify what your project aims to accomplish and what you are researching. Remember to use clear language.

Thinking about the project’s purpose will help you set realistic goals and inform how you divide tasks and assign responsibilities. These individual tasks will be your stepping stones to reach your overarching goal.

Additionally, you’ll want to identify the specific problem, the usability metrics needed, and the intended solutions.

Know the following three things about your project’s purpose before you outline anything else:

What you’re doing

Why you’re doing it

What you expect from it

Identify individual objectives

With your overarching project objectives in place, you can identify any individual goals or steps needed to reach those objectives. Break them down into phases or steps. You can work backward from the project goal and identify every process required to facilitate it.

Be mindful to identify each unique task so that you can assign responsibilities to various team members. At this point in your research plan development, you’ll also want to assign priority to those smaller, more manageable steps and phases that require more immediate or dedicated attention.

Select research methods

Once you have outlined your goals, objectives, steps, and tasks, it’s time to drill down on selecting research methods . You’ll want to leverage specific research strategies and processes. When you know what methods will help you reach your goals, you and your teams will have direction to perform and execute your assigned tasks.

Research methods might include any of the following:

User interviews : this is a qualitative research method where researchers engage with participants in one-on-one or group conversations. The aim is to gather insights into their experiences, preferences, and opinions to uncover patterns, trends, and data.

Field studies : this approach allows for a contextual understanding of behaviors, interactions, and processes in real-world settings. It involves the researcher immersing themselves in the field, conducting observations, interviews, or experiments to gather in-depth insights.

Card sorting : participants categorize information by sorting content cards into groups based on their perceived similarities. You might use this process to gain insights into participants’ mental models and preferences when navigating or organizing information on websites, apps, or other systems.

Focus groups : use organized discussions among select groups of participants to provide relevant views and experiences about a particular topic.

Diary studies : ask participants to record their experiences, thoughts, and activities in a diary over a specified period. This method provides a deeper understanding of user experiences, uncovers patterns, and identifies areas for improvement.

Five-second testing: participants are shown a design, such as a web page or interface, for just five seconds. They then answer questions about their initial impressions and recall, allowing you to evaluate the design’s effectiveness.

Surveys : get feedback from participant groups with structured surveys. You can use online forms, telephone interviews, or paper questionnaires to reveal trends, patterns, and correlations.

Tree testing : tree testing involves researching web assets through the lens of findability and navigability. Participants are given a textual representation of the site’s hierarchy (the “tree”) and asked to locate specific information or complete tasks by selecting paths.

Usability testing : ask participants to interact with a product, website, or application to evaluate its ease of use. This method enables you to uncover areas for improvement in digital key feature functionality by observing participants using the product.

Live website testing: research and collect analytics that outlines the design, usability, and performance efficiencies of a website in real time.

There are no limits to the number of research methods you could use within your project. Just make sure your research methods help you determine the following:

What do you plan to do with the research findings?

What decisions will this research inform? How can your stakeholders leverage the research data and results?

Recruit participants and allocate tasks

Next, identify the participants needed to complete the research and the resources required to complete the tasks. Different people will be proficient at different tasks, and having a task allocation plan will allow everything to run smoothly.

Prepare a thorough project summary

Every well-designed research plan will feature a project summary. This official summary will guide your research alongside its communications or messaging. You’ll use the summary while recruiting participants and during stakeholder meetings. It can also be useful when conducting field studies.

Ensure this summary includes all the elements of your research project . Separate the steps into an easily explainable piece of text that includes the following:

An introduction: the message you’ll deliver to participants about the interview, pre-planned questioning, and testing tasks.

Interview questions: prepare questions you intend to ask participants as part of your research study, guiding the sessions from start to finish.

An exit message: draft messaging your teams will use to conclude testing or survey sessions. These should include the next steps and express gratitude for the participant’s time.

Create a realistic timeline

While your project might already have a deadline or a results timeline in place, you’ll need to consider the time needed to execute it effectively.

Realistically outline the time needed to properly execute each supporting phase of research and implementation. And, as you evaluate the necessary schedules, be sure to include additional time for achieving each milestone in case any changes or unexpected delays arise.

For this part of your research plan, you might find it helpful to create visuals to ensure your research team and stakeholders fully understand the information.

Determine how to present your results

A research plan must also describe how you intend to present your results. Depending on the nature of your project and its goals, you might dedicate one team member (the PI) or assume responsibility for communicating the findings yourself.

In this part of the research plan, you’ll articulate how you’ll share the results. Detail any materials you’ll use, such as:

Presentations and slides

A project report booklet

A project findings pamphlet

Documents with key takeaways and statistics

Graphic visuals to support your findings

• Format your research plan

As you create your research plan, you can enjoy a little creative freedom. A plan can assume many forms, so format it how you see fit. Determine the best layout based on your specific project, intended communications, and the preferences of your teams and stakeholders.

Find format inspiration among the following layouts:

Written outlines

Narrative storytelling

Visual mapping

Graphic timelines

Remember, the research plan format you choose will be subject to change and adaptation as your research and findings unfold. However, your final format should ideally outline questions, problems, opportunities, and expectations.

• Research plan example

Imagine you’ve been tasked with finding out how to get more customers to order takeout from an online food delivery platform. The goal is to improve satisfaction and retain existing customers. You set out to discover why more people aren’t ordering and what it is they do want to order or experience. 

You identify the need for a research project that helps you understand what drives customer loyalty . But before you jump in and start calling past customers, you need to develop a research plan—the roadmap that provides focus, clarity, and realistic details to the project.

Here’s an example outline of a research plan you might put together:

Project title

Project members involved in the research plan

Purpose of the project (provide a summary of the research plan’s intent)

Objective 1 (provide a short description for each objective)

Objective 2

Objective 3

Proposed timeline

Audience (detail the group you want to research, such as customers or non-customers)

Budget (how much you think it might cost to do the research)

Risk factors/contingencies (any potential risk factors that may impact the project’s success)

Remember, your research plan doesn’t have to reinvent the wheel—it just needs to fit your project’s unique needs and aims.

Customizing a research plan template

Some companies offer research plan templates to help get you started. However, it may make more sense to develop your own customized plan template. Be sure to include the core elements of a great research plan with your template layout, including the following:

Introductions to participants and stakeholders

Background problems and needs statement

Significance, ethics, and purpose

Research methods, questions, and designs

Preliminary beliefs and expectations

Implications and intended outcomes

Realistic timelines for each phase

Conclusion and presentations

How many pages should a research plan be?

Generally, a research plan can vary in length between 500 to 1,500 words. This is roughly three pages of content. More substantial projects will be 2,000 to 3,500 words, taking up four to seven pages of planning documents.

What is the difference between a research plan and a research proposal?

A research plan is a roadmap to success for research teams. A research proposal, on the other hand, is a dissertation aimed at convincing or earning the support of others. Both are relevant in creating a guide to follow to complete a project goal.

What are the seven steps to developing a research plan?

While each research project is different, it’s best to follow these seven general steps to create your research plan:

Defining the problem

Identifying goals

Choosing research methods

Recruiting participants

Preparing the brief or summary

Establishing task timelines

Defining how you will present the findings

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 6 February 2023

Last updated: 6 October 2023

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 7 March 2023

Last updated: 9 March 2023

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

Log in or sign up

Get started for free

We use cookies to give you the best experience possible. By continuing we’ll assume you’re on board with our cookie policy

• A Research Guide
• Research Paper Guide

How to Write a Research Plan

• Research plan definition
• Purpose of a research plan
• Research plan structure
• Step-by-step writing guide

Tips for creating a research plan

• Research plan examples

Research plan: definition and significance

What is the purpose of a research plan.

• Bridging gaps in the existing knowledge related to their subject.
• Reinforcing established research about their subject.
• Introducing insights that contribute to subject understanding.

Research plan structure & template

Introduction.

• What is the existing knowledge about the subject?
• What gaps remain unanswered?
• How will your research enrich understanding, practice, and policy?

Literature review

Expected results.

• Express how your research can challenge established theories in your field.
• Highlight how your work lays the groundwork for future research endeavors.
• Emphasize how your work can potentially address real-world problems.

5 Steps to crafting an effective research plan

Step 1: define the project purpose, step 2: select the research method, step 3: manage the task and timeline, step 4: write a summary, step 5: plan the result presentation.

• Brainstorm Collaboratively: Initiate a collective brainstorming session with peers or experts. Outline the essential questions that warrant exploration and answers within your research.
• Prioritize and Feasibility: Evaluate the list of questions and prioritize those that are achievable and important. Focus on questions that can realistically be addressed.
• Define Key Terminology: Define technical terms pertinent to your research, fostering a shared understanding. Ensure that terms like “church” or “unreached people group” are well-defined to prevent ambiguity.
• Organize your approach: Once well-acquainted with your institution’s regulations, organize each aspect of your research by these guidelines. Allocate appropriate word counts for different sections and components of your research paper.

Research plan example

• Writing a Research Paper
• Research Paper Title
• Research Paper Sources
• Research Paper Problem Statement
• Research Paper Thesis Statement
• Hypothesis for a Research Paper
• Research Question
• Research Paper Outline
• Research Paper Summary
• Research Paper Prospectus
• Research Paper Proposal
• Research Paper Format
• Research Paper Styles
• AMA Style Research Paper
• MLA Style Research Paper
• Chicago Style Research Paper
• APA Style Research Paper
• Research Paper Structure
• Research Paper Cover Page
• Research Paper Abstract
• Research Paper Introduction
• Research Paper Body Paragraph
• Research Paper Literature Review
• Research Paper Background
• Research Paper Methods Section
• Research Paper Results Section
• Research Paper Discussion Section
• Research Paper Conclusion
• Research Paper Appendix
• Research Paper Bibliography
• APA Reference Page
• Annotated Bibliography
• Bibliography vs Works Cited vs References Page
• Research Paper Types
• What is Qualitative Research

Receive paper in 3 Hours!

• Choose the number of pages.
• Select your deadline.
• Complete your order.

Number of Pages

550 words (double spaced)

Deadline: 10 days left

By clicking "Log In", you agree to our terms of service and privacy policy . We'll occasionally send you account related and promo emails.

Sign Up for your FREE account

How to Write a Research Plan

Your answers to these questions form your research strategy. Most likely, you’ve addressed some of these issues in your proposal. But you are further along now, and you can flesh out your answers. With your instructor’s help, you should make some basic decisions about what information to collect and what methods to use in analyzing it. You will probably develop this research strategy gradually and, if you are like the rest of us, you will make some changes, large and small, along the way. Still, it is useful to devise a general plan early, even though you will modify it as you progress. Develop a tentative research plan early in the project. Write it down and share it with your instructor. The more concrete and detailed the plan, the better the feedback you’ll get.

Academic Writing, Editing, Proofreading, And Problem Solving Services

Get 10% off with 24start discount code.

This research plan does not need to be elaborate or time-consuming. Like your working bibliography, it is provisional, a work in progress. Still, it is helpful to write it down since it will clarify a number of issues for you and your professor.

Writing a Research Plan

To write out your research plan, begin by restating your main thesis question and any secondary ones. They may have changed a bit since your original proposal. If these questions bear on a particular theory or analytic perspective, state that briefly. In the social sciences, for example, two or three prominent theories might offer different predictions about your subject. If so, then you might want to explore these differences in your thesis and explain why some theories work better (or worse) in this particular case. Likewise, in the humanities, you might consider how different theories offer different insights and contrasting perspectives on the particular novel or film you are studying. If you intend to explore these differences, state your goal clearly in the research plan so you can discuss it later with your professor. Next, turn to the heart of this exercise, your proposed research strategy. Try to explain your basic approach, the materials you will use, and your method of analysis. You may not know all of these elements yet, but do the best you can. Briefly say how and why you think they will help answer your main questions.

Be concrete. What data will you collect? Which poems will you read? Which paintings will you compare? Which historical cases will you examine? If you plan to use case studies, say whether you have already selected them or settled on the criteria for choosing them. Have you decided which documents and secondary sources are most important? Do you have easy access to the data, documents, or other materials you need? Are they reliable sources—the best information you can get on the subject? Give the answers if you have them, or say plainly that you don’t know so your instructor can help. You should also discuss whether your research requires any special skills and, of course, whether you have them. You can—and should—tailor your work to fit your skills.

If you expect to challenge other approaches—an important element of some theses—which ones will you take on, and why? This last point can be put another way: Your project will be informed by some theoretical traditions and research perspectives and not others. Your research will be stronger if you clarify your own perspective and show how it usefully informs your work. Later, you may also enter the jousts and explain why your approach is superior to the alternatives, in this particular study and perhaps more generally. Your research plan should state these issues clearly so you can discuss them candidly and think them through.

If you plan to conduct tests, experiments, or surveys, discuss them, too. They are common research tools in many fields, from psychology and education to public health. Now is the time to spell out the details—the ones you have nailed down tight and the ones that are still rattling around, unresolved. It’s important to bring up the right questions here, even if you don’t have all the answers yet. Raising these questions directly is the best way to get the answers. What kinds of tests or experiments do you plan, and how will you measure the results? How will you recruit your test subjects, and how many will be included in your sample? What test instruments or observational techniques will you use? How reliable and valid are they? Your instructor can be a great source of feedback here.

Your research plan should say:

• What materials you will use
• What methods you will use to investigate them
• Whether your work follow a particular approach or theory

There are also ethical issues to consider. They crop up in any research involving humans or animals. You need to think carefully about them, underscore potential problems, and discuss them with your professor. You also need to clear this research in advance with the appropriate authorities at your school, such as the committee that reviews proposals for research on human subjects.

Not all these issues and questions will bear on your particular project. But some do, and you should wrestle with them as you begin research. Even if your answers are tentative, you will still gain from writing them down and sharing them with your instructor. That’s how you will get the most comprehensive advice, the most pointed recommendations. If some of these issues puzzle you, or if you have already encountered some obstacles, share them, too, so you can either resolve the problems or find ways to work around them.

Remember, your research plan is simply a working product, designed to guide your ongoing inquiry. It’s not a final paper for a grade; it’s a step toward your final paper. Your goal in sketching it out now is to understand these issues better and get feedback from faculty early in the project. It may be a pain to write it out, but it’s a minor sting compared to major surgery later.

Checklist for Conducting Research

• Familiarize yourself with major questions and debates about your topic.
• Is appropriate to your topic;
• Addresses the main questions you propose in your thesis;
• Relies on materials to which you have access;
• Can be accomplished within the time available;
• Uses skills you have or can acquire.
• Divide your topic into smaller projects and do research on each in turn.
• Write informally as you do research; do not postpone this prewriting until all your research is complete.

Back to How To Write A Research Paper .

ORDER HIGH QUALITY CUSTOM PAPER

FLEET LIBRARY | Research Guides

Rhode island school of design, create a research plan: research plan.

• Research Plan
• Literature Review
• Ulrich's Global Serials Directory
• Related Guides

A research plan is a framework that shows how you intend to approach your topic. The plan can take many forms: a written outline, a narrative, a visual/concept map or timeline. It's a document that will change and develop as you conduct your research. Components of a research plan

1. Research conceptualization - introduces your research question

2. Research methodology - describes your approach to the research question

3. Literature review, critical evaluation and synthesis - systematic approach to locating,

    reviewing and evaluating the work (text, exhibitions, critiques, etc) relating to your topic

4. Communication - geared toward an intended audience, shows evidence of your inquiry

Research conceptualization refers to the ability to identify specific research questions, problems or opportunities that are worthy of inquiry. Research conceptualization also includes the skills and discipline that go beyond the initial moment of conception, and which enable the researcher to formulate and develop an idea into something researchable ( Newbury 373).

Research methodology refers to the knowledge and skills required to select and apply appropriate methods to carry through the research project ( Newbury 374) .

Method describes a single mode of proceeding; methodology describes the overall process.

Method - a way of doing anything especially according to a defined and regular plan; a mode of procedure in any activity

Methodology - the study of the direction and implications of empirical research, or the sustainability of techniques employed in it; a method or body of methods used in a particular field of study or activity *Browse a list of research methodology books  or this guide on Art & Design Research

Literature Review, critical evaluation & synthesis

A literature review is a systematic approach to locating, reviewing, and evaluating the published work and work in progress of scholars, researchers, and practitioners on a given topic.

Critical evaluation and synthesis is the ability to handle (or process) existing sources. It includes knowledge of the sources of literature and contextual research field within which the person is working ( Newbury 373).

Literature reviews are done for many reasons and situations. Here's a short list:

Sources to consult while conducting a literature review:

Online catalogs of local, regional, national, and special libraries

meta-catalogs such as worldcat , Art Discovery Group , europeana , world digital library or RIBA

subject-specific online article databases (such as the Avery Index, JSTOR, Project Muse)

digital institutional repositories such as Digital Commons @RISD ; see Registry of Open Access Repositories

Open Access Resources recommended by RISD Research LIbrarians

works cited in scholarly books and articles

print bibliographies

the internet-locate major nonprofit, research institutes, museum, university, and government websites

search google scholar to locate grey literature & referenced citations

trade and scholarly publishers

fellow scholars and peers

Communication                              

Communication refers to the ability to

• structure a coherent line of inquiry
• communicate your findings to your intended audience
• make skilled use of visual material to express ideas for presentations, writing, and the creation of exhibitions ( Newbury 374)

Research plan framework: Newbury, Darren. "Research Training in the Creative Arts and Design." The Routledge Companion to Research in the Arts . Ed. Michael Biggs and Henrik Karlsson. New York: Routledge, 2010. 368-87. Print.

About the author

Except where otherwise noted, this guide is subject to a Creative Commons Attribution license

source document

  Routledge Companion to Research in the Arts

• Next: Literature Review >>
• Last Updated: Sep 20, 2023 5:05 PM
• URL: https://risd.libguides.com/researchplan

• Design for Business
• Most Recent
• Presentations
• Infographics
• Data Visualizations
• Forms and Surveys
• Video & Animation
• Case Studies
• Digital Marketing
• Design Inspiration
• Visual Thinking
• Product Updates
• Visme Webinars
• Artificial Intelligence

8 Research Proposal Examples & Template to Use

Written by: Raja Mandal

So you have a groundbreaking research idea you've spent months or even years developing, and now you're ready to take the next step.

How do you get funding for your research, and how should you approach potential funders? The answer is to create a convincing research proposal.

Unfortunately, most research proposals often get rejected. According to the European Research Council, the success rate for repeat proposal applications was only 14.8% in 2023 .

Pitching a novel research concept isn’t enough. To increase your chances of securing funding, your research proposal must check the right boxes in terms of clarity, feasibility, aesthetic appeal and other factors.

If you’re looking for inspiration to create a persuasive and feasible proposal, you’re in the right place. In this article, we have compiled a list of research proposal examples to help you create yours.

These examples will help you understand how to organize your proposal, what information to include and how to present it in a way that encourages others to support your project.

Let's dive in!

Table of Contents

What is a research proposal, what to include in a research proposal, 8 research proposal examples & templates, research proposal faqs.

• A research proposal is a document that outlines your proposed research project, explaining what you plan to study, why it's important and how you will conduct your research.
• A well-structured research proposal includes a title page, abstract and table of contents, introduction, literature review, research design and methodology, contribution to knowledge, research schedule, timeline and budget.
• Visme's research proposal examples and templates offer a great starting point for creating engaging and well-structured proposals.
• Choose a template from Visme's research proposal examples and customize it to fit your needs.
• With Visme’s proposal maker , you can create a research proposal that stands out. Access a drag-and-drop editor and advanced features like AI tools , collaboration features, brand wizard and more.

A research proposal is a structured document that outlines the core idea of your research, the methods you intend to use, the required resources and the expected results.

Think of it as a sales pitch for your research. It answers some big questions: What are you planning to explore? Why is it important to conduct the research? What are your research objectives and the methods you’ll use to achieve them? What are the potential outcomes or contributions of this research to the field?

A research proposal serves two primary purposes. First, it convinces funding bodies or academic committees to support your research project expected to bring new ideas and insights. Second, it provides a roadmap for your research journey, helping you stay focused, organized and on track.

Now, we'll discuss what to include in a research proposal. You'll learn about the important parts of a research proposal template and how they help present your research idea clearly.

Here’s an infographic that you can use to understand the elements of a research proposal quickly.

1. Title Page

Start your research proposal with a title page that clearly states your research. The title page is like a book cover, giving the first impression of your project. Therefore, you must ensure the design is engaging enough to attract your audience at first glance.

Include the following details on your title page:

• Title of your research
• Contact Details
• Name of the department or organization
• Date of submission

2. Abstract and Table of Contents

After the title page comes the abstract and the table of contents.

The abstract is a concise summary of your project that briefly outlines your research question, the reasons behind the study and the methods you intend to use. It is a quick way for readers to understand your proposal without reading the entire document.

The table of contents is a detailed list of the sections and subsections in your proposal, with page numbers. It helps readers navigate through your document and quickly locate different parts they're interested in.

3. Introduction

The introduction of your research proposal sets the tone for the rest of the document. It should grab the reader's attention and make them want to learn more. It's your chance to make a strong case for why your research is worth investigating and how it can fill a gap in current knowledge or solve a specific problem.

Make sure that your introduction covers the following:

• Background Information: Set the stage with a brief snapshot of existing research and why your topic is relevant.
• Research Problem: Identify the specific problem or knowledge gap that your study will address.
• Research Questions or Hypotheses: Present the central question or hypothesis that guides your research focus.
• Aims and Objectives: Outline your research's main goal and the steps you'll take to achieve it.
• Significance and Contribution: Explain how your research will add value to the field and what impact it could have.

4. Literature Review

A literature review is a list of the scholarly works you used to conduct your research. It helps you demonstrate your current knowledge about the topic.

Here's how this part works:

• Summary of Sources: Talk about the main ideas or findings from your research materials and explain how they connect to your research questions.
• Finding Gaps: Show where the current research falls short or doesn't give the full picture—this is where your research comes in!
• Key Theories: Tell the readers about any theories or ways of thinking that help shape your research.
• Learning from Methods: Discuss what previous researchers worked on and how their methods might guide your research.
• Recognizing Authors and Studies: Honor the pioneers whose work has had a major influence on your topic.

5. Research Design and Methodology

This section outlines your plan for answering your research question. It explains how you intend to gather and analyze information, providing a clear roadmap of the investigation process.

Here are the key components:

Population and Sample

Describe the entire group you're interested in (the population). This could be all teachers in a specific state or all social media platform users. After that, you will need to explain how you will choose a smaller group, known as a sample, to study directly. This sample should be selected to accurately represent the larger population you are interested in studying.

To choose the right sampling method, you need to assess your population properly. For instance, to obtain general insights, you can use random sampling to select individuals without bias. If the population consists of different categories, such as professionals and students, you can use stratified sampling to ensure that each category is represented in the sample.

Other popular sampling methods include systematic, convenience, purposive, cluster, and probability sampling techniques.

Research Approach

There are three main approaches for the research: qualitative (focusing on experiences and themes), quantitative (using numbers and statistics), or mixed methods (combining both). Your choice will depend on your research question and the kind of data you need.

Data Collection

This section details the specific methods you'll use to gather information. Will you distribute surveys online or in person? Conduct interviews? Perhaps you'll use existing data sets. Here, you'll also explain how you'll ensure the data collection process is reliable and ethical.

Data Analysis

Once you have collected your data, the next step is to analyze it to obtain meaningful insights. The method you choose depends on the available data type.

If you have quantitative data, you can employ statistical tests to analyze it. And if you're dealing with qualitative data, coding techniques can help you spot patterns and themes in your collected data.

6. Contribution to Knowledge

In this section, you need to explain how your research will contribute to the existing knowledge in your field. You should describe whether your study will fill a knowledge gap, challenge conventional ideas or beliefs or offer a fresh perspective on a topic.

Clearly outline how your work will advance your field of study and why this new knowledge is essential.

7. Research Schedule and Timeline

Create a timeline with important milestones, such as finishing your literature review, completing data collection and finalizing your analysis.

This shows that you've carefully considered the scope of your project and can manage your time effectively. Furthermore, account for possible delays and be prepared to adapt your schedule accordingly.

To create this timeline, consider using a visual tool like a Gantt chart or a simple spreadsheet. These tools will help you organize individual tasks, assign deadlines, and visualize the project's overall progress.

Choose a Gantt chart template from Visme's library and customize it to create your timeline quickly. Here's an example template:

The budget section is your opportunity to show them that you've carefully considered all necessary expenses and that your funding request is justified.

Here's how you can approach this part:

• Understand the Rules: Before making calculations, thoroughly review the funding agency's guidelines. Pay attention to what types of expenses are allowed or excluded and whether there are any budget caps.
• Personnel: Salaries and benefits for yourself, research assistants, or collaborators.
• Equipment: Specialized tools, software, or lab supplies.
• Travel: Transportation, lodging and meals if data collection requires travel.
• Dissemination: Costs for publishing results or presenting at conferences.
• Provide Justifications: Don't just list a cost. Briefly explain why each expense is crucial for completing your research.
• Be Thorough and Realistic: Research prices for specific items using quotes or online comparisons. Don't underestimate expenses, as this can raise troubles about the project's feasibility.
• Don't Forget Contingencies: Include a small buffer (around 5% of your total budget) for unexpected costs that might arise.

Using these research proposal examples and templates, you can create a winning proposal in no time. You will find templates for various topics and customize every aspect of them to make them your own.

Visme’s drag-and-drop editor, advanced features and a vast library of templates help organizations and individuals worldwide create engaging documents.

Here’s what a research student who uses Visme to create award-winning presentations has to say about the tool:

Chantelle Clarke

Research Student

Now, let’s dive into the research proposal examples.

1. Research Proposal Presentation Template

This research proposal presentation template is a powerful tool for presenting your research plan to stakeholders. The slides include specific sections to help you outline your research, including the research background, questions, objectives, methodology and expected results.

The slides create a coherent narrative, highlighting the importance and significance of your research. Overall, the template has a calming and professional blue color scheme with text that enables your audience to grasp the key points.

If you need help creating your presentation slides in a fraction of the time, check out Visme's AI presentation maker . Enter your requirements using text prompts, and the AI tool will generate a complete presentation with engaging visuals, text and clear structure. You can further customize the template completely to your needs.

2. Sales Research Proposal Template

Sales research gives you a deeper understanding of their target audience. It also helps you identify gaps in the market and develop effective sales strategies that drive revenue growth. With this research proposal template, you can secure funding for your next research project.

It features a sleek and professional grayscale color palette with a classic and modern vibe. The high-quality images in the template are strategically placed to reinforce the message without overwhelming the reader. Furthermore, the template includes a vertical bar graph that effectively represents budget allocations, enabling the reader to quickly grasp the information.

Use Visme's interactive elements and animations to add a dynamic layer to your research proposals. You can animate any object and add pop-ups or link pages for a more immersive experience. Use these functionalities to highlight key findings, demonstrate trends or guide readers through your proposal, making the content engaging and interactive.

3. General Funding Research Proposal Template

This proposal template is a great tool for securing funding for any type of research project. It begins with a captivating title page that grabs attention. The beautiful design elements and vector icons enhance the aesthetic and aid visual communication.

This template revolves around how a specific user group adopts cryptocurrencies like Bitcoin and Ethereum. The goal is to assess awareness, gauge interest and understand key factors affecting cryptocurrency adoption.

The project methodology includes survey design, data collection, and market research. The expected impact is to enhance customer engagement and position the company as a customer-centric brand.

Do you need additional help crafting the perfect text for your proposal? Visme's AI writer can quickly generate content outlines, summaries and even entire sections. Just explain your requirements to the tool using a text prompt, and the tool will generate it for you.

4. Product Research Proposal Template

Creating a product that delights users begins with detailed product research. With this modern proposal template, you can secure buy-in and funding for your next research.

It starts with a background that explains why the research is important. Next, it highlights what the research is set to achieve, how the research will be conducted, how much it will cost, the timeline and the expected outcomes. With a striking color scheme combining black, yellow, and gray, the template grabs attention and maintains it until the last page.

What we love about this template is the smart use of visuals. You'll find a flowchart explaining the methodology, a bar graph for the budget, and a timeline for the project. But that’s just the tip of the iceberg regarding the visual elements you’ll find in Visme.

Visme offers data visualization tools with 30+ data widgets, such as radial gauges, population arrays, progress bars and more. These tools can help you turn complex data into engaging visuals for your research proposal or any other document.

For larger data sets, you can choose from 20+ types of charts and graphs , including bar graphs , bubble charts , Venn diagrams and more.

5. Tech Research Proposal Template

If you’re a tech researcher, we’ve got the perfect template for you. This research proposal example is about predictive analytics in e-commerce. However, you can customize it for any other type of research proposal.

It highlights the project's objectives, including the effectiveness of predictive analysis, the impact of product recommendations and supply chain optimization. The methods proposed for achieving these objectives involve A/B testing and data analysis, a comprehensive budget and a 12-month timeline for clear project planning.

The title page has a unique triptych-style layout that immediately catches the reader's attention. It has plenty of white space that enhances readability, allowing your audience to focus on the critical points.

Submitting to different funding agencies? You don’t have to manually make changes to your document. Visme's dynamic fields can help save time and eliminate repetitive data entry.

Create custom fields like project names, addresses, contact information and more. Any changes made to these fields will automatically populate throughout the document.

6. Marketing Research Proposal Template

Artificial intelligence (AI) is taking the world by storm and the marketing niche isn’t left out. With this eye-catching template, you can attract attention to your proposed marketing research project for an AI-driven platform.

The main goal of the research is to evaluate the platform's feasibility and marketing potential. To achieve this goal, the scope of work includes a comprehensive analysis of the market and competitors and pilot testing. The proposal also contains a budget overview that clearly outlines the allocation of funds, ensuring a well-planned and transparent approach.

Using Visme's Brand Design Tool , you can easily customize this template to suit your branding with just one click. Simply enter your URL into the brand wizard, and the tool will automatically extract your company logo, brand colors, and brand fonts . Once saved, you or your team members can apply the branding elements to any document. It's that simple!

7. Environmental Research Proposal Template

The environmental research proposal example focuses on carbon emissions, identifies their contributing factors, and suggests sustainable practices to address them. It uses an appropriate sample size and data collection techniques to gather and evaluate data and provide sustainable recommendations to reduce industrial carbon footprints and waste.

From a design standpoint, the green and white color combination matches the theme of nature and environmental friendliness. In addition to its aesthetic appeal, the proposal includes relevant images that support ecological advocacy, making it informative and visually aligned with its purpose.

A key feature of this template is its detailed breakdown of the project's timeline. It uses a Gantt chart to clearly present stages, milestones and deadlines.

Collaborate with your team members to customize these research proposal templates using Visme’s collaborative design features . These features allow you to leave feedback, draw annotations and even make live edits. Invite your teammates via email or a shareable link and allow them to work together on projects.

8. General Approval Research Proposal Template

This research proposal template is a total game-changer - you can use it for any research proposal and customize it however you want. It features a modern and refreshing color scheme that immediately makes it stand out, providing a contemporary look that can adapt to any project's needs.

The template's layout is thoughtfully designed with primary fields that users can easily personalize by changing text, adjusting colors, or swapping images. No matter the research topic, you can tailor the template to fit your specific needs.

Once you're done customizing your research proposal template on Visme, you can download, share and publish it in different ways. For offline usage, you may download the proposal in PDF, PNG, or JPG format. To share it online, you can use a private or public link or generate a code snippet that you can embed anywhere on the web.

Want to create other types of proposals? Here are 29 proposal templates that you can easily customize in Visme.

Q. What Are the Five Steps of Writing a Research Proposal?

Follow these steps to write a solid research proposal:

• Choose a topic within your field of study that can be explored and investigated.
• Research existing literature and studies to build a foundational understanding and prepare your research question.
• Outline your research proposal: introduction, literature review, proposed methodology, budget and timeline.
• Conduct more detailed studies to strengthen your proposition, refine your research question and justify your methodology.
• Follow your outline to write a clear and organized proposal, then review and edit for accuracy before submitting.

If you want to learn more about creating an expert research proposal , we highly recommend checking out our in-depth guide.

Q. How Long Is a Research Proposal?

Research proposals can range from 1,000 to 5,000 words. For smaller projects or when specific requirements aren't provided, aim for a concise and informative proposal that effectively outlines your research plan.

However, the ideal length depends on these factors:

• Projects with complex methodologies or multiple phases may require longer proposals to explain the scope and procedures in detail.
• Universities, academic institutions and funding agencies often have guidelines of a specific length. Always check their requirements beforehand.
• When writing a proposal, adjust the level of study based on the audience. Academic proposals may require comprehensive explanations, while business or non-profit proposals require a more streamlined approach.

Q. How Long Does It Take to Write a Research Proposal?

The time it takes to write a research proposal depends on a few factors:

• Complex research with extensive data collection or analysis will naturally take longer to plan and write about.
• If you're new to writing research proposals, expect to spend more time learning the format and best practices.
• If you've already conducted some research or a thorough literature review, the writing process might go faster.
• Funding applications often have strict deadlines that will dictate your timeline.

Set aside several weeks to a couple of months for researching, writing, and revising your proposal. Start early to avoid stress and produce your best work.

Q. What Not to Do for a Research Proposal?

There are several factors that can make a research proposal weak. Here are some of the most common errors that you should avoid while preparing your research proposal:

• Don’t choose a topic that’s too broad. Focus on a specific area you can thoroughly explore within your proposal’s limits.
• Don’t ignore the rules for formatting and submitting your proposal. Always adhere to the requirements set by your institution or funding body.
• Don’t forget to conduct a thorough literature review. It's crucial to show your grasp of existing research related to your topic.
• Don't be vague about your methods. Ensure they're clearly defined and suitable for answering your research question.
• Don't overlook errors in grammar, typos or structure. A well-proofread proposal reflects professionalism, so review it carefully before submitting it.

Craft Professional & Engaging Proposals with Visme

Writing a compelling research proposal takes effort, but with the right tools, the process becomes a breeze. Use the research proposal examples and templates in this article as a launching point to write your own proposal.

The best part? Visme provides easy-to-use tools with a vast collection of customizable templates, design elements and powerful features.

Whether you're a seasoned researcher or a student, Visme has the resources to help you create visually appealing and well-structured research proposals. In addition to research proposals, Visme helps you create many other document types, such as presentations , infographics , reports and more.

Ready to create your own research proposal? Check out Visme's proposal maker and start crafting professional and engaging proposals in minutes!

Create professional research proposals with Visme

Trusted by leading brands

Recommended content for you:

Create Stunning Content!

Design visual brand experiences for your business whether you are a seasoned designer or a total novice.

About the Author

Raja Antony Mandal is a Content Writer at Visme. He can quickly adapt to different writing styles, possess strong research skills, and know SEO fundamentals. Raja wants to share valuable information with his audience by telling captivating stories in his articles. He wants to travel and party a lot on the weekends, but his guitar, drum set, and volleyball court don’t let him.

• Privacy Policy

Home » Research Paper – Structure, Examples and Writing Guide

Research Paper – Structure, Examples and Writing Guide

Table of Contents

Research Paper

Definition:

Research Paper is a written document that presents the author’s original research, analysis, and interpretation of a specific topic or issue.

It is typically based on Empirical Evidence, and may involve qualitative or quantitative research methods, or a combination of both. The purpose of a research paper is to contribute new knowledge or insights to a particular field of study, and to demonstrate the author’s understanding of the existing literature and theories related to the topic.

Structure of Research Paper

The structure of a research paper typically follows a standard format, consisting of several sections that convey specific information about the research study. The following is a detailed explanation of the structure of a research paper:

The title page contains the title of the paper, the name(s) of the author(s), and the affiliation(s) of the author(s). It also includes the date of submission and possibly, the name of the journal or conference where the paper is to be published.

The abstract is a brief summary of the research paper, typically ranging from 100 to 250 words. It should include the research question, the methods used, the key findings, and the implications of the results. The abstract should be written in a concise and clear manner to allow readers to quickly grasp the essence of the research.

Introduction

The introduction section of a research paper provides background information about the research problem, the research question, and the research objectives. It also outlines the significance of the research, the research gap that it aims to fill, and the approach taken to address the research question. Finally, the introduction section ends with a clear statement of the research hypothesis or research question.

Literature Review

The literature review section of a research paper provides an overview of the existing literature on the topic of study. It includes a critical analysis and synthesis of the literature, highlighting the key concepts, themes, and debates. The literature review should also demonstrate the research gap and how the current study seeks to address it.

The methods section of a research paper describes the research design, the sample selection, the data collection and analysis procedures, and the statistical methods used to analyze the data. This section should provide sufficient detail for other researchers to replicate the study.

The results section presents the findings of the research, using tables, graphs, and figures to illustrate the data. The findings should be presented in a clear and concise manner, with reference to the research question and hypothesis.

The discussion section of a research paper interprets the findings and discusses their implications for the research question, the literature review, and the field of study. It should also address the limitations of the study and suggest future research directions.

The conclusion section summarizes the main findings of the study, restates the research question and hypothesis, and provides a final reflection on the significance of the research.

The references section provides a list of all the sources cited in the paper, following a specific citation style such as APA, MLA or Chicago.

How to Write Research Paper

You can write Research Paper by the following guide:

• Choose a Topic: The first step is to select a topic that interests you and is relevant to your field of study. Brainstorm ideas and narrow down to a research question that is specific and researchable.
• Conduct a Literature Review: The literature review helps you identify the gap in the existing research and provides a basis for your research question. It also helps you to develop a theoretical framework and research hypothesis.
• Develop a Thesis Statement : The thesis statement is the main argument of your research paper. It should be clear, concise and specific to your research question.
• Plan your Research: Develop a research plan that outlines the methods, data sources, and data analysis procedures. This will help you to collect and analyze data effectively.
• Collect and Analyze Data: Collect data using various methods such as surveys, interviews, observations, or experiments. Analyze data using statistical tools or other qualitative methods.
• Organize your Paper : Organize your paper into sections such as Introduction, Literature Review, Methods, Results, Discussion, and Conclusion. Ensure that each section is coherent and follows a logical flow.
• Write your Paper : Start by writing the introduction, followed by the literature review, methods, results, discussion, and conclusion. Ensure that your writing is clear, concise, and follows the required formatting and citation styles.
• Edit and Proofread your Paper: Review your paper for grammar and spelling errors, and ensure that it is well-structured and easy to read. Ask someone else to review your paper to get feedback and suggestions for improvement.
• Cite your Sources: Ensure that you properly cite all sources used in your research paper. This is essential for giving credit to the original authors and avoiding plagiarism.

Research Paper Example

Note : The below example research paper is for illustrative purposes only and is not an actual research paper. Actual research papers may have different structures, contents, and formats depending on the field of study, research question, data collection and analysis methods, and other factors. Students should always consult with their professors or supervisors for specific guidelines and expectations for their research papers.

Research Paper Example sample for Students:

Title: The Impact of Social Media on Mental Health among Young Adults

Abstract: This study aims to investigate the impact of social media use on the mental health of young adults. A literature review was conducted to examine the existing research on the topic. A survey was then administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO (Fear of Missing Out) are significant predictors of mental health problems among young adults.

Introduction: Social media has become an integral part of modern life, particularly among young adults. While social media has many benefits, including increased communication and social connectivity, it has also been associated with negative outcomes, such as addiction, cyberbullying, and mental health problems. This study aims to investigate the impact of social media use on the mental health of young adults.

Literature Review: The literature review highlights the existing research on the impact of social media use on mental health. The review shows that social media use is associated with depression, anxiety, stress, and other mental health problems. The review also identifies the factors that contribute to the negative impact of social media, including social comparison, cyberbullying, and FOMO.

Methods : A survey was administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The survey included questions on social media use, mental health status (measured using the DASS-21), and perceived impact of social media on their mental health. Data were analyzed using descriptive statistics and regression analysis.

Results : The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO are significant predictors of mental health problems among young adults.

Discussion : The study’s findings suggest that social media use has a negative impact on the mental health of young adults. The study highlights the need for interventions that address the factors contributing to the negative impact of social media, such as social comparison, cyberbullying, and FOMO.

Conclusion : In conclusion, social media use has a significant impact on the mental health of young adults. The study’s findings underscore the need for interventions that promote healthy social media use and address the negative outcomes associated with social media use. Future research can explore the effectiveness of interventions aimed at reducing the negative impact of social media on mental health. Additionally, longitudinal studies can investigate the long-term effects of social media use on mental health.

Limitations : The study has some limitations, including the use of self-report measures and a cross-sectional design. The use of self-report measures may result in biased responses, and a cross-sectional design limits the ability to establish causality.

Implications: The study’s findings have implications for mental health professionals, educators, and policymakers. Mental health professionals can use the findings to develop interventions that address the negative impact of social media use on mental health. Educators can incorporate social media literacy into their curriculum to promote healthy social media use among young adults. Policymakers can use the findings to develop policies that protect young adults from the negative outcomes associated with social media use.

References :

• Twenge, J. M., & Campbell, W. K. (2019). Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study. Preventive medicine reports, 15, 100918.
• Primack, B. A., Shensa, A., Escobar-Viera, C. G., Barrett, E. L., Sidani, J. E., Colditz, J. B., … & James, A. E. (2017). Use of multiple social media platforms and symptoms of depression and anxiety: A nationally-representative study among US young adults. Computers in Human Behavior, 69, 1-9.
• Van der Meer, T. G., & Verhoeven, J. W. (2017). Social media and its impact on academic performance of students. Journal of Information Technology Education: Research, 16, 383-398.

Appendix : The survey used in this study is provided below.

Social Media and Mental Health Survey

• How often do you use social media per day?
• Less than 30 minutes
• 30 minutes to 1 hour
• 1 to 2 hours
• 2 to 4 hours
• More than 4 hours
• Which social media platforms do you use?
• Others (Please specify)
• How often do you experience the following on social media?
• Social comparison (comparing yourself to others)
• Cyberbullying
• Fear of Missing Out (FOMO)
• Have you ever experienced any of the following mental health problems in the past month?
• Do you think social media use has a positive or negative impact on your mental health?
• Very positive
• Somewhat positive
• Somewhat negative
• Very negative
• In your opinion, which factors contribute to the negative impact of social media on mental health?
• Social comparison
• In your opinion, what interventions could be effective in reducing the negative impact of social media on mental health?
• Education on healthy social media use
• Counseling for mental health problems caused by social media
• Social media detox programs
• Regulation of social media use

Thank you for your participation!

Applications of Research Paper

Research papers have several applications in various fields, including:

• Advancing knowledge: Research papers contribute to the advancement of knowledge by generating new insights, theories, and findings that can inform future research and practice. They help to answer important questions, clarify existing knowledge, and identify areas that require further investigation.
• Informing policy: Research papers can inform policy decisions by providing evidence-based recommendations for policymakers. They can help to identify gaps in current policies, evaluate the effectiveness of interventions, and inform the development of new policies and regulations.
• Improving practice: Research papers can improve practice by providing evidence-based guidance for professionals in various fields, including medicine, education, business, and psychology. They can inform the development of best practices, guidelines, and standards of care that can improve outcomes for individuals and organizations.
• Educating students : Research papers are often used as teaching tools in universities and colleges to educate students about research methods, data analysis, and academic writing. They help students to develop critical thinking skills, research skills, and communication skills that are essential for success in many careers.
• Fostering collaboration: Research papers can foster collaboration among researchers, practitioners, and policymakers by providing a platform for sharing knowledge and ideas. They can facilitate interdisciplinary collaborations and partnerships that can lead to innovative solutions to complex problems.

When to Write Research Paper

Research papers are typically written when a person has completed a research project or when they have conducted a study and have obtained data or findings that they want to share with the academic or professional community. Research papers are usually written in academic settings, such as universities, but they can also be written in professional settings, such as research organizations, government agencies, or private companies.

Here are some common situations where a person might need to write a research paper:

• For academic purposes: Students in universities and colleges are often required to write research papers as part of their coursework, particularly in the social sciences, natural sciences, and humanities. Writing research papers helps students to develop research skills, critical thinking skills, and academic writing skills.
• For publication: Researchers often write research papers to publish their findings in academic journals or to present their work at academic conferences. Publishing research papers is an important way to disseminate research findings to the academic community and to establish oneself as an expert in a particular field.
• To inform policy or practice : Researchers may write research papers to inform policy decisions or to improve practice in various fields. Research findings can be used to inform the development of policies, guidelines, and best practices that can improve outcomes for individuals and organizations.
• To share new insights or ideas: Researchers may write research papers to share new insights or ideas with the academic or professional community. They may present new theories, propose new research methods, or challenge existing paradigms in their field.

Purpose of Research Paper

The purpose of a research paper is to present the results of a study or investigation in a clear, concise, and structured manner. Research papers are written to communicate new knowledge, ideas, or findings to a specific audience, such as researchers, scholars, practitioners, or policymakers. The primary purposes of a research paper are:

• To contribute to the body of knowledge : Research papers aim to add new knowledge or insights to a particular field or discipline. They do this by reporting the results of empirical studies, reviewing and synthesizing existing literature, proposing new theories, or providing new perspectives on a topic.
• To inform or persuade: Research papers are written to inform or persuade the reader about a particular issue, topic, or phenomenon. They present evidence and arguments to support their claims and seek to persuade the reader of the validity of their findings or recommendations.
• To advance the field: Research papers seek to advance the field or discipline by identifying gaps in knowledge, proposing new research questions or approaches, or challenging existing assumptions or paradigms. They aim to contribute to ongoing debates and discussions within a field and to stimulate further research and inquiry.
• To demonstrate research skills: Research papers demonstrate the author’s research skills, including their ability to design and conduct a study, collect and analyze data, and interpret and communicate findings. They also demonstrate the author’s ability to critically evaluate existing literature, synthesize information from multiple sources, and write in a clear and structured manner.

Characteristics of Research Paper

Research papers have several characteristics that distinguish them from other forms of academic or professional writing. Here are some common characteristics of research papers:

• Evidence-based: Research papers are based on empirical evidence, which is collected through rigorous research methods such as experiments, surveys, observations, or interviews. They rely on objective data and facts to support their claims and conclusions.
• Structured and organized: Research papers have a clear and logical structure, with sections such as introduction, literature review, methods, results, discussion, and conclusion. They are organized in a way that helps the reader to follow the argument and understand the findings.
• Formal and objective: Research papers are written in a formal and objective tone, with an emphasis on clarity, precision, and accuracy. They avoid subjective language or personal opinions and instead rely on objective data and analysis to support their arguments.
• Citations and references: Research papers include citations and references to acknowledge the sources of information and ideas used in the paper. They use a specific citation style, such as APA, MLA, or Chicago, to ensure consistency and accuracy.
• Peer-reviewed: Research papers are often peer-reviewed, which means they are evaluated by other experts in the field before they are published. Peer-review ensures that the research is of high quality, meets ethical standards, and contributes to the advancement of knowledge in the field.
• Objective and unbiased: Research papers strive to be objective and unbiased in their presentation of the findings. They avoid personal biases or preconceptions and instead rely on the data and analysis to draw conclusions.

Advantages of Research Paper

Research papers have many advantages, both for the individual researcher and for the broader academic and professional community. Here are some advantages of research papers:

• Contribution to knowledge: Research papers contribute to the body of knowledge in a particular field or discipline. They add new information, insights, and perspectives to existing literature and help advance the understanding of a particular phenomenon or issue.
• Opportunity for intellectual growth: Research papers provide an opportunity for intellectual growth for the researcher. They require critical thinking, problem-solving, and creativity, which can help develop the researcher’s skills and knowledge.
• Career advancement: Research papers can help advance the researcher’s career by demonstrating their expertise and contributions to the field. They can also lead to new research opportunities, collaborations, and funding.
• Academic recognition: Research papers can lead to academic recognition in the form of awards, grants, or invitations to speak at conferences or events. They can also contribute to the researcher’s reputation and standing in the field.
• Impact on policy and practice: Research papers can have a significant impact on policy and practice. They can inform policy decisions, guide practice, and lead to changes in laws, regulations, or procedures.
• Advancement of society: Research papers can contribute to the advancement of society by addressing important issues, identifying solutions to problems, and promoting social justice and equality.

Limitations of Research Paper

Research papers also have some limitations that should be considered when interpreting their findings or implications. Here are some common limitations of research papers:

• Limited generalizability: Research findings may not be generalizable to other populations, settings, or contexts. Studies often use specific samples or conditions that may not reflect the broader population or real-world situations.
• Potential for bias : Research papers may be biased due to factors such as sample selection, measurement errors, or researcher biases. It is important to evaluate the quality of the research design and methods used to ensure that the findings are valid and reliable.
• Ethical concerns: Research papers may raise ethical concerns, such as the use of vulnerable populations or invasive procedures. Researchers must adhere to ethical guidelines and obtain informed consent from participants to ensure that the research is conducted in a responsible and respectful manner.
• Limitations of methodology: Research papers may be limited by the methodology used to collect and analyze data. For example, certain research methods may not capture the complexity or nuance of a particular phenomenon, or may not be appropriate for certain research questions.
• Publication bias: Research papers may be subject to publication bias, where positive or significant findings are more likely to be published than negative or non-significant findings. This can skew the overall findings of a particular area of research.
• Time and resource constraints: Research papers may be limited by time and resource constraints, which can affect the quality and scope of the research. Researchers may not have access to certain data or resources, or may be unable to conduct long-term studies due to practical limitations.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Tables in Research Paper – Types, Creating Guide...

Literature Review – Types Writing Guide and...

Research Paper Conclusion – Writing Guide and...

Future Research – Thesis Guide

Research Questions – Types, Examples and Writing...

Critical Analysis – Types, Examples and Writing...

Educational resources and simple solutions for your research journey

How to Write a Research Paper Outline (with Examples)

Writing a research paper is an essential part of an academic career. However, the task can be quite challenging especially for early career researchers unfamiliar with the nuances of academic research and writing. Creating an impactful research paper demands meticulous attention to detail, an in depth understanding of the topic and research methodology, and the ability to communicate the findings in an accurate and easy to understand way. This is where a research paper outline becomes useful. Writing a research paper can be made simpler and more efficient with a well-organized plan. A well-structured research paper outline offers the fundamental foundation on which researchers can construct their narratives logically, ensuring that the study report is well-presented and interesting for readers.   

Table of Contents

This article takes a look now at the benefits of having a good research paper outline and also provides guidance on creating one.  

4 steps to create a well-structured research paper outline    

List the key components  .

To begin with, researchers must list down the key components that should be included in the research paper outline . Start with identifying your research question. Organize your key ideas and thoughts so that you are able to clearly convey the various aspects of your research question or thesis statement. Create separate points for the introduction, literature review, methodology, results, significance of your research along with its limitations. These sections will help you organize your thoughts and ensure that all relevant information is included in your research manuscript.  

Structure the outline logically  

As you create your outline, make sure that there is logical flow of ideas and arguments. Think through the sequence in which you will present your topic and ideas. Structure the research paper outline in a way that allows a clear and continuous narrative that is easy to understand. For example, the introduction must be concise and engaging and must clearly introduce the research topic. The main paragraphs must focus on the research problem and arguments with supporting evidence. Experts suggest using headings and sub-heads to help organize ideas and data into sub-groups. The concluding section should have a summary of your study’s main points and key takeaways with recommendations for future research.   

Provide supporting evidence  

It is important to provide adequate supporting evidence and examples that underpin your key idea or argument. This helps to fit your study into the larger context of your subject area. It may be a good idea to collect all your data and relevant sources right from the start. Experts suggest providing at last three supporting evidences for each of your main ideas and including appropriate and accurate citations in the research paper outline .  

Review and edit  

Finally, take time to review the outline and make necessary modifications as you come across new data and information. To do so, you must have sufficient knowledge of the existing and current literature on the topic. Make sure that your ideas are in a logical order, and you have not missed out anything from your research notes.  

3 tips to draft a great research paper outline   

• Be concise and clear: Avoid adding unnecessary details to your research paper outline . Try instead, to focus only on the key ideas, information and supporting evidence for your study. Experts suggest avoiding the use of lengthy sentences and recommend the use of short phrases, sub-heads, and bullet points to outline ideas.  
• Stay consistent with formatting: To ensure consistency in formatting, researchers can choose from different kinds of research paper outline templates. The most commonly used ones are:
• The alpha-numerical template where the points are written as short sentences,   
• The full sentence format where whole sentences are written with specific points   
• The decimal format where the main point is presented as a whole number (1, 2) and sub-points are given as decimal points (1.1, 1.2).   
• Seek feedback from supervisors: Once you have completed the outline, it is a good idea to share it with your supervisors and mentors and seek their insights. Their inputs will help ensure that your research paper outline is on track.   

Research paper outline example

Given below is a research paper outline example that you can use as a starting point.

I. Introduction

• Background and context of the research topic
• Problem statement and research question  
• Significance of the study

II. Literature Review

• Overview of relevant literature  
• Discussion of previous research and findings  
• Identification of gaps and areas for further exploration  

III. Methodology 

• Explanation of the research design  
• Description of data collection methods  
• Discussion of data analysis techniques

IV. Results

• Presentation of research findings  
• Data visualization (tables, graphs, charts, etc.)  
• Explanation of key results

V. Discussion

• Interpretation of the results  
• Comparison with existing literature  
• Addressing limitations and implications of the study

VI. Conclusion

• Summary of the research paper  
• Final remarks and suggestions for future research   

Researcher.Life is a subscription-based platform that unifies the best AI tools and services designed to speed up, simplify, and streamline every step of a researcher’s journey. The Researcher.Life All Access Pack is a one-of-a-kind subscription that unlocks full access to an AI writing assistant, literature recommender, journal finder, scientific illustration tool, and exclusive discounts on professional publication services from Editage.  

Based on 21+ years of experience in academia, Researcher.Life All Access empowers researchers to put their best research forward and move closer to success. Explore our top AI Tools pack, AI Tools + Publication Services pack, or Build Your Own Plan. Find everything a researcher needs to succeed, all in one place –  Get All Access now starting at just $17 a month !    

Related Posts

Take Top AI Tools for Researchers for a Spin with the Editage All Access 7-Day Pass!

Thesis Defense: How to Ace this Crucial Step

Reference management. Clean and simple.

Getting started with your research paper outline

Levels of organization for a research paper outline

First level of organization, second level of organization, third level of organization, fourth level of organization, tips for writing a research paper outline, research paper outline template, my research paper outline is complete: what are the next steps, frequently asked questions about a research paper outline, related articles.

The outline is the skeleton of your research paper. Simply start by writing down your thesis and the main ideas you wish to present. This will likely change as your research progresses; therefore, do not worry about being too specific in the early stages of writing your outline.

A research paper outline typically contains between two and four layers of organization. The first two layers are the most generalized. Each layer thereafter will contain the research you complete and presents more and more detailed information.

The levels are typically represented by a combination of Roman numerals, Arabic numerals, uppercase letters, lowercase letters but may include other symbols. Refer to the guidelines provided by your institution, as formatting is not universal and differs between universities, fields, and subjects. If you are writing the outline for yourself, you may choose any combination you prefer.

This is the most generalized level of information. Begin by numbering the introduction, each idea you will present, and the conclusion. The main ideas contain the bulk of your research paper 's information. Depending on your research, it may be chapters of a book for a literature review , a series of dates for a historical research paper, or the methods and results of a scientific paper.

I. Introduction

II. Main idea

III. Main idea

IV. Main idea

V. Conclusion

The second level consists of topics which support the introduction, main ideas, and the conclusion. Each main idea should have at least two supporting topics listed in the outline.

If your main idea does not have enough support, you should consider presenting another main idea in its place. This is where you should stop outlining if this is your first draft. Continue your research before adding to the next levels of organization.

• A. Background information
• B. Hypothesis or thesis
• A. Supporting topic
• B. Supporting topic

The third level of organization contains supporting information for the topics previously listed. By now, you should have completed enough research to add support for your ideas.

The Introduction and Main Ideas may contain information you discovered about the author, timeframe, or contents of a book for a literature review; the historical events leading up to the research topic for a historical research paper, or an explanation of the problem a scientific research paper intends to address.

• 1. Relevant history
• 2. Relevant history
• 1. The hypothesis or thesis clearly stated
• 1. A brief description of supporting information
• 2. A brief description of supporting information

The fourth level of organization contains the most detailed information such as quotes, references, observations, or specific data needed to support the main idea. It is not typical to have further levels of organization because the information contained here is the most specific.

• a) Quotes or references to another piece of literature
• b) Quotes or references to another piece of literature

Tip: The key to creating a useful outline is to be consistent in your headings, organization, and levels of specificity.

• Be Consistent : ensure every heading has a similar tone. State the topic or write short sentences for each heading but avoid doing both.
• Organize Information : Higher levels of organization are more generally stated and each supporting level becomes more specific. The introduction and conclusion will never be lower than the first level of organization.
• Build Support : Each main idea should have two or more supporting topics. If your research does not have enough information to support the main idea you are presenting, you should, in general, complete additional research or revise the outline.

By now, you should know the basic requirements to create an outline for your paper. With a content framework in place, you can now start writing your paper . To help you start right away, you can use one of our templates and adjust it to suit your needs.

After completing your outline, you should:

• Title your research paper . This is an iterative process and may change when you delve deeper into the topic.
• Begin writing your research paper draft . Continue researching to further build your outline and provide more information to support your hypothesis or thesis.
• Format your draft appropriately . MLA 8 and APA 7 formats have differences between their bibliography page, in-text citations, line spacing, and title.
• Finalize your citations and bibliography . Use a reference manager like Paperpile to organize and cite your research.
• Write the abstract, if required . An abstract will briefly state the information contained within the paper, results of the research, and the conclusion.

An outline is used to organize written ideas about a topic into a logical order. Outlines help us organize major topics, subtopics, and supporting details. Researchers benefit greatly from outlines while writing by addressing which topic to cover in what order.

The most basic outline format consists of: an introduction, a minimum of three topic paragraphs, and a conclusion.

You should make an outline before starting to write your research paper. This will help you organize the main ideas and arguments you want to present in your topic.

• Consistency: ensure every heading has a similar tone. State the topic or write short sentences for each heading but avoid doing both.
• Organization : Higher levels of organization are more generally stated and each supporting level becomes more specific. The introduction and conclusion will never be lower than the first level of organization.
• Support : Each main idea should have two or more supporting topics. If your research does not have enough information to support the main idea you are presenting, you should, in general, complete additional research or revise the outline.

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

Planning and Writing a Research Paper

Mastering the complex academic skill of writing a research paper will prepare you to enter the discourse community of your chosen area of study with excitement and confidence. Writing a research paper can seem like a daunting task, but if you take the time in the pages ahead to learn how to break the writing process down, you will be amazed at the level of comfort and control you feel when preparing your assignment. 

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved June 25, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• TemplateLab
• Research Poposal Templates

40 Best Research Proposal Templates & Format Examples

If you are doing academic research or any research for the company you work for, you will need to present the material in a professional fashion. A research proposal will help explain the intention behind the research you plan to conduct. It will also highlight the research techniques you plan to use. Making clear your intentions and methods is imperative if you want your proposal to impress those who must approve the project. This is where a research proposal template serves as a valuable and helpful resource.

Table of Contents

• 1 Research Proposal Templates
• 2 Who should use a research proposal?
• 3 What are the research proposal template requirements?
• 4 Research Proposal Examples
• 5.1 Research Paper Title
• 5.2 Table of Contents
• 5.3 Abstract
• 5.4 Background and Rationale
• 5.5 Research Questions or Introduction
• 5.6 Literary Review
• 6 Research Proposal Samples
• 7 Final Thoughts
• 8 Research Proposal Format

A research paper proposal template breaks down all the necessary sections of the proposal into segments. You can use a research proposal example to help in designing your own template. But, you also have the choice of using a ready-made research paper outline template to make things easier for you.

When filling out the research proposal template you will still need to take considerable care in developing the proposal’s presentation. The proposal needs to be exact, easy to understand, and concise. Academic or business-oriented writing and language are essential. You will need to fill the template in such a way that your ideas still be in linear order. The most basic of formats must be the final shape of the presentation. In other words, your writing is fluid and move from the introduction to the body of the proposal. It must then continue with a fluid presentation into the conclusion of the document.

Research Proposal Templates

Who should use a research proposal?

If you are writing a paper for an independent study or you are writing a dissertation in college, you will need to write a research proposal. The document must express the subject of your research. It must also include how you plan to go forth with the research. This document is of tremendous importance since it is the precursor to a larger body of work. The research paper proposal template is ideal for the following people:

• Students conducting independent studies.
• Students preparing for a dissertation.
• Students practice writing for future research papers.
• Employees looking to write a research paper for their employer.
• Faculty members at universities looking to write research papers.
• People writing a proposal to get a grant for research.

What are the research proposal template requirements?

Every research paper differs and is dependent upon the subject matter. You should talk with the supervisor of your project to find out about project expectations and deadlines. You can verify proposal deadlines and if the research you are interested in pursuing is acceptable.

The academic level and the purpose of the paper, whether a dissertation or some other type of assignment, plays a role in what goes into the initial proposal. However, there are some general elements one can expect to find in a proposal.

A quality research proposal example will reveal a document offering a clear, well-written outline describing your project’s undertaking. The plans you have for completing the research is something to include in the proposal. The goal of the proposal is so you can sway project funders to provide you with the monies needed to complete the necessary research. Or, it might be that your goal is to impress your professor by proving the knowledge you’ve accumulated will contribute to a successful research project.

No matter what you decide to research, the proposal will have some basic requirements to meet. The proposal must include the following:

• Your name and course name (if applicable).
• Your student identification number (if applicable).
• The Governing Education Department and the Supervisor/Professor.
• The Course Code (if applicable).
• The date of the paper’s submission.
• Paper Title.
• An introduction to the longer body of work. This introduction needs to present your argument and what you are setting out to prove with your research.
• An explanation of your planned research methods.
• A timetable revealing the length of time for the completion of research (if applicable).
• A mention of any ethical considerations.
• A paragraph or more describing anything that might limit the scope of the research you plan to conduct.
• A bibliography and citations with the correct citation format (Of works you plan to work with; the list is something you can expand upon and edit as you conduct your research).

Research Proposal Examples

What is the proper length for a research proposal?

It’s imperative that the abstract stays brief. The expectation is about 250 to 300 words in length. That’s about 10 percent of the total length of the research proposal. Longer than that, and it might deter the reader’s interest. It will also give you less room to discuss the goals and aims of the research project.

The length of the research proposal template is not long. You’ll find most writing requirements demand a word count of 2500 to 3000 words. Depending on spacing requirements, this might be three to five pages of information. This ensures concise writing. Of course, the study supervisor makes the final call on proposal length and necessary inclusions.

Research Paper Title

The research proposal template will have an area where you can put the title of the document. This title needs to be short, concise, and to the point. It should not take up more space than necessary. You’ll need to have a title that is absent of acronyms or abbreviations. The title needs to have a hook or stimulating words that will stir up the reader’s interest in what the research paper is about; you can use a research proposal example to get an idea about excellent title writing.

Within the title, if it is possible for you to do so, you need to give reference to the independent and dependent variables. If it is possible, the title needs to still be under 15 words. It should not be shorter than five words. There will be room for the catchy title on a research paper proposal template; it is here you will win the attentions of that all-important reader.

The title should be on a separate page and set apart from the rest of the template. Here, the author’s name, the class or course name, and the date of the work is necessary. The page numbers begin on the next page, but do not appear on the first page of the template.

Part of presenting a well-documented and researched paper is to make it available with ease of access. The table of contents follows the title page. The main chapters or sections of the paper must appear neat and orderly. Sub-headings are something you can add to make it easy to find the detailed portions of each section or chapter as well. If you can create a mock table of contents, it can serve as an outline for the paper’s structure. You can always edit the table later as you develop your research and expand on the subject matter.

The abstract is the next section you will see on a well-constructed research proposal template. This area has information about what the paper holds. The abstract is a summary so the reader gets an immediate understanding about the arguments or discoveries in the paper.  The abstract should be in active voice if it is possible to do so.

The proper use of grammar and sentence style is also necessary when you are writing the abstract. You might benefit from writing the abstract portion of the paper last. This gives you an opportunity to work your way through the rest of the research paper outline template.  It allows for the opportunity to develop a clear description of all the essential information the paper holds.

The abstract of the research proposal template needs to be short. You do not mention your references or research material at this point. All sentences should be complete and you should not use ellipses. Terms, jargon, and related abbreviations should appear the body of the paper. Do not reference your images at this point either. It is a clear and unambiguous explanation of the paper’s purpose only.

Background and Rationale

Depending on the research proposal template and assignment specifications, you might see a section where you must present some background information and the rationale for the paper. This is a section allowing you to describe how you decided to examine the subject of the paper.

You can help draw the interest of the reader if you explain how you became interested in what the paper explores. The answer needs to be brief, but by explaining the development of your interests, it will help the reader understand the point of you view you take when examining your subject matter. Here, a bit of your background and experience can also prove both beneficial and revealing.

In this section, you can touch on some of the supporting literature you will use to back your arguments. Your familiarity with the subject matter will become clear as you make mention of the types of work you’ve explored during the process of your planned research. The information of most value, however, will be the reason the project is a worthwhile expedition or endeavor; it’s your job to tell the reader the knowledge you expect to gain through the study.

Research Questions or Introduction

As you examine one research proposal example after another, you will see the style of the paper differs on the type of coursework relating to the paper in question. You’ll also see differences in a research proposal template design when you are proposing diverse types of research. Some papers need an area where explore one or more research questions. Others might want a cut and dry introduction revealing the questions you plan to explore. The subject of your investigation and the questions you plan to answer need to have a clear definition. You need to know in advance what you plan to focus on with the work you are developing. Here you need to convince the reader you have a solid understanding of the subject and how you will approach it.

Literary Review

Some research paper proposal template selections will include an area where you can include a literary review. This section allows you to make a list of the literature you have explored to support your research. It can also list the literature you plan to examine to find evidence supporting your theories or position. The literature review is a place to give the reader with a summary of existing literature, and your interpretation of that literature. Emphasis is on your interpretation here; you don’t want to regurgitate someone else’s work. You want to interpret what you’ve found and enhance it by adding your own opinion to it. In this section, you’ll also need to find any gaps that might be in existing research. If your theory or presentation plans to address this gap in knowledge, you can identify that fact in this section by explaining how it will supply the missing knowledge.

The Framework or Methodology of the Proposal

This is the part of the research proposal template that needs information in relation to the methods you plan to use to research and support your argument. Are there special methods or procedures you must use? Here, you’ll have to source your information and data. The reader can internalize the information and consider whether it is valid or not. Your means of analysis will also undergo the reader’s scrutiny. The inclusion of the method section of is imperative. No matter what subject you are covering, if the information you are using is outdated, outmoded, spurious, or if your focus is too rigid, you can end up with a research project featuring weak or refutable arguments.

Since there are many methods for conducting research, you’ll need to clarify your research techniques. This section of the research template proposal gives you the chance to reveal the reasons why you choose one method of research over others. For example, did you use an observational method when you conducted your research ? Did you interact with any subjects in the study? If so, how did you interact? Did you offer a questionnaire for students to answer? What size was the representative sample for the questionnaire? Clarifying your method will allow the reader to understand how you approached the research.

Research Proposal Samples

Final Thoughts

A research proposal template can help simplify the task of proposing research for a dissertation, job, or research grant approval. While the template helps in designing a proposal that helps a supervisor or governing body understand your theories and methods, it is also a paper that helps you in developing an understanding of how to continue with writing the longer dissertation. It will serve as a documented outline of the literary pieces you’ll need to review for project completion. The proposal also defines your theory in concrete terms, so you can compare your findings with the initial assertions you set forth in the proposal. You may find your argument and findings evolve during the course of the research.

Research Proposal Format

More Templates

Collaboration Agreements

Industry Analysis Examples

Literature Review Templates

Qualitative Research Plan

Every drop counts . Because research requires the input of resources—money or kind—it should have a justified return. You may be fine with throwing away a few dollars, but what about thousands of dollars? And what if you could shorten five months of hard labor into half? Think of all the other things you could have done with your time and money. When you have a research plan , you can save yourself the avoidable hassle of losing your mind to stress at 3 AM.

Before the board or your academic mentors give your study a signal, you have to show them that you know what you’re doing. A research plan is your research roadmap. And like any map, you use the plan to steer you and your team in the right direction. In essence, it is a document that reminds the researcher of the important details about the study.

Plan vs. Proposal

A research plan is different from a research proposal . Although both talks about the study, the proposal is meant to sway opinion to favoring the conduct of the study. You also use proposals when you want to acquire study grants from higher institutions. A plan is for your perusal. As a researcher, you tend to become immersed in your study. By following all the promising trails, you may get lost in the process. Having a plan at arms reach lets you keep yourself on track. When you include a project timeline in your document, you can also track your progress along the way.

Qualitative vs. Quantitative

The rift goes way beyond numbers or the lack of thereof. The difference between the two isn’t because one is better than the other. In fact, a lot of research fields can benefit from the input of both methods. The choice between the two lies in what kind of question you want to answer. Qualitative research is appropriate for pioneer studies or those that require a deeper understanding of opinion, experiences, and encounters. Some things cannot be reduced to ones and zeroes. There are different methods for performing qualitative research. You can use interviews, focus groups, surveys , or observations. The versatility and cost-effectiveness of these methods make them a popular resort to researchers.

However, we cannot reduce quantitative research as a cold way to see the world. Quantitative research places measurements on things like opinion, behavior, and other variables. This method is more analytical and structured than qualitative research. Because most of the subjectivity is removed in data collection and analysis, the findings that are true for a small group can be used to generalize a bigger population. Most research in hard sciences is quantitative because the replicability of the results generally makes for credible results, especially when the only witnesses of the described event are the scientists in that lab. This research also makes use of surveys and questionnaires, provided that the observations can be represented in numerical data afterward.

Plan Framework

In general, the plans adhere to the same format, although you can see derivations in the names of the headers or the arrangement of the sections. The document is like a proposal, except that the details are made for the researchers themselves. Research plans can be a precursor to research proposals. Hence they tend to have similarities in the document structure.

Research Question:  This is the cold brew of your research study that kickstarts the entire research endeavor. This is the challenge or the issue that you want to address with your study. When you have a poorly-defined research question, you might as well forget about getting that research grant . The question is a lead on what the study will cover and the gaps in related literature.

Hypotheses:  These are your well-educated predictions on the results of the study in answer to your research questions. Your entire research design is grounded in testing these hypotheses. That is why your guesses must be backed by established and credible information. It is also these hypotheses that will be supported or refuted by succeeding studies.

Objectives: Objectives will influence the research design because what you want to accomplish will direct the methods you’ll use. When well-defined, they will steer you straight in the right direction. This means that they should be appropriate for your study. In devising your objectives , you should remember to make them specific, measurable, achievable, relevant, and time-based.

Research Design:  Because a research plan is like a rough sketch of your study, it includes your actual plan on how you will perform your investigation, as well as your list of materials and equipment. The details don’t have to be refined and specific, but they should convey the general idea. You can create a research flowchart of your methods to visualize the process better. Aside from being a map of the research, it is also an inventory check to see if you have the things you need for the study.

Examples of Qualitative Research Plans

People learn by example. Check out the following qualitative research plans that would help you with your content. You can download these PDF files as your guide.

1. Research Plan Sample

Size: 22 KB

2. Research Plan Guide

Size: 264 KB

3. Research Plan Abstract

Size: 73 KB

4. Research Plan Outline

Size: 106 Kb

5. Research Plan Example

Size: 116 KB

6. Funded Research Example

Size: 89 KB

7. Data Analysis Plan

Size: 941 KB

Preparing a Research Plan

Your research plan is for your use. It is meant to guide you throughout the entire research conduct . However, when you’ve set your standards too high and your plan is too idealistic, your performance and results might disappoint you. How do you make a plan that will work for you?

1. Research Your Research

When you want to answer a problem, you first have to be knowledgeable about it. Especially when you are applying for a research grant, your benefactors should have the impression that you know what you’re doing. You have to scour sources for related literature. Maybe the study has already been done, or there is a similar problem that has already been solved. By being diligent in your literature review, you can get a grasp of the issue’s relevance to society. Because you are learning more about the subject, you can identify methods and approaches that you can apply. By now, your study is taking shape.

2. Draw a Complete Map

This is a large section of your research plan. It describes what you want to come out of this study and your expectations. You will also write about your course of action to realize those goals. There is a domino relationship shared by your research questions, objectives, and methodology . The former two determine your methods. And the three will have a significant bearing on your results. You can use established methods provided that you justify why you use them. You can be as specific as possible. But because the plan is preliminary, you can expect changes along the way.

3. Be Practical and Realistic

As a researcher, you would want to make a significant contribution to the world. However, being too ambitious without the capacity to back it up will have negative consequences for your study. Therefore, when you plan a study, you have to look at your available resources. If you plan on procuring materials for the study, will they arrive on time? Is your expected schedule for deliverables realistic? Is your expectation for the study reasonable? You can add a timetable and a breakdown of foreseen expenses in your plan. That way, you can stick to your schedule and your budget.

4. Track Your Progress

Your research plan should be with you throughout the study period as a reference. You can view it to review your next steps or spot the ones you missed. Will the activities still fit the determined period? The chances that you will run out of time on an activity. Don’t create a rigid time frame. The future is unpredictable, so you should include a time allowance for each activity. You can also use Gantt charts to monitor your progress. The charts will let you see how much you have accomplished and how much work is left.

In any research endeavor, it pays to be prepared. We can’t predict the future, but when we have a plan on how to live with this uncertainty, we can mitigate losses. As a researcher, you can integrate research plans in the conduct of your studies. The document can influence the success of your investigation.

Text prompt

• Instructive
• Professional

Create a study plan for final exams in high school

Develop a project timeline for a middle school science fair.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 19 June 2024

Detecting hallucinations in large language models using semantic entropy

• Sebastian Farquhar   ORCID: orcid.org/0000-0002-9185-6415 1   na1 ,
• Jannik Kossen 1   na1 ,
• Lorenz Kuhn 1   na1 &
• Yarin Gal   ORCID: orcid.org/0000-0002-2733-2078 1  

Nature volume  630 ,  pages 625–630 ( 2024 ) Cite this article

57k Accesses

1456 Altmetric

Metrics details

• Computer science
• Information technology

Large language model (LLM) systems, such as ChatGPT 1 or Gemini 2 , can show impressive reasoning and question-answering capabilities but often ‘hallucinate’ false outputs and unsubstantiated answers 3 , 4 . Answering unreliably or without the necessary information prevents adoption in diverse fields, with problems including fabrication of legal precedents 5 or untrue facts in news articles 6 and even posing a risk to human life in medical domains such as radiology 7 . Encouraging truthfulness through supervision or reinforcement has been only partially successful 8 . Researchers need a general method for detecting hallucinations in LLMs that works even with new and unseen questions to which humans might not know the answer. Here we develop new methods grounded in statistics, proposing entropy-based uncertainty estimators for LLMs to detect a subset of hallucinations—confabulations—which are arbitrary and incorrect generations. Our method addresses the fact that one idea can be expressed in many ways by computing uncertainty at the level of meaning rather than specific sequences of words. Our method works across datasets and tasks without a priori knowledge of the task, requires no task-specific data and robustly generalizes to new tasks not seen before. By detecting when a prompt is likely to produce a confabulation, our method helps users understand when they must take extra care with LLMs and opens up new possibilities for using LLMs that are otherwise prevented by their unreliability.

Similar content being viewed by others

Testing theory of mind in large language models and humans

Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT

ThoughtSource: A central hub for large language model reasoning data

‘Hallucinations’ are a critical problem 9 for natural language generation systems using large language models (LLMs), such as ChatGPT 1 or Gemini 2 , because users cannot trust that any given output is correct.

Hallucinations are often defined as LLMs generating “content that is nonsensical or unfaithful to the provided source content” 9 , 10 , 11 but they have come to include a vast array of failures of faithfulness and factuality. We focus on a subset of hallucinations which we call ‘confabulations’ 12 for which LLMs fluently make claims that are both wrong and arbitrary—by which we mean that the answer is sensitive to irrelevant details such as random seed. For example, when asked a medical question “What is the target of Sotorasib?” an LLM confabulates by sometimes answering KRASG12 ‘C’ (correct) and other times KRASG12 ‘D’ (incorrect) despite identical instructions. We distinguish this from cases in which a similar ‘symptom’ is caused by the following different mechanisms: when LLMs are consistently wrong as a result of being trained on erroneous data such as common misconceptions 13 ; when the LLM ‘lies’ in pursuit of a reward 14 ; or systematic failures of reasoning or generalization. We believe that combining these distinct mechanisms in the broad category hallucination is unhelpful. Our method makes progress on a portion of the problem of providing scalable oversight 15 by detecting confabulations that people might otherwise find plausible. However, it does not guarantee factuality because it does not help when LLM outputs are systematically bad. Nevertheless, we significantly improve question-answering accuracy for state-of-the-art LLMs, revealing that confabulations are a great source of error at present.

We show how to detect confabulations by developing a quantitative measure of when an input is likely to cause an LLM to generate arbitrary and ungrounded answers. Detecting confabulations allows systems built on LLMs to avoid answering questions likely to cause confabulations, to make users aware of the unreliability of answers to a question or to supplement the LLM with more grounded search or retrieval. This is essential for the critical emerging field of free-form generation in which naive approaches, suited to closed vocabulary and multiple choice, fail. Past work on uncertainty for LLMs has focused on simpler settings, such as classifiers 16 , 17 and regressors 18 , 19 , whereas the most exciting applications of LLMs relate to free-form generations.

The term hallucination in the context of machine learning originally comes from filling in ungrounded details, either as a deliberate strategy 20 or as a reliability problem 4 . The appropriateness of the metaphor has been questioned as promoting undue anthropomorphism 21 . Although we agree that metaphor must be used carefully with LLMs 22 , the widespread adoption of the term hallucination reflects the fact that it points to an important phenomenon. This work represents a step towards making that phenomenon more precise.

To detect confabulations, we use probabilistic tools to define and then measure the ‘semantic’ entropy of the generations of an LLM—an entropy that is computed over meanings of sentences. High entropy corresponds to high uncertainty 23 , 24 , 25 —so semantic entropy is one way to estimate semantic uncertainties. Semantic uncertainty, the broader category of measures we introduce, could be operationalized with other measures of uncertainty, such as mutual information, instead. Entropy in free-form generation is normally hard to measure because answers might mean the same thing (be semantically equivalent) despite being expressed differently (being syntactically or lexically distinct). This causes naive estimates of entropy or other lexical variation scores 26 to be misleadingly high when the same correct answer might be written in many ways without changing its meaning.

By contrast, our semantic entropy moves towards estimating the entropy of the distribution of meanings of free-form answers to questions, insofar as that is possible, rather than the distribution over the ‘tokens’ (words or word-pieces) which LLMs natively represent. This can be seen as a kind of semantic consistency check 27 for random seed variation. An overview of our approach is provided in Fig. 1 and a worked example in Supplementary Table 1 .

a , Naive entropy-based uncertainty measures variation in the exact answers, treating ‘Paris’, ‘It’s Paris’ and ‘France’s capital Paris’ as different. But this is unsuitable for language tasks for which sometimes different answers mean the same things. Our semantic entropy clusters answers which share meanings before computing the entropy. A low semantic entropy shows that the LLM is confident about the meaning. b , Semantic entropy can also detect confabulations in longer passages. We automatically decompose a long generated answer into factoids. For each factoid, an LLM generates questions to which that factoid might have been the answer. The original LLM then samples  M possible answers to these questions. Finally, we compute the semantic entropy over the answers to each specific question, including the original factoid. Confabulations are indicated by high average semantic entropy for questions associated with that factoid. Here, semantic entropy classifies Fact 1 as probably not a confabulation because generations often mean the same thing, despite very different wordings, which a naive entropy would have missed.

Intuitively, our method works by sampling several possible answers to each question and clustering them algorithmically into answers that have similar meanings, which we determine on the basis of whether answers in the same cluster entail each other bidirectionally 28 . That is, if sentence A entails that sentence B is true and vice versa, then we consider them to be in the same semantic cluster. We measure entailment using both general-purpose LLMs and natural language inference (NLI) tools developed specifically for detecting entailment for which we show direct evaluations in Supplementary Tables 2 and 3 and Supplementary Fig. 1 . Textual entailment has previously been shown to correlate with faithfulness 10 in the context of factual consistency 29 as well as being used to measure factuality in abstractive summarization 30 , especially when applied at the right granularity 31 .

Semantic entropy detects confabulations in free-form text generation across a range of language models and domains, without previous domain knowledge. Our evaluations cover question answering in trivia knowledge (TriviaQA 32 ), general knowledge (SQuAD 1.1; ref. 33 ), life sciences (BioASQ 34 ) and open-domain natural questions (NQ-Open 35 ) derived from actual queries to Google Search 36 . In addition, semantic entropy detects confabulations in mathematical word problems (SVAMP 37 ) and in a biography-generation dataset, FactualBio, accompanying this paper.

Our results for TriviaQA, SQuAD, BioASQ, NQ-Open and SVAMP are all evaluated context-free and involve sentence-length answers (96 ± 70 characters, mean ± s.d.) and use LLaMA 2 Chat (7B, 13B and 70B parameters) 38 , Falcon Instruct (7B and 40B) 39 and Mistral Instruct (7B) 40 . In the Supplementary Information , we further consider short-phrase-length answers. Results for FactualBio (442 ± 122 characters) use GPT-4 (ref. 1 ). At the time of writing, GPT-4 (ref. 1 ) did not expose output probabilities 41 or hidden states, although it does now. As a result, we propose a discrete approximation of our estimator for semantic entropy which allows us to run experiments without access to output probabilities, which we use for all GPT-4 results in this paper and which performs similarly well.

Our confabulation detection with semantic entropy is more robust to user inputs from previously unseen domains than methods which aim to ‘learn’ how to detect confabulations from a set of example demonstrations. Our method is unsupervised, meaning that we do not need labelled examples of confabulations. By contrast, supervised methods detect confabulations by learning patterns behind examples of confabulations, assuming that future questions preserve these patterns. But this assumption is often untrue in new situations or with confabulations that human overseers are unable to identify (compare Fig. 17 of ref. 24 ). As a strong supervised baseline, we compare to an embedding regression method inspired by ref. 24 which trains a logistic regression classifier to predict whether the model correctly answered a question on the basis of the final ‘embedding’ (hidden state) of the LLM. We also use the P (True) method 24 which looks at the probability with which an LLM predicts that the next token is ‘True’ when few-shot prompted to compare a main answer with ‘brainstormed’ alternatives.

Confabulations contribute substantially to incorrect answers given by language models. We show that semantic entropy can be used to predict many incorrect model answers and to improve question-answering accuracy by refusing to answer those questions the model is uncertain about. Corresponding to these two uses, we evaluate two main metrics. First, the widely used area under the receiver operating characteristic (AUROC) curve for the binary event that a given answer is incorrect. This measure captures both precision and recall and ranges from 0 to 1, with 1 representing a perfect classifier and 0.5 representing an un-informative classifier. We also show a new measure, the area under the ‘rejection accuracy’ curve (AURAC). This studies the case in which the confabulation detection score is used to refuse to answer the questions judged most likely to cause confabulations. Rejection accuracy is the accuracy of the answers of the model on the remaining questions and the area under this curve is a summary statistic over many thresholds (representative threshold accuracies are provided in Supplementary Material ). The AURAC captures the accuracy improvement which users would experience if semantic entropy was used to filter out questions causing the highest entropy.

Detecting confabulations in QA and math

In Fig. 2 , we show that both semantic entropy and its discrete approximation outperform our best baselines for sentence-length generations. These results are averaged across datasets and provide the actual scores on the held-out evaluation dataset. We report the raw average score across held-out evaluation datasets without standard error because the distributional characteristics are more a property of the models and datasets selected than the method. Consistency of relative results across different datasets is a stronger indicator of variation in this case.

Semantic entropy outperforms leading baselines and naive entropy. AUROC (scored on the y -axes) measures how well methods predict LLM mistakes, which correlate with confabulations. AURAC (likewise scored on the y -axes) measures the performance improvement of a system that refuses to answer questions which are judged likely to cause confabulations. Results are an average over five datasets, with individual metrics provided in the Supplementary Information .

Semantic entropy greatly outperforms the naive estimation of uncertainty using entropy: computing the entropy of the length-normalized joint probability of the token sequences. Naive entropy estimation ignores the fact that token probabilities also express the uncertainty of the model over phrasings that do not change the meaning of an output.

Our methods also outperform the supervised embedding regression method both in- and out-of-distribution. In pale-yellow bars we show that embedding regression performance deteriorates when its training data do not match the deployment distribution—which mirrors the common real-world case in which there is a distribution shift between training and deployment 42 —the plotted value is the average metric for embedding regression trained on one of the four ‘off-distribution’ datasets for that evaluation. This is critical because reliable uncertainty is most important when the data distribution shifts. Semantic entropy also outperforms P (True) which is supervised ‘in-context’; that is, it is adapted to the deployment task with a few training examples provided in the LLM prompt itself. The discrete variant of semantic entropy performs similarly to our standard estimator, despite not requiring exact output probabilities.

Averaged across the 30 combinations of tasks and models we study, semantic entropy achieves the best AUROC value of 0.790 whereas naive entropy (0.691), P (True) (0.698) and the embedding regression baseline (0.687) lag behind it. Semantic entropy performs well consistently, with stable performance (between 0.78 and 0.81 AUROC) across the different model families (LLaMA, Falcon and Mistral) and scales (from 7B to 70B parameters) which we study (we report summary statistics for each dataset and model as before). Although semantic entropy outperforms the baselines across all model sizes, P (True) seems to improve with model size, suggesting that it might become more competitive for very capable honest models in settings that the model understands well (which are, however, not the most important cases to have good uncertainty). We use ten generations to compute entropy, selected using analysis in Supplementary Fig. 2 . Further results for short-phrase generations are described in Supplementary Figs. 7 – 10 .

The results in Fig. 2 offer a lower bound on the effectiveness of semantic entropy at detecting confabulations. These evaluations determine whether semantic entropy and baseline methods can detect when the answers of the model are incorrect (which we validate against human correctness evaluations in Supplementary Table 4 ). In addition to errors from confabulations (arbitrary incorrectness), this also includes other types of mistakes for which semantic entropy is not suited, such as consistent errors learned from the training data. The fact that methods such as embedding regression are able to spot other kinds of errors, not just confabulations, but still are outperformed by semantic entropy, suggests that confabulations are a principal category of errors for actual generations.

Examples of questions and answers from TriviaQA, SQuAD and BioASQ, for LLaMA 2 Chat 70B, are shown in Table 1 . These illustrate how only semantic entropy detects when the meaning is constant but the form varies (the first row of the table) whereas semantic entropy and naive entropy both correctly predict the presence of confabulations when the form and meaning vary together (second row) and predict the absence of confabulations when the form and meaning are both constant across several resampled generations (third row). In the final row, we give an example in which semantic entropy is erroneously high as a result of overly sensitive semantic clustering relative to the reference answer. Our clustering method distinguishes the answers which provide a precise date from those which only provide a year. For some contexts that would have been correct but in this context the distinction between the specific day and the year is probably irrelevant. This highlights the importance of context and judgement in clustering, especially in subtle cases, as well as the shortcomings of evaluating against fixed reference answers which do not capture the open-ended flexibility of conversational deployments of LLMs.

Detecting confabulations in biographies

Semantic entropy is most natural for sentences that express a single proposition but the idea of semantic equivalence is trickier to apply to longer passages which express many propositions which might only agree partially 43 . Nevertheless, we can use semantic entropy to detect confabulations in longer generations, such as entire paragraphs of text. To show this, we develop a dataset of biographical generations from GPT-4 (v.0613) for 21 individuals notable enough to have their own Wikipedia page but without extensive online biographies. From each biography generated by GPT-4, we automatically extract propositional factual claims about the individual (150 factual claims in total), which we manually label as true or false.

Applying semantic entropy to this problem is challenging. Naively, one might simply regenerate each sentence (conditioned on the text so far) and then compute semantic entropy over these regenerations. However, the resampled sentences often target different aspects of the biography: for example, one time describing family and the next time profession. This is analogous to the original problem semantic entropy was designed to resolve: the model is uncertain about the right ordering of facts, not about the facts themselves. To address this, we break down the entire paragraph into factual claims and reconstruct questions which might have been answered by those claims. Only then do we apply semantic entropy (Fig. 1 ) by generating three new answers to each question (selected with analysis in Supplementary Figs. 3 and 4 ) and computing the semantic entropy over those generations plus the original factual claim. We aggregate these by averaging the semantic entropy over all the questions to get an uncertainty score for each proposition, which we use to detect confabulations. Unaggregated results are shown in Supplementary Figs. 5 and 6 .

As GPT-4 did not allow access to the probability of the generation at the time of writing, we use a discrete variant of semantic entropy which makes the further approximation that we can infer a discrete empirical distribution over semantic meaning clusters from only the generations ( Methods ). This allows us to compute semantic entropy using only the black-box outputs of an LLM. However, we were unable to compute the naive entropy baseline, the standard semantic entropy estimator or the embedding regression baseline for GPT-4 without output probabilities and embeddings.

In Fig. 3 we show that the discrete variant of semantic entropy effectively detects confabulations on this dataset. Its AUROC and AURAC are higher than either a simple ‘self-check’ baseline—which just asks the LLM whether the factoid is likely to be true—or a variant of P (True) which has been adapted to work for the paragraph-length setting. Discrete semantic entropy has better rejection accuracy performance until 20% of the questions have been rejected at which point P (True) has a narrow edge. This indicates that the questions predicted to cause confabulations are indeed more likely to be wrong.

The discrete variant of our semantic entropy estimator outperforms baselines both when measured by AUROC and AURAC metrics (scored on the y -axis). The AUROC and AURAC are substantially higher than for both baselines. At above 80% of questions being answered, semantic entropy has the highest accuracy. Only when the top 20% of answers judged most likely to be confabulations are rejected does the answer accuracy on the remainder for the P (True) baseline exceed semantic entropy.

Our probabilistic approach, accounting for semantic equivalence, detects an important class of hallucinations: those that are caused by a lack of LLM knowledge. These are a substantial portion of the failures at present and will continue even as models grow in capabilities because situations and cases that humans cannot reliably supervise will persist. Confabulations are a particularly noteworthy failure mode for question answering but appear in other domains too. Semantic entropy needs no previous domain knowledge and we expect that algorithmic adaptations to other problems will allow similar advances in, for example, abstractive summarization. In addition, extensions to alternative input variations such as rephrasing or counterfactual scenarios would allow a similar method to act as a form of cross-examination 44 for scalable oversight through debate 45 .

The success of semantic entropy at detecting errors suggests that LLMs are even better at “knowing what they don’t know” than was argued by ref. 24 —they just don’t know they know what they don’t know. Our method explicitly does not directly address situations in which LLMs are confidently wrong because they have been trained with objectives that systematically produce dangerous behaviour, cause systematic reasoning errors or are systematically misleading the user. We believe that these represent different underlying mechanisms—despite similar ‘symptoms’—and need to be handled separately.

One exciting aspect of our approach is the way it makes use of classical probabilistic machine learning methods and adapts them to the unique properties of modern LLMs and free-form language generation. We hope to inspire a fruitful exchange of well-studied methods and emerging new problems by highlighting the importance of meaning when addressing language-based machine learning problems.

Semantic entropy as a strategy for overcoming confabulation builds on probabilistic tools for uncertainty estimation. It can be applied directly to any LLM or similar foundation model without requiring any modifications to the architecture. Our ‘discrete’ variant of semantic uncertainty can be applied even when the predicted probabilities for the generations are not available, for example, because access to the internals of the model is limited.

In this section we introduce background on probabilistic methods and uncertainty in machine learning, discuss how it applies to language models and then discuss our contribution, semantic entropy, in detail.

Uncertainty and machine learning

We aim to detect confabulations in LLMs, using the principle that the model will be uncertain about generations for which its output is going to be arbitrary.

One measure of uncertainty is the predictive entropy of the output distribution, which measures the information one has about the output given the input 25 . The predictive entropy (PE) for an input sentence x is the conditional entropy ( H ) of the output random variable Y with realization y given x ,

A low predictive entropy indicates an output distribution which is heavily concentrated whereas a high predictive entropy indicates that many possible outputs are similarly likely.

Aleatoric and epistemic uncertainty

We do not distinguish between aleatoric and epistemic uncertainty in our analysis. Researchers sometimes separate aleatoric uncertainty (uncertainty in the underlying data distribution) from epistemic uncertainty (caused by having only limited information) 46 . Further advances in uncertainty estimation which separate these kinds of uncertainty would enhance the potential for our semantic uncertainty approach by allowing extensions beyond entropy.

Joint probabilities of sequences of tokens

Generative LLMs produce strings of text by selecting tokens in sequence. Each token is a wordpiece that often represents three or four characters (though especially common sequences and important words such as numbers typically get their own token). To compute entropies, we need access to the probabilities the LLM assigns to the generated sequence of tokens. The probability of the entire sequence, s , conditioned on the context, x , is the product of the conditional probabilities of new tokens given past tokens, whose resulting log-probability is \(\log P({\bf{s}}| {\boldsymbol{x}})={\sum }_{i}\log P({s}_{i}| {{\bf{s}}}_{ < i},{\boldsymbol{x}})\) , where s i is the i th output token and s < i denotes the set of previous tokens.

Length normalization

When comparing the log-probabilities of generated sequences, we use ‘length normalization’, that is, we use an arithmetic mean log-probability, \(\frac{1}{N}{\sum }_{i}^{N}\log P({s}_{i}| {{\bf{s}}}_{ < i},{\boldsymbol{x}})\) , instead of the sum. In expectation, longer sequences have lower joint likelihoods because of the conditional independence of the token probabilities 47 . The joint likelihood of a sequence of length N shrinks exponentially in N . Its negative log-probability therefore grows linearly in N , so longer sentences tend to contribute more to entropy. We therefore interpret length-normalizing the log-probabilities when estimating the entropy as asserting that the expected uncertainty of generations is independent of sentence length. Length normalization has some empirical success 48 , including in our own preliminary experiments, but little theoretical justification in the literature.

Principles of semantic uncertainty

If we naively calculate the predictive entropy directly from the probabilities of the generated sequence of tokens, we conflate the uncertainty of the model over the meaning of its answer with the uncertainty over the exact tokens used to express that meaning. For example, even if the model is confident in the meaning of a generation, there are still usually many different ways for phrasing that generation without changing its meaning. For the purposes of detecting confabulations, the uncertainty of the LLM over meanings is more important than the uncertainty over the exact tokens used to express those meanings.

Our semantic uncertainty method therefore seeks to estimate only the uncertainty the LLM has over the meaning of its generation, not the choice of words. To do this, we introduce an algorithm that clusters model generations by meaning and subsequently calculates semantic uncertainty. At a high level this involves three steps:

Generation: sample output sequences of tokens from the predictive distribution of a LLM given a context x .

Clustering: cluster sequences by their meaning using our clustering algorithm based on bidirectional entailment.

Entropy estimation: estimate semantic entropy by summing probabilities of sequences that share a meaning following equation ( 2 ) and compute their entropy.

Generating a set of answers from the model

Given some context x as input to the LLM, we sample M sequences, { s (1) , …,  s ( M ) } and record their token probabilities, { P ( s (1) ∣ x ), …,  P ( s ( M ) ∣ x )}. We sample all our generations from a single model, varying only the random seed used for sampling from the token probabilities. We do not observe the method to be particularly sensitive to details of the sampling scheme. In our implementation, we sample at temperature 1 using nucleus sampling ( P  = 0.9) (ref. 49 ) and top- K sampling ( K  = 50) (ref. 50 ). We also sample a single generation at low temperature (0.1) as an estimate of the ‘best generation’ of the model to the context, which we use to assess the accuracy of the model. (A lower sampling temperature increases the probability of sampling the most likely tokens).

Clustering by semantic equivalence

To estimate semantic entropy we need to cluster generated outputs from the model into groups of outputs that mean the same thing as each other.

This can be described using ‘semantic equivalence’ which is the relation that holds between two sentences when they mean the same thing. We can formalize semantic equivalence mathematically. Let the space of tokens in a language be \({\mathcal{T}}\) . The space of all possible sequences of tokens of length N is then \({{\mathcal{S}}}_{N}\equiv {{\mathcal{T}}}^{N}\) . Note that N can be made arbitrarily large to accommodate whatever size of sentence one can imagine and one of the tokens can be a ‘padding’ token which occurs with certainty for each token after the end-of-sequence token. For some sentence \({\bf{s}}\in {{\mathcal{S}}}_{N}\) , composed of a sequence of tokens, \({s}_{i}\in {\mathcal{T}}\) , there is an associated meaning. Theories of meaning are contested 51 . However, for specific models and deployment contexts many considerations can be set aside. Care should be taken comparing very different models and contexts.

Let us introduce a semantic equivalence relation, E (  ⋅  ,  ⋅  ), which holds for any two sentences that mean the same thing—we will operationalize this presently. Recall that an equivalence relation is any reflexive, symmetric and transitive relation and that any equivalence relation on a set corresponds to a set of equivalence classes. Each semantic equivalence class captures outputs that can be considered to express the same meaning. That is, for the space of semantic equivalence classes \({\mathcal{C}}\) the sentences in the set \(c\in {\mathcal{C}}\) can be regarded in many settings as expressing a similar meaning such that \(\forall {\bf{s}},{{\bf{s}}}^{{\prime} }\in c:E({\bf{s}},{{\bf{s}}}^{{\prime} })\) . So we can build up these classes of semantically equivalent sentences by checking if new sentences share a meaning with any sentences we have already clustered and, if so, adding them into that class.

We operationalize E (  ⋅  ,  ⋅  ) using the idea of bidirectional entailment, which has a long history in linguistics 52 and natural language processing 28 , 53 , 54 . A sequence, s , means the same thing as a second sequence, s ′, only if the sequences entail (that is, logically imply) each other. For example, ‘The capital of France is Paris’ entails ‘Paris is the capital of France’ and vice versa because they mean the same thing. (See later for a discussion of soft equivalence and cases in which bidirectional entailment does not guarantee equivalent meanings).

Importantly, we require that the sequences mean the same thing with respect to the context—key meaning is sometimes contained in the context. For example, ‘Paris’ does not entail ‘The capital of France is Paris’ because ‘Paris’ is not a declarative sentence without context. But in the context of the question ‘What is the capital of France?’, the one-word answer does entail the longer answer.

Detecting entailment has been the object of study of a great deal of research in NLI 55 . We rely on language models to predict entailment, such as DeBERTa-Large-MNLI 56 , which has been trained to predict entailment, or general-purpose LLMs such as GPT-3.5 (ref. 57 ), which can predict entailment given suitable prompts.

We then cluster sentences according to whether they bidirectionally entail each other using the algorithm presented in Extended Data Fig. 1 . Note that, to check if a sequence should be added to an existing cluster, it is sufficient to check if the sequence bidirectionally entails any of the existing sequences in that cluster (we arbitrarily pick the first one), given the transitivity of semantic equivalence. If a sequence does not share meaning with any existing cluster, we assign it its own cluster.

Computing the semantic entropy

Having determined the classes of generated sequences that mean the same thing, we can estimate the likelihood that a sequence generated by the LLM belongs to a given class by computing the sum of the probabilities of all the possible sequences of tokens which can be considered to express the same meaning as

Formally, this treats the output as a random variable whose event-space is the space of all possible meaning-classes, C , a sub- σ -algebra of the standard event-space S . We can then estimate the semantic entropy (SE) as the entropy over the meaning-distribution,

There is a complication which prevents direct computation: we do not have access to every possible meaning-class c . Instead, we can only sample c from the sequence-generating distribution induced by the model. To handle this, we estimate the expectation in equation ( 3 ) using a Rao–Blackwellized Monte Carlo integration over the semantic equivalence classes C ,

where \(P({C}_{i}| {\boldsymbol{x}})=\frac{P({c}_{i}| {\boldsymbol{x}})}{{\sum }_{c}P(c| {\boldsymbol{x}})}\) estimates a categorical distribution over the cluster meanings, that is, ∑ i P ( C i ∣ x ) = 1. Without this normalization step cluster ‘probabilities’ could exceed one because of length normalization, resulting in degeneracies. Equation ( 5 ) is the estimator giving our main method that we refer to as semantic entropy throughout the text.

For scenarios in which the sequence probabilities are not available, we propose a variant of semantic entropy which we call ‘discrete’ semantic entropy. Discrete semantic entropy approximates P ( C i ∣ x ) directly from the number of generations in each cluster, disregarding the token probabilities. That is, we approximate P ( C i ∣ x ) as \({\sum }_{1}^{M}\frac{{I}_{c={C}_{i}}}{M}\) , the proportion of all the sampled answers which belong to that cluster. Effectively, this just assumes that each output that was actually generated was equally probable—estimating the underlying distribution as the categorical empirical distribution. In the limit of M the estimator converges to equation ( 5 ) by the law of large numbers. We find that discrete semantic entropy results in similar performance empirically.

We provide a worked example of the computation of semantic entropy in Supplementary Note  1 .

Semantic entropy is designed to detect confabulations, that is, model outputs with arbitrary meaning. In our experiments, we use semantic uncertainty to predict model accuracy, demonstrating that confabulations make up a notable fraction of model mistakes. We further show that semantic uncertainty can be used to improve model accuracy by refusing to answer questions when semantic uncertainty is high. Last, semantic uncertainty can be used to give users a way to know when model generations are probably unreliable.

We use the datasets BioASQ 34 , SQuAD 33 , TriviaQA 32 , SVAMP 37 and NQ-Open 35 . BioASQ is a life-sciences question-answering dataset based on the annual challenge of the same name. The specific dataset we use is based on the QA dataset from Task B of the 2023 BioASQ challenge (11B). SQuAD is a reading comprehension dataset whose context passages are drawn from Wikipedia and for which the answers to questions can be found in these passages. We use SQuAD 1.1 which excludes the unanswerable questions added in v.2.0 that are deliberately constructed to induce mistakes so they do not in practice cause confabulations to occur. TriviaQA is a trivia question-answering dataset. SVAMP is a word-problem maths dataset containing elementary-school mathematical reasoning tasks. NQ-Open is a dataset of realistic questions aggregated from Google Search which have been chosen to be answerable without reference to a source text. For each dataset, we use 400 train examples and 400 test examples randomly sampled from the original larger dataset. Note that only some of the methods require training, for example semantic entropy does not use the training data. If the datasets themselves are already split into train and test (or validation) samples, we sample our examples from within the corresponding split.

All these datasets are free-form, rather than multiple choice, because this better captures the opportunities created by LLMs to produce free-form sentences as answers. We refer to this default scenario as our ‘sentence-length’ experiments. In Supplementary Note  7 , we also present results for confabulation detection in a ‘short-phrase’ scenario, in which we constrain model answers on these datasets to be as concise as possible.

To make the problems more difficult and induce confabulations, we do not provide the context passages for any of the datasets. When the context passages are provided, the accuracy rate is too high for these datasets for the latest generations of models to meaningfully study confabulations.

For sentence-length generations we use: Falcon 39 Instruct (7B and 40B), LLaMA 2 Chat 38 (7B, 13B and 70B) and Mistral 40 Instruct (7B).

In addition to reporting results for semantic entropy, discrete semantic entropy and naive entropy, we consider two strong baselines.

Embedding regression is a supervised baseline inspired by the P (IK) method 24 . In that paper, the authors fine-tune their proprietary LLM on a dataset of questions to predict whether the model would have been correct. This requires access to a dataset of ground-truth answers to the questions. Rather than fine-tuning the entire LLM in this way, we simply take the final hidden units and train a logistic regression classifier to make the same prediction. By contrast to their method, this is much simpler because it does not require fine-tuning the entire language model, as well as being more reproducible because the solution to the logistic regression optimization problem is not as seed-dependent as the fine-tuning procedure. As expected, this supervised approach performs well in-distribution but fails when the distribution of questions is different from that on which the classifier is trained.

The second baseline we consider is the P (True) method 24 , in which the model first samples M answers (identically to our semantic entropy approach) and then is prompted with the list of all answers generated followed by the highest probability answer and a question whether this answer is “(a) True” or “(b) False”. The confidence score is then taken to be the probability with which the LLM responds with ‘a’ to the multiple-choice question. The performance of this method is boosted with a few-shot prompt, in which up to 20 examples from the training set are randomly chosen, filled in as above, but then provided with the actual ground truth of whether the proposed answer was true or false. In this way, the method can be considered as supervised ‘in-context’ because it makes use of some ground-truth training labels but can be used without retraining the model. Because of context-size constraints, this method cannot fit a full 20 few-shot examples in the context when input questions are long or large numbers of generations are used. As a result, we sometimes have to reduce the number of few-shot examples to suit the context size and we note this in the  Supplementary Material .

Entailment estimator

Any NLI classification system could be used for our bidirectional entailment clustering algorithm. We consider two different kinds of entailment detector.

One option is to use an instruction-tuned LLM such as LLaMA 2, GPT-3.5 (Turbo 1106) or GPT-4 to predict entailment between generations. We use the following prompt:

We are evaluating answers to the question {question} Here are two possible answers: Possible Answer 1: {text1} Possible Answer 2: {text2} Does Possible Answer 1 semantically entail Possible Answer 2? Respond with entailment, contradiction, or neutral.

Alternatively, we consider using a language model trained for entailment prediction, specifically the DeBERTa-large model 56 fine-tuned on the NLI dataset MNLI 58 . This builds on past work towards paraphrase identification based on embedding similarity 59 , 60 and BERT-style models 61 , 62 . We template more simply, checking if DeBERTa predicts entailment between the concatenation of the question and one answer and the concatenation of the question and another answer. Note that DeBERTa-large is a relatively lightweight model with only 1.5B parameters which is much less powerful than most of the LLMs under study.

In Supplementary Note 2 , we carefully evaluate the benefits and drawbacks of these methods for entailment prediction. We settle on using GPT-3.5 with the above prompt, as its entailment predictions agree well with human raters and lead to good confabulation detection performance.

In Supplementary Note  3 , we provide a discussion of the computational cost and choosing the number of generations for reliable clustering.

Prompting templates

We use a simple generation template for all sentence-length answer datasets:

Answer the following question in a single brief but complete sentence. Question: {question} Answer:

Metrics and accuracy measurements

We use three main metrics to evaluate our method: AUROC, rejection accuracy and AURAC. Each of these is grounded in an automated factuality estimation measurement relative to the reference answers provided by the datasets that we use.

AUROC, rejection accuracy and AURAC

First, we use the AUROC curve, which measures the reliability of a classifier accounting for both precision and recall. The AUROC can be interpreted as the probability that a randomly chosen correct answer has been assigned a higher confidence score than a randomly chosen incorrect answer. For a perfect classifier, this is 1.

Second, we compute the ‘rejection accuracy at X %’, which is the question-answering accuracy of the model on the most-confident X % of the inputs as identified by the respective uncertainty method. If an uncertainty method works well, predictions on the confident subset should be more accurate than predictions on the excluded subset and the rejection accuracy should increase as we reject more inputs.

To summarize this statistic we compute the AURAC—the total area enclosed by the accuracies at all cut-off percentages X %. This should increase towards 1 as given uncertainty method becomes more accurate and better at detecting likely-inaccurate responses but it is more sensitive to the overall accuracy of the model than the AUROC metric.

In Supplementary Note  5 , we provide the unaggregated rejection accuracies for sentence-length generations.

Assessing accuracy

For the short-phrase-length generation setting presented in Supplementary Note  7 , we simply assess the accuracy of the generations by checking if the F1 score of the commonly used SQuAD metric exceeds 0.5. There are limitations to such simple scoring rules 63 but this method is widely used in practice and its error is comparatively small on these standard datasets.

For our default scenario, the longer sentence-length generations, this measure fails, as the overlap between the short reference answer and our long model answer is invariably too small. For sentence-length generations, we therefore automatically determine whether an answer to the question is correct or incorrect by using GPT-4 to compare the given answer to the reference answer. We use the template:

We are assessing the quality of answers to the following question: {question} The expected answer is: {reference answer} The proposed answer is: {predicted answer} Within the context of the question, does the proposed answer mean the same as the expected answer? Respond only with yes or no.

We make a small modification for datasets with several reference answers: line two becomes “The following are expected answers to this question:” and the final line asks “does the proposed answer mean the same as any of the expected answers?”.

In Supplementary Note 6 , we check the quality of our automated ground-truth evaluations against human judgement by hand. We find that GPT-4 gives the best results for determining model accuracy and thus use it in all our sentence-length experiments.

In this section we describe the application of semantic entropy to confabulation detection in longer model generations, specifically paragraph-length biographies.

We introduce a biography-generation dataset—FactualBio—available alongside this paper. FactualBio is a collection of biographies of individuals who are notable enough to have Wikipedia pages but not notable enough to have large amounts of detailed coverage, generated by GPT-4 (v.0613). To generate the dataset, we randomly sampled 21 individuals from the WikiBio dataset 64 . For each biography, we generated a list of factual claims contained in each biography using GPT-4, with 150 total factual claims (the total number is only coincidentally a round number). For each of these factual claims, we manually determined whether the claim was correct or incorrect. Out of 150 claims, 45 were incorrect. As before, we apply confabulation detection to detect incorrect model predictions, even though there may be model errors which are not confabulations.

Prompting and generation

Given a paragraph-length piece of LLM-generated text, we apply the following sequence of steps:

Automatically decompose the paragraph into specific factual claims using an LLM (not necessarily the same as the original).

For each factual claim, use an LLM to automatically construct Q questions which might have produced that claim.

For each question, prompt the original LLM to generate M answers.

For each question, compute the semantic entropy of the answers, including the original factual claim.

Average the semantic entropies over the questions to arrive at a score for the original factual claim.

We pursue this slightly indirect way of generating answers because we find that simply resampling each sentence creates variation unrelated to the uncertainty of the model about the factual claim, such as differences in paragraph structure.

We decompose the paragraph into factual claims using the following prompt:

Please list the specific factual propositions included in the answer above. Be complete and do not leave any factual claims out. Provide each claim as a separate sentence in a separate bullet point.

We found that we agreed with the decompositions in all cases in the dataset.

We then generate six questions for each of the facts from the decomposition. We generate these questions by prompting the model twice with the following:

Following this text: {text so far} You see the sentence: {proposition} Generate a list of three questions, that might have generated the sentence in the context of the preceding original text, as well as their answers. Please do not use specific facts that appear in the follow-up sentence when formulating the question. Make the questions and answers diverse. Avoid yes-no questions. The answers should not be a full sentence and as short as possible, e.g. only a name, place, or thing. Use the format “1. {question} – {answer}”.

These questions are not necessarily well-targeted and the difficulty of this step is the main source of errors in the procedure. We generate three questions with each prompt, as this encourages diversity of the questions, each question targeting a different aspect of the fact. However, we observed that the generated questions will sometimes miss obvious aspects of the fact. Executing the above prompt twice (for a total of six questions) can improve coverage. We also ask for brief answers because the current version of GPT-4 tends to give long, convoluted and highly hedged answers unless explicitly told not to.

Then, for each question, we generate three new answers using the following prompt:

We are writing an answer to the question “{user question}”. So far we have written: {text so far} The next sentence should be the answer to the following question: {question} Please answer this question. Do not answer in a full sentence. Answer with as few words as possible, e.g. only a name, place, or thing.

We then compute the semantic entropy over these answers plus the original factual claim. Including the original fact ensures that the estimator remains grounded in the original claim and helps detect situations in which the question has been interpreted completely differently from the original context. We make a small modification to handle the fact that GPT-4 generations often include refusals to answer questions. These refusals were not something we commonly observe in our experiments with LLaMA 2, Falcon or Mistral models. If more than half of the answers include one of the strings ‘not available’, ‘not provided’, ‘unknown’ or ‘unclear’ then we treat the semantic uncertainty as maximal.

We then average the semantic entropies for each question corresponding to the factual claim to get an entropy for this factual claim.

Despite the extra assumptions and complexity, we find that this method greatly outperforms the baselines.

To compute semantic entailment between the original claim and regenerated answers, we rely on the DeBERTa entailment prediction model as we find empirically that DeBERTa predictions result in higher train-set AUROC than other methods. Because DeBERTa has slightly lower recall than GPT-3.5/4, we use a modified set-up for which we say the answers mean the same as each other if at least one of them entails the other and neither is seen to contradict the other—a kind of ‘non-defeating’ bidirectional entailment check rather than true bidirectional entailment. The good performance of DeBERTa in this scenario is not surprising as both factual claims and regenerated answers are relatively short. We refer to Supplementary Notes 2 and 3 for ablations and experiments regarding our choice of entailment estimator for paragraph-length generations.

We implement two baselines. First, we implement a variant of the P (True) method, which is adapted to the new setting. For each factoid, we generate a question with answers in the same way as for semantic entropy. We then use the following prompt:

Question: {question} Here are some brainstormed ideas: {list of regenerated answers} Possible answer: {original answer} Is the possible answer true? Respond with “yes” or “no”.

As we cannot access the probabilities GPT-4 assigns to predicting ‘yes’ and ‘no’ as the next token, we approximate this using Monte Carlo samples. Concretely, we execute the above prompt ten times (at temperature 1) and then take the fraction of answers which was ‘yes’ as our unbiased Monte Carlo estimate of the token probability GPT-4 assigns to ‘yes’.

As a second, simpler, baseline we check if the model thinks the answer is true. We simply ask:

Following this text: {text so far} You see this statement: {proposition} Is it likely that the statement is true? Respond with ‘yes’ or ‘no’.

It is interesting that this method ought to perform very well if we think that the model has good ‘self-knowledge’ (that is, if “models mostly know what they don’t know” 24 ) but in fact semantic entropy is much better at detecting confabulations.

Data availability

The data used for the short-phrase and sentence-length generations are publicly available and the released code details how to access it. We release a public version of the FactualBio dataset as part of the code base for reproducing the paragraph-length experiments.

Code availability

We release all code used to produce the main experiments. The code for short-phrase and sentence-length experiments can be found at github.com/jlko/semantic_uncertainty and https://doi.org/10.5281/zenodo.10964366 (ref. 65 ). The code for paragraph-length experiments can be found at github.com/jlko/long_hallucinations and https://doi.org/10.5281/zenodo.10964366 (ref. 65 ).

GPT-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2023).

Gemini: a family of highly capable multimodal models. Preprint at https://arxiv.org/abs/2312.11805 (2023).

Xiao, Y. & Wang, W. Y. On hallucination and predictive uncertainty in conditional language generation. In Proc. 16th Conference of the European Chapter of the Association for Computational Linguistics 2734–2744 (Association for Computational Linguistics, 2021).

Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T. & Saenko, K. Object hallucination in image captioning. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing (eds Riloff, E., Chiang, D., Hockenmaier, J. & Tsujii, J.) 4035–4045 (Association for Computational Linguistics, 2018).

Weiser, B. Lawyer who used ChatGPT faces penalty for made up citations. The New York Times (8 Jun 2023).

Opdahl, A. L. et al. Trustworthy journalism through AI. Data Knowl. Eng . 146 , 102182 (2023).

Shen, Y. et al. ChatGPT and other large language models are double-edged swords. Radiology 307 , e230163 (2023).

Article   PubMed   Google Scholar  

Schulman, J. Reinforcement learning from human feedback: progress and challenges. Presented at the Berkeley EECS Colloquium. YouTube www.youtube.com/watch?v=hhiLw5Q_UFg (2023).

Ji, Z. et al. Survey of hallucination in natural language generation. ACM Comput. Surv. 55 , 248 (2023).

Maynez, J., Narayan, S., Bohnet, B. & McDonald, R. On faithfulness and factuality in abstractive summarization. In Proc. 58th Annual Meeting of the Association for Computational Linguistics (eds Jurafsky, D., Chai, J., Schluter, N. & Tetreault, J.) 1906–1919 (Association for Computational Linguistics, 2020).

Filippova, K. Controlled hallucinations: learning to generate faithfully from noisy data. In Findings of the Association for Computational Linguistics: EMNLP 2020 (eds Webber, B., Cohn, T., He, Y. & Liu, Y.) 864–870 (Association for Computational Linguistics, 2020).

Berrios, G. Confabulations: a conceptual history. J. Hist. Neurosci. 7 , 225–241 (1998).

Article   CAS   PubMed   Google Scholar  

Lin, S., Hilton, J. & Evans, O. Teaching models to express their uncertainty in words. Transact. Mach. Learn. Res. (2022).

Evans, O. et al. Truthful AI: developing and governing AI that does not lie. Preprint at https://arxiv.org/abs/2110.06674 (2021).

Amodei, D. et al. Concrete problems in AI safety. Preprint at https://arxiv.org/abs/1606.06565 (2016).

Jiang, Z., Araki, J., Ding, H. & Neubig, G. How can we know when language models know? On the calibration of language models for question answering. Transact. Assoc. Comput. Linguist. 9 , 962–977 (2021).

Article   Google Scholar  

Desai, S. & Durrett, G. Calibration of pre-trained transformers. In Proc. 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (eds Webber, B., Cohn, T., He, Y. & Liu, Y.) 295–302 (Association for Computational Linguistics, 2020).

Glushkova, T., Zerva, C., Rei, R. & Martins, A. F. Uncertainty-aware machine translation evaluation. In Findings of the Association for Computational Linguistics: EMNLP 2021 (eds Moens, M-F., Huang, X., Specia, L. & Yih, S.) 3920–3938 (Association for Computational Linguistics, 2021).

Wang, Y., Beck, D., Baldwin, T. & Verspoor, K. Uncertainty estimation and reduction of pre-trained models for text regression. Transact. Assoc. Comput. Linguist. 10 , 680–696 (2022).

Baker, S. & Kanade, T. Hallucinating faces. In Proc. Fourth IEEE International Conference on Automatic Face and Gesture Recognition . 83–88 (IEEE, Catalogue no PR00580, 2002).

Eliot, L. AI ethics lucidly questioning this whole hallucinating AI popularized trend that has got to stop. Forbes Magazine (24 August 2022).

Shanahan, M. Talking about large language models. Commun. Assoc. Comp. Machinery 67 , 68–79 (2024).

MacKay, D. J. C. Information-based objective functions for active data selection. Neural Comput. 4 , 590–604 (1992).

Kadavath, S. et al. Language models (mostly) know what they know. Preprint at https://arxiv.org/abs/2207.05221 (2022).

Lindley, D. V. On a measure of the information provided by an experiment. Ann. Math. Stat. 27 , 986–1005 (1956).

Article   MathSciNet   Google Scholar  

Xiao, T. Z., Gomez, A. N. & Gal, Y. Wat zei je? Detecting out-of-distribution translations with variational transformers. In Workshop on Bayesian Deep Learning at the Conference on Neural Information Processing Systems (NeurIPS, Vancouver, 2019).

Christiano, P., Cotra, A. & Xu, M. Eliciting Latent Knowledge (Alignment Research Center, 2021); https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit .

Negri, M., Bentivogli, L., Mehdad, Y., Giampiccolo, D. & Marchetti, A. Divide and conquer: crowdsourcing the creation of cross-lingual textual entailment corpora. In Proc. 2011 Conference on Empirical Methods in Natural Language Processing 670–679 (Association for Computational Linguistics, 2011).

Honovich, O. et al. TRUE: Re-evaluating factual consistency evaluation. In Proc. Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering 161–175 (Association for Computational Linguistics, 2022).

Falke, T., Ribeiro, L. F. R., Utama, P. A., Dagan, I. & Gurevych, I. Ranking generated summaries by correctness: an interesting but challenging application for natural language inference. In Proc. 57th Annual Meeting of the Association for Computational Linguistics 2214–2220 (Association for Computational Linguistics, 2019).

Laban, P., Schnabel, T., Bennett, P. N. & Hearst, M. A. SummaC: re-visiting NLI-based models for inconsistency detection in summarization. Trans. Assoc. Comput. Linguist. 10 , 163–177 (2022).

Joshi, M., Choi, E., Weld, D. S. & Zettlemoyer, L. TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. In Proc. 55th Annual Meeting of the Association for Computational Linguistics 1601–1611 (Association for Computational Linguistics. 2017).

Rajpurkar, P., Zhang, J., Lopyrev, K. & Liang, P. SQuAD: 100,000+ questions for machine compression of text. In Proc. 2016 Conference on Empirical Methods in Natural Language Processing (eds Su, J., Duh, K. & Carreras, X.) 2383–2392 (Association for Computational Linguistics, 2016).

Tsatsaronis, G. et al. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics 16 , 138 (2015).

Article   PubMed   PubMed Central   Google Scholar  

Lee, K., Chang, M.-W. & Toutanova, K. Latent retrieval for weakly supervised open domain question answering. In Proc. 57th Annual Meeting of the Association for Computational Linguistics 6086–6096 (Association for Computational Linguistics, 2019).

Kwiatkowski, T. et al. Natural questions: a benchmark for question answering research. Transact. Assoc. Comput. Linguist. 7 , 452–466 (2019).

Patel, A., Bhattamishra, S. & Goyal, N. Are NLP models really able to solve simple math word problems? In Proc. 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Toutanova, K. et al.) 2080–2094 (Assoc. Comp. Linguistics, 2021).

Touvron, H. et al. Llama 2: open foundation and fine-tuned chat models. Preprint at https://arxiv.org/abs/2307.09288 (2023).

Penedo, G. et al. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. In Proc. 36th Conference on Neural Information Processing Systems (eds Oh, A. et al.) 79155–79172 (Curran Associates, 2023)

Jiang, A. Q. et al. Mistral 7B. Preprint at https://arxiv.org/abs/2310.06825 (2023).

Manakul, P., Liusie, A. & Gales, M. J. F. SelfCheckGPT: Zero-Resource Black-Box hallucination detection for generative large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023 (eds Bouamor, H., Pino, J. & Bali, K.) 9004–9017 (Assoc. Comp. Linguistics, 2023).

Mukhoti, J., Kirsch, A., van Amersfoort, J., Torr, P. H. & Gal, Y. Deep deterministic uncertainty: a new simple baseline. In IEEE/CVF Conference on Computer Vision and Pattern Recognition 24384–24394 (Computer Vision Foundation, 2023).

Schuster, T., Chen, S., Buthpitiya, S., Fabrikant, A. & Metzler, D. Stretching sentence-pair NLI models to reason over long documents and clusters. In Findings of the Association for Computational Linguistics: EMNLP 2022 (eds Goldberg, Y. et al.) 394–412 (Association for Computational Linguistics, 2022).

Barnes, B. & Christiano, P. Progress on AI Safety via Debate. AI Alignment Forum www.alignmentforum.org/posts/Br4xDbYu4Frwrb64a/writeup-progress-on-ai-safety-via-debate-1 (2020).

Irving, G., Christiano, P. & Amodei, D. AI safety via debate. Preprint at https://arxiv.org/abs/1805.00899 (2018).

Der Kiureghian, A. & Ditlevsen, O. Aleatory or epistemic? Does it matter? Struct. Saf. 31 , 105–112 (2009).

Malinin, A. & Gales, M. Uncertainty estimation in autoregressive structured prediction. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=jN5y-zb5Q7m (2021).

Murray, K. & Chiang, D. Correcting length bias in neural machine translation. In Proc. Third Conference on Machine Translation (eds Bojar, O. et al.) 212–223 (Assoc. Comp. Linguistics, 2018).

Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The curious case of neural text degeneration. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=rygGQyrFvH (2020).

Fan, A., Lewis, M. & Dauphin, Y. Hierarchical neural story generation. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (eds Gurevych, I. & Miyao, Y.) 889–898 (Association for Computational Linguistics, 2018).

Speaks, J. in The Stanford Encyclopedia of Philosophy (ed. Zalta, E. N.) (Metaphysics Research Lab, Stanford Univ., 2021).

Culicover, P. W. Paraphrase generation and information retrieval from stored text. Mech. Transl. Comput. Linguist. 11 , 78–88 (1968).

Google Scholar  

Padó, S., Cer, D., Galley, M., Jurafsky, D. & Manning, C. D. Measuring machine translation quality as semantic equivalence: a metric based on entailment features. Mach. Transl. 23 , 181–193 (2009).

Androutsopoulos, I. & Malakasiotis, P. A survey of paraphrasing and textual entailment methods. J. Artif. Intell. Res. 38 , 135–187 (2010).

MacCartney, B. Natural Language Inference (Stanford Univ., 2009).

He, P., Liu, X., Gao, J. & Chen, W. Deberta: decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations https://openreview.net/forum?id=XPZIaotutsD (2021).

Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33 , 1877–1901 (2020).

Williams, A., Nangia, N. & Bowman, S. R. A broad-coverage challenge corpus for sentence understanding through inference. In Proc. 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Walker, M. et al.) 1112–1122 (Assoc. Comp. Linguistics, 2018).

Yu, L., Hermann, K. M., Blunsom, P. & Pulman, S. Deep learning for answer sentence selection. Preprint at https://arxiv.org/abs/1412.1632 (2014).

Socher, R., Huang, E., Pennin, J., Manning, C. D. & Ng, A. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Proceedings of the 24th Conference on Neural Information Processing Systems (eds Shawe-Taylor, J. et al.) (2011)

He, R., Ravula, A., Kanagal, B. & Ainslie, J. Realformer: Transformer likes residual attention. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (eds Zhong, C., et al.) 929–943 (Assoc. Comp. Linguistics, 2021).

Tay, Y. et al. Charformer: fast character transformers via gradient-based subword tokenization. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=JtBRnrlOEFN (2022).

Kane, H., Kocyigit, Y., Abdalla, A., Ajanoh, P. & Coulibali, M. Towards neural similarity evaluators. In Workshop on Document Intelligence at the 32nd conference on Neural Information Processing (2019).

Lebret, R., Grangier, D. & Auli, M. Neural text generation from structured data with application to the biography domain. In Proc. 2016 Conference on Empirical Methods in Natural Language Processing (eds Su, J. et al.) 1203–1213 (Association for Computational Linguistics, 2016).

Kossen, J., jlko/semantic_uncertainty: Initial release v.1.0.0. Zenodo https://doi.org/10.5281/zenodo.10964366 (2024).

Download references

Acknowledgements

We thank G. Irving, K. Perlin, J. Richens, L. Rimell and M. Turpin for their comments or discussion related to this work. We thank K. Handa for his help with the human evaluation of our automated accuracy assessment. We thank F. Bickford Smith and L. Melo for their code review. Y.G. is supported by a Turing AI Fellowship funded by the UK government’s Office for AI, through UK Research and Innovation (grant reference EP/V030302/1), and delivered by the Alan Turing Institute.

Author information

These authors contributed equally: Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn

Authors and Affiliations

OATML, Department of Computer Science, University of Oxford, Oxford, UK

Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn & Yarin Gal

You can also search for this author in PubMed   Google Scholar

Contributions

S.F. led the work from conception to completion and proposed using bidirectional entailment to cluster generations as a way of computing entropy in LLMs. He wrote the main text, most of the Methods and Supplementary Information and prepared most of the figures. J.K. improved the mathematical formalization of semantic entropy; led the extension of semantic entropy to sentence- and paragraph-length generations; wrote the code for, and carried out, all the experiments and evaluations; wrote much of the Methods and Supplementary Information and prepared drafts of many figures; and gave critical feedback on the main text. L.K. developed the initial mathematical formalization of semantic entropy; wrote code for, and carried out, the initial experiments around semantic entropy and its variants which demonstrated the promise of the idea and helped narrow down possible research avenues to explore; and gave critical feedback on the main text. Y.G. ideated the project, proposing the idea to differentiate semantic and syntactic diversity as a tool for detecting hallucinations, provided high-level guidance on the research and gave critical feedback on the main text; he runs the research laboratory in which the work was carried out.

Corresponding author

Correspondence to Sebastian Farquhar .

Ethics declarations

Competing interests.

S.F. is currently employed by Google DeepMind and L.K. by OpenAI. For both, this paper was written under their University of Oxford affiliation. The remaining authors declare no competing interests.

Peer review

Peer review information.

Nature thanks Mirella Lapata and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended data fig. 1 algorithm outline for bidirectional entailment clustering..

Given a set of outputs in response to a context, the bidirectional entailment answer returns a set of sets of outputs which have been classified as sharing a meaning.

Supplementary information

Supplementary information.

Supplementary Notes 1–7, Figs. 1–10, Tables 1–4 and references. Includes, worked example for semantic entropy calculation, discussion of limitations and computational cost of entailment clustering, ablation of entailment prediction and clustering methods, discussion of automated accuracy assessment, unaggregated results for sentence-length generations and further results for short-phrase generations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Farquhar, S., Kossen, J., Kuhn, L. et al. Detecting hallucinations in large language models using semantic entropy. Nature 630 , 625–630 (2024). https://doi.org/10.1038/s41586-024-07421-0

Download citation

Received : 17 July 2023

Accepted : 12 April 2024

Published : 19 June 2024

Issue Date : 20 June 2024

DOI : https://doi.org/10.1038/s41586-024-07421-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Fall 2024 Semester

Undergraduate courses.

Composition courses that offer many sections (ENGL 101, 201, 277 and 379) are not listed on this schedule unless they are tailored to specific thematic content or particularly appropriate for specific programs and majors.

• 100-200 level

ENGL 151.S01: Introduction to English Studies

Tuesday and Thursday, 11 a.m.-12:15 p.m.

Sharon Smith

ENGL 151 serves as an introduction to both the English major and the discipline of English studies. In this class, you will develop the thinking, reading, writing and research practices that define both the major and the discipline. Much of the semester will be devoted to honing your literary analysis skills, and we will study and discuss texts from several different genres—poetry, short fiction, the novel, drama and film—as well as some literary criticism. As we do so, we will explore the language of the discipline, and you will learn a variety of key literary terms and concepts. In addition, you will develop your skills as both a writer and researcher within the discipline of English.

ENGL 201.ST1 Composition II: The Mind/Body Connection

In this section of English 201, students will use research and writing to learn more about problems that are important to them and articulate ways to address those problems. The course will focus specifically on issues related to the mind, the body and the relationship between them. The topics we will discuss during the course will include the correlation between social media and body image; the efficacy of sex education programs; the degree to which beliefs about race and gender influence school dress codes; and the unique mental and physical challenges faced by college students today. In this course, you will be learning about different approaches to argumentation, analyzing the arguments of others and constructing your own arguments. At the same time, you will be honing your skills as a researcher and developing your abilities as a persuasive and effective writer.

ENGL 201.S10 Composition II: Environmental Writing   

Monday/Wednesday/Friday 1-1:50 p.m.

Gwen Horsley

English 201 will help students develop the ability to think critically and analytically and to write effectively for other university courses and careers. This course will provide opportunities to develop analytical skills that will help students become critical readers and effective writers. Specifically, in this class, students will:

• Focus on the relationships between world environments, land, animals and humankind.
• Read various essays by environmental, conservational and regional authors.
• Produce student writings. 

Students will improve their writing skills by reading essays and applying techniques they witness in others’ work and those learned in class. This class is also a course in logical and creative thought. Students will write about humankind’s place in the world and our influence on the land and animals, places that hold special meaning to them or have influenced their lives and stories of their own families and their places and passions in the world. Students will practice writing in an informed and persuasive manner, in language that engages and enlivens readers by using vivid verbs and avoiding unnecessary passives, nominalizations and expletive constructions.

Students will prepare writing assignments based on readings and discussions of essays included in "Literature and the Environment " and other sources. They may use "The St. Martin’s Handbook," as well as other sources, to review grammar, punctuation, mechanics and usage as needed.

ENGL 201.13 Composition II: Writing the Environment

Tuesday and Thursday 9:30-10:45 a.m.

Paul Baggett

For generations, environmentalists have relied on the power of prose to change the minds and habits of their contemporaries. In the wake of fires, floods, storms and droughts, environmental writing has gained a new sense of urgency, with authors joining activists in their efforts to educate the public about the grim realities of climate change. But do they make a difference? Have reports of present and future disasters so saturated our airwaves that we no longer hear them? How do writers make us care about the planet amidst all the noise? In this course, students will examine the various rhetorical strategies employed by some of today’s leading environmental writers and filmmakers. And while analyzing their different arguments, students also will strengthen their own strategies of argumentation as they research and develop essays that explore a range of environmental concerns.

ENGL 201 Composition II: Food Writing

S17 Tuesday and Thursday 12:30-1:45 p.m.

S18 Tuesday and Thursday 2-3:15 p.m.

Jodi Andrews

In this composition class, students will critically analyze essays about food, food systems and environments, food cultures, the intersections of personal choice, market forces and policy and the values underneath these forces. Students will learn to better read like writers, noting authors’ purpose, audience organizational moves, sentence-level punctuation and diction. We will read a variety of essays including research-intensive arguments and personal narratives which intersect with one of our most primal needs as humans: food consumption. Students will rhetorically analyze texts, conduct advanced research, reflect on the writing process and write essays utilizing intentional rhetorical strategies. Through doing this work, students will practice the writing moves valued in every discipline: argument, evidence, concision, engaging prose and the essential research skills for the 21st century.

ENGL 221.S01 British Literature I

Michael S. Nagy

English 221 is a survey of early British literature from its inception in the Old English period with works such as "Beowulf" and the “Battle of Maldon,” through the Middle Ages and the incomparable writings of Geoffrey Chaucer and the Gawain - poet, to the Renaissance and beyond. Students will explore the historical and cultural contexts in which all assigned reading materials were written, and they will bring that information to bear on class discussion. Likely themes that this class will cover include heroism, humor, honor, religion, heresy and moral relativity. Students will write one research paper in this class and sit for two formal exams: a midterm covering everything up to that point in the semester, and a comprehensive final. Probable texts include the following:

• The Norton Anthology of English Literature: The Middle Ages. Ed. Alfred David, M. H. Abrams, and Stephen Greenblatt. 9th ed. New York: W. W. Norton & Company, 2012.
• The Norton Anthology of English Literature: The Sixteenth Century and Early Seventeenth Century. Ed. George M. Logan, Stephen Greenblatt, Barbara K Lewalski, and M. H. Abrams. 9th ed. New York: W. W. Norton & Company, 2012.
• The Norton Anthology of English Literature: The Restoration and the Eighteenth Century. Ed. George M. Logan, Stephen Greenblatt, Barbara K Lewalski, and M. H. Abrams. 9th ed. New York: W. W. Norton & Company, 2012.
• Gibaldi, Joseph. The MLA Handbook for Writers of Research Papers. 6th ed. New York: The Modern Language Association of America, 2003.
• Any Standard College Dictionary.

ENGL 240.S01 Juvenile Literature Elementary-5th Grade

Monday, Wednesday and Friday noon-12:50 p.m.

April Myrick

A survey of the history of literature written for children and adolescents, and a consideration of the various types of juvenile literature. Text selection will focus on the themes of imagination and breaking boundaries.

ENGL 240.ST1 Juvenile Literature Elementary-5th Grade

Randi Anderson

In English 240 students will develop the skills to interpret and evaluate various genres of literature for juvenile readers. This particular section will focus on various works of literature at approximately the K-5 grade level. We will read a large range of works that fall into this category, as well as information on the history, development and genre of juvenile literature.

Readings for this course include classical works such as "Hatchet," "Little Women", "The Lion, the Witch and the Wardrobe" and "Brown Girl Dreaming," as well as newer works like "Storm in the Barn," "Anne Frank’s Diary: A Graphic Adaptation," "Lumberjanes," and a variety of picture books. These readings will be paired with chapters from "Reading Children’s Literature: A Critical Introduction " to help develop understanding of various genres, themes and concepts that are both related to juvenile literature and also present in our readings.

In addition to exposing students to various genres of writing (poetry, historical fiction, non-fiction, fantasy, picture books, graphic novels, etc.) this course will also allow students to engage in a discussion of larger themes present in these works such as censorship, race and gender. Students’ understanding of these works and concepts will be developed through readings, research, discussion posts, exams and writing assignments designed to get students to practice analyzing poetry, picture books, informational books and transitional/easy readers.

ENGL 241.S01: American Literature I

Tuesday and Thursday 12:30-1:45 p.m.

This course provides a broad, historical survey of American literature from the early colonial period to the Civil War. Ranging across historical periods and literary genres—including early accounts of contact and discovery, narratives of captivity and slavery, poetry of revolution, essays on gender equality and stories of industrial exploitation—this class examines how subjects such as colonialism, nationhood, religion, slavery, westward expansion, race, gender and democracy continue to influence how Americans see themselves and their society.

Required Texts

• The Norton Anthology of American Literature: Package 1, Volumes A and B Beginnings to 1865, Ninth Edition. (ISBN 978-0-393-26454-8)

ENGL 283.S01 Introduction to Creative Writing

Steven Wingate

Students will explore the various forms of creative writing (fiction, nonfiction and poetry) not one at a time in a survey format—as if there were decisive walls of separation between then—but as intensely related genres that share much of their creative DNA. Through close reading and work on personal texts, students will address the decisions that writers in any genre must face on voice, rhetorical position, relationship to audience, etc. Students will produce and revise portfolios of original creative work developed from prompts and research. This course fulfills the same SGR #2 requirements ENGL 201; note that the course will involve a research project. Successful completion of ENGL 101 (including by test or dual credit) is a prerequisite.

ENGL 283.S02 Introduction to Creative Writing

Jodilyn Andrews

This course introduces students to the craft of writing, with readings and practice in at least two genres (including fiction, poetry and drama).

ENGL 283.ST1 Introduction to Creative Writing

Amber Jensen, M.A., M.F.A.

This course explores creative writing as a way of encountering the world, research as a component of the creative writing process, elements of craft and their rhetorical effect and drafting, workshop and revision as integral parts of writing polished literary creative work. Student writers will engage in the research practices that inform the writing of literature and in the composing strategies and writing process writers use to create literary texts. Through their reading and writing of fiction, poetry and creative nonfiction, students will learn about craft elements, find examples of those craft elements in published works and apply these elements in their own creative work, developed through weekly writing activities, small group and large group workshop and conferences with the instructor. Work will be submitted, along with a learning reflection and revision plan in each genre and will then be revised and submitted as a final portfolio at the end of the semester to demonstrate continued growth in the creation of polished literary writing.

• 300-400 level

ENGL 424.S01 Language Arts Methods grades 7-12  

Tuesday 6-8:50 p.m.

Danielle Harms

Techniques, materials and resources for teaching English language and literature to middle and secondary school students. Required of students in the English education option.

AIS/ENGL 447.S01: American Indian Literature of the Present 

Thursdays 3-6 p.m.

This course introduces students to contemporary works by authors from various Indigenous nations. Students examine these works to enhance their historical understanding of Indigenous peoples, discover the variety of literary forms used by those who identify as Indigenous writers, and consider the cultural and political significance of these varieties of expression. Topics and questions to be explored include:

• Genre: What makes Indigenous literature indigenous?
• Political and Cultural Sovereignty: Why have an emphasis on tribal specificity and calls for “literary separatism” emerged in recent decades, and what are some of the critical conversations surrounding such particularized perspectives?
• Gender and Sexuality: What are the intersecting concerns of Indigenous Studies and Women, Gender and Sexuality Studies, and how might these research fields inform one another?
• Trans-Indigeneity: What might we learn by comparing works across different Indigenous traditions, and what challenges do such comparisons present?
• Aesthetics: How do Indigenous writers understand the dynamics between tradition and creativity?
• Visual Forms: What questions or concerns do visual representations (television and film) by or about Indigenous peoples present?

Possible Texts

• Akiwenzie-Damm, Kateri and Josie Douglas (eds), Skins: Contemporary Indigenous Writing. IAD Press, 2000. (978-1864650327)
• Erdrich, Louise, The Sentence. Harper, 2021 (978-0062671127)
• Harjo, Joy, Poet Warrior: A Memoir. Norton, 2021 (978-0393248524)
• Harjo, Sterlin and Taika Waititi, Reservation Dogs (selected episodes)
• Talty, Morgan. Night of the Living Rez, 2022, Tin House (978-1953534187)
• Wall Kimmerer, Robin. Braiding Sweet Grass, Milkweed Editions (978-1571313560)
• Wilson, Diane. The Seed Keeper: A Novel. Milkweed Editions (978-1571311375)
• Critical essays by Alexie, Allen, Cohen, Cox, King, Kroeber, Ortiz, Piatote, Ross and Sexton, Smith, Taylor, Teuton, Treuer, Vizenor, and Womack.

ENGL 472.S01: Film Criticism

Tuesdays 2-4:50 p.m.

Jason McEntee

Do you have an appreciation for, and enjoy watching, movies? Do you want to study movies in a genre-oriented format (such as those we typically call the Western, the screwball comedy, the science fiction or the crime/gangster, to name a few)? Do you want to explore the different critical approaches for talking and writing about movies (such as auteur, feminist, genre or reception)?

In this class, you will examine movies through viewing and defining different genres while, at the same time, studying and utilizing different styles of film criticism. You will share your discoveries in both class discussions and short writings. The final project will be a formal written piece of film criticism based on our work throughout the semester. The course satisfies requirements and electives for all English majors and minors, including both the Film Studies and Professional Writing minors. (Note: Viewing of movies outside of class required and may require rental and/or streaming service fees.)

ENGL 476.ST1: Fiction

In this workshop-based creative writing course, students will develop original fiction based on strong attention to the fundamentals of literary storytelling: full-bodied characters, robust story lines, palpable environments and unique voices. We will pay particular attention to process awareness, to the integrity of the sentence, and to authors' commitments to their characters and the places in which their stories unfold. Some workshop experience is helpful, as student peer critique will be an important element of the class.

ENGL 479.01 Capstone: The Gothic

Wednesday 3-5:50 p.m.

With the publication of Horace Walpole’s "The Castle of Otranto " in 1764, the Gothic officially came into being. Dark tales of physical violence and psychological terror, the Gothic incorporates elements such as distressed heroes and heroines pursued by tyrannical villains; gloomy estates with dark corridors, secret passageways and mysterious chambers; haunting dreams, troubling prophecies and disturbing premonitions; abduction, imprisonment and murder; and a varied assortment of corpses, apparitions and “monsters.” In this course, we will trace the development of Gothic literature—and some film—from the eighteenth-century to the present time. As we do so, we will consider how the Gothic engages philosophical beliefs about the beautiful and sublime; shapes psychological understandings of human beings’ encounters with horror, terror, the fantastic and the uncanny; and intervenes in the social and historical contexts in which it was written. We’ll consider, for example, how the Gothic undermines ideals related to domesticity and marriage through representations of domestic abuse, toxicity and gaslighting. In addition, we’ll discuss Gothic texts that center the injustices of slavery and racism. As many Gothic texts suggest, the true horrors of human existence often have less to do with inexplicable supernatural phenomena than with the realities of the world in which we live. 

ENGL 485.S01: Undergraduate Writing Center Learning Assistants 

Flexible Scheduling

Nathan Serfling

Since their beginnings in the 1920s and 30s, writing centers have come to serve numerous functions: as hubs for writing across the curriculum initiatives, sites to develop and deliver workshops and resource centers for faculty as well as students, among other functions. But the primary function of writing centers has necessarily and rightfully remained the tutoring of student writers. This course will immerse you in that function in two parts. During the first four weeks, you will explore writing center praxis—that is, the dialogic interplay of theory and practice related to writing center work. This part of the course will orient you to writing center history, key theoretical tenets and practical aspects of writing center tutoring. Once we have developed and practiced this foundation, you will begin work in the writing center as a tutor, responsible for assisting a wide variety of student clients with numerous writing tasks. Through this work, you will learn to actively engage with student clients in the revision of a text, respond to different student needs and abilities, work with a variety of writing tasks and rhetorical situations, and develop a richer sense of writing as a complex and negotiated social process.

Graduate Courses

Engl 572.s01: film criticism, engl 576.st1 fiction.

In this workshop-based creative writing course, students will develop original fiction based on strong attention to the fundamentals of literary storytelling: full-bodied characters, robust story lines, palpable environments and unique voices. We will pay particular attention to process awareness, to the integrity of the sentence and to authors' commitments to their characters and the places in which their stories unfold. Some workshop experience is helpful, as student peer critique will be an important element of the class.

ENGL 605.S01 Seminar in Teaching Composition

Thursdays 1-3:50 p.m.

This course will provide you with a foundation in the pedagogies and theories (and their attendant histories) of writing instruction, a foundation that will prepare you to teach your own writing courses at SDSU and elsewhere. As you will discover through our course, though, writing instruction does not come with any prescribed set of “best” practices. Rather, writing pedagogies stem from and continue to evolve because of various and largely unsettled conversations about what constitutes effective writing and effective writing instruction. Part of becoming a practicing writing instructor, then, is studying these conversations to develop a sense of what “good writing” and “effective writing instruction” might mean for you in our particular program and how you might adapt that understanding to different programs and contexts.

As we read about, discuss and research writing instruction, we will address a variety of practical and theoretical topics. The practical focus will allow us to attend to topics relevant to your immediate classroom practices: designing a curriculum and various types of assignments, delivering the course content and assessing student work, among others. Our theoretical topics will begin to reveal the underpinnings of these various practical matters, including their historical, rhetorical, social and political contexts. In other words, we will investigate the praxis—the dialogic interaction of practice and theory—of writing pedagogy. As a result, this course aims to prepare you not only as a writing teacher but also as a nascent writing studies/writing pedagogy scholar.

At the end of this course, you should be able to engage effectively in the classroom practices described above and participate in academic conversations about writing pedagogy, both orally and in writing. Assessment of these outcomes will be based primarily on the various writing assignments you submit and to a smaller degree on your participation in class discussions and activities.

ENGL 726.S01: The New Woman, 1880–1900s 

Thursdays 3–5:50 p.m.

Katherine Malone

This course explores the rise of the New Woman at the end of the nineteenth century. The label New Woman referred to independent women who rebelled against social conventions. Often depicted riding bicycles, smoking cigarettes and wearing masculine clothing, these early feminists challenged gender roles and sought broader opportunities for women’s employment and self-determination. We will read provocative fiction and nonfiction by New Women writers and their critics, including authors such as Sarah Grand, Mona Caird, George Egerton, Amy Levy, Ella Hepworth Dixon, Grant Allen and George Gissing. We will analyze these exciting texts through a range of critical lenses and within the historical context of imperialism, scientific and technological innovation, the growth of the periodical press and discourse about race, class and gender. In addition to writing an argumentative seminar paper, students will complete short research assignments and lead discussion.

ENGL 792.ST1 Women in War: Female Authors and Characters in Contemporary War Lit

In this course, we will explore the voices of female authors and characters in contemporary literature of war. Drawing from various literary theories, our readings and discussion will explore the contributions of these voices to the evolving literature of war through archetypal and feminist criticism. We will read a variety of short works (both theoretical and creative) and complete works such as (selections subject to change): "Eyes Right" by Tracy Crow, "Plenty of Time When We Get Home" by Kayla Williams, "You Know When the Men are Gone" by Siobhan Fallon, "Still, Come Home" by Katie Schultz and "The Fine Art of Camouflage" by Lauren Johnson.

What’s Included: Research Paper Template

If you’re preparing to write an academic research paper, our free research paper template is the perfect starting point. In the template, we cover every section step by step, with clear, straightforward explanations and examples .

The template’s structure is based on the tried and trusted best-practice format for formal academic research papers. The template structure reflects the overall research process, ensuring your paper will have a smooth, logical flow from chapter to chapter.

The research paper template covers the following core sections:

• The title page/cover page
• Abstract (sometimes also called the executive summary)
• Section 1: Introduction 
• Section 2: Literature review 
• Section 3: Methodology
• Section 4: Findings /results
• Section 5: Discussion
• Section 6: Conclusion
• Reference list

Each section is explained in plain, straightforward language , followed by an overview of the key elements that you need to cover within each section. We’ve also included links to free resources to help you understand how to write each section.

The cleanly formatted Google Doc can be downloaded as a fully editable MS Word Document (DOCX format), so you can use it as-is or convert it to LaTeX.

FAQs: Research Paper Template

What format is the template (doc, pdf, ppt, etc.).

The research paper template is provided as a Google Doc. You can download it in MS Word format or make a copy to your Google Drive. You’re also welcome to convert it to whatever format works best for you, such as LaTeX or PDF.

What types of research papers can this template be used for?

The template follows the standard best-practice structure for formal academic research papers, so it is suitable for the vast majority of degrees, particularly those within the sciences.

Some universities may have some additional requirements, but these are typically minor, with the core structure remaining the same. Therefore, it’s always a good idea to double-check your university’s requirements before you finalise your structure.

Is this template for an undergrad, Masters or PhD-level research paper?

This template can be used for a research paper at any level of study. It may be slight overkill for an undergraduate-level study, but it certainly won’t be missing anything.

How long should my research paper be?

This depends entirely on your university’s specific requirements, so it’s best to check with them. We include generic word count ranges for each section within the template, but these are purely indicative. 

What about the research proposal?

If you’re still working on your research proposal, we’ve got a template for that here .

We’ve also got loads of proposal-related guides and videos over on the Grad Coach blog .

How do I write a literature review?

We have a wealth of free resources on the Grad Coach Blog that unpack how to write a literature review from scratch. You can check out the literature review section of the blog here.

How do I create a research methodology?

We have a wealth of free resources on the Grad Coach Blog that unpack research methodology, both qualitative and quantitative. You can check out the methodology section of the blog here.

Can I share this research paper template with my friends/colleagues?

Yes, you’re welcome to share this template. If you want to post about it on your blog or social media, all we ask is that you reference this page as your source.

Can Grad Coach help me with my research paper?

Within the template, you’ll find plain-language explanations of each section, which should give you a fair amount of guidance. However, you’re also welcome to consider our private coaching services .

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Why Are Companies That Lose Money Still So Successful?

• Vijay Govindarajan,
• Shivaram Rajgopal,
• Anup Srivastava,
• Aneel Iqbal,
• Elnaz Basirian

New research on how to identify investments that produce delayed but real profits — not just those that produce short-term accounting profits.

In a well-functioning capital market, profits should be the sole criterion for firm survival; that is, firms reporting losses should disappear. Of late, however, loss-making firms are highly sought after by investors — often more than some profitable firms. Unicorns, or startups with valuations exceeding a billion dollars, are examples of such loss-making firms. What has changed over time? When and why did losses lose their meaning? The authors’ series of new research papers provide some answers, guiding managers to make the right investments: those that produce delayed but real profits — not just those that produce short-term accounting profits but decimate shareholder wealth in long run.

In 1979, psychologists Daniel Kahneman and Amos Tversky famously posited that losses loom larger than gains in human decision-making. For example, a dollar of loss affects our behavior more than a dollar of profits . Likewise, when a firm announces losses, its stock price declines more dramatically than it increases for the same dollar amount of profits. Investors abandon and lenders tend to stop financing loss-making firms , which then start restructuring their business lines and laying off employees. Some firms go even further, conducting M&A transactions without substance and “managing earnings” to report profits instead of a loss.

• Vijay Govindarajan is the Coxe Distinguished Professor at Dartmouth College’s Tuck School of Business, an executive fellow at Harvard Business School, and faculty partner at the Silicon Valley incubator Mach 49. He is a New York Times and Wall Street Journal bestselling author. His latest book is Fusion Strategy: How Real-Time Data and AI Will Power the Industrial Future . His Harvard Business Review articles “ Engineering Reverse Innovations ” and “ Stop the Innovation Wars ” won McKinsey Awards for best article published in HBR. His HBR articles “ How GE Is Disrupting Itself ” and “ The CEO’s Role in Business Model Reinvention ” are HBR all-time top-50 bestsellers. Follow him on LinkedIn . vgovindarajan
• Shivaram Rajgopal is the Roy Bernard Kester and T.W. Byrnes Professor of Accounting and Auditing and Vice Dean of Research at Columbia Business School. His research examines financial reporting and executive compensation issues and he is widely published in both accounting and finance.
• Anup Srivastava holds Canada Research Chair in Accounting, Decision Making, and Capital Markets and is a full professor at Haskayne School of Business, University of Calgary. In a series of HBR articles, he examines the management implications of digital disruption. He specializes in the valuation and financial reporting challenges of digital companies. Follow Anup on  LinkedIn .
• Aneel Iqbal is an assistant professor at Thunderbird School of Global Management, Arizona State University. He examines the accounting measurement and financial disclosures for new-economy firms and incorporates his wide-ranging industry experience into his research and teaching. He is a seasoned accounting and finance professional with diverse experience in auditing, financial analysis, business advisory, performance management, and executive training. Follow Aneel on LinkedIn .
• Elnaz Basirian is a PhD student at the Haskayne School of Business. She examines the influence and role of intangibles in accounting and finance, aimed at improving valuation and market efficiency. She brings a decade of work experience in international financial markets. Follow Elnaz on LinkedIn .

Partner Center

share this!

June 24, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

New research uncovers hidden phenomena in ultra-clean quantum materials

by Forschungsverbund Berlin e.V. (FVB)

In a paper published today in Nature Communications , researchers unveiled previously unobserved phenomena in an ultra-clean sample of the correlated metal SrVO 3 . The study offers experimental insights that challenge the prevailing theoretical models of these unusual metals.

The international research team—from the Paul Drude Institute of Solid State Electronics (PDI), Germany; Oak Ridge National Laboratory (ORNL); Pennsylvania State University; University of Pittsburgh; the Pittsburgh Quantum Institute; and University of Minnesota—believes their findings will prompt a re-evaluation of current theories on electron correlation effects, shedding light on the origins of valuable phenomena in these systems, including magnetic properties , high-temperature superconductivity , and the unique characteristics of highly unusual transparent metals.

The perovskite oxide material SrVO 3 is classified as a Fermi liquid—a state describing a system of interacting electrons in a metal at sufficiently low temperatures.

In conventional metals, electrons that conduct electricity move independently, commonly referred to as a Fermi gas. In contrast, Fermi liquids feature significant mutual interactions between electrons, meaning the motion of one electron strongly influences the others. This collective behavior can lead to unique electronic properties with profound technological applications, providing insights into the interactions between electrons in correlated metals.

SrVO 3 serves as an ideal model system for studying electron correlation phenomena due to its crystalline and electronic simplicity. This simplicity is crucial for understanding complex phenomena such as magnetic order or superconductivity, which can complicate theoretical and experimental studies.

Another crucial factor in understanding experimental results that guide theoretical models for electron correlation effects is the presence or absence of defects in the material itself. Dr. Roman Engel-Herbert, study lead and Director of PDI in Berlin, said, "If you want to get to the bottom of one of the best-kept secrets in condensed matter physics, then you must study it in its purest form; in the absence of any extrinsic disturbance. High-quality materials that are virtually defect-free are essential. You need to synthesize ultra-clean materials."

Achieving a defect-free sample of SrVO 3 has been a seemingly insurmountable challenge until now. By employing an innovative thin film growth technique that combines the advantages of molecular beam epitaxy and chemical vapor deposition , the team achieved an unprecedented level of material purity.

Dr. Matt Brahlek, first author of the study, quantifies the improvement: "A simple measure of material purity is the ratio of how easily electricity flows at room temperature compared to low temperature, called the residual resistivity ratio, RRR value. If the metal contains many defects, RRR values are low, typically around 2–5.

"We have been able to synthesize SrVO 3 films with RRR nearly 100 times larger, 200, opening the door to study the true properties of the correlated metal SrVO 3 . In particular, the high material quality allowed accessing special regime at high magnetic fields for the first time, where surprises were found."

The interdisciplinary team of scientists was surprised to discover a series of peculiar transport phenomena that were in sharp contrast to the transport properties measured previously on highly defective samples. Their findings challenge the long-standing scientific consensus regarding SrVO 3 as a simple Fermi liquid.

Engel-Herbert explains, "This situation was very exciting but also puzzling. While we reproduced previously reported transport behavior of SrVO 3 in our highly defective samples, identical measurements in ultraclean samples with high RRR values differed."

Results from defective samples allowed a straight-forward interpretation of the results that matched theoretical expectation. These results were used as experimental evidence that the theoretical understanding correctly captured the electron correlation effects in SrVO 3 . However, the team found that measurements on the ultraclean samples could not be explained so easily.

Brahlek added, "An observation that stands out is the expectation that the number of electrons that carry electricity in a metal is independent of temperature and magnetic field. This is of course true, but the interpretation of the measured quantity is not a direct measure of the carrier concentration.

"Rather, this quantity is mixed up with other aspects of the material properties, such as how defects and temperature impact the flow of electricity. We had to delve deeper into the physics to understand what we saw. That is what makes it so important and exciting."

The researchers believe their discovery can serve as a basis to refine theoretical models and prompt a re-examination of established views and interpretations of materials exhibiting a sizeable electron correlation.

Engel-Herbert says, "Our job as experimental physicists is to push beyond the boundaries of the current understanding of nature. This is where discoveries can be made, where we advance science. As condensed matter physicists, it is key to keep perfecting our object of study by challenging ourselves to push the limits of perfecting materials.

"This can potentially give new insights into the true behavior of this class of materials and enables a comprehensive explanation of the phenomena measured and observed. It takes an interdisciplinary team of experts to do this.

"While the job is not yet completed, our results are an opportunity for the community to recalibrate their theories; re-examining materials we believed were well-understood and re-evaluate their potential for applications."

Journal information: Nature Communications

Provided by Forschungsverbund Berlin e.V. (FVB)

Explore further

Feedback to editors

Synthetic pathway for promising nitride compounds discovered

6 minutes ago

Researchers determine that crystal cells in fruit flies help transport oxygen throughout the body

7 minutes ago

A new CRISPR-driven technology for gene drive in plants

18 minutes ago

Synthetic fuels and chemicals from CO₂: Ten experiments in parallel

Researchers find intriguing new tool for tendon healing: Nanoparticles for precision drug delivery

Wind tunnel study shows hypersonic jet engine flow can be controlled optically

Re-analyzing LHC Run 2 data with cutting-edge analysis techniques allowed physicists to address old discrepancy

New method visualizes ligands on gold nanoparticles in liquid

Key mechanism of herpesvirus egress uncovered, offering new target for antiviral therapies

New strategy to obtain carbon nanotube fibers with higher dynamic strength

Relevant physicsforums posts, bulk and slab electronic structure differences.

Jun 26, 2024

Gap Equations Plotting Error in Python: Need Help Debugging

Jun 20, 2024

How Do You Solve Harper's Equation in Quantum Mechanics?

Jun 3, 2024

Latest explanation for "stability of high multiplicity states"

May 28, 2024

How to do Convergent-Close Coupling (CCC)

May 24, 2024

Anyone knows the slow light in EIT？

May 22, 2024

More from Atomic and Condensed Matter

Related Stories

How a transparent conductor responds to strain

Aug 4, 2023

Researchers show an old law still holds for quirky quantum materials

Nov 30, 2023

Quantum materials: A new state of matter with chiral properties

Feb 7, 2024

Thanks to trapped electrons, a material expected to be a conducting metal remains an insulator

Jul 14, 2023

Scientists identify mechanism that explains the characteristic properties of 'strange metals'

Aug 17, 2023

A new material for light-matter interactions

Jun 26, 2020

Recommended for you

Understanding quantum states: New research shows importance of precise topography in solid neon qubits

17 hours ago

A high-temperature superconductor with zero resistance that exhibits strange metal behavior

Jun 25, 2024

Foregoing quantum chaos to achieve high-fidelity quantum state transfer

Researchers discover new flat electronic bands, paving way for advanced quantum materials

Quantum annealer improves understanding of quantum many-body systems

Quantum effects forbid the formation of black holes from high concentrations of intense light, say physicists

Jun 24, 2024

Let us know if there is a problem with our content

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter