basis for a quality research paper

How To Write A Research Paper

Step-By-Step Tutorial With Examples + FREE Template

By: Derek Jansen (MBA) | Expert Reviewer: Dr Eunice Rautenbach | March 2024

For many students, crafting a strong research paper from scratch can feel like a daunting task – and rightly so! In this post, we’ll unpack what a research paper is, what it needs to do , and how to write one – in three easy steps. 🙂 

Overview: Writing A Research Paper

What (exactly) is a research paper.

• How to write a research paper
• Stage 1 : Topic & literature search
• Stage 2 : Structure & outline
• Stage 3 : Iterative writing
• Key takeaways

Let’s start by asking the most important question, “ What is a research paper? ”.

Simply put, a research paper is a scholarly written work where the writer (that’s you!) answers a specific question (this is called a research question ) through evidence-based arguments . Evidence-based is the keyword here. In other words, a research paper is different from an essay or other writing assignments that draw from the writer’s personal opinions or experiences. With a research paper, it’s all about building your arguments based on evidence (we’ll talk more about that evidence a little later).

Now, it’s worth noting that there are many different types of research papers , including analytical papers (the type I just described), argumentative papers, and interpretative papers. Here, we’ll focus on analytical papers , as these are some of the most common – but if you’re keen to learn about other types of research papers, be sure to check out the rest of the blog .

With that basic foundation laid, let’s get down to business and look at how to write a research paper .

Overview: The 3-Stage Process

While there are, of course, many potential approaches you can take to write a research paper, there are typically three stages to the writing process. So, in this tutorial, we’ll present a straightforward three-step process that we use when working with students at Grad Coach.

These three steps are:

• Finding a research topic and reviewing the existing literature
• Developing a provisional structure and outline for your paper, and
• Writing up your initial draft and then refining it iteratively

Let’s dig into each of these.

Need a helping hand?

Step 1: Find a topic and review the literature

As we mentioned earlier, in a research paper, you, as the researcher, will try to answer a question . More specifically, that’s called a research question , and it sets the direction of your entire paper. What’s important to understand though is that you’ll need to answer that research question with the help of high-quality sources – for example, journal articles, government reports, case studies, and so on. We’ll circle back to this in a minute.

The first stage of the research process is deciding on what your research question will be and then reviewing the existing literature (in other words, past studies and papers) to see what they say about that specific research question. In some cases, your professor may provide you with a predetermined research question (or set of questions). However, in many cases, you’ll need to find your own research question within a certain topic area.

Finding a strong research question hinges on identifying a meaningful research gap – in other words, an area that’s lacking in existing research. There’s a lot to unpack here, so if you wanna learn more, check out the plain-language explainer video below.

Once you’ve figured out which question (or questions) you’ll attempt to answer in your research paper, you’ll need to do a deep dive into the existing literature – this is called a “ literature search ”. Again, there are many ways to go about this, but your most likely starting point will be Google Scholar .

If you’re new to Google Scholar, think of it as Google for the academic world. You can start by simply entering a few different keywords that are relevant to your research question and it will then present a host of articles for you to review. What you want to pay close attention to here is the number of citations for each paper – the more citations a paper has, the more credible it is (generally speaking – there are some exceptions, of course).

Ideally, what you’re looking for are well-cited papers that are highly relevant to your topic. That said, keep in mind that citations are a cumulative metric , so older papers will often have more citations than newer papers – just because they’ve been around for longer. So, don’t fixate on this metric in isolation – relevance and recency are also very important.

Beyond Google Scholar, you’ll also definitely want to check out academic databases and aggregators such as Science Direct, PubMed, JStor and so on. These will often overlap with the results that you find in Google Scholar, but they can also reveal some hidden gems – so, be sure to check them out.

Once you’ve worked your way through all the literature, you’ll want to catalogue all this information in some sort of spreadsheet so that you can easily recall who said what, when and within what context. If you’d like, we’ve got a free literature spreadsheet that helps you do exactly that.

Step 2: Develop a structure and outline

With your research question pinned down and your literature digested and catalogued, it’s time to move on to planning your actual research paper .

It might sound obvious, but it’s really important to have some sort of rough outline in place before you start writing your paper. So often, we see students eagerly rushing into the writing phase, only to land up with a disjointed research paper that rambles on in multiple

Now, the secret here is to not get caught up in the fine details . Realistically, all you need at this stage is a bullet-point list that describes (in broad strokes) what you’ll discuss and in what order. It’s also useful to remember that you’re not glued to this outline – in all likelihood, you’ll chop and change some sections once you start writing, and that’s perfectly okay. What’s important is that you have some sort of roadmap in place from the start.

At this stage you might be wondering, “ But how should I structure my research paper? ”. Well, there’s no one-size-fits-all solution here, but in general, a research paper will consist of a few relatively standardised components:

• Introduction
• Literature review
• Methodology

Let’s take a look at each of these.

First up is the introduction section . As the name suggests, the purpose of the introduction is to set the scene for your research paper. There are usually (at least) four ingredients that go into this section – these are the background to the topic, the research problem and resultant research question , and the justification or rationale. If you’re interested, the video below unpacks the introduction section in more detail. 

The next section of your research paper will typically be your literature review . Remember all that literature you worked through earlier? Well, this is where you’ll present your interpretation of all that content . You’ll do this by writing about recent trends, developments, and arguments within the literature – but more specifically, those that are relevant to your research question . The literature review can oftentimes seem a little daunting, even to seasoned researchers, so be sure to check out our extensive collection of literature review content here .

With the introduction and lit review out of the way, the next section of your paper is the research methodology . In a nutshell, the methodology section should describe to your reader what you did (beyond just reviewing the existing literature) to answer your research question. For example, what data did you collect, how did you collect that data, how did you analyse that data and so on? For each choice, you’ll also need to justify why you chose to do it that way, and what the strengths and weaknesses of your approach were.

Now, it’s worth mentioning that for some research papers, this aspect of the project may be a lot simpler . For example, you may only need to draw on secondary sources (in other words, existing data sets). In some cases, you may just be asked to draw your conclusions from the literature search itself (in other words, there may be no data analysis at all). But, if you are required to collect and analyse data, you’ll need to pay a lot of attention to the methodology section. The video below provides an example of what the methodology section might look like.

By this stage of your paper, you will have explained what your research question is, what the existing literature has to say about that question, and how you analysed additional data to try to answer your question. So, the natural next step is to present your analysis of that data . This section is usually called the “results” or “analysis” section and this is where you’ll showcase your findings.

Depending on your school’s requirements, you may need to present and interpret the data in one section – or you might split the presentation and the interpretation into two sections. In the latter case, your “results” section will just describe the data, and the “discussion” is where you’ll interpret that data and explicitly link your analysis back to your research question. If you’re not sure which approach to take, check in with your professor or take a look at past papers to see what the norms are for your programme.

Alright – once you’ve presented and discussed your results, it’s time to wrap it up . This usually takes the form of the “ conclusion ” section. In the conclusion, you’ll need to highlight the key takeaways from your study and close the loop by explicitly answering your research question. Again, the exact requirements here will vary depending on your programme (and you may not even need a conclusion section at all) – so be sure to check with your professor if you’re unsure.

Step 3: Write and refine

Finally, it’s time to get writing. All too often though, students hit a brick wall right about here… So, how do you avoid this happening to you?

Well, there’s a lot to be said when it comes to writing a research paper (or any sort of academic piece), but we’ll share three practical tips to help you get started.

First and foremost , it’s essential to approach your writing as an iterative process. In other words, you need to start with a really messy first draft and then polish it over multiple rounds of editing. Don’t waste your time trying to write a perfect research paper in one go. Instead, take the pressure off yourself by adopting an iterative approach.

Secondly , it’s important to always lean towards critical writing , rather than descriptive writing. What does this mean? Well, at the simplest level, descriptive writing focuses on the “ what ”, while critical writing digs into the “ so what ” – in other words, the implications. If you’re not familiar with these two types of writing, don’t worry! You can find a plain-language explanation here.

Last but not least, you’ll need to get your referencing right. Specifically, you’ll need to provide credible, correctly formatted citations for the statements you make. We see students making referencing mistakes all the time and it costs them dearly. The good news is that you can easily avoid this by using a simple reference manager . If you don’t have one, check out our video about Mendeley, an easy (and free) reference management tool that you can start using today.

Recap: Key Takeaways

We’ve covered a lot of ground here. To recap, the three steps to writing a high-quality research paper are:

• To choose a research question and review the literature
• To plan your paper structure and draft an outline
• To take an iterative approach to writing, focusing on critical writing and strong referencing

Remember, this is just a b ig-picture overview of the research paper development process and there’s a lot more nuance to unpack. So, be sure to grab a copy of our free research paper template to learn more about how to write a research paper.

You Might Also Like:

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Planning to Write

Q: What is the step to conducting a quality research?

I wish to create a good and healthy food product.

Asked on 19 Aug, 2020

Quality research involves not one but many steps, the most important of which is to frame the question clearly . ‘How to create a good and healthy food product?’ is a good question, but can you narrow it down? Do you have in mind a ready-to-eat product (like a bar of chocolate) or something that requires some processing (like soybean chunks)? When you say ‘healthy,’ do you mean something like a vitamin supplement or a protein supplement? Or, something that builds immunity?

Once you formulate the research question, the next step would be to consider possible approaches, review what has been done before (products that would be competitors to your product, for example), consider what the disadvantages or shortcomings of existing products are, and so on. In formal research, this is what a review of literature is all about.

Also, in the specific question you have asked, you could think about the target consumer group (children, teenagers, the elderly, etc.).

The next step is to draw up a plan of action : how exactly you plan to go about carrying out the required research. If you already have a particular food in mind, you need to think of the evidence you would need to demonstrate that your product actually confers the benefits it promises, that it is better than its competitors. In formal research, this is what the design of experiment is all about: how you would recruit volunteer subjects, how many of them, what criteria they must meet, how much of your product will they be consuming and for how long, and so on. This corresponds to the materials and methods section of a typical research paper.

To sum up, quality research comes from a clear objective and a clear plan that ensures that the results of your research will be scientifically valid.

Hope that helps. For more information on the various aspects discussed above, you may refer to the following resources:

• How to choose a research question
• What a journal editor expects to see in a literature review
• What is experimental design?

All the best for conducting quality research and creating a good and healthy food product! Do send us a sample when it’s done – both of the paper and the product. ;-)

[with inputs by  Yateendra Joshi ]

Answered by Irfan Syed on 25 Aug, 2020

• Upvote this Answer

This content belongs to the Conducting Research Stage

Confirm that you would also like to sign up for free personalized email coaching for this stage.

Trending Searches

• Statement of the problem
• Background of study
• Scope of the study
• Types of qualitative research
• Rationale of the study
• Concept paper
• Literature review
• Introduction in research
• Under "Editor Evaluation"
• Ethics in research

Recent Searches

• Review paper
• Responding to reviewer comments
• Predatory publishers
• Scope and delimitations
• Open access
• Plagiarism in research
• Journal selection tips
• Editor assigned
• Types of articles
• "Reject and Resubmit" status
• Decision in process
• Conflict of interest

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Research paper

How to Create a Structured Research Paper Outline | Example

Published on August 7, 2022 by Courtney Gahan . Revised on August 15, 2023.

A research paper outline is a useful tool to aid in the writing process , providing a structure to follow with all information to be included in the paper clearly organized.

A quality outline can make writing your research paper more efficient by helping to:

• Organize your thoughts
• Understand the flow of information and how ideas are related
• Ensure nothing is forgotten

A research paper outline can also give your teacher an early idea of the final product.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

Research paper outline example, how to write a research paper outline, formatting your research paper outline, language in research paper outlines.

• Definition of measles
• Rise in cases in recent years in places the disease was previously eliminated or had very low rates of infection
• Figures: Number of cases per year on average, number in recent years. Relate to immunization
• Symptoms and timeframes of disease
• Risk of fatality, including statistics
• How measles is spread
• Immunization procedures in different regions
• Different regions, focusing on the arguments from those against immunization
• Immunization figures in affected regions
• High number of cases in non-immunizing regions
• Illnesses that can result from measles virus
• Fatal cases of other illnesses after patient contracted measles
• Summary of arguments of different groups
• Summary of figures and relationship with recent immunization debate
• Which side of the argument appears to be correct?

Prevent plagiarism. Run a free check.

Follow these steps to start your research paper outline:

• Decide on the subject of the paper
• Write down all the ideas you want to include or discuss
• Organize related ideas into sub-groups
• Arrange your ideas into a hierarchy: What should the reader learn first? What is most important? Which idea will help end your paper most effectively?
• Create headings and subheadings that are effective
• Format the outline in either alphanumeric, full-sentence or decimal format

There are three different kinds of research paper outline: alphanumeric, full-sentence and decimal outlines. The differences relate to formatting and style of writing.

• Alphanumeric
• Full-sentence

An alphanumeric outline is most commonly used. It uses Roman numerals, capitalized letters, arabic numerals, lowercase letters to organize the flow of information. Text is written with short notes rather than full sentences.

• Sub-point of sub-point 1

Essentially the same as the alphanumeric outline, but with the text written in full sentences rather than short points.

• Additional sub-point to conclude discussion of point of evidence introduced in point A

A decimal outline is similar in format to the alphanumeric outline, but with a different numbering system: 1, 1.1, 1.2, etc. Text is written as short notes rather than full sentences.

• 1.1.1 Sub-point of first point
• 1.1.2 Sub-point of first point
• 1.2 Second point

To write an effective research paper outline, it is important to pay attention to language. This is especially important if it is one you will show to your teacher or be assessed on.

There are four main considerations: parallelism, coordination, subordination and division.

Parallelism: Be consistent with grammatical form

Parallel structure or parallelism is the repetition of a particular grammatical form within a sentence, or in this case, between points and sub-points. This simply means that if the first point is a verb , the sub-point should also be a verb.

Example of parallelism:

• Include different regions, focusing on the different arguments from those against immunization

Coordination: Be aware of each point’s weight

Your chosen subheadings should hold the same significance as each other, as should all first sub-points, secondary sub-points, and so on.

Example of coordination:

• Include immunization figures in affected regions
• Illnesses that can result from the measles virus

Subordination: Work from general to specific

Subordination refers to the separation of general points from specific. Your main headings should be quite general, and each level of sub-point should become more specific.

Example of subordination:

Division: break information into sub-points.

Your headings should be divided into two or more subsections. There is no limit to how many subsections you can include under each heading, but keep in mind that the information will be structured into a paragraph during the writing stage, so you should not go overboard with the number of sub-points.

Ready to start writing or looking for guidance on a different step in the process? Read our step-by-step guide on how to write a research paper .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Gahan, C. (2023, August 15). How to Create a Structured Research Paper Outline | Example. Scribbr. Retrieved April 18, 2024, from https://www.scribbr.com/research-paper/outline/

Is this article helpful?

Courtney Gahan

Other students also liked, research paper format | apa, mla, & chicago templates, writing a research paper introduction | step-by-step guide, writing a research paper conclusion | step-by-step guide, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

How to Reference & Use QM in Research

Quality Matters encourages studies related to QM Rubrics and processes. Because the validity of the QM Rubrics and Peer Review process is held in place by rigorously applied Standards and procedures as defined by QM, any adaptation of the QM Rubrics or processes in research must be carefully considered and clearly identified in documents about the research results. Note that the results of research with adapted versions of the QM tools and processes compromises the validity of their application and, thus, would not indicate effects of official use of the fully annotated QM Rubric available to QM members.

For individuals and organizations planning to do research related to QM, QM Research provides:

• Consultations regarding QM-focused research
• Guidance on QM Professional Development that would provide evidence of  understanding of the QM Rubric and process 
• Information about how to obtain permission for use of the QM Rubric in research ( request permission ) 

Essential Information About Use of QM in Research

In any report or document about research that uses QM, a description of the research and how it incorporates or relates to QM should be included, along with links to the appropriate section of the QM website (e.g., the Quality Matters homepage for a reference to QM in general, or Specific Review Standards from the QM Higher Education Rubric, Seventh Edition for a reference to the Specific Review Standards from the QM Higher Education Rubric, Seventh Edition, etc.). 

The QM Rubrics are the intellectual property of Quality Matters, Inc., and, therefore, cannot be copied, published, or made publicly available. QM tracks the use of the QM Rubrics on an annual basis. If you are starting or completing research related to QM, notify Barbra Burch , QM Manager of Research and Development.

Reports on research related to QM must indicate whether the QM Rubrics and processes were used as designed by QM or adapted for purposes of the research and, if adapted, how they were adapted. 

How to Reference the QM Rubric or Standards from the QM Rubric in Research

Annotated QM Higher Ed. Rubric, Seventh Edition

• In-Text Citation for the Quality Matters Higher Education Rubric, Seventh Edition: “Quality Matters Higher Education Rubric, Seventh Edition”
• End-of-Text APA-Style Citation: “QM Higher Education Rubric, Seventh Edition, 2023. Quality Matters. Used under license. All rights reserved. Retrieved from MyQM .”

  Standards from the QM Higher Education Rubric, Seventh Edition

• In-Text Citation for the Quality Matters Higher Education Rubric, Seventh Edition:  “Standards from the Quality Matters Higher Education Rubric, Seventh Edition”
• End-of-Text APA-Style Citation: “Standards from the Quality Matters Higher Education Rubric, Seventh Edition. Quality Matters. Retrieved from Specific Review Standards from the QM Higher Education Rubric, Seventh Edition "

Adaptation of the QM Rubric for Research

Research reports including the use of an adapted form of the Standards from the QM Rubric should state, “The QM Rubrics have been developed and are regularly updated through a rigorous process that examines relevant research, data, and practitioner perspectives. They consist of Standards supported by detailed Annotations explaining the application of the Standards and are intended to support the continuous improvement of courses with constructive feedback provided by trained and certified Peer Reviewers using a specific review protocol. Additionally, the QM Rubric and QM Course Review process are focused solely on course design. QM recognizes that design is only one influential factor in online learning quality and that this quality can be impacted by other aspects of online learning, such as online teaching, student readiness, and institutional support and technology. This research includes an adapted form of the Standards from the Quality Matters Rubric . [Statement from researcher about the research that has been conducted and how it has been conducted.]”

Request Permission to Reference QM Materials in Research

Evaluating the quality of scientific research papers in entrepreneurship

• Published: 15 October 2021
• Volume 56 , pages 3013–3027, ( 2022 )

Cite this article

• Yoganandan G.   ORCID: orcid.org/0000-0002-3000-9183 1 &
• Vasan M.   ORCID: orcid.org/0000-0003-4600-4683 2  

940 Accesses

1 Altmetric

Explore all metrics

The study aims to find the quality of research papers published in the domain of entrepreneurship in India. This study covers 100 research papers. A standardized measurement tool developed by the earlier researchers was used to evaluate the research quality. The data compiled using the measurement tool were analyzed with the support of the SPSS. The statistical tools such as descriptive statistics, Friedman’s test, factor analysis, two-sample ‘t’ test, and ANOVA are applied to analyze the data. The study findings reported that the quality of research papers published in the field of entrepreneurship is not up to the quality standards. The quality of multiple-author papers is better than single-author papers. Similarly, the quality of papers published by foreign authors is comparatively better than Indian authors. Further, the quality of papers published with the combination of foreign and Indian authors is substantially good. The quality of papers published in foreign journals is higher as compared with Indian journals. Further, the standard of papers published under the qualitative approach was comparatively better than the quantitative approach. The authors developed a Conceptual Model of Process and Product of Research (YOVA model). This model clearly shows that the whole research process yields six levels of research products. The study recommended that the researchers need to go for international collaborations to improve the quality of the publication. The funding agencies, higher learning institutions and research institutions should focus on enhancing research infrastructure. The study examined the validity of research articles searched by novice researchers in India in Google by using keywords related to entrepreneurship and, as such this non-focused approach is a big impediment to quality research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Source: Developed by Authors

Similar content being viewed by others

Entrepreneurship Research in Iran: Current Trends and Future Agendas

The evolution of university entrepreneurship over the past 20 years: a bibliometric analysis

Ali Morovati Sharifabadi & Elham Mofateh Zadeh

The art of crafting a systematic literature review in entrepreneurship research

Sascha Kraus, Matthias Breier & Sonia Dasí-Rodríguez

Abdel Moneim Aly: Quality of medical journals with special reference to the Eastern Mediterranean health journal. Saudi Med. J. 25 (1), 18–20 (2004)

Google Scholar  

Adams, J.: The fourth age of research. Nature 497 , 557–560 (2013)

Article   Google Scholar  

Akkerman, S., Admiraal, W., Brekelmans, M., Oost, H.: Auditing quality of research in social sciences. Qual. Quant. 42 (2), 257–274 (2006)

Aphinyanaphongs, Y., Tsamardinos, I., Statnikov, A., Hardin, D., Aliferis, C.F.: Text categorization models for high-quality article retrieval in internal medicine. J. Am. Med. Inform. Assoc. 12 (2), 207–217 (2005)

Bonaccorsi, A., Cicero, T., Ferrara, A., Malgarini. M., (2015). Journal Ratings as Predictors of Articles Quality in Arts, Humanities and Social Sciences: An Analysis based on the Italian Research Evaluation Exercise. F1000Research , 4, 196.

Bornmann, L., Schier, H., Marx, W., Daniel, H.-D.: Does the h index for assessing single publications really work? a case study on papers published in chemistry”. Scientometrics 89 , 835–843 (2011)

Bornmann, L., Schier, H., Marx, W., Daniel, H.-D.: What factors determine citation counts of publications in chemistry besides their quality? J. Informet. 6 , 11–18 (2012)

Bornmanna, L., Daniel, H.-D.: The Citation speed index: a useful bibliometric indicator to add to the h index. J. Informet. 4 , 444–446 (2010)

Chawla, R., Gupta, M., Anand, N.: Ranking of Indian journals with popular international journals: a comparative study. Int. J. Sci. Technol. Res. 9 (3), 72–81 (2020)

Cho, M.K., Bero, L.A.: Instruments for assessing the quality of drug studies published in the medical literature. J. Am. Med. Assoc. 272 , 101–104 (1994)

Djulbegovic, B., Lacevic, M., Cantor, A.: The uncertainty principle and industry-sponsored research. Lancet 356 , 635–638 (2000)

Durieux, V., Gevenois, P.A.: Bibliometric indicators: quality measurements of scientific publication. Radiology 255 (2), 342–351 (2010)

Ghazavi, R., Taheri, B., Ashrafi-rizi, H.: Article quality indicator: proposing a new indicator for measuring article quality in scopus and web of science. J. Sci. Res. 8 (1), 9–17 (2019)

Gupta, P., Kaur, G., Sharma, B., Shah, D., Choudhury, P.: What is submitted and what gets accepted in Indian Pediatrics: analysis of submissions, review process, decision making, and criteria for rejection. Indian Pediatr. 43 (6), 479–489 (2006)

Gupta, R., Tiwari, R., Mueen Ammed, K.K.: Dengue research in India: a scientometric analysis of publications - 2003–12. Int. J. Med. Public Health 4 (1), 1–9 (2014)

Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E.: Multivariate Data Analysis, 7th edn. Pearson Education, U.K (2010)

Hair, H.H., Jr, Black, W.C., Babin, B.J., Anderson, R.E., & Tatham, R.L., (2009). Multivariate Data Analysis . London, U.K: Pearson Education.

Hudson, J.: Trends in multi-authored papers in economics. J. Econom. Perspect. 10 (3), 153–158 (1996)

Inayatullah, S., Fitzgerald, J.: Gene Discourses: politics, culture, law, and futures. Technol. Forecast. Soc. Chang. 52 (2–3), 161–183 (1996)

James, J.B., Vincent, C.B.: Perception of journal quality. Account. Rev. 49 (2), 360–362 (1974)

Lawani, S.M.: Some bibliometric correlates of quality in scientific research. Scientometrics 9 , 13–25 (1986)

Li, E.Y., Liao, C.H., Yen, H.R.: Co-authorship networks and research impact: a social capital perspective. Res. Policy 42 (9), 1515–1530 (2013)

Low, W.Y., Ng, K.H., Kabir, M.A., et al.: Trend and impact of international collaboration in clinical medicine papers published in Malaysia. Scientometrics 98 , 1521–1533 (2014)

Mårtensson, P., Fors, U., Wallin, S.-B., Zander, U., Nilsson, G.H.: Evaluating research: a multidisciplinary approach to assessing research practice and quality. Res. Policy 45 , 593–603 (2015)

Mattsson, P., Laget, P., Nilsson, A., & Sundberg, C.J. (2008). Intra-EU Vs. Extra-EU Scientific Co-publication Patterns in EU. Scientometrics , 75(3), 555–574.

Mays, N., & Pope, C. (2000). Quality in Qualitative Health Research. In: Mays, N., and Pope, C. Qualitative Research in Health Care . 2nd ed. London: BMJ Books.

Michael, J. Cuellar, Duane, P. Truex, & Hirotoshi Takeda. (2016). Can we trust journal ranking to assess article quality? Proceedings of the Twenty-second Americas Conference on Information Systems , San Diego. 1–11.

Nanjundaiah & Dinesh, K.S. : Ranking and comparison of journals published in india with special reference to humanities and social sciences. Int. J. Inf. Dissem. Technol. 6 (4), 251–257 (2016)

National Center for the Dissemination of Disability Research - NCDDR. (2002). A Technical Brief: What Are the Standards for Quality Research? Southwest Educational Development Laboratory.

Popay, R., Rogers, A., Williams, G.: Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Research 8 , 341–351 (1998)

Ram, S.: A quantitative assessment of “chikungunya” research publications, 2004–2013. Trop. J. Med. Res. 19 , 52–60 (2016)

RAND. (2010). Technical Report: Standards for High-Quality Research and Analysis . RAND Corporation.

Salimi, N.: Quality assessment of scientific outputs using the BWM. Scientometrics 112 , 195–213 (2017)

Seglen, P.O.: Why the impact factor of journals should not be used for evaluating research? BMJ 314 (7079), 497 (1997)

Sendhilkumar, S., Elakkiya, E., Mahalakshmi, G.S.: Citation semantic based approaches to identify article quality. Comp Sci Inform Technol 1 , 411–420 (2013)

Sheikh Tariq Mahmood: Factors affecting the quality of research in education: student’s perceptions. J. Educ. Pract. 2 (11 & 12), 34–39 (2011)

Shewfelt, R.L.: What is quality? Postharvest Biol. Technol. 15 , 197–200 (1999)

Stokols, D., Harvey, R., Gress, J., Fuqua, J., Phillips, K.: In vivo studies of transdisciplinary scientific collaboration: lessons learned and implications for active living research. Am. J. Prev. Med. 28 (2), 202–213 (2005)

Moyses Szklo. (2006). Quality of Scientific Articles. Rev Saúde Pública , 40 (N Esp), 30–35.

Timmer, A., Sutherland, L.R., Hilsden, R.J.: Development and evaluation of a quality score for abstracts. BMC Med Res Methodol 3 , 2 (2003)

Victor, B.G., Hodge, D.R., Perron, B.E., Vaughn, M.G., Salas-Wright, C.P.: The rise of co-authorship in social work scholarship: a longitudinal study of collaboration and article quality, 1989–2013. Br. J. Soc. Work. 47 (8), 1–16 (2016)

Walter, J., Lechner, C., Kellermanns, F.W.: Knowledge transfer between and within alliance partners: private versus collective benefits of social capital. J. Bus. Res. 60 (7), 698–710 (2007)

Wasko, M.M., Faraj, S.: Why should I share? examining social capital and knowledge contribution in electronic networks of practice. MIS Q. 29 (1), 35–57 (2005)

Welch, C., Piekkari, R.: How should we (not) judge the ‘quality’ of qualitative research? a re-assessment of current evaluative criteria in international business. J. World Bus. 52 (5), 714–725 (2017)

Yokuş, G., Akdaği, H.: Identifying quality criteria of a scientific research adopted by academic community: a case study. Int. J. Eurasia Soc. Sci. 10 (36), 516–527 (2019)

Zuber-Skerritt, O., Fletcher, M.: The quality of an action research thesis in the social sciences. Qual. Assur. Educ. 15 (4), 413–436 (2007)

Download references

No funding received.

Author information

Authors and affiliations.

Department of Management Studies, Periyar University, Salem, Tamilnadu, 636 011, India

Yoganandan G.

Department of Commerce, National College (Autonomous), Tiruchirappalli, Tamilnadu, 620 001, India

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Vasan M. .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Yoganandan, G., Vasan, M. Evaluating the quality of scientific research papers in entrepreneurship. Qual Quant 56 , 3013–3027 (2022). https://doi.org/10.1007/s11135-021-01254-z

Download citation

Accepted : 30 September 2021

Published : 15 October 2021

Issue Date : October 2022

DOI : https://doi.org/10.1007/s11135-021-01254-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Entrepreneurship
• Quality criteria
• Research collaboration
• Research quality
• Find a journal
• Publish with us
• Track your research

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: physcene: physically interactable 3d scene synthesis for embodied ai.

Abstract: With recent developments in Embodied Artificial Intelligence (EAI) research, there has been a growing demand for high-quality, large-scale interactive scene generation. While prior methods in scene synthesis have prioritized the naturalness and realism of the generated scenes, the physical plausibility and interactivity of scenes have been largely left unexplored. To address this disparity, we introduce PhyScene, a novel method dedicated to generating interactive 3D scenes characterized by realistic layouts, articulated objects, and rich physical interactivity tailored for embodied agents. Based on a conditional diffusion model for capturing scene layouts, we devise novel physics- and interactivity-based guidance mechanisms that integrate constraints from object collision, room layout, and object reachability. Through extensive experiments, we demonstrate that PhyScene effectively leverages these guidance functions for physically interactable scene synthesis, outperforming existing state-of-the-art scene synthesis methods by a large margin. Our findings suggest that the scenes generated by PhyScene hold considerable potential for facilitating diverse skill acquisition among agents within interactive environments, thereby catalyzing further advancements in embodied AI research. Project website: this http URL .

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

This paper is in the following e-collection/theme issue:

Published on 17.4.2024 in Vol 26 (2024)

Service Quality and Residents’ Preferences for Facilitated Self-Service Fundus Disease Screening: Cross-Sectional Study

Authors of this article:

Original Paper

• Senlin Lin 1, 2, 3 * , MSc   ; 
• Yingyan Ma 1, 2, 3, 4 * , PhD   ; 
• Yanwei Jiang 5 * , MPH   ; 
• Wenwen Li 6 , PhD   ; 
• Yajun Peng 1, 2, 3 , BA   ; 
• Tao Yu 1, 2, 3 , BA   ; 
• Yi Xu 1, 2, 3 , MD   ; 
• Jianfeng Zhu 1, 2, 3 , MD   ; 
• Lina Lu 1, 2, 3 , MPH   ; 
• Haidong Zou 1, 2, 3, 4 , MD  

1 Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China

2 National Clinical Research Center for Eye Diseases, Shanghai, China

3 Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China

4 Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China

5 Shanghai Hongkou Center for Disease Control and Prevention, Shanghai, China

6 School of Management, Fudan University, Shanghai, China

*these authors contributed equally

Corresponding Author:

Haidong Zou, MD

Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital

School of Medicine

Tongji University

No 1440, Hongqqiao Road

Shanghai, 200336

Phone: 86 02162539696

Email: [email protected]

Background: Fundus photography is the most important examination in eye disease screening. A facilitated self-service eye screening pattern based on the fully automatic fundus camera was developed in 2022 in Shanghai, China; it may help solve the problem of insufficient human resources in primary health care institutions. However, the service quality and residents’ preference for this new pattern are unclear.

Objective: This study aimed to compare the service quality and residents’ preferences between facilitated self-service eye screening and traditional manual screening and to explore the relationships between the screening service’s quality and residents’ preferences.

Methods: We conducted a cross-sectional study in Shanghai, China. Residents who underwent facilitated self-service fundus disease screening at one of the screening sites were assigned to the exposure group; those who were screened with a traditional fundus camera operated by an optometrist at an adjacent site comprised the control group. The primary outcome was the screening service quality, including effectiveness (image quality and screening efficiency), physiological discomfort, safety, convenience, and trustworthiness. The secondary outcome was the participants’ preferences. Differences in service quality and the participants’ preferences between the 2 groups were compared using chi-square tests separately. Subgroup analyses for exploring the relationships between the screening service’s quality and residents’ preference were conducted using generalized logit models.

Results: A total of 358 residents enrolled; among them, 176 (49.16%) were included in the exposure group and the remaining 182 (50.84%) in the control group. Residents’ basic characteristics were balanced between the 2 groups. There was no significant difference in service quality between the 2 groups (image quality pass rate: P =.79; average screening time: P =.57; no physiological discomfort rate: P =.92; safety rate: P =.78; convenience rate: P =.95; trustworthiness rate: P =.20). However, the proportion of participants who were willing to use the same technology for their next screening was significantly lower in the exposure group than in the control group ( P <.001). Subgroup analyses suggest that distrust in the facilitated self-service eye screening might increase the probability of refusal to undergo screening ( P =.02).

Conclusions: This study confirms that the facilitated self-service fundus disease screening pattern could achieve good service quality. However, it was difficult to reverse residents’ preferences for manual screening in a short period, especially when the original manual service was already excellent. Therefore, the digital transformation of health care must be cautious. We suggest that attention be paid to the residents’ individual needs. More efficient man-machine collaboration and personalized health management solutions based on large language models are both needed.

Introduction

Vision impairment and blindness are caused by a variety of eye diseases, including cataracts, glaucoma, uncorrected refractive error, age-related macular degeneration, diabetic retinopathy, and other eye diseases [ 1 ]. They not only reduce economic productivity but also harm the quality of life and increase mortality [ 2 - 6 ]. In 2020, an estimated 43.3 million individuals were blind, and 1.06 billion individuals aged 50 years and older had distance or near vision impairment [ 7 ]. With an increase in the aging population, the number of individuals affected by vision loss has increased substantially [ 1 ].

High-quality public health care for eye disease prevention, such as effective screening, can assist in eliminating approximately 57% of all blindness cases [ 8 ]. Digital technologies, such as telemedicine, 5G telecommunications, the Internet of Things, and artificial intelligence (AI), have provided the potential to improve the accessibility, availability, and productivity of existing resources and the overall efficiency of eye care services [ 9 , 10 ]. The use of digital technology not only reduces the cost of eye disease screening and improves its efficiency, but also assists residents living in remote areas to gain access to eye disease screening [ 11 - 13 ]. Therefore, an increasing number of countries (or regions) are attempting to establish eye screening systems based on digital technology [ 9 ].

Fundus photography is the most important examination in eye disease screening because the vast majority of diagnoses of blinding retinal diseases are based on fundus photographs. Diagnoses can be made by human experts or AI software. However, traditional fundus cameras must be operated by optometrists, who are usually in short supply in primary health care institutions when faced with the large demand for screening services.

Fortunately, the fully automatic fundus camera has been developed on the basis of digital technologies including AI, industrial automation, sensors, and voice navigation. It can automatically identify the person’s left and right eyes, search for pupils, adjust the lens position and shooting focus, and provide real-time voice feedback during the process, helping the residents to understand the current inspection steps clearly and cooperatively complete the inspection. Therefore, a facilitated self-service eye screening pattern has been newly established in 2022 in Shanghai, China.

However, evidence is inadequate about whether this new screening pattern performs well and whether the residents prefer it. Therefore, this cross-sectional study aims to compare the service quality and residents’ preferences of this new screening pattern with that of the traditional screening pattern. We aimed to (1) investigate whether the facilitated self-service eye screening can achieve service quality similar to that of traditional manual screening, (2) compare residents’ preferences between the facilitated self-service eye screening and traditional manual screening, and (3) explore the relationship between the screening service quality and residents’ preferences.

Study Setting

This study was conducted in Shanghai, China, in 2022. Since 2010, Shanghai has conducted an active community-based fundus disease telemedicine screening program. After 2018, an AI model was adopted ( Figure 1 ). At the end of 2021, the fully automatic fundus camera was adopted, and the facilitated self-service fundus disease screening pattern was established ( Figure 1 ). Within this new pattern, residents could perform fundus photography by themselves without professionals’ assistance ( Multimedia Appendix 1 ). The fundus images were sent to the cloud server center of the AI model, and the screening results were fed back immediately.

Study Design

We conducted a cross-sectional study at 2 adjacent screening sites. These 2 sites were expected to be very similar in terms of their socioeconomic and educational aspects since they were located next to each other. One site provided facilitated self-service fundus disease screening, and the residents who participated therein comprised the exposure group; the other site provided screening with a traditional fundus camera operated by an optometrist, and the residents who participated therein comprised the control group. All the adult residents could participant in our screening program, but their data were used for analysis only if they signed the informed consent form. Residents could opt out of the study at any time during the screening.

In the exposure group, the residents were assessed using an updated version of the nonmydriatic fundus camera Kestrel 3100m (Shanghai Top View Industrial Co Ltd) with a self-service module. In the process of fundus photography, the residents pressed the “Start” button by themselves. All checking steps (including focusing, shooting, and image quality review) were undertaken automatically by the fundus camera ( Figure 2 ). Screening data were transmitted to the AI algorithm on a cloud-based server center through the telemedicine platform, and the screening results were fed back immediately. Residents were fully informed that the assessment was fully automated and not performed by the optometrist.

In the control group, the residents were assessed using the basic version of the same nonmydriatic fundus camera. The optical components were identical to those in the exposure group but without the self-service module. In the process of fundus photography, all steps were carried out by the optometrist (including focusing, shooting, and image quality review). Screening data were transmitted to the AI algorithm on a cloud-based server center through the telemedicine platform, and the screening results were fed back immediately. Residents were also fully informed.

Measures and Outcomes

The primary outcome was the screening service’s quality. Based on the World Health Organization’s recommendations for the evaluation of AI-based medical devices [ 14 ] and the European Union’s Assessment List for Trustworthy Artificial Intelligence [ 15 ], 5 dimensions were selected to reflect the service quality of eye disease screening: effectiveness, physiological discomfort, safety, convenience, and trustworthiness.

Furthermore, effectiveness was based on 2 indicators: image quality and screening efficiency. A staff member recorded the time required for each resident to take fundus photographs (excluding the time taken for diagnosis) at the screening site. Then, a professional ophthalmologist evaluated the quality of each fundus photograph after the on-site experiment. The ophthalmologist was blinded to the grouping of participants. Image quality was assessed on the basis of the image quality pass rate, expressed as the number of eyes with high-quality fundus images per 100 eyes. Screening efficiency was assessed on the basis of the average screening time, expressed as the mean of the time required for each resident to take fundus photographs.

To assess physiological discomfort, safety, convenience, and trustworthiness of screening services, residents were asked to finish a questionnaire just after they received the screening results. A 5-point Likert scale was adopted for each dimension, from the best to the worst, except for the physiological discomfort ( Multimedia Appendix 2 ). A no physiological discomfort rate was expressed as the number of residents who chose the “There is no physiological discomfort during the screening” per 100 individuals in each group. Safety rate is expressed as the number of residents who chose “The screening is very safe” or “The screening is safe” per 100 individuals in each group. Convenience rate is expressed as the number of residents who chose “The screening is very convenient” or “The screening is convenient” per 100 individuals in each group. The trustworthiness rate is expressed as the number of residents who chose “The screening result is very trustworthy” or “The screening result is trustworthy” per 100 individuals in each group.

The secondary outcome was the preference rate, expressed as the number of residents who were willing to use the same technology for their next screening per 100 individuals. In detail, in the exposure group, the preference rate was expressed as the number of the residents who preferred facilitated self-service eye screening per 100 individuals, while in the control group, it was expressed as the number of residents who preferred traditional manual screening per 100 individuals.

To understand the residents’ preference, a video displaying the processes of both facilitated self-service eye screening and traditional manual screening was shown to the residents. Then, the following question was asked: “At your next eye disease screening, you can choose either facilitated self-service eye screening or traditional manual screening. Which one do you prefer?” A total of 4 alternatives were set: “Prefer traditional manual screening,” “Prefer facilitated self-service eye screening,” “Both are acceptable,” and “Neither is acceptable (Refusal of screening).” Each resident could choose only 1 option, which best reflected their preference.

Sample Size

The rule of events per variable was used for sample size estimation. In this study, 2 logit models were established for the 2 groups separately, each containing 8 independent variables. We set 10 events per variable in general. According to a previous study [ 16 ], when the decision-making process had high uncertainty, the proportion of individuals who preferred the algorithms was about 50%. This led us to arrive at a sample size of 160 (8 variables multiplied by 10 events each, with 50% of individuals potentially preferring facilitated screening [ie, 50% of 8×10]) for each group.

Every dimension of the screening service quality and the preference rate were calculated separately. Chi-square and t tests were used to test whether the service quality or the residents’ preferences differed between the 2 groups. A total of 7 hypotheses were tested, as shown in Textbox 1 .

• H1: image quality pass rate exposure group ≠ image quality pass rate control group H0: image quality pass rate exposure group =image quality pass rate control group
• H1: screening time exposure group ≠screening time control group H0: screening time exposure group =screening time control group
• H1: no discomfort rate exposure group ≠no discomfort rate control group H0: no discomfort rate exposure group = no discomfort rate control group
• H1: safety rate exposure group ≠safety rate control group H0: safety rate exposure group = safety rate control group
• H1: convenience rate exposure group ≠convenience rate control group H0: convenience rate exposure group = convenience rate control group
• H1: trustworthiness rate exposure group ≠trustworthiness rate control group H0: trustworthiness rate exposure group = trustworthiness rate control group
• H1: preference rate exposure group ≠preference rate control group H0: preference rate exposure group = preference rate control group

If any of the hypotheses among hypotheses 1-6 ( Textbox 1 ) were significant, it indicated that the service quality was different between facilitated self-service eye screening and traditional manual screening. If hypothesis 7 was significant, it meant that the residents’ preference for facilitated self-service eye screening was different from that for traditional manual screening.

Additionally, subgroup analyses in the exposure and control groups were conducted to explore the relationships between the screening service quality and the residents’ preferences, using generalized logit models. The option “Prefer facilitated self-service eye screening” was used as the reference level for the dependent variable in the models. The independent variables included age, sex, image quality, screening efficiency, physiological discomfort, safety, convenience, and trustworthiness. All statistics were performed using SAS (version 9.4; SAS Institute).

Ethical Considerations

The study adhered to the ethical principles of the Declaration of Helsinki and was approved by the Shanghai General Hospital Ethics Committee (2022SQ272). All participants provided written informed consent before participating in this study. The study data were anonymous, and no identification of individual participants in any images of the manuscript or supplementary material is possible.

Participants’ Characteristics

A total of 358 residents enrolled; among them, 176 (49.16%) were in the exposure group and the remaining 182 (50.84%) were in the control group. Residents’ basic characteristics were balanced between the 2 groups. The mean age was 65.05 (SD 12.28) years for the exposure group and 63.96 (SD 13.06) years for the control group; however, this difference was nonsignificant ( P =.42). The proportion of women was 67.05% (n=118) for the exposure group and 62.09% (n=113) for the control group; this difference was also nonsignificant between the 2 groups ( P =.33).

Screening Service Quality

In the exposure group, high-quality fundus images were obtained for 268 out of 352 eyes (image quality pass rate=76.14%; Figure 3 ). The average screening time was 81.03 (SD 36.98) seconds ( Figure 3 ). In the control group, high-quality fundus images were obtained for 274 out of 364 eyes (image quality pass rate=75.27%; Figure 3 ). The average screening time was 78.22 (SD 54.01) seconds ( Figure 3 ). There was no significant difference in the image quality pass rate ( χ 2 1 =0.07, P =.79) and average screening time ( t 321.01 =–0.58 [Welch–Satterthwaite–adjusted df ], P =.56) between the 2 groups ( Figure 3 ).

For the other dimensions, detailed information is shown in Figure 3 . There were no significant differences between any of these rates between the 2 groups (no physiological discomfort rate: χ 2 1 =0.01, P =.92; safety rate: χ 2 1 =0.08, P =.78; convenience rate: χ 2 1 =0.004, P =.95; trustworthiness rate: χ 2 1 =1.63, P =.20).

Residents’ Preferences

In the exposure group, 120 (68.18%) residents preferred traditional manual screening, 19 (10.80%) preferred facilitated self-service eye screening, 19 (10.80%) preferred both, and the remaining 18 (10.23%) preferred neither. In the control group, 123 (67.58%) residents preferred traditional manual screening, 14 (7.69%) preferred facilitated self-service eye screening, 20 (10.99%) preferred both, and the remaining 25 (13.74%) preferred neither.

The proportion of residents who chose the category “Prefer facilitated self-service eye screening” in the exposure group was significantly lower than that of residents who chose the category “Prefer traditional manual screening” in the control group ( χ 2 1 =120.57, P <.001; Figure 3 ).

Subgroup Analyses

In the exposure group, 4 generalized logit models were generated ( Table 1 ). Regarding the effectiveness of facilitated self-service eye screening, neither the image quality nor the screening time had an impact on the residents’ preferences. Regarding the other dimensions for facilitated self-service eye screening service quality, models 3 and 4 demonstrated that distrust in the results of facilitated self-service eye screening might decrease the probability of preferring this screening service and increase the probability of preferring neither of the 2 screening services.

a Age and gender were adjusted in model 1. Age, gender, image quality, and screening efficiency were adjusted in model 2. Age, gender, physiological discomfort, safety, convenience, and trustworthiness were adjusted in model 3. Age, gender, image quality, screening efficiency, physiological discomfort, safety, convenience, and trustworthiness were adjusted in model 4.

b In the exposure group, distrust in the results of facilitated self-service eye screening might decrease the probability of preferring this screening service and increase the probability of preferring neither the traditional nor the facilitated self-service screening services.

c Not available.

In the control group, another 4 generalized logit models were generated ( Table 2 ). Men were more likely to choose a preference both screening services. The probability of preferring manual screening might increase with age, as long as the probability of preferring facilitated self-service eye screening decreased. Regarding the effectiveness of traditional manual screening, neither the image quality pass rate nor the screening time had an impact on the residents’ preferences. For the other dimensions of the quality of traditional manual screening, models 7 and 8 showed that if the residents feel unsafe about traditional manual screening, their preference for traditional manual screening might decrease, and they might turn to facilitated self-service eye screening.

a Age and gender were adjusted in model 5. Age, gender, image quality, and screening efficiency were adjusted in model 6. Age, gender, physiological discomfort, safety, convenience, and trustworthiness were adjusted in model 7. Age, gender, image quality, screening efficiency, physiological discomfort, safety, convenience, and trustworthiness were adjusted in model 8.

b In the control group, if the residents feel unsafe about traditional manual screening, their preference for traditional manual screening might decrease, and they might turn to facilitated self-service eye screening.

A new fundus disease screening pattern was established using the fully automatic fundus camera without any manual intervention. Our findings suggest that facilitated self-service eye screening can achieve a service quality similar to that of traditional manual screening. The study further evaluated the residents’ preferences and associated factors for the newly established self-service fundus disease screening. Our study found that the residents’ preference for facilitated self-service eye screening is significantly less than that for traditional manual screening. This implies that the association between the service quality of the screening technology and residents’ preferences was weak, suggesting that aversion to the algorithm might exist. In addition, the subgroup analyses suggest that even the high quality of facilitated self-service eye screening cannot increase the residents’ preference for this new screening pattern. Worse still, distrust in the results of this new pattern may lead to lower usage of eye disease screening services as a whole. To the best of our knowledge, this study is one of the first to evaluate service quality and residents’ preferences for facilitated self-service fundus disease screening.

Previous studies have suggested that people significantly prefer manual services to algorithms in the field of medicine [ 16 - 18 ]. Individuals have an aversion to algorithms underlying digital technology, especially when they see errors in the algorithm’s functioning [ 18 ]. The preference for algorithms does not increase even if the residents are told that the algorithm outperforms human doctors [ 19 , 20 ]. Our results confirm that fundus image quality in the exposure group is similar to that in the control group in our study, and both are similar to or even better than those reported in previous studies [ 21 , 22 ]. However, the preference for facilitated self-service fundus disease screening is significantly less than that for traditional manual screening. One possible explanation is that uniqueness neglect—a concern that algorithm providers are less able than human providers to account for residents’ (or patients’) unique characteristics and circumstances—drives consumer resistance to digital medical technology [ 23 ]. Therefore, personalized health management solutions based on large language models should be developed urgently [ 24 ] to meet the residents’ individual demands. In addition, a survey of population preferences for medical AI indicated that the most important factor for the public is that physicians are ultimately responsible for diagnosis and treatment planning [ 25 ]. As a result, man-machine collaboration, such as human supervision, is still necessary [ 26 ], especially in the early stages of digital transformation to help residents understand and accept the digital technologies.

Furthermore, our study suggests that distrust in the results of facilitated self-service fundus disease screening may cause residents to abandon eye disease screening, irrespective of whether it is provided using this new screening pattern or via the traditional manual screening pattern. This is critical to digital transformation in medicine. This implies that if the digital technology does not perform well, residents will not only be averse to the digital technology itself but also be more likely to abandon health care services as a whole. Digital transformation is a fundamental change to the health care delivery system. This implies that it can self-disrupt its ability to question the practices and production models of existing health care services. As a result, it may become incompatible with the existing models, processes, activities, and even cultures [ 27 ]. Therefore, it is important to assess whether the adoption of digital technologies contributes to health system objectives in an optimal manner, and this assessment should be carried out at the level of health services but not at the level of digital transformation [ 28 ].

The most prominent limitation of our study is that it was conducted only in Shanghai, China. Because of the sound health care system in Shanghai, residents have already received high-quality eye disease screening services before the adoption of the facilitated self-service eye screening pattern. Consequently, residents are bound to demand more from this new pattern. This situation is quite different from that in lower-income regions. Digital technology was adapted in poverty-stricken areas to build an eye care system, but it did not replace the original system that is based on manually delivered services [ 13 ]. Therefore, the framing effect may be weak [ 29 ], and there is little practical value in comparing digital technology and manual services in these regions. Second, our study is an observational study and blind grouping was not practical due to the special characteristics of fundus examination. However, we have attempted to use blind processing whenever possible. For instance, ophthalmologists’ evaluation of image quality was conducted in a blinded manner. Third, the manner in which we inquired about residents’ preferences might affect the results. For example, participants in the exposure group generally have experience with manual screening, but those in the control group may not have had enough experience with facilitated screening despite having been shown a video. This might make the participants in the control group more likely to choose manual screening because the new technology was unfamiliar. Finally, individual-level socioeconomic factors or educational level were not recorded, so we cannot rule out the influence of these factors on residents’ preferences.

In summary, this study confirms that the facilitated self-service fundus disease screening pattern could achieve high service quality. The preference of the residents for this new mode, however, was not ideal. It was difficult to reverse residents’ preference for manual screening in a short period, especially when the original manual service was already excellent. Therefore, the digital transformation of health care must proceed with caution. We suggest that attention be paid to the residents’ individual needs. Although more efficient man-machine collaboration is necessary to help the public understand and accept new technologies, personalized health management solutions based on large language models are required.

Acknowledgments

This study was funded by the Shanghai Public Health Three-Year Action Plan (GWVI-11.1-30, GWVI-11.1-22), Science and Technology Commission of Shanghai Municipality (20DZ1100200 and 23ZR1481000), Shanghai Municipal Health Commission (2022HP61, 2022YQ051, and 20234Y0062), Shanghai First People's Hospital featured research projects (CCTR-2022C08) and Medical Research Program of Hongkou District Health Commission (Hongwei2202-07).

Data Availability

Data are available from the corresponding author upon reasonable request.

Authors' Contributions

SL, YM, and YJ contributed to the conceptualization and design of the study. SL, YM, YJ, YP, TY, and YX collected the data. SL and YM analyzed the data. SL, YM, and YJ drafted the manuscript. WL, YX, JZ, LL, and HZ extensively revised the manuscript. All authors read and approved the final manuscript submitted.

Conflicts of Interest

None declared.

Video of the non-mydriatic fundus camera Kestrel-3100m with the self-service module.

Questions for screening service quality.

• GBD 2019 BlindnessVision Impairment Collaborators, Vision Loss Expert Group of the Global Burden of Disease Study. Causes of blindness and vision impairment in 2020 and trends over 30 years, and prevalence of avoidable blindness in relation to VISION 2020: the Right to Sight: an analysis for the Global Burden of Disease Study. Lancet Glob Health. Feb 2021;9(2):e144-e160. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marques AP, Ramke J, Cairns J, Butt T, Zhang JH, Muirhead D, et al. Global economic productivity losses from vision impairment and blindness. EClinicalMedicine. May 2021;35:100852. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jan C, Li S, Kang M, Liu L, Li H, Jin L, et al. Association of visual acuity with educational outcomes: a prospective cohort study. Br J Ophthalmol. Nov 18, 2019;103(11):1666-1671. [ CrossRef ] [ Medline ]
• Chai YX, Gan ATL, Fenwick EK, Sui AY, Tan BKJ, Quek DQY, et al. Relationship between vision impairment and employment. Br J Ophthalmol. Mar 16, 2023;107(3):361-366. [ CrossRef ] [ Medline ]
• Nayeni M, Dang A, Mao AJ, Malvankar-Mehta MS. Quality of life of low vision patients: a systematic review and meta-analysis. Can J Ophthalmol. Jun 2021;56(3):151-157. [ CrossRef ] [ Medline ]
• Wang L, Zhu Z, Scheetz J, He M. Visual impairment and ten-year mortality: the Liwan Eye Study. Eye (Lond). Aug 19, 2021;35(8):2173-2179. [ FREE Full text ] [ CrossRef ] [ Medline ]
• GBD 2019 BlindnessVision Impairment Collaborators, Vision Loss Expert Group of the Global Burden of Disease Study. Trends in prevalence of blindness and distance and near vision impairment over 30 years: an analysis for the Global Burden of Disease Study. Lancet Glob Health. Feb 2021;9(2):e130-e143. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cheng C, Wang N, Wong TY, Congdon N, He M, Wang YX, et al. Vision Loss Expert Group of the Global Burden of Disease Study. Prevalence and causes of vision loss in East Asia in 2015: magnitude, temporal trends and projections. Br J Ophthalmol. May 28, 2020;104(5):616-622. [ CrossRef ] [ Medline ]
• Li JO, Liu H, Ting DS, Jeon S, Chan RP, Kim JE, et al. Digital technology, tele-medicine and artificial intelligence in ophthalmology: a global perspective. Prog Retin Eye Res. May 2021;82:100900. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ting DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. Feb 25, 2019;103(2):167-175. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xie Y, Nguyen QD, Hamzah H, Lim G, Bellemo V, Gunasekeran DV, et al. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health. May 2020;2(5):e240-e249. [ CrossRef ]
• Tang J, Liang Y, O'Neill C, Kee F, Jiang J, Congdon N. Cost-effectiveness and cost-utility of population-based glaucoma screening in China: a decision-analytic Markov model. Lancet Glob Health. Jul 2019;7(7):e968-e978. [ CrossRef ]
• Xiao X, Xue L, Ye L, Li H, He Y. Health care cost and benefits of artificial intelligence-assisted population-based glaucoma screening for the elderly in remote areas of China: a cost-offset analysis. BMC Public Health. Jun 04, 2021;21(1):1065. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Generating Evidence for Artificial Intelligence Based Medical Devices: A Framework for Training Validation and Evaluation. World Health Organization. URL: https://www.who.int/publications/i/item/9789240038462 [accessed 2024-03-27]
• The Assessment List for Trustworthy Artificial Intelligence. URL: https://altai.insight-centre.org/ [accessed 2024-03-27]
• Dietvorst BJ, Bharti S. People reject algorithms in uncertain decision domains because they have diminishing sensitivity to forecasting error. Psychol Sci. Oct 11, 2020;31(10):1302-1314. [ CrossRef ] [ Medline ]
• DeCamp M, Tilburt JC. Why we cannot trust artificial intelligence in medicine. Lancet Digit Health. Dec 2019;1(8):e390. [ CrossRef ]
• Frank D, Elbæk CT, Børsting CK, Mitkidis P, Otterbring T, Borau S. Drivers and social implications of artificial intelligence adoption in healthcare during the COVID-19 pandemic. PLoS One. Nov 22, 2021;16(11):e0259928. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Juravle G, Boudouraki A, Terziyska M, Rezlescu C. Trust in artificial intelligence for medical diagnoses. Prog Brain Res. 2020;253:263-282. [ CrossRef ] [ Medline ]
• Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. Oct 2019;1(6):e271-e297. [ CrossRef ]
• Scanlon PH, Foy C, Malhotra R, Aldington SJ. The influence of age, duration of diabetes, cataract, and pupil size on image quality in digital photographic retinal screening. Diabetes Care. Oct 2005;28(10):2448-2453. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cen L, Ji J, Lin J, Ju S, Lin H, Li T, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun. Aug 10, 2021;12(1):4828. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Longoni C, Bonezzi A, Morewedge C. Resistance to medical artificial intelligence. J Consum Res. 2019;46:650. [ CrossRef ]
• Huang AS, Hirabayashi K, Barna L, Parikh D, Pasquale LR. Assessment of a Large Language Model's Responses to Questions and Cases About Glaucoma and Retina Management. JAMA Ophthalmol. Feb 22, 2024. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ploug T, Sundby A, Moeslund TB, Holm S. Population preferences for performance and explainability of artificial intelligence in health care: choice-based conjoint survey. J Med Internet Res. Dec 13, 2021;23(12):e26611. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Young AT, Amara D, Bhattacharya A, Wei ML. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review. Lancet Digit Health. Sep 2021;3(9):e599-e611. [ CrossRef ]
• Alami H, Gagnon M, Fortin J. Digital health and the challenge of health systems transformation. Mhealth. Aug 08, 2017;3:31-31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ricciardi W, Pita Barros P, Bourek A, Brouwer W, Kelsey T, Lehtonen L, et al. Expert Panel on Effective Ways of Investing in Health (EXPH). How to govern the digital transformation of health services. Eur J Public Health. Oct 01, 2019;29(Supplement_3):7-12. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khan WU, Shachak A, Seto E. Understanding decision-making in the adoption of digital health technology: the role of behavioral economics' prospect theory. J Med Internet Res. Feb 07, 2022;24(2):e32714. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 06.01.23; peer-reviewed by B Li, A Bate, CW Pan; comments to author 13.09.23; revised version received 15.10.23; accepted 12.03.24; published 17.04.24.

©Senlin Lin, Yingyan Ma, Yanwei Jiang, Wenwen Li, Yajun Peng, Tao Yu, Yi Xu, Jianfeng Zhu, Lina Lu, Haidong Zou. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.