research paper on virtual environment

SYSTEMATIC REVIEW article

Designing immersive virtual environments for human behavior research.

• 1 Department of Design and Environmental Analysis, Cornell University, Ithaca, NY, United States
• 2 Department of Communication, Cornell University, Ithaca, NY, United States

What are strategies for the design of immersive virtual environments (IVEs) to understand environments’ influence on behaviors? To answer this question, we conducted a systematic review to assess peer-reviewed publications and conference proceedings on experimental and proof-of-concept studies that described the design, manipulation, and setup of the IVEs to examine behaviors influenced by the environment. Eighteen articles met the inclusion criteria. Our review identified key categories and proposed strategies in the following areas for consideration when deciding on the level of detail that should be included when prototyping IVEs for human behavior research: 1) the appropriate level of detail (primarily visual) in the environment: important commonly found environmental accessories, realistic textures, computational costs associated with increased details, and minimizing unnecessary details, 2) context: contextual element, cues, and animation social interactions, 3) social cues: including computer-controlled agent-avatars when necessary and animating social interactions, 4) self-avatars, navigation concerns, and changes in participants’ head directions, and 5) nonvisual sensory information: haptic feedback, audio, and olfactory cues.

Introduction

The influence of surrounding environments on behavior: research limitations.

Our surrounding physical environment can influence behavior ( Waterlander et al., 2015 ) as it “affords” (per Gibson, 1979 ) the activities of the broader social, political, and cultural world. By understanding how our surrounding environment affects occupants, researchers can identify evidence-based design approaches such as developing standardized evaluation toolkits ( Joseph et al., 2014 ; Rollings and Wells, 2018 ), identifying design moderators ( Rollings and Evans, 2019 ), and ultimately informing policy, including guidelines governing how facilities are built, renovated, and maintained ( Sachs, 2018 ). By understanding how environments affect behaviors on a microbehavioral (i.e., unconscious) level, researchers can identify appropriate interventions (e.g., providing more sidewalks to encourage physical activity) and thereby inform the development of more effective informational and environmental interventions to improve desirable behavior ( Marcum et al., 2018 ).

However, experimentally examining the influence of our surrounding environment on behavior is challenging. Real-life environmental manipulations may be costly and even politically challenging to implement ( Schwebel et al., 2008 ). On the other hand, behaviors induced in conventional lab-based environments may not be generalizable to real-life environments ( Ledoux et al., 2013 ). The influence of the surrounding environment on behaviors might be better understood ( Ledoux et al., 2013 ) if researchers could immerse participants in complex physical and social environments that are ecologically valid while being highly controlled ( Veling et al., 2016 ). Because of this, simulations are sometimes used to explore the relationship between environment and behavior ( Marans and Stokols, 2013 ). Potential simulations can include mockups, sketches, photographs, models, and immersive virtual environments (IVEs). While CAVE automatic virtual environments (CAVEs, Cruz-Neira et al., 1993 ) and head-mounted displays (HMDs) have both been used to simulate such environments, the recent increase in the availability of consumer HMDs means that many more researchers can now use IVEs to answer questions about the effects of surrounding environments on behaviors. In this review, we reviewed and synthesized peer-reviewed research that used IVEs presented in HMDs for research on behavior influenced by our surrounding environment, with the aim of showcasing the solutions found by previous researchers. As virtual reality (VR) and IVEs will be frequently mentioned in this review, it is important to distinguish “VR” as the technology used to create “IVEs.”

Immersive Virtual Environment Tools for Human Behavior Research: Making the Case

Past research suggests that VR is a useful research tool to simulate real-life environmental features, as it allows researchers to immerse participants in hypothetical contexts and study their responses to controlled environmental manipulations otherwise difficult to examine in real-life environments ( Parsons et al., 2007 ; Schwebel et al., 2008 ; Poelman et al., 2017 ; Ahn, 2018 ). Considerable work has demonstrated VR’s ability to elicit behavioral responses to virtual environments, even when the participant is well aware that the environment is not “real” as in demonstrations of the classic “pit demo” ( Meehan et al., 2003 ).

In 2002, Blascovich and colleagues foresaw the advantages of VR as a tool for research in the social sciences. Although Blascovich’s original article discussed the use of VR as a tool for social psychology specifically, the advantages he describes for balancing experimental control and mundane realism and improving replicability and representative sampling have made it a tool of interest for researchers in several social science fields. VR has a high degree of realism: users tend to react to scenarios as if they were occurring in the real world. VR allows for a high degree of experimental control. Environments, events, and even virtual people can be programmed to appear to every user in the same way. Thus, VR has already been used extensively for diagnosis ( Parsons et al., 2007 ), clinical education ( Lok et al., 2006 ; Atesok et al., 2016 ), and clinical and experimental interventions ( Difede and Hoffman, 2002 ; Wiederhold and Wiederhold, 2010 ; Wiederhold, 2017 ).

VR provides critical benefits over other methods available for behavior research ( Schwebel et al., 2008 ). These advantages are particularly applicable when considering the influence of environments on behavior. VR has the potential to examine how people behave in real-life situations, without exposing participants to the risk and inconsistency of real-world environments ( Blascovich et al., 2002 ). Participants can safely experience immersion in the virtual environment when the real environment is hazardous ( Viswanathan and Choudhury, 2011 ), permitting researchers to ethically examine potentially dangerous behaviors ( Schwebel et al., 2012 ). Additionally, it is relatively easy to manipulate environmental factors such as noise and crowding in virtual environments ( Neo et al., 2019 ).

The Design of VR Environments for Behavior Studies: Research Gap

A prototype “is an artifact that approximates a feature (or multiple features) of a product, service, or system” ( Otto and Wood, 2001 ; Camburn et al., 2017 , p. 1) and “a virtual prototype is one which is developed (and tested) on a computational platform” ( Camburn et al., 2017 , p. 17). VR, especially its prototyping functions (i.e., the test-refinement-completion of designs using digital mockups, Ulrich and Eppinger, 2012 ), has been increasingly applied to behavior research. In this review, we examine VR’s potential to address environmental effects on behavior. In these cases, IVEs should be designed such that interactions between the individual and the virtual environment are as analogous as possible to interactions that would take place if the individual were in the actual environment, with the ultimate goal of developing a more robust way of examining the impact of the surrounding environment on behavior.

VR is generally considered to be a high-presence medium. Presence refers to the sense of “being there” in the VR environment ( Heeter, 1992 ; Slater et al., 2009 ). While presence and immersion are terms sometimes used interchangeably, researchers have distinguished between the subjective psychological sense of presence and immersion, which can be considered a quality of the technology ( Slater, 2018 ). A virtual reality setup that provides highly detailed visual content, spatialized sound, and haptic feedback (e.g., through vibrating controllers) would be considered more immersive than a scene rendered on a desktop monitor. Greater immersion is generally considered to increase presence ( Cummings and Bailenson, 2016 ). Because consumer HMDs have reduced cost and expense while retaining a high sense of presence; it is plausible for many more researchers to use VR for prototyping applications; thus, we focus our recommendations on this larger pool of potential researchers.

While other considerable valuable works have used CAVE or desktop-based virtual environments to examine behavior, we have limited our analysis in this review to studies that use HMDs, to study behavior as it relates to the environment. The relatively lower cost and portability of new consumer HMDs mean that researchers who have not previously engaged with virtual reality now have the opportunity to use these systems for their research. This review aims to provide a summary of design considerations pulled from existing research in virtual reality that might prove useful to potential researchers who are not experienced in this area.

The qualities of HMDs provide special opportunities and constraints. HMDs combine portability with the ability to block out the surrounding environment, making them good for “in-the-wild” studies ( Oh et al., 2016 ). The greater presence HMDs can provide is particularly important to these behavioral studies but comes with tradeoffs. Users do not see their real bodies, so researchers must decide whether or not to include avatars. HMDs allow users to experience spaces that may be larger than the physical space that they are actually in, meaning that users’ abilities to navigate must be programmed and controlled. Such environments allow for the ready tracking of behavioral data ( Yaremych and Persky, 2019 ) and interaction with objects, but all of these interactions must be designed. In this review, we highlight the solutions and tradeoffs that previous researchers have made in this context.

Best Practices for Successful IVE-Based Experimental Studies

Heydarian and Becerik-Gerber (2017) describe “four phases of IVE-based experimental studies” and discuss best practices for consideration in different phases of experimental studies ( Figure 1 , see Heydarian and Becerik-Gerber, 2017 for in-depth discussion).

FIGURE 1 . Four phases of IVE-based experimental studies.

In this review, we focus on the “development of experimental procedure” phase, described by Heydarian and Becerik-Gerber as Phase 2. This includes the design and setup of the IVEs, especially considerations involving the level of detail required (i.e., factor(s) recognizable by participants; Heydarian and Becerik-Gerber, 2017 ). This may differ between studies and can include visual appearance, behavioral realism, and virtual human behavior. To meet the study objectives, a sense of presence is key, allowing study participants to feel “there” and thus behave as if they were in the actual environment.

However, information on the design process in Phase II can be hard to find. Researchers typically describe the “final environment” they have designed in publications, but justifications for the many design decisions they have made in the development of the virtual environment are less common, probably at least in part due to publication length limitations. However, this information is extremely valuable. The following review expands on the work by Heydarian and Becerik-Gerber (2017) by reviewing and synthesizing strategies from 18 studies using IVEs for behavior research. In addition, we have created a wiki ( https://osf.io/gyadu/ ) to collect citations for other papers that use IVEs for this purpose so that this database can be updated. We hope this synthesis and this wiki will be an additional resource for researchers new to this space to build on the knowledge of previous researchers to make informed choices when they are designing such IVEs.

Inclusion Criteria

Inclusion criteria were as follows: (a) the study examined nonclinical populations, (b) the study used IVEs presented via HMDs to examine behaviors influenced by the environment, (c) users experienced a virtual environment that plausibly represented an actual environment, and (d) the study provided sufficient details on how it designed and set up the IVE.

Review Procedure and Data Extraction

After consultation with a research librarian, we applied the following keywords ( Table 1 ) and MeSH search terms: virtual reality, behavior, prototype, and design ( Table 2 ). Terms were combined with the Boolean operators “and” and “or” to identify relevant studies. We did not conduct searches separately by specific behaviors, e.g., “grocery shopping” or environments, e.g., “grocery store,” but narrowed down the results from an initial search focusing on VR. Using PubMed, Web of Science, Scopus, and Google Scholar databases, we conducted a systematic search to identify English-language, peer-reviewed publications, and conference proceedings on experimental and proof-of-concept studies that described the design, manipulation, and setup of the IVEs to examine behaviors influenced by the environment. The search targeted articles were published before May 15, 2020 (i.e., no lower bound cutoff date). Reviewer 1 assessed retrieved texts to determine if they met the inclusion criteria. If a study was deemed potentially eligible, the full article was retrieved for further assessment and inclusion. A second reviewer screened all included articles. The selection was finalized after discussion and consensus between reviewers. We identified additional studies by searches of the references provided in the included publications ( Greenhalgh and Peacock, 2005 ). Once the finalized list of papers was determined, these data were extracted: first author, year, behavior, environment, and strategies in designing IVEs for behavior research.

TABLE 1 . List of keywords.

TABLE 2 . List of search terms.

Data Synthesis and Analysis

Due to the heterogeneity of research in the field, quantitative synthesis was not planned. To conduct a narrative synthesis, two reviewers independently categorized studies based on strategies in designing IVEs for behavior research. Based on past use of VR in behavior research, our research question was as follows: What are strategies that researchers identified as effective in designing IVEs for behavior research?

We screened 203 citations and reviewed 61 full texts, of which 18 met the inclusion criteria ( Figures 2 and 3 ). All the studies we found were in the range of 2015–2020. Exclusion criteria were as follows: (a) the study examined clinical populations (57 citations), (b) the study did not use IVE presented via HMDs to examine behaviors influenced by the environment (32), (c) the study did not use VR to create a virtual environment (28), and (d) the study did not provide sufficient details on how it designed and set up the IVE (50). Case studies that did not describe behavioral outcomes were thus also excluded ( Lovreglio et al., 2018 ). As some studies were rejected for more than one reason, the sum of studies is greater than 142. For brevity, the types of behaviors and environments are summarized in Table 3 .

FIGURE 2 . Flow diagram.

FIGURE 3 . A yes/no flowchart to decide if a study provided sufficient details on how it designed and set up the IVE.

TABLE 3 . Types of behavior and environment.

Building from recent research using IVEs as a prototyping tool to examine the relationship between the environment and human behaviors, this review provides strategies drawn from previous IVEs designed to understand environments’ influence on behaviors. This discussion is grounded in our analysis of what design researchers have reported as effective in previous experiments and on the outcomes in the reviewed studies. Five key categories emerged from this analysis: (1) the appropriate level of detail (primarily visual) in the environment, (2) context, (3) social cues, (4) participant tracking and rendering, and (5) nonvisual sensory information ( Figure 4 . A high-resolution version of this figure is available as Supplementary Figure S1 ).

FIGURE 4 . Flow diagrams summarizing the strategies for the design of IVEs to understand environments’ influence on behaviors.

In this review, many recommendations (e.g., including dynamic representation of the participant’s body) are dependent on the research goal and tradeoffs (e.g., additional cost, development time, and technical capabilities). Hence, before any meaningful discussion of what researchers have reported as effective in designing IVEs for environment-behavior research, all strategies and decisions should be evaluated in the context of two design principles: (1) the research goal and (2) potential tradeoffs to design choices, such as development cost and scalability. Additionally, the recommendations featured in this review were specific examples used by researchers in past studies. It is important to note that there are various ways to approach these considerations that may or may not have been discussed in past studies.

Level of Detail in the Environment

Important commonly found environmental accessories.

Researchers proposed that IVEs should incorporate typical elements (features such as furniture or features) of actual environments. In a study examining gambling behavior, Dickinson et al. (2020) included items such as paper slips, pens, and stools in the betting shop ( Figure 5 ).

FIGURE 5 . Dickinson et al. (2020) included items such as paper slips, pens, and stools in the betting shop.

Realistic Textures

Realistic textures can be created by taking photos of the actual product before attaching it to the virtual objects. This is particularly important if participants are expected to move around the environment and pick up objects to examine them ( Morrongiello et al., 2015 ; Lombart et al., 2019 ; Siegrist et al., 2019 ). For example, in a study examining purchase behavior, Lombart et al. (2019) used real product textures with high-resolution pictures from real products ( Figure 6 ).

FIGURE 6 . Lombart et al. (2019) used real product textures with high-resolution pictures from real products.

Computational Costs Associated With Increased Details

Realistic rendering can increase an individual’s sense of presence in the virtual environment. According to Slater et al. (2009) , visual realism has two components: geometric realism (i.e., virtual and real objects look alike) and illumination realism (i.e., the fidelity of the lighting model). However, building complex IVEs can be time-consuming, requires heavy computational algorithms, and may even decrease frame rate when participants view the IVE ( Sobhani et al., 2017 ; Lin et al., 2020 ).

Researchers must then decide which extraneous features in the surrounding environment are not key to the research question and can be removed while maintaining ecological validity. For example, in examining distracted pedestrian crossing behavior, Sobhani et al. (2017) excluded other pedestrians and cyclists, focusing on vehicles and the participant ( Figure 7 ). Before finalizing the research design and hypotheses, researchers should always consider the complexity of their desired IVE since the impact of a “suboptimal” IVE (e.g., an IVE that lacks critical details of the surrounding environment) could have significant effects on participants’ behaviors ( Lin et al., 2020 ).

FIGURE 7 . Sobhani et al. (2017) excluded other pedestrians and cyclists, focusing on vehicles and the participant.

Minimizing Unnecessary Detail

IVEs may be perceived as more immersive than lab conditions ( Dickinson et al., 2020 ). However, high levels of visual realism (i.e., consistency between one’s virtual vs. real-world experience Witmer and Singer, 1998 ) might increase expectations for other aspects (e.g., nonvisual and tactile) of the simulation to be equally realistic ( Dickinson et al., 2020 ). This raises a critical consideration in designing IVEs for research: while realistic IVEs generally increase the ecological validity of research, some kinds of realism (e.g., adding avatars) inherently increase the complexity and confounding variables associated with experimental environments. Here, we define a confounding variable as an “extraneous variable whose presence affects the variables being studied so results do not reflect the actual relationship between the variables under study” ( Pourhoseingholi et al., 2012 , p. 79).

Too much detail can reintroduce some challenges associated with research in actual environments ( Dickinson et al., 2020 ). Before designing an IVE, researchers can use input from past research, end-users, and subject experts to identify possible confounds and evaluate the risks and benefits of including a given feature into the IVE ( Persky et al., 2018 ). For example, in a study examining navigation behaviors in powered wheelchair driving simulators, Alshaer et al. (2017) did not tell participants about the passability of the doorframes or gaps to preserve realism and did not include furnishings or decorations to avoid distractions and to remove cues to size and distance provided by familiar objects ( Figure 8 ).

FIGURE 8 . Alshaer et al. (2017) did not tell participants about the passability of the doorframes or gaps to preserve realism. Furnishings or decorations were excluded to avoid distractions.

Before creating IVEs to examine behaviors, researchers need to identify what contextual cues are necessary and evaluate whether VR can incorporate all the required elements. Social elements involve perhaps the most complicated tradeoffs, discussed in the following section.

Contextual Elements

In IVEs, participants give special attention to objects most relevant to looking behaviors, such as windows ( Lee et al., 2019 ). Participants may also engage in more exploratory behaviors (e.g., looking intensely at objects irrelevant to the research questions), behaviors that may not be as salient in actual environments ( Lee et al., 2019 ). For example, in Lee et al. (2019) , participants spent more time looking at “see-through” surfaces such as windows ( Figure 9 ). Perhaps, participants learned that available behaviors in IVEs were mostly “looking around” and focused their attention on windows, instead of objects with functionality (e.g., portability) in the real but not virtual world ( Lee et al., 2019 ). Participants’ attention to display surfaces or windows highlights the notion of multiple embeddedness during interactions in virtual environments ( Lee et al., 2019 ). A person’s experience in an IVE is still embedded in the surrounding environment ( Lee et al., 2019 ). For example, when participants allocate visual attention to window surfaces displaying extra information regarding the exterior, they may be creating a mental model of the IVE’s location and themselves in the virtual space ( Lee et al., 2019 ). Researchers should consider whether such features (e.g., windows) should be included or excluded as participants may devote unnecessary attention to “checking them out.” For example, including a “skybox” and trees as a surrounding exterior makes users think that they are inside a building with windows (i.e., virtual realism) ( Morrongiello et al., 2015 ; Nordbo et al., 2015 ).

FIGURE 9 . In Lee et al. (2019) , participants spent more time looking at “see-through” surfaces such as windows.

Stimulatory and instructional cues can provide relevant information and navigation within IVEs. In a study examining energy consumption behavior, Saeidi et al. (2019) used stimulatory cues related to the spatial and temporal configuration of the IVE to simulate relevant information about the IVE, such as the sense of time, weather, and crowding. Specifically, Saeidi et al. (2019) used lighting and moving traffic to evoke a sense of time. However, researchers should consider using such cues in moderation. For example, in a study examining evacuation behavior by Tucker et al. (2018) , smoke was incorporated only to enhance participants’ anxiety and perceived hazard and not to the extent where the simulating impacts of smoke hinder visibility ( Tucker et al., 2018 ) ( Figure 10 ).

FIGURE 10 . Tucker et al. (2018) incorporated smoke only to enhance participants’ anxiety and perceived hazard and not to the extent where the simulating impacts of smoke hinder visibility.

Saeidi et al. (2019) also used instructional cues to help participants navigate within the IVE and distinguish and interact with operable virtual objects. For example, as participants hover the controller over operable objects, they would start blinking, signaling activation ( Saeidi et al., 2019 ) ( Figure 11 ).

FIGURE 11 . Saeidi et al. (2019) used instructional cues to help participants navigate within the IVE and distinguish and interact with operable virtual objects.

Animating Interactions

In an IVE, a feature (e.g., traffic and avatar) may be moving at a simulated speed. Identifying and creating the appropriate speed may be important for research examining behaviors such as road crossing and emergency evacuation. To reduce potential confounders (i.e., walking speed), in this example, the researchers kept the demonstration avatar’s walking speed constant based on the staff’s walking speed ( Shi et al., 2019 ). If possible, researchers should use pretests to ensure that calibration accuracy between participants’ actual movements and virtual animations is acceptable to participants ( Shi et al., 2019 ). Specifically, there should be minimal movement mismatches in the virtual environment when participants rotate their bodies as these mismatches can affect their feelings of presence ( Shi et al., 2019 ). In a study by Lin et al. (2020) examining evacuation behaviors, the range of avatar speeds depended on their age and gender.

Social Cues

If social aspects are important, researchers need to find ways to incorporate associated cues or agents into the IVE ( Neo et al., 2019 ).

Include Computer-Controlled Agent-Avatars When Necessary

Computer-controlled agent-avatars may enhance the realism of IVEs and facilitate the examination of behaviors such as distractions and emergency evacuation. However, avatars may also unduly distract participants from engaging in the behavior of interest. For example, in Dickinson et al. (2020) , no agent-avatars were placed in the IVE, so as not to distract participants playing the electronic gaming machine.

Animating Social Interactions

In general, introducing avatars of the self or of others requires a number of decisions. If agent-avatars are introduced, they must be animated appropriately. A number of animations are available for free or for purchase (e.g., Mixamo.com ) can be designed using modeling programs or can be generated from human motion ( Gonzalez-Franco et al., 2020 ). These animations can be automatically triggered by the actions of the participant, but sometimes a “wizard-of-Oz” scenario, in which a key digital element is actually controlled by a human user (for example, see Lucas et al., 2019 ) may be more useful. For example, in examining fall risk, instead of relative to playing a prerecorded animation, Shi et al. (2019) used a lab staff member to control another avatar in real time as the human-controlled avatar seemed more “natural and acceptable” to the participants.

Participant Tracking and Rendering

Self-avatars.

Self-avatars can enhance a user’s sense of presence ( Fox and Bailenson, 2010 ). People tend to experience an elevated sense of presence when there is a virtual representation of oneself in the VR environment and when other users (avatars) recognize them ( Nash et al., 2000 ; Fox and Bailenson, 2010 ).

When investigating certain behaviors, such as pedestrian road crossing, it may be helpful for a participant to see a dynamic representation of their body to increase their sense of presence ( Kooijman et al., 2019 ). Kooijman et al. (2019) propose that the representation needs to be realistic (i.e., not robot-looking) and gender-specific and provides synchronous tactile-visual feedback to evoke a full-body illusion ( Kooijman et al., 2019 ). However, as other researchers have found, a first-person perspective alone can aid in creating a useful body-ownership illusion ( Slater et al., 2010 ).

As suggested by Kooijman et al. (2019) , motion suits are increasingly used in road safety research and may become more common for other types of VR-based research. Different headsets may have different capabilities to track and represent user behavior. Some newer HMDs, for example, the Oculus Quest, now offer the ability to track and render users’ hands without requiring them to hold hand controllers.

Navigation Concerns

Navigation via real walking can enhance one’s sense of presence in IVEs ( Shi et al., 2019 ); however, this option is constrained by the physical space available ( Iryo-Asano et al., 2018 ; Kooijman et al., 2019 ). For example, in Kooijman et al. (2019) , which examined pedestrian crossing behaviors, participants were asked not to walk beyond the third “zebra stripe” due to the space constraints of the laboratory. Given this conflict, allowing participants to control their walking direction and speed can help them feel as if they are walking in VR environments ( Morrongiello et al., 2015 ; Marcum et al., 2018 ).

Changes in Participants’ Head Directions

Mizuchi and Inamura (2018) evaluated human behavior difference with a restricted field of view in real and IVEs and found that large changes in the head direction and some head-mounted display properties affect spatial perception about recognition speed and manipulation skill in IVEs. A scenario in which participants must frequently and dramatically change head direction may be unfavorable for observing behaviors in IVEs. In a study by Shi et al. (2019) examining fall risk behaviors, slight movement mismatches in the IVE when participants rotate their heads can affect their sense of presence (i.e., being there in the simulated world) ( Figure 12 ). IVEs and scenarios should be designed to avoid large changes in participants’ head direction, for example, putting participants in a narrow space ( Mizuchi and Inamura, 2018 ).

FIGURE 12 . Shi et al. (2019) highlighted that slight movement mismatches in the IVE when participants rotate their heads can affect their sense of presence. Shi et al. (2019) used real planks to provide haptic feedback while participants walked on “virtual planks.”

Nonvisual Sensory Information

The realism of virtual simulations was highly rated in various studies; however, most simulations lack some characteristics of the real environment (e.g., haptic feedback, sounds, smell, and other sensory contents) ( Ledoux et al., 2013 ). As with other elements that might increase immersion but create technical difficulties, adding additional sensory modalities requires the careful consideration of tradeoffs.

Haptic Feedback

Some VR-based studies integrate real elements besides the VR displays, for example, to provide passive haptic feedback. For example, in a study examining fall risk behaviors, Shi et al. (2019) used real planks to provide haptic feedback while participants walked on “virtual planks.” With such integration, the real and virtual element (e.g., a plank) should be carefully calibrated to enhance realism, and yet address safety issues (e.g., a participant falling off a plank) ( Shi et al., 2019 ). To enhance setups with real and virtual elements, avatars may be included to reflect any interactions observed with the real elements, such as falling off a real plank ( Shi et al., 2019 ).

IVEs may include relevant (i.e., typical of the actual environment) audio to isolate viewers from the real world ( Dickinson et al., 2020 ). Researchers should first determine if the sounds can enhance the IVE’s realism or if they might confound the study ( Morrongiello et al., 2015 ). In a study examining evacuation behaviors, Markwart et al. (2019) included sounds of a storm (i.e., rain; thunder), wind, cars driving by, splatter sounds from the water fountain, and footsteps. To make virtual experiences more realistic, certain sounds, such as avatars' voices, should be gender-specific ( Markwart et al., 2019 ).

Olfactory Cues

None of the studies included olfactory cues in the IVE. However, some studies cited the lack of characteristics of a real food choice environment (e.g., food smell) as a key limitation ( Ledoux et al., 2013 ; Marcum et al., 2018 ). For example, the lack of olfactory feedback might become a factor if participants were asked to pick up the virtual food ( Ledoux et al., 2013 ; Marcum et al., 2018 ). Olfactory cues have been used in studies examining virtual food; for example, Li and Bailenson (2018) explored the role of olfactory cues of a virtual donut on satiation and eating behavior and Stelick et al. (2018) developed a proof-of-concept study to determine if the pungent flavor notes of blue cheese may be enhanced by showing participants a virtual dairy barn. In Li and Bailenson (2018) , the authors attached a scented cotton bud to the front of the HMD with Velcro strips at the exact same time the participants saw and smelled a virtual donut.

Overall, more targeted haptic, auditory, and olfactory feedback will likely become possible with the rapid growth of VR technology. Nonvisual sensory information may be more important if participants engage in, for example, food selection behavior, where aesthetic and reward-oriented features may be more important ( Marcum et al., 2018 ). Adding nonvisual sensory information will also help researchers who need to capture the potential responses of a more diverse participant pool, for example, participants with low vision ( Zhao et al., 2019 ).

Limitations of This Review

This review summarizes published information about researchers’ design decisions when creating IVEs to test the effect of environments on behavior. However, we recognize that much valuable information remains unpublished due to page limits and other constraints of academic publishing. While we hope this paper can be useful, especially for researchers less familiar with this area, providing experienced researchers with resources to share their design experiences will greatly aid the research community. This would allow for the inclusion of projects that did not include behavioral data at the time of publication of this paper but that are likely to contain valuable information, for example, ( Lovreglio et al., 2018 ).

Due to the heterogeneity of human behaviors, we could not design a search strategy for each behavior (e.g., human behavior in a supermarket; human behavior in pedestrian crossing). Our search returned limited results due to our strict inclusion criteria (i.e., a study must use HMDs) and the resulting small number of studies. Our decision to limit this review to studies using HMDs may have reduced the breadth and depth of our analysis, as well as the variety of environments and behaviors. Variations in research questions and perspectives also limited our results, despite our best efforts to systematically identify and categorically include relevant studies. Some studies met the inclusion criteria but provided no discussion on how to design IVE to examine behaviors. Due to a reduced pool of studies from which to draw conclusions, some environmental considerations highlighted in this review (e.g., haptic feedback; olfactory cues) depended on the intended application and purpose. By using the term “human behavior” relatively broadly as the starting point, recommendations can be generalized to a larger and more diverse audience including researchers, designers, and practitioners.

Our review analyzed the use of IVEs in behavioral science research and provided design considerations when prototyping IVEs for research on the interaction between the environment and human behavior. We found that rather than trying to replicate every aspect of the physical environment, researchers carefully considered the level of detail needed for each element. We also found that interdisciplinary collaboration is required to conceptualize, plan, design, and execute a VR study to examine behaviors ( Metsis et al., 2019 ; Neo et al., 2019 ). We provided some sample workflows illustrated with examples from published work.

With these design considerations gleaned from experienced VR researchers in mind, other behavioral scientists may be able to better use VR to develop designs in order to examine behaviors. Ultimately, by enhancing our knowledge of the design and use of VR, researchers can better understand environmental factors that influence behavior and how to effectively alter environmental and/or policy-based interventions. However, the valuable information about the design decisions researchers makes in creating these virtual environments can be hard to find. Studies currently available only as pilots or case studies (e.g., Lovreglio et al., 2018 ), if used in the future to examine and capture behavioral data, provide promising opportunities to help researchers better understand the environments’ influence on behaviors.

While this review aims to provide a summary of relevant research up to the present, finding ways for more researchers to be able to easily share their hard-won knowledge is an important need for the community of current and interested researchers.

Data Availability Statement

The original contributions presented in the study are included in the article/ Supplementary Material ; further inquiries can be directed to the corresponding author.

Author Contributions

JN was involved in conceptualization, data curation, formal analysis, methodology, project administration, writing-original draft preparation, review, and editing. AW and MS were involved in formal analysis, methodology, writing-original draft preparation, review, and editing.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to acknowledge the help of Ms. Amelia Kallaher (Applied Social Science Librarian, Cornell University) for developing the search strategy.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.603750/full#supplementary-material

Ahn, S. J. (2018). Virtual exemplars in health promotion campaigns, J. Media Psychol. Theor. Methods Appl . 30, 91–103. doi:10.1027/1864-1105/a000184

CrossRef Full Text | Google Scholar

Alshaer, A., Regenbrecht, H., and O'Hare, D. (2017). Immersion factors affecting perception and behaviour in a virtual reality power wheelchair simulator. Appl. Ergon . 58, 1–12. doi:10.1016/j.apergo.2016.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Atesok, K., Satava, R. M., Van Heest, A., Hogan, M. V., Pedowitz, R. A., Fu, F. H., et al. (2016). Retention of skills after simulation-based training in orthopaedic surgery. J. Am. Acad. Orthop. Surg . 24 (8), 505–514. doi:10.5435/JAAOS-D-15-00440

Camburn, B., Viswanathan, V., Linsey, J., Anderson, D., Jensen, D., Crawford, R., et al. (2017). Design prototyping methods: state of the art in strategies, techniques, and guidelines. Des. Sci . 3, E13. doi:10.1017/dsj.2017.10

Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. (1993). Surround-screen projection-based virtual reality: the design and implementation of the CAVE,” in Proceedings of the 20th annual conference on computer graphics and interactive techniques , 135–142.

Google Scholar

Cummings, J. J., and Bailenson, J. N. (2016). How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychol . 19 (2), 272–309. doi:10.1080/15213269.2015.1015740

Dickinson, P., Gerling, K., Wilson, L., and Parke, A. (2020). Virtual reality as a platform for research in gambling behaviour. Comput. Hum. Behav . 107, 106293. doi:10.1016/j.chb.2020.106293

Difede, J., and Hoffman, H. G. (2002). Virtual reality exposure therapy for world trade center post-traumatic stress disorder: a case report. Cyberpsychol. Behav . 5 (6), 529–535. doi:10.1089/109493102321018169

Fox, J., and Bailenson, J. N. (2010). The use of doppelgängers to promote health and behavior change. Cyberther. Rehabil . 3 (2), 16–17. doi:10.1037/e530522011-003

Gibson, J. J. (1979). The ecological approach to visual perception . Boston: Houghton Mifflin .

Glotzbach, P. A., and Heft, H. (1982). Ecological and phenomenological approaches to perception . Hoboken: Nous , 16108–16121.

Gonzalez, G. M., Lehr, J., Krämer, N., and Gratch, J. (2019). “The effectiveness of social influence tactics when used by a virtual agent,”in Proceedings of the 19th ACM international conference on intelligent virtual agents , 22–29.

Gonzalez-Franco, M., Egan, Z., Won, G. M., and Wang, G. M. (2020). MoveBox: democratizing MoCap for the Microsoft Rocketbox Avatar Library (IEEE AIVR 2020) . Berlin: Springer .

Greenhalgh, T., and Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 331 (7524), 1064–1065. doi:10.1136/bmj.38636.593461.68

Heeter, C. (1992). Being there: the subjective experience of presence. Presence Teleop. Virt. Environ . 1 (2), 262–271. doi:10.1162/pres.1992.1.2.262

Heydarian, A., and Becerik-Gerber, B. (2017). Use of immersive virtual environments for occupant behaviour monitoring and data collection. J. Build. Perform. Simul . 10 (5–6), 484–498. doi:10.1080/19401493.2016.1267801

Hurwitz, J., Loomis, J., Beall, A. C., Swinth, K. R., Hoyt, C. L., and Bailenson, J. N. (2002). Target article: immersive virtual environment technology as a methodological tool for social psychology. Psychol. Inq . 13, 103–124. doi:10.1207/s15327965pli1302_01

Iryo-Asano, M., Hasegawa, Y., and Dias, C. (2018). Applicability of virtual reality systems for evaluating pedestrians' perception and behavior. Transp. Res. Proced . 34, 67–74. doi:10.1016/j.trpro.2018.11.015

Joseph, A., Quan, X., Keller, A. B., Taylor, E., Nanda, U., and Hua, Y. (2014). Building a knowledge base for evidence-based healthcare facility design through a post-occupancy evaluation toolkit. Intell. Build. Int . 6 (3), 155–169. doi:10.1080/17508975.2014.903163

Kooijman, L., Happee, R., and de Winter, J. (2019). How do eHMIs affect pedestrians' crossing behavior? A study using a head-mounted display combined with a motion suit. Information 10 (12), 386. doi:10.3390/info10120386

Ledoux, T., Nguyen, A. S., Bakos-Block, C., and Bordnick, P. (2013). Using virtual reality to study food cravings. Appetite 71, 396–402. doi:10.1016/j.appet.2013.09.006

Lee, J., Eden, A., Ewoldsen, D. R., Beyea, D., and Lee, S. (2019). Seeing possibilities for action: orienting and exploratory behaviors in VR. Comput. Hum. Behav . 98, 158–165. doi:10.1016/j.chb.2019.03.040

Li, B. J., and Bailenson, J. N. (2018). Exploring the influence of haptic and olfactory cues of a virtual donut on satiation and eating behavior. Presence Teleop. Virt. Environ . 26 (03), 337–354. doi:10.1162/pres_a_00300

Lin, J., Cao, L., and Li, N. (2020). How the completeness of spatial knowledge influences the evacuation behavior of passengers in metro stations: a VR-based experimental study. Autom. Construct . 113, 103136. doi:10.1016/j.autcon.2020.103136

Lok, B., Ferdig, R. E., Raij, A., Johnsen, K., Dickerson, R., Coutts, J., et al. (2006). Applying virtual reality in medical communication education: current findings and potential teaching and learning benefits of immersive virtual patients. Virt. Real . 10 (3–4), 185–195. doi:10.1007/s10055-006-0037-3

Lombart, C., Millan, M., Normand, J.-M., Verhulst, A., Labbé-Pinlon, B., and Moreau, G. (2019). Consumer perceptions and purchase behavior toward imperfect fruits and vegetables in an immersive virtual reality grocery store. J. Retail. Consum. Serv. 48, 28–40. doi:10.1016/j.jretconser.2019.01.010

Lovreglio, R., Gonzalez, V., Feng, Z., Amor, R., Spearpoint, M., Thomas, J., et al. (2018). Prototyping virtual reality serious games for building earthquake preparedness: the Auckland City Hospital case study. Adv. Eng. Inform . 38, 670–682. doi:10.1016/j.aei.2018.08.018

Marcum, C. S., Goldring, M. R., McBride, C. M., and Persky, S. (2018). Modeling dynamic food choice processes to understand dietary intervention effects. Ann. Behav. Med. 52, 252–261. doi:10.1093/abm/kax041

Markwart, H., Vitera, J., Lemanski, S., Kietzmann, D., Brasch, M., and Schmidt, S. (2019). Warning messages to modify safety behavior during crisis situations: a virtual reality study. Int. J. Disaster Risk Reduct . 38, 101235. doi:10.1016/j.ijdrr.2019.101235

Meehan, M., Razzaque, S., Whitton, M. C., and Brooks, F. P. (2003). “Effect of latency on presence in stressful virtual environments,” in Proceedings of the IEEE virtual reality , 141–148.

Metsis, V., Lawrence, G., Trahan, M., Smith, K. S., Tamir, D., and Selber, K. (2019). 360 Video: a prototyping process for developing virtual reality interventions. J. Technol. Hum. Serv . 37 (1), 32–50. doi:10.1080/15228835.2019.1604291

Mizuchi, Y., and Inamura, T. (2018). “Evaluation of human behavior difference with restricted field of view in real and VR environments,” in 27th IEEE international symposium on robot and human interactive communication, RO-MAN , 196–201.

Morrongiello, B. A., Corbett, M., Switzer, J., and Hall, T. (2015). Using a virtual environment to study pedestrian behaviors: how does time pressure affect children's and adults' street crossing behaviors. J. Pediatr. Psychol. 40, 697–703. doi:10.1093/jpepsy/jsv019

Nash, E. B., Edwards, G. W., Thompson, J. A., and Barfield, W. (2000). A review of presence and performance in virtual environments. Int. J. Hum.-Comput. Interact . 12 (1), 1–41. doi:10.1207/s15327590ijhc1201_1

Neo, J. R. J., Won, A. S., and Shepley, M. M. (2019). “Virtual reality as a prototyping tool in health behavior research: Current research, limitations, and recommendations for future research,” in Paper presented at the 69th International Communication Association Conference, DC, USA .

Nordbo, K., Milne, D., Calvo, R. A., and Allman-Farinelli, M. (2015). “Virtual food court: a VR environment to assess people's food choices,” in Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, 2015 (New York, NY: ACM), 69–72.

Oh, S. Y., Shriram, K., Laha, B., Baughman, S., Ogle, E., and Bailenson, J. (2016). Immersion at scale: researcher's guide to ecologically valid mobile experiments. IEEE Virt. Real . 14, 249–250. doi:10.1109/vr.2016.7504747

Otto, K., and Wood, K. (2001). Product design: techniques in reverse engineering and new product design . Upper Saddle River: Prentice-Hall .

Parsons, T. D., Bowerly, T., Buckwalter, J. G., and Rizzo, A. A. (2007). A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child. Neuropsychol . 13, 363–381. doi:10.1080/13825580600943473

Persky, S., Goldring, M. R., Turner, S. A., Cohen, R. W., and Kistler, W. D. (2018). Validity of assessing child feeding with virtual reality. Appetite 123, 201–207. doi:10.1016/j.appet.2017.12.007

Poelman, M., Kroeze, W., Waterlander, W., de Boer, M., and Steenhuis, I. (2017). Food taxes and calories purchased in the virtual supermarket: a preliminary study. BFJ 119, 2559–2570. doi:10.1108/bfj-08-2016-0386

Pourhoseingholi, M. A., Baghestani, A. R., and Vahedi, M. (2012). How to control confounding effects by statistical analysis. Gastroenterol. Gastroenterol Hepatol Bed Bench 5 (2), 79–83. doi:10.1002/0471476471.ch13

Rollings, K. A., and Evans, G. W. (2019). Design moderators of perceived residential crowding and chronic physiological stress among children. Environ. Behav . 51 (5), 590–621. doi:10.1177/0013916518824631

Rollings, K. A., and Wells, N. M. (2018). Cafeteria assessment for elementary schools (CAFES): development, reliability testing, and predictive validity analysis. BMC Public Health 18, 1154. doi:10.1186/s12889-018-6032-2

R. W. Marans, and D. Stokols (2013). Environmental simulation: research and policy issues . Berlin: Springer Science and Business Media .

Sachs, N. A. (2018). EBD at the macro Level: how research informs policy. Health Environ. Res. Des. J . 11 (4), 108–110. doi:10.1177/1937586718812120

Saeidi, S., Chokwitthaya, C., Zhua, Y., and Sun., Ming. (2019). Spatial-temporal event-driven modeling for occupant behavior studies using immersive virtual environments. Autom. Construct . 94, 381–382. doi:10.1016/j.autcon.2018.07.019

Schwebel, D. C., Gaines, J., and Severson, J. (2008). Validation of virtual reality as a tool to understand and prevent child pedestrian injury. Accid. Anal. Prev . 40, 1394–1400. doi:10.1016/j.aap.2008.03.005

Schwebel, D. C., Stavrinos, D., Byington, K. W., Davis, T., O'Neal, E. E., and de Jong, D. (2012). Distraction and pedestrian safety: how talking on the phone, texting, and listening to music impact crossing the street. Accid. Anal. Prev . 45 (2), 266–271. doi:10.1016/j.aap.2011.07.011

Shi, Y., Du, J., Ahn, C. R., and Ragan, E. (2019). Impact assessment of reinforced learning methods on construction workers' fall risk behavior using virtual reality. Autom. Construct . 104, 197–214. doi:10.1016/j.autcon.2019.04.015

Siegrist, M., Ung, C.-Y., Zank, M., Marinello, M., Kunz, A., Hartmann, C., et al. (2018). Consumers' food selection behaviors in three-dimensional (3D) virtual reality. Food Res. Int. 117, 50–59. doi:10.1016/j.foodres.2018.02.033

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. Br. J. Psychol . 109 (3), 431–433. doi:10.1111/bjop.12305

Slater, M., Khanna, P., Mortensen, J., and Yu, I. (2009). Visual realism enhances realistic response in an immersive virtual environment. IEEE Comput. Graph Appl . 29, 76–84. doi:10.1109/mcg.2009.55

Slater, M., Spanlang, B., Sanchez-Vives, M. V., and Blanke, O. (2010). First person experience of body transfer in virtual reality. PloS one 5 (5), e10564. doi:10.1371/journal.pone.0010564

Sobhani, A., Farooq, B., and Zhong, Z. (2017). “Distracted pedestrians crossing behaviour: application of immersive head mounted virtual reality,” in IEEE 20th international conference intelligent transportation systems (ITSC) , 1–6.

Stelick, A., Penano, A. G., Riak, A. C., and Dando, R. (2018). Dynamic context sensory testing—a proof of concept study bringing virtual reality to the sensory booth. J. Food Sci . 83, 2047–2051. doi:10.1111/1750-3841.14275

Tucker, A., Marsh, K. L., Gifford, T., Lu, X., Luh, P. B., and Astur, R. S. (2018). The effects of information and hazard on evacuee behavior in virtual reality. Fire Safe. J . 99, 1–11. doi:10.1016/j.firesaf.2018.04.011

Ulrich, K. T., and Eppinger, S. D. (2012). Product and design development . 5th ed. New York, NY: McGraw Hill Companies, Inc.

Veling, W., Counotte, J., Pot-Kolder, R., van Os, J., and van der Gaag, M. (2016). Childhood trauma, psychosis liability and social stress reactivity: a virtual reality study. Psychol. Med . 46, 3339–3348. doi:10.1017/S0033291716002208

Viswanathan, A. M., and Choudhury, B. (2011). Virtual reality anatomy: is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anat. Sci. Educ . 4 (3), 119–125. doi:10.1002/ase.214

Waterlander, W. E., Jiang, Y., Steenhuis, I. H., and Ni Mhurchu, C. (2015). Using a 3D virtual supermarket to measure food purchase behavior: a validation study. J. Med. Internet Res . 17, e107. doi:10.2196/jmir.3774

Wiederhold, B. K. (2017). What can behavioral healthcare learn from digital medicine? Cyberpsychol. Behav. Social Netw . 20 (12), 725–726. doi:10.1089/cyber.2017.29092.bkw

Wiederhold, B. K., and Wiederhold, M. D. (2010). Virtual reality treatment of posttraumatic stress disorder due to motor vehicle accident. Cyberpsychol. Behav. Soc. Netw . 13 (1), 21–27. doi:10.1089/cyber.2009.0394

Witmer, B. G., and Singer, M. J. (1998). Measuring presence in virtual environments: a presence questionnaire. Presence 7 (3), 225–240. doi:10.1162/105474698565686

Yaremych, H. E., and Persky, S. (2019). Tracing physical behavior in virtual reality: a narrative review of applications to social psychology. J. Exp. Soc. Psychol. 85, 1–8. doi:10.1016/j.jesp.2019.103845

Zhao, Y., Cutrell, E., Holz, C., Morris, M. R., Ofek, E., and Wilson, A. D. (2019). “SeeingVR: a set of tools to make virtual reality more accessible to people with low vision,” in Proceedings of the 2019 CHI conference on human factors in computing systems , 1–14.

Keywords: immersive virtual environment, human behavior, design, prototype development, environmental psychology, virtual reality

Citation: Neo JRJ, Won AS and Shepley MM (2021) Designing Immersive Virtual Environments for Human Behavior Research. Front. Virtual Real. 2:603750. doi: 10.3389/frvir.2021.603750

Received: 07 September 2020; Accepted: 13 January 2021; Published: 04 March 2021.

Reviewed by:

Copyright © 2021 Neo, Won and Shepley. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Jun Rong Jeffrey Neo, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Effects of virtual learning environments: A scoping review of literature

Laura caprara.

Graduate Studies in Education, St. Francis Xavier University, Antigonish, Nova Scotia Canada

Cataldo Caprara

Associated data.

Not Applicable.

The purpose of this scoping review is to isolate and investigate the existing data and research that identifies if the synchronous face-to-face visual presence of a teacher in a virtual learning environment (VLE) is a significant factor in a student’s ability to maintain good mental health. While the present research on this explicit interaction among VLE implementation and student mental health is limited, the material suggests a framework for strong utilization of VLEs. Overall, our research has shown that authentic, high quality VLEs are ones that have as their primary focus the communication between students and their teachers and between students and their peers. This communication is best generated through synchronous connections where there exists the ability to convey the student’s immediate needs in real-time. Our research results and discussion will outline how a team approach that brings together teachers, students, administration, counsellors, mental health support staff, instructional designers, and ICT specialists is necessary to create a genuinely enriching VLE where both learning and social-emotional needs can be met. The authors present a case for further study in order to reveal the nature of the interaction among VLEs and student mental health.

Introduction

Increasingly educators are turning to digital online tools and resources to supplement and enhance their teaching in the classroom. Independent of the move to online learning as a result of the COVID-19 pandemic, there is a clear upward trend to the prevalence and variety of online learning opportunities available to students of all ages. In their meta-analysis, Means et al. ( 2009 ) show that the rise of online learning is for a variety of reasons:

Online learning has become popular because of its potential for providing more flexible access to content and instruction at any time, from any place. Frequently, the focus entails (a) increasing the availability of learning experiences for learners who cannot or choose not to attend traditional face-to-face offerings, (b) assembling and disseminating instructional content more cost efficiently, or (c) enabling instructors to handle more students while maintaining learning outcome quality that is equivalent to that of comparable face-to-face instruction. (p. 22)

In Ontario, online learning can be accessed through traditional school boards using eLearning Ontario and Brightspace or through available online private schools such as VirtualHighSchool.com. These virtual learning environments (VLEs), online spaces that facilitate the delivery of curriculum content, assessment, and evaluation activities – deliver said curriculum in an asynchronous manner, that is “learning that is not delivered in real time. Asynchronous learning may involve students watching pre-recorded video lessons, completing assigned tasks, or contributing to online discussion boards” (Ontario Ministry of Education, 2020 , PPM 164). This is different than synchronous learning which is defined as “Learning that happens in real time. Synchronous learning involves using text, video, or voice communication in a way that enables educators…to instruct and connect with students in real time” (Ontario Ministry of Education, 2020 , PPM 164). It is believed that “synchronous learning supports the well-being and academic achievement of all students…by providing educators and students with an interactive and engaging way to learn. It helps teachers provide immediate feedback to students and enables students to interact with one another” (Ontario Ministry of Education, 2020 , PPM 164). Despite this fulsome definition of synchronous learning, little research-driven policy or infrastructure exists whereby educators can be trained and supported in creating an ideal VLE for their students’ educational and social-emotional needs (Jones, 2015 ; Kent et al., 2018 ). This was certainly the case for all Ontario educators when, in March 2020, schools were closed due to the SARS-COVID-19 global pandemic and learning shifted to online-only instruction as contextualised in the following section.

On Thursday, March 12, 2020 at 4:00p.m. we tuned into the local news channel to hear that the Ontario Provincial Government was putting the province into lockdown to avert the upcoming threat of the novel Coronavirus, SARS-COVID-19. In this same news conference, we learned that Friday, March 13, 2020 would be our last day in schools; schools would be closed for two weeks after the March Break which began the following Monday. We spent the day photocopying, distributing, and explaining work packages for our students while ourselves wondering how we would negotiate the transition to distance learning and the new expectation to work – teach – from home. After these two weeks, a continuation of the lockdown was announced and Ontario educators were tasked with connecting with students and families procedure to gauge student’s communications technology readiness, to monitor their social-emotional needs arising from the lockdown and to explain that basic emergency learning was now in place.

At the same time, educators were told to create virtual learning environments (VLEs) to deliver this emergency instruction. Hours of instruction guidelines were provided by the Ontario Ministry of Education (April 2020): for JK-Grade 6, no more than six hours per week; for Grades 7–8, no more than eleven hours per week and for Grades 9–12, no more than three hours per course per week. During this time, normal attendance procedures were paused and replaced with anecdotal teacher monitoring and the mode and delivery of teaching was not standardized via a specific virtual platform. Guidance from Ontario teacher unions was to avoid videoconferencing and webcasting (OECTA, 2020 ) to maintain the privacy of both teacher and student.

The unprecedented scope and unforeseen turmoil caused by the COVID-19 crisis resulted in a delayed and mixed response for how educators could meet the daily social-emotional needs of their students who were now being taught in ways that were not conducive to learning such as having no face-to-face communication. Part of this mixed response included the ministry directive that students’ marks and grades would be frozen to the date prior to the initial closure (Miller, 2020 ), though opportunities for improvement would be provided. While the intentions of policymakers were rooted in compassion and wishing not to amplify any feelings of stress, anxiety, or hardship already being caused by the pandemic, it did not serve to inspire students to be continually engaged in their educational community or to feel wholly supported by said community. Overwhelmingly, students in both of our schools stopped engaging in course content; knowing that their grades could not change truly impacted their motivation, diligence, and general commitment to learning. When this disconnect occurred, teachers were advised to continue to call the student and encourage them to login to their VLEs, but this consistent check-in by four teachers at least once a week to each family began to feel at best like nagging and, at worst, an intrusion. This disconnect had another negative effect; now that students were not explicitly required to complete work for school, businesses were free to demand that they work throughout the day. Many students shared that they had taken on full-time hours at their part-time jobs. Had the student–teacher-classroom relationship continued throughout the closure somehow through routine, synchronous, live-video or live-voice communication, perhaps students and families would have felt a social and emotional pull to preserve their work habits and would have benefitted from helping to maintain the class community.

Essentially, our students went from daily, face-to-face interaction that built relationships and promoted positive social-emotional skills to the bare minimum interaction of a posting or an email a few times a week. Compounding this growing detachment was the social isolation that being in lockdown necessitated both from friends and family members who were at work. Students were at home, cut off from teachers, school, and friends, alone with the computer screens.

When schools reopened toward the end of September 2020, protocols and safety standards had been improved and implemented in ways that would encourage some families to opt for in-person learning while for others, the risk was still too great. For these families, school boards in Ontario created whole virtual schools or adopted a form of hybrid learning that would accommodate student from Junior Kindergarten through to Grade 12.

Our personal experience of in-person learning after the initial lockdown was atypical: though there was slight excitement the first day, this waned quickly and by the end of the week students were behaving very differently than they had been in March. Classes were silent; students were not speaking to each other, and group discussion or student–teacher conferences were very difficult to craft and nurture. In cases where students were priorly familiar with one another and the teacher, as was the case in one author’s classroom, students remained quiet, distanced, and generally withdrawn. These personal observations were similarly attested by several colleagues. For one author who had been assigned to the new virtual school, similar observations were made. Here, students from all over the city were being placed in VLEs together, disconnected from their home schools. Every attempt was made to create positive community bonds but still, save a handful of eager students, online classrooms were quiet both auditorily and in the chat box.

Unfortunately, COVID-19 continued to spread rapidly through the province and by the winter of 2021, the threat of the new, more infectious variants of COVID-19 forced the hand of the Ontario Ministry of Education to close schools once again for the remainder of the school year. The most marked difference in this lockdown is the continuation of the collection of attendance – students are at least driven to login to their VLEs to have their attendance recorded to avoid the consequences of truancy. Many students have shared with the authors that this second lockdown had evaporated their hopes of any social-emotional normalcy for their educational experience. They have attested that they are tired of feeling alone, overly challenged at the thought of having to continue their learning experiences solely through their computer screen.

The path to inquiry

In the summer of 2020, The Hospital for Sick Children ( 2019 ) released a report that detailed how online learning and increased screen time could result in an increase of negative mental health effects. In addition, the prevailing advice of Public Health Ontario (2015) for the total number of hours of screen-time children and youth should be engaged in had not moved from the recommended “no more than 2 hours per day”. Instead of heeding the warning and advice of these public institutions, the Ontario Ministry of Education Policy/Program Memorandum No. 164 ( 2020 ) defining synchronous learning as “using text, video, or voice communication” in real time, set in place the following expectations for daily synchronous learning: 180 min for Kindergarten and 225 min for Grades 1–12.

Our desire to study the mental health effects of online learning takes its root in the experience of students and their guardians who have directly voiced difficulty with engagement in online schooling. Specifically, our students shared that their experiences in courses facilitated by teachers who did not utilize real-time video for the delivery of their synchronous lessons were difficult, hard to manage, and at times felt alienating.

Certainly, the few noted anecdotes above cannot constitute a direct cause for this review. Nonetheless, it is the view of the authors that such comments appearing from within the unique context of the mandated and sweeping switch to online learning warrant an investigation of what research might exist to establish acceptable standards of VLE implementation. In the case of the COVID-19 context, it must be understood that the changes to educational architectures were motivated by the physical requirement of stopping the spread of disease rather than what research has shown to be the best mode of education for the learner and their mental health needs. Further, even within the confines of mass VLE implementation, deeming the use of real-time video as strictly optional instead of prescribing a definitive modality for engaging in best practice displays a gap in an understanding of VLE significance. Further still, as online learning continues to be a the only viable option for many families, educators in Ontario must ensure that the creation of a VLE and the curriculum delivered through it is guided by the Ontario College of Teachers (OCT) Ethical Standards of Care , where it is understood that Ontario Teachers ought to avoid practices that may not support the welfare of all students placed in their care.

The Ontario College of Teachers ( 2020 ) defines the ethical standard of care as follows: “Members express their commitment to students' well-being and learning through positive influence, professional judgment and empathy in practice.” Further, the Ontario Education Act, 1990 Sect. 264(1c) clarifies that part of a teacher’s duty is to impress the “highest regard for…humanity…and all other virtues” to their students. In addition to the duty of a Principal to provide “assiduous attention to the health and comfort of the pupils” (Ontario Education Act, 1990 , Sect. 265(1j)) they must do so in such a way that they set the standard for the teacher. Without daily, live interaction with students, how is it possible to wholly meet these expectations?

Interestingly, the concept of all persons having to remain at home during the new COVID-19 online education paradigm seemed to create a vague understanding of whether it was even possible for the professional liability of care to take place. In consideration of the area of attendance, it must be understood that the traditional taking of attendance has a layered purpose: to ensure the “safe arrival” of students, to give proof of student attendance for funding allocation, and to link a student to the legal liability of a teacher as well as to allow teachers to exercise interventions for students who are truant. In the first stage of lockdown, teachers were asked to not take attendance and so were unable to ensure regular student contact. In this way, it became much more difficult to distinguish which students were not able to keep up with their studies and who required appropriate interventions for their learning, development, and wellness.

Tronick et al. ( 1978 ) showed that when the face-to-face interaction between infants and their mothers becomes distorted in such a way whereby the mother is unresponsive and still, “infants reacted with intense wariness and eventual withdrawal” (p. 1). They concluded that it was vital to have “interactional reciprocity” (Tronick et al., 1978 , p. 1) to learn how to regulate their own emotional reactions. In their meta-analysis of this Still-Face Paradigm, Mesman et al. ( 2009 ) found that not only had the paradigm been consistent through multiple studies in infants, but its negative effects could also be found in both youth and adults. Experiments with adults resulted in “quite severe disrupting effects on social interactions, making people angry, confused, or upset…[where] [t]he perceived necessity for following the ground rules of social interactions is likely to stem from the evolutionary roots of human social life” (Mesman et al., 2009 , p. 156). These results would suggest that part of the educator’s duty of care to a student is to maintain this quintessentially human behaviour of consistent face-to-face interaction. Without this fundamental interaction, it seems that humans are unable to properly regulate their emotional expression. Certainly social-emotional learning and the formation and maintenance of positive relationships with others is a core part of the care mandate of teachers.

Purpose of scoping review

While it could be surmised that the guiding education authorities’ combined lack of clarity in declaring best practice directives and providing systems of care contributed to negative student experiences, the notion of how these organizational failures might have impacted student mental health must be drawn into focus. Indeed, the most notable shift in education has been toward the use of the online classroom and it is foreseeable that the continuation of this paradigm will be unavoidable. However, with the ambiguous messaging that surrounds that which defines quality implementation of a VLE, there lies a discernible gap in gauging the importance of teachers using live video in the delivery of synchronous instruction and if significant mental health implications are present in the decision to do so.

The purpose of our scoping review is to isolate and investigate the existing data and research that identifies if the real-time visual presence of a teacher in a VLE is a significant factor in a student’s ability to maintain good mental health. Such research might reconcile the void of definitive directives educational authorities have offered concerning teachers’ utilization of real-time video in a VLE and, more importantly, provide students with a higher standard of care while enduring the context of increased vulnerability the pandemic has introduced.

In short, this scoping review aims to answer two questions:

• Does the synchronous face-to-face and interactive presence of a teacher in a VLE contribute positively to student learning and mental health and well-being?
• What are the characteristics of a VLE that meets the social-emotional care requirements and needs of the student?

Method and parameters

This scoping review works to capture the current “size and location of the literature” (Anderson et al., 2008 , p.7) to better define the state of understanding of our research questions. Further, it is our hope that this scoping review might work to ascertain common threads among the research and note any gaps to inspire further inquiry in the topic. All of the material will be charted so it can be viewed in tandem for the above purposes.

Boundaries of the review

Our search was limited to English-only peer-reviewed research literature that investigated the efficacy of online teaching and VLEs for educating the whole student. More specifically, we completed a search of all research articles published between 1 January 2010 and 31 January 2021 (an 11-year period). The rationale for focusing on this span of just over a decade was to draw our attention to the most recent research evidence related to our question. This date range also reflects the time span in which we had secured permanent work as educators; tracing the development and evolution of online learning since that point is of great personal interest.

Our search utilized the following terms: virtual learning environment (in all text) OR online instruction (in all text) AND mental health (in all text). These search terms were selected as they provided the widest catchment for the screening process required for this scoping review; in particular, the search term virtual learning environment was selected to identify research that was specific to an interactive context of online learning, as opposed to, for instance, online learning structured as a correspondence course. Databases that we utilized include ProQuest, ProQuest/ERIC, APA PsychINFO, and SAGE Journals Online because together they encompass a comprehensive and diverse catalogue of education-related literature as well as research regarding the psychological aspects of education. These database searches occurred between January 31, 2021, and February 7, 2021.

Special considerations of the topic search: defining a student

Ideally, the search terms could have been narrowed to refer to students specifically in a high school setting, using common platforms such as Google Classroom, but these terms did not yield any results. In an effort to compile the most useful sources for our research, a broadening rather than narrowing approach to our search terms was employed as it was discovered that limited resources existed in the field we were exploring. Considering this, the limits we set required special considerations of demarcation. For instance, many sources pertaining to the student mental health in a VLE context dealt exclusively with nursing students (e.g., Shea & Rovera, 2021 ). These studies offered excellent data sets but were omitted because the data could not be understood as congruous with a universalized definition of what constitutes a student. That is, the specialized nature of these studies involved students who had the responsibility of dealing with patient care, using VLEs to interact with their patients. In this way, the nursing paradigm muddied the demarcation between the VLE experience of a student and that of an authority.

Inclusion criteria

In initially assessing and screening the results of the database searches, we ventured to remove sources that did not pertain to the contexts of VLEs and the student’s experience of those VLEs. Here, a primary list of 268 articles was redacted to a sum of 63 articles via an evaluation on the content of the title and abstract of each article. The inclusion of the remaining 63 articles was based on their titles and abstracts having expressed: (1) the use of an online modality that would implicitly involve the visual presence of an educator, and (2) the interest in the contexts of the learners’ experiences being a result of the instruction provided. Sources that were concerned with the experience of post-secondary students were only included if they involved Year 1 students as part of their purview. This limit was imposed on the research as we perceived this domain of data as relevant to the prospect of constructing an information-set surrounding the transitional stage of Grade 12 into post-secondary. Further, this inclusion might allow for a greater basis of insight into the still emerging reality of widespread online education. Notable exclusions from the research that might have helped develop such a picture were many articles that focused on the VLE experiences of graduate and post-graduate level students (e.g., Chugani & Houtrow, 2020 ; Gardner, 2020 ; Shawaqfeh et al., 2020 ) and articles concerned with the mental health of educators who utilized VLEs (e.g., Watermeyer et al., 2021 ; Rowe et al., 2020 ; Ault et al., 2020 ; Alkarani & Thobaity, 2020 ; Schlesselman, 2020 ). The subsequent 63 articles were read in their entirety to determine whether their content would be appropriate for this scoping review. This stage of assessment resulted in the inclusion of 38 articles.

Manual inclusions

Additionally, a reference check of all 38 articles yielded the inclusion of a meta-analysis authored by Cavanaugh et al. ( 2004 ) we deemed relevant insofar as it provided an overview of the effects of online learning in a strictly academic sense allowing a possible comparison between achievement and positive mental health. A manual search conducted prior to the exercise of prescribed database search-terms warranted the inclusion of one other meta-analysis authored by Mesman et al. ( 2009 ) which is noted in the introduction section of this scoping review. The specific data and information presented in the work of Mesman et al. ( 2009 ) serves as a correlational tool in observing the trends apparent in the included articles. As a final article addition, a second manual journal search was conducted where two studies were found to include data that encompassed student perceptions of their experiences with VLEs, thus, resulting in a total sum of 42 articles to be included in this scoping review. The total review process is noted visually in Fig.  1 .

Review process and results

Summary of research findings

The purpose of this scoping review is to isolate and investigate the existing data and research that identifies if the real-time visual presence of a teacher in a virtual learning environment (VLE) is a significant factor in a student’s ability to maintain good mental health. Overall, our research has shown that authentic, high quality VLEs are ones that have as their primary focus the communication between students and their teachers and between students and their peers. This communication is best generated through synchronous connections where there exists the ability to convey the student’s immediate needs in real-time. The demands placed upon an individual educator to facilitate an online education for students, inclusive of the creation and maintenance of an effectual and engaging VLE often serving as a proxy to a face-to-face (F2F) base-school, “require[s] extensive time commitments” (Wingo et al., 2016 , p. 437) which are far beyond the standard workload formulas calculated by many institutions (Wingo et al., 2016 ). Indeed, the complexity of this task while also providing sufficient social-emotional services for students and families is not fully understood in terms of student and teacher equity. Our research results and discussion will outline how a team approach that brings together teachers, students, administration, counsellors, mental health support staff, instructional designers, and ICT specialists is necessary to create a genuinely enriching VLE where both learning and social-emotional needs can be met.

Summaries of the 42 articles pertaining to this research are provided in Table ​ Table1. 1 . In addition to information related to authors, years of publication, countries (i.e., where the research was conducted), participant profiles, intervention programs and timelines (where available) and data sources, we have also summarised the researchers’ aims, research designs, findings, and conclusions.

Research summary of the various iterations of virtual learning environments (VLE)

Note:  3DVR: 3D Virtual Reality, 3DVE: 3D Virtual Environment, AR: Augmented Reality, BLM: Blended Learning Model, F2F: Face-to-Face, LMS: Learning Management System, SCR: Screen Capture Recording, VLE: Virtual Learning Environment

Our research provided studies that had been conducted from nearly every continent of Earth, save South America and Antarctica. Notably, many of the articles were completed by researchers in the United States ( n  = 16). Fifteen were completed by researchers in Europe (Belgium, Cyprus, Georgia, Germany, Greece, Poland, Portugal, Slovenia, The Netherlands, Turkey, Ukraine, and the United Kingdom), and six were completed by researchers in Asia (India, South Korea, Taiwan, and Vietnam). The remaining research was conducted in Australia, Canada, Jordan, and Kenya. The research designs employed by the various researchers in our summary are quite varied and sometimes particularly nuanced and range from case study to meta-analysis. However, a preference for experimental, quasi-experimental, qualitative analysis, and mixed-method correlational analysis did emerge. Overall, few articles had a singular focus, which can be attributed to the expansive field of VLEs and their many intricate pieces. Some central concepts were highlighted, however, in the research: fifteen observed the effectiveness, benefits or challenges of some aspect of VLE (i.e., using virtual reality simulations or synchronous video); ten investigated student readiness, either on a social-emotional basis, technology know-how, or academic; six focused on the mental health of the learners in VLEs; six gathered data on the experiences and perceptions of either students, students’ families, or faculty, and four researched the current infrastructure and available policies for online learning. The participant profiles of our research were also quite varied, identifying elementary and secondary school-aged children, both undergraduate and graduate students, school faculty, and student family members, sometimes all within the same study.

Our research results have been organized in Table ​ Table1 1 below. It has been constructed in such a manner whereby an informational narrative that reflects the essential themes found within the research can be revealed.

Results and discussion of themes

The articles included in Table ​ Table1 1 represent the most current and relevant research in considering the embedded inquiry of this scoping review which involves uncovering the nature, implications, and best iterations of practice within VLE contexts. In our reading and review of the data therein, the themes of insufficient data surrounding VLEs, VLE benefits, the challenge of VLE readiness, and that which constitutes the ideal VLE emerged as pivotal. The objective of this section is to elucidate these themes, thereby, providing a modest basis for recommendations regarding VLE implementations and, perhaps, a view to offer directionality for future research.

Insufficient data

A key note thread found within many of articles was the self-admission of insufficient data. This theme of insufficient data is expressed in varying capacities that range from claims of there being a limited or even non-existent body of research, to more systemic causes for the insufficiencies. While the lack of data is often presented as a cautionary device for the demarcation of limits to implementation outside the context of the studies and provide exhortation for further research to be conducted, the admissions of insufficient data also point to the novel nature of the area of inquiry in question. Kumar and Owston (2015) begin their study on e-learning accessibility by stating that their field of inquiry had “not been explored, nor have methods to generate data” (p. 264) expressing that there is “a dearth of studies'' (p. 268) in the literature, and concluding that “[c]ontinued work in the area of developing methods to evaluate e-learning accessibility is thus urgently needed” (p. 280). Archambault et al. ( 2013 ) also identified their research scope of basic virtual school policies as being novel in nature, having no representation in the existing literature. Many researchers make note of the existing data as being too insufficient to draw more universal conclusions (Barbour & LaBronte, 2019 ; Cavanaugh et al., 2004 ; Engelbertink et al., 2020 ; Gillis & Krull, 2020 ; Ho et al., 2014; Jena, 2016 ; Zhu & van Winkel, 2016 ). In addition to this paucity of research, the attrition of study participants is noted as being a barrier to gathering full data sets (Manthey et al., 2016 ).

Some systemic issues which led to shortages in the available data are noted in Johnston et al. ( 2014 ) where school districts are slow to institute policy. Cavanaugh et al. ( 2004 ) mentioned a similar dynamic in considering that common goals are needed in policy making to identify the effectiveness of an intervention and policy makers and evaluators are exhorted to work together in partnership to ameliorate this. A further systemic barrier to data production that is noted is the problem of implementation of programming without conducting research (Cavanaugh et al., 2004 ).

Benefits of VLEs

In response to our first research question regarding the benefits of a wholly synchronous VLE experience, the research is generally favourable toward academic achievement with some degree of attestation to its social-emotional benefits. The benefits to VLEs and their implementation are assumed among most of our research in how they can be potential vehicles delivering some form of meaningful intervention or program within a given context. Further, some of the articles underline fundamental goods that can be uniquely exploited via VLEs. Driscoll et al. ( 2012 ) cites VLEs as an opportunity to better promote a constructivist framework for learning in saying that it inherently “creates a structural impetus for this style of learning that is not automatically present in F2F classrooms” (p. 314). Cavanaugh et al. ( 2004 ) provides multiple examples of how the institutional advantages of virtual schools “represent the best hope for bringing high school reform quickly to large numbers of students” (p. 22). Building upon the pervasive benefits to VLEs as a concept, Roblek et al. ( 2019 ) frame VLE dynamics as an essential component of human advancement where “social relations will be formed through the building of collective intelligence” (p. 96). Similarly, VLEs and their relation to ICT literacy as a global objective is observed throughout the research (Blayone et al., 2018 ; Cavanaugh et al., 2004 ; Crea & Sparnon, 2017 ; Davies, 2014 ; Gibson & Smith, 2018 ; Huang et al., 2011 ; Hursen, 2019 ; Jena, 2016 ; Mallya et al., 2019 ).

The strengths of specifically synchronous VLEs emerge in the research with highlighting synchronous learning as an essential component to student engagement with technology, peers, and educators. Concerning technology fluency, even in a blended learning context, synchronous VLEs offered a unique opportunity to implement technology in a meaningful way (Ho et al., 2016 ). Using a device in a synchronous context meant that students felt more engaged with material, subsequently feeling more confident with presenting work using technology, and students enjoyed being able to revisit an interactive lesson digitally after the synchronous session was over (Davies, 2014 ; Driscoll et al., 2012 ; Kumar & Owston, 2016 ). In terms of supporting engagement among classmates, synchronous learning was seen to offer increased avenues for peer-to-peer learning while allowing for teacher involvement throughout, thus increasing effectiveness (Crea & Sparnon, 2017 ; Johnston et al, 2014 ). Synchronous VLEs that include video also offer opportunities to be present to a class setting in a way that attends to learning retention, academic engagement, resiliency, and self-regulation (Archambault et al., 2013 ; Driscoll et al., 2012 ). When VLEs employ best-possible real-time communication, education processes can be more active, constructive, cooperative, and more attentive to a student’s meta-cognitive abilities than the traditional classroom (Cavanaugh et al., 2004 ). These latter points concerning real-time visual instruction potentially align with a foundational dynamic noted by Mesman et al. ( 2009 ) where it is stated that an “infant needs an external regulator to achieve optimal arousal levels and will show disorganization of emotion and behaviour when the regulator is absent or non-optimal” (p.122). Such a relationship becomes apparent in the work of.

Baker et al. ( 2019 ) which observed quiz results decrease among those students whose instructor withdrew communication and synchronous availability after originally being quite attendant to their needs and in the work of Engelbertink et al. ( 2020 ) where student motivation dropped significantly when the teacher no longer demonstrated an interest in the student’s homework. Throughout the research, it is evident that student engagement and achievement is well-supported in a synchronous VLE.

The barrier to a VLE: the challenge of readiness

Across all our research, it became clear that one of the primary factors curtailing the effectiveness of any VLE or LMS was the various states of readiness of the institution, the teacher, and the student.

At an institutional level it can be said that most schools are not equipped to create VLEs where students can thrive, even those schools that are virtual by design. The infrastructure required to create a holistic learning experience for the student, and one that embodies fair and equitable working conditions for the online educator, requires a considerable investiture of human resources and technological tools (Archambault et al., 2013 ; Cairns et al., 2020 ; Jones, 2015 ). Many LMSs that institutions use for online learning are bulky and inefficient (Gillis & Krull, 2020 ; Jones, 2015 ; Kumar & Owston, 2016 ; Lee et al., 2016 ) which can lead to their being used as places where information is simply disseminated, rather than genuine VLEs where the design and curriculum content can come together to connect students with each other for interaction and collaboration (Jones, 2015 ; Stone, 2019 ). Elementary schools, for instance, can be said to provide many opportunities for families to increase their informal social capital and high schools, colleges and universities often provide a student with guidance and counseling services not easily accessible elsewhere. In moving to online learning, these institutions must not forget their “organizational brokerage” (Domina et al., 2021 , p. 4) in facilitating and maintaining these social connections lest their students suffer in isolation (Crea & Sparnon, 2017 ). Ultimately, the VLE experience begins with the institution; if there is no commitment to ensuring the use of a high-quality LMS and no focus on securing and maintaining the human resource social supports that students and families have come to rely on the school to provide, then the mental health and academic achievement of its students can deteriorate (Cairns et al., 2020 ; Cavanaugh et al., 2004 ; Domina et al., 2021 ; Gillis & Krull, 2020 ; Jones, 2015 ; Lee & Oh, 2017 ; Merlin-Knoblich et al., 2019 ; Rogowska et al., 2020 ; Stone, 2019 ; Xavier et al., 2020 ; Zhu & van Winkel, 2016 ).

As Blayone et al. ( 2018 ) points out, vital to the VLE experience is “high quality activity design, strong environmental and motivational supports, and competent online facilitators” (p.15). Teacher readiness for both the technological scope of VLEs and for the new expectations that they are the sole social-emotional support for students and families (at the very least a proxy to such supports) is generally low. Training is essential for educators who are navigating new technologies and creating resources that provide meaningful opportunities for knowledge construction, reflection, and practice (Davies, 2014 ; Gibson & Smith, 2018 ). Teachers must also be taught how to “adjust and find their own rhythm, providing sufficient presence while avoiding feeling perpetually ‘on call’” (Jones, 2015 , p. 227). Teachers lacked access to suitable training and felt ill-prepared to offer and provide to students with special needs or disabilities the appropriate accommodations within the VLE (Kent et al., 2018 ). Substantial professional development is needed to ensure that teachers know how to provide social opportunities in the VLE that encourages group work, formal and informal interactions, and peer-to-peer cooperative learning (Cavanaugh et al., 2004 ; Johnston et al., 2014 ; Zhu & Van Winkel, 2016 ). Cultivating this social-emotional component is an essential task of the online educator; when a student can trust their teacher and their classmates, their self-efficacy and motivation increases and generally so does their performance and progress (Johnston et al., 2014 ). To accomplish this, institutions must increase their efforts in training and supporting their faculty to be ready for online instruction (Crea & Sparnon, 2017 ).

Jena ( 2016 ) defines student learning readiness as “the body of skills needed by learners to learn” (p. 950). This body of skills and aptitudes includes, but is not limited to, motivation, self-regulation, perceived usefulness, confidence with using various technology, attitude, self-efficacy, computational abilities, communication skills, and research and critical thinking competence (Baker et al., 2019 ; Blayone et al., 2018 ; Du et al., 2019 ; Hursen, 2019 ; Johnston et al., 2014 ; Jones, 2015 ; Mallya et al., 2019 ). Beyond these attributes of learning readiness is also a strong necessity for a certain level of social-emotional maturity, most especially if the online learning was a result of the COVID-19 pandemic or of illness (i.e., not a free choice). Soft qualities such as resilience, flexibility, and positivity (Lee & Oh, 2017 ) made it more possible for students to survive the transition from the routine and collaboration of a physical classroom to the more solitary and independent learning space of the VLE (Crea & Sparnon, 2017 ; Gibson & Smith, 2018 ; Jena, 2016 ). In addition to these crucial factors, is the technology-readiness of students. Students may not have access to their own personal device to do their schoolwork, and if they do, there is no guarantee that it is a device equipped with the sufficient technological specifications to handle the resource heavy online tools or that the student has access to high-speed internet to allow full and equal participation in the lesson and VLE (Domina et al., 2021 ; Gillis & Krull, 2020 ; Hursen, 2019 ). It cannot be assumed that because students use technology at very high rates for personal relationships and entertainment that they can directly transfer those skills to the sophisticated and critical digital literacy necessary and conducive to learning in a VLE (Blayone et al., 2018 ; Roblek et al., 2019 ). Indeed, the various online tools that are familiar to institutions and educators are rarely in the purview of students, though when the need arises, students do want to be taught how to use the many programs and LMSs available to them effectively (Stone, 2019 ) and thus system readiness, student readiness, student inclusion, student achievement and teacher readiness are inseparable (Huang et al., 2011 ; Kumar & Owston, 2016 ; Pryjmachuk et al., 2012 ; Yilmaz, 2019 ).

The ideal VLE

Among the reviewed articles, the answer to our second research question concerning the criteria of an ideal VLE emerged. VLEs which supported students both academically and emotionally and whereby online educators were engaged and motivated were highly organized and inventive, and if given that no barriers of readiness existed, could be implemented in every school system willing to pivot to this necessary focus. Firstly, policies and procedures that focus on the progress and social-emotional needs of the student must be in place (Archambault et al., 2013 ). This can only be achieved if a full set of human resources such as guidance teachers, attendance officers, counsellors and special education resource teachers are available both on a central campus and online (Johnston et al., 2014 ) offering “inclusion, communication, connection with others and proactive institutional support” (Stone, 2019 , p. 7) by way of a school-home mentorship model (Barbour & LaBonte, 2019 ). In this way, the student’s isolation is lessened and, united with the educational team, the VLE teacher can focus on lending their subject and pedagogical expertise to their students (Driscoll et al., 2012 ; Du et al., 2019 ; Engelbertink et al., 2020 ; Wingo et al., 2016 ; Zhu & van Winkel, 2016 ). Secondly, the VLE must be easy to use, accessible, flexible, and innovative. Institutions must select uncomplicated LMSs for teachers to use to deliver their program. The expectations of use must also be communicated to all faculty to ensure a seamless experience for students (Jones, 2015 ). As well, in either a synchronous VLE or BLM, having easy access to recorded lessons is crucial, especially for students with disabilities or who are still learning the language (Davies, 2014 ; Dommett et al., 2019 ; Kumar & Owston, 2016 ). Investment in innovative tools and technologies is necessary to keep the VLE from becoming stagnant for students and, depending on the technology, can promote healthy, rich, and meaningful student interactions (Du et al., 2019 ). There is promising research in the use of tools such as AR, VR, 3DVR and 3DVE to create experiences and spaces that allow students to attend to one another virtually. These tools help to cultivate positive relationships, academic and personal confidence, and good mental health (Huang et al., 2019 ; Lan et al., 2018 ; Papanastasiou et al., 2019 ; Stone, 2019 ). Thirdly, there must be, at best, a live-video synchronous component to the VLE, or at minimum, the availability of synchronous office-hours (Stone, 2019 ; Wingo et al., 2016 ; Zeren, 2015 ; Zhu & van Winkel, 2016 ). When students and teachers were engaged face-to-face, body language and tone could be better understood and relationship markers such as trust and care could be better perceived (Driscoll et al., 2012 ; Johnston et al., 2014 ; Wingo et al., 2016 ). Finally, the VLE must engage students in becoming digital citizens together. VLEs that provide opportunities for students to engage formally and informally enable students to increase their academic self-efficacy, increase their learning outcomes, and mitigate any mental health issues that may result from the perceived isolation of online learning (Driscoll et al., 2012 ; Du et al., 2019 ; Engelbertink et al., 2020 ; Johnston et al., 2014 ; Stone, 2019 ; Yilmaz, 2019 ; Zhu & van Winkel, 2016 ).

Discussion of gaps and limitations in the research and suggestions for further inquiry

The attempt to study any observable intersection of VLE implementation and student mental health presents unique logistical and philosophical queries that remain unquelled. Such wonderings involve the state of how participant numbers are determined, the founding modalities in which self-reported qualitative data is obtained, the rationale, or lack thereof, of why specific LMS platforms were used in the existing studies, and the generally perceived evolving nature of VLEs. Taken together, the various streams of inadequate information fret deeply and, perhaps, create quite significant gaps. In the following discussion of these gaps, we will humbly aim to make moderate suggestions for further inquiry that could enrich the current available research.

Concerning the limitations in obtaining meaningful participation, a key area that remained challenging among the research was ensuring that participant profiles were not assembled out of simply convenient contexts of implementation. Indeed, quality research is exhorted to communicate, as narrowly as possible, the contexts in which they are situated. However, our search yielded a number of studies that were isolated case studies (e.g., Archambault et al., 2013 ; Johnston et al., 2014 ; Jones, 2015 ; Kumar & Owston, 2016 ) or were relegated to being singularly quasi-experimental (e.g., Blayone et al., 2018 ; Davies, 2018; Driscoll et al., 2012 ; Ho et al., 2016 ; Huang et al., 2011 ; Lee et al., 2016 ) in nature due to the fact that their implementation was imposed upon pre-existing participant groupings – those who happened to be enrolled in the class that was chosen for intervention. In extension to this, adequate control conditions were not always apparent, especially those which considered many factors that were changed in the experience of intervention groups. That is all to say that the interventions themselves were multifaceted, and one could surmise a possible inability to distinguish which key facet or combination was pivotal in the intervention. This issue may be considered a specific function of the sheer complexity of studying VLE implementations themselves. It is further compounded in the noting of pre-existing intervention groupings as it is perhaps the result of simple pragmatism in observing VLE implementations where they are available to be observed. This point recognizes that VLEs require specific access to resources that may be limited, making widespread and universally approachable studies a challenge. Here, it is possible that an underlying dynamic exists in the research where actioning any opportunity for study, however limited, is better than conducting no study at all. In our view, further inquiry into VLE efficacy and its relation to the mental health of students, should endeavour to include randomized trials, whereby there is no observed previous relationship between the intervention group and the researcher.

Another limitation to this scoping review related to participant selection is the scale and size of many of the studies. Several studies combined the type of participant, blending the experiences of students, faculty, and education support staff, thus limiting a focus on the unique perspective of the student as the end-user (e.g., Crea & Sparnon, 2017 ; Dommett et al., 2019 ; Engelbertink et al., 2020 ; Johnston et al., 2014 ; Stone, 2019 ; Wingo et al., 2016 ). Additionally, some studies that reported findings concerning students directly were of an extremely small student sample size of thirty or less (e.g., Cairns et al., 2020 ; Dommett et al., 2019 ; Engebertink et al., 2020 ; Hursen, 2019 ; Johnston et al., 2014 ; Kumar & Owston, 2016 ; Lan et al., 2018 ; Wingo et al., 2016 ; Zeren, 2015 ; Zhu & van Winkel, 2016 ). We note this small sample size in order to frame the perceived usefulness of these studies in the Ontario education context noting that Ontario Regulation 484/20, s. 4(14.1) states that “the average size in a school year of a board's online learning classes shall not exceed 30”. It is our view that findings of studies with a less than thirty sample size should be interpreted cautiously, as the dynamics and pressures conspicuous in an average sized class cannot be accurately measured. For further inquiry, we would suggest research that included groups of whole divisions across multiple school boards allowing for parallel interpretation and consistency.

A further limitation of this scoping review is the lack of consistency in LMS research. In as many facets as teachers differ so too do the online VLE tools that may be utilized to deliver programming and the effects of each LMS’s nuances can be difficult to account for and isolate as non-contributing factors within the studies. Several studies looked specifically at the Blackboard LMS (e.g., Crea & Sparnon, 2017 ; Davies, 2014 ; Du et al., 2019 ; Engebertink et al., 2020 ; Kent et al., 2018 ; Lee et al., 2016 ; Pryjmachuk et al., 2012 ) noting that in most cases its use was pragmatically chosen as it was already in use by the hosting institution. As well, multiple studies looked at either outdated programs, such as Facebook and RSS feeds (e.g., Huang et al., 2011 ; Hursen, 2018; Yilmaz, 2019 ) or expensive and new technology, such as 3DVR and iPads (e.g., Davies, 2014 ; Huang et al., 2019 ; Lan et al., 2018 ), that would be quite financially out of reach for most Ontario school boards to implement in any widespread and equitable fashion. Overall, researchers instead focused on studying only the perceptions of online learning in general or one specific piece of the online learning experience (for instance, the posting of recorded lectures or an asynchronous discussion board section) without giving any precise attention to the LMS used to create the VLE. In fact, it can be seen that the largest gap in the research is ignoring the LMS as a true, unto itself environment. We find this to be crucial to our research focus of determining how the perceived humanity of the VLE affects the mental health of the student; after all, it is the space in which the student will be spending most of their learning hours.

Many of the studies relied upon qualitative data gathered via surveys during the intervention which required participants to self-report on their perceived well-being and mental health. This paradigm of understanding reads as akin to consumer-based research where one who is satisfied with a product is more likely to repeat consumption regardless of whether the product ultimately increases quality of life. Just as the terms pleasure and contentment are not interchangeable, in the context of the research, it remains unclear if a participant’s self-reporting on perceived levels of anxiety is congruent with clinical definitions of the terms. While studies which utilized this form of data collection are free of equivocation by way of maintaining qualifying language such as “perceived level of anxiety” and not simply state “anxiety”, the question of the results being meaningful still remains. A suggestion for further inquiry may entail the implementation of standardized data sources such as the PSS-10 , the CES-D , and the MBI-SS utilized by Lee and Oh ( 2017 ). Caution must be employed when developing questionnaires, interview questions, and surveys that provide opportunities to participants for open-ended self-reporting.

A final point on the limits of VLE research rests in the concern that LMSs may evolve at rates that do not allow for consistent implementation and ample research to be conducted in a timely manner. In an earlier section of this scoping review, we referenced the lack of research as a product of the field being relatively novel in nature. However, for several years now, LMSs have offered functionalities that extend into the realm of real-time collaboration that is inclusive of visual presence with teachers as well as being fully capable of allowing students fluid peer-to-peer real-time engagement. In considering this it is astonishing that our research showed such a vast range in how VLE operated in relation to the available technology. This has led us to wonder if the technology is available, why is it not being utilized and studied in a way that reflects its full capabilities? Aside from the concepts of readiness that have been earlier itemized and discussed, the level of investment that an institution is willing to make of a platform and the rate of making that LMS available with a fully optioned suite, becomes the pivoting element. Further, if a gradual investment model is employed, where not all features of the LMS are available at the onset, each time a new feature is doled out, it becomes a potential point of relearning and creates inefficiencies if there are not ample professional development opportunities for educators.

While determining our final research terms for this scoping review, we had initially searched for studies that were exclusively for the K-12 sphere and unsuspectingly this did not prove to be fruitful. We noted earlier that a peculiar number of studies had been published using nursing students as the study participants, reporting on their role as student and practitioner. We believe that the near exclusivity of undergraduate students in the research limited our ability to present a complete picture of the current state of VLEs for all students. In The Human Face of Mental Health and Mental Illness in Canada , the Government of Canada ( 2006 ) reported that “two-thirds (68.8%) of young adults aged 15–24 years with a mood or anxiety disorder reported that their symptoms had started before the age of 15” (p. 34). Boak et al. ( 2016 ) support these conclusions, showing in their report that “one-third (34%) of students indicate a moderate-to-serious level of psychological distress (symptoms of anxiety and depression)” (p. iv) and “one-in-eight (12%) students had serious thoughts about suicide in the past year” (p. iv), a statistic that has remained consistent over the past fifteen years of reporting. Given these unsettling statistics, we would have thought that the rich and varied K-12 arena would have been a sphere where there would be a surfeit of mental health research related to VLE utilization. Instead, our search results yielded studies that disproportionately represented participant groups outside of our desired range of inquiry; just six included the experiences of K-12 students, and in each of those studies, the students’ caregivers were the primary data source. K-12 students often spend more than half their waking hours in school environments and VLEs; it is notably unclear why most of the studies observed in this scoping review are unilaterally disinterested in exploring an identified area of need for mental health support. We believe it would be prudent to prioritize research on the mental health of K-12 students engaged in VLEs which Domina et al. ( 2021 ) has shown can be isolating, psychologically disruptive, and exhausting experiences.

Suggestions for educator practice

Though there is variety in the identified gaps detailed above, our research maintained a consistent thread that related to the criteria of the ideal VLE for both the success of the educator but also the well-being and dignity of the student. From this work, we endeavour to make a few moderate suggestions for online educators:

• Where possible and where privacy concerns can be mitigated, conduct lessons and office hours using live videoconferencing. Whether the VLE is a secondary component in a BLM arrangement, or it is the primary mode of program delivery, maintaining a personal face-to-face connection is an essential component to a student’s feelings of connectedness and motivation.
• Avoid viewing the VLE as merely a space to bank work packages and collect evaluations. Rather, aim to create a space where both formal and informal interactions can occur, synchronously and asynchronously. Many LMSs have the capability to incorporate a variety of third-party online teaching and learning tools to aid educators in creating a multi-modal experience for students.
• Be vulnerable and take advantage of learning opportunities when they become available. It takes an enormous amount of energy and resources to run stimulating programs that speak honestly to curriculum content, allow for individual learning needs, and that are cognizant to the social-emotional well-being of students. The responsibilities and conditions of an online educator are well-primed for strain; be mindful of the added pressure and allow the professional development that is available to inform practice but not make hurried demands of that practice.

If educational research involves an ethical component, it would be incumbent on institutions to see that research reflects areas of need within communities. It is our hope that this scoping review might provide modest insight into the current state of research that concerns student mental health in VLE contexts, while casting light on the need for new research initiatives to be undertaken in the K-12 sphere. As it stands, there lies the strong possibility that K-12 students are experiencing VLE implementations that do not actively partake in the qualities of a VLE that soundly offer best practices, working to support the mental health needs of students. To build strong VLE’s for K-12 students, research campaigns ought to offer architectures that are universalized in their implementation and fundamentally repeatable. This requires a commitment beyond that of the researchers involved, but also a willingness of the institutions who serve the participants of such a sweeping study to abide by the research. Without such research, institutions which utilize VLEs can only continue on sometimes arbitrary perceptions of how best to serve student wellness. Persisting in the status quo as such leaves students vulnerable to practices that might institutionally under-serve them and have potential generational implications. Interestingly, one might argue that, without such research, institutions who offer VLEs might garner the ability to omit themselves of the direct responsibility to provide those qualities of VLEs that would be found to support mental health and exclude those qualities that are found to diminish mental health.

As a closing thought and to return to the experiential modus and inquiry of this review, we adjure future research to be guided by the question of how the student encounters their teacher within the VLE. Emmanuel Levinas, a philosopher who wrote extensively on the innate ethical experience that is garnered through face-to-face interaction, took a rare moment in his writing to offer insight on the dynamics of education. In Levinas’ Totality and Infinity, he notes that in being called to respond to the Other, “[teaching] designates an interior being that is capable of a relation with the exterior and does not take its own interiority for the totality of being” (Levinas, 1969, as cited in Zhao, 2016 , p. 324). Here, Levinas may appear to point to the disposition of the educator as one that offers the presence of self for the sake of the students’ being. This sentiment, taken along with the intriguing meta-analysis offered by Mesman et al. ( 2009 ) may do little to establish the “how” of education as conveyed through this inquiry, but certainly makes a tremendous stride in the realm of the “why” that institutions ought to work to expound among the current VLE modalities that they are imposing upon learning communities.

Acknowledgements

The authors would like to thank Dr. Christopher Gilham for his comments and guidance on a previous version of this manuscript.

Data availability

Code availability, declarations.

Not applicable.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Al Karani A, Thobaity A. Medical staff members’ experiences with Blackboard at Taif University, Saudi Arabia. Journal of Multidisciplinary Healthcare. 2020; 2020 (13):1629–1634. doi: 10.2147/JMDH.S287389. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research and Policy Systems. 2008; 6 (7):1–12. doi: 10.1186/1478-4505-6-7. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Archambault, L., Kennedy, K., & Bender, S. (2013). Cyber-truancy: Addressing issues of attendance in the digital age. Journal of Research on Technology in Education , 46 (1), 1–28.  http://libproxy.stfx.ca/login?url=https://www-proquest-com.libproxy.stfx.ca/scholarly-journals/cyber-truancy-addressing-issues-attendance/docview/1492734861/se-2?accountid=13803 . Accessed 9 Feb 2021.
• Ault MJ, Courtrade G, Miracle SA, Bruce AE. Providing support for rural special educators during nontraditional instruction: One state’s response. Rural Special Education Quarterly. 2020; 39 (4):193–200. doi: 10.1177/8756870520959653. [ CrossRef ] [ Google Scholar ]
• Baker R, Evans B, Li Q, Cung B. Does inducing students to schedule lecture watching in online classes improve their academic performance? An experimental analysis of a time management intervention. Research in Higher Education. 2019; 60 (4):521–552. doi: 10.1007/s11162-018-9521-3. [ CrossRef ] [ Google Scholar ]
• Barbour, M.K., & Labonte, R. (2019). Sense of irony or perfect timing: Examining the research supporting proposed e-learning changes in Ontario. International Journal of E-Learning & Distance Education , 34 (2), 1–30.  http://libproxy.stfx.ca/login?url=https://www-proquest-com.libproxy.stfx.ca/scholarly-journals/sense-irony-perfect-timing-examining-research/docview/2350114007/se-2?accountid=13803 . Accessed 9 Feb 2021.
• Blayone TJB, Mykhailenko O, Kavtaradze M, Kokhan M, van Oostveen R, Barber W. Profiling the digital readiness of higher education students for transformative online learning in the post-soviet nations of Georgia and Ukraine. International Journal of Educational Technology in Higher Education. 2018; 15 :1–22. doi: 10.1186/s41239-018-0119-9. [ CrossRef ] [ Google Scholar ]
• Boak, A., Hamilton, H., Adlaf, E., Henderson, J., & Mann, R. (2016). The mental health and well-being of Ontario students, 1991–2015: Detailed OSDUHS findings . CAMH Research Document Series no. 43. Toronto: Centre for Addiction and Mental Health.
• Cairns, M.R., Ebinger, M., Stinson, C., & Jordan, J. (2020). COVID-19 and human connection: Collaborative research on loneliness and online worlds from a socially-distanced academy. Human Organization , 79 (4), 281–291.  https://www-proquest-com.libproxy.stfx.ca/docview/2472333748/fulltextPDF/84C6807DFE224A65PQ/1?accountid=13803 . Accessed 9 Feb 2021.
• Cavanaugh, C., Gillian, K.J., Kromrey, J., Hess, M., & Blomeyer, R. (2004) The effects of distance education on K-12 student outcomes: A meta-analysis . Learning Point Associates.  https://eric.ed.gov/?id=ED489533 . Accessed 9 Feb 2021.
• Chugani C, Houtrow A. Effect of the COVID-19 pandemic on college students with disabilities. American Journal of Public Health. 2020; 110 (12):1722–1723. doi: 10.2105/AJPH.2020.305983. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Crea TM, Sparnon N. Democratizing education at the margins: Faculty and practitioner perspectives on delivering online tertiary education for refugees. International Journal of Educational Technology in Higher Education. 2017; 14 :1–19. doi: 10.1186/s41239-017-0081-y. [ CrossRef ] [ Google Scholar ]
• Davies M. Using the Apple iPad to facilitate student-led group work and seminar presentation. Nurse Education in Practice. 2014; 14 (4):363–367. doi: 10.1016/j.nepr.2014.01.006. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Domina, T., Renzulli, L., Murray, B., Garza, A.N. & Perez, L. (2021). Remote or removed: Predicting successful engagement with online learning during COVID-19. Socius: Sociological Research for a Dynamic World , 7 , 1–15. 10.1177/2378023120988200
• Dommett EJ, Gardner B, van Tilburg W. Staff and student views of lecture capture: A qualitative study. International Journal of Educational Technology in Higher Education. 2019; 16 (23):1–12. doi: 10.1186/s41239-019-0153-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Driscoll A, Jicha K, Hunt AN, Tichavsky L, Thompson G. Can online courses deliver in-class results?: A comparison of student performance and satisfaction in an online versus a face-to-face introductory sociology course. Teaching Sociology. 2012; 40 (4):312–331. doi: 10.1177/0092055X12446624. [ CrossRef ] [ Google Scholar ]
• Du J, Fan X, Xu J, Wang C, Sun L, Liu F. Predictors for students’ self-efficacy in online collaborative groupwork. Educational Technology Research and Development. 2019; 67 :767–791. doi: 10.1007/s11423-018-9631-9. [ CrossRef ] [ Google Scholar ]
• Engelbertink MMJ, Kelders SM, Woudt-Mittendorff KM, Westerhof GJ. Participatory design of persuasive technology in a blended learning course: A qualitative study. Education and Information Technologies. 2020; 25 (5):4115–4138. doi: 10.1007/s10639-020-10147-x. [ CrossRef ] [ Google Scholar ]
• Gardner, M. (2020). Lecture captures adds value to attending psychological research methods lectures. Scholarship of Teaching and Learning in Psychology . Advance online publication. http://dx.doi.org/10.1037/stl0000212
• Gibson PF, Smith S. Digital literacies: Preparing pupils and students for their information journey in the twenty-first century. Journal of Information and Learning Science. 2018; 119 (12):733–742. doi: 10.1108/ILS-07-2018-0059. [ CrossRef ] [ Google Scholar ]
• Gillis A, Krull LM. COVID-19 remote learning transition in spring 2020: Class structures, student perceptions, and inequality in college courses. Teaching Sociology. 2020; 48 (4):283–299. doi: 10.1177/0092055X20954263. [ CrossRef ] [ Google Scholar ]
• Government of Canada. (2006). The human face of mental health and mental illness in Canada .  https://www.phac-aspc.gc.ca/publicat/human-humain06/pdf/human_face_e.pdf . Accessed 9 Feb 2021. 
• Ho V-T, Nakamori Y, Ho T-B, Ping Lim C. Blended learning model on hands-on approach for in-service secondary school teachers: Combination of e-learning and face-to-face discussion. Education and Information Technologies. 2016; 21 (1):185–208. doi: 10.1007/s10639-014-9315-y. [ CrossRef ] [ Google Scholar ]
• The Hospital for Sick Children. (2021, January 21). COVID-19: Guidance for school operation during the pandemic .  https://www.sickkids.ca/siteassets/news/news-archive/2021/covid19-guidance-for-school-operation-sickkids.pdf . Accessed 9 Feb 2021.
• Huang, T.-C., Huang, Y.-M., & Yu, F.-Y. (2011). Cooperative weblog learning in higher education: Its facilitating effects on social interaction, time lag, and cognitive load. Journal of Educational Technology & Society , 14 (1), 95–106.  http://libproxy.stfx.ca/login?url=https://www-proquest-com.libproxy.stfx.ca/scholarly-journals/cooperative-weblog-learning-higher-education/docview/1287031679/se-2?accountid=13803 . Accessed 9 Feb 2021.
• Huang Y, Backman SJ, Backman KF, McGuire FA, DeWayne M. An investigation of motivation and experience in virtual learning environments: A self-determination theory. Education and Information Technologies. 2019; 24 (1):591–611. doi: 10.1007/s10639-018-9784-5. [ CrossRef ] [ Google Scholar ]
• Hursen C. The effect of technology supported problem-based learning approach on adults’ self-efficacy perception for research-inquiry. Education and Information Technologies. 2019; 24 (2):1360–2357. doi: 10.1007/s10639-018-9822-3. [ CrossRef ] [ Google Scholar ]
• Jena RK. Investigating the interrelation between attitudes, learning readiness, and learning styles under virtual learning environment: A study among Indian students. Behavior and Information Technology. 2016; 35 (11):946–957. doi: 10.1080/0144929X.2016.1212930. [ CrossRef ] [ Google Scholar ]
• Johnston, S.C., Greer, D., & Smith, S.J. (2014). Peer learning in virtual schools. Journal of Distance Education , 28 (1), 1–31.  http://www.ijede.ca/index.php/jde/article/view/853 . Accessed 9 Feb 2021.
• Jones S. Benefits and challenges of online education for clinical social work: Three examples. Clinical Social Work Journal. 2015; 43 (2):225–235. doi: 10.1007/s10615-014-0508-z. [ CrossRef ] [ Google Scholar ]
• Kent, M., Ellis, K., & Giles, M. (2018). Students with disabilities and eLearning in Australia: Experiences of accessibility and disclosure at Curtin University. TechTrends , 62 (6), 654–663. 10–1007/s11528–018–0337-y
• Kumar KL, Owston R. Evaluating e-learning accessibility by automated and student-centered methods. Educational Technology, Research and Development. 2016; 64 (2):263–283. doi: 10.1007/s11423-015-9413-6. [ CrossRef ] [ Google Scholar ]
• Lan, Y.-J., Hsiao, I.Y.T., & Shih, M.-F. (2018). Effective learning design of game-based 3D virtual language learning environments for special education students. Journal of Educational Technology & Society , 21 (3), 213–227.  https://www-proquestcom.libproxy.stfx.ca/docview/2147865547?accountid=13803 . Accessed 9 Feb 2021.
• Lee SJ, Ngampornchai A, Trail-Constant T, Abril A, Srinivasan S. Does a case-based online group project increase students’ satisfaction with interaction in online courses? Active Learning in Higher Education. 2016; 17 (3):249–260. doi: 10.1177/1469787416654800. [ CrossRef ] [ Google Scholar ]
• Lee, S.M., & Oh., Y. (2017). The mediator role of perceived stress in the relationship between academic stress and depressive symptoms among e-learning students with visual impairments. Journal of Visual Impairment & Blindness , 111 (2), 123–127. https://files.eric.ed.gov/fulltext/EJ1137410.pdf [accessed 9 February 2021].
• Mallya, J., Lakshminarayanan, S., & Payini, V. (2019). Self-efficacy as an antecedent to students’ behavioral intention to use the internet for academic purposes: A structural equation modelling approach. Library Philosophy and Practice , 1–12.  http://libproxy.stfx.ca/login?url=https://www-proquest-com.libproxy.stfx.ca/scholarly-journals/self-efficacy-as-antecedent-students-behavioral/docview/2299764012/se-2?accountid=13803 . Accessed 9 Feb 2021.
• Manthey L, Vehreschild V, Renner K-H. Effectiveness of two cognitive interventions promoting happiness with video-based online instructions. Journal of Happiness Studies. 2016; 17 (1):319–339. doi: 10.1007/s10902-014-9596-2. [ CrossRef ] [ Google Scholar ]
• Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. U.S. Department of Education. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies .  https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf . Accessed 9 Feb 2021.
• Merlin-Knoblich, C., Harris, P.N., & Mason, E.C.M. (2019). Examining student classroom engagement in flipped and non-flipped counsellor education courses. The Professional Counselor , 9 (2), 109–125. 10.15241/cmk.9.2.109
• Mesman J, van Ijzendoorn M, Bakermans-Kranenburg M. The many faces of the still-face paradigm: A review and meta-analysis. Developmental Review. 2009; 29 (2):120–162. doi: 10.1016/j.dr.2009.02.001. [ CrossRef ] [ Google Scholar ]
• Miller, J. (2020, April). Learning from home won’t drag down grades for high school students, promises Ottawa board. Ottawa Citizen ,  https://ottawacitizen.com/news/local-news/learning-from-home-wont-drag-down-grades-for-high-school-students-promises-ottawa-board. Accessed 9 Feb 2021.
• Ontario College of Teachers. (2020). Ethical Standards . https://www.oct.ca/public/professional-standards/ethical-standards
• Ontario Education Act (1990) O Reg. 484/20, s. 4(14.1). Retrieved from Ontario e-Laws website: https://www.ontario.ca/laws/regulation/120132
• Ontario Education Act, R.S.O. 1990, c. E.2. Retrieved from Ontario e-Laws website: https://www.ontario.ca/laws/statute/90e02#BK441
• OECTA. (May 11, 2020). Statement from OECTA president Liz Stuart on “Synchronous Learning” .  https://www.catholicteachers.ca/News-Events/News/Releases/Statement-from-OECTA-President-Liz-Stuart-on-the-M . Accessed 9 Feb 2021.
• Ontario Ministry of Education. (2020). Policy/Program Memorandum No. 164.  http://www.edu.gov.on.ca/extra/eng/ppm/164.html . Accessed 9 Feb 2021.
• Papanastasiou G, Drigas A, Skianis C, Lytras M, Papanastasiou E. Virtual and augmented reality effects on K-12, higher and tertiary education students’ twenty-first century skills. Virtual Reality. 2019; 23 (4):425–436. doi: 10.1007/s10055-018-0363-2. [ CrossRef ] [ Google Scholar ]
• Pryjmachuk S, Gill A, Wood P, Olleveant N, Keeley P. Evaluation of an online study skills course. Active Learning in Higher Education. 2012; 13 (2):155–168. doi: 10.1177/1469787412441298. [ CrossRef ] [ Google Scholar ]
• Roblek V, Mesko M, Dimovski V, Peterlin J. Smart technologies as social innovation and complex social issues of the Z generation. Kybernetes. 2019; 48 (1):91–107. doi: 10.1108/K-09-2017-0356. [ CrossRef ] [ Google Scholar ]
• Rogowska AM, Kuśnierz C, Bokszczanin A. Examining anxiety, life satisfaction, general health, stress and coping styles during COVID-19 pandemic in Polish sample of university students. Psychology Research and Behavior Management. 2020; 13 :797–811. doi: 10.2147/PRBM.S266511. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rowe DA, Carter E, Gajjar S, Maves EA, Wall JC. Supporting strong transitions remotely: Considerations and complexities for rural communities during COVID-19. Rural Special Education Quarterly. 2020; 39 (4):220–232. doi: 10.1177/8756870520958199. [ CrossRef ] [ Google Scholar ]
• Schlesselman L. Perspective from a teaching and learning center during emergency remote teaching. American Journal of Pharmaceutical Education. 2020; 84 (8):1042–1044. doi: 10.5688/ajpe8142. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shawaqfeh M, Al Bekairy A, Al-Azayzih A, Alkatheri A, Qandil A, Obaidat A, Al Harbi S, Muflih S. Pharmacy students perceptions of their distance online learning experience during the COVID-19 pandemic: A cross-sectional survey study. Journal of Medical Education and Curricular Development. 2020; 7 :1–9. doi: 10.1177/2382120520963039. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shea KL, Rovera EJ. Preparing for the COVID-19 pandemic and its impact on a nursing simulation curriculum. Journal of Nursing Education. 2021; 60 (1):52–55. doi: 10.3928/01484834-20201217-12. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stone C. Online learning in Australian higher education: Opportunities, challenges and transformations. Student Success. 2019; 10 (2):1–11. doi: 10.5204/ssj.v10i2.1299. [ CrossRef ] [ Google Scholar ]
• Toulouse C. Screen capture recordings enhance connectedness among students, course content, and faculty. Journal of Nursing Education. 2020; 59 (9):531–535. doi: 10.3928/01484834-20200817-11. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tronick E, Als H, Adamson L, Wise S, Brazelton E. The infant's response to entrapment between contradictory messages in face-to-face interaction. Journal of the American Academy of Child Psychiatry. 1978; 17 (1):1–13. doi: 10.1016/S0002-7138(09)62273-1. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Watermeyer R, Crick T, Knight C, Goodall J. COVID-19 and digital disruption in UK universities: Afflictions and affordances of emergency online migration. Higher Education. 2021; 81 (3):623–641. doi: 10.1007/s10734-020-00561-y. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wingo NP, Peters GB, Ivankova NV, Gurley DK. Benefits and challenges of teaching nursing online: Exploring perspectives of different stakeholders. Journal of Nursing Education. 2016; 55 (8):433–440. doi: 10.3928/01484834-20160715-03. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Xavier, B., Camarneiro, A.P., Loureiro, L., Menino, E., Cunha-Oliveira, A., & Monteiro, A.P. (2020). Impact of COVID-19 on the family, social, and academic dynamics of nursing students in Portugal. Revista de Enfemagem Referência , 5 (4), 1–10. 10.12707/RV20104
• Yilmaz FKG. Exploring the role of Facebook adoption and virtual environment loneliness on knowledge sharing behaviors in a Facebook learning community. Education and Information Technologies. 2019; 24 (5):1699–1714. doi: 10.1007/s10639-018-09854-3. [ CrossRef ] [ Google Scholar ]
• Zeren, S.G. (2015). Face-to-face and online counselling: Client problems and satisfaction. Education and Science , 40 (182), 127–141. 10.15390/EB.2015.4696
• Zhao G. Levinas and the philosophy of education. Educational Philosophy and Theory. 2016; 48 (4):323–330. doi: 10.1080/00131857.2015.1041007. [ CrossRef ] [ Google Scholar ]
• Zhu C, van Winkel L. A virtual learning environment for the continuation of education and its relationship with the mental well-being of chronically ill adolescents. Educational Psychology. 2016; 36 (8):1429–1442. doi: 10.1080/01443410.2014.992393. [ CrossRef ] [ Google Scholar ]

Virtual, mixed, and augmented reality: a systematic review for immersive systems research

• Original Article
• Published: 03 January 2021
• Volume 25 , pages 773–799, ( 2021 )

Cite this article

• Matthew J. Liberatore   ORCID: orcid.org/0000-0002-5741-6723 1 &
• William P. Wagner 2  

7999 Accesses

67 Citations

1 Altmetric

Explore all metrics

Immersive systems can be used to capture new data, create new experiences, and provide new insights by generating virtual elements of physical and imagined worlds. Immersive systems are seeing increased application across a broad array of fields. However, in many situations it is unknown if an immersive application performs as well or better than the existing application in accomplishing a specific task. The purpose of this study is to conduct a systematic review of the literature that addresses the performance of immersive systems. This review assesses those applications where experiments, tests, or clinical trials have been performed to evaluate the proposed application. This research addresses a broad range of application areas and considers studies that compared one or more immersive systems with a control group or evaluated performance data for the immersive system pre- and post-test. The results identify those applications that have been successfully tested and also delineate areas of future research where more data may be needed to assess the effectiveness of proposed applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Immersive media experience: a survey of existing methods and tools for human influential factors assessment

XR Accessibility – Learning from the Past and Addressing Real User Needs for Inclusive Immersive Environments

Inclusivity Requirements for Immersive Content Consumption in Virtual and Augmented Reality

Explore related subjects.

• Artificial Intelligence

Availability of data and material

Results of systematic literature search are available.

Code availability

Code used for literature search included as “ Appendix .”

Abdullah M, Shaikh ZA (2018) An effective virtual reality based remedy for acrophobia. Int J Adv Comput Sci Appl 9(6):162–167

Google Scholar  

Anstadt S, Bradley S, Burnett A (2013) Virtual worlds: relationship between real life and experience in second life. Int Rev Res Open Distance Learn 14(4):160–190

Article   Google Scholar  

Azuma R (1997) A survey of augmented reality. Presence: Teleoper Virtual Environ 6(4):355–385

Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, Macintyre B (2001) Recent advances in augmented reality. IEEE Comput Graphics Appl 21:34–47

Blum J, Rockstroh C, Göritz AS (2019) Heart rate variability biofeedback based on slow-paced breathing with immersive virtual reality nature scenery. Front Psychol 10:2172

Boas Y (2013) Overview of virtual reality technologies. School of Electronics and Computer Science, University of Southampton, Southampton, UK. http://static1.squarespace.com/static/537bd8c9e4b0c89881877356/t/5383bc16e4b0bc0d91a758a6/1401142294892/yavb1g12_25879847_finalpaper.pdf . Accessed 1 Aug 2020

Boell S, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews in IS. J Inf Technol 30(2):161–173

Bogicevica V, Seob S, Kandampullyc J, Liuc S, Rudd N (2019) Virtual reality presence as a preamble of tourism experience: the role of mental imagery. Tour Manag 74:55–64

Bordnick PS, Traylor AC, Carter BL, Graap KM (2012) A feasibility study of virtual reality-based coping skills training for nicotine dependence. Res Soc Work Pract 22(3):293–300

Borsci S, Lawson G, Jha B, Burges M, Salanitri D (2016) Effectiveness of a multidevice 3D virtual environment application to train car service maintenance procedures. Virtual Real 20(1):41–55

Bouzit M, Popescu G, Burdea G, Boian R (2002) The Rutgers master II-ND force feedback glove. In: Proceedings of IEEE VR 2002 haptics symposium, Orlando FL, March, pp 256–263

Bowman D, McMahan R (2007) Virtual reality: how much immersion is enough? Computer 40:36–43

Bramley I, Goode A, Anderson L, Mary E (2018) Researching in-store, at home: using virtual reality within quantitative surveys. Int J Mark Res 60(4):344–351

Cardoş RA, David OA, David DO (2017) Virtual reality exposure therapy in flight anxiety: a quantitative meta-analysis. Comput Hum Behav 72:371–380

Carlson P, Peters A, Gilbert SB, Vance JM, Luse A (2015) Virtual training: learning transfer of assembly tasks. IEEE Trans Visual Comput Gr 21(6):770–782

Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M (2011) Augmented reality technologies, systems and applications. Multimedia Tools Appl 51(1):341–377

Carù A, Cova B (2006) Expériences de marque: comment favoriser l’immersion du consommateur? Décis Mark 41:43–52

Choi S, Cheung H (2008) A versatile virtual prototyping system for rapid product development. Comput Ind 59(5):477–488

Corbetta D, Imeri F, Gatti R (2015) Rehabilitation that incorporates virtual reality is more effective than standard rehabilitation for improving walking speed, balance and mobility after stroke: a systematic review. J Physiother 61(3):117–124

Cruz-Neira C, Sandlikn D, DeFanti T, Kenyon R, Hart J (1992) The CAVE: audio visual experience automatic virtual environment. Commun ACM 35(6):65–72

Cummings J, Bailenson J (2016) How immersive is Enough? A meta-analysis of the effects of immersive technology on user preference. Media Psychol 19:272–309

CyberGlove Systems (2020) CyberGrasp. http://www.cyberglovesystems.com/cybergrasp . Accessed 11 Nov 2020

Czub M, Piskorz J (2018) Body movement reduces pain intensity in virtual reality–based analgesia. Int J Hum-Comput Interact 34(11):1045–1051

de Rooij IJ, van de Port IG, Meijer JWG (2016) Effect of virtual reality training on balance and gait ability in patients with stroke: systematic review and meta-analysis. Phys Ther 96(12):1905–1918

Dobrowolski P, Pochwatko G, Skorko M, Bielecki M (2014) The effects of virtual experience on attitudes toward real brands. Cyberpsychol Behav Soc Netw 17(2):125–128

Dunn J (2017) A visual history of Nintendo’s video game consoles. http://www.businessinsider.com/nintendo-consoles-in-history-photos-switch-2017-1/#before-there-was-the-nes-there-was-the-color-tv-game-nintendo-first-dipped-its-toes-into-console-gaming-by-launching-five-of-these-rectangles-between-1977-and-1980-all-in-its-native-japan-1. Accessed 1 Aug 2020

Ferrer-Garcia M, Pla-Sanjuanelo J, Dakanalis A, Vilalta-Abella F, Riva G, Fernandez-Aranda F, Ribas-Sabaté J (2019) A randomized trial of virtual reality-based cue exposure second-level therapy and cognitive behavior second-level therapy for bulimia nervosa and binge-eating disorder: outcome at six-month follow up. Cyberpsychol Behav Soc Netw 22(1):60–68

Fink A (2005) Conducting research literature reviews: From the Internet to paper. Sage Publications, Thousand Oaks

Flavián C, Ibáñez-Sánchez S, Orús C (2019) Integrating virtual reality devices into the body: effects of technological embodiment on customer engagement and behavioral intentions toward the destination. J Travel Tour Mark 36(7):12. https://doi.org/10.1080/10548408.2019.1618781

Fodor L, Coteț C, Cuijpers P, Szamoskozi Ș, David D, Cristea I (2018) The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: a meta-analysis. Sci Rep 8(1):10323. https://doi.org/10.1038/s41598-018-28113-6

Gerçeker G, Binay Ş, Bilsin E, Kahraman A, Yılmaz H (2018) Effects of virtual reality and external cold and vibration on pain in 7-to 12-year-old children during phlebotomy: a randomized controlled trial. J PeriAnesthesia Nurs 33(6):981–989

Gleasure R, Feller J (2015) A rift in the ground: theorizing the evolution of anchor values in crowdfunding communities through the oculus rift case study. J Assoc Comput Syst 17(1):708–736

Glennon C, McElroy S, Connelly L, Lawson L, Bretches A, Gard A, Newcomer L (2018) Use of virtual reality to distract from pain and anxiety. Oncol Nurs Forum 45(4):545–552. https://doi.org/10.1188/18.ONF.545-552

Glueck A, Han D (2020) Improvement potentials in balance and visuo-motor reaction time after mixed reality action game play: a pilot study. Virt Real 24(2):223–229. https://doi.org/10.1007/s10055-019-00392-y

Gonzalez-Franco M, Pizarro R, Cermeron J, Li K, Thorn J, Hutabarat W, Bermell-Garcia P (2017) Immersive mixed reality for manufacturing training. Frontiers in Robotics and AI, 4. http://journal.frontiersin.org/article/10.3389/frobt.2017.00003/full . Accessed 1 Aug 2020

Gordon NS, Merchant J, Zanbaka C, Hodges LF, Goolkasian P (2011) Interactive gaming reduces experimental pain with or without a head mounted display. Comput Hum Behav 27(6):2123–2128

Gumaa M, Rehan Youssef A (2019) Is virtual reality effective in orthopedic rehabilitation? A systematic review and meta-analysis. Phys Ther 99(10):1304–1325

Guo C, Deng H, Yang J (2015) Effect of virtual reality distraction on pain among patients with hand injury undergoing dressing change. J Clin Nurs 24(1–2):115–120

Huang TL (2019) Psychological mechanisms of brand love and information technology identity in virtual retail environments. J Retail Consum Serv 47:251–264

Igna R, Stefan S, Onac I, Ungur RA, Tatar AS (2014) Mindfulness-based cognitive-behavior therapy (MCBT) versus virtual reality (VR) enhanced CBT, versus treatment as usual for chronic back pain. A clinical trial. J Evid-Based Psychother 14(2):229

igroup.org (2016) igroup presence questionnaire (IPQ) overview. http://www.igroup.org/pq/ipq/index.php Accessed 1 Aug 2020

Israel K, Zerres C, Tscheulin DK (2019) Presenting hotels in virtual reality: does it influence the booking intention? J Hosp Tour Technol 10(3):473–493

Javornik A (2016) Augmented reality: research agenda for studying the impact of its media characteristics on consumer behavior. J Retail Consumer Serv 30:252–261

Jennett C, Cox A, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A (2008) Measuring and defining the experience of immersion in games. Int J Hum Comput Stud 66(9):641–661

Jo D, Kim GJ (2019) IoT + AR: pervasive and augmented environments for “digi-log” shopping experience. Hum-Centric Comput Inf Sci. https://doi.org/10.1186/s13673-018-0162-5

Juan MC, Calatrava J (2011) An augmented reality system for the treatment of phobia to small animals viewed via an optical see-through HMD: comparison with a similar system viewed via a video see-through HMD. Int J Hum–Comput Interact 27(5):436–449

Kalawsky RS (1996) AGOCG Report. Exploiting virtual reality techniques in education and training: technological issues. http://www.agocg.ac.uk/reports/virtual/vrtech/toc.htm Accessed 1 Aug 2020

Karafotias G, Korres G, Teranishi A, Park W, Eid M (2017) Mid-air tactile stimulation for pain distraction. IEEE Trans Haptics 11(2):185–191

Kawulich B, D’Alba A (2019) Teaching qualitative research methods with second life: a 3-dimensional online virtual environment. Virtual Real 23(4):375–384

Kim IC, Lee BH (2012) Effects of augmented reality with functional electric stimulation on muscle strength, balance and gait of stroke patients. J Phys Ther Sci 24(8):755–762

Krumins A (2017) Haptic bodysuits and the strange new landscape of immersive VR. Jan 4, blog entry at https://www.extremetech.com/extreme/241917-haptic-bodysuits-strange-new-landscape-immersive-virtual-reality . Accessed 1 Aug 2020

Ku J, Kim YJ, Cho S, Lim T, Lee HS, Kang YJ (2019) Three-dimensional augmented reality system for balance and mobility rehabilitation in the elderly: a randomized controlled trial. Cyberpsychol Behav Soc Netw 22(2):132–141

Kumar R, Oskiper T, Naroditsky O, Samarasekera S, Zhu Z, Kim J (2017) System and method for generating a mixed reality environment. US Patent No. 9,600,067 B2

Laha B, Sensharma K, Schiffbauer JD, Bowman DA (2012) Effects of immersion on visual analysis of volume data. IEEE Trans Visual Comput Gr 19(4):597–606

Latif U, Shin S (2019) OP-MR: the implementation of order picking based on mixed reality in a smart warehouse. Vis Comput. https://doi.org/10.1007/s00371-019-01745-z

Lau K (2015) Organizational learning goes virtual? A study of employees’ learning achievement in stereoscopic 3D virtual reality. Learn Organ 22(5):289–303

Lee C, Kim Y, Lee B (2014) Augmented reality-based postural control training improves gait function in patients with stroke: randomized controlled trial. Hong Kong Physiother J 32(2):51–57

Lee J, Yoo H, Lee B (2017) Effects of augmented reality-based Otago exercise on balance, gait, and physical factors in elderly women to prevent falls: a randomized controlled trial. J Phys Ther Sci 29(9):1586–1589

Lessiter J, Freeman J, Keogh E, Davidoff J (2001) A cross-media presence questionnaire: the ITC sense of presence inventory. Presence: Teleoper Virtual Environ 10(3):282–297

Li C, Liang W, Quigley C, Zhao Y, Yu L (2017) Earthquake safety training through virtual drills. IEEE Trans Vis Comput Gr 23(4):1275–1284

Liberati N (2013) Improving the embodiment relations by means of phenomenological analysis on the “reality” of ARs. In: 2013 IEEE international symposium on mixed and augmented reality-arts, media, and humanities (ISMAR-AMH) 0, 13–17, 2013. http://doi.ieeecomputersociety.org/10.1109/ISMAR-AMH.2012.6483983

Liberati N (2016) Augmented reality and ubiquitous computing: the hidden potentialities of augmented reality. AI Soc 31(1):17–28

Lima J, McCabe-Bennett H, Antony M (2018) Treatment of storm fears using virtual reality and progressive muscle relaxation. Behav Cognit Psychother 46(2):251–256

Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. J Comput-Med Commun 3(2):1083–6101

Lombard M, Ditton T, Weinstein L (2013) Measuring presence: the temple presence inventory (TPI). Updated September, 15 . http://matthewlombard.com/research/p2_ab.html . Accessed 1 Aug 2020

Loreto-Quijada D, Gutiérrez-Maldonado J, Nieto R, Gutiérrez-Martínez O, Ferrer-García M, Saldana C, Liutsko L (2014) Differential effects of two virtual reality interventions: distraction versus pain control. Cyberpsychol Behav Soc Netw 17(6):353–358

Manzoni GM, Cesa GL, Bacchetta M, Castelnuovo G, Conti S, Gaggioli A, Riva G (2016) Virtual reality–enhanced cognitive–behavioral therapy for morbid obesity: a randomized controlled study with 1 year follow-up. Cyberpsychol Behav Soc Netw 19(2):134–140

Martínez-Navarro J, Bigné E, Guixeres J, Alcañiz M, Torrecilla C (2019) The influence of virtual reality in e-commerce. J Bus Res 100:475–482

Maskey M, Rodgers J, Grahame V, Glod M, Honey E, Kinnear J, Parr J (2019) A randomised controlled feasibility trial of immersive virtual reality treatment with cognitive behaviour therapy for specific phobias in young people with autism spectrum disorder. J Autism Dev Disord 49(5):1912–1927

McLay R, Wood D, Webb-Murphy J, Spira J, Wiederhold M, Pyne J, Wiederhold B (2011) A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychol Behav Soc Netw 14(4):223–229

McLay R, Baird A, Webb-Murphy J, Deal W, Tran L, Anson H, Klam W, Johnston S (2017) A randomized, head-to-head study of virtual reality exposure therapy for posttraumatic stress disorder. Cyberpsychol Behav Soc Netw 20(4):218–224

McMahan A (2003) Immersion, engagement and presence: a method for analyzing 3-D video games. In: Wolf M, Perron B (eds) The video game theory reader, chap 3. Routledge, New York, pp 67–86

Meng F, Zhang W, Yang R (2014) The development of a panorama manifestation virtual reality system for navigation and a usability comparison with a desktop system. Behav Inf Technol 33(2):133–143

Merel T (2017) The reality of VR/AR growth. Tech Crunch. https://techcrunch.com/2017/01/11/the-reality-of-vrar-growth/ . Accessed 1 Aug 2020

Michaliszyn D, Marchand A, Bouchard S, Martel M, Poirier-Bisson J (2010) A randomized, controlled clinical trial of in virtuo and in vivo exposure for spider phobia. Cyberpsychol Behav Soc Netw 13(6):689–695

Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):1321–1329

Montero-López E, Santos-Ruiz A, García-Ríos M, Rodríguez-Blázquez R, Pérez-García M, Peralta-Ramírez M (2016) A virtual reality approach to the Trier Social Stress Test: contrasting two distinct protocols. Behav Res Methods 48(1):223–232

Motraghi T, Seim R, Meyer E, Morissette S (2014) Virtual reality exposure therapy for the treatment of posttraumatic stress disorder: a methodological review using CONSORT guidelines. J Clin Psychol 70(3):197–208

Muhanna M (2015) Virtual reality and the CAVE: taxonomy, interaction challenges and research directions. J King Saud Univ—Comput Inf Sci 27(3):344–361

Murcia-Lopez M, Steed A (2018) A comparison of virtual and physical training transfer of bimanual assembly tasks. IEEE Trans Vis Comput Gr 24(4):1574–1583

Narayan M, Waugh L, Zhang X, Bafna P, Bowman D (2005) Quantifying the benefits of immersion for collaboration in virtual environments. In: Proceedings of the ACM symposium on virtual reality software and technology, Monterey, California, USA, 7–9 November

Neguţ A, Matu S, Sava F, David D (2016) Task difficulty of virtual reality-based assessment tools compared to classical paper-and-pencil or computerized measures: a meta-analytic approach. Comput Hum Behav 54:414–424

Ng Y-L, Ma F, Ho F, Ip P, Fu K-W (2019) Effectiveness of virtual and augmented reality-enhanced exercise on physical activity, psychological outcomes, and physical performance: a systematic review and meta-analysis of randomized controlled trials. Comput Hum Behav 99:278–291

Nilsson S, Johansson B, Jonsson A (2010) Cross-organizational collaboration supported by augmented reality. IEEE Trans Visual Comput Graphics 17(10):1380–1392

Nilsson N, Nordahl R, Serafin S (2016) Immersion revisited: a review of existing definitions of immersion and their relation to different theories of presence. Hum Technol 12(2):108–134

Okoli C, Schabram K (2010) A guide to conducting a systematic literature review of information systems research. Sprouts: working papers on information systems, vol 10, no. 26. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1954824 . Accessed 1 Aug 2020

Oleksy T, Wnuk A (2016) Augmented places: an impact of embodied historical experience on attitudes towards places. Comput Hum Behav 57:11–16

Paré G, Jaana M, Sicotte C (2007) Systematic review of home telemonitoring for chronic diseases: the evidence base. J Am Med Inform Assoc 14(3):269–277. https://doi.org/10.1197/jamia.M2270

Paré G, Trudel M-C, Jaana M, Kitsiou S (2015) Synthesizing information systems knowledge: a typology of literature reviews. Inf Manag 52(2):183–199

Paré G, Tate M, Johnstone D, Kitsiou S (2016) Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews. Eur J Inf Syst 25(6):493–508

Parsons T, Gaggioli A, Riva G (2017) Virtual reality research in social neuroscience. Brain Sci 7(4):42

Parveau M, Adda M (2018) 3iVClass: a new classification method for virtual, augmented and mixed realities. Procedia Comput Sci 141:263–270. https://doi.org/10.1016/j.procs.2018.10.180

Perret J, Vander Poorten E (2018) Touching virtual reality: a review of haptic gloves. In: ACTUATOR 2018: 16th international conference on new actuators, June 25–27, pp 270–274

Pickering C, Byrne J (2013) The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early-career researchers. High Educ Res Dev 33(3):534–548. https://doi.org/10.1080/07294360.2013.841651

Pickering C, Grignon J, Steven R, Guitart D, Byrne J (2015) Publishing not perishing: how research students transition from novice to knowledgeable using systematic quantitative literature reviews. Stud High Educ 40(10):1756–1769. https://doi.org/10.1080/03075079.2014.914907

Piskorz J, Czub M (2014) Distraction of attention with the use of virtual reality. Influence of the level of game complexity on the level of experienced pain. Pol Psychol Bull 45(4):480–487

Regenbrecht H, Schubert T (2002) Real and illusory interactions enhance presence in virtual environments. Presence: Teleoper Virtual Environ 11(4):425–434

Reger G, Koenen-Woods P, Zetocha K, Smolenski D, Holloway K, Rothbaum B, Gahm G (2016) Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consul Clin Psychol 84(11):946–959

Rehman U, Cao S (2019) Comparative evaluation of augmented reality-based assistance for procedural tasks: a simulated control room study. Behav Inf Technol. https://doi.org/10.1080/0144929X.2019.1660805

Repetto C, Gaggioli A, Pallavicini F, Cipresso P, Raspelli S, Riva G (2013) Virtual reality and mobile phones in the treatment of generalized anxiety disorders: a phase-2 clinical trial. Pers Ubiquit Comput 17(2):253–260

Riva G, Waterworth JA (2003) Presence and the self: a cognitive neuroscience approach. Presence-Connect, 3(3)

Rodríguez C, Areces D, Garcia T, Cueli M, González Castro P (2018) Comparison between two continuous performance tests for identifying ADHD: traditional vs. virtual reality. Int J Clin Health Psychol 18:254–263

Ronchi E, Mayorga D, Lovreglio R, Wahlqvist J, Nilsson D (2019) Mobile-powered head-mounted displays versus cave automatic virtual environment experiments for evacuation research. Comput Anim Virtual Worlds 30(6):e1873. https://doi.org/10.1002/cav.1873

Rowe F (2014) What literature review is not: diversity, boundaries, and recommendations. Eur J Inf Syst 23(3):241–255

Sacks R, Perlman A, Barak R (2013) Construction safety training using immersive virtual reality. Constr Manag Econ 31(9):1005–1017. https://doi.org/10.1080/01446193.2013.828844

Sadowsky W, Stanney K (2002) Measuring and managing presence in virtual environments. In: Stanney K (ed) Handbook of virtual environments technology. Lawrence Erlbaum Associates, Mahway, pp 791–806

Schoonheim M, Heyden R, Wiecha JM, Henden T (2014) Use of a virtual world computer environment for international distance education: lessons from a pilot project using second life. BMC Med Educ. https://doi.org/10.1186/1472-6920-14-36

Schroeder R (1996) Possible worlds: the social dynamic of virtual reality technologies. Westview Press, Boulder

Schryen G (2015) Writing qualitative IS literature reviews—guidelines for synthesis, interpretation and guidance of research. Commun Assoc Inf Syst 37:286–325

MathSciNet   Google Scholar  

Schryen G, Benlian A, Rowe F, Shirley G, Larsen K, Petter S, Wagner G, Haag S, Yasasin E (2017) Literature reviews in IS research: what can be learnt from the past and other fields? Commun Assoc Inf Syst. https://doi.org/10.17705/1CAIS.04130

Schubert T, Friedmann F, Regenbrecht H (2001) The experience of presence: factor analytic insights. Teleoper Virtual Environ 10(3):266–281

Shu Y, Huang YZ, Chang SH, Chen MY (2019) Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments. Virtual Real 23(4):437–446

Slater M (2003) A note on presence terminology. Presence Connect 3(3):1–5

Slater M, Wilbur S (1997) A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence: Teleoper Virtual Environ 6(6):603–616

Slater M, Usoh M, Steed A (1994) Depth of presence in virtual environments. Presence: Teleoper Virtual Environ 3(2):130–144

Smink A, Frowijn S, van Reijmersdal E, van Noort G, Neijens P (2019) Try online before you buy: how does shopping with augmented reality affect brand responses and personal data disclosure. Electron Commer Res Appl 35:100854

Solomon B (2014) Facebook buys oculus, virtual reality gaming startup, for $2 billion. https://www.forbes.com/sites/briansolomon/2014/03/25/facebook-buys-oculus-virtual-reality-gaming-startup-for-2-billion/#d8d8b7024984 . Accessed 1 Aug 2020

Suh A, Prophet J (2018) The state of immersive technology research: a literature analysis. Comput Hum Behav 86:77–90

Suso-Ribera C, Fernández-Álvarez J, García-Palacios A, Hoffman HG, Bretón-López J, Banos RM, Botella C (2019) Virtual reality, augmented reality, and in vivo exposure therapy: a preliminary comparison of treatment efficacy in small animal phobia. Cyberpsychol Behav Soc Netw 22(1):31–38

Tang Y, Au K, Lau H, Ho G, Wu G (2020) Evaluating the effectiveness of learning design with mixed reality (MR) in higher education. Virtual Real. https://doi.org/10.1007/s10055-020-00427-9

Teel E, Gay M, Johnson B, Slobounov S (2016) Determining sensitivity/specificity of virtual reality-based neuropsychological tool for detecting residual abnormalities following sport-related concussion. Neuropsychology 30(4):474–483

Templier M, Paré G (2018) Transparency in literature reviews: an assessment of reporting practices across review types and genres in top IS journals. Eur J Inf Syst 27(5):503–550. https://doi.org/10.1080/0960085X.2017.1398880

Thompson C (2017) Stereographs were the original virtual reality. Smithsonian Magazine . https://www.smithsonianmag.com/innovation/sterographs-original-virtual-reality-180964771/ . Accessed 1 Aug 2020

Thompson T, Steffert T, Steed A, Gruzelier J (2011) A randomized controlled trial of the effects of hypnosis with 3-d virtual reality animation on tiredness, mood, and salivary cortisol. Int J Clin Exp Hypn 59(1):122–142

Turk V (2016) Face electrodes let you taste and chew in virtual reality. https://www.newscientist.com/article/2111371-face-electrodes-let-you-taste-and-chew-in-virtual-reality/ . Accessed 1 Aug 2020

UQO Cyberpsychology Lab. Presence Questionnaire. (2002). http://w3.uqo.ca/cyberpsy/wp-content/uploads/2019/04/QEP_vf.pdf . Accessed August 1, 2020

Valtchanov D, Barton KR, Ellard C (2010) Restorative effects of virtual nature settings. Cyberpsychol Behav Soc Netw 13(5):503–512

Van Baren J, IJsselsteijn W (2004) Measuring presence: a guide to current measurement approaches. http://www8.informatik.umu.se/~jwworth/PresenceMeasurement.pdf . Accessed 1 Aug 2020

Van Kerrebroeck H, Brengman M, Willems K (2017) When brands come to life: experimental research on the vividness effect of virtual reality in transformational marketing communications. Virtual Real 21(4):177–191

vom Brocke J, Simons A, Riemer K, Niehaves B, Plattfaut R, Cleven A (2015) Standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. Commun Assoc Inf Syst 37:205–224

Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26(2):xiii–xxiii

Wechsler TF, Mühlberger A, Kümpers F (2019) Inferiority or even superiority of virtual reality exposure therapy in phobias?—A systematic review and quantitative meta-analysis on randomized controlled trials specifically comparing the efficacy of virtual reality exposure to gold standard in vivo exposure in agoraphobia, specific phobia and social phobia. Front Psychol 10:1758. https://doi.org/10.3389/fpsyg.2019.01758

Westerfield G, Mitrovic A, Billinghurst M (2015) Intelligent augmented reality training for motherboard assembly. Int J Artif Intell Educ 25(1):157–172

Wiederhold M, Crisci M, Patel V, Nonaka M, Wiederhold B (2019) Physiological monitoring during augmented reality exercise confirms advantages to health and well-being. Cyberpsychol Behav Soc Netw 22(2):122–126

Wilkerson W, Avstreih D, Gruppen L, Beier K-P, Woolliscroft J (2008) Using immersive simulation for training first responders for mass casualty incidents. Acad Emerg Med 15(11):1152–1159. https://doi.org/10.1111/j.1553-2712.2008.00223.x

Wissmath B, Weibel D, Mast F (2010) Measuring presence with verbal versus pictorial scales: a comparison between online- and ex post- ratings. Virtual Real 14(1):43–53

Witmer B, Singer M (1998) Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoper Virtual Environ 7(3):225–240

Witmer B, Jerome C, Singer M (2005) The factor structure of the presence questionnaire. Presence 14(3):298–312

Yang S, Xiong G (2019) Try it on! Contingency effects of virtual fitting rooms. J Manag Inf Syst 36(3):789–822

Yang Z, Shi J, Jiang W, Sui Y, Wu Y, Ma S, Li H (2019) Influences of augmented reality assistance on performance and cognitive loads in different stages of assembly task. Front Psychol 10:1703. https://doi.org/10.3389/fpsyg.2019.01703

Yoo SC, Drumwright M (2018) Nonprofit fundraising with virtual reality. Nonprofit Manag Leadersh 29(1):11–27

Download references

Not applicable.

Author information

Authors and affiliations.

Department of Management and Operations, Villanova School of Business, Villanova University, Villanova, PA, 19085, USA

Matthew J. Liberatore

Department of Accounting and Information Systems, Villanova School of Business, Villanova University, Villanova, PA, 19085, USA

William P. Wagner

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed equally to this research.

Corresponding author

Correspondence to Matthew J. Liberatore .

Ethics declarations

Conflicts of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Keywords used in Scopus literature search

((TITLE (“immersive system” OR “virtual reality” OR “mixed reality” OR “augmented reality”) OR ABS (“immersive system” OR “virtual reality” OR “mixed reality” OR “augmented reality”))) AND ((TITLE (“experiment” OR “trial” OR “test”) OR ABS (“experiment” OR “trial” OR “test”))) AND (LIMIT-TO (SRCTYPE, “j”)) AND (EXCLUDE (SUBJAREA, “MEDI”) OR EXCLUDE (SUBJAREA, “MATH”) OR EXCLUDE (SUBJAREA, “PHYS”) OR EXCLUDE (SUBJAREA, “NEUR”) OR EXCLUDE (SUBJAREA, “MATE”) OR EXCLUDE (SUBJAREA, “BIOC”) OR EXCLUDE (SUBJAREA, “AGRI”) OR EXCLUDE (SUBJAREA, “EART”) OR EXCLUDE (SUBJAREA, “ENVI”) OR EXCLUDE (SUBJAREA, “CENG”) OR EXCLUDE (SUBJAREA, “IMMU”) OR EXCLUDE (SUBJAREA, “DENT”) OR EXCLUDE (SUBJAREA, “PHAR”) OR EXCLUDE (SUBJAREA, “VETE”) OR EXCLUDE (SUBJAREA, “Undefined”) OR EXCLUDE (SUBJAREA, “ENER”) OR EXCLUDE (SUBJAREA, “CHEM”) OR EXCLUDE (SUBJAREA, “ENGI”)) AND (EXCLUDE (PUBYEAR, 2009) OR EXCLUDE (PUBYEAR, 2008) OR EXCLUDE (PUBYEAR, 2007) OR EXCLUDE (PUBYEAR, 2006) OR EXCLUDE (PUBYEAR, 2005) OR EXCLUDE (PUBYEAR, 2004) OR EXCLUDE (PUBYEAR, 2003) OR EXCLUDE (PUBYEAR, 2002) OR EXCLUDE (PUBYEAR, 2001) OR EXCLUDE (PUBYEAR, 2000) OR EXCLUDE (PUBYEAR, 1999) OR EXCLUDE (PUBYEAR, 1998) OR EXCLUDE (PUBYEAR, 1997) OR EXCLUDE (PUBYEAR, 1996) OR EXCLUDE (PUBYEAR, 1995) OR EXCLUDE (PUBYEAR, 1994) OR EXCLUDE (PUBYEAR, 1993) OR EXCLUDE (PUBYEAR, 1992) OR EXCLUDE (PUBYEAR, 1991) OR EXCLUDE (PUBYEAR, 1984)) AND (EXCLUDE (EXACTSRCTITLE, “Computers And Education”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Advanced Oxidation Technologies”) OR EXCLUDE (EXACTSRCTITLE, “Computer Applications In Engineering Education”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Online Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Computing In Civil Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Turkish Online Journal Of Educational Technology”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of recent Technology And Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Advanced Engineering Informatics”) OR EXCLUDE (EXACTSRCTITLE, “Computers In Education Journal”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Computing And Information Science In Engineering”) OR EXCLUDE (EXACTSRCTITLE, “AES Journal Of The Audio Engineering Society”) OR EXCLUDE (EXACTSRCTITLE, “Advances In Mechanical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Transactions On Biomedical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Multimedia And Ubiquitous Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Telecommunication Electronic And Computer Engineering”) OR EXCLUDE (EXACTSRCTITLE, “British Journal Of Educational Technology”) OR EXCLUDE (EXACTSRCTITLE, “Educational Technology And Society”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Advanced Research In Dynamical And Control Systems”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Medical And Biological Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Professional Issues In Engineering Education And Practice”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Science Education And Technology”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Journal On Emerging And Selected Topics In Circuits And Systems”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Innovative Technology And Exploring Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Educational Computing Research”) OR EXCLUDE (EXACTSRCTITLE, “Annals Of Biomedical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Transactions On Circuits And Systems For Video Technology”) OR EXCLUDE (EXACTSRCTITLE, “Interactive Technology And Smart Education”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Applied Engineering Research”) OR EXCLUDE (EXACTSRCTITLE, “Asia Pacific Education Researcher”)) AND (LIMIT-TO (LANGUAGE, “English”)) AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”)).

Rights and permissions

Reprints and permissions

About this article

Liberatore, M.J., Wagner, W.P. Virtual, mixed, and augmented reality: a systematic review for immersive systems research. Virtual Reality 25 , 773–799 (2021). https://doi.org/10.1007/s10055-020-00492-0

Download citation

Received : 02 August 2020

Accepted : 27 November 2020

Published : 03 January 2021

Issue Date : September 2021

DOI : https://doi.org/10.1007/s10055-020-00492-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Immersive systems
• Virtual reality
• Augmented reality
• Mixed reality
• Empirical research
• Systematic review
• Find a journal
• Publish with us
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Echosee: an assistive mobile application for real-time 3d environment reconstruction and sonification supporting enhanced navigation for people with vision impairments  †.

Graphical Abstract

1. Introduction

• What is the frequency with which a participant encounters an obstacle when blindfolded and assisted by the application compared to when blindfolded and unassisted?
• How quickly can a participant traverse the obstacle course blindfolded with the assistance of EchoSee as compared to without it?
• How much “seeking” behavior (head turning off axis from the direction of travel) occurs in participants during obstacle course traversal with the assistance of the application as compared to when blindfolded or unassisted?

2. Materials and Methods

2.1. hardware and software, 2.2. participants, 2.3. experimental protocol, 2.4. experimental metrics, 3.1. application development and testing, 3.2. developmental testing of analytics, 3.3. analytic results of study protocol, 3.4. qualitative results of study protocol, 3.5. results summary.

• Real-time 3D Soundscape: Successfully implemented EchoSee, a system generating real-time 3D audio soundscapes on a mobile device by integrating real-time 3D scanning, AR technologies, and spatial audio.
• Analytical Framework: Developed methods for aligning and extracting various metrics from 3D trajectory data that are relevant to a user’s navigation performance.
• Obstacle Avoidance: Demonstrated a 39.8% reduction in obstacle contacts when using EchoSee (181 unassisted vs. 109 assisted).
• Environmental Awareness: Observed substantially increased seeking behavior (slope of 2.28 assisted vs. −1.11 unassisted), indicating improved environmental investigation.
• Navigation Behavior: Noted a 34.0% increase in average completion time (50s unassisted vs. 67s assisted). This additional time suggests more thorough explorations with the benefit of reduced collisions.
• SPI & ADREV Performance: EchoSee-assisted navigation showed improved trends in both the SPI and ADREV metrics.
• User Perception: Recorded high user ratings for EchoSee providing actionable navigation information (0.861/1.0) and improved navigation confidence (0.833/1.0).
• Learning Effects: Observed increased learning rate within assisted conditions for collision reduction, suggesting navigation benefits with EchoSee after only a few trials.

4. Discussion

4.1. development of echosee, 4.2. experimental development, 4.3. learning effects, 4.4. training with virtual echosee, 4.5. safety performance index (spi), 4.6. real-world considerations and limitations, 4.7. usability and accessibility, 4.8. future directions, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest, abbreviations.

• Varma, R.; Vajaranant, T.S.; Burkemper, B.; Wu, S.; Torres, M.; Hsu, C.; Choudhury, F.; McKean-Cowdin, R. Visual impairment and blindness in adults in the United States: Demographic and geographic variations from 2015 to 2050. JAMA Ophthalmol. 2016 , 134 , 802–809. [ Google Scholar ] [ CrossRef ]
• Bourne, R.R.; Flaxman, S.R.; Braithwaite, T.; Cicinelli, M.V.; Das, A.; Jonas, J.B.; Keeffe, J.; Kempen, J.H.; Leasher, J.; Limburg, H.; et al. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: A systematic review and meta-analysis. Lancet Glob. Health 2017 , 5 , e888–e897. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Blasch, B.B.; Stuckey, K.A. Accessibility and mobility of persons who are visually impaired: A historical analysis. J. Vis. Impair. Blind. 1995 , 89 , 417–422. [ Google Scholar ] [ CrossRef ]
• Wiberg, H.J. Be My Eyes. Available online: https://www.bemyeyes.com/ (accessed on 7 March 2023).
• Aira Tech Corp. Aira. Available online: https://www.aira.io/ (accessed on 7 March 2023).
• Fishman, G.A. When your eyes have a wet nose: The evolution of the use of guide dogs and establishing the seeing eye. Surv. Ophthalmol. 2003 , 48 , 452–458. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Office of the Federal Register, National Archives and Records Administration. 28 CFR § 35.135—Personal Devices and Services. Available online: https://www.govinfo.gov/app/details/CFR-2021-title28-vol1/CFR-2021-title28-vol1-sec35-135 (accessed on 7 March 2023).
• Haskins Laboratories, New York; The National Academy of Sciences Committee on Sensory Devices. Research on Guidance Devices for the Blind: A Progress Report of Work Done at the Haskins Laboratories, New York City, Between February 15, 1944 and December 1, 1946 ; Haskins Laboratories: New York City, NY, USA, 1946. [ Google Scholar ]
• Beurle, R.L. Electronic aids for blind people. Br. J. Psychol. Sect. 1951 , 42 , 164–171. [ Google Scholar ] [ CrossRef ]
• Kay, L. An ultrasonic sensing probe as a mobility aid for the blind. Ultrasonics 1964 , 2 , 53–59. [ Google Scholar ] [ CrossRef ]
• Kay, L. A sonar aid to enhance spatial perception of the blind: Engineering design and evaluation. Radio Electron. Eng. 1974 , 44 , 605–627. [ Google Scholar ] [ CrossRef ]
• Easton, R.D.; Jackson, R.M. Pilot test of the Trisensor, a new generation sonar sensory aid. J. Vis. Impair. Blind. 1983 , 77 , 446–449. [ Google Scholar ] [ CrossRef ]
• Kish, C.D. Evaluation of an Echo-Mobility Program for Young Blind People. Master’s Thesis, California State University, San Bernardino, CA, USA, 1995. [ Google Scholar ]
• Kolarik, A.J.; Cirstea, S.; Pardhan, S.; Moore, B.C. A summary of research investigating echolocation abilities of blind and sighted humans. Hear. Res. 2014 , 310 , 60–68. [ Google Scholar ] [ CrossRef ]
• Kolarik, A.J.; Moore, B.C.; Zahorik, P.; Cirstea, S.; Pardhan, S. Auditory distance perception in humans: A review of cues, development, neuronal bases, and effects of sensory loss. Atten. Percept. Psychophys. 2016 , 78 , 373–395. [ Google Scholar ] [ CrossRef ]
• Teng, S.; Whitney, D. The acuity of echolocation: Spatial resolution in the sighted compared to expert performance. J. Vis. Impair. Blind. 2011 , 105 , 20. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rowan, D.; Papadopoulos, T.; Edwards, D.; Holmes, H.; Hollingdale, A.; Evans, L.; Allen, R. Identification of the lateral position of a virtual object based on echoes by humans. Hear. Res. 2013 , 300 , 56–65. [ Google Scholar ] [ CrossRef ]
• Thaler, L. Echolocation may have real-life advantages for blind people: An analysis of survey data. Front. Physiol. 2013 , 4 , 98. [ Google Scholar ] [ CrossRef ]
• Tonelli, A.; Brayda, L.; Gori, M. Depth echolocation learnt by novice sighted people. PLoS ONE 2016 , 11 , e0156654. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Thaler, L.; Goodale, M.A. Echolocation in humans: An overview. Wiley Interdiscip. Rev. Cogn. Sci. 2016 , 7 , 382–393. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Thaler, L.; Castillo-Serrano, J. People’s ability to detect objects using click-based echolocation: A direct comparison between mouth-clicks and clicks made by a loudspeaker. PLoS ONE 2016 , 11 , e0154868. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• de Vos, R.; Hornikx, M. Human ability to judge relative size and lateral position of a sound reflecting board using click signals: Influence of source position and click properties. Acta Acust. United Acust. 2018 , 104 , 131–144. [ Google Scholar ] [ CrossRef ]
• Thaler, L.; De Vos, H.; Kish, D.; Antoniou, M.; Baker, C.; Hornikx, M. Human click-based echolocation of distance: Superfine acuity and dynamic clicking behaviour. J. Assoc. Res. Otolaryngol. 2019 , 20 , 499–510. [ Google Scholar ] [ CrossRef ]
• Tirado, C.; Gerdfeldter, B.; Nilsson, M.E. Individual differences in the ability to access spatial information in lag-clicks. J. Acoust. Soc. Am. 2021 , 149 , 2963–2975. [ Google Scholar ] [ CrossRef ]
• Thaler, L.; Norman, L.; De Vos, H.; Kish, D.; Antoniou, M.; Baker, C.; Hornikx, M.C. Human Echolocators Have Better Localization Off Axis. Psychol. Sci. 2022 , 33 , 1143–1153. [ Google Scholar ]
• González-Mora, J.L.; Rodriguez-Hernandez, A.; Burunat, E.; Martin, F.; Castellano, M.A. Seeing the world by hearing: Virtual Acoustic Space (VAS) a new space perception system for blind people. In Proceedings of the IEEE 2006 2nd International Conference on Information & Communication Technologies, Damascus, Syria, 24–28 April 2006; Volume 1, pp. 837–842. [ Google Scholar ]
• Waters, D.A.; Abulula, H.H. Using bat-modelled sonar as a navigational tool in virtual environments. Int. J. Hum.-Comput. Stud. 2007 , 65 , 873–886. [ Google Scholar ] [ CrossRef ]
• Torres-Gil, M.; Casanova-Gonzalez, O.; Gonzalez-Mora, J. Virtual reality simulator for sonification studies. In Proceedings of the 8th WSEAS International Conference on Computational Intelligence, Man–Machine Systems and Cybernetics (CIMMACS’ 09), Canary Islands, Spain, 14–16 December 2009; pp. 134–138. [ Google Scholar ]
• Rodríguez-Hernández, A.F.; Merino, C.; Casanova, O.; Modrono, C.; Torres, M.A.; Montserrat, R.; Navarrete, G.; Burunat, E.; González-Mora, J.L. Sensory substitution for visually disabled people: Computer solutions. WSEAS Trans. Biol. Biomed. 2010 , 7 , 1–10. [ Google Scholar ]
• Torres-Gil, M.; Casanova-Gonzalez, O.; González-Mora, J.L. Applications of virtual reality for visually impaired people. WSEAS Trans. Comput. 2010 , 9 , 184–193. [ Google Scholar ]
• Maidenbaum, S.; Levy-Tzedek, S.; Chebat, D.R.; Amedi, A. Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the “EyeCane”: Feasibility study. PLoS ONE 2013 , 8 , e72555. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Maidenbaum, S.; Chebat, D.R.; Levy-Tzedek, S.; Amedi, A. Depth-to-audio sensory substitution for increasing the accessibility of virtual environments. In Proceedings of the Universal Access in Human–Computer Interaction—Design and Development Methods for Universal Access: 8th International Conference, UAHCI 2014, held as Part of HCI International 2014, Heraklion, Crete, Greece, 22–27 June 2014; Proceedings, Part I 8. Springer: Cham, Switzerland, 2014; pp. 398–406. [ Google Scholar ]
• Picinali, L.; Afonso, A.; Denis, M.; Katz, B.F. Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge. Int. J. Hum.-Comput. Stud. 2014 , 72 , 393–407. [ Google Scholar ] [ CrossRef ]
• Neidhardt, A.; Liebal, J.; Paasonen, J. Human echolocation in virtual acoustic environments: Estimating the direction of a close wall. In Proceedings of the 4th International Conference on Spatial Audio (ICSA), Graz, Austria, 7 September 2017. [ Google Scholar ]
• Andrade, R.; Waycott, J.; Baker, S.; Vetere, F. Echolocation as a means for people with visual impairment (PVI) to acquire spatial knowledge of virtual space. ACM Trans. Access. Comput. (TACCESS) 2021 , 14 , 1–25. [ Google Scholar ] [ CrossRef ]
• Fujitsuka, Y.; Sumiya, M.; Ashihara, K.; Yoshino, K.; Nagatani, Y.; Kobayasi, K.I.; Hiryu, S. Two-dimensional shape discrimination by sighted people using simulated virtual echoes. JASA Express Lett. 2021 , 1 , 011202. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhao, Y.; Bennett, C.L.; Benko, H.; Cutrell, E.; Holz, C.; Morris, M.R.; Sinclair, M. Enabling people with visual impairments to navigate virtual reality with a haptic and auditory cane simulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; pp. 1–14. [ Google Scholar ]
• Dodsworth, C.; Norman, L.J.; Thaler, L. Navigation and perception of spatial layout in virtual echo-acoustic space. Cognition 2020 , 197 , 104185. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fialho, L.; Oliveira, J.; Filipe, A.; Luz, F. Soundspace VR: Spatial navigation using sound in virtual reality. Virtual Real. 2021 , 27 , 397–405. [ Google Scholar ] [ CrossRef ]
• Real, S.; Araujo, A. VES: A Mixed-Reality Development Platform of Navigation Systems for Blind and Visually Impaired. Sensors 2021 , 21 , 6275. [ Google Scholar ] [ CrossRef ]
• Steffens, H.; Schutte, M.; Ewert, S.D. Acoustically driven orientation and navigation in enclosed spaces. J. Acoust. Soc. Am. 2022 , 152 , 1767–1782. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Steffens, H.; Schutte, M.; Ewert, S.D. Auditory orientation and distance estimation of sighted humans using virtual echolocation with artificial and self-generated sounds. JASA Express Lett. 2022 , 2 , 124403. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Massiceti, D.; Hicks, S.L.; van Rheede, J.J. Stereosonic vision: Exploring visual-to-auditory sensory substitution mappings in an immersive virtual reality navigation paradigm. PLoS ONE 2018 , 13 , e0199389. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Grayson, M.; Thieme, A.; Marques, R.; Massiceti, D.; Cutrell, E.; Morrison, C. A dynamic AI system for extending the capabilities of blind people. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–4. [ Google Scholar ]
• Deemer, A.D.; Swenor, B.K.; Fujiwara, K.; Deremeik, J.T.; Ross, N.C.; Natale, D.M.; Bradley, C.K.; Werblin, F.S.; Massof, R.W. Preliminary evaluation of two digital image processing strategies for head-mounted magnification for low vision patients. Transl. Vis. Sci. Technol. 2019 , 8 , 23. [ Google Scholar ] [ CrossRef ]
• Strumillo, P.; Bujacz, M.; Baranski, P.; Skulimowski, P.; Korbel, P.; Owczarek, M.; Tomalczyk, K.; Moldoveanu, A.; Unnthorsson, R. Different approaches to aiding blind persons in mobility and navigation in the “Naviton” and “Sound of Vision” projects. In Mobility of Visually Impaired People ; Springer: Cham, Switzerland, 2018; pp. 435–468. [ Google Scholar ]
• Bujacz, M.; Skulimowski, P.; Strumillo, P. Naviton—A prototype mobility aid for auditory presentation of three-dimensional scenes to the visually impaired. J. Audio Eng. Soc. 2012 , 60 , 696–708. [ Google Scholar ]
• Caraiman, S.; Morar, A.; Owczarek, M.; Burlacu, A.; Rzeszotarski, D.; Botezatu, N.; Herghele‘, P.; Moldoveanu, F.; Strumillo, P.; Moldoveanu, A. Computer vision for the visually impaired: The sound of vision system. In Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy, 22–29 October 2017; pp. 1480–1489. [ Google Scholar ]
• Shinohara, K.; Wobbrock, J.O. In the shadow of misperception: Assistive technology use and social interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, 7–12 May 2011; pp. 705–714. [ Google Scholar ]
• Giudice, N.A.; Legge, G.E. Blind navigation and the role of technology. In The Engineering Handbook of Smart Technology for Aging, Disability, and Independence ; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2008; pp. 479–500. [ Google Scholar ]
• Meijer, P.B. An experimental system for auditory image representations. IEEE Trans. Biomed. Eng. 1992 , 39 , 112–121. [ Google Scholar ] [ CrossRef ]
• Meijer, P.B. The vOICe—New Frontiers in Artificial Vision. Available online: https://www.seeingwithsound.com/ (accessed on 15 January 2023).
• Microsoft. Seeing AI App from Microsoft. Available online: https://www.microsoft.com/en-us/ai/seeing-ai (accessed on 30 January 2023).
• Schwartz, B.S.; Bell, T. Assistive mobile application for real-time 3D spatial audio soundscapes toward improving safe and independent navigation. In Proceedings of the 3D Imaging and Applications 2023, Electronics Imaging, Society for Imaging Science and Technology, San Francisco, CA, USA, 15–19 January 2023; pp. 108-1–108-5. [ Google Scholar ] [ CrossRef ]
• Unity Technologies. v2022.1.3f1. Unity. Available online: https://unity.com/ (accessed on 7 March 2023).
• Apple Inc. ARKit. v6. Available online: https://developer.apple.com/augmented-reality/arkit/ (accessed on 7 March 2023).
• Unity Technologies. AR Foundation. v5.0. Available online: https://docs.unity3d.com/Packages/ [email protected] /manual/index.html (accessed on 7 March 2023).
• The MathWorks, Inc. MATLAB. R2022b.5. Available online: https://www.mathworks.com/products/matlab.html (accessed on 13 March 2023).
• Warrian, K.J.; Lorenzana, L.L.; Lankaranian, D.; Dugar, J.; Wizov, S.S.; Spaeth, G.L. Assessing age-related macular degeneration with the ADREV performance-based measure. Retina 2009 , 29 , 80–90. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Real, S.; Araujo, A. VES: A mixed-reality system to assist multisensory spatial perception and cognition for blind and visually impaired people. Appl. Sci. 2020 , 10 , 523. [ Google Scholar ] [ CrossRef ]
• Theodorou, P.; Meliones, A. Towards a Training Framework for Improved Assistive Mobile App Acceptance and Use Rates by Blind and Visually Impaired People. Educ. Sci. 2020 , 10 , 58. [ Google Scholar ] [ CrossRef ]
• Leporini, B.; Buzzi, M.C.; Buzzi, M. Interacting with mobile devices via VoiceOver: Usability and accessibility issues. In Proceedings of the 24th Australian Computer-Human Interaction Conference, OzCHI ’12, Melbourne, VIC, Australia, 26–30 November 2012; Association for Computing Machinery: New York, NY, USA, 2012; pp. 339–348. [ Google Scholar ] [ CrossRef ]
• Brown, F.E.; Sutton, J.; Yuen, H.M.; Green, D.; Van Dorn, S.; Braun, T.; Cree, A.J.; Russell, S.R.; Lotery, A.J. A novel, wearable, electronic visual aid to assist those with reduced peripheral vision. PLoS ONE 2019 , 14 , e0223755. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Schwartz, B.S.; King, S.; Bell, T. EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments. Bioengineering 2024 , 11 , 831. https://doi.org/10.3390/bioengineering11080831

Schwartz BS, King S, Bell T. EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments. Bioengineering . 2024; 11(8):831. https://doi.org/10.3390/bioengineering11080831

Schwartz, Broderick S., Seth King, and Tyler Bell. 2024. "EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments" Bioengineering 11, no. 8: 831. https://doi.org/10.3390/bioengineering11080831

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals