Method and materials

Related works
The literature analysis for the development of this segment seeks to mainly highlight
certain essential points in the application of mixed reality in the higher education department.
Further, the analysis also highlighted certain core literature reviews conducted within the past
three years by various researchers to offer an extensive insight of the conducted researches
and the gaps that the research sought to fill within this analysis. The studies based on the
aspect of technology have been conducted mainly to illustrate the particularities involved in
the design and development of the features of mixed reality application, and the presently
accessible platforms and tools that can be used to ensure the application of such reality in
education. Yu et al. (2019) presented an analysis on the prevailing platforms that are designed
to enable the application of mixed reality. On the other hand, Wu et al. (2019) conducted
analysis by comparing the existing platforms used in the development of mobile mixed
reality strategies and based their analysis on the developed resources of mixed reality that
such strategies provide for the construction of the educational resolutions mainly based on the
mobile devices.
Another research conducted by Murugan, Balaji and Rajkumar (2019) surmised that
the mixed reality platforms in the educational procedures is the most current that are updated
by both the developer and user in a cooperative manner and enables for a particular
perception of the tools applied. Wish-Baratz et al. (2020) in their research addressed the
concept of maintainability in relation to the mixed reality aspects and attributes, mainly
identifying and describing the discrepancy of the integration between the soft and hardware,
and the mixed reality resolutions. Moreover, from the educational viewpoint, numerous
researches have explored initiatives via various literature reviews. For instance, Gerup,
Soerensen, and Dieckmann (2020) assessed the application of the mixed and augmented
realities in healthcare particularly in the surgical training. Although the augment and mixed
reality strategies designed did not possess a clear pedagogical structure, it involved the
application of both mixed and augmented realities in teaching and training the healthcare
students. The study also evaluated the visions for the future and chances for the future studies
within the augmented and mixed reality application for educational settings. Kounlaxay and
Kim (2020) in their research described the results of their review that the particular direction
of the mixed reality strategies would potentially result in some advantages including the
increase in students’ motivation to learn. The study by Al Janabi et al. (2020) further
supported the findings by Kounlaxay and Kim, and highlighted that the mixed reality
approaches would facilitate a better learning performance among the students, since the most
identified challenges by most of the researchers is focused on the difficulty for the learners to
effectively use this form of application in its optimal form to facilitate learning activities.
Method and materials
A systematic literature review was conducted to identify and summarize all the
researches published in a thorough research progress to conduct analysis of the studies in the
research. It was systematically organized as aligned to the preferred reporting item for
systematic review and meta-analysis, PRISMA, standards, as a three-stage process consisting
of planning, conducting and reporting (Asar et al., 2016).
Research method and research questions
The preferred reporting for the PRISMA statements was adopted in the literature
mainly by extracting the key information for the study and specifying the research
characteristics adopted from PICO model, as suggested by Bramer et al. (2018), in the five
identified digital libraries, that is the SciTech Premium Collection, Australian Education
Index, APA PsycInfo®, ERIC; Social Science Premium Collection, and the Scopus. The
search established that there are numerous study questions that are mainly focused on the
systematizing and structuring the study on mixed reality application for the higher education,
which are mainly aligned to attain the objectives of the study including;
i.
What mixed reality technologies are used in higher education?
ii.
What characteristics or design elements of the mixed reality model are applied to higher
education?
iii.
What do learning theories examine the use of mixed reality model in higher education?
iv.
What are the higher education domains that use mixed reality models to examine the
learning theories?
v.
What are the differences in the learning outcomes of higher education students who use
the MR model and those who do not?
vi.
How are the MR model and higher education students’ performance or learning
outcomes related?
The Review process and search strategy
The review process encompassed certain procedures, decisions and considerations,
which led to the consolidated list of the articles to be analyzed in-depth. The systematic review
in the selected model consists of four phases of articles selection procedure, the identification,
screening, eligibility, and included, as illustrated in the figure below.
Identification
SciTech Premium
Collection
database
(n = 41)
Australian
Education Index
database
(n = 1)
Scopus
database
(n = 40)
search in web of
sciences, DGRL,
and Scopus. (n = 7)
Screening
Records after duplicates removed
(n = 88)
Records screened
(n =70)
Eligibility
Full-text articles assessed
for eligibility
(n =52)
ERIC; Social
Science Premium
Collection
database
(n = 3)
APA
PsycInfo®
database
(n = 4)
Records excluded
(n =18)
Full-text articles excluded,
with reasons
(n =27)
Included
Studies included
(n =25)
Figure 1: The article-selection process
Definition of review scope, keywords and research question
In the definition of the review, we followed the procedure, beginning with the
selection of keywords search approaches in all the relevant digital libraries. The digital
libraries used in the study includes SciTech Premium Collection, Australian Education Index,
APA PsycInfo®, ERIC; Social Science Premium Collection, and the Scopus. SciTech
Premium Collection is often a rich repository, which mainly covers the domains of the
computer science, engineering, information technology, and other software related
technology. On the other hand, the Scopus database offers a wide array of publications
domains, which cut across all the fields encompassing of the natural sciences, technology,
social sciences, information technologies, and medicine. The other databases aids in the
provision in other fields that were not presented in the first two databases, including the
social sciences, humanities spheres, and arts. The keywords using in the search for the
articles to be used in the study consisted of mixed reality and educate, with various
combinations for the keywords for educations, learning paradigm, higher education,
university students, learning outcome and theories, plus the combined keywords and subject
headings in the identification of the technical mixed reality articles. Hence, we defined the
following search string for the database search as shown in the figure below
Databases
Search terminologies in the titles and keywords
SciTech Premium
‘Mixed reality’ OR ‘MR’ OR ‘learn’ OR ‘teach’ OR
Collection, Australian
‘higher education’ OR ‘train’ OR ‘undergraduate’
Education Index, APA
OR ‘college’ OR ‘graduate’.
PsycInfo®, ERIC; Social
Science Premium
Collection, and the Scopus
Initial articles search in five databases
For the article search in the databases, we specifically defined the search string to
‘higher education’ OR ‘university’ OR ‘college’, which was significant in reducing the
inappropriate areas like the application of the mixed reality in primary, vocational, or even
the secondary education. Further, by adding such keywords, we reduced the results of the
databases from approximately 200 articles to roughly 150 articles. Besides, the term “not”
(‘artificial intelligence’ OR ‘machine learning’ OR ‘neural network’ OR ‘deep learning’) was
added to eliminate all the articles that were based on artificial intelligence lacking the
inclusion of the context of human learning. The results from the databases primarily covered
the peer-reviewed scientific papers and conference articles that were published 2017 to date.
The amassed results for the original search were about 89 journals. Since we had a huge
number of results from the initial search, we adopted two-phased filtering procedures, that is
the semi-automatic filters for the inclusion and exclusion strategies, and the manual filters in
the identification of the potential articles.
Exclusion and inclusion method
We first performed the content analysis of all the databases identified form the
research by KH Coder 3, which is applicable in the quantitative content analysis,
computational linguistic and text mining purpose. To perform this analysis, pre-processing
was carried out by removing punctuation marks. Also, we removed the stop words that offers
no additional inferences to the sentences. Additionally, the review was mainly restricted to
the English-language researches that were published from 2017, which were perceived as
effective since the databases revealed an increase interest in the topic of the virtual and mixed
reality since 2016 with the release of the immersive HTC Vive headset. Hence, training the
database to begin the search in 2017 greatly increased the possibility of acquiring immersive
mixed reality-based learning articles. Further, because of the novelty in the MR domain, the
inclusion of the conference articles was considered necessary, since most of the innovations
surrounding the MR application in education were published in the conference papers rather
than the journal articles. Thus, the search was restricted to the scholarly and conference
articles and proceedings. Hence, the eligibility of every research article was assessed based
on the title, which led to 18 out of 70 articles being excluded from the digital libraries
reducing the number to 52.
Removing duplicate documents
Due to different databases used in the articles search, finding duplicate was an
irreplaceable, hence making this process significant in the search. From the articles identified
for the research, there was merely one article that was duplicated to be removed by the find
duplicates attributes within the article’s endnotes after the importation of the article from the
database.
Manual selection process 1: Reading the titles and abstract
From the 52 articles remaining, the abstract identification in the search were assessed
for the inclusion, which allowed for the researchers to identify the articles with abreacts that
satisfied the listed criteria. The assessment of the 52 abstracts and titles that allowed them to
be market as either irrelevant or relevant for the research. This process resulted in the exclusion
of about 10 articles.
Manual selection process 2: Reading the contents
The assessment of abstract was preceded with the reading of the contents and
classification of the articles into various concepts that were defined and described to avert
multiple interpretations. This step allowed the number of the articles to be further reduced by
seven articles.
Manual selection process 3: Further exclusion of irrelevant entries
The remaining articles were read further, which enables the researchers to remove all
the irrelevant articles. This resulted in only 25 articled being considered as relevant for the
research, which are included in the systematic review.
Data collection process
During the data extraction process, the relevant data needed for the research was
recorded from all the selected 25 articles to offer sufficient information on the research
questions (Cumpston et al., 2019). The resulting data was tabulated to create patterns and
trends in the identified articles, which enabled further analysis and summary of the articles.
Additionally, to extract the required information, a worksheet was used to record the metaanalysis for the 25 journals. Overall, to execute the meta-analysis of the 25 articles deemed
relevant for the research, all the articles were independently reviewed and the data regarding
the method applied, domain to which the strategy is applied, MR, and application of the MR
in higher education was extracted. The final stage for the research regarded the synthesis of
the data extracted that encompassed of particular phases in the evaluation (Campbell et al.,
2020). The researchers first conducted the systematic analysis of the raw data through the
described literature review process and enlisted the results in the preceding section of the
study. Additional to the collection of the metadata concerning the selected articles in the
study’s literature review, similar metadata concerning the researches included in the topic.
Moreover, the authors compared the data derived for all the meta-data features for the 25
articles and generated patterns and extensive results, which were recorded, discussed and lists
among the manuscripts, and aided the process of designing drafted recommendation for the
research. Thus, the research agenda is primarily based on the information assessed from the
25 articles.
References
Al Janabi, H.F., Aydin, A., Palaneer, S., Macchione, N., Al-Jabir, A., Khan, M.S., Dasgupta,
P. and Ahmed, K., 2020. Effectiveness of the HoloLens mixed-reality headset in
minimally invasive surgery: a simulation-based feasibility study. Surgical
endoscopy, 34(3), pp.1143-1149.
Asar, S., Jalalpour, S.H., Ayoubi, F., Rahmani, M.R. and Rezaeian, M., 2016. PRISMA;
preferred reporting items for systematic reviews and meta-analyses. Journal of
Rafsanjan University of Medical Sciences, 15(1), pp.68-80.
Bramer, W.M., de Jonge, G.B., Rethlefsen, M.L., Mast, F. and Kleijnen, J., 2018. A
systematic approach to searching: an efficient and complete method to develop
literature searches. Journal of the Medical Library Association: JMLA, 106(4), p.531.
Campbell, M., McKenzie, J.E., Sowden, A., Katikireddi, S.V., Brennan, S.E., Ellis, S.,
Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J. and Welch, V., 2020.
Synthesis without meta-analysis (SWiM) in systematic reviews: reporting
guideline. bmj, 368.
Cumpston, M., Li, T., Page, M.J., Chandler, J., Welch, V.A., Higgins, J.P. and Thomas, J.,
2019. Updated guidance for trusted systematic reviews: a new edition of the Cochrane
Handbook for Systematic Reviews of Interventions. Cochrane Database Syst Rev, 10,
p.ED000142.
Gerup, J., Soerensen, C.B. and Dieckmann, P., 2020. Augmented reality and mixed reality for
healthcare education beyond surgery: an integrative review. International journal of
medical education, 11, p.1.
Kounlaxay, K. and Kim, S.K., 2020. Design of Learning Media in Mixed Reality for Lao
Education. CMC-COMPUTERS MATERIALS & CONTINUA, 64(1), pp.161-180.
Murugan, A., Balaji, G.A. and Rajkumar, R., 2019, November. AnatomyMR: A Multi-User
Mixed Reality Platform for Medical Education. In Journal of Physics: Conference
Series (Vol. 1362, No. 1, p. 012099). IOP Publishing.
Wish-Baratz, S., Crofton, A.R., Gutierrez, J., Henninger, E. and Griswold, M.A., 2020.
Assessment of mixed-reality technology use in remote online anatomy
education. JAMA Network Open, 3(9), pp.e2016271-e2016271.
Wu, W., Hartless, J., Tesei, A., Gunji, V., Ayer, S. and London, J., 2019. Design assessment
in virtual and mixed reality environments: Comparison of novices and
experts. Journal of Construction Engineering and Management, 145(9), p.04019049.
Yu, H., Zhou, Z., Lei, X., Liu, H., Fan, G. and He, S., 2019. Mixed Reality− Based
Preoperative Planning for Training of Percutaneous Transforaminal Endoscopic
Discectomy: A Feasibility Study. World neurosurgery, 129, pp.e767-e775.
…