Smart Operators: How Augmented and Virtual Technologies Are Affecting the Worker’s Performance in Manufacturing Contexts

Purpose: The correct interaction between the workforce and augmented, virtual, and mixed reality technologies represents a crucial aspect of the success of the smart factory. This interaction is, indeed, affected by the variability of human behavior and its reliability, which can strongly influence the quality, safety, and productivity standards. For this reason, this paper aims to provide a clear and complete analysis of the impacts of these technologies on the worker’s performance. Design/methodology/approach: A Systematic Literature Review (SLR) was conducted to identify peerreviewed papers that focused on the implementation of augmented and virtual technologies in manufacturing systems and their effects on human performance. Findings: In total, 61 papers were selected and thoroughly analyzed. The findings of this study reveal that augmented, virtual and mixed reality can be applied for several applications in manufacturing systems with different types of devices, that involve various advantages and disadvantages. The worker’s performances that are influenced by the use of these technologies are above all time to complete a task, error rate, and mental and physical workload. Originality/value: Over the years augmented, virtual and mixed reality technologies in manufacturing systems have been investigated by researchers. Several studies mostly focused on technological issues, have been conducted. The role of the operator, whose tasks may be influenced positively or negatively by the use of new devices, has been hardly ever analyzed and a deep analysis of human performance affected by these technologies is missing. This study represents a preliminary analysis to fill this gap. The results obtained from the SLR allowed us to develop a conceptual framework that investigates the current stateof-the-art knowledge about the topic and highlights gaps in the current research.


Background and Motivation
The fourth industrial revolution is affecting the workforce at strategical, tactical, and operational levels and it is leading to the development of new careers with precise and specific skills and competence (Di Pasquale, De Simone, Miranda & Riemma, 2021). The implementation of enabling technologies in the industrial context involves new types of interactions between workers and machines, interactions that transform the industrial workforce and have significant implications for the nature of the work (Romero, Stahre, Wuest, Noran, Bernus, Fast-Berglund et al., 2016). This interaction will certainly be affected by the variability of human behavior and its reliability, which can strongly influence safety, quality, and productivity standards.
Among the various enabling technologies, Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) have been proposed in recent years as disruptive technologies within the Industry 4.0 paradigm (Damiani, Demartini, Guizzi, Revetria & Tonelli, 2018). Their use has spread widely in the industrial field from the design phase to production, assembly, and also maintenance and the development of new approaches and techniques of mixed, augmented, and virtual reality allow to enhance many industrial applications from an ergonomic and economic point of view. Augmented reality is a technology that enriches the real factory environment of an operator with information and digital data superimposed in real-time in its field of sight (e.g. headphones, smartphones, tablets, or AR space projectors). The use of AR to improve processes in production has a long history and there are various established use cases in industry and research. Industrial use-cases contain applications of factory planning and several studies on maintenance and inspection (Bottani & Vignali, 2019). Virtual reality is, instead, a computer-simulated multimedia reality that can digitally replicate a design environment and allow the intelligent operator to interact with any presence inside (e.g. a product, a machine tool, a robot, a line production, a factory), with reduced risk, real-time feedback, and cost minimization. Virtual reality holds the promise of potential low-cost solutions for the design and prototyping of new products and the obvious advantage of using it as a training tool. VR offers a safe environment for making errors and makes it possible to practice scenarios that might not occur very often or are difficult to simulate in the real world (e.g. emergencies and events, or involvement of machines or tools unavailable, etc.). Mixed reality and increasing networking performances have, indeed, the capacity to enhance the experience and communication between collaborators in geographically distributed locations. Nevertheless, barriers to their seamless integration in everyday production processes remain.
The results of a recent review of the authors have shown an increasing interest in smart operators, with particular attention to Augmented and Virtual Operators (Di Pasquale et al., 2021). AR and VR involve various company stakeholders, as Managers, Employees, Maintenance Operator, Production Operators, and Logistics Operators, through a purely cognitive type of interaction. However, the significant benefits, such as faster cycle times and reduction of completion times; improvement of reliability and reduction of error rates; shorter learning curve; improvement of health and safety, reported in several research studies about AR, VR, and MR contrast with the limitations and implementation difficulties encountered in the field and the laboratory. The review emphasized that despite the numerous benefits that technologies can bring to the development of augmented and virtual operators, there are several questions still to be addressed to overcome the disadvantages, including several ethical and social issues. The other gap that emerged from the review is that the role of the smart operator is still placed in the background compared to the single technology. In fact, in all the papers analyzed, the focus has never been on the study of the worker's performance but rather on the implementation of the technology itself.
Other researchers have analyzed the scientific literature published on these technologies, examining the current state-of-the-art of AR and VR technology and highlight their key benefits within the industry (Bottani & Vignali, 2019;Damiani et al., 2018;Danielsson, Holm & Syberfeldt, 2020). Despite these studies have underlined the benefits of AR, VR, and MR in various industries and have identified them as powerful tools to improve flexibility and process efficiency, their real use is still not recurrent and to the best of the authors' knowledge, no study has investigated the relationship between technologies and workers' performance (Palmarini, Erkoyuncu, Roy & Torabmostaedi, 2018;Di Pasquale et al., 2021).
Considering the results obtained and the gaps highlighted by the previous study, this paper focuses on the analysis of the impact of AR, VR, and MR technologies on the performance of operators in manufacturing contexts. The purpose is to study and evaluate how the use of AR, VR, and MR technologies support operators in carrying out their work activities and what advantages/disadvantages they entail. The specific research questions, addressed by this study, are reported below: • RQ1: Which are the main applications and types of tasks in which AR/VR/MR are applied? • RQ2: What are the most used types of devices and what are their main characteristics?
• RQ3: What are the impacts of AR/VR/MR technologies on the worker's performance?
• RQ4: Which are the main social challenges to apply AR/VR/MR in production environments?
To answer these RQs, a Systematic Literature Review (SLR) was conducted to identify peer-reviewed papers that focused on the implementation of AR/VR/MR technologies in manufacturing systems and their effects on human performance. The obtained results allowed us to develop a conceptual framework that investigates the current state-of-the-art knowledge about the topic and highlights gaps in the current research.
Since skilled, flexible, and motivated employees have become crucial "assets" for companies to handle all the current challenges and to remain competitive, it is necessary to investigate and direct the study of these technologies not only from a technological but also from a social and human point of view. The development of technologies must be associated with a careful and accurate assessment of the impact on the performance of the operator and consequently on the system, taking into account the great uncertainty and variability of human behavior.
This article is organized as follows. Section 2 provides the research method used for the analysis. Section 3 presents the results of the systematic literature review, which are deeply discussed. Finally, Section 4 presents the main conclusions of the work and future research directions.

Systematic Literature Review
Following a defined and rigorous protocol, a systematic literature review (SLR) allows a critical and reproducible evaluation of relevant studies to answer formulated research questions. A SLR, based on the methodology proposed by (Bayonne, Marin-Garcia & Alfalla-Luque, 2020;Di Pasquale, Miranda, Neumann & Setayesh, 2018), has been conducted to understand how the use of AR, VR, and MR in manufacturing systems affects worker's performance.
The literature search consisted of four consecutive steps: the definition of suitable keywords; the literature database search under constraint; paper selection according to screening criteria; analysis of selected papers and data extraction. To answer the research questions of the paper, four groups of relevant keywords, related respectively to the operator, enabling technologies, performance, and sector were defined (Table 1). The groups of keywords were combined with logical operator AND whereas the keywords of each group with logical operator OR. All the possible combinations of these keywords have been searched in the scientific database Scopus in January 2021. To collect as many papers as possible on the topics, the search covered all the fields managed by the Scopus database. Moreover, the search was limited to the papers published or in press in peer-reviewed scientific journals or conference proceedings from 2015, which provided an English version, and with full-text available. After running the search, all papers were imported into Mendeley, a software tool for managing literature citations.
The selection screening was divided into two stages. The first one involved the reading of the title and the abstract. Papers were classified as included, excluded, or undefined according to specific exclusion criteria: • papers whose main key concept is the enabling technology and not the effects on workers; • papers with no relevant case/user study; • duplicate papers of the same study (only the most complete version was included in the review); • papers not related to manufacturing systems (i.e. medical, educational, or military application).
The second stage consisted of the reading of the full text of the papers which were included in the previous one and represented a definitive assessment for the identification of the most relevant papers based on the exclusion criteria described above.
The exclusion criteria used in the two screening phases were used to select only the papers that addressed the problem of worker's performance or directly (through a focus on the variation of worker's performance with and without the use of AR/VR or MR) or indirectly with the development of a solution such as a model, method, or platform and the relative evaluation of this solution through a user study.

Bibliometric and Content Analysis
The main information of the eligible papers was extracted and stored in a spreadsheet to facilitate data analysis according to several criteria: type of publication (journal or conference), year of publication, source title, keywords. Based on these analysis criteria, a bibliometric analysis of papers was conducted to investigate the trend in literature over the years about the research topic. In particular, the software VosViewer was applied to examine keywords co-occurrence trend of papers.
Furthermore, a content analysis was performed to analyze in detail the selected papers. A Microsoft Excel customized and shared workbook, and related spreadsheets were designed to capture, organize, and store the information collected by two of the authors after the full-text reading of the paper. The following fields were included in the analysis of each paper: • Type of enabling technology dealt with in the paper (augmented, virtual, mixed reality or a combination of them).
• Type of contribution: papers were classified including different types of contribution (development of a solution, survey, or case study for evaluating impacts of technology).
• Type of comparison on which the evaluation of impacts on human and system performances is based. The selected papers were distinguished between studies that compared the worker and system performance with and without the implementation of one of the enabling technologies from studies that, instead, compared the impacts on operators of different solutions/devices for a specific technology. However, in some studies, both types of strategy were pursued, whereas some others focused on the analysis of how technologies affect performances without a real comparison of this type.
• The research environment of the case/user study (laboratory, industry, or both).
• Type of application. The implementation of AR, VR, or MR can be a support for different types of applications in manufacturing systems. Those included in our classification were assistance, training, and remote collaboration. Also, the combination of two or more types of application, if applicable, was considered as an option.
• Type of task. Over the general applications, AR, VR, or MR can be a support for different and specific types of tasks. Assembly, maintenance, order picking, procedural task, quality, and set-up were considered as possible options of this field. Even in this case, the combination of two or more options, if applicable, was included in the classification.
• Types of technological devices were distinguished between Head-Mounted Display, projector, hand-held device, glove, haptic device, and desktop PC. Even these options were not mutually exclusive for each paper.
• Human performance under investigation or analyzed. Four different performances were included in our data collection: time to complete the task, error rate, workload, and health and safety. For each paper, it was analyzed if the single performance was or not one of the objects of study, and, in positive case, the technology's type of impacts on the performance was collected distinguishing between "negative" (performance worsening), "positive" (performance improvement), "non-significant" (performance not significantly affected by the technology), and "solution-dependent" (performance affected by the specific solution/device used).
• Lastly, human factors affected by the implementation of the technology and the main advantages and disadvantages reported by users, their preference, their concerns, and other qualitative evaluations were reported for each paper.
Considering all information gained from the papers, the answers to the RQs, the summary of the state of the art in the research field of interest, and the gaps in the current research were addressed.

SLR Results
The total number of studies that resulted from the database search was 4809 papers. Following the selection screening process described in section 2.1, after the title and abstract reading (first stage), 556 papers were identified as relevant. However, after the full-text reading of papers (second stage), the sample of eligible papers was reduced to 61 papers excluding the articles that do not evaluate, directly or indirectly, the effect of technologies on workers or those that do not present a relevant user/case study. The systematic literature review process, described in the section above, and the relative results are depicted in Figure 1. In Appendix A, the full list of selected papers is reported. The identification numbers reported in Appendix A will be used in the subsequent sections for facilitating the discussion of the RQs.

Bibliometric and Content Analysis Results and Discussions
The number of published articles per year is shown in Figure 2. The publication frequency distribution underlines the growing interest in this topic. In particular, 33 papers (54% of selected papers) were published in the last three years with a peak of 21 papers in 2020.
With a total of 35 sources, the selected papers are not concentrated in any specific journal or conference. Among the selected papers, 27 of these were published in peer-reviewed journals, mainly the International Journal of Advanced Manufacturing Technology (5 papers), Journal of Manufacturing Systems (2 papers), Journal on Multimodal User Interfaces (2 papers), Multimedia Tools and Applications (2 papers). Most of the papers selected appeared in journals mainly relating to the subject area of 'Computer Science' (51.6%), and 'Engineering' (36.8%). Other papers belonged to the category of 'Social Sciences', 'Decision Science', 'Business, Management, and Accounting', 'Mathematics' (11.6 %). The remaining 34 papers were published in conference proceedings, mainly ACM International Conference Proceeding Series (9 papers), Procedia CIRP (4 papers), IFAC-PapersOnLine (3 papers), Conference on Human Factors in Computing Systems -Proceedings (3 papers), Procedia Manufacturing (3 papers).
The research topics of the selected papers were explored by analyzing the network and the distribution of keywords over the years. Initially, this was useful to support our research and identify the main trends in the state of the art. In particular, the keywords co-occurrence analysis was conducted using VOSviewer software and the main results are shown in Figure 3. Both author and index keywords were considered as a unit of analysis. The minimum number of occurrences used for each keyword was 5. With this constraint, 23 keywords met the threshold defining 4 clusters and 166 links. Figure 3 shows the keywords overlay visualization, namely the keywords co-occurrence network map where keywords' colors indicate the average publication year. The weight of nodes, defined as the number of keyword occurrences or their frequency, is indicated by the size of the nodes. For this reason, a greater weight has a larger size of the node. The line between two keywords indicates that they have appeared together, whereas the thickness of the line represents the "link strength" between the two keywords which denotes the frequency of co-occurrence and defines quantitatively the relationship between two nodes. Indeed, VOSviewer allows defining for each node the "total link strength" which is the sum of its link strengths over all the other nodes. Lastly, the color of items is determined by the score attribute represented by the average publication year of occurrences (van Eck & Waltman, 2013).  The results allowed us to identify the main topics of the papers, mainly related to the type of technology and their main fields of application. The keyword "augmented reality" has the highest frequency, whereas the other two enabling technologies investigated in this study, "virtual reality" and "mixed reality", have a lower frequency.
Regarding the fields of application, keywords with the highest overall frequencies are related to assembly (29 with keywords "assembly", "assembly tasks", "manual assembly") and training (25 with keywords "e-learning", "training" and "personnel training"). In terms of frequency, it's clear that "maintenance" (5) and "remote collaboration" (7) are less investigated by researchers in the selected papers.
The effects and the impacts of one technology's implementation on human and production systems are represented by few keywords, such as "ergonomics" and "task completion time", whereas the development trend to improve the relationship between humans and enabling technologies is represented by the keywords "Human engineering", "human computer interaction" and "user interfaces".
Considering the trends over the year, Figure 3 made it clear that "mixed reality" is on average a more recent topic (yellow keywords) compared to "augmented reality" or "virtual reality" (green keywords), whereas for the fields of application, over the years, the researchers focused firstly on assembly and after on maintenance, whereas remote collaboration is the most recent one.
The relationship between "augmented reality" and assembly shows the main field of application for augmented reality technology in manufacturing systems, whereas the relationship between "augmented reality" and "head mounted displays", "helmet mounted display", and "spatial augmented realities" highlighted a primary link between technology and different type of devices/solutions used, which can have a significant impact on the results in terms of performance.
Lastly, the node "virtual reality" has ticker lines with "e-learning" (8), "personnel training" (5), "training"(4), and "augmented reality"(4), whereas the node "mixed reality" has ticker lines with "remote collaboration" (7). These relationships reveal the main field of application for virtual and mixed reality, or rather respectively training and remote collaboration.
In conclusion, despite the selection of the articles strongly focused on the social and human aspects of technologies, this first result highlights how this topic is still not very detailed compared to the purely technical aspects of AR/VR and MR.

RQ1: Which Are the Main Applications and Types of Tasks in Which AR/VR/MR Are Applied?
Qualitative analysis of the selected papers determined that 41 papers (67,2% of the sample) concentrated on one or more case studies to evaluate the impacts of technology on operators and production systems, whereas 19 papers (31,1% of the sample) concentrated on the development of a solution such as a model, method, or platform. Only one paper investigated the expectations of operators on smart glasses through a survey. Most case/user studies or experiments illustrated in papers were conducted in a laboratory (40 papers), whereas only 20 papers examined an industrial application. One paper analyzed multiple case-study developed both in laboratory and industry. To evaluate the impacts of technology on human and system performances, in most papers (31 papers) researchers have been compared the implementation of technology with the baseline (generally paper-based instructions). 9 papers, instead, investigated the dependence of single technology's performances on the type of device or instruction technique through a comparison between different solutions. 18 papers investigated both comparisons, while 3 papers dealt with a particular implementation of technology without comparing results to other ones.
Each paper included in the analysis has been classified according to the type of technology investigated (AR/VR/MR). Most of the papers focused on a single technology, as shown in Figure 4, whereas only one paper dealt with two technologies (augmented and virtual reality).
As also highlighted by the keywords network, studies on augmented reality are more numerous than those on other technologies. For mixed reality, this is also linked to its recent diffusion in the industrial field. Regarding the type of applications, augmented, virtual and mixed reality in manufacturing systems have been identified as interesting support for different applications such as: • assistance = 32 of selected papers (30 papers related to AR, and 2 papers related to MR).
• training = 16 of selected papers (12 papers related to VR, 2 papers to AR, 1 paper to AR and VR, and 1 paper to MR).
• remote collaboration = 11 of selected papers (7 papers related to MR, and 4 to AR).
Only two papers dealt with technical support for both assistance and training.
The main tasks supported by these applications are, instead: • assembly = 36 of selected papers (25 papers related to AR, 7 papers to VR, 3 papers to MR, and 1 paper to AR and VR).
• maintenance = 10 of selected papers (7 papers related to AR, and 3 papers to VR).
• order picking = 4 of selected papers (2 papers related to AR, 1 paper to VR, and 1 paper to MR).
• procedural and search tasks = 4 of selected papers (3 papers related to MR, and 1 paper to AR).
• quality = 3 of the selected papers related to MR.
The remaining 4 papers analyzed packing, health, and safety and more than one type of task supported by technology. Figure 5 shows a Sankey diagram that emphasizes the main papers' qualitative characteristics and the relationship between them. For further details, see Appendix A. As evident from Figure 5, the selected articles are quite heterogeneous and dealt with the interaction between humans and AR/VR/MR technologies in different industrial contexts and with different purposes. However, some more relevant trends can be highlighted. The use of AR is strongly aimed at assisting operators in their daily activities, with a particular focus on manual assembly and maintenance. This result reflects one of the main characteristics and advantages of AR, namely making operators capable of performing tasks that require a higher level of qualification, avoiding replacing manpower with machines to improve work quality. Virtual reality, on the other hand, confirms its wide use for training. Mixed reality is used to remotely collaborate and guide novices with great potential in many applications (e.g. mechanical maintenance, remote supervision of complex mechanical assembly, training, collaborative design, and decision making in engineering).

RQ2: What Are the Most Used Types of Devices and What Are Their Main Characteristics?
It is worth considering the type of device investigated in selected papers because the effects on human performance are affected by this element. Most of the studies (42 papers) have been based on the use of Head-Mounted Display (HMD) for AR (24 papers), MR (10 papers), and VR (8 papers) applications. Alternative technological solutions to HMD are projectors (15 papers), hand-held devices, as smartphones or tablets, analyzed in 14 of the selected papers, and desktop PC (9 papers). Few studies investigated, instead, haptic devices (4 papers) or gloves (1 paper). Figure 6 reports the classification of the different devices based on the technology (see also Appendix A). Figure 6. Type of devices for augmented, virtual and mixed reality As highlighted in the results, especially for AR there is a great diffusion of HMD. Research by Kubenke and Kunz (2019) reports that while performing a machine-setting task with a higher amount of specific information, the IT glass turned out to be the most efficient assistive system due to the stepwise visualization of the work instruction. As well as, HMD has proved to be more suitable for the operator's training than paper manual and computer screen (Mengoni, Ceccacci, Generosi & Leopardi, 2018). However, as analyzed in more detail in the following sections, several studies highlight disadvantages as bad ergonomics, low resolution, an excess of weight, and limited/fixed focal depth, headaches, and often the users reported a limited field of view in which the annotations were visible as well as handling problems if the user already was wearing regular glasses (Leutert & Schilling, 2018;Princle, Campbell, Hutka, Torrasso, Couper, Strunden et al., 2018;Uva, Gattullo, Manghisi, Spagnulo, Cascella & Fiorentino, 2018). Wearable devices, in general, resulted invasive for the operator and limit the possibility to focus on short and long-distance objects. In particular, the head-mounted displays can be unsuitable for some tasks in manufacturing, and many local workers do not want to wear them, especially, when manipulating some large and heavy metal mechanical parts or working in a small space.
In comparison, mobile devices such as smartphones and tablets (hand-held devices) are cheap, flexible, and accessible to many customers. They are capable of position tracking and visualizing information. Therefore, these mobile devices are becoming popular augmented reality tools. Using a tablet for an AR application has advantages, almost everybody is used to tablets nowadays lowering the entry-level for the users. It also has both a camera and a screen which enables a fully functioning AR application in one single device. Although this kind of device is very easy to implement due to the availability of low-cost and powerful devices, in practice, it has various limitations. One of the most important is that operators should employ one or even two hands for the visualization, thus limiting their ability to operate (Uva et al., 2018). For example, the handling of the tablet was reported to be cumbersome -users preferred to have their hands free for work (Leutert & Schilling, 2018). A tablet is inexpensive and a standardized off-the-shelf product enabling multiple users without large budget depositions. An AR application in a tablet either occupies one hand of the operator or, if placed in a stand, must be constantly re-located for being in the operator's field of view. It is often in the way and at the stand, it is not in the line of sight of the operator during work. Another disadvantage is if the hand or arm of the operator or a tool comes in the way of the camera, the virtual objects disappear since the picture used as the AR anchor is covered. However, the use of AR/VR and MR instructions on mobile devices has not been evaluated in many studies yet. So, it is unclear whether the instructions implemented on a mobile device will result in favorable task performance processes.
A display solution that does not suffer from the problem above (ergonomics, not free hands, angular offset) is represented by spatial augmented reality (SAR) using digital projectors. The added value of SAR is the possibility to locate with graphic signs on the real product, the parts involved in the task. The SAR highly reduces the task complexity because the technical information is conveyed illuminating the component/part/item to be pulled out/inserted/assembly/pick based on the type of task. Projectors seem to be more efficient, intuitive, and fast than other visualization devices (Funk, Kosch & Schmidt, 2016;Mengoni et al., 2018) and the perceived workload of operators is significantly reduced when projected instructions were compared to screen instructions (Bosch, Van Rhijn, Krause, Könemann, Wilschut & De Looze, 2020;Bosch, Könemann, De Cock & Van Rhijn, 2017). With the SAR, we have then two great benefits: (1) the reduction of the mental workload associated with a task because only the task-relevant information is displayed; (2) the reduction of the cognitive distance because of the information space and the physical space coincides.
However, the main disadvantages of SAR, as pointed out by Uva et al. (2018) are the following: surface-based distortions; brightness, contrast, and visibility of projection; lower color fidelity; higher latency; and registration. The problem most reported with the projector-based system was occlusion by the user (Leutert & Schilling, 2018). Despite the issues, the hands-free character of in-situ projection will have great potential for instruction systems at the workplace, as HMD instructions have problems being accepted by workers and tablet instructions interfere with a two-hand assembly .
Another device that is adopted very frequently for VR is the computer screen or desktop PC. However, several results highlighted that a computer screen is not suitable for productive context, despite it does not have problems of visual occlusion, because it distracts the operator's attention (Mengoni et al., 2018).
It is evident that the implementations of AR/VR and MR are very influenced by the devices used. For this reason, several studies have compared the worker's performance in the execution of a specific task using different devices (paper IDs 3,5,8,10,13,14,17,19,23,24,27,28,31,34,35,38,39,41,(46)(47)(48)51,52,(57)(58)(59)(60). The objective in many studies was, in fact, to understand, concerning the task under investigation, which was the most suitable device in terms of performance and preference of the operators. However, a clear and complete framework on this aspect is not yet present in the literature. Furthermore, even for single device classes, technological development is still growing, especially to improve their usability, the user experience (measuring, for example, levels of Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty), but also the collaborative experience and the social presence. Several articles have compared different interfaces or different communication systems to identify the most technologically valid and performing solutions for a specific task (paper IDs 3,5,13,17,24,28,38,39,51). The results obtained from the review underline the ongoing development of the various devices and the efforts of the researchers to identify the fields of application most suitable for each type of device, considering the advantages and disadvantages just discussed.

RQ3: What Are the Impacts of AR/VR/MR Technologies on the Worker's Performance?
The analysis of the selected papers and the study of the reported user/case studies made it possible to carry out a preliminary analysis on the impacts of the individual technologies on the performance of the operators.
Going into greater detail for each performance parameter, the impact on performance was assessed, paying particular attention to the correlations between time, error and workload, and technology identified in each study to answer RQ3. AR/VR/MR can contribute negatively, positively, or insignificantly to worker's performance. A positive correlation indicated a decrease in the time to complete a task, error rate, or physical and mental load with AR/VR/MR compared to performing the same task without the technologies. Furthermore, considering the studies that compare the impacts on operators of different solutions/devices for the same technology, in many cases, it was not possible to determine the type of correlation because it is a function of the different solutions analyzed. For this reason, the results reported in Tables 3, 4, and 5 show the positive, negative, non-significant, and solution-dependent impacts.
As reported in Table 3, most of the selected papers have evaluated the benefits that AR systems can generate in industrial operations, compared with traditional techniques, by means either of user studies or technical tests.
Many of the studies have shown a positive impact on the performance of operators both in terms of time taken to perform the activity, as well as in the number of mistakes made and an improvement in mental and physical load. In particular, the use of augmented reality turns out to be very effective in increasing the reliability of the operators and reducing the errors committed, thanks to the various support and assistance features they provide. Compared to time, however, a higher number of negative impacts were identified. This is linked in several cases to the inexperience of operators in the use of AR devices but also to problems related to the devices, as reported in the previous section. However, it should be emphasized that many studies have not analyzed the long-term impacts of the use of technologies and, therefore, the acquisition of greater confidence and familiarity with the devices could probably have a positive effect on the performance of operators.  Table 3. Impacts of augmented reality on worker's performance.
For virtual reality and mixed reality (Tables 4 and 5), as also highlighted in Figure 7, the results obtained from the review are limited. For mixed reality, most of the results are related to comparisons between different solutions.
The workload for AR, VR, and MR reported conflicting and not perfectly indicative results, given the heterogeneity of the data. This is also due to the strong relationship between mental and physical load and the type of device used. Concerning the workload, it is also interesting to underline that only one recent study (Drouot, Le Bigot, Bolloc'h, Bricard, de Bougrenet & Nourrit, 2021) has focused, specifically, on the impacts on the visual system of operators. Research by Drouot et al. (2021) has specifically investigated the visual system and subjective fatigue symptoms (blurred vision, sore/aching eyes, and headache) during working activities. The sample's large size of papers selected for this study does not allow us to investigate this aspect in depth. Starting from this gap, a future study will have to investigate the ergonomics level and the human factors involved in the use of different devices to be able to define a standard scheme of impacts on the physical and mental load of the technologies to then evaluate the effects on performance. Another aspect on which the analysis has focused is that of security. Unlike time, error, and workload, measuring performance in safety terms is more complex because it would be necessary to evaluate the use of devices for long periods and measure incidents, injuries, and other events. As reported in section 3.2, however, most of the studies were conducted in a laboratory (40 papers) with a reduced number of replicas, and only 20 papers conducted the user study in an industrial context, but also in this case for limited periods. A medium-long-term analysis has not been carried out and this makes studies on performance in terms of safety scarce.
The only studies that addressed this problem (5 papers -IDs 7, 10, 15, 32, 37) have highlighted how AR/VR/MR systems will influence HFE at the workplace. The study by Vorraber, Gasser, Webb, Neubacher and Url (2020) reported that 67% of participants in the study agreed that risk was minimized using AR-based remote maintenance processes supported by cutting-edge optical head-mounted display technology (paper ID 7). The research by Aromaa, Väätänen, Kaasinen, Uimonen and Siltanen (2018) pointed out how the use of hand-held devices may also raise new kinds of safety issues as people are viewing the environment via AR devices (paper ID 37). Indeed, some participants in their study were concerned about the "safety" of the tablet: it should not break, and it should tolerate the harsh industry conditions. However, the participants did not feel that there would be any safety risks when using the AR system. Participants even suggested that the use of an AR system could improve safety because instructions are followed in systematic order. In addition, the use of AR can be safer than talking on the phone to get help.
In addition to the main worker's performances, a limited number of papers, investigating the use of technologies for training, analyzed and measured the impacts on learning ability or learning curves. For example, in the research by Büttner, Prilla and Röcker (2020), the worker's performance achieved was measured thanks to the use of AR as a learning tool compared to traditional paper-based methods. The results showed that, while after one day, traditionally trained operators report lower error rates than AR-trained operators, opposite data are recorded after one week. The impact on learning is also assessed in (Schwarz, Regal, Kempf & Schatz, 2020) which highlighted the potential of virtual reality as positive training transfer into practice and positive skill retention over time compared to conventional training outcomes could be achieved.
Finally, it should be emphasized that regardless of the measured performance, especially in the laboratory studies, the participants showed a good predisposition towards AR/VR/MR technologies. In the study by Smith, Semple, Evans, McRae and Blackwell (2020), although performance when using paper instructions was better than AR for task time, error count, and cognitive effort, users reported finding the AR system easier and more enjoyable to use than its paper counterpart, and this is reflected in the system usability scores. The expert workers involved in the case study of Muñoz, Martí, Mahiques, Gracia, Solanes & Tornero (2020) highlighted the great robustness of the application working in an industrial environment and pointed out that the device and virtual objects did not affect their view, which means an improvement of the worker safety. Moreover, they highlighted the easiness of placing the virtual interface anywhere in the environment and modifying its size, reducing the physical stress of the expert while performing the task, which means an improvement of the worker ergonomics. Hoover, Miller, Gilbert and Winer (2020) reported several positive feedbacks of the 35 participants who used the HoloLens AR instructions. Twenty-nine percent of the HoloLens participants mentioned that they thought the HoloLens AR instructions were easy to use, and 11% said that it was easier to use than a paper manual. Moreover, 11% of the surveyed participants reported that they felt the HoloLens AR instructions reduced their mental workload in some way. However, also negative feedbacks were reported. 26% of participants mentioned problems with the 3D tracking or registration of virtual images with the real environment. Fourteen percent of participants mentioned that the HoloLens was heavy or uncomfortable to wear during the study. Another 11% suggested that having the AR graphics constantly in their field of view was distracting or annoying.

RQ4: Which Are the Main Social Challenges to Apply AR/VR and MR in Production Environments?
Review results show an increasing interest in the interaction between workers and AR/VR/MR technologies but a low focus on the social challenges related to the implementation of these technologies in industrial environments.
One of the first things that emerge from the review is that the technologies under investigation are not new and already exist in the industrial context but the challenge that most of the researchers aim to overcome is to interconnect these technologies to enhance the role of the operator. The operator must benefit from technologies that do not hinder him/her but enhance him and make him/her carry out each action with confidence without mental or physical stress or insecurities. As pointed out in section 2.1, for the purpose of this study, only the papers that addressed the problem of worker's performance or directly (through a focus on the variation of worker's performance with and without the use of AR/VR or MR) or indirectly (with the development of a solution such as a model, method, or platform and the relative evaluation of this solution through a user study) have been selected. Despite this, in most of the papers analyzed the focus has never been on the study of the performance of the operators but rather on the implementation of the technology itself. What is evident is that the role of the operator is still placed in the background compared to the technology that characterizes it and this represents an important research social challenge on which we must focus in the future.
The analysis of the papers made it possible to identify also other challenges and barriers still present in research and practice.
First, as highlighted in the reply to RQ2, the devices used still have considerable problems that make it difficult for workers to use, especially for long periods. Participants in various studies reported symptoms of nausea, sickness, and/or disorientation, as well as excessive physical exertion, which can lead to musculoskeletal disorders. Another problem is that related to the usability of the devices, namely the appropriateness to the purpose of the AR/VR/MR solutions. Numerous studies (paper IDs 5,6,8,10,13,16,(22)(23)(24)26,28,30,40,44,45,54) have measured the system usability through specific questionnaires or System Usability Scale SUS standard. The SUS questionnaire was widely used to test the usability of the proposed interface due to its simplicity, speed, and ease of use (Brooke, 1996). The usability scores recorded in the various studies are not always comparable as they strongly depend on the questionnaire submitted and on the technical and graphic specifications of the device used.
However, what can be seen is that many of the problems related to the usability of the devices are linked to the graphic interfaces and the communication systems used, as well as to the AR/VR/MR contents.
It should also be emphasized that several papers have highlighted the problem of experience, as a factor influencing the performance of operators. On the one hand, the researchers highlighted that for workers with insufficient experience in the task, the visual support provided by AR allows significantly improving performance, both in terms of errors and time taken (paper IDs 4, 6). This highlights the importance of educating and including operators early in the implementation project to gain the expected benefits. On the other hand, previous experience with technology is another discriminating factor to be taken into consideration.
Furthermore, although not many papers have been identified that analyze the use of technology as the age varies, the implementation of AR/VR technologies must take into account age-related problems. Targeting the needs and capabilities of aging employees is inevitable to address the ongoing demographic change and remain competitive.
Moreover, the trust in technologies and their acceptance by the operators represent one of the crucial challenges to be overcome. Over the years, automation and digitalization have been associated with the reduction of jobs and this often leads operators to react with distrust in new technologies. It is obvious that user acceptance often falls by the wayside, even if it is a crucial aspect of the diffusion of new technologies in society and working environments.

Conclusions
The main objective of the present research work is to investigate the recent scientific literature on the analysis of the impact of AR, VR, and MR technologies on the performance of operators in manufacturing contexts. The 61 selected papers allowed us to answer the 4 RQs, highlighting the growing interest in this topic but also the several research gaps.
What is evident is that such technologies certainly have numerous potentials to be used in different work environments, although many efforts still need to be made to improve their usability by operators and eliminate the negative effects on health and performance. AR/VR and MR are technologies strongly linked to operators and cannot prescind from their physical, cognitive, perceptive, and psychosocial characteristics. It would be useful to establish a set of human factors design guidelines that can help match device capabilities with task and environmental requirements of the workplace.
The review presents several aspects to be improved in the future to investigate the field of interest in greater depth. The heterogeneity of the data did not allow in some cases to support univocal statements on the impacts of technologies on the performance of operators. Future studies will have to focus on individual technologies by thoroughly evaluating the impacts of varying devices used, in order to identify and define the most suitable fields of application. Furthermore, the study of factors such as experience, age, trust, acceptance should be further investigated to evaluate their effect on the performance of operators.

Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The research work is part of the activities carried out in the context of the SO4SIMS project (