S PC -Based Model for Evaluation of Training Processes in Industrial Context

: Purpose: This article aims to present successful practices in the management of training processes based on virtual reality and augmented reality, namely a strategy for evaluating the process with the principle of continuous improvement in mind, and monitoring its performance in terms of productivity and waste levels. It is proposed to apply Statistical Process Control (SPC) tools to develop control charts for monitoring individual events (i-charts). Design/methodology/approach: The methodology is based on a case study developed in an industrial project and is guided by a literature review on Work-Based Learning (WBL) and SPC. Findings: The developed work shows that SPC tools are suitable for supporting decision making in situations where the data to be analyzed is generated by human-computer interactions, e.g., involving students and virtual learning environments. Originality/value: The innovative aspect presented in the article lies in the evaluation of the effectiveness of pedagogical resources arranged in simulation environments based on virtual and augmented reality. The accumulated knowledge about the application of SPC in service areas, and others that demand data analysis, reinforces the hypothesis of the suitability of its application in the case presented. This is an original application of SPC, normally used in business processes quality control, but which in this case is applied in an innovative way to the evaluation of industrial training processes, with the same spirit for which it was designed, i.e. to provide the means to manage the quality of a process.


Introduction
Business Process Management (BPM) research exhaustively discusses the importance of process monitoring for performance analysis (Kohlbacher, 2010).This may be seen as an essential activity for quality control in industrial environments, supporting companies with the ability to keep entropy controllable and to guarantee the capacity of reaction to events whose behavior is not completely known or controllable.The referred business processes are performed by functional companies' resources, which apply their different abilities in an interconnected way to deliver intended outcomes (Lima, 2010), and as can be foreseen, these processes are transversal to all company functional areas and activities.It is usual that companies have to develop their own training processes (De Vin, Jacobsson, Odhe & Wickberg, 2017), adaptable to their own production systems.In this case, companies struggle with evaluation of the quality of those processes and with the related continuous improvement process.The evaluation of a training process aims to provide information to support the decision making of the trainer, the manager of the process and other decision makers.Tools in data analysis are fundamental requirements in digital learning environments for decision making as they presuppose learner autonomy for the exploitation of learning resources.In order for learners to interact with the environment, trainers and managers need the means to assess whether this communication is in accordance with the intended objectives in order to decide on the most appropriate interventions to be undertaken when they are needed.In addition, not only immediate corrective actions are necessary to maintain the effectiveness of the model, the learner's experience in the learning environment as a whole is also important for the evolution of the model.The tools adopted for these purposes, namely the SPC and statistical classification of experiences, have proved adequate for this purpose.
Statistical Process Control (SPC) is an instrument used in the field of quality management that has proven to be efficient in the management of professional training processes in the workplace by providing the necessary means for monitoring process performance and assisting in decision making for corrective actions or reconfigurations to meet new scenarios.According to Kenett, Zacks and Amberti (2014), SPC is a technique that has as principle the analysis of control limits.Those limits can fluctuate and tend to stabilize as the monitored process matures or can be initially set using the parameters defined previously according to the experience of practitioners.SPC is a traditional tool in Industrial Engineering in the field of quality management and, nowadays, with the advance of technological resources for data collection and analysis, is gaining new nuances of importance in its role of enabling coordination in process management (Mirzaei, Niroomand & Zare, 2016).
SPC is a tool that, based on performance data collected from the process, enables to set a pattern of its natural variability Öberg, Hammersberg and Fundin (2017).Such a pattern is then used to compare further outcomes of the process, so that unusual variations can be detected.In this sense, that is a tool that adjusts itself according to the system behavior.In other words, it learns from the system and offers managers the ability to articulate strategies based on factual data rather than planning actions to achieve goals concerning optimal scenarios (Banks, 1993).These characteristics are quite appropriate for training processes because each training is ultimately unique as each participant responds to it in a particular way.The more the training processes manage to respond to the individual particularities of the participants, the more effective they are (Sousa & Dinis-Carvalho, 2020).Therefore, a significant challenge for training managers focused on competence development and for trainers, who are at the forefront of this activity, is precisely the identification of the individual needs of the trainees.Despite this characteristic, training programs are usually generic and applied to everyone indistinctly.As mentioned above, in business process management, each process is designed to achieve a goal, and its performance is continuously evaluated to enable decision making in case of inefficiency.The objective of the training process in the work environment is the development of competencies related to the functions performed by the participants.However, unlike sales processes, for example, where a target such as reducing the number of missed delivery times by 5% is quite tangible, targets such as increasing the effectiveness of training or reducing training time without impairing its effectiveness are somehow abstract and too elusive to the many control tools available.
Therefore, the objective of this article is to present a proposal for evaluating the performance of a business process responsible for training newly hired operators.Thus, it is important to consider that the performance of the process is directly related to its ability to provide learning, that is, to the effectiveness of learning resources.This can be measured from the analysis of the data generated in the learners' interaction with these resources.In order to consider that such resources are designed to satisfy a frequent behavioral profile (which in the traditional form of training is identifiable only from the experience and competence of the trainers) the SPC based model presented in this paper allows real-time monitoring of the learning process and agile decision-making.This is accomplished by identifying exceptions to control limits that are dynamically determined from historical data on learning resource usage.This paper aims to demonstrate the applicability of SPC as a suitable tool for the evaluation of business processes for the management of training in the workplace, and therefore intends to answer the following questions: 1. What is the SPC implementation roadmap to achieve the objective of analyzing data generated in learning environments with a view to decision making for the maintenance or progression of the effectiveness of available resources for autonomous learning?2. What methodology should be adopted to evaluate training processes in order to represent their performance and assist in decision-making?
2. Theoretical Background Ellen and Boguslaw (2003) affirm that conventional approaches to dealing with training, related to the professional environment, traditionally conceptualize this issue in order to segment the roles of individuals, trainers and managers.For the authors, this segmentation is problematic because it does not consider the contexts of each person involved in the training process and ignores the holistic and complex nature of the relationships involved.
Therefore, training models oriented towards the simple execution of a program, which consider everyone involved as passive agents in the process, are thus in question.The evaluation of the effectiveness of the teaching and learning resources has its relevance associated with the methods adopted by the trainer, who has control and can directly perceive the reactions of the learners in their training path.However, this aspect becomes critical in models that foresee the digitalization of many tasks and the consequent increase of the participants' autonomy.For the definition of an evaluation model of a digital teaching and learning environment based on virtual reality and augmented reality, one must consider that this format has specific characteristics and demands.
In the traditional teaching and learning model, the training programs are defined, planned and executed, so that participants are indiscriminately submitted to a process of reproducing a program in such a way that their action, therefore, has a passive character (Ellen & Boguslaw, 2003).In the digital model, participants play an active role because the environment, as described, requires exploratory action and foresees progressive levels of autonomy.Thus, the digital model does not deal with the mechanical dynamics of events, but rather with individual activity that refers to a "social mechanism", since the system needs to identify the behavior of individuals during their training journey and present relevant information to trainers and managers for decision-making.Rikku and Chakrabarty (2013) consider that one of the most important steps in the development of training programs in the workplace is the analysis of existing needs within the training process.The authors suggest the term Training Need Analysis (TNA) to refer to efforts to monitor continuously and permanently the requirements of training processes and therefore their performance.For this purpose, they propose a model based on three dimensions: Organizational analysis, task analysis, person analysis.In a similar approach to Rikku and Chakrabarty (2013), Bodily, Graham and Bush (2017) defend the use of learning analytics to support the engagement of participants to address three perspectives involving teaching and learning processes: didactic considerations, technological issues and concerns about interface design, i.e. the effectiveness of didactic resources in interactions with learners.
For the same purpose of Ellen and Boguslaw (2003), which is to consider the specific demands of each stakeholder, while being part of a process that fosters dynamic and complex relationships, Bodily et al. (2017) suggest the application of data analysis resources, based on the results of interactions in teaching and learning environments, to support decision making and maintain continuous improvement.

Training Evaluation
According to León-Medina (2017), "Social Mechanism" is a term used to refer to the analytical spirit present in different traditions of sociological studies.According to this author, the aim of studies using the concept of mechanism is to avoid the black boxes that exist in explanations based on statistics.The black boxes exist in the state of a system, that is, in the macro perspective, when the causal link between two phenomena, which occurs in the micro perspective, is absent from the explanation concerning them.The black boxes contain social mechanisms that demonstrate the high-level explanations, so if we are careful to make the mechanisms evident in our explanations, we avoid the existence of black boxes and guarantee the clarity of our constructions.A mechanism-based model provides the description of a system dynamic, i.e. the behavior of the causal chain of micro-level events that generate the macro-level behavior of the system.
León-Medina (2017) presents a telling example of the use of the mechanism-based approach that refers to the collective behavior of ants to decide between a longer and shorter path between the anthill and the food source.
According to the author, experiments indicate that when ants discover a food source, it goes to the nest and leaves a trail of pheromones along the way.Initially the ants that were in the anthill choose randomly among the various pheromone paths available and, as these trails evaporate, they realize which is the shortest path and thus the collective behavior of the ants tends to become more and more homogeneous.The mechanism-based approach, therefore, seeks the understanding of events from the micro perspective to bring out elements that promote the understanding of complex systems from a macro perspective.
A study using a similar approach was carried out by McEneaney (2016) to assess the effectiveness of evaluation based on simulation of hierarchical structures of learning objectives with instructional technologies.According to the author, instructional technologies are critically dependent on a systematic design of the training process and on hierarchically presented learning objectives.A simulation-based environment continuously calibrated by data generated by the system itself can cause two implications: (1) various learning pathways can be equally effective in terms of learning, and, (2) the interaction between the skills demonstrated by the trainees and the learning pathways can disadvantage the less capable trainees.As demonstrated in the study, from a statistical modeling of occurrences where mean and standard deviation are set within maximum and minimum limits it is possible to identify both the most efficient pathways and, like in the study of ants, the exceptions that should be treated with specific strategies.
Learning Analytics is the term used by Bodily et al. (2017) to define the process of selecting, collecting, analyzing and reporting data on learning activities and interactions to promote improvement in the teaching and learning process.The authors recommend that the data be classified in order to allow assessment parameters and provide analysis at different levels, according to the needs of the stakeholders, who in training processes may be managers, trainers, designers and trainees.Bodily et al. (2017) present the following possibilities of data nature presented according to the interests and needs of the stakeholders: • Administrators: Finance, enrollment, student outcomes, student engagement, satisfaction; • Designers: Averages, assessment, student engagement; • Instructors: Interaction data, assessment, student outcomes data, student engagement; Students: Student assessment data, compare to class, compare to students, student outcomes data.
Additionally, the authors recommend the presentation of evaluation reports on dashboards using scatter plots, network plots, histograms, polygraphs and others.Carlucci, Renna, Izzo and Schiuma (2019) proposes a framework to analyze adequately the quality of the teaching process in the light of the imprecision and uncertainties present in subjective assessments.The instrument integrates two methods, the u-control chart and the ABC analysis using fuzzy weights.By means of the control charts, students' assessments are analyzed to detect courses that are outside the control limits, and ABC analysis using fuzzy weights deals with the imprecision and uncertainty of students' assessments in order to provide a risk map of potential areas for improvement.In general, the authors present a management tool capable of indicating the need for short-term corrective measures, by means of the control charts, and point to areas that have potential for improvement in the long term.
The main reason why the authors advocate data analytics for the evaluation of training processes in place of the traditional self-reflection questionnaires answered by the students, according to them, is that in general the problems that appear in the surveys are related to infrastructure, materials or specific occurrences.The answers hardly help to solve problems in the training system as a whole.They sought support in literature and found that the evaluations of the trainees should no longer be seen as a review of their educational experiences, which undoubtedly serve as a reflection for the stakeholders, but are far from being material enough to indicate actions that promote effective improvement in the process.Thus, Carlucci et al. (2019) present a framework for quality improvement analysis, and prioritization of problems occurring in higher education courses.This framework proved to be effective and its decision flow is based in two dimensions, namely, problems and opportunities for improvement.These opportunities are classified as short and medium term for the occurrences whose causes can be mapped at the technical level, and as long-term for occurrences whose causes are at the level of the system as a whole, involving new strategies and approaches.
According to Mirzaei et al. (2016), Vicentin, Silva, Piccirillo, Bueno and Oprime (2018), John and Singhal (2019) and Prata, Chaves, Gomes & Passos (2020), SPC is an instrument made up of statistical procedures that aims to measure and analyze variations in processes.Together with diagnostic tools such as the Ishikawa diagram, cause and effect matrix, histograms, and others, SPC is able to identify problems with agility and reasonable assertiveness before the damage reaches a greater impact beyond those linked to its immediate cause.Although it is widely used in the goods industry, it is also an essential tool in the service sector where the concept of quality is not defined only by objective parameters, but mainly by the judgment of clients.Carlucci et al. (2019) claim that Statistical Process Control Charts are divided into two categories: variable and attribute control charts.According to the authors, the variable control charts are used to evaluate processes in which quality is defined by numerical factors such as weight, height, etc., and in cases where quality is defined by "conformity" or "nonconformity", the attribute control charts should be applied.Of these control charts, the most commonly used are p-control chart, np-control chart, u-control chart and c-control chart.
The control charts provided by SPC have wide application in production systems of both goods and services, besides being recognized as an effective resource for performance evaluation in process management, including in training processes for the evaluation of resources and teaching methods.Several authors have used SPC beyond its original usage, being the following ones, examples of such works published in the recent years: • Demonstration of the usefulness of SPC for monitoring paradata for assessing the quality of data obtained in surveys (Jin, Vandenplas & Loosveldt, 2019).
• Proposal of a mechanism for process improvement in product design with Internet of Things, statistical hypothesis testing and process capability indices chart applied in a bicycle manufacturer (Lin, Su, Chao, Hsieh & Tsai, 2016).
• Application of SPC in Professional Service (PS) companies (Khan, Kaviani, Galli & Ishtiaq, 2019).The literature review allowed the identification of significant experiences in the exploration and application of SPC beyond defect control in production lines in the manufacturing context.As can be seen, there are successful cases in the adoption of this instrument in circumstances that demand data analysis, but the experience of using SPC for the analysis of the effectiveness of educational resources is still a gap to be explored.Kenett et al. (2014) explain that variable control charts can be of many types, but the most common are X & R and I & MR.The first are used to identify quality problems in situations involving groups of individuals or lots that have been produced under the same conditions.The I & MR charts should be used for the evaluation of individual items, i.e. when their production involves individual conditions.This process control approach provides for the use of two cards: one to monitor the average of occurrences and other to monitor the variance.Both are complementary for problem diagnosis.If values outside the control limits begin to appear on cards IX, but the variance remains within expectations, this means that the process has changed and individuals have started to behave differently than they had been doing and therefore outside the control limits until these limits are reconfigured.The cause may be a problem, or a change in the process.In cases where the variance also becomes very large, it may be an indication that the process is not achieving the expected effectiveness due to an instability caused by a localized problem (Kenett et al., 2014).According to Mirzaei et al. (2016), SPC's graphical control tools are used in the services area to evaluate, for example, the level of consumer satisfaction, quality of service, statistical survey of quality and analysis of process results, benchmarking for evaluation of competitiveness and others.Therefore, the quantification of the evaluation starts with the identification of parameters that interfere in the perceived quality.They suggest that the implementation of SPC follows four steps:

SPC Specification
1. Definition of the process; 2. Diagnosis of the process; 3. Actions and measurements; 4. Definition of the control charts.
Abdul-Halim-Lim, Antony, He and Arshed (2017) explain that the classical application of the SPC is through the charts X (average) e R (range) which shows the average value of occurrences related to process quality plus one estimate of standard deviation of the quality characteristic under analysis, based on the Range value of samples.In this instrument, the Central Limit Theorem is basic for any of its applications.Durmusoglu (2018) and Carlucci et al. (2019) mention the instrument called Statistical Control Chart (SCC) composed of the variables that determine the limits of analysis, namely Upper Control Limit (UCL) and Lower Control Limit (LCL).It is important to note that the X/R control charts are indicated for the analysis of observation subgroups, that is, for each sample several occurrences are considered.This makes a lot of sense in industrial application where the objective is the identification in a production line of occurrences out of control limits, for example, a significant increase of defects, which could mean a malfunction in a machine.In these cases, the elements that make up the process have a behavior that obeys a specification and the trend is always a stability in the process.
In the case of a training process, we cannot consider that each learner will always have the same reaction before the learning objects and in the interaction with the scenarios.Although there is the expectation of a minimum condition that enables learning, in no way can a human individual be observed as a machine.Besides, as opposed to the typical application of SPC in industry, especially in large scale production processes, in the process of training, the amount of values measured or counted are quite small, which, in general, could make a definition of large samples or subgroups unfeasible.Therefore, the proposal is the application of charts for individuals, instead of using charts for samples.Moving Range charts, known as X/MR or I-MR, are indicated in this case.The I-MR chart is then composed of two charts, the I-chart where the values for each individual is plotted, along with the MR-chart, where the moving range between two successive values are plotted.These values are then compared to the limit values for each of such charts in order to evaluate if each individual value can be considered as having the expected value.The calculation of the limit lines, as well as the central line of each chart is based on the history values recorded as the trainees are submitted to the learning objects and tasks.According to Kenett et al. (2014), for the calculation of the limit lines, given k history values of the measured values, the i-th value denoted x i .The i-th value of the moving range is calculated as: (1) In the Individual graphics procedure, the central line is defined as the average value, estimated as follows (Kenett et al., 2014): (2) Control limits must be calculated for both I and MR charts.The control limits for the I chart are calculated as: (3) (4) where, (5) The control limits for the MR chart are calculated as: The value of E2, D3 and D4 depends on the sample size.Assuming that the chart for individuals (I-MR chart) is used, since the Moving Range is calculated based on two successive values, the sample size is always fixed at 2, and then, E2, D3 and D4 are constants, assuming the following values: E2 = 2,660; D3 = 0; D4 = 3,267.

Methodology
The research is performed as an exploratory case study informed by the literature review for the purpose of theoretical background and study of precedents.This case study was developed as a data research procedure and was integrated in a larger research project called Factory of the Future (FoF), which results from a partnership between University of Minho and the company Bosch Car Multimedia Portugal.The project goal is related to the implementation of a more effective training process for new employees, based in gamification, and augmented and virtual reality.
The exploratory case study approach was adopted in response to the context of the project, which required immersion in the application environment, analysis and diagnosis of the problem, and the development of a solution based on the theoretical background, guided by the previous experiences of the researchers and experts.From tests and simulations of the developed solution, the evaluation model of the training process presented in this article was developed.Therefore, the observation research procedure was adopted for data collection and subsequent process modeling.The time available for the research was a limitation that was partially overcome by using scenario simulation.Therefore, the results were obtained from supervised, expert-ratified simulations.
For the literature review, the knowledge bases JSTOR, Emerald Insight and Springer were used.Guided by the experience gained during the immersion process in the application environment and by previous experiences, we conducted the search for references using the following keywords: Adaptive learning, control chart, data analysis, Game Based Learning, individualized learning, industrial training, serious game, Work Based Learning, workplace learning, Statistical Process Control, statistical quality control, virtual reality and augmented reality.The articles were chosen initially according to the titles, then by the abstract and finally from the reading of the whole text.The process of collecting and selecting the articles was carried out collaboratively by the authors and supported by the Zotero software.

SPC Model for Evaluation of the Training Process
The proposed approach to analyze the quality of the training process through the SPC forces the use of analytical methods for the individual study of phenomena, given the characteristics of the object of study that correspond to the interactions between learners and learning resources arranged in a virtual environment, in an exploratory way and with a high level of autonomy.These resources can be learning objects, through which content is studied in varied and combined formats, such as texts, videos, quizzes, texts-quizzes, videos-quizzes, texts-videos-quizzes, etc., and scenarios that represent real-world situations and problems and impose challenges on participants.
In this way, one can analyze each learner as a unique universe, although it is a goal of the process to maintain stability within the limits of control.For this, one should always use two combined graphs for monitoring process performance, namely, the graph for monitoring scenario execution time and interaction with learning objects and the graph for monitoring the moving range which should identify very large variations between different individuals in scenario execution or interaction with learning objects.Therefore, process improvement guided by the SPC follows the flow represented in Figure 1.The system has the resources to perform continuous data collection on the interactions made by students with the learning objects, both those built to provide assimilation of content and instructions and those that are challenges to be overcome.The control charts monitor the interactions according to the time in which they are carried out and when the time is not sufficient to draw conclusions, it is possible to correlate the times with points obtained in quiz-type assessments and thus obtain more refined parameters.In problems, the variance is re-established in the control limits after being corrected for faults, in cases where there has been a change in the process, the control limits will be readjusted as the occurrences accumulate.An example of a problem is the performance of a student who has dyslexia when studying a five-page text, while an example of a change in the process is a trainer changing a learning object or altering it in such a way as to have an impact on interaction times and points.The application of SPC aims at monitoring the performance of three important factors: (1) learning objects, (2) learning tasks (which are dimensions that concern short and medium term for decision making and the solution efforts take place at technical and operational level), and, (3) expected quality of training (which is a long-term dimension for decision making and requires a way of prioritization because the needs for solution efforts are unclear and require further analysis and action).
It is important to mention that the learning objects can be of the following types: • Simple, such as a video, a text, a quiz, an animation, etc.
• Composed, as for example a virtual class composed of several simple objects, including a quiz or several quizzes.
• Scenario, which is a composite object with the difference that it is designed for the learner to solve a problem.In general, it corresponds to a virtual simulation of a situation that can be experienced in the real world.
The data structure has an entity for recording the tasks performed by learners.The tasks are carried out within the framework of the scenarios.This entity has an attribute to identify the points obtained in the accomplishment of the task and another attribute for the time employed.There is also an entity for the storage of data concerning the learning objects in which it is possible to obtain an average of the points achieved by the learners as they interact with them and with this attribute, it is possible to identify the average performance of all the learners for each learning object.It is important to note that this assessment is only valid for composite learning objects in which they comprise a quiz element through which learners' performance is inferred.

Analysis of Results
Learning objects are designed and created to suit the learning styles.For example, a video whose aim is to present the overview of a procedure should meet profiles of learners who find it easier to learn techniques presented by a contextualization narrative than by simply demonstrating them.It is therefore necessary to monitor the performance of learners as they interact with learning objects designed to meet their respective profiles.I-MR control charts should be used to identify learners' behavior when interacting with a learning object or implementing a scenario.In this way, these charts can be used to both assess learners' performance and evaluate teaching resources, i.e. learning objects and scenarios.On learners' performance, we will return to this question in the next section.
For the evaluation of learning objects and scenarios, the I-charts in Figures 3 shows a pattern of learners' behavior in their interaction with one of these resources.Each control chart is the representation of the recorded times of learners in interaction with a learning object or scenario, so that the time histories of all learners are used for plotting the Central Line (LC) and the lower and upper limits (UCL and LCL).This control chart can serve as parameters for trainers and managers to evaluate whether learning objects have been properly designed.It is expected that learning objects will be architected to suit the learning styles.It is a plausible hypothesis that the interaction with these artifacts takes place in a time range effective for assimilation.An interaction at a very low time or at a very high time may indicate that the learning object does not meet the trainer's objective and therefore should be reviewed.It is possible to observe how long a particular learning object is used by learners.In all occurrences except one the times have varied within the control limits, which means that the learning object under analysis is complying with the object that is expected from it.The recurrence of times outside the control limits can mean high level of obviousness, high level of difficulty or still can indicate deviations in the learning profile of the learner in question so that it needs to be redone.For example, if the learning profile informs that the learner has a textual profile and the learner does not perform well with learning objects intended for this characteristic then it is necessary to confirm if the style is still the same or if the learning object does not fit this profile.In any case, therefore, these control charts are instruments so that the trainer or the manager can carry out his investigations about problems in the process.
However, as the average and control limits are established by a history, they may not reflect the quality of the resources as if they are poorly done, everyone will have times consistent with that low quality.To solve this problem, we suggest using the I-chart in Figure 4 that demonstrates the ratio between times and points.
In fact, it is reasonable to assume that a correlation exists between the time a trainee dedicates to learn a subject using a learning object, like a video or a text, for example, and the trainee´s performance in a test related to that subject.A ratio between the time dedicated by a trainee to that learning object and the performance in the test could be a good indicator of the ability of that learning object to "aggregate" knowledge in the learning process.
Unlike the chart per time which is applicable in any kind of learning object, including scenarios, the chart of ratio between times and points will be possible only for analyzing scenarios and for analyzing learning objects that end with a quiz.An example about this last case is a learner that reads a PDF document and to finish this operation he/she must answer some questions to measure their understanding of the text.With this it is possible based on the history to draw a pattern and control limits that take into account the correlation of time and points.
Another point is that with the I-chart of correlation with point and time it is possible to identify learning objects or scenarios with bad quality.In fact, if the learners need to hold on to a learning object for a long time to achieve a good performance, an investigation may be necessary to verify if this factor is inherent to a possible complexity of the operation or if there are problems that need to be corrected.In this specific case, MR charts should be used for the observation of time variance and correlation times and points in the training process as a way to identify changes in the behavior of the process that deserves to be investigated.Variance is the measure by which one evaluates how far away a value is from the expected value.In a common situation, it is normal that sporadic cases of variation outside the limit of control occur.However, in situations where these occurrences became frequent and subsequent, they might indicate that an investigative action needs to be carried out because something happened in the process that made it have an atypical behavior, or the process changed in such a way that its behavior from that time on has changed, i.e. it became another process.In a training process, this can occur in cases where the learning objects or scenarios are revised, or some change occurred in the training path, causing a change in the learning outcomes.
With this instrument, the trainer can carry out a more detailed investigation in relation to these learning objects like to do more tests in controlled environments, simulations, to verify if they meet the objectives for which they were created.
The management implications of applying SPC for learning objects evaluation is equivalent to a shift in strategy from push to pull production.In the conventional training process, its design provides for the planning of a program to transmit content.Thus, the evaluation of its effectiveness is on the ability to reproduce that content.This approach is equivalent to the pushed production model, because it is based on an expectation of capacity.
The model presented in this paper evaluates the ability of learning objects to simulate the real-life environment and their effectiveness in promoting learning.The evaluation is based on data that is generated in the interaction of learners with this simulation environment, i.e., it is based on action data.Therefore, it corresponds to a pull model in which actions aimed at correcting the fit between prediction and execution are reduced to a minimum.
In this way, it is estimated that both the training time of an operator and the dedication time of a trainer to the process will decrease, as the operator performs fewer adjustment actions to correct gaps between the training environment and the working environment, and trainers spend less time on retraining procedures and other forms of rework.Examination of this hypothesis should be undertaken in the future, when sufficient data is available to make the necessary comparisons.

Final Remarks
The Factory of the Future (FoF) program, carried out in partnership between the University of Minho and Bosch Car Multimedia Portugal, is composed of several innovation projects, among them the design and implementation of a training and evaluation process with application of virtual and augmented reality for the training of new employees to work on the production line of automatic insertion.
The case in question corresponds to a training activity that is complex because its effectiveness is expected and necessary; the process has a high level of informality where evaluation occurs by means of subjective estimates.It is not clear how all the stakeholders understand the whole process and its relationship with the organization's strategic objectives.This is why the level of commitment of people to it is to the extent that immediate needs dictate.Faced with this scenario, a reconfiguration of the training process that would transform a conventional method from end to end to a method that is innovative in all senses would be a reckless leap.Thus, the following strategy was chosen: • Introduction of the company in the universe of investigations, discussions and initiatives guided by the Work-Based Learning (WBL) theme for the development of the perception of professional training as a critical factor for success in highly qualified environments and at the same time unique in terms of competence requirements.
• Adoption of Instructional Design to effectively enable a training process as it is a framework that defines dimensions, interdependent procedures, monitoring, roles and responsibilities.
• Application of Bloom Taxonomy to research and plan necessary competences in a way directed to the particularities of each trainee's profile.In addition, a Storytelling approach associated with Game-Based Learning (GBL) was adopted as a guideline for the development of content in the virtual reality environment.Therefore, the article can answer the initial questions as the follow: (1) What is the SPC implementation roadmap to achieve the objective of analyzing data generated in learning environments with a view to decision making for the maintenance or progression of the effectiveness of available resources for autonomous learning?In section 4, Figure 1, a roadmap for the implementation of SPC for the evaluation of training processes can be observed.In section 5 it is presented in detail the procedures for implementation.It is important to note that the control charts present the behavior of the process as learners interact with the content and scenario learning objects.These charts should be used for the identification of occurrences that present results beyond the limits of control, so that from the competencies of trainers, managers and other stakeholders, the causes can be assessed, whether of problems or changes in the process, and decisions can be made.Therefore, it is an instrument for monitoring the process and supporting decision-making and continuous improvement measures.
(2) What methodology should be adopted to evaluate training processes in order to represent the performance of the processes and assist in decision making?The control charts adopted allow the performance of learning objects and scenarios to be monitored as learners interact with them.Initially evaluation can be carried out on the basis of the correlation between points and times and when there is sufficient experience to consider the appropriate time for the appropriate achievement of points, the charts controlling time can be sufficient for analysis.Changes in the process or in the artifacts of the process may produce occurrences in the control charts which monitor the values as a function of the control means and in the variation, in order to stabilize when the new results become standard, or when there are no changes, the results indicate difficulties with the learners, either by specific and localized problems which must be investigated with each individual or problems in the identification of the learning profile, which in this case must be remade, since the learning objects and the scenarios are designed and selected by the augmented reality and virtual reality environment for application in order to meet particular skills.
The resources adopted guaranteed the capacity to monitor the process in critical factors for different stakeholders, as administrator, designers, instructors and learners, such as the performance of learning objects, the performance of the trainees and the capacity of each task to promote learning through challenges as advocated by GBL.With these resources, the training process has regulatory mechanisms to respond in an agile manner to both localized and larger problems.The study time is a limitation of the research presented in this article, which implies the analysis of data generated in simulations.The exploratory nature of the research in case study minimizes this problem because the simulations are performed in the application environment and ratified by process experts.
The work done also raises new research opportunities as the solution gains in maturity; among them, one should highlight the expansion of the system's autonomy in adjusting to the particularities of each individual in the learning process from data analysis and intervention when control limits are exceeded.

Figure 1 .
Figure 1.Process improvement flow with SPC

Figure 2 .
Figure 2. I-chart example for learning objects (scenario)

Figure 3 .
Figure 3. Ratio times and points for Learning Objects or ScenariosBoth charts are therefore supposed to complement each other.Through the correlation chart, it is possible to identify the time in which learning objects can provide good performance.After this indication it is possible to use charts which only times are plotted.An important complement for I-chart is the MR-Chart, illustrated in Figure5.Control charts for variables, such as X-R, X-s and I-MR charts, for example, always come in pairs.The first of the pair (X or I) tracks the value of the variable itself (with the mean X of a sample, or with the individual value I).The second in the pair (R, s or MR) measures the variance in some way (for samples, we use the R amplitude or the "s" standard deviation of the sample; for individual values, we measure MR, the variation between two successive I points).

Figure 4 .
Figure 4. MR-chart for learning objects