Formative Assessment Framework Proposal for Transversal Competencies : Application to Analysis and Problem-Solving Competence

Purpose: In the last years, there is an increasing interest in the manner that transversal competences (TC) are introduced in the curricula. Transversal competences are generic and relevant skills that students have to develop through the several stages of the educational degrees. This paper analyses TCs in the context of the learning process of undergraduate and postgraduate courses. The main aim of this paper is to propose a framework to improve results. The framework facilities the student's training and one of the important pieces is undoubtedly that he has constant feedback from his assessments that allowing to improve the learning. An applying in the analysis and problem solving competence in the context of Master Degree in Advanced Engineering Production, Logistics and Supply Chain at the UPV is carried out. Design/methodology/approach: The work is the result of several years of professional experience in the application of the concept of transversal competence in the UPV with undergraduate and graduate students. As a result of this work and various educational innovation projects, a team of experts has been created, which has been discussing some aspects relevant to the improvement of the teaching-learning process. One of these areas of work has been in relation to the integration of various proposals on the application and deployment of transversal competences. With respect to this work, a conceptual proposal is proposed that has subsequently been empirically validated through the analysis of the results of several groups of students in a degree. Findings: The main result that is offered in the work is a framework that allows identifying the elements that are part of the learning process in the area of transversal competences. Likewise, the different items that are part of the framework are linked to the student's life cycle, and a temporal scope is established for their deployment. Practical implications: One of the most noteworthy practical implications is that the proposed framework includes a tool that allows a clear measurement of the student's evolution throughout his / her formative life cycle. In this way the student has a more consistent and robust vision of his / her training and the academic directors of the titles can have a vision of the impact of the decisions on the learning processes. Originality/value: The analysis of transversal competences is usually presented in the context of a subject. In this paper we propose an approach to cross-curricular competences but in the scope of the student's complete life cycle. The consideration of the entire formative process as well as the identification of the relevant elements that are part of this process are the most original aspects of the work.


Introduction and Literature Review
The Universitat Politècnica de València (UPV) introduced in the curricula of the new undergraduate and graduate degrees not only the assessment of specific competences but also the assessment of the generic ones.Thirteen generic or transversal competences (TC) were defined and introduced within all the curricula (Dimensiones Competenciales UPV, 2014): Understanding and integration (TC1), Application practical thinking (TC-2), Analysis and problem solving (TC-3), Innovation, creativity and entrepreneurship (TC-4), Project design (TC-5), Teamwork and leadership (TC-6), Professional and ethical responsibility (TC-7), Effective communication (TC-8), Critical thinking (TC-9), Knowledge of contemporary issues (TC-10), Continuous learning (TC-11), Planning and time management (TC-12), and Instrumental specific (TC-13).
In 2012, after the completion of the ABET (Accreditation Board for Engineering and Technology) accreditation process of four degrees at the UPV, it was questioned that UPV had defined and implemented systematic procedures to assess the degree of achievement of transversal competences.From this accreditation process, it was concluded that it was necessary to define a general procedure to evaluate the progress and certify the acquisition of transversal competences by students.This issue is not completed yet, and instructors and Universities are developing new approaches to face it such as the works developed by Villa and Poblete (2007), Blanco-Fernández, Learreta-Ramos, Alba-Ferré, Asensio-Castañeda, Blanco-Archilla, Bonsón-Aventín et al. (2009), García-García, Terrón-López and Blanco-Archilla (2009), Rodríguez-Gómez, (2009) and Ibarra-Sáiz (2010).Most of the transversal competences were not explicitly defined in the previous degrees and their development and assessment is new for instructors and professors who are specialised in the assessment of specific competences.Transversal competences are to be assessed within the specific activities developed through the subjects/courses.For that purpose, the UPV has designed an institutional project called "UPV transversal competences" in order to guide the general implementation of the generic competencies in the different degrees.
On the other hand, there is a distinction between two types of complementary assessments (Scriven, 1967;Bloom, Hasting & Madaus, 1971): summative and formative assessment.Traditional assessment usually relies on summative or final assessment with the main purpose of providing a qualification at the end of a period of time (quarter, semester, etc.) (Morales, 2000).However, when the focus is to improve the learning process, the final and summative activities should be complemented with formative and continuous assessment activities.There are several advantages of introducing formative assessment in the learning-teaching process.First, performing continuous learning activities provide feedback to students about their achievements and difficulties as well as motivate them to the achievement of the next goals (de Miguel-Díaz, 2006).Second, the continuous and formative assessment provides feedback not only to students but also to the instructor allowing reorienting the learningteaching process easily and efficiently (Huhta, 2010).Third, formative assessment allows improving student's awareness of how they learn (Shepard, 2005).
The main purpose of this paper is to review how transversal competences could be achieved in different skill levels during the whole life of students and propose a framework in order to develop a teaching and learning process integrating a formative assessment.This paper is structured as follows.In the second section a review of transversal competence along the life of the student is analysed, in the third section a framework for designing and implementing transversal competences is presented, in the fourth section previous concepts are applied to TC-3 competence.Finally, conclusions are exposed.

Approach to Teaching-Learning Life Cycle Concept in Transversal Competences
A transversal competence, like any other competence that we want to deal with an individual, relies on a set of natural capacities that must be strengthened, modelled and combined until reaching the desired ability.The authors of this work share the idea that one can only speak of a competence if one can measure the degree of skill that the individual achieves using it.From this brief reflection the hypothesis is established that all competences can be approached as a teaching-learning process that must be measured with one or several key performance indicators (KPI).
All the processes must have associated one or several objectives, which must be established before the KPIs, being that these are derived from the objectives.Normally, the objectives are multiple and are grouped by dimensions, so that one dimension is linked to several aspect of the same skill.It is proposed to approach competences as a cyclical process consisting of 4 basic steps, as shown in Figure 2. It should include: Definition of objectives of each dimension and identification of the corresponding KPIs, design of the didactic plan, deployment of didactic activities and, finally, analysis of the results and improvement of the process.It is considered as a cyclical process because it will be repeated in each of the 3 cycles in which it is used.The only activity that differs is the one corresponding to the definition of the dimensions associated with the competence that is only performed at the beginning of cycle 1.
It is important to clarify that the analysis phase refers to the process in its entirety and not to the punctual result that can be obtained in a given individual.This activity requires sufficient time and data, as well as a teamwork in which the owners of the process and its most significant participants should participate.

TELETCO: A Framework for Designing and Implementing Transversal Competences
The TEaching-LEarning process in Transversal COmpetences (TELETCO) is proposed in this paragraph.The proposal aims to facilitate the stages of the life cycle Figure 2. As shown in Figure 3, the proposal is based on the existence of a set of prescribing organisms that are those that by standards or tendencies in good practices propose transversal competences and define their dimensions and levels in a general way.On the other hand, the Academic Responsible of the Course (RAC) should be able to interpret and qualify such bidders to adapt the title to which they will apply.The RAC must specify and adjust the objectives of the different levels according to the cycles covered by the course.
Given that the teaching-learning process is developed in the subjects, the subjects responsible could assume the role of representatives of the operational level, and in collaboration with the RAC should define the aspects related to the design, the deployment and the analysis of transversal competences.
The aspects related to the design should be based on a Teaching-Learning Plan that allows the student to be trained in the selected competence throughout the cycle with a global vision, complete and shared by the subjects that are most suitable.The plan of activities should be included, each of which would have its focus, levels of achievement and time of dedication, and should also differentiate between teacher-supported activities and homework activities.
The aspects related to the deployment can be flexible and allow the teacher to develop the teaching-learning plan within of the subjects.This flexibility can only be achieved if there is a precise knowledge of all the teachers regarding the objectives of the competence and the criteria to evaluate it.That is why a Unification Plan is proposed that will allow us to assess skills in the same way in any subject.
Finally, the aspects related to the analysis and improvement are supported by a Performance Measurement System that allows to define the measurable ones, to obtain the data, to facilitate its understanding and to start adjustment mechanisms if they were necessary.

Applying TELETCO to Analysis and Problem-Solving Competence in Master Degree at UPV
All the transversal competences of the UPV have been designed to be developed in 3 stages where the basic level is reached in year 1 and 2 of the degree, the means in the year 3-4 of the degree and the advanced during the studies of Master or doctorate.According to the UPV's catalogue of transversal competences, the ability to analyse and solve problems is referred to as CT-3. Figure 4 shows the 4 dimensions of the CT-3 where each has 3 levels associated with cycles 1, 2 and 3.According to the TELETCO framework, the UPV would play the role of prescribing organism, and these would be the general objectives to be achieved.
In this case, the development of the competence in the Master's Degree in Advanced Engineering of Production, Logistics and Supply Chain (MUIAPLCS) with 60 ECTS credits and duration of an academic course is addressed.The Responsible Academic of the Course should adapt the last level of the 4 dimensions of Figure 4 to the scope of the master.By way of example, the dimension related to "The problem" could be transformed into "Consideration of causes and effects in the framework of the supply chain".It could be considered the result of the tuning of objectives proposed in the frame.In this case this task is reviewed every 5 years or in case of redefinition of transversal competences by the UPV.
The MUIAPLCS select 2 subjects to each competence, corresponding one to the first semester and another to the second.The objective is to be able to observe the evolution of the student throughout the course.This assignment is reviewed each new academic year.The two professors responsible for the CT-3 define a training plan that allows the evaluation of the competence and provides information to the student about its evolution and the teacher about the training of the student.The plan follows the guidelines indicated in Figure 5, it is unique for both subjects and in its first semester it is more oriented to the diagnosis and diffusion of the relevant criteria of the competence, and in the second is focused on the collection of evidence and the evaluation.Another feature of the plan is that in semester A the work of the student is mostly in the classroom and in the semester B is fundamentally autonomous.
The unification of criteria proposed in TELETCO is achieved through the use of a common rubric in relation to CT-3 for all the activities programmed in the teaching-learning plan as proposed by Gómez-Gasquet, Verdecho, Rodríguez-Rodríguez, Alfaro-Saiz (2016).These rubrics are reviewed annually to introduce the lessons learned during the course.In this case it addresses the following 4 dimensions: • Analyses the causes and effects of problems from a global long-term approach.
• Applies advanced search criteria information for troubleshooting and evaluates the quality of information.
• Organized in a systematic way to work decision.
• Evaluates possible solutions as viable scientific and technical difficulty of implementation.
The last aspect proposed in TELETCO is the use of a set of KPIs that shared by both subjects is visible by the student during the entire course.In a first approximation can be done using a simple excel sheet online.Beyond the tool the important thing is students can visualize from the first moment the KPIs that contribute to reach each one of the dimensions of the competences and can associate through scoreborad its formative actions to the said KPIs.

Conclusions
This paper has allowed to contextualize the process of development and acquisition of transversal competences and locate its development beyond the scope of a subject.In addition, a framework has been established, which, linked to the life-cycle concept, allows us to consider the elements necessary to achieve the objectives in the acquisition of the selected transversal competence.All of this has been applied to the transversal competence 3 related to the analysis and resolution of problems.

Figure 1 .
Figure 1.Representation of competence dimensions

Figure 2 .
Figure 2. Life cycle of transversal competence at UPV

Figure 4 .
Figure 4. Dimensions by levels of CT-3 at UPV

Figure 5 .
Figure 5. Characteristics of Teaching-Learning Plan