Document Type
Theses, Ph.D
Rights
Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence
Disciplines
2. ENGINEERING AND TECHNOLOGY
Abstract
Metacognition entails the conscious evaluation and control of one's cognitive processes. This meta-level control of cognitive process is not essential for all activities, but in the domain of problem solving and the development of new expertise, conscious control of mental functioning is essential to success. Previous studies have shown a relationship between metacognitive knowledge & skills and student self-regulated learning, self-efficacy and more generally, with success in academic and non-academic endeavours; they represent critical skills for an aspiring engineer to possess for their future employability. Metacognition can be stimulated by allowing students to engage and reflect on the problem-solving process. Studies in STEM education focus almost entirely on the use of technical problems for the source of this stimulation. The drawback of this approach is that these problems generally require prior knowledge of physics or mathematics for the students to engage in the process. Recent research utilising naturalistic observations of students’ behaviour while they were engaged in technical problem solving found that metacognitive knowledge and skills can be categorised into discrete metacognitive behaviours. Specifically, metacognitive behaviour can be measured through analysis of students’ discourse with one another as they engage in the problem-solving process.
This research utilised a sequential mixed methods design, which contained two strands – the first sought to develop a Situational Judgment Test (SJT) while the second strand sought to utilise the SJT as a stimulus of metacognitive behaviour. An SJT was developed, evaluated by fifty-three engineering professionals in eleven expert panels and rolled out to three hundred and third four final year and masters level engineering students at TU Dublin and KU Leuven, who took the SJT as a test. The SJT items were then delivered to a further fifty-five first year engineering students at TU Dublin, this time in groups, for them to choose responses and discuss them with their peers. The items which stimulated metacognitive behaviour amongst these students were identified using the Naturalistic Observations of Metacognition in Engineering students (NOME) protocol. The resulting items were provided to a group of eight first year engineering students and the NOME protocol was re-applied to evaluate the efficacy of the new metacognitive learning resource in stimulating metacognitive behaviour.
The development of a means of stimulating metacognitive behaviour that was not conditional on students’ having prior knowledge of physics and mathematics or a reliance on inventory style assessment allowed iii for a better-quality assessment of a students’ metacognitive knowledge and skills. Allowing students to apply their metacognitive knowledge and skills in groups permitted students to construct tools of higher mental functioning though peer dialogue, using an SJT in the stimulation of this dialogue had pedagogical merit, as particular SJT items proved highly effective in eliciting the use of metacognitive skills.
This research work aims to add to engineering education scholarship in three ways. Firstly, to provide an engineering specific SJT to enable educators to identify areas of relative strength and weakness in students’ professional judgements in order to better prepare them for their future careers. Secondly, to use the insights and resources generated from the development and evaluation of the SJT to develop a resource for engineering educators to stimulate students’ metacognitive behaviour that does not rely on a students’ prior knowledge of physics and mathematics, in order to provide them with the skills to self-regulate their learning. Thirdly, this research provides fresh insights into how engineering student’s exhibit metacognitive behaviours when working in groups, adding to an existing body of literature about how students exhibit these behaviours during the problem-solving process.
DOI
https://doi.org/10.21427/2dvm-fb16
Recommended Citation
Carthy, D. (2021). Development of a Situational Judgement Test and an assessment of its efficacy as a stimulus of metacognitive behaviour in engineering students. Technological University Dublin. DOI: 10.21427/2DVM-FB16
Publication Details
Submitted in partial fulfilment of the requirements for the award of PhD, Technological University Dublin, 2021.