The EARLI21-contributions cover studies from the interdisciplinary Open Research Area project, FLoRA- Faciliating Self-Regulated Learning with Personalized Scaffolds on Students’ own Regulation Activities as part of two symposiums at the 19th Biennial European Association for Research on Learning and Instruction - EARLI 2021 conference. The project members of the FLoRA project, which investigates and improves trace data and subsequently develops and tests personalized scaffolds based on individual learning processes, hold their inaugural symposium.
Besides, Prof. Maria Bannert will also join the other members of the EARLI-Centre for Innovative Research (E-CIR), “Measuring and Supporting Student’s Self-Regulated Learning in Adaptive Educational Technologies“, in an expert panel discussion. In light of the COVID-19 pandemic, the EARLI 2021 conference will take place online with the conference theme – Education and Citizenship: learning and Instruction and the Shaping of Futures.
The contributions are as follows:
Project FLoRA EARLI Symposium: New Ways of Measuring, Analysing and Scaffolding Self-Regulated Learning
Organizer: Joep van der Graaf, Radboud University, Netherlands
Discussant: Philip Winne, Simon Fraser University, Canada
Date & Time: 26 Aug 2021, 15:45 - 16:45
Location: Session Room 5 - Session T
New ways of measuring and analysing Self-Regulated Learning (SRL) are rapidly emerging. This has important implications for theoretical frameworks of SRL, methodological approaches, and for current educational practices. The first aim of this symposium is to present and discuss new approaches to measurement and analysis of SRL. The second aim is to provide suggestions on the design of educational materials that provide additional insights into students’ learning processes and support their SRL. The presenters are a group of international researchers, who have a strong interest in learning analytics and/or SRL. The presentations are incrementally ordered, moving from measurement and analysis of SRL (1, 2) to learning outcomes (3) and ending with a digital learning tool (4). The four presentations revolve around the following main questions: a) How can multimodal data improve the granularity of measurement of SRL? b) How does SRL unfold in high versus low performing students? c) How do SRL activities relate to different learning outcomes? And d) How can we support students by visualising their SRL process? By addressing these questions, this symposium provides suggestions for theory and methodological development as well as educational practice.
Improving the granularity for the measurement of self-regulated learning using multi-channel data.
Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi-channel data. Though, to what extent can the addition of peripheral and eye-tracking data with navigational data improve the granularity of measurement of SRL are key questions that require further investigation. Hence, we conducted a study that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. Based on the measurement protocol proposed in this study, we were able to compare the process models of SRL of n=25 students across different data channels. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In addition, we also concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.
Understanding Self-Regulated Learning Processes through Process Mining.
Lyn Lim, Technical University of Munich, Germany; Maria Bannert, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia.
Self-regulated learning (SRL) is related to better learning outcomes and observation of SRL using think aloud data has been shown to be more insightful in determining SRL activities and predicting students’ learning achievements than self-reports. Educational process mining, moreover with think aloud data, enables a deeper understanding and a more fine-grained analysis of SRL processes. This study based on a pre-post design aimed to investigate how students differ in SRL learning processes and how this affects learning performance. There were 32 university students who participated in the study to learn about the theme, “Artificial Intelligence in Education”, and they had to write an essay in a digital learning environment within a 45-minute learning session while thinking aloud. The results showed that there is a significant learning gain in the knowledge test. Besides, the top performers showed more metacognitive and cognitive activities during learning. Furthermore, process mining using HeuristicMiner algorithm based on post hoc coded think aloud protocols examined differences in process structures of SRL for the high and low performers. In general, comparing resulting process mining models with prior process mining models will help to better generalize findings of prior research.
How Self-Regulated Learning Affects Different Learning Outcomes.
Joep van der Graaf, Radboud University, Netherlands; Lyn Lim, Technical University of Munich, Germany; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
Self-regulated learning (SRL) fosters transfer, but effects on other learning outcomes, such as domain knowledge are mixed. SRL potentially has a differential impact on learning outcomes with different characteristics, deep vs surface knowledge, and independent vs connected concepts. Therefore, we assessed how surface knowledge measured with a domain test (independent), and a concept map (connected) and deep knowledge measured with a transfer test (independent) and an essay (connected) are associated to SRL activities during learning and to prior metacognitive knowledge. Forty-five university students performed a 45-minute problem-solving task integrating three topics into a vision on future of education. SRL activities were measured using think aloud. Results revealed learning occurred. Surface knowledge measures, independent and connected concepts, were related to each other and associated with low cognitive activities during learning. Deep knowledge of independent concepts was associated with low cognitive processes, while deep knowledge of connected concepts was associated with a mixture of low and high cognitive processes. In addition, we found that prior metacognitive knowledge was associated with deep knowledge of independent concepts. To conclude, taking the level and structure of knowledge into account helps to specify effects of SRL processes on learning outcomes.
Visualising student’s learning strategies in online learning to support self-regulation.
Shaveen Singh, Monash University, Australia; Mladen Rakovic, Monash University, Australia, Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Maria Bannert, Technical University of Munich, Germany; Dragan Gasevic, Monash University, Australia.
Visualisations provide an effective way for learners to gain insight into their learning process which, in turn, may promote their self-regulated learning. Yet few learner-facing visualisations have been developed to support learners’ self-regulation. To this purpose, we propose a collection of personalised, theory-based and empirically driven visual interfaces. We harnessed trace data from multiple channels to generate clear and actionable recommendations for learners to improve their regulation. Guided by a quasi-experimental study in an university context (n=25), we investigated the student’s critical learning processes in SRL, such as, planning, content consumption, working on task, monitoring and evaluation. In the presentation, we describe the learning environment to collect data about those processes, and suggest visualizations that rely upon these data sources. In an ongoing study, we will prompt learners to engage in metacognitive monitoring of their learning using visualisations to support their regulation and learning.
Contribution to the EARLI Symposium: Leveraging SRL research into intelligent learning technologies
Organizer: Sanna Järvelä, University of Oulu, Finland; Inge Molenaar, Radboud University Nijmegen, Netherlands
Discussant: Arthur Graesser, University of Memphis, United States
Date & Time: 26 Aug 2021, 17:30 - 18:30
Location: Session Room 3 - Session U
Multimodal Data Analysis of Student’s own Regulation Activities to Advance Personalized Scaffolds.
Maria Bannert, Technical University of Munich, Germany; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia.
Education has been geared towards students’ ability to regulate their own learning within technology-enhanced learning environments. Prior research has shown that self-regulated learning (SRL) leads to better learning performance but students often experience difficulties to adequately self-regulate their learning. They can be supported by instructional scaffolds which consequently improve learning outcomes. However, scaffolds are often standardized and not personalized. Learning analytics and machine learning offer an approach to better understand SRL-processes during learning. Yet, current approaches often lack validity or require extensive analysis after the learning process. Hence, in the interdisciplinary Flora project the combination of both research expertise in the fields of self-regulated learning and learning analytics provide superior opportunities to develop and test more effective adaptive learning technologies. The general aim of this research is to advance instructional support given to students by improving unobtrusive data collection and machine learning techniques to gain better measurement and understanding of SRL-processes. So far, two empirical studies (Study 1: n=36; Study 2: n= 46 university students) about self-regulated learning in a digital learning environment were carried out and their main results will be discussed. Study 1 and 2 validate measurements of SRL combining multimodal data assessment and traditional think aloud measurements. This leads to improved measurements of SRL during learning and hence can be used to design personalized scaffolds based on individual SRL processes.
Panel Discussion: Multimodal Measurement of SRL in Advanced Learning Technologies: 5 years of pioneering research.
Inge Molenaar, Radboud University, Netherlands; Roger Azevedo, University of Central Florida, United States; Sanna Järvelä, University of Oulu, Finland; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany.
Date & Time: 23rd August, 10:45 - 11:45
The E-CIR “Measuring and supporting students' self-regulated learning in adaptive educational technologies” has been working on this theme for 5 years. In this panel, we like to discuss the progress and the development of the field over the last 5 years. In a concerted interdisciplinary dialogue, we have combined psychology, educational sciences with learning analytics and artificial intelligence in our research efforts to further develop methodologies to measure cognition, metacognition, affect and emotions during learning. Where our focus initially was on the measurement of these SRL processes, it slowly moved to incorporating new measurement into new forms of support for SRL during learning. In the panel, we will highlight lessons learned as described in the two special issues, discuss future endeavors and emphasis the importance of a new type of SRL theory as proposed by Peter Reimann.