Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study

Dirk T. Tempelaar, Bart Rienties, Bas Giesbers

Research output: Chapter in Book/Report/Conference proceedingChapterAcademic

Abstract

In this empirical contribution, a follow-up study of previous research [1], we focus on the issues of stability and sensitivity of Learning Analytics based prediction models. Do predictions models stay intact, when the instructional context is repeated in a new cohort of students, and do predictions models indeed change, when relevant aspects of the instructional context are adapted? Applying Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics combined with formative assessments and learning management systems, we compare two cohorts of a large module introducing mathematics and statistics. Both modules were based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials, and have similar instructional design, except for an intervention into the design of quizzes administered in the module. We analyse bivariate and multivariate relationships of module performance and track and disposition data to provide evidence of both stability and sensitivity of prediction models.
Original languageEnglish
Title of host publicationCommunications in Computer and Information Science, Vol. 583
Subtitle of host publicationComputer Supported Education
EditorsS Zvacek, M. T. Restivo, J. Uhomoibhi, M. Helfert
PublisherSpringer Verlag
Pages256-273
VolumeCommunications in Computer and Information Science
Edition583
DOIs
Publication statusPublished - 11 Feb 2016

Publication series

SeriesComputer Supported Education
Volume583
ISSN1865-0929

Cite this

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2016). Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study. In S. Zvacek, M. T. Restivo, J. Uhomoibhi, & M. Helfert (Eds.), Communications in Computer and Information Science, Vol. 583: Computer Supported Education (583 ed., Vol. Communications in Computer and Information Science, pp. 256-273). [Chapter 15] Springer Verlag. Computer Supported Education, Vol.. 583 https://doi.org/10.1007/978-3-319-29585-5_15
Tempelaar, Dirk T. ; Rienties, Bart ; Giesbers, Bas. / Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study. Communications in Computer and Information Science, Vol. 583: Computer Supported Education. editor / S Zvacek ; M. T. Restivo ; J. Uhomoibhi ; M. Helfert. Vol. Communications in Computer and Information Science 583. ed. Springer Verlag, 2016. pp. 256-273 (Computer Supported Education, Vol. 583).
@inbook{c6b617d9bdda4b0391dc057f1ba73155,
title = "Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study",
abstract = "In this empirical contribution, a follow-up study of previous research [1], we focus on the issues of stability and sensitivity of Learning Analytics based prediction models. Do predictions models stay intact, when the instructional context is repeated in a new cohort of students, and do predictions models indeed change, when relevant aspects of the instructional context are adapted? Applying Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics combined with formative assessments and learning management systems, we compare two cohorts of a large module introducing mathematics and statistics. Both modules were based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials, and have similar instructional design, except for an intervention into the design of quizzes administered in the module. We analyse bivariate and multivariate relationships of module performance and track and disposition data to provide evidence of both stability and sensitivity of prediction models.",
author = "Tempelaar, {Dirk T.} and Bart Rienties and Bas Giesbers",
note = "Data source : use of self-collected survey data only; no data-base data.",
year = "2016",
month = "2",
day = "11",
doi = "10.1007/978-3-319-29585-5_15",
language = "English",
volume = "Communications in Computer and Information Science",
series = "Computer Supported Education",
pages = "256--273",
editor = "S Zvacek and Restivo, {M. T.} and J. Uhomoibhi and M. Helfert",
booktitle = "Communications in Computer and Information Science, Vol. 583",
publisher = "Springer Verlag",
address = "Germany",
edition = "583",

}

Tempelaar, DT, Rienties, B & Giesbers, B 2016, Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study. in S Zvacek, MT Restivo, J Uhomoibhi & M Helfert (eds), Communications in Computer and Information Science, Vol. 583: Computer Supported Education. 583 edn, vol. Communications in Computer and Information Science, Chapter 15, Springer Verlag, Computer Supported Education, vol. 583, pp. 256-273. https://doi.org/10.1007/978-3-319-29585-5_15

Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study. / Tempelaar, Dirk T.; Rienties, Bart; Giesbers, Bas.

Communications in Computer and Information Science, Vol. 583: Computer Supported Education. ed. / S Zvacek; M. T. Restivo; J. Uhomoibhi; M. Helfert. Vol. Communications in Computer and Information Science 583. ed. Springer Verlag, 2016. p. 256-273 Chapter 15 (Computer Supported Education, Vol. 583).

Research output: Chapter in Book/Report/Conference proceedingChapterAcademic

TY - CHAP

T1 - Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study

AU - Tempelaar, Dirk T.

AU - Rienties, Bart

AU - Giesbers, Bas

N1 - Data source : use of self-collected survey data only; no data-base data.

PY - 2016/2/11

Y1 - 2016/2/11

N2 - In this empirical contribution, a follow-up study of previous research [1], we focus on the issues of stability and sensitivity of Learning Analytics based prediction models. Do predictions models stay intact, when the instructional context is repeated in a new cohort of students, and do predictions models indeed change, when relevant aspects of the instructional context are adapted? Applying Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics combined with formative assessments and learning management systems, we compare two cohorts of a large module introducing mathematics and statistics. Both modules were based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials, and have similar instructional design, except for an intervention into the design of quizzes administered in the module. We analyse bivariate and multivariate relationships of module performance and track and disposition data to provide evidence of both stability and sensitivity of prediction models.

AB - In this empirical contribution, a follow-up study of previous research [1], we focus on the issues of stability and sensitivity of Learning Analytics based prediction models. Do predictions models stay intact, when the instructional context is repeated in a new cohort of students, and do predictions models indeed change, when relevant aspects of the instructional context are adapted? Applying Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics combined with formative assessments and learning management systems, we compare two cohorts of a large module introducing mathematics and statistics. Both modules were based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials, and have similar instructional design, except for an intervention into the design of quizzes administered in the module. We analyse bivariate and multivariate relationships of module performance and track and disposition data to provide evidence of both stability and sensitivity of prediction models.

U2 - 10.1007/978-3-319-29585-5_15

DO - 10.1007/978-3-319-29585-5_15

M3 - Chapter

VL - Communications in Computer and Information Science

T3 - Computer Supported Education

SP - 256

EP - 273

BT - Communications in Computer and Information Science, Vol. 583

A2 - Zvacek, S

A2 - Restivo, M. T.

A2 - Uhomoibhi, J.

A2 - Helfert, M.

PB - Springer Verlag

ER -

Tempelaar DT, Rienties B, Giesbers B. Verifying the Stability and Sensitivity of Learning Analytics Based Prediction Models: An Extended Case Study. In Zvacek S, Restivo MT, Uhomoibhi J, Helfert M, editors, Communications in Computer and Information Science, Vol. 583: Computer Supported Education. 583 ed. Vol. Communications in Computer and Information Science. Springer Verlag. 2016. p. 256-273. Chapter 15. (Computer Supported Education, Vol. 583). https://doi.org/10.1007/978-3-319-29585-5_15