Valid Prediction Intervals for Course Grades with Conformal Prediction

Raphaël Morsomme, Evgueni Smirnov

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

Conformal prediction is a statistical-learning framework that complements predictions with a reliable measure of confidence [22]. It allows the construction of prediction intervals for any type of regression models under the assumption that the data are exchangeable. Conformal prediction intervals have provably valid frequentist coverage for finite data; i.e. they contain the true value of the response variable for any test data with probability at least 1 - ? for any significance level ?.Due to the validity of its prediction intervals, we argue that the conformal-prediction framework has to be used for course-grade prediction. In this paper, we experiment with two conformal predictors: Inductive Conformal Machines (ICMs) [11] and Cross-Conformal Machines (CCMs) [21]. They are compared in terms of informational efficiency (width and stability of prediction intervals) and computational efficiency on course-grade prediction tasks from a liberal education program of Maastricht University, the Netherlands. We show that ICMs enjoy substantial computational benefits while CCMs has a better informational efficiency. Moreover, CCMs extend the applicability of the conformal-prediction framework to course-grade prediction tasks for data sets of as few as 30 instances.

Original languageEnglish
Title of host publicationProceedings of the 19th IEEE International Conference on Machine Learning and Applications
Subtitle of host publicationICMLA
EditorsM. Arif Wani, Feng Luo, Xiaolin Li, Dejing Dou, Francesco Bonchi
PublisherIEEE
Pages936-941
ISBN (Electronic)9781728184708
DOIs
Publication statusPublished - Dec 2020
Event19th IEEE International Conference on Machine Learning and Applications - Online, Miami, United States
Duration: 14 Dec 202017 Dec 2020
Conference number: 19
https://www.icmla-conference.org/icmla20/

Conference

Conference19th IEEE International Conference on Machine Learning and Applications
Abbreviated titleIEEE ICMLA2020
Country/TerritoryUnited States
CityMiami
Period14/12/2017/12/20
Internet address

Cite this