Program evaluation B: Evaluation, big data, and artificial intelligence: two sides of one coin

This chapter is about evaluation, big data, and artificial intelligence. It suggest that despite the rapid digitization and dataification of society since the early 2000’s, evaluations that use Big Data and Artificial Intelligence (BD & AI) as part of their methodology have till recently been an underdeveloped field of interest. This chapter addresses the following questions: which role(s) can BD & AI) play when doing evaluations? why is evaluating (the roles of) BD & AI in policy making and policy implementation important? when evaluating BD & AI, what are (some of) the roles evaluation then can play? The chapter starts with showing the gap between digitization and dataification of society and the limited attention paid to these developments by the evaluation community. Only recently this situation has changed (somewhat). Also a few central concepts are discussed. In section 3 five roles that BD & AI (can) play(s) in program and policy evaluation are described and discussed. The following two sections deal with the other side of the BD & AI ‘coin’: not what BD & AI can contribute to the evaluation (profession), but why evaluations have to address (the roles of) BD & AI in policy making and policy implementation and what that work, if and when done, includes. Working with BD & AI goes hand in hand with (new) challenges for evaluators regarding theory and methodology. They are discussed in section 7. One challenge concerns the role of theories in policy and program evaluation, the other is how to combine (quasi-) experimental evaluations with BD & AI. Finally, section 8 draws some conclusions.