Nothing about it all, without it all: on evaluation as an agnostic practice

Description

In this presentation I will discuss an sociocultural intervention approach: video-reflexive ethnography (VRE). VRE facilitates interdisciplinary dialogues by filming everyday practices, editing footage, and using clips for co-analysis in reflexive meetings with the involved professionals aiming for practice improvement. Using the application of VRE in a Dutch hospital as a case study, allows me to discuss some challenges we face while evaluating interventions.
First, impact studies assume a clear-defined intervention-instrument and a researcher in full control in a stabile context. However, the complexity at hand does not permit such levels of control and stability. Moreover, a sociocultural intervention like VRE does not allow a cleardefined boundary of who and what belongs to the intervention-instrument at what moment in time. Who then should be considered as the one that intervenes? To whom to grant the successes and failures? When is an intervention actually done…if ever?
Second, it is hard to define what is affected and what stays out of reach? Especially its
relational effects, like re-appreciation, re-awareness and issues of professional identity, are less observable. In other words, the context of intervention, the instrument itself as well as its sphere of influence lacks the clarity that is assumed by mainstream evaluation approaches. How then studying the impact of VRE?
Third, impact studies have potential to disrupt relationships that have been carefully
developed over time. As a participatory method VRE frames participants as co-researchers on an equal basis. However, evaluation, as I will show, can unintentionally disentangle the engaged unity between researcher and participants and create a distance. How to compress this distance and align with the indigenous paradigm of ‘nothing about us, without us’ (Chatterji’s, 2001). How can I deal with these challenges when evaluating the impact of my VRE project? Iedema et al. (2019) refer to Donna Mertens (2009) ‘transformative paradigm’ in order to deal with the above challenges. In her paradigm participants feature as co-researchers who are transformative and act of co-evaluators of impact. In addition, Iedema and co-authors provide a typology of impacts that allows to include also ‘more cognitive-mental/ less observable’ impacts. For them, the evaluation of sociocultural interventions like VRE is an agnostic practice. What then is the added value of this agnostic dimension for the evaluation of my own project. An answer to this question and a more detailed analysis of the challenges we have to deal with, will be the focus of my presentation.

Period	10 Oct 2019 → 11 Oct 2019
Event title	(Re)assessing impact: A conference about interventions, experiments and entanglements
Event type	Workshop
Location	Oxelösund, SwedenShow on map
Degree of Recognition	International