The assessment of quantitative problem-solving skills with 'none of the above'-items

In this contribution we concentrate oil the features of a particular item format: items having as the last option "none of the above" (NOTA items). There is considerable dispute oil the advisability of the usage of NOTA items in testing. Some authors come to the conclusion that NOTA items should be avoided, some come to neutral conclusions while others argue that NOTA items are optimal test items. In this article, we provide evidence to this discussion by conducting protocol analysis oil written statements of examinees while answering NOTA items. In our investigation, a test containing 30 multiple-choice items was administered from 169 university students, The results show that NOTA options appear to be more attractive than options with specified solutions in those cases where a problemsolver fails. Also, a relationship is found between the quality of (incorrect) problemsolving and the choice of NOTA items: the more qualitative the incorrect problemsolving process is, the more likely the student is to choose for NOTA items. Overall, our research supports the statement that 'the more confidence all examinee has in his worked solution, which is inconsistent with one of the specified solutions, the more eager he seems to choose "none of the above".