Detecting Linked Data quality issues via crowdsourcing: A DBpedia study

Maribel Acosta, Amrapali Zaveri, Elena Simperl, Dimitris Kontokostas, Fabian Flöck, Jens Lehmann

Research output: Contribution to journalArticleAcademicpeer-review

10 Citations (Web of Science)


In this paper we examine the use of crowdsourcing as a means to detect Linked Data quality problems that are difficult to uncover automatically. We base our approach on the analysis of the most common errors encountered in the DBpedia dataset, and a classification of these errors according to the extent to which they are likely to be amenable to crowdsourcing. We then propose and study different crowdsourcing approaches to identify these Linked Data quality issues, employing DBpedia as our use case: (i) a contest targeting the Linked Data expert community, and (ii) paid microtasks published on Amazon Mechanical Turk. We secondly focus on adapting the Find-Fix-Verify crowdsourcing pattern to exploit the strengths of experts and lay workers. By testing two distinct Find-Verify workflows (lay users only and experts verified by lay users) we reveal how to best combine different crowds' complementary aptitudes in Linked Data quality issue detection. Empirical results show that a combination of the two styles of crowdsourcing is likely to achieve more effective results than each of them used in isolation, and that human computation is a promising and affordable way to enhance the quality of DBpedia.
Original languageEnglish
Pages (from-to)303-335
Number of pages33
JournalSemantic web
Issue number3
Early online date2016
Publication statusPublished - 12 Apr 2018
Externally publishedYes


  • 2016 event_eswc group_aksw sys:relevantFor:infai sys:relevantFor:bis lehmann MOLE

Cite this