Crowdsourcing Linked Data quality assessment

Maribel Acosta, Amrapali Zaveri, Elena Simperl, Dimitris Kontokostas, Sören Auer, Jens Lehmann

Research output: Chapter in Book/Report/Conference proceedingChapterAcademic


In this paper we look into the use of crowdsourcing as a means to handle Linked Data quality problems that are challenging to be solved automatically. We analyzed the most common errors encountered in Linked Data sources and classified them according to the extent to which they are likely to be amenable to a specific crowdsourcing approach. Based on this analysis, we implemented and compared two quality assessment methods for Linked Data that leverage the wisdom of the crowds in different ways: (i) a contest format targeting an expert crowd of researchers and Linked Data enthusiasts; and (ii) paid microtasks published on Amazon Mechanical Turk. We evaluated the two methods empirically in terms of their capacity to spot quality issues in DBpedia and investigated how the contributions of the two crowds could be optimally integrated into Linked Data curation processes. The results showed that the two styles of crowdsourcing are complementary, and that crowdsourcing-enabled quality assessment is a promising and affordable way to enhance the quality of Linked Data sets.
Original languageEnglish
Title of host publication12th International Semantic Web Conference, 21-25 October 2013, Sydney, Australia
Number of pages17
ISBN (Electronic)978-3-642-41338-4
Publication statusPublished - 2013
Externally publishedYes

Publication series

SeriesLecture Notes in Computer Science


  • zaveri auer lehmann kontokostas group_aksw sys:relevantFor:infai sys:relevantFor:bis sys:relevantFor:lod2 lod2page 2013 event_ISWC dbpediadqcrowd sys:relevantFor:geoknow topic_Crowdsourcing
  • topic_QualityAnalysis dataquamole MOLE

Cite this