FOREAL: RoBERTa Model for Fake News Detection based on Emotions

Vladislav Kolev*, Gerhard Weiss, Gerasimos Spanakis

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review


Detecting false information in the form of fake news has become a bigger challenge than anticipated. There are multiple promising ways of approaching such a problem, ranging from source-based detection, linguistic feature extraction, and sentiment analysis of articles. While analyzing the sentiment of text has produced some promising results, this paper explores a rather more fine-grained strategy of classifying news as fake or real, based solely on the emotion profile of an article's title. A RoBERTa model was first trained to perform Emotion Classification, achieving test accuracy of about 90%. Six basic emotions were used for the task, based on the prominent psychologist Paul Ekman - fear, joy, anger, sadness, disgust and surprise. A seventh emotional category was also added to represent neutral text. Model performance was also validated by comparing classification results to other state-of-the-art models, developed by other groups. The model was then used to make inference on the emotion profile of news titles, returning a probability vector, which describes the emotion that the title conveys. Having the emotion probability vectors for each article's title, another Binary Random Forest classifier model was trained to evaluate news as either fake or real, based solely on their emotion profile. The model achieved up to 88% accuracy on the Kaggle Fake and Real News Dataset, showing there is a connection present between the emotion profile of news titles and if the article is fake or real.

Original languageEnglish
Title of host publicationProceedings of the 14th International Conference on Agents and Artificial Intelligence (ICAART 2022)
EditorsAna Paula Rocha, Luc Steels, Jaap van den Herik
PublisherScitepress - Science And Technology Publications
Number of pages12
ISBN (Print)978-989-758-547-0
Publication statusPublished - Feb 2022
Event14th International Conference on Agents and Artificial Intelligence - Online, Setúbal, Portugal
Duration: 3 Feb 20225 Feb 2022
Conference number: 14


Conference14th International Conference on Agents and Artificial Intelligence
Abbreviated titleICAART 2022
Internet address


  • Emotion Analysis
  • Emotion Classification
  • Fake News
  • Natural Language Processing
  • RoBERTa
  • Sentiment Analysis
  • Transformers

Cite this