FOREAL: RoBERTa Model for Fake News Detection based on Emotions

Detecting false information in the form of fake news has become a bigger challenge than anticipated. There are multiple promising ways of approaching such a problem, ranging from source-based detection, linguistic feature extraction, and sentiment analysis of articles. While analyzing the sentiment of text has produced some promising results, this paper explores a rather more fine-grained strategy of classifying news as fake or real, based solely on the emotion profile of an article's title. A RoBERTa model was first trained to perform Emotion Classification, achieving test accuracy of about 90%. Six basic emotions were used for the task, based on the prominent psychologist Paul Ekman - fear, joy, anger, sadness, disgust and surprise. A seventh emotional category was also added to represent neutral text. Model performance was also validated by comparing classification results to other state-of-the-art models, developed by other groups. The model was then used to make inference on the emotion profile of news titles, returning a probability vector, which describes the emotion that the title conveys. Having the emotion probability vectors for each article's title, another Binary Random Forest classifier model was trained to evaluate news as either fake or real, based solely on their emotion profile. The model achieved up to 88% accuracy on the Kaggle Fake and Real News Dataset, showing there is a connection present between the emotion profile of news titles and if the article is fake or real.