Low-Resource Deontic Modality Classification in EU Legislation

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

In law, it is important to distinguish between obligations, permissions, prohibitions, rights, and powers. These categories are called deontic modalities. This paper evaluates the performance of two deontic modality classification models, LEGAL-BERT and a Fusion model, in a low-resource setting. To create a generalized dataset for multi-class classification, we extracted random provisions from European Union (EU) legislation. By fine-tuning previously researched and published models, we evaluate their performance on our dataset against fusion models designed for low-resource text classification. We incorporate focal loss as an alternative for cross-entropy to tackle issues of class imbalance. The experiments indicate that the fusion model performs better for both balanced and imbalanced data with a macro F1-score of 0.61 for imbalanced data, 0.62 for balanced data, and 0.55 with focal loss for imbalanced data. When focusing on accuracy, our experiments indicate that the fusion model performs better with scores of 0.91 for imbalanced data, 0.78 for balanced data, and 0.90 for imbalanced data with focal loss.
Original languageEnglish
Title of host publicationProceedings of the Natural Legal Language Processing Workshop 2023
EditorsDaniel Preoțiuc-Pietro, Catalina Goanta, Ilias Chalkidis, Leslie Barrett, Gerasimos (Jerry) Spanakis, Nikolaos Aletras
Place of PublicationStroudsburg
PublisherAssociation for Computational Linguistics (ACL)
Pages149-158
Number of pages10
ISBN (Electronic)9798891760547
ISBN (Print)979-8-89176-054-7
Publication statusPublished - Dec 2023
Event5th Natural Legal Language Processing Workshop - Singapore, Singapore
Duration: 7 Dec 20237 Dec 2023
Conference number: 5
https://nllpw.org/workshop/

Workshop

Workshop5th Natural Legal Language Processing Workshop
Abbreviated titleNLLP 2023
Country/TerritorySingapore
CitySingapore
Period7/12/237/12/23
Internet address

Keywords

  • deontic modalities
  • classification
  • focal loss

Fingerprint

Dive into the research topics of 'Low-Resource Deontic Modality Classification in EU Legislation'. Together they form a unique fingerprint.

Cite this