From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models

Abdulmuizz Khalak; Abderrahmane Issam; Gerasimos Spanakis

doi:10.18653/v1/2026.vardial-1.16

From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

Abstract

Arabic Language Models (LMs) are pretrained predominately on Modern Standard Arabic (MSA) and are expected to transfer to its dialects. While MSA as the standard written variety is commonly used in formal settings, people speak and write online in various dialects that are spread across the Arab region. This poses limitations for Arabic LMs, since its dialects vary in their similarity to MSA. In this work we study cross-lingual transfer of Arabic models using probing on 3 Natural Language Processing (NLP) Tasks, and representational similarity. Our results indicate that transfer is possible but disproportionate across dialects, which we find to be partially explained by their geographic proximity. Furthermore, we find evidence for negative interference in models trained to support all Arabic dialects. This questions their degree of similarity, and raises concerns for cross-lingual transfer in Arabic models.

Original language	English
Title of host publication	VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop
Editors	Yves Scherrer, Noemi Aepli, Verena Blaschke, Tommi Jauhiainen, Nikola Ljubesic, Preslav Nakov, Jorg Tiedemann, Marcos Zampieri
Publisher	Association for Computational Linguistics (ACL)
Pages	196-209
Number of pages	14
ISBN (Electronic)	9798891763722
DOIs	https://doi.org/10.18653/v1/2026.vardial-1.16
Publication status	Published - 2026
Event	13th Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial 2026 - co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026 - Rabat, Morocco Duration: 29 Mar 2026 → 29 Mar 2026 https://2026.eacl.org/program/workshops/

Workshop

Workshop	13th Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial 2026 - co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026
Country/Territory	Morocco
City	Rabat
Period	29/03/26 → 29/03/26
Internet address	https://2026.eacl.org/program/workshops/

Access to Document

10.18653/v1/2026.vardial-1.16Licence: CC BY

Cite this

Khalak, A., Issam, A., & Spanakis, G. (2026). From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models. In Y. Scherrer, N. Aepli, V. Blaschke, T. Jauhiainen, N. Ljubesic, P. Nakov, J. Tiedemann, & M. Zampieri (Eds.), VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop (pp. 196-209). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2026.vardial-1.16

Khalak, Abdulmuizz ; Issam, Abderrahmane ; Spanakis, Gerasimos. / From FusHa to Folk : Exploring Cross-Lingual Transfer in Arabic Language Models. VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop. editor / Yves Scherrer ; Noemi Aepli ; Verena Blaschke ; Tommi Jauhiainen ; Nikola Ljubesic ; Preslav Nakov ; Jorg Tiedemann ; Marcos Zampieri. Association for Computational Linguistics (ACL), 2026. pp. 196-209

@inproceedings{1807c391485b4f49b28988d8c3960ec1,

title = "From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models",

abstract = "Arabic Language Models (LMs) are pretrained predominately on Modern Standard Arabic (MSA) and are expected to transfer to its dialects. While MSA as the standard written variety is commonly used in formal settings, people speak and write online in various dialects that are spread across the Arab region. This poses limitations for Arabic LMs, since its dialects vary in their similarity to MSA. In this work we study cross-lingual transfer of Arabic models using probing on 3 Natural Language Processing (NLP) Tasks, and representational similarity. Our results indicate that transfer is possible but disproportionate across dialects, which we find to be partially explained by their geographic proximity. Furthermore, we find evidence for negative interference in models trained to support all Arabic dialects. This questions their degree of similarity, and raises concerns for cross-lingual transfer in Arabic models.",

author = "Abdulmuizz Khalak and Abderrahmane Issam and Gerasimos Spanakis",

note = "Publisher Copyright: {\textcopyright} 2026 Association for Computational Linguistics.; 13th Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial 2026 - co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026 ; Conference date: 29-03-2026 Through 29-03-2026",

year = "2026",

doi = "10.18653/v1/2026.vardial-1.16",

language = "English",

pages = "196--209",

editor = "Yves Scherrer and Noemi Aepli and Verena Blaschke and Tommi Jauhiainen and Nikola Ljubesic and Preslav Nakov and Jorg Tiedemann and Marcos Zampieri",

booktitle = "VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

address = "United States",

url = "https://2026.eacl.org/program/workshops/",

}

Khalak, A, Issam, A & Spanakis, G 2026, From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models. in Y Scherrer, N Aepli, V Blaschke, T Jauhiainen, N Ljubesic, P Nakov, J Tiedemann & M Zampieri (eds), VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop. Association for Computational Linguistics (ACL), pp. 196-209, 13th Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial 2026 - co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026, Rabat, Morocco, 29/03/26. https://doi.org/10.18653/v1/2026.vardial-1.16

From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models. / Khalak, Abdulmuizz; Issam, Abderrahmane ; Spanakis, Gerasimos.
VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop. ed. / Yves Scherrer; Noemi Aepli; Verena Blaschke; Tommi Jauhiainen; Nikola Ljubesic; Preslav Nakov; Jorg Tiedemann; Marcos Zampieri. Association for Computational Linguistics (ACL), 2026. p. 196-209.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - From FusHa to Folk

T2 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial 2026 - co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026

AU - Khalak, Abdulmuizz

AU - Issam, Abderrahmane

AU - Spanakis, Gerasimos

PY - 2026

Y1 - 2026

N2 - Arabic Language Models (LMs) are pretrained predominately on Modern Standard Arabic (MSA) and are expected to transfer to its dialects. While MSA as the standard written variety is commonly used in formal settings, people speak and write online in various dialects that are spread across the Arab region. This poses limitations for Arabic LMs, since its dialects vary in their similarity to MSA. In this work we study cross-lingual transfer of Arabic models using probing on 3 Natural Language Processing (NLP) Tasks, and representational similarity. Our results indicate that transfer is possible but disproportionate across dialects, which we find to be partially explained by their geographic proximity. Furthermore, we find evidence for negative interference in models trained to support all Arabic dialects. This questions their degree of similarity, and raises concerns for cross-lingual transfer in Arabic models.

AB - Arabic Language Models (LMs) are pretrained predominately on Modern Standard Arabic (MSA) and are expected to transfer to its dialects. While MSA as the standard written variety is commonly used in formal settings, people speak and write online in various dialects that are spread across the Arab region. This poses limitations for Arabic LMs, since its dialects vary in their similarity to MSA. In this work we study cross-lingual transfer of Arabic models using probing on 3 Natural Language Processing (NLP) Tasks, and representational similarity. Our results indicate that transfer is possible but disproportionate across dialects, which we find to be partially explained by their geographic proximity. Furthermore, we find evidence for negative interference in models trained to support all Arabic dialects. This questions their degree of similarity, and raises concerns for cross-lingual transfer in Arabic models.

U2 - 10.18653/v1/2026.vardial-1.16

DO - 10.18653/v1/2026.vardial-1.16

M3 - Conference article in proceeding

SP - 196

EP - 209

BT - VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop

A2 - Scherrer, Yves

A2 - Aepli, Noemi

A2 - Blaschke, Verena

A2 - Jauhiainen, Tommi

A2 - Ljubesic, Nikola

A2 - Nakov, Preslav

A2 - Tiedemann, Jorg

A2 - Zampieri, Marcos

PB - Association for Computational Linguistics (ACL)

Y2 - 29 March 2026 through 29 March 2026

ER -

Khalak A, Issam A , Spanakis G. From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models. In Scherrer Y, Aepli N, Blaschke V, Jauhiainen T, Ljubesic N, Nakov P, Tiedemann J, Zampieri M, editors, VarDial 2026 - 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2026. p. 196-209 doi: 10.18653/v1/2026.vardial-1.16

From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models

Abstract

Workshop

Access to Document

Fingerprint

Cite this