Notes on NLP for Moroccan Darija
Darija breaks every assumption standard NLP pipelines make: no standardized orthography, code-switching mid-sentence, and almost no labeled data. Field notes from building sentiment and transcription tools.
Moroccan Darija is spoken by over 30 million people and under-served by virtually every NLP system. Working on sentiment analysis and transcription for Darija surfaced problems you don't see in high-resource languages.
No canonical spelling
The same sentence exists in Arabic script, Latin script (arabizi with digits: "3", "7", "9" mapping Arabic letters), and mixed forms — sometimes within one comment. Any preprocessing pipeline must normalize across scripts before it can do anything else.
Code-switching is the default
Real Moroccan social media text switches between Darija, Modern Standard Arabic, French and English mid-sentence. Language detection at the document level is useless; you need token-level awareness.
Data scarcity is a protocol problem
With ~71% accuracy on 3-class sentiment classification using classical ML, the bottleneck wasn't the model — it was annotation consistency. Two annotators rarely agree on sarcasm in Darija. Before fine-tuning anything, invest in:
- A written annotation guideline with examples
- Inter-annotator agreement measurement
- A held-out test set frozen before experimentation
Why this matters for research
Darija is a natural laboratory for low-resource NLP: transfer learning from MSA, dialect adapters, speech-to-text for oral-first languages. It's also where applied AI meets real users in Morocco — which is exactly the intersection I want for doctoral research.