Morphology data for 22 languages
No Thumbnail Available
Restricted Availability
Date
2020-07-02, 2020-07-02
Persistent identifier of the Data Catalogue metadata
Creator/contributor
Editor
Journal title
Journal volume
Publisher
Publication Type
dataset
dataset
dataset
Peer Review Status
Repositories
Access rights
Open
ISBN
ISSN
Description
Most people will want to download only the train_data.zip file. This contains folders named after language ISO codes. Under each language, you will find separate folders for the lemmatization, analysis and generation tasks. Each one of these folders has source and target files for train, val and test. There is also a pred.txt file that has the predictions of the baseline system. If you need more control over how these datasets are created, keep on reading.
To create the dataset from scratch, run the data_formatter.py. The morphological data is stored in the fst.zip, if you want to download that and skip some parts of making the data from scratch, extract it to the same folder as the data_formatter.py.
Cite:
Hämäläinen, M., Partanen, N., Rueter, J., & Alnajjar, K. (2021). Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa 2021)