Data for Finnish Dialect Detection
No Thumbnail Available
Restricted Availability
Date
2021-08-29, 2021-08-29
Persistent identifier of the Data Catalogue metadata
Creator/contributor
Editor
Journal title
Journal volume
Publisher
Publication Type
dataset
dataset
dataset
Peer Review Status
Repositories
Access rights
Open
ISBN
ISSN
Description
The data used in the paper "Finnish Dialect Identification: The Effect of Audio and Text".
If you use the data, please cite:
Hämäläinen, Mika; Alnajjar, Khalid; Partanen, Niko & Rueter, Jack (2021). Finnish Dialect Identification: The Effect of Audio and Text. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP).
The metadata.json contains the dialectal and normalized transcriptions, the length of the wav files in milliseconds, the path to the wav file, the role of the speaker and the dialect. The data.zip* files contain the wav files. They are partial zip files to make Zenodo upload easier. Code_models.zip contains the code for training the bimodal model and the final trained model presented in the paper. There is also an OpenNMT model that is the text only model described in the paper. You can use it by running: python3 -m onmt.bin.translate -model nmt-model_step_100000.pt -src source_test.txt -output pred.txt -replace_unk
Our dataset is licensed under CC BY NC ND 4.0. Academic use only.
The corpus is based on Suomenkielen näytteitä (CC BY Kotimaisten kielten keskus).