Name: High-Quality Multilingual Translation Data
Creator: Kaggle
License: https://creativecommons.org/publicdomain/zero/1.0/

About this Dataset

High-Quality Multilingual Translation Data

13 Languages for Machine Learning

By Huggingface Hub [source]

About this dataset

This extensive collection of multilingual translation data provides an invaluable resource for the furtherance of machine learning research. With language pairs spanning both English and non-English languages, this dataset delivers a comprehensive selection of high-quality text translations with thousands of records per language pair. Each file within the folder for a given language pair contains two distinct columns – id and translation – providing identification numbers associated with each translation record as well as the corresponding translation text itself. This highly structured data set is sure to be an invaluable asset in the pursuit of advanced machine learning techniques!

More Datasets

For more datasets, click here.

Featured Notebooks

🚨 Your notebook can be here! 🚨!

How to use the dataset

How to Use this Dataset

This multilingual translation data can be used for a variety of tasks, from training machine learning models to understanding language nuances. Below are some steps to get you started using this dataset:

Select the language pair that you would like to work with (ex. English-Spanish). This selection can be found in the filename (eg. en-es_train).

Download and extract the file containing your selected language pair from this Kaggle dataset. You will find two files for both training and testing within this folder - Training_File and Test_File.

Open your chosen file in a spreadsheet program such as Microsoft Excel or Google Sheets, so that you may explore the contents of the dataset. You will find two columns present: id (unique identifier for each translation pair) and translation which contains information about translations from either English or a non-English language depending on which file you are accessing (training vs test).

With these files you may then generate machine learning models in order apply natural language processing techniques, or simply explore transnational correlations between languages amongst other interesting research applications!

Research Ideas

Developing machine translation models: This dataset can be used to train and evaluate a variety of different machine translation models. The data could be used to optimize existing algorithms as well as train entirely new models tailored specifically for multilingual applications.

Improving natural language understanding: This corpus could be used to help build better artificial intelligence systems with an enhanced ability to process natural language inputs, thus allowing them to rapidly translate and respond accurately in multiple languages.

Translating web content dynamically: This dataset can be leveraged by web developers who want their websites and applications to automatically detect a visitor's language and generate translations instantly in the correct language pair format. The rapid response time would eliminate the need for cumbersome inter-language switching among users

Acknowledgements

If you use this dataset in your research, please credit the original authors.
Data Source

License

License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication
No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Columns

File: ca-en_train.csv

Column name	Description
translation	Contains both English and non-English translations side by side. (String)

File: en-fi_train.csv

Column name	Description
translation	Contains both English and non-English translations side by side. (String)

File: en-es_train.csv

Column name	Description
translation	Contains both English and non-English translations side by side. (String)

Acknowledgements

If you use this dataset in your research, please credit the original authors.
If you use this dataset in your research, please credit Huggingface Hub.

Tables

En Pl Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.en_pl_train

406.13 KB
2831 rows
2 columns


CREATE TABLE en_pl_train (
  "id" BIGINT,
  "translation" VARCHAR
);

En Pt Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.en_pt_train

199.36 KB
1404 rows
2 columns


CREATE TABLE en_pt_train (
  "id" BIGINT,
  "translation" VARCHAR
);

En Ru Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.en_ru_train

2.96 MB
17496 rows
2 columns


CREATE TABLE en_ru_train (
  "id" BIGINT,
  "translation" VARCHAR
);

En Sv Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.en_sv_train

530.42 KB
3095 rows
2 columns


CREATE TABLE en_sv_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Eo Es Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.eo_es_train

273.75 KB
1677 rows
2 columns


CREATE TABLE eo_es_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Eo Fr Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.eo_fr_train

271.21 KB
1588 rows
2 columns


CREATE TABLE eo_fr_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Eo Hu Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.eo_hu_train

267.7 KB
1636 rows
2 columns


CREATE TABLE eo_hu_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Eo It Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.eo_it_train

259.73 KB
1453 rows
2 columns


CREATE TABLE eo_it_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Eo Pt Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.eo_pt_train

204.24 KB
1259 rows
2 columns


CREATE TABLE eo_pt_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es Fi Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_fi_train

485.6 KB
3344 rows
2 columns


CREATE TABLE es_fi_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es Fr Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_fr_train

9.13 MB
56319 rows
2 columns


CREATE TABLE es_fr_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es It Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_it_train

5 MB
28868 rows
2 columns


CREATE TABLE es_it_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es Nl Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_nl_train

5.75 MB
32247 rows
2 columns


CREATE TABLE es_nl_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es No Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_no_train

489.83 KB
3585 rows
2 columns


CREATE TABLE es_no_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es Pt Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_pt_train

207.54 KB
1327 rows
2 columns


CREATE TABLE es_pt_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Es Ru Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.es_ru_train

3.05 MB
16793 rows
2 columns


CREATE TABLE es_ru_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fi Fr Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fi_fr_train

505.14 KB
3537 rows
2 columns


CREATE TABLE fi_fr_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fi No Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fi_no_train

463.96 KB
3414 rows
2 columns


CREATE TABLE fi_no_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fi Pl Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fi_pl_train

426.79 KB
2814 rows
2 columns


CREATE TABLE fi_pl_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr Hu Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_hu_train

14.9 MB
89337 rows
2 columns


CREATE TABLE fr_hu_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr It Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_it_train

3.07 MB
14692 rows
2 columns


CREATE TABLE fr_it_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr Nl Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_nl_train

6.58 MB
40017 rows
2 columns


CREATE TABLE fr_nl_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr No Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_no_train

460.74 KB
3449 rows
2 columns


CREATE TABLE fr_no_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr Pl Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_pl_train

424.31 KB
2825 rows
2 columns


CREATE TABLE fr_pl_train (
  "id" BIGINT,
  "translation" VARCHAR
);

Fr Pt Train

@kaggle.thedevastator_high_quality_multilingual_translation_data.fr_pt_train

207.16 KB
1263 rows
2 columns


CREATE TABLE fr_pt_train (
  "id" BIGINT,
  "translation" VARCHAR
);