Name: WikiANN
Creator: Kaggle
License: https://creativecommons.org/publicdomain/zero/1.0/

About this Dataset

WikiANN

Lombard Language Training Dataset

Lombard Language Training Data

By wikiann (From Huggingface) [source]

About this dataset

Overview

WikiANN (sometimes called PAN-X) is a multilingual named entity recognition dataset consisting of Wikipedia articles annotated with LOC (location), PER (person), and ORG (organisation) tags in the IOB2 format. This version corresponds to the balanced train, dev, and test splits of Rahimi et al. (2019), which supports 176 of the 282 languages from the original WikiANN corpus.

Columns

File: pdc_train.csv

Column name	Description
tokens	This column contains individual words or tokens in the Lombard language. (Text)
ner_tags	This column contains named entity recognition (NER) tags associated with each token. NER tags help identify and classify named entities such as names, locations, organizations, etc. (Text)
langs	This column indicates the language of each token. In this dataset, it specifically represents the Lombard language. (Text)
spans	This column provides information about the position or span of each token within the text. (Text)

File: sr_validation.csv

Column name	Description
tokens	This column contains individual words or tokens in the Lombard language. (Text)
ner_tags	This column contains named entity recognition (NER) tags associated with each token. NER tags help identify and classify named entities such as names, locations, organizations, etc. (Text)
langs	This column indicates the language of each token. In this dataset, it specifically represents the Lombard language. (Text)
spans	This column provides information about the position or span of each token within the text. (Text)

File: uz_train.csv

Column name	Description
tokens	This column contains individual words or tokens in the Lombard language. (Text)
ner_tags	This column contains named entity recognition (NER) tags associated with each token. NER tags help identify and classify named entities such as names, locations, organizations, etc. (Text)
langs	This column indicates the language of each token. In this dataset, it specifically represents the Lombard language. (Text)
spans	This column provides information about the position or span of each token within the text. (Text)

Research Ideas

Named Entity Recognition (NER) Training: The dataset can be used to train models for NER tasks specific to the Lombard language. By utilizing the ner_tags column, developers can create models that identify and classify named entities in Lombard text, such as names of people, places, organizations, and more.

Language Classification: Since the langs column indicates the language of each token in the dataset (which will always be Lombard), this dataset can be used for training language classification models. These models can then be utilized to automatically detect whether a given piece of text is in Lombard or another language.

Span Identification: The spans column provides information about the position or span of each token within the text. This information can be utilized to develop algorithms or applications that require analyzing specific spans within sentences or paragraphs in Lombard text. For example, it could help identify important phrases or extract certain sections of text from a larger document

Acknowledgements

If you use this dataset in your research, please credit the original authors.
Data Source

License

License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication
No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Acknowledgements

If you use this dataset in your research, please credit the original authors.
If you use this dataset in your research, please credit wikiann (From Huggingface).

Tables

Cbk Zam Test

@kaggle.thedevastator_lombard_language_training_dataset.cbk_zam_test

16.13 KB
100 rows
4 columns


CREATE TABLE cbk_zam_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cbk Zam Train

@kaggle.thedevastator_lombard_language_training_dataset.cbk_zam_train

17.46 KB
100 rows
4 columns


CREATE TABLE cbk_zam_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cbk Zam Validation

@kaggle.thedevastator_lombard_language_training_dataset.cbk_zam_validation

14.57 KB
100 rows
4 columns


CREATE TABLE cbk_zam_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cdo Test

@kaggle.thedevastator_lombard_language_training_dataset.cdo_test

13.33 KB
100 rows
4 columns


CREATE TABLE cdo_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cdo Train

@kaggle.thedevastator_lombard_language_training_dataset.cdo_train

13.93 KB
100 rows
4 columns


CREATE TABLE cdo_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cdo Validation

@kaggle.thedevastator_lombard_language_training_dataset.cdo_validation

14.01 KB
100 rows
4 columns


CREATE TABLE cdo_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ceb Test

@kaggle.thedevastator_lombard_language_training_dataset.ceb_test

10.07 KB
100 rows
4 columns


CREATE TABLE ceb_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ceb Train

@kaggle.thedevastator_lombard_language_training_dataset.ceb_train

9.53 KB
100 rows
4 columns


CREATE TABLE ceb_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ceb Validation

@kaggle.thedevastator_lombard_language_training_dataset.ceb_validation

9.88 KB
100 rows
4 columns


CREATE TABLE ceb_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ce Test

@kaggle.thedevastator_lombard_language_training_dataset.ce_test

13.49 KB
100 rows
4 columns


CREATE TABLE ce_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ce Train

@kaggle.thedevastator_lombard_language_training_dataset.ce_train

12.69 KB
100 rows
4 columns


CREATE TABLE ce_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ce Validation

@kaggle.thedevastator_lombard_language_training_dataset.ce_validation

12.68 KB
100 rows
4 columns


CREATE TABLE ce_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ckb Test

@kaggle.thedevastator_lombard_language_training_dataset.ckb_test

49.88 KB
1000 rows
4 columns


CREATE TABLE ckb_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ckb Train

@kaggle.thedevastator_lombard_language_training_dataset.ckb_train

52.86 KB
1000 rows
4 columns


CREATE TABLE ckb_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Ckb Validation

@kaggle.thedevastator_lombard_language_training_dataset.ckb_validation

50.52 KB
1000 rows
4 columns


CREATE TABLE ckb_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Co Test

@kaggle.thedevastator_lombard_language_training_dataset.co_test

10.06 KB
100 rows
4 columns


CREATE TABLE co_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Co Train

@kaggle.thedevastator_lombard_language_training_dataset.co_train

10.19 KB
100 rows
4 columns


CREATE TABLE co_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Co Validation

@kaggle.thedevastator_lombard_language_training_dataset.co_validation

9.76 KB
100 rows
4 columns


CREATE TABLE co_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Crh Test

@kaggle.thedevastator_lombard_language_training_dataset.crh_test

11.18 KB
100 rows
4 columns


CREATE TABLE crh_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Crh Train

@kaggle.thedevastator_lombard_language_training_dataset.crh_train

11.66 KB
100 rows
4 columns


CREATE TABLE crh_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Crh Validation

@kaggle.thedevastator_lombard_language_training_dataset.crh_validation

10.82 KB
100 rows
4 columns


CREATE TABLE crh_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Csb Test

@kaggle.thedevastator_lombard_language_training_dataset.csb_test

13.17 KB
100 rows
4 columns


CREATE TABLE csb_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Csb Train

@kaggle.thedevastator_lombard_language_training_dataset.csb_train

14.18 KB
100 rows
4 columns


CREATE TABLE csb_train (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Csb Validation

@kaggle.thedevastator_lombard_language_training_dataset.csb_validation

13.13 KB
100 rows
4 columns


CREATE TABLE csb_validation (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);

Cs Test

@kaggle.thedevastator_lombard_language_training_dataset.cs_test

680.91 KB
10000 rows
4 columns


CREATE TABLE cs_test (
  "tokens" VARCHAR,
  "ner_tags" VARCHAR,
  "langs" VARCHAR,
  "spans" VARCHAR
);