HANS (Invalid NLI Heuristics Benchmark) by Kaggle | Other

About this Dataset

HANS (Invalid NLI Heuristics Benchmark)

Test specific invalid heuristics that NLI models are likely to learn

Source

Huggingface Hub: link

About this dataset

The HANS dataset is an NLI evaluation set that tests specific hypotheses about invalid heuristics that NLI models are likely to learn.

How to use the dataset

Research Ideas

Training better NLI models

Understanding how NLI models make predictions

Investigating the types of heuristics that NLI models learn

Acknowledgements

License

> License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication
> No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Columns

File: validation.csv

Column name	Description
premise	The premise of the example. (string)
hypothesis	The hypothesis of the example. (string)
label	The label of the example. (string)
parse_premise	The parse of the premise. (string)
parse_hypothesis	The parse of the hypothesis. (string)
binary_parse_premise	The binary parse of the premise. (string)
binary_parse_hypothesis	The binary parse of the hypothesis. (string)
heuristic	The heuristic that the example is based on. (string)
subcase	The subcase of the heuristic that the example is based on. (string)
template	The template that the example is based on. (string)

File: train.csv

Column name	Description
premise	The premise of the example. (string)
hypothesis	The hypothesis of the example. (string)
label	The label of the example. (string)
parse_premise	The parse of the premise. (string)
parse_hypothesis	The parse of the hypothesis. (string)
binary_parse_premise	The binary parse of the premise. (string)
binary_parse_hypothesis	The binary parse of the hypothesis. (string)
heuristic	The heuristic that the example is based on. (string)
subcase	The subcase of the heuristic that the example is based on. (string)
template	The template that the example is based on. (string)

Tables

Train

@kaggle.thedevastator_heuristic_analysis_of_natural_language_inference.train

2.57 MB
30000 rows
10 columns


CREATE TABLE train (
  "premise" VARCHAR,
  "hypothesis" VARCHAR,
  "label" BIGINT,
  "parse_premise" VARCHAR,
  "parse_hypothesis" VARCHAR,
  "binary_parse_premise" VARCHAR,
  "binary_parse_hypothesis" VARCHAR,
  "heuristic" VARCHAR,
  "subcase" VARCHAR,
  "template" VARCHAR
);

Validation

@kaggle.thedevastator_heuristic_analysis_of_natural_language_inference.validation

2.57 MB
30000 rows
10 columns


CREATE TABLE validation (
  "premise" VARCHAR,
  "hypothesis" VARCHAR,
  "label" BIGINT,
  "parse_premise" VARCHAR,
  "parse_hypothesis" VARCHAR,
  "binary_parse_premise" VARCHAR,
  "binary_parse_hypothesis" VARCHAR,
  "heuristic" VARCHAR,
  "subcase" VARCHAR,
  "template" VARCHAR
);