Test specific invalid heuristics that NLI models are likely to learn
Dataset Description
HANS (Invalid NLI Heuristics Benchmark)
Test specific invalid heuristics that NLI models are likely to learn
Source
Huggingface Hub: link
About this dataset
The HANS dataset is an NLI evaluation set that tests specific hypotheses about invalid heuristics that NLI models are likely to learn.
How to use the dataset
Research Ideas
- Training better NLI models
- Understanding how NLI models make predictions
- Investigating the types of heuristics that NLI models learn
Acknowledgements
License
> License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication
> No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.
Columns
File: validation.csv
| Column name | Description |
|---|---|
| premise | The premise of the example. (string) |
| hypothesis | The hypothesis of the example. (string) |
| label | The label of the example. (string) |
| parse_premise | The parse of the premise. (string) |
| parse_hypothesis | The parse of the hypothesis. (string) |
| binary_parse_premise | The binary parse of the premise. (string) |
| binary_parse_hypothesis | The binary parse of the hypothesis. (string) |
| heuristic | The heuristic that the example is based on. (string) |
| subcase | The subcase of the heuristic that the example is based on. (string) |
| template | The template that the example is based on. (string) |
File: train.csv
| Column name | Description |
|---|---|
| premise | The premise of the example. (string) |
| hypothesis | The hypothesis of the example. (string) |
| label | The label of the example. (string) |
| parse_premise | The parse of the premise. (string) |
| parse_hypothesis | The parse of the hypothesis. (string) |
| binary_parse_premise | The binary parse of the premise. (string) |
| binary_parse_hypothesis | The binary parse of the hypothesis. (string) |
| heuristic | The heuristic that the example is based on. (string) |
| subcase | The subcase of the heuristic that the example is based on. (string) |
| template | The template that the example is based on. (string) |
Related Datasets
-
SuperGLUE
@kaggle
-
Dummy Monster
@owid