Commonsense Reasoning in Interpersonal Conflict

Abstract

We propose to use the subreddit named r/AITA as the corpus for studying social commonsense reasoning. Compared to existing popular corpora, it contains social situations with more complex structures. We show that current NLP systems have worse performance on the subset of the corpus where humans have a lower agreement. We show that, across different subsets, RoBERTa outperforms BERT. Intermediate task finetuning only produces similar performance on the subsets with a low agreement. Jointly learning to classify and generate improves the performance of BERT but produces similar results for RoBERTa on the subsets with a low agreement. Finally, we propose to use the adversarial attack technique to study the bias of NLP models. We provide preliminary algorithms and results on applying that technique to study the bias in different parts of the social situations.