Abstract

• PLM : near-human performance 만큼 achieved, BUT human-interpretable evidence에는 약함

→ 이를 해결하기 위해 PLM을 “contrast alternatives”를 이용한 explanation prompts로 완성

e.g. peanuts are usally salty while raisins are sweet

1. Introduction

• PLM achieved impressive performance on CSR tasks,

  → 한계 : little human-interpretable evidence of the intermediate commonsense knowledge or reasoning they use

  해결 : “그들의 reasoning pattern에 free-form natural language explanations” 을 생성

  → 한계

  (1) space of possible free-form explanations is incredibly large&ambiguous& difficult to annotate or evaluate

  (2) difficult to quantifying the model’s dependence on free-form explanations

  해결

  

  이 모델

  설명에서 다른 possible answers들을 대조할수 있도록 contrastive prompts를 이용한 unsupervised method

  • CSR task에는 보통 comparison or contrast of plausible alternatives 많이 사용

  • 사람들은 보통 “why Q?”가 아닌 “why Q rather than P?”를 추론할 때 더 많이 사용

    → 이유 : focusing on the limited set of reasons

  • 목표 : PLM이 위의 이점을 얻기 위함

  • 방법 : in-filled by a PLM인 a small set of contrastive generation Prompts 개발

    • 템플릿은 인간이 어떻게 entities를 비교하고 대조하는 지를 cover함
    • 그러면 또다른 PLM은 generated contrastive explanation과 original input을 비교하여 final answer를 예측함

  • WSC & multiple-choice QA 실험

    • zero-shot setting에서 available PLM보다 11% 성능이 높음

  • contrastive explanations가 일반적인 clarification questions보다 유용

2. Related Works

CSR

• Models that rationalize their decisions by extract- ing a contiguous subsequence of the input as an explanation : CSR에 inadequate
• use free-form textual explanations : 너무 많은 space 차지, ambiguous
• uses an unsupervised approach to generate free-form explanations as sequences of tokens : not well-formed sentences
• 우리 모델 : uses specialized prompts to generate well-formed human-interpretable explana- tions without any additional supervision

가장 비슷한 테스크의 최근 모델

• an unsupervised approach using a fixed set of clarification questions as prompts

contrastive reasoning