이진욱님의 블로그

https://arxiv.org/pdf/2405.07863

https://arxiv.org/pdf/2406.01297Self-correction is an approach to improving responses from large language models (LLMs) by refining the responses using LLMs during inference. Prior work has proposed various self-correction frameworks using different sources of feedback, including self-evaluation and external feedback. However, there is still no consensus on the question of when LLMs can correct ..

https://arxiv.org/pdf/2409.12917point : 모델의 distribution에서 가능한 가장 좋은 final answer을 생성해내기 위함 + 모델 collapse를 막기완전히 스스로 생성한 데이터를 통해 self-correct 능력을 향상방식스스로 생성한 데이터 - distribution mistmatch 회피두단계로 훈련 stage - minimal edit strategy 의 실패 경우의 모델 collapse 회피를 위한것 (STaR)  LLM의 self correction 능력은 비효율적이다 (e.g llm cannot self correct yet 논문)Existing approach는 self-correct을 위해 여러개의 모델, more capable LLM, 혹은 ..

x = question , a1 = 첫번째 대답, a2 = 두번째 대답y = a1, a2 중에 뭐가 좋은지 = pseudo-labelingwe only select those data where the model exhibits high confidence

https://arxiv.org/abs/2305.10425

https://arxiv.org/pdf/2401.10727

https://arxiv.org/pdf/2408.03314Related WorkLanguage model reasoning.Language model performance on challenging mathematical reasoning tasks has rapidly improved in recent years [20, 22, 25, 32, 39]. These improvements can be attributed to four primary factors: 1) running continued pretraining on large corpora of math focused data [20, 22, 32, 39]; 2) improving the LLM proposal distribution by ei..

https://arxiv.org/abs/2401.17686

https://arxiv.org/pdf/2305.00633