이진욱님의 블로그

https://arxiv.org/abs/2404.01730

https://arxiv.org/abs/2402.01694  ARGS: Alignment as Reward-Guided SearchAligning large language models with human objectives is paramount, yet common approaches including RLHF suffer from unstable and resource-intensive training. In response to this challenge, we introduce ARGS, Alignment as Reward-Guided Search, a novel framearxiv.orgAligning large language models with human objectives is para..

https://arxiv.org/pdf/2402.10038Reinforcement learning from human feedback (RLHF) has been extensively employed to align large language models with user intent. However, proximal policy optimization (PPO) based RLHF is occasionally unstable requiring significant hyperparameter finetuning, and computationally expensive to maximize the estimated reward during alignment. Recently, direct preference..

https://arxiv.org/abs/2404.11912

https://arxiv.org/pdf/2405.00675Standard reinforcement learning from human feedback (RLHF) approaches relying on parametric models like the Bradley-Terry model fall short in capturing the intransitivity and irrationality in human preferences. Recent advancements suggest that directly working with preference probabilities can yield a more accurate reflection of human preferences, enabling more fl..