이진욱님의 블로그

https://arxiv.org/pdf/2308.08998요약 : Reward model + SFT 핵심 Grow stage에서 데이터셋을 샘플링 ~ current policy modelreward model을 통해 filtered improve 파트 NLL loss  Figure 1 모식도 pseudo - 알고리즘 Reward 부여 방식 : 토큰 뒤에 scalar reward로

https://arxiv.org/pdf/2401.06591Assessing long-form responses generated by Vision-Language Models (VLMs) is challenging. It not only requires checking whether the VLM follows the given instruction but also verifying whether the text output is properly grounded on the given image. Inspired by the recent approach of evaluating LMs with LMs, in this work, we propose to evaluate VLMs with VLMs. For ..

https://arxiv.org/pdf/2405.14622요약reward 부여 방식: self-generated instruction-following score( calculated using the language decoder of the LVLM , 이거 하나로만 안되는 이유 : modality misalignment, potentially overlooking visual input information ), + the image-response relevance score, R^I (s).( We leverage CLIP-score [17] for this calculation ) 3 Calibrated Self-Rewarding Vision Language Models To address t..

https://arxiv.org/pdf/2404.01258v2 Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM) However, in tasks involving video instruction following, providing informative feedback, especially for detecting hallucinations in generated responses, remains a significant challenge Previous..

https://arxiv.org/pdf/2401.00789

https://arxiv.org/pdf/2304.08485Instruction tuning large language models (LLMs) using machine-generated instruction-following data has been shown to improve zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field we present the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data. By instruction tuning on suc..

https://arxiv.org/pdf/2407.06023

https://arxiv.org/pdf/2406.10858

https://arxiv.org/pdf/2408.16737Training on high-quality synthetic data from strong language models (LMs) is a common strategy to improve the reasoning performance of LMs. In this work, we revisit whether this strategy is computeoptimal under a fixed inference budget (e.g., FLOPs). To do so, we investigate the trade-offs between generating synthetic data using a stronger but more expensive (SE) ..

https://arxiv.org/abs/2409.20537