이진욱님의 블로그

Efficient self-improvement in multimodal large language models: A model-level judge-free approach. Strengthening multimodal large language model with bootstrapped preference optimization. CLIP-DPO: Vision-language models as a source of preference for fixing hallucinations in lvlms. Enhancing large vision language models with self-training on image comprehension. RLAIF-V: Aligning mllms through o..

https://arxiv.org/html/2412.17451v1#bib.bib33

https://arxiv.org/pdf/2412.03704v2

https://arxiv.org/html/2501.13926v1

https://arxiv.org/html/2501.07542v1

https://arxiv.org/abs/2411.10440

RLAIF-V assigned trustworthiness score to atomic claims of each candidate responses using open-source MLLM.  SIMA  utilizes critic prompt considering various factor to obtain the preference pairs dataset.  VL-feedback uses gpt-4v to assess various decoded responses from other LMM regarding Helpfulness, Visual Faithfulness, Ethical Considerations.  FGAIF  trains reward model to assign score to va..

TEST-TIME ADAPTATION WITH CLIP REWARD FOR ZERO-SHOT GENERALIZATION IN VISION-LANGUAGE MODELShttps://openreview.net/pdf?id=kIP0duasBbEfficient Test-Time Prompt Tuning for Vision-Language Modelshttps://arxiv.org/html/2408.05775v1Self-Generated Critiques Boost Reward Modeling for Language Modelshttps://arxiv.org/html/2411.16646v2Rewarding Progress: Scaling Automated Process Verifiers for LLM Reason..