이진욱님의 블로그

https://arxiv.org/abs/2409.15268

2.3 RLAIF AND LLM-AS-A-JUDGE Reinforcement Learning from AI Feedback (RLAIF) presents an alternative approach to the standard RLHF pipeline. Bai et al. (2022b) demonstrate the efficacy of RLAIF in training helpful and harmless models without relying on human feedback labels for harmlessness assessment. Their work shows that as language model capabilities improve, AI identification of harms incre..

https://arxiv.org/abs/2410.17050 UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMsThe key components of machine learning are data samples for training, model for learning patterns, and loss function for optimizing accuracy. Analogously, unlearning can potentially be achieved through anti-data samples (or anti-samples), unlearning methodarxiv.orgThe key components of machine lear..

https://arxiv.org/pdf/2410.17131

https://arxiv.org/pdf/2410.12832Reinforcement Learning from Human Feedback (RLHF) has greatly improved the performance of modern Large Language Models (LLMs). The RLHF process is resource-intensive and technically challenging, generally requiring a large collection of human preference labels over model-generated outputs. Reinforcement Learning from AI Feedback (RLAIF) addresses this data collect..

https://arxiv.org/pdf/2407.04842While text-to-image models like DALLE-3 and Stable Diffusion are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, ..

https://arxiv.org/abs/2306.13394 MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language ModelsMultimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image. However, it is difficult for these case studies to fully reflect tarxiv.orgMultimodal Large Langu..

https://arxiv.org/abs/2311.16502 MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIWe introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exaarxiv.orgWe i..

https://arxiv.org/pdf/2410.08193

https://arxiv.org/pdf/2410.13787Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind (e.g., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect? We define introspection as acquiring knowledge that is not contained in or derived from training data but ..