이진욱님의 블로그

https://arxiv.org/pdf/2407.04842While text-to-image models like DALLE-3 and Stable Diffusion are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, ..

https://arxiv.org/abs/2306.13394 MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language ModelsMultimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image. However, it is difficult for these case studies to fully reflect tarxiv.orgMultimodal Large Langu..

https://arxiv.org/abs/2311.16502 MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIWe introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exaarxiv.orgWe i..

https://arxiv.org/pdf/2410.08193

https://arxiv.org/pdf/2410.13787Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind (e.g., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect? We define introspection as acquiring knowledge that is not contained in or derived from training data but ..

https://arxiv.org/abs/2410.04343

https://arxiv.org/pdf/2410.10934

https://arxiv.org/pdf/2410.12381

https://arxiv.org/abs/2410.07112Current benchmarks for assessing vision-language models (VLMs) often focus on their perception or problem-solving capabilities and neglect other critical aspects such as fairness, multilinguality, or toxicity.Furthermore, they differ in their evaluation procedures and the scope of the evaluation, making it diff i cult to compare models. To address these issues, we..

https://arxiv.org/pdf/2406.11069Recent breakthroughs in vision-language models (VLMs) emphasize the necessity of benchmarking human preferences in real-world multimodal interactions. To address this gap, we launched WILDVISION-ARENA (WV-ARENA), an online platform that collects human preferences to evaluate VLMs. We curated WVBENCH by selecting 500 high-quality samples from 8,000 user submissions..