이진욱님의 블로그

3IXC2.5-RewardData PreparationReward models are trained using pairwise preference annotations (e.g., prompts x with chosen responses yc and rejected responses yr) that reflect human preferences. While existing public preference data is primarily textual, with limited image and scarce video examples, we train IXC-2.5-Reward using both open-source data and a newly collected dataset to ensure broad..

3.1 PreliminariesBefore introducing our framework, we first present MCTS to clarify the motivation behind our work.  MCTS Coulom (2006) is a widely used planning algorithm and famously employed in AlphaGo Silver et al. (2016). Taking the Game of Go as an example, the algorithm assists in selecting the best possible action in the current state of the board based on their average rewards.  These r..

2 MethodIn this section, we present our preliminary attempts to adapt MLLMs by equipping them with slow-thinking capacities for complex multimodal tasks. We explore two straightforward adaptation methods: (1) transferring slow-thinking abilities using text-based long thought data, and (2) distilling multimodal long thought data from existing slow-thinking MLLMs. Our aim is to investigate how slo..

3 Step-by-Step Visual Reasoning Benchmark: VRC-Bench목적 to facilitate a thorough assesment of the reasoning capabilties in complex scenarios4 Proposed Step-by-Step Visual Reasoning Model: LlamaV-o1  시스템 프롬프트 점수 매기기 You are a reasoning evaluator designed to assess the alignment , coherence , and quality of reasoning steps in text responses . Your task is to evaluate reasoning steps between the * g..

https://arxiv.org/html/2411.14432v1#S3간단하게 두개의 MLLM 사용 reasoning, summarizationTo fully leverage the reasoning capabilities of MLLMs, we propose Insight-V, a novel system comprising two MLLMs dedicated to reasoning and summarization, respectively. reasoning model - detailed reasoning process 생성summary model - reasoning을 supplementray info 보조적인 정보로 사용해 정답에 대한 relevance utilility 를 평가  3.2 Constru..

https://arxiv.org/html/2412.09078v1#S4 Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM ReasoningForest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning Zhenni Bi    Kai Han    Chuanjian Liu    Yehui Tang    Yunhe Wang Abstract Large Language Models (LLMs) have shown remarkable abilities across various language tasks,arxiv.orgbenchmark : GSM 8k , MATH 3.1 FoT frame..

나중에

has emerged as indicate represent denote  identify despite, nevertheless, However, Although advances, development, advancemente.g) despite these advances,  have demonstrated shown propose present compelling comparable superior  significant remarkable impressive ~ing, thereby, thuse.g) Mastering multi-step visual reasoning requires the integration of multimodal information, along with rigorous ad..

https://aclanthology.org/2024.tacl-1.78/ When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMsRyo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, Rui Zhang. Transactions of the Association for Computational Linguistics, Volume 12. 2024.aclanthology.orgWhen Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs https://arxiv..

Efficient self-improvement in multimodal large language models: A model-level judge-free approach. Strengthening multimodal large language model with bootstrapped preference optimization. CLIP-DPO: Vision-language models as a source of preference for fixing hallucinations in lvlms. Enhancing large vision language models with self-training on image comprehension. RLAIF-V: Aligning mllms through o..