table 이해 및 Q&A 모델
https://arxiv.org/pdf/2404.07181
table 이해
제약 - Ligand 3D 디자인 : 디퓨전 사용
https://arxiv.org/pdf/2403.07902
그래픽 레이아웃 디자인
https://arxiv.org/pdf/2404.14368
그래픽 레이아웃 생성 멀티모달 모델
RGB-A 이미지를 인풋으로 써서 json draft 프로토콜을 결과로 생성한다
사용한 데이터셋은 아래와 같다
https://huggingface.co/datasets?other=graphic%20design
Hugging Face – The AI community building the future.
huggingface.co
https://huggingface.co/Lin-Chen/ShareGPT4V-7B
Lin-Chen/ShareGPT4V-7B · Hugging Face
ShareGPT4V-7B Model Card Model details Model type: ShareGPT4V-7B is an open-source chatbot trained by fine-tuning CLP vision tower and LLaMA/Vicuna on GPT4-Vision-assisted ShareGPT4V data and LLaVA instruction-tuning data. Model date: ShareGPT4V-7B was tra
huggingface.co
https://huggingface.co/datasets/cyberagent/crello
cyberagent/crello · Datasets at Hugging Face
[ [ 226, 216, 224 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 250, 171, 100 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 250, 171, 100 ], [ 230, 231, 232 ], [ 230, 231, 232 ], [ 230,
huggingface.co
사용한 off the shelf 모델 즉, 대체가능한 모델에는
RGBA-Encoder로 ViT-L/14 (with 224 × 224 four-channel input)
CLIP visual tower 파라미터로 훈련 첫 진행(initailize)한다
https://huggingface.co/openai/clip-vit-large-patch14
openai/clip-vit-large-patch14 · Hugging Face
Model Card: CLIP Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. Model Details The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision
huggingface.co
LLM
Qwen1.5-0.5B - tiny 버젼 https://huggingface.co/Qwen/Qwen1.5-0.5B
Qwen/Qwen1.5-0.5B · Hugging Face
Qwen1.5-0.5B Introduction Qwen1.5 is the beta version of Qwen2, a transformer-based decoder-only language model pretrained on a large amount of data. In comparison with the previous released Qwen, the improvements include: 8 model sizes, including 0.5B, 1.
huggingface.co
Qwen1.5-7B - small 버젼를 사용했다