LLaVa is an open-source chatbot trained by fine-tuning LlamA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. In other words, it is an multi-modal version of LLMs fine-tuned for chat / …

[2024/05/10] 🔥 LLaVA-NeXT (Stronger) models are released, stronger LMM with support of LLama-3 (8B) and Qwen-1.5 (72B/110B). [Blog] [Checkpoints] [Demo] [Code] [2024/05/10] 🔥 LLaVA-NeXT (Video) is released. The image-only-trained LLaVA-NeXT model is surprisingly strong on video tasks with zero-shot modality transfer.

Apr 28, 2024 · llava-llama-3-8b-v1_1-hf is a LLaVA model fine-tuned from meta-llama/Meta-Llama-3-8B-Instruct and CLIP-ViT-Large-patch14-336 with ShareGPT4V-PT and InternVL-SFT by XTuner. Note: This model is in HuggingFace LLaVA format.

LLaVA represents a novel end-to-end trained large multimodal model that combines a vision encoder and Vicuna for general-purpose visual and language understanding, achieving impressive chat capabilities mimicking spirits of the multimodal GPT-4 and setting a new state-of-the-art accuracy on Science QA.

Jul 18, 2023 · LLaVA is a multimodal model that combines a vision encoder and Vicuna for general-purpose visual and language understanding, achieving impressive chat capabilities mimicking spirits of the multimodal GPT-4.

llava-llama-3-8b is a large multimodal model (LMM) trained using the LLaVA-v1.5 framework with the 8-billion parameter meta-llama/Meta-Llama-3-8B-Instruct model as language backbone and the CLIP-based vision encoder.

Nov 24, 2024 · The Llama prefix indicates that it includes both the text embedding layer and the Llama decoder. This class inherits from the LlamaPreTrainedModel class, which provides several handy...

LLaVA is an open-source project, collaborating with research community to advance the state-of-the-art in AI. LLaVA represents the first end-to-end trained large multimodal model (LMM) that achieves impressive chat capabilities mimicking spirits of the multimodal GPT-4.

Currently this implementation supports llava-v1.5 variants, as well as llava-1.6 llava-v1.6 variants. The pre-converted 7b and 13b models are available. For llava-1.6 a variety of prepared gguf models are available as well 7b-34b. After API is confirmed, …

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond. - haotian-liu/LLaVA