LLaVA-Plus-CodebaseLLaVA-VL/LLaVA-Plus-Codebase
Provides a reusable training and inference codebase for multimodal LLMs (LLaVA family),…
No ratings yet
Rate this skill
Stars771
Forks60
Watchers771
Open Issues22
InstallOne-line install
UnconfirmedWhether this repository follows the Agent Skills spec (that is, whether it contains a SKILL.md) has not been verified yet. Until that is settled, clone it and use what you need:
git clone https://github.com/LLaVA-VL/LLaVA-Plus-Codebase.git
OverviewOverview
Key Features
Provides a reusable training and inference codebase for multimodal LLMs (LLaVA family), addressing three engineering concerns: vision-language connector selection, image preprocessing strategy, and launching tool-using agents. Key capabilities include: 1. Vision-language connector configuration: the --mm_projector_type flag selects connectors such as mlp2x_gelu (a two-layer MLP) to map vision-encoder outputs into the language model's hidden space, making image-text alignment trainable and swappable; 2. Image aspect-ratio preprocessing: the --image_aspect_ratio pad option pads and resizes non-square images instead of stretching them, preventing distortion and keeping input sizes consistent across batches; 3. Tool worker launch: a Launch tool workers script entry starts multiple tool service processes in bulk so the agent can call external tools in parallel during inference; 4. Multimodal agent training and inference scripts: integrates model, data, and tool-calling flows to reproduce and fine-tune multimodal agents on custom data.
Use Cases
Multimodal LLM fine-tuning experiments, comparing and selecting vision-language connectors, building image preprocessing pipelines, local deployment and debugging of tool-using agents.
Target Roles
AI / Algorithm Engineer, Backend Developer, Full-stack / Indie Developer.
TopicsTopics
RatingUser ratings
Nobody has rated this skill yet. Yours would be the first on the site — your vote becomes the initial average.
RepositoryRepository
| Skill ID | 1816 |
|---|---|
| Data source | GitHub |
| Repository | LLaVA-VL/LLaVA-Plus-Codebase |
| Category | AI & Multimodal / Multimodal Agents & Tool Use |
| Primary language | Python |
| License | Apache License 2.0 |
| Default branch | main |
| GitHub repository ID | 715592775 |
| Node ID | R_kgDOKqcURw |
| Rank | #928 (position within the source ranking) |
| Site-wide popularity rank | #1045 (computed live from Stars, across all 5833 entries) |
GrowthStar / Fork growth
One snapshot per day, tracking how Stars and Forks changed
- Now771
- Previous point · 09-27 771
- vs previous point ±0
- 2 data points 14 days
- Now60
- Previous point · 09-27 61
- vs previous point -1
- 2 data points 14 days
TimelineTimeline
Repository created2023-11-07 13:06:022023-11-07
Data last updated2026-10-26 12:06:1721 days ago
Last commit2024-02-01 12:51:102024-02-01
CommitLatest commit
Version [latest] committed by HaoZhang534. Update Category: Feature Addition & Enhancement. Commit details: Add opensedd and controlnet worker (#18). This update introduces new capabilities and module extensions.