Image-Text-to-Text
Transformers
TensorBoard
Safetensors
multilingual
internvl_chat
feature-extraction
internvl
custom_code
conversational
Instructions to use OpenGVLab/InternVL-Chat-V1-5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenGVLab/InternVL-Chat-V1-5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="OpenGVLab/InternVL-Chat-V1-5", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenGVLab/InternVL-Chat-V1-5", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OpenGVLab/InternVL-Chat-V1-5 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OpenGVLab/InternVL-Chat-V1-5" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenGVLab/InternVL-Chat-V1-5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/OpenGVLab/InternVL-Chat-V1-5
- SGLang
How to use OpenGVLab/InternVL-Chat-V1-5 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OpenGVLab/InternVL-Chat-V1-5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenGVLab/InternVL-Chat-V1-5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OpenGVLab/InternVL-Chat-V1-5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenGVLab/InternVL-Chat-V1-5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use OpenGVLab/InternVL-Chat-V1-5 with Docker Model Runner:
docker model run hf.co/OpenGVLab/InternVL-Chat-V1-5
Update README.md
Browse files
README.md
CHANGED
|
@@ -161,12 +161,12 @@ generation_config = dict(
|
|
| 161 |
do_sample=False,
|
| 162 |
)
|
| 163 |
|
| 164 |
-
# single-round conversation
|
| 165 |
question = "请详细描述图片"
|
| 166 |
response = model.chat(tokenizer, pixel_values, question, generation_config)
|
| 167 |
print(question, response)
|
| 168 |
|
| 169 |
-
# multi-round conversation
|
| 170 |
question = "请详细描述图片"
|
| 171 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=None, return_history=True)
|
| 172 |
print(question, response)
|
|
@@ -174,8 +174,35 @@ print(question, response)
|
|
| 174 |
question = "请根据图片写一首诗"
|
| 175 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
|
| 176 |
print(question, response)
|
| 177 |
-
```
|
| 178 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
|
| 180 |
## Citation
|
| 181 |
|
|
|
|
| 161 |
do_sample=False,
|
| 162 |
)
|
| 163 |
|
| 164 |
+
# single-round single-image conversation
|
| 165 |
question = "请详细描述图片"
|
| 166 |
response = model.chat(tokenizer, pixel_values, question, generation_config)
|
| 167 |
print(question, response)
|
| 168 |
|
| 169 |
+
# multi-round single-image conversation
|
| 170 |
question = "请详细描述图片"
|
| 171 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=None, return_history=True)
|
| 172 |
print(question, response)
|
|
|
|
| 174 |
question = "请根据图片写一首诗"
|
| 175 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
|
| 176 |
print(question, response)
|
|
|
|
| 177 |
|
| 178 |
+
# multi-round multi-image conversation
|
| 179 |
+
pixel_values1 = load_image('./examples/image1.jpg', max_num=6).to(torch.bfloat16).cuda()
|
| 180 |
+
pixel_values2 = load_image('./examples/image2.jpg', max_num=6).to(torch.bfloat16).cuda()
|
| 181 |
+
pixel_values = torch.cat((pixel_values1, pixel_values2), dim=0)
|
| 182 |
+
|
| 183 |
+
question = "详细描述这两张图片"
|
| 184 |
+
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=None, return_history=True)
|
| 185 |
+
print(question, response)
|
| 186 |
+
# 第一张图片是一只红熊猫,它有着独特的橙红色皮毛,脸部、耳朵和四肢的末端有白色斑块。红熊猫的眼睛周围有深色的环,它的耳朵是圆形的,上面有白色的毛。它正坐在一个木制的结构上,看起来像是一个平台或休息的地方。背景中有树木和竹子,这表明红熊猫可能在一个模拟自然环境的动物园或保护区内。
|
| 187 |
+
#
|
| 188 |
+
# 第二张图片是一只大熊猫,它是中国的国宝,以其黑白相间的皮毛而闻名。大熊猫的眼睛、耳朵和四肢的末端是黑色的,而它的脸部、耳朵内侧和身体其他部分是白色的。大熊猫正坐在地上,周围有竹子,这是它们的主要食物来源。背景中也有树木,这表明大熊猫可能在一个为它们提供自然栖息地模拟的动物园或保护区内。
|
| 189 |
+
|
| 190 |
+
question = "这两张图片的相同点和区别分别是什么"
|
| 191 |
+
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
|
| 192 |
+
print(question, response)
|
| 193 |
+
# 这两张图片的相同点:
|
| 194 |
+
#
|
| 195 |
+
# 1. 都展示了熊猫,这是两种不同的熊猫物种。
|
| 196 |
+
# 2. 熊猫都处于一个看起来像是模拟自然环境的场所,可能是动物园或保护区。
|
| 197 |
+
# 3. 熊猫周围都有竹子,这是它们的主要食物来源。
|
| 198 |
+
#
|
| 199 |
+
# 这两张图片的区别:
|
| 200 |
+
#
|
| 201 |
+
# 1. 熊猫的种类不同:第一张图片是一只红熊猫,第二张图片是一只大熊猫。
|
| 202 |
+
# 2. 熊猫的皮毛颜色和图案不同:红熊猫的皮毛是橙红色,脸部、耳朵和四肢的末端有白色斑块;而大熊猫的皮毛是黑白相间的,眼睛、耳朵和四肢的末端是黑色的,脸部、耳朵内侧和身体其他部分是白色的。
|
| 203 |
+
# 3. 熊猫的姿态和位置不同:红熊猫坐在一个木制的结构上,而大熊猫坐在地上。
|
| 204 |
+
# 4. 背景中的植被和环境细节略有不同,但都包含树木和竹子。
|
| 205 |
+
```
|
| 206 |
|
| 207 |
## Citation
|
| 208 |
|