Instructions to use Repoaner/llama_guard_vision with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Repoaner/llama_guard_vision with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Repoaner/llama_guard_vision")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Repoaner/llama_guard_vision")
model = AutoModelForImageTextToText.from_pretrained("Repoaner/llama_guard_vision")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Repoaner/llama_guard_vision with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Repoaner/llama_guard_vision"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Repoaner/llama_guard_vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Repoaner/llama_guard_vision

SGLang

How to use Repoaner/llama_guard_vision with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Repoaner/llama_guard_vision" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Repoaner/llama_guard_vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Repoaner/llama_guard_vision" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Repoaner/llama_guard_vision",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Repoaner/llama_guard_vision with Docker Model Runner:
```
docker model run hf.co/Repoaner/llama_guard_vision
```

llama_guard_vision / chat_template.json

Repoaner

Upload folder using huggingface_hub

24a8883 verified 10 months ago

raw

history blame contribute delete

2.7 kB

	{
	"chat_template": "{%- if messages\|length % 2 == 0 -%}\n {%- set role = 'Agent' -%}\n{%- else -%}\n {%- set role = 'User' -%}\n{%- endif -%}\n{%- if categories is not defined -%}\n {%- \n set categories = ({\n \"S1\": \"Violent Crimes.\",\n \"S2\": \"Non-Violent Crimes.\",\n \"S3\": \"Sex Crimes.\",\n \"S4\": \"Child Exploitation.\",\n \"S5\": \"Defamation.\",\n \"S6\": \"Specialized Advice.\",\n \"S7\": \"Privacy.\",\n \"S8\": \"Intellectual Property.\",\n \"S9\": \"Indiscriminate Weapons.\",\n \"S10\": \"Hate.\",\n \"S11\": \"Self-Harm.\",\n \"S12\": \"Sexual Content.\",\n \"S13\": \"Elections.\",\n })\n -%}\n{%- endif -%}\n\n{{- \"<\|begin_of_text\|><\|start_header_id\|>user<\|end_header_id\|>\\n\\n\" -}}\n{%- set ns = namespace(contains_image=false) -%}\n{%- for message in messages -%}\n {%- for content in message['content'] \| selectattr('type', 'equalto', 'image') -%}\n {%- set ns.contains_image = true -%}\n {%- endfor -%}\n{%- endfor -%}\n{%- if ns.contains_image -%}\n{{- \"<\|image\|> \"-}}\n{%- endif -%}\n{{- \"Task: Check if there is unsafe content in '\" + role + \"' messages in conversations according our safety policy with the below categories.\\n\\n\" -}}\n{{- \"<BEGIN UNSAFE CONTENT CATEGORIES>\\n\"-}}\n\n{%- for key in categories -%}\n {%- if key not in excluded_category_keys -%}\n {{-key + \": \" + categories[key] + \"\\n\" -}}\n {%- endif -%}\n{%- endfor -%}\n\n{{- \"<END UNSAFE CONTENT CATEGORIES>\\n\\n\"-}}\n{{- \"<BEGIN CONVERSATION>\\n\\n\"-}}\n{%- for message in messages -%}\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n {{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...')-}}\n {%- endif -%}\n {%- set content = message['content'] -%}\n {%- if message['role'] == 'user' -%}\n {%- set role = 'User' -%}\n {%- elif message['role'] == 'assistant' -%}\n {%- set role = 'Agent' -%}\n {%- endif -%}\n {%- for content in message['content'] \| selectattr('type', 'equalto', 'text') -%}\n{{- role + ': ' + content['text'] \| trim + '\\n\\n'-}}\n {%- endfor -%}\n{%- endfor -%}\n{{- \"<END CONVERSATION>\\n\\n\"-}}\n{{- \"Provide your safety assessment for ONLY THE LAST \" + role + \" message in the above conversation:\\n\"-}}\n{{- \" - First line must read 'safe' or 'unsafe'.\\n\"-}}\n{{- \" - If unsafe, a second line must include a comma-separated list of violated categories. <\|eot_id\|><\|start_header_id\|>assistant<\|end_header_id\|>\"-}}\n"
	}