Instructions to use mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1", device_map="auto")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1")
model = AutoModelForCausalLM.from_pretrained("mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1

SGLang

How to use mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1 with Docker Model Runner:
```
docker model run hf.co/mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1
```

ShearedPlats-7b Instruct

This is an ShearedPlats-7b model that has been fine-tuned on 2 epochs of the Open-Platypus dataset.

The modified version of the dataset can be found here

Prompt Template

### Instruction:

{query}

### Response:
<Leave new line for model to respond>

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM,pipeline

tokenizer = AutoTokenizer.from_pretrained("mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1")
model = AutoModelForCausalLM.from_pretrained("mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1")
query = "Provide step-by-step instructions for making a sweet chicken bugger"
text_gen = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=350)
output = text_gen(f"### Instruction:\n{query}\n### Response:\n")
print(output[0]['generated_text'])
"""
### Instruction:
Provide step-by-step instructions for making a sweet chicken bugger
### Response:
Step 1: Prepare the ingredients

You will need a mixture of ground chicken, breadcrumbs, butter, Worcestershire sauce, garlic powder, onion powder, salt, and pepper.

Step 2: Form the bugger

Take a piece of chicken breast meat and use a sharp knife to cut it into small cubes. Place the cubes in a bowl and add the remaining ingredients: breadcrumbs, butter, Worcestershire sauce, garlic powder, onion powder, salt, and pepper. Mix the ingredients together until they are well combined.

Step 3: Shape the bugger

Take a piece of the bugger mixture and form it into a ball. Place the ball on a plate or in a bag and refrigerate it for 30 minutes.

Step 4: Cook the bugger

Heat a grill pan or grill to medium-high heat. Take the bugger out of the refrigerator and place it on the grill. Cook the bugger for 5-7 minutes on each side, or until it is cooked through.

Step 5: Serve and enjoy!

Once the bugger is cooked, serve it hot and enjoy!

Note: You can also use a sweet chicken bugger mix to make sweet chicken buggers. Simply follow the instructions above, but use the sweet chicken bugger mix instead of the ground chicken.

Enjoy your sweet chicken buggers!
"""

Evals

|  Tasks  |Version|Filter|n-shot| Metric |Value |   |Stderr|
|---------|-------|------|-----:|--------|-----:|---|-----:|
|hellaswag|Yaml   |none  |     0|acc     |0.5283|±  |0.0050|
|         |       |none  |     0|acc_norm|0.7068|±  |0.0045|


|  Groups  |Version|Filter|n-shot|  Metric   | Value |   |Stderr|
|----------|-------|------|-----:|-----------|------:|---|-----:|
|truthfulqa|N/A    |none  |     0|acc        | 0.3411|±  |0.0016|
|          |       |none  |     0|bleu_max   |19.4174|±  |0.6888|
|          |       |none  |     0|bleu_acc   | 0.3378|±  |0.0166|
|          |       |none  |     0|bleu_diff  |-4.4165|±  |0.6611|
|          |       |none  |     0|rouge1_max |43.6923|±  |0.8239|
|          |       |none  |     0|rouge1_acc | 0.3305|±  |0.0165|
|          |       |none  |     0|rouge1_diff|-6.4023|±  |0.7680|
|          |       |none  |     0|rouge2_max |28.4074|±  |0.8883|
|          |       |none  |     0|rouge2_acc | 0.2827|±  |0.0158|
|          |       |none  |     0|rouge2_diff|-6.7716|±  |0.8844|
|          |       |none  |     0|rougeL_max |40.2657|±  |0.8218|
|          |       |none  |     0|rougeL_acc | 0.3023|±  |0.0161|
|          |       |none  |     0|rougeL_diff|-6.5447|±  |0.7706|

|----------|-------|------|-----:|------|-----:|---|-----:|
|winogrande|Yaml   |none  |     0|acc   |0.6464|±  |0.0134|

|-------------|-------|------|-----:|--------|-----:|---|-----:|
|arc_challenge|Yaml   |none  |     0|acc     |0.3652|±  |0.0141|
|             |       |none  |     0|acc_norm|0.3908|±  |0.0143|

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	41.13
AI2 Reasoning Challenge (25-Shot)	40.19
HellaSwag (10-Shot)	70.08
MMLU (5-Shot)	28.12
TruthfulQA (0-shot)	41.23
Winogrande (5-shot)	65.04
GSM8k (5-shot)	2.12

Downloads last month: 6

Safetensors

Model size

3B params

Tensor type

F16

Model tree for mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1

Base model

vihangd/shearedplats-2.7b-v2

Finetuned

(1)

this model

Dataset used to train mwitiderrick/shearedplats-2.7b-v2-instruct-v0.1

Evaluation results

hellaswag(0-Shot) on hellaswag
self-reported

0.528
winogrande(0-Shot) on winogrande
self-reported

0.646
arc_challenge(0-Shot) on arc_challenge
shearedplats-2.7b-v2-instruct-v0.1 model card

0.365
normalized accuracy on AI2 Reasoning Challenge (25-Shot)
test set Open LLM Leaderboard

40.190
normalized accuracy on HellaSwag (10-Shot)
validation set Open LLM Leaderboard

70.080
accuracy on MMLU (5-Shot)
test set Open LLM Leaderboard

28.120
mc2 on TruthfulQA (0-shot)
validation set Open LLM Leaderboard

41.230
accuracy on Winogrande (5-shot)
validation set Open LLM Leaderboard

65.040