Instructions to use nmitchko/dr-niko-70B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nmitchko/dr-niko-70B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nmitchko/dr-niko-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nmitchko/dr-niko-70B")
model = AutoModelForCausalLM.from_pretrained("nmitchko/dr-niko-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use nmitchko/dr-niko-70B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nmitchko/dr-niko-70B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nmitchko/dr-niko-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nmitchko/dr-niko-70B

SGLang

How to use nmitchko/dr-niko-70B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nmitchko/dr-niko-70B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nmitchko/dr-niko-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nmitchko/dr-niko-70B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nmitchko/dr-niko-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nmitchko/dr-niko-70B with Docker Model Runner:
```
docker model run hf.co/nmitchko/dr-niko-70B
```

Dr. Niko (70B)

This repository contains the full merged model.

The Dr-Niko model is designed to assist medical professionals, researchers, and students with a wide range of tasks, including answering medical and scientific questions, summarizing research papers and clinical notes, generating medical reports and documentation, providing medical advice and recommendations (with appropriate disclaimers), and assisting with medical decision-making and diagnosis (in a supporting role).

Model Details

Model Name: Dr-Niko
Model Type: Medical Large Language Model (LLM)
Model Size: 70 billion parameters (base model is miqu-70B)
Training Data: The model was fine-tuned on a curated dataset of high-quality medical and scientific literature.
Fine-Tuning Approach: The model was then fine-tuned on a medical and scientific dataset using LLaMa-Factory for 1.5 epochs.
Intended Use: The Dr-Niko model is designed to assist medical professionals, researchers, and students with a wide range of tasks, including:
- Answering medical and scientific questions
- Summarizing research papers and clinical notes (in a supporting role)
- Generating medical reports and documentation (in a supporting role)
- Providing medical advice and recommendations (with appropriate disclaimers)
- Assisting with medical decision-making and diagnosis (in a supporting role)

Model Description

This model is next in a series of medical finetuning attempts, following medfalcon and medguanaco.

Developed by: Nick Mitchko
Funded by : [My Bank Account]
Shared by : [My Internet]
Model type: [LLaMa-70B variant]
Language(s) (NLP): [English]
License: [See here]
Finetuned from model [optional]: Miqu-70B

Examples

Model Sources

Repository: [Coming Soon]
Paper []: [Maybe]

Demo []: Maybe Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed] -->

Metrics

[Formal Evaluation Coming Soon]

Citation [optional]

Information Coming Soon

License - NOMERGE

NOMERGE License

Copyright (c) 2024 152334H

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, NOT merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

All tensors ("weights") provided by the Software shall not be conjoined with
other tensors ("merging") unless given explicit permission by the license holder.
Utilities including but not limited to "mergekit", "MergeMonster", are forbidden
from use in conjunction with this Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Downloads last month: 4

Safetensors

Model size

69B params

Tensor type

BF16

F32

Model tree for nmitchko/dr-niko-70B

Base model

152334H/miqu-1-70b-sf

Quantized

(14)

this model

Paper for nmitchko/dr-niko-70B

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 47