Update README.md

README.md

@@ -37,7 +37,62 @@ repo_path = snapshot_download("RobbiePasquale/lightbulb")
 
 print(f"Repository downloaded to: {repo_path}")
 ```
+### 0. Distill Large model into your own small model
 
+## Minimal quick testing
+
+```bash
+python main_menu_new.py \
+    --task distill_full_model \
+    --teacher_model_name gpt2 \
+    --student_model_name distilgpt2 \
+    --dataset_name wikitext
+```
+
+## Full Distillation
+
+```bash
+python main_menu_new.py \
+    --task distill_full_model \
+    --teacher_model_name gpt2 \
+    --student_model_name distilgpt2 \
+    --dataset_name wikitext \
+    --config wikitext-2-raw-v1 \
+    --num_epochs 5 \
+    --batch_size 8 \
+    --max_length 256 \
+    --learning_rate 3e-5 \
+    --temperature 2.0 \
+    --save_path ./distilled_full_model \
+    --log_dir ./logs/full_distillation \
+    --checkpoint_dir ./checkpoints/full_distillation \
+    --early_stopping_patience 2
+```
+
+## Domain Specific Distillation
+
+Use domain specific distillation to distill the part of the model relevant for you- if you like how llama 3.1 7B responds to healthcare prompts for example, you could use:
+
+```bash
+python main_menu_new.py \
+    --task distill_domain_specific \
+    --teacher_model_name gpt2 \
+    --student_model_name distilgpt2 \
+    --dataset_name wikitext \
+    --config wikitext-2-raw-v1 \
+    --query_terms healthcare medicine pharmacology \
+    --num_epochs 5 \
+    --batch_size 8 \
+    --max_length 256 \
+    --learning_rate 3e-5 \
+    --temperature 2.0 \
+    --save_path ./distilled_healthcare_model \
+    --log_dir ./logs/healthcare_distillation \
+    --checkpoint_dir ./checkpoints/healthcare_distillation \
+    --early_stopping_patience 2
+
+
+```
 
 ### 1. Train a Web Search Agent