iski-lora

LoRA adapter trained on İSKİ (İstanbul Su ve Kanalizasyon İdaresi) corporate Q&A data for domain-specific Turkish language understanding.

Model Information

Field	Value
Model type	PEFT LoRA Adapter
Base model	unsloth/Qwen3.6-35B-A3B
Framework	Transformers + PEFT (Unsloth)
Precision	bfloat16
LoRA rank	r=32, alpha=64
LoRA target modules	q_proj, k_proj, v_proj, o_proj
LoRA dropout	0
Training samples	~7,100 İSKİ Q&A pairs
Language	Turkish
Domain	Municipal water & sewerage administration

Training Configuration

Parameter	Value
Epochs	3
Learning rate	5e-5
LR scheduler	cosine
Optimizer	paged_adamw_8bit
Batch size (per device)	8
Gradient accumulation	4 (effective batch: 32)
Weight decay	0.01
Warmup ratio	0.03
Max sequence length	4096
Early stopping patience	2
Best model metric	eval_loss

Data Pipeline

Dataset is split using GroupShuffleSplit (group key: source_file) with a 95/5 train/eval ratio to prevent data leakage across documents from the same source.

Each training sample is formatted as a three-turn conversation using the Qwen3 chat template:

System    → domain context (category, source_file, source_year)
User      → instruction [+ optional input]
Assistant → expected output

Loss is computed only on assistant responses (train_on_responses_only).

Usage

Load the Adapter

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "unsloth/Qwen3.6-35B-A3B"
adapter_id = "iskiai/iski-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Inference

SYSTEM_BASE = "Sen kurum ici dokumanlar konusunda uzman, yardimci bir asistansin."

def build_system_message(category="", source_file="", source_year=""):
    return (f"{SYSTEM_BASE} Baglam Bilgisi - Kategori: {category}, "
            f"Kaynak: {source_file} ({source_year}).")

messages = [
    {"role": "system", "content": build_system_message()},
    {"role": "user",   "content": "İSKİ su aboneliği nasıl açılır?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    # Qwen3: thinking mode kapatılmalı
    chat_template_kwargs={"enable_thinking": False}
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        use_cache=True,
    )

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

⚠️ Thinking Mode: Qwen3 modelleri varsayılan olarak chain-of-thought düşünme modunu etkinleştirir. Üretim ortamında enable_thinking: false set edilmezse yanıtlar <think>...</think> bloğu içerebilir.

Serving with vLLM

vllm serve unsloth/Qwen3.6-35B-A3B \
  --enable-lora \
  --lora-modules iski-lora=iskiai/iski-lora \
  --max-lora-rank 32 \
  --max-model-len 8192 \
  --port 5001

from openai import OpenAI

client = OpenAI(base_url="http://localhost:5001/v1", api_key="dummy")

response = client.chat.completions.create(
    model="iski-lora",
    messages=[
        {"role": "system", "content": build_system_message()},
        {"role": "user",   "content": "Su sayacı arızası nasıl bildirilir?"},
    ],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}}
)
print(response.choices[0].message.content)

Files

iski-lora/
├── adapter_model.safetensors   # LoRA weights
├── adapter_config.json         # PEFT configuration
├── tokenizer_config.json
└── README.md

Training Outputs

The training script produces the following artifacts alongside the adapter:

Artifact	Description
egitim_loglari.jsonl	Step-by-step training logs (loss, eval_loss, lr, epoch) in JSON Lines format
benchmark_sonuclari.json	Final metrics: perplexity, VRAM usage, sample generations
grafikler/	Per-metric PNG plots + combined summary dashboard
runs/	TensorBoard logs (local, no external connection required)

View training curves locally:

tensorboard --logdir runs

Evaluation

Final model quality is assessed by:

1. Perplexity on the held-out eval split (computed from eval_loss)
2. Sample generations — 5 randomly drawn eval examples are generated greedily and saved to benchmark_sonuclari.json for manual inspection
3. DeepEval GEval — downstream evaluation using a vLLM-backed judge model

License

This repository contains a LoRA adapter trained on the Qwen3.6-35B-A3B base model. Users must obtain and use the base model in accordance with its original license. This adapter is distributed under the Apache 2.0 License.

Author

Created by İSKİ AI Team · iskiai