wattson-qwen3.5-4b-pyt

Qwen3.5-4B fine-tuned (LoRA, merged to bf16) with scripted thinking to answer AC power-flow questions by writing Python that calls a solver, and never computing numbers itself. Part of WattsOn / wattson. The pyt suffix is Python shell plus thinking; its non-thinking sibling recipe is described in docs/results.md.

What it does

Same task family and run_python shell as wattson-granite-4.2-8b-py: fits, N-1 reports, worst-of-three planning, refusals of non-existent elements, and derived questions (a sum, difference or mean over several solver results).

Evaluation (held-out networks; three training seeds: 42, 43, 44)

Suite Seed 42 Seed 43 Seed 44
Held-out fits and reports (90) 90 89 85
Smoke (20) 20 20 20
Judgement (40) 40 40 40
Unsolvable (28) 26 19 26
Planning (40) 40 40 40
Derived arithmetic (40) 39 29 38
Derived: attempts with a number no tool or prompt gave the model 1 3 1

These are the seed-42 weights. Seed-to-seed spread is large on the unsolvable and derived suites. The untrained Qwen3.5-4B with the same shell scores 35 of 40 on the derived suite; the same recipe without thinking scores 31, 30, 27. See docs/results.md.

Training

  • Base: Qwen/Qwen3.5-4B at revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a (Apache-2.0).
  • Data: 826 oracle-scripted trajectories whose reasoning is scripted prose generated by a deterministic program (every number in a thought is already in the prompt or an earlier tool reply), not model-generated reasoning.
  • LoRA r=32, alpha=64; 2 epochs; learning rate 1e-4 cosine; sequence length 8192; supervised-tail loss; seed 42.
  • Weight integrity: SHA-256 of every file is listed in hash_manifest.json in this repository.

Intended use and limits

  • For the WattsOn harness and research on tool-grounded numerical reasoning, not for operational power-system decisions.
  • The Python shell it calls is not a security boundary (restricted python -I child; allowlist; time and memory limits).
  • Thinking is not free of the failure it is meant to avoid: 5 attempts across three seeds wrote a number nothing had provided, where the non-thinking recipe wrote none.
  • This model is released as evidence for the recipe; the preferred base for new work is Granite 4.2 8B (wattson-granite-4.2-8b-py).
Downloads last month
414
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for PromptExecutionAU/wattson-qwen3.5-4b-pyt

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,557)
this model
Quantizations
1 model