Instructions to use PromptExecutionAU/wattson-qwen3.5-4b-pyt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PromptExecutionAU/wattson-qwen3.5-4b-pyt with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="PromptExecutionAU/wattson-qwen3.5-4b-pyt") messages = [ { "role": "user", "content": [ {"type": "image", "url": "/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("PromptExecutionAU/wattson-qwen3.5-4b-pyt") model = AutoModelForMultimodalLM.from_pretrained("PromptExecutionAU/wattson-qwen3.5-4b-pyt", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use PromptExecutionAU/wattson-qwen3.5-4b-pyt with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "PromptExecutionAU/wattson-qwen3.5-4b-pyt" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PromptExecutionAU/wattson-qwen3.5-4b-pyt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/PromptExecutionAU/wattson-qwen3.5-4b-pyt
- SGLang
How to use PromptExecutionAU/wattson-qwen3.5-4b-pyt with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "PromptExecutionAU/wattson-qwen3.5-4b-pyt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PromptExecutionAU/wattson-qwen3.5-4b-pyt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "PromptExecutionAU/wattson-qwen3.5-4b-pyt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PromptExecutionAU/wattson-qwen3.5-4b-pyt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use PromptExecutionAU/wattson-qwen3.5-4b-pyt with Docker Model Runner:
docker model run hf.co/PromptExecutionAU/wattson-qwen3.5-4b-pyt
wattson-qwen3.5-4b-pyt
Qwen3.5-4B fine-tuned (LoRA, merged to bf16) with scripted thinking to answer AC power-flow questions by writing Python that
calls a solver, and never computing numbers itself. Part of WattsOn / wattson.
The pyt suffix is Python shell plus thinking; its non-thinking sibling recipe is described in docs/results.md.
What it does
Same task family and run_python shell as wattson-granite-4.2-8b-py: fits, N-1 reports, worst-of-three planning, refusals of
non-existent elements, and derived questions (a sum, difference or mean over several solver results).
Evaluation (held-out networks; three training seeds: 42, 43, 44)
| Suite | Seed 42 | Seed 43 | Seed 44 |
|---|---|---|---|
| Held-out fits and reports (90) | 90 | 89 | 85 |
| Smoke (20) | 20 | 20 | 20 |
| Judgement (40) | 40 | 40 | 40 |
| Unsolvable (28) | 26 | 19 | 26 |
| Planning (40) | 40 | 40 | 40 |
| Derived arithmetic (40) | 39 | 29 | 38 |
| Derived: attempts with a number no tool or prompt gave the model | 1 | 3 | 1 |
These are the seed-42 weights. Seed-to-seed spread is large on the unsolvable and derived suites. The untrained Qwen3.5-4B with
the same shell scores 35 of 40 on the derived suite; the same recipe without thinking scores 31, 30, 27. See docs/results.md.
Training
- Base:
Qwen/Qwen3.5-4Bat revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a(Apache-2.0). - Data: 826 oracle-scripted trajectories whose reasoning is scripted prose generated by a deterministic program (every number in a thought is already in the prompt or an earlier tool reply), not model-generated reasoning.
- LoRA r=32, alpha=64; 2 epochs; learning rate 1e-4 cosine; sequence length 8192; supervised-tail loss; seed 42.
- Weight integrity: SHA-256 of every file is listed in
hash_manifest.jsonin this repository.
Intended use and limits
- For the WattsOn harness and research on tool-grounded numerical reasoning, not for operational power-system decisions.
- The Python shell it calls is not a security boundary (restricted
python -Ichild; allowlist; time and memory limits). - Thinking is not free of the failure it is meant to avoid: 5 attempts across three seeds wrote a number nothing had provided, where the non-thinking recipe wrote none.
- This model is released as evidence for the recipe; the preferred base for new work is Granite 4.2 8B
(
wattson-granite-4.2-8b-py).
- Downloads last month
- 414