SHARED WEIGHTS / EXPERIMENT NOTEBOOK

Hub post preview

Local preview of the publication draft. Hugging Face typography may differ.

Serving Decision Workloads with LLMs

One checkpoint, two paths: score choices from logits or generate chat. We evaluated SGLang across four models without fine-tuning or separate decision weights.

BFCL-derived tool selection: correct / attempted; median seconds over valid outputs.

Checkpoint / weights Decision correct Chat-key correct Chat-JSON correct Chat-JSON valid Decision median (s) Chat-key median (s) Chat-JSON median (s)
Qwen3.5-9B · BF16 29/30 29/30 6/30 6/30 0.224 0.246 3.553
Gemma-4-A4B · BF16 30/30 29/30 29/30 30/30 0.291 0.304 1.927
GPT-OSS-120B · mixed MXFP4, experimental 27/30 29/30 24/30 25/30 0.297 1.251 3.057
Nemotron 3.5 Super VL · mixed NVFP4 30/30 30/30 13/30 13/30 0.431 0.535 2.704

Decision JSON vs chat JSON choice agreement (valid pairs only):

Checkpoint Decision / chat-JSON choice agreement Valid-pair coverage
Qwen3.5-9B · BF16 100.0% (6/6) 6/30 (20.0%)
Gemma-4-A4B · BF16 96.7% (29/30) 30/30 (100.0%)
GPT-OSS-120B · mixed MXFP4, experimental 88.0% (22/25) 25/30 (83.3%)
Nemotron 3.5 Super VL · mixed NVFP4 100.0% (13/13) 13/30 (43.3%)

Agreement includes shared wrong answers, not equal probabilities. Low coverage matters. GPT-OSS uses experimental forced-final scoring that skips reasoning. These are small subsets, not official leaderboard scores.

Results and request logs · Full article · SGLang prefill-only work