← Leaderboard
Qwen3.8-27B MLX 8bit (lmstudio-community)
Retired Rank #53 of 53 · 2/8 GAUNTLET progress
Hollow markers & dashed spokes: axis not yet scored
G
—
A
97.5
U
—
N
—
T
n/d
L
—
E
—
T
5
Specification
- Parameters
- 27B dense
- Architecture
- qwen3_5
- Size on disk
- 28 GB
- Quantization
- 8bit
- Format
- MLX
- Runtime
- LM Studio
- Class
- Standard
- Reasoning (CoT)
- Yes — emits reasoning tokens
- Internal ID
- M25g
- Mean speed
- 17.1 tok/s across suites
- Model card
- huggingface.co
Suite results
Suite Score Avg / 20 Tests n tok/s Run
Agentic tool-calling & protocol adherence 1b 156 / 160 19.5 8/8 n=1 17.1 run 2026-09-12
| Test | Category | Score | tok/s | |
|---|---|---|---|---|
| WA1 | Tool Calling & Protocol Adherence | 20/20 | 15.1 | |
| WA2 | Tool Calling & Protocol Adherence | 20/20 | 15.8 | |
| WA3 | Tool Calling & Protocol Adherence | 20/20 | 16.4 | |
| WA4 | Tool Calling & Protocol Adherence | 20/20 | 14.9 | |
| WA5 | Tool Calling & Protocol Adherence | 20/20 | 17.4 | |
| WB1 | Agent Robustness & State Management | 20/20 | 15.5 | |
| WB2 | Agent Robustness & State Management | 20/20 | 15.5 | |
| WB3 | Agent Robustness & State Management | 16/20 | 16.9 |
Category mean Tool Calling & Protocol Adherence 20Agent Robustness & State Management 18.7
Each test is LLM-judged 0–20 against a fixed rubric; suite max = tests × 20. An ADJ chip marks a cell carrying a subjective adjustment from the maintainer; the judged score is the one shown, is unchanged, and remains what the axes, ranks and tie bands are computed from. Hover for the reason. Rows with a ▶ expand to the per-test breakdown. Where fewer tests ran than the suite total, unrun tests count as zero toward the axis score. See methodology.