The Qwen3.8-27B Buyer's Guide, Part 1: What Compression Actually Costs
TLDR: Compressing this model harder did not make it worse. From 8-bit down to 4-bit, across every test we could score, the quality spread was 0.33 points on a 20-point scale — noise. The 30 GB build scored below the 15 GB one. If you are choosing a Qwen3.8-27B today, pick on the size that fits your RAM, not on the quant.
ELI5: A big AI model gets squeezed to fit on your computer, and there are several squeeze settings. Everyone assumes squeezing harder makes it dumber. We tested that on fourteen versions of the same model and it mostly isn’t true — the small squeezed one answered our questions about as well as the big one. This part covers how smart each version is. How fast each one is turns out to be a harder question to answer honestly, so it gets its own article.
Why this model gets a series
Qwen3.8-27B is the model most often handed to someone setting up their first local LLM, and the recommendation almost never says which Qwen3.8-27B. There are at least fourteen distinct packagings of these same weights. On this bench they have ranged from 15 GB to 30 GB on disk — a 2x spread in what you download and in how much memory you give up — for a model that, as you will see, answers about the same either way.
Every number here came off one machine: a MacBook Pro M5 Max with 128 GB of unified memory, running the same prompts through the same judge.
A note on what “the same model” means. All of these share one set of trained weights. What differs is packaging: the file format (GGUF or MLX), how aggressively the numbers inside were rounded off to save space (the quant), and who did the rounding. Nobody retrained anything. That is what makes this a fair test of compression specifically.
The builds
| Build | Format / quant | On disk | 1A score | Coverage | 1B score |
|---|---|---|---|---|---|
| Q6_K GGUF | GGUF Q6_K | 23.4 GB | 19.8 | 13/13 | 19.2 |
| MLX 4-bit | MLX 4bit | 15.0 GB | 19.5 | 11/13 | 20.0 |
| MLX 5-bit | MLX 5bit | 19.0 GB | 19.4 | 10/13 | 20.0 |
| Q4_K_M GGUF | GGUF Q4_K_M | 17.7 GB | 19.1 | 13/13 | 20.0 |
| UD-Q6_K_XL (Unsloth, on Ollama) | GGUF Q6_K_XL | 25.3 GB | 19.1 | 13/13 | 20.0 |
| Q8_0 GGUF | GGUF Q8_0 | 30.0 GB | 18.8 | 13/13 | 19.1 |
| nvfp4 MLX (on Ollama) | MLX nvfp4 | 16.5 GB | 17.4 | 13/13 | 20.0 |
| MLX 6-bit | MLX 6bit | 21.0 GB | withheld | 9/13 | 20.0 |
| MLX 8-bit | MLX 8bit | 28.0 GB | withheld | 9/13 | 20.0 |
| Uncensored Q5_K_P | GGUF Q5_K_P | 18.8 GB | withheld | 3/13 | — |
| Uncensored Q6_K_P | GGUF Q6_K_P | 24.1 GB | no score | 0/13 | — |
Scores are out of 20, judged against a fixed rubric. Coverage is how many of suite 1A’s 13 tests actually produced an answer to judge. A build under 75% coverage gets no published score, because an average over a handful of tests is not a score, it is an anecdote. Three builds are in that state and are shown here as withheld rather than quietly dropped.
Finding 1: the quant ladder is flat
This is the headline, and it was not the expected result.
Five builds were raced head-to-head on one night, on one engine, over an identical subset of tests — the only way to compare them without confounds. The scores:
| MLX 8-bit | MLX 6-bit | MLX 5-bit | MLX 4-bit | Q6_K GGUF |
|---|---|---|---|---|
| 19.50 | 19.67 | 19.33 | 19.67 | 19.67 |
A 0.33-point spread on a 20-point scale. That is one judge point on one question. There is no ladder here — halving the file from 8-bit to 4-bit cost nothing we could measure, and the 4-bit build tied the reference exactly.
The going assumption in local-LLM communities is that quantization is a dial trading quality for size, and that you should stay as high as you can afford. On this model, in this size class, on these tests, that dial does nothing until at least 4-bit. Stop agonising over the quant. Pick what fits comfortably in RAM and spend the attention elsewhere.
Finding 2: the biggest file is not the best model
The Q8_0 GGUF is the largest build here at 30 GB, and it scored 18.8 — below the 23 GB Q6_K (19.8), below the 17.7 GB Q4_K_M (19.1), and below the 15 GB MLX 4-bit (19.5). All four have full or near-full coverage, so this is not a measurement artifact.
“Download the biggest one you can fit” is common advice. Here it costs you 15 GB and buys you nothing.
Finding 3: the program that serves the model can change the score
Correction, October 7, 2026. This section originally said the Unsloth UD-Q6_K_XL build completed zero of thirteen tests under LM Studio and all thirteen under Ollama, and concluded that the runtime matters more than the quant. A re-run does not support that. We have rewritten it.
Take the Unsloth UD-Q6_K_XL build. The identical file was served by two different programs on the same laptop. In August, an LM Studio run of it failed every attempt mid-stream. When we re-ran it on October 7, LM Studio completed all thirteen tests on the first attempt, with no stream deaths. So the August failure was not a property of LM Studio serving this file. We have not pinned down its cause. Our test rig had a bug in that period that killed generations right after a server restart, which is the likeliest culprit, but we have not proven it for that run.
What did differ is the score: 17.8 on LM Studio, 19.1 on Ollama. The cause is a setting, not the program’s speed or stability. This model has a reasoning effort switch, and our tests ask for medium. LM Studio did not apply it to this file, which was downloaded by hand rather than from LM Studio’s own catalog, so the model ran at its most verbose default. On several tests it spent its whole 16,383-token budget thinking and never wrote an answer. Where it did answer, it scored about the same as on Ollama. We wrote up the mechanism and a one-minute test you can run yourself in LM Studio Silently Drops Reasoning Effort on Sideloaded Models.
The lesson that survives is narrower than the one we first drew. The serving program is not a neutral pipe: it can quietly change what the model is told. In this case the gap between programs (1.3 points) was larger than the entire spread across quants (0.33 points), but only because of one setting that one program did not pass along. Any number that depends on the serving program, above all throughput and anything involving a reasoning model, has to be measured with the program held fixed and the rendered prompt checked. Which brings us to what this article deliberately does not tell you.
What Part 1 will not tell you: speed
You may have noticed there is no tokens-per-second column above. That is deliberate, and it is worth explaining rather than hiding.
We have speed numbers for every build on that table. They are not comparable to each other. The GGUF builds were last timed in mid-August; the MLX builds were timed in early September, after the underlying inference engines had been updated. Engine builds change throughput on their own — that is most of what they are for — so a table ranking August GGUF numbers against September MLX numbers would be measuring the calendar as much as the model.
We know precisely how bad this is, because the one thing we did measure cleanly shows the size of the effect: in the five-build race above, run on a single night on a single engine, throughput ranged from 9.1 to 22.9 tokens per second — a 2.5x spread across builds of the same model. An effect that large cannot be reported from mixed-era measurements and called a finding.
Worse, the August runs are unattributable in a specific way: their run manifests record no engine version at all, because they predate the instrumentation that captures it. It is not that we know they ran on an old engine and want a newer one. It is that those numbers cannot be assigned to any engine, so they can never be honestly ranked against numbers that can.
This bench has a standing rule against publishing comparisons that span measurement eras without a crosswalk run to join them. Part 1 obeys it by saying nothing about speed.
What Part 2 will do
Part 2 is the crosswalk, and it is already queued.
The run. Five builds — the surviving GGUF rung (Q6_K) and all four MLX rungs (4, 5, 6 and 8-bit) — re-run on suite 1A in one pass, on one engine, on one night. Same prompts, same judge, same machine, nothing else moving. That produces five throughput figures that can legitimately sit in one column, plus first-token latency, which behaves differently from throughput and matters more than people expect for chat-style use.
Two gaps it closes on the way. The MLX 6-bit and 8-bit builds are the two withheld rows above, sitting at 9 of 13 tests; a full re-run restores them to the quality table as well. And the MLX 4-bit build — the one Part 1 recommends on quality — is scored on 11 of 13, where the two missing tests are a Python debugging task and a multi-step arithmetic task. Those are precisely the two most likely to expose damage from aggressive rounding, and they failed for serving-layer reasons rather than model ones. So “4-bit costs nothing” is currently established on the easy and medium questions and untested on the hardest two. Part 2 answers that too.
One gap it cannot close. The Q4_K_M and Q8_0 GGUF builds cannot be re-timed, because their weights are no longer on this machine — only Q6_K survives in that directory. Their quality scores in the table above stand, since those were full-coverage runs and quality is far less engine-sensitive than speed. But they will carry no comparable speed figure unless they are downloaded again, and that is a deliberate decision rather than something to do by reflex.
So: Part 1 is the quality answer, and it is a real one. Part 2 is the speed answer, and it will be worth having precisely because we refused to fake it here.
All figures from the GAUNTLET bench: MacBook Pro M5 Max, 128 GB. Suite 1A is a 13-test general-capability set; 1B is an 8-test agentic set. Judge scores are Claude at temperature 0 against a fixed rubric (v2). Transient stream failures are recorded as did-not-finish and excluded from averages rather than counted as zeros — a distinction that changed several previously published numbers. See the methodology pages for why.