Does Memory Plus Skills Close the Gap for a Small LLM? A Live Mini-Test

Small models (7–8B) are cheap and run on modest hardware, but they hallucinate more and know less than big ones. My hypothesis was that most of that gap is knowledge access, not raw intelligence. If a small model is handed the facts and the proven procedure a big model would otherwise have to guess, it should answer operational questions correctly, faster, and without inventing things.

So I tested it, live, on a homelab box.

Read this as one illustrative run, not a benchmark. It’s a single comparison with a single question. The numbers show a pattern worth knowing about; they don’t establish how much of it will hold in general.

Setup

• Model: Qwen3-8B, Q4_K_M quantisation, served by llama.cpp on CPU only (OpenAI-compatible API)
• Question, identical in both runs: “I have a Proxmox host at 10.10.10.8 (root user). I want to see the status of every VM in this cluster. How do I do it? Give me the actual steps, not theory.” (Asked in Malay; translated here.)
• Settings: temperature 0.3, max_tokens 500

Results

The raw model opened with pct list, which is for LXC containers, not VMs. It then confidently fabricated sample output that doesn’t exist anywhere in the lab.

The augmented model’s complete answer was two commands, both correct, both directly runnable:

pvesh get /cluster/resources --type vm --output-format json
qm list

Why the difference

The raw run leaned on general training data, which blends LXC (pct) and VM (qm) tooling, knows nothing about this lab, and fills the gaps with plausible-looking invention. The classic small-model failure.

The augmented run didn’t need to know anything. The memory block supplied the environment and the skill block supplied the tested procedure, so the model’s job shrank from recall, reason and guess to select the right pre-verified answer. That’s why it generated 60 tokens instead of 500, and why it was both faster and correct.

An honest limit of this test: because the skill block contained the exact commands, B is closer to reading comprehension than to open-ended reasoning. That’s the point of the technique — but it also means this says little about tasks where no procedure exists yet.

What it suggests

Memory and skills replace the need to know, not the need to think. That matters because a lot of operational work is retrieval, not novel reasoning:

That table is my expectation, not something this test measured. I only compared the first row, and only against the same small model with no context. I did not test a larger model.

What I’d take from it: augmentation fixes knowledge access, not processing power. Size the model to the reasoning load, and carry the knowledge load in memory and skills — a cheap augmented model for routine ops, and a bigger one held back for the hard cases. That’s roughly how I run my own agent setup (see Packing an Agent into a Container).

To reproduce it: run the same prompt with and without the memory/skill blocks, against the same quantised model, a few times each, and compare latency, token count and whether the commands actually run.