benchmark note - Local Model Bench

Local Model Bench

Sign in Subscribe

benchmark note

A collection of 2 posts

Bigger was not always better in the paperwork benchmark

Bigger was not always better in the paperwork benchmark

In the current nine-case paperwork suite, two larger-sounding local model choices produced fewer finished cases than leaner alternatives. That does not prove small models are better. It shows that parameter count and model labels a

Gemini 3.5 Flash can draw SVG. The runner was the problem.

Gemini 3.5 Flash can draw SVG. The runner was the problem.

A first Gemini 3.5 Flash SVG run looked like a model failure. It was not. The OpenAI-compatible API path let hidden thinking consume the output budget, so the model returned planning fragments instead of a complete SVG. The native Gemini API with thinkingBudget: 0 produced a valid City Plan SVG and