I'm trying out the Gemma4-12B-QAT-Q4 model for the first time today.
It's nice that it doesn't use up a lot of VRAM capacity,
but it's a bit awkward on my RTX 5060Ti 16GB.
With the Gemma4-26B-A4B-QAT-Q4 model, 16GB is a bit tight on memory,
and I even have to allocate some layers to main memory,
yet its speed turned out to be better than the Gemma4-12B-QAT-Q4 model.
I guess that's the difference between a Dense model and a MoE model?
This might vary depending on the conversation topic, but in my first comparison test,
Gemma4-26B-A4B-QAT-Q4 was slightly faster.
So there's no point in using the Gemma4-12B model on my PC.
There seems to be significant progress in multimodal-related features, so it would probably be useful for leveraging that,
or in special situations like when the system is running heavy due to other programs,
that's when it would be used.
Among the various models, the smartest one that runs at a usable speed on my PC
seems to be Qwen3.6-35B-A3B.
