https://huggingface.co/jcbtc/Qwen3.8-27B-IU4-Kairic-Edge
I thought INT4 acceleration would only be available on DGX Spark,
but there's also IU4 for RDNA3.5.
Roughly twice the performance of FP16 / FP8 (which is about 1/3 of Spark NV4)
It seems to be quite difficult to use.
That's why there are a few things attached next to the model, and llamacpp is not just a fork level but something more complex.
I built it, but since it's structured so that it can't be used with frontends like lemonade, I just gave up.
If you only need a text-only LLM Qwen 3.8 27B environment, it seems like it would be okay.
It's said to have reached 47 t/s... ㄷ ㄷ ㄷ
Even if it's made, it's not properly supported, it's complicated, and as expected, ROCm....
