기존 사용하던 qwen3.5-122b가 생각보다 멍청하고 램을 많이 사용해서 여유램이 부족하니 다른걸 찾아 봐야지 하고 찾아 봤슴미다
허깅페이스에 nvidia에서 개조한 버전이 올라왔는데
qwen3.6-27b dense , qwen3.6-35b-a3b 이렇게 임미다
처음 덴스모델이 좋다고 해서 다운받고 사용해 봤는데... 넘넘 느리네요
그래서 다시 122b로 가야 하나 하다가 35b가 비슷한 성능이라고 해서 일단 다운받았는데 스파크용 옵션이 있슴미다
그래서 다 적용하고 vllm에 올리고 토큰생석속도를 측정해 보니
| 항목 | 27B (이전) | 35B-A3B (현재) |
| ---- | -------- | ------------ |
| 컨텍스트 | 131K | 262K |
| VRAM | 98GB | 22GB |
| RAM | 109GB | 79GB (캐시 포함) |
| 속도 | 9 tok/s | 105 tok/s |
대충 이정도 나옴미다 105토큰은 오픈코드 적용 안 한 상태에서 한거라 실제로 오픈코드 적용한건 더 낮은 토큰생성이 나오는데 그래도 80개 이상은 나와줌미다
이정도면 실사용에 아무런 문제가 없죠
스파크 2대면 딥싴4플래시를 운용 가능하다던데.... 미니맥스3도 된다던데... 부럽요
Model qwen3.5-122b yang saya gunakan sebelumnya ternyata tidak sebaik yang saya bayangkan dan menggunakan banyak RAM, sehingga memori tersisa tidak cukup, jadi saya mencari alternatif lain
Ada versi yang dimodifikasi oleh Nvidia di Hugging Face
qwen3.6-27b dense, qwen3.6-35b-a3b seperti ini
Pada awalnya saya mendengar model dense itu bagus, jadi saya mengunduhnya dan mencobanya... tapi sangat lambat
Jadi saya berpikir apakah harus kembali ke 122b, tapi karena saya mendengar 35b memiliki performa serupa, saya mengunduhnya terlebih dahulu, dan ternyata ada opsi untuk Spark
Jadi saya menerapkan semuanya, mengunggahnya ke vllm, dan mengukur kecepatan generasi token
| Item | 27B (Sebelumnya) | 35B-A3B (Sekarang) |
| ---- | -------- | ------------ |
| Konteks | 131K | 262K |
| VRAM | 98GB | 22GB |
| RAM | 109GB | 79GB (termasuk cache) |
| Kecepatan | 9 tok/s | 105 tok/s |
Kurang lebih hasilnya seperti ini. 105 token adalah tanpa menerapkan OpenCode, jadi ketika OpenCode diterapkan, generasi token akan lebih rendah, tetapi masih menghasilkan lebih dari 80
Pada tingkat ini, tidak ada masalah untuk penggunaan praktis
Katanya dengan 2 Spark bisa menjalankan DeepSeek 4 Flash.... katanya juga bisa MinMax 3.... iri saya
The existing qwen3.5-122b I was using seemed less intelligent than expected and consumed a lot of RAM, leaving insufficient free memory, so I decided to look for an alternative and searched around
A version modified by NVIDIA was uploaded to Hugging Face
qwen3.6-27b dense, qwen3.6-35b-a3b like this
At first, I heard the dense model was good, so I downloaded and tried using it... but it's very slow
So I was thinking about going back to 122b, but since I heard 35b had similar performance, I downloaded it first, and it has Spark options
So I applied everything and uploaded it to vLLM and measured the token generation speed
| Item | 27B (Previous) | 35B-A3B (Current) |
| ------- | -------------- | ----------------- |
| Context | 131K | 262K |
| VRAM | 98GB | 22GB |
| RAM | 109GB | 79GB (including cache) |
| Speed | 9 tok/s | 105 tok/s |
Roughly this is what I got. The 105 tokens were measured without OpenCode applied, so when OpenCode is actually applied, I get lower token generation, but it still produces more than 80
At this level, there's no problem with actual use
I heard that with 2 Sparks you can operate DeepSeek-4 Flash.... I heard MiniMax3 works too.... I'm jealous
