I'm testing using nemotron3 super(nvfp4 120b).
I was using vllm with several llms composed, but I decided to try just one for a long time. Recently, I heard that 120b is available, so I set it up and tried it.
Model loading and response are successful. It's great that it works smoothly without any problems. It outputs 15 tokens per second.
I was thinking about the usage environment, and decided to make it an app. I started making it with rust, but then I decided to go web so that I could access it from both mobile and PC while keeping spark running for session maintenance. The structure is to use only claude and codex to design and create work instructions, and leave the coding to nemo.
Claude finished coding the app. It works when I tested it. Now for load testing.
Huh?
It died. It just shut down. What's going on? I reproduced the error 4 times with temperature measurement, and I don't know where the problem is - cpu/gpu lane, or insufficient power. I'm testing all possible methods. I left the testing to Claude, and it seems to be doing a lot of tests.
The culprit has been narrowed down to power. Is it an adapter problem?
Hmm... Could there be a problem with the RAM? Usually, you can fix these things by removing the RAM from the motherboard and wiping it with an eraser, but I can't do that here. I'm running a load test on VLLM with only cpu and ram.
It hasn't died even after 20 minutes? Wow. I need to expand the tests. I'll suggest a few methods to conduct broader testing. It's gratifying to see the behavior of a superstar AI with limitations in deductive reasoning, as a human (ADHD) myself.
Aha!
I caught the culprit. There seems to have been a bug in the vllm build I was using. I was running it on a spark build. I updated to the latest version and it hasn't died yet.
I'm going to run a 60-minute test, and it's been 49 minutes and it's still alive. The green light is on. I enjoyed watching Green Lantern, but this is even more fun.
See you again. Bye bye.
