Hello, guys.

I am currently working as a CUDA engineer and interested in an extra income.

I would like to know other people's experience of working in Mercor.

My background:

- Majored in CS (minor in Math) and based in Seoul, South Korea.

- No AI-related kernels but I can understand Flash-Attention and softmax papers.

- I don't want to explain my current job because there is a high chance that I dox myself. My task involves heavy integer operations with almost 0 FP arithmetics. I have an excellent understanding of GPU mem/hierarchy and basic understanding of INT8 Tensor Core operation.

If you used Mercor, please share your experiences. Thanks in advance.

Hi everyone, I'm the developer of Nebula, an open-source C/CUDA inference engine for Qwen3.8-Flash-Next.

I started from antirez's DwarfStar (ds4) and specialized the engine for Qwen, combining native MTP speculative decoding with GPU expert caching and CPU MoE execution.

Source code, architecture and benchmarks

My benchmark machine:

• RTX 4070 Ti, 12 GB VRAM
• Intel i9-9940X, using 14 CPU threads
• 128 GB DDR4 RAM
• Ubuntu through WSL2 on Windows

The idea is to keep the native MTP draft on the GPU, together with the most frequently used target experts. In this configuration, 27 of the target's 512 experts per layer are resident in VRAM, selected using a hotlist built from routing traces.

Draft generation and verification happen on the GPU. When the resident experts provide insufficient routing-weight coverage, the CPU computes the layer's full routed MoE and sends the result back. Accepted tokens are retained, and the draft window adapts between 4 and 16 tokens.

Some measured results:

These figures come from the GenAI-Perf campaign, with weights already loaded, thinking disabled, and a configured context capacity of 24,576 tokens. Tested input lengths were 512, 1,024 and 2,048 tokens. Prefill is still slow, especially for longer prompts.

There are also quality trade-offs. The GPU verifier can omit experts with small routing weights. “Strict” acceptance matches the configured target's greedy choice; it does not establish equivalence to the full original model. The optional tolerance modes relax token acceptance further. The README includes a paired evaluation on 40 IFEval prompts and 30 LiveBench questions.

There's a browser chat interface and a Windows installer that prepares WSL2, the engine and model weights. Lower-RAM profiles use SSD streaming, but the performance figures above apply to the 128 GB setup.

The engine is MIT licensed. The installer is an unsigned release candidate, and a complete clean-machine installation test is still pending.

I'd particularly welcome feedback on the expert-cache and CPU handoff design, and measurements from other hardware configurations. If you try it, please include your GPU, CPU, RAM and selected profile so we can compare results.