A
DENSE · 16 GB OF WEIGHTS
Qwen3-8B
Eight KV heads across 36 layers, so the cache stays cheap and decode bandwidth is what binds.
MODELS
Different models.
Different demands.
Meet the two model configurations used in Sera’s demo and the workloads that make co-residency worth testing.
INSIDE THE DEMO
Three open-weight models.
One card at a time.
Pick a model and Sera tunes how it is served on a single RTX Pro 6000 Blackwell: one lever per trial, every change measured, and anything that breaks the latency target or the quality floor reverted and recorded.
B
DENSE · 30 GB OF WEIGHTS
DeepSeek-R1-Distill-Qwen-14B
A reasoning distill, so answers run long and every decode step re-reads all 30 GB.
C
MIXTURE OF EXPERTS · 31 GB OF WEIGHTS
Moonlight-16B-A3B
Moonshot’s Kimi-family MoE. Latent attention keeps the cache unusually small for the parameter count, so it tolerates high concurrency.
Demo environment: one NVIDIA RTX Pro 6000 Blackwell, 96 GB, on molab. Each run tunes a single model.
THE EXPERIMENT MATTERS
A model is only
part of the workload.
Request rate, input length, output length, and burstiness all affect serving behavior. Sera evaluates the configuration against the workload and requirements supplied in the experiment.
Prompts and responses.
Long prompts concentrate work in prefill. Long responses keep decode active for more steps.
Memory and device time.
A model’s footprint matters, but sharing also introduces competition for device time.
Requirements for each model.
A shared configuration is accepted only when both models clear their individual gates.
These are the demo’s configured models, not a claim of universal model support or physical hardware benchmark results.
SEE THE LOOP IN ACTION