Find each model’s balance.
Quantization, batching, and parallelism specialists propose changes. An arbiter decides which ideas earn a trial.
INTELLIGENCE, WORKING TOGETHER
Every model.
In its element.
Find the right balance between models and machines. Sera brings specialist agents together to tune inference and test what can share a GPU.
Interactive demo · No GPU or API key needed
Quantization
Batching
Parallelism
THREE SPECIALISTS. ONE SHARED OBJECTIVE.
FROM EXPERIMENT TO EVIDENCE
Good instincts.
Measured decisions.
Fitting in memory is a start.
Running well together is the test.
Bring them together.
Sera checks memory fit, then runs both models on one shared device. Each model must still meet its own limits.
Keep the whole story.
Every trial enters the ledger, including failures. See what worked, what didn’t, and why a decision was made.
MEET YOUR RESEARCH TEAM
Different perspectives.
Better-informed decisions.
Three specialists reason from the same evidence. Each owns a different part of the serving configuration, so you can trace a result back to the change that caused it.
THE MEMORY SPECIALIST
Smaller footprint.
Quality still comes first.
Explore weight and KV-cache precision to reduce memory demand. Every candidate still has to clear the quality and performance gates.
Weight precisionKV-cache precision
RESEARCH PRINCIPLE
If memory isn’t the bottleneck, the specialist can say so. A useful answer doesn’t always require a change.
TWO QUESTIONS. ONE CONNECTED LOOP.
Tune the model.
Then rethink the hardware.
The best configuration for an isolated model may not be the right configuration for a shared GPU. Sera keeps both questions in view.
How should each model run?
Set workload requirements, latency limits, quality floors, and a trial budget. Let the specialists propose changes and use measured trials to decide what stays.
ProposeMeasureValidate
You get A history of tested configurations and their outcomes.
Can two models share a GPU?
Revisit the ledger for viable single-device configurations. Check memory fit, run the models together, and evaluate each model against its own limits.
FitRun togetherGate
You get A sharing verdict, with evidence behind the decision.
A RESULT YOU CAN QUESTION
The failed trials
belong in the story.
A configuration can fit in memory and still fail under contention. Sera records reverts, checks predictions against measurements, and brings the evidence back to its specialists.
Follow an experimentDECISION RECORDIllustrative example
Memory fitPassed
Shared-device latencyLimit exceeded
DecisionRevert
WHY IT MATTERS
Enough memory doesn’t mean enough device time. Keep the models separate when sharing breaches a model’s requirements.
INSIDE THE DEMO
Two models.
Two very different workloads.
The demo pairs long prompts and short answers with short prompts and longer answers, so the shared device faces different demands.
A
PREFILL-HEAVY · BURSTY TRAFFIC
Qwen2.5–1.5B Instruct
Longer inputs and shorter outputs put prefill work at the center of the experiment.
B
DECODE-HEAVY · STEADY TRAFFIC
Llama 3.2–1B Instruct
Shorter inputs and longer outputs make decode performance a different kind of constraint.
Demo environment: two simulated 24 GB devices. Co-residency is tested on one shared device.
A FEW USEFUL DETAILS
Before you
enter the lab.
Do I need a GPU to try the demo?
No. The project’s default experiment uses a simulator and requires no GPU or API key. This website currently opens an interactive design walkthrough.
Are the results hardware benchmarks?
Simulator results describe the modeled experiment, not measured performance on a physical GPU. The project records the trial substrate so simulated and measured results can be distinguished.
Does Sera always recommend sharing?
No. A memory fit is followed by a joint trial. Each model must pass its own performance and quality gates; a failed gate can lead to a revert.
What makes this different from a parameter sweep?
Sera connects individual tuning to consolidation, records specialist predictions, and routes new constraints back into the loop. The current project does not claim better trial efficiency than random search.
A CLEARER VIEW OF YOUR INFERENCE