Find each model’s balance.
Three investigators inspect evidence, propose changes, and review each other’s findings. The arbiter selects the next GPU experiment.
INTELLIGENCE, WORKING TOGETHER
Every model.
In its element.
Find the right balance between models and machines. Sera brings specialist agents together to tune inference and test what can share a GPU.
Offline replay: no keys · Live runs: GPU + API keys
Memory / context
Scheduling
Output quality
THREE SPECIALISTS. ONE SHARED OBJECTIVE.
FROM EXPERIMENT TO EVIDENCE
Good instincts.
Measured decisions.
Fitting in memory is a start.
Running well together is the test.
Bring them together.
Sera checks memory fit, then runs both models on one shared device. Each model must still meet its own limits.
Keep the whole story.
Weave records outputs, measurements, and agent decisions. Investigators read those traces in later rounds. ARIA provides an optional, separate review of the completed run.
MEET YOUR RESEARCH TEAM
Different perspectives.
Better-informed decisions.
Three investigators bring different areas of expertise to shared evidence. They inspect traces, compare proposals, and revise the next experiment from measured results.
THE MEMORY SPECIALIST
Smaller footprint.
Quality still comes first.
Inspect memory use and context demand. Propose supported cache and context settings; keep the quality and performance gates fixed.
KV-cache precisionContext limitsMemory budget
RESEARCH PRINCIPLE
If memory isn’t the bottleneck, the specialist can say so. A useful answer doesn’t always require a change.
TWO QUESTIONS. ONE CONNECTED LOOP.
Tune the model.
Then rethink the hardware.
The best configuration for an isolated model may not be the right configuration for a shared GPU. Sera keeps both questions in view.
How should each model run?
Set the workload, quality floor, and progress threshold. The swarm proposes, reviews, and measures candidates until a valid stop condition. The demo has no fixed total trial cap.
ProposeMeasureValidate
You get A tested configuration, a usable runner, a saved report, and replay downloads.
Can two models share a GPU?
Revisit the ledger for viable single-device configurations. Check memory fit, run the models together, and evaluate each model against its own limits.
FitRun togetherGate
You get A sharing verdict, with evidence behind the decision.
THE DEMO
Watch a run. Then run your own.
useseraai.com/demo replays a measured optimization loop end to end. useseraai.com/notebook is the live marimo notebook — pick a model, press Run, in your browser or on a Blackwell.
A MEASURED AGENT LOOP
Three investigators.
A faster configuration.
Three Astra investigators read Weave traces, reviewed each other’s proposals, and completed three search rounds. Prefix caching and graph execution reduced p95 latency by 19.77% on eight warmed, repeated tasks.
See demo resultsMEASURED RUNQwen2.5-72B · FP8 weights
Baseline p95771.05 ms
Selected p95618.59 ms
Answer checks8 / 8 passed
WHAT THE LOOP LEARNED
Graph execution alone did not beat prefix caching. The final combination won. A fresh request through the returned runner passed, then Sera closed the runner. ARIA provided a separate post-run review.
INSIDE THE DEMO
A model for each test.
One physical GPU.
The demo tunes Qwen2.5-72B on one RTX Pro 6000 Blackwell. Eight exact-answer JSON tasks, a 99% quality floor, and concurrency 1/2/4/8 stay fixed while the swarm searches.
A
DEMO · FP8 WEIGHTS
Qwen2.5-72B
FP8 weights and BF16 KV cache. Run 1 optimizes latency. Run 2 optimizes latency, then throughput while preserving the earlier limit.
B
SHARED-GPU TEST · BF16 WEIGHTS
Qwen3-0.6B
The smaller service in the separate two-model test. It must pass its own answer checks and shared-device latency limit.
C
SHARED-GPU TEST · FP8 WEIGHTS
GLM-4-9B
The second service in the sharing test. Both services must pass quality and stay within 10% of their isolated p95 latency.
Hardware: one 96 GB RTX Pro 6000 on Molab. The separate sharing test declares a 24 GiB memory budget; it does not emulate a smaller card’s compute speed. An earlier joint run passed; the latest repeat missed GLM’s latency limit.
A FEW USEFUL DETAILS
Before you
enter the lab.
Do I need a GPU to try the demo?
The offline replay and local simulator need no GPU or API keys. Live demo needs a compatible GPU, an OpenAI key for investigators, and a W&B key and Weave project. Clone the notebook before entering keys. GPU time and hosted model calls can cost money.
Are the results hardware benchmarks?
The recorded Qwen2.5-72B run measured 19.77% lower p95 latency on eight warmed, repeated tasks, with 8/8 answer checks passing. Simulator numbers are modeled, not measured GPU results.
Does Sera always recommend sharing?
No. A memory fit is followed by a joint trial. Each model must pass its own performance and quality gates; a failed gate can lead to a revert.
What makes this different from a parameter sweep?
Investigators read Weave evidence, review each other’s proposals, and adapt after each trial. The full run demonstrates this loop. A general search-efficiency advantage over grid or random search is not established.
A CLEARER VIEW OF YOUR INFERENCE