I started by attempting to reconstruct the work itself, but that isn’t realistic for me, given skill/resource constraints. So I’ve instead pivoted to porting their Python mechanism that uses a base Qwen model to build their coordinator: Trinity Coordinator. (Right now it’s been deconstructed to use a local path-dep in mix.exs related to an inference library I’m building to generalize abstracting LLM providers, thus not “clone friendly” yet.)
Just seeing if this is of interest to anyone. Certainly open to input/feedback/ideas/critiques on approach. Please respond here or open an issue with your candid feedback. There must be someone out there with more knowledge/experience on such matters who can provide guidance?
I’ve created the safetensors file from the original python scripts, so nx can talk numbers properly. I’ve been working on a staged verification process so that the resulting coordinator based on Qwen will behave the same as the generated .pt file from their Python system. There are some nuances related to numpy -> nx and others that might prevent perfect alignment but I’m aiming for behavioral and functional parity.
One thing I see often these days is people creating amazing work and ideas in Elixir, but often hard coupled to providers and the like. One goal for trinity_coordinator is to have a working standalone system with built in routing to LLM’s, but also pluggable/modular for integration into any other codebase/framework/system.
ps: I wasn’t sure if this is the right forum category, but there was a note that said to use the nx forum if it’s nx related.
I put in some elbow grease yesterday. Seems to be pretty solid so far. Using a modified, tiny 0.6B Qwen SLM for robust routing might turn out to be quite useful!
Sakana is using their version of TRINITY (obviously, theirs is not served from Elixir/Bumblebee/etc) in their forthcoming commercial AI framework. So, this is pretty cutting edge. Glad to stand on the shoulders of giants (both in the research community, as well as the ML community in Elixir)
I’m running this on a 5060 Ti 16GB, but it should work on most any GPU since it’s an SLM.
Right now it’s just saying what model it would call, so it’s not complete framework yet, just a proving ground.
$ XLA_TARGET=cuda12 mix run examples/qwen_router_prompt_eval.exs
=== QWEN ROUTER PROMPT EVAL ===
What this does
Loads the local adapted Qwen router once, sends fixed transcripts through it,
and checks whether the selected agent slot and role match expectations.
No external LLM/provider calls are made.
Agent labels
The agent names are labels from the original Sakana checkpoint.
Example: agent 4 is labeled "google/gemma-3-27b-it".
This eval does not call Gemma; it only reports that the router selected
checkpoint slot 4.
Artifact
priv/sakana_trinity/adapted_qwen3_0_6b_layer26
Model
base router model: Qwen/Qwen3-0.6B
router head shape: [10, 1024]
assertion mode: strict
Native logs
hidden in normal mode: tmp/examples/qwen_router_prompt_eval.native.log
use --debug-native-logs to print XLA/CUDA compiler logs inline
[1/12] math_direct - PASS
Prompt sent to router:
user: What is 17 + 25? Answer briefly.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 15
[2/12] math_proof - PASS
Prompt sent to router:
user: Prove that the sum of the first n odd positive integers is n squared. Route this to the best next role.
Expected route:
agent 0: gpt-5
role 0: Worker
Router returned:
agent 0: gpt-5
role 0: Worker
Router input tokens: 26
[3/12] code_debug - PASS
Prompt sent to router:
user: A Python function mutates its default list argument across calls. Identify the bug and propose the smallest fix.
Expected route:
agent 0: gpt-5
role 0: Worker
Router returned:
agent 0: gpt-5
role 0: Worker
Router input tokens: 23
[4/12] security_review - PASS
Prompt sent to router:
user: Review this login flow for security risks: passwords are hashed, sessions are cookies, and reset tokens never expire.
Expected route:
agent 4: google/gemma-3-27b-it
role 1: Thinker
Router returned:
agent 4: google/gemma-3-27b-it
role 1: Thinker
Router input tokens: 24
[5/12] planning - PASS
Prompt sent to router:
user: Create a concise implementation plan for migrating a small Elixir service from in-memory state to Postgres.
Expected route:
agent 4: google/gemma-3-27b-it
role 0: Worker
Router returned:
agent 4: google/gemma-3-27b-it
role 0: Worker
Router input tokens: 22
[6/12] verification_after_worker - PASS
Prompt sent to router:
user: Calculate 6 * 7 and verify the answer.
assistant: Worker answer: 6 * 7 = 42.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 28
[7/12] needs_revision - PASS
Prompt sent to router:
user: Check whether the answer is correct: 19 + 24 = 41.
assistant: Worker answer: 19 + 24 = 41.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 38
[8/12] ambiguous_decomposition - PASS
Prompt sent to router:
user: This problem has unclear requirements. We may need to split it into assumptions, risks, and a concrete next action.
Expected route:
agent 0: gpt-5
role 2: Verifier
Router returned:
agent 0: gpt-5
role 2: Verifier
Router input tokens: 25
[9/12] creative_but_constrained - PASS
Prompt sent to router:
user: Draft a friendly but precise support reply explaining a billing correction. Keep it under 120 words.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 23
[10/12] longer_context - PASS
Prompt sent to router:
system: You are routing work inside a three-role TRINITY loop. Worker solves, Thinker plans or redirects, Verifier checks.
user: Given a release checklist, identify the next best role. The feature compiles, unit tests pass, docs changed, but no smoke test has been run yet.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 62
[11/12] provider_failure_triage - PASS
Prompt sent to router:
user: The last provider call timed out after 30 seconds. Decide whether to retry, ask a thinker for a smaller plan, or verify the partial answer.
Expected route:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router returned:
agent 4: google/gemma-3-27b-it
role 2: Verifier
Router input tokens: 33
[12/12] final_answer_check - PASS
Prompt sent to router:
user: Solve and then verify: the capital of France is Paris.
assistant: Worker answer: Paris is the capital of France.
assistant: Thinker note: this is a factual lookup and likely ready.
Expected route:
agent 4: google/gemma-3-27b-it
role 0: Worker
Router returned:
agent 4: google/gemma-3-27b-it
role 0: Worker
Router input tokens: 41
Summary
passed: 12
failed: 0
roles selected: Worker=4, Thinker=1, Verifier=7
agent slots selected: 0=3, 4=9
PASS qwen_router_prompt_eval
Just for fun, I ported this to work on my 24GB M4 MBP using the Emily backend. In doing so, during the export, I ran into a limitation of the current native mlx libraries - their svd functions have no ‘thin’ mode and always materialise the full matrix. For the Qwen embedder that turned out to be ~92GB of memory.
I updated Emily to support this mode (in specific cases) directly via the Gram matrix, could do the one-time export in ~2s and was able to run the qwen router example.
If you are interested, the changes I made are here
@ausimian, the Emily result is extremely useful to know. The fact that native MLX SVD materialized the full matrix, but the Gram-matrix path made the one-time export work on a 24GB M4, is exactly the kind of backend detail I would not have known to check.
@polvalente, your point about Nx’s default implementation is really helpful. If the default implementation avoids explicit evals, then it may avoid the immediate materialization issue seen in the native MLX SVD path.
This helped me separate two concerns: what TRINITY needs from Nx today, and what backend-neutral code can safely assume. In TRINITY, the export path only needs the reduced SVD outputs for reconstruction. As an application author, though, it is tempting to read:
Nx.LinAlg.svd(tensor, full_matrices?: false)
as a “thin SVD” guarantee in both output shape and execution/memory behavior. The safer cross-backend interpretation seems to be:
full_matrices?: false is an output-shape contract.
Execution strategy and memory behavior are backend-dependent.
If that is the intended boundary, I have two practical questions:
If a backend’s native SVD is known to materialize full matrices anyway, should the preferred backend behavior be to notice full_matrices?: false and fall back to Nx’s default implementation when that is likely to behave better?
Would the Nx team be open to a small docs PR clarifying that full_matrices? controls SVD output shapes but does not guarantee a reduced-memory execution profile?
For the current TRINITY path, this does not force an immediate change: it is still fine to call Nx.LinAlg.svd/2 and let Nx/the selected backend choose native SVD, default Nx lowering, or a backend-specific workaround. But the next rework is to make backend/runtime selection a profile concern instead of something CUDA-shaped in the application code, so I want to make sure I am relying on the right Nx contract before generalizing it. I do not want application code branching on Emily vs EXLA vs Torchx unless there is no better option.
Side note: I did go down a rabbit hole thinking about how production code could ask a backend about these paths before starting a large SVD. I wrote that up here: Concrete proposal: SVD execution capabilities in Nx. It is probably too much architecture for this immediate issue, but I am leaving it as background in case backend-specific memory behavior becomes a larger Nx API discussion later.
@gtcode any backend-specific issues should be addressed in the specific backends. For instance, EMLX, which is the original MLX-based backend for Nx, does not fall back to the binary backend at all, and relies on these default implementations from Nx for the most part when things aren’t supported by the native backend.
I think it doesn’t really make sense to document this specific issue in Nx because this can happen with literally any option, as different backends are limited by different constraints. Maybe a general comment on this would be welcome.
Also, feel free to test this out with EMLX in the github main version, as I’m actively working on improvements. v0.3 should be out soon.
I gave a very quick pass through the proposal, and I think there’s something being overlooked there. It focuses on Nx.block, but the same concerns are pertinent to all Nx callbacks.
Also keep in mind that blocks and block structs can be defined by Nx users, not just Nx itself.
Additionally, even Nx.dot might have different execution profiles depending on the backend.
Finally, lowered vs eval’d is also something that’s not a concern for the backend, but for the compiler.
Do feel welcome to open an issue for us to discuss there and hopefully achieve a great feature, but I suggest rethinking the approach to something more general that includes all Nx callbacks.
@ausimian — this is great, thank you. The ~92 GB → ~2 s number on the embedder is very helpful – I would not have been able to diagnose that failure mode from the CUDA side, and your Emily Gram path is the right idea.
I went through your two commits and the Emily PR. The diagnosis and the backend-level fix both look mostly right. The one thing I’d note is that since you opened that PR, the trinity_coordinator has moved to a runtime-profile architecture (mix … --runtime-profile <name>), and the prompt-eval suite is now 37 fixed cases with snapshot + determinism + margin-floor checks rather than the original 12. So, I don’t want to merge the PR verbatim. config :nx, default_backend: Emily.Backend globally and making "emily" a valid XLA_TARGET would blast through a few things on our end. But, I do want to land an explicit :emily runtime profile that takes the same intent: Emily as the default backend when the operator chooses that lane, untouched otherwise.
The blocker for me is that I don’t have Apple hardware. Would you be up for one validation pass against current main so I can shape the upstream integration with confidence? I think it’s two commands:
# Pull current main, add Emily 0.4.0 (the Gram fix is already in there), then:
mix trinity.sakana.export_adapted \
--force \
--svd-compute-type f32 \
--out tmp/emily_adapted_qwen3_0_6b_layer26
mix run examples/qwen_router_prompt_eval.exs \
--artifact-dir tmp/emily_adapted_qwen3_0_6b_layer26 \
--snapshot examples/fixtures/qwen_router_prompt_eval_logits.json \
--determinism-runs 2
What I’d love to see in the output:
export reaches status=complete with no OOM,
per-tensor SVD time for the embedder,
37/37 prompt cases pass,
if anything fails, whether it’s route_hash (likely backend-sensitive) or one of the decision-stable fields (agent_id / role_id / token_count / transcript_hash) — that distinction tells me whether Emily needs its own snapshot lane or whether something deeper drifted.
If you’d rather not run the full thing, even just the export half is useful. Either way, thanks again for doing this — turning it into a real Apple lane upstream is now basically a paperwork exercise on my side because of the work you did.
I was able to adapt it locally. I hit the same 92GB allocation, so I’ll see if I can add a better upstream Nx implementation for Thin SVD so that EXLA can also take advantage of it.
I do want to encourage people to reach out on the Nx issues tracker when they hit these types of limitations.
It uses a smaller memory footprint which enabled EMLX to run the export without OOMs and without having to rewrite anything. This also benefits EXLA, which uses the same Nx base code.
PR should be merged soon, I just gotta fix some test assertions and documentation.
With just this change and replacing all EXLA.Backend cuda config with the equivalent EMLX, device: :gpu setup, I got the outputs:
@ausimian@polvalente — both of your runs landed on main this afternoon. Quick summary of what shipped, because the rollup matters more than the diff.
We now have a first-class --runtime-profile emily lane. The :emily clause in RuntimeProfile.resolve/1 mirrors :emlx’s Apple-shaped flags but routes to Emily.Backend and ships the empirical per-profile margin floors directly from ausimian’s validation pass — agent worst 0.417 on two_assistant_turns, role worst 1.029 on escalate_to_human, 80% rule, so agent: 0.33 / role: 0.82. The previous {:custom, Emily.Backend, []} recipe still works for one-off overrides, but the day-to-day operator path is just:
No --min-*-margin flags needed; the profile carries them. Paulo, your bare-EMLX 37/37 result is now the reference for :emlx; ausimian’s bare-Emily 36/37 + per-profile-floor result is the reference for :emily. Both lanes are decision-stable against the CUDA snapshot (agent_id, role_id, token_count, transcript_hash all match); route_hash drifts on every case for both, which is what the per-profile snapshot fixture lane below is designed for.
A few other things that landed in the same push, attributed: ausimian’s “honest per-tensor SVD timings” follow-up became Phase 1 (Exporter.sync_tensor!/1 is a one-line backend-agnostic force point inside timed_decompose/2 + timed_reconstruct/3 — closes the decompose_elapsed_ms: 2 artefact for any lazy backend, costs ~nothing on EXLA). Paulo’s runtime.ex:56 review nudge became Phase 2 (tensor_backend/1 default is now tensor.data.__struct__ |> Module.split() |> Enum.join(".")) plus a Phase 3 audit that found two more sites in Sakana.{Artifact, PythonImporter} doing the same brittle string-cond — they’re consolidated behind Runtime.BackendLabel.from_label/1 now, with EMLX label coverage that the prior :emlx lane was actually missing.
Per-profile snapshot fixtures: drop examples/fixtures/runtime_profiles/emily/qwen_router_prompt_eval_logits.json and the eval picks it up automatically when --runtime-profile emily is passed without an explicit --snapshot. The seed run is mix run examples/qwen_router_prompt_eval.exs --runtime-profile emily --snapshot-out <path>. If anyone wants to send me the Emily-side rounded logits + route_hashes for all 37 cases, I’ll commit it as priv-clean and the next person to try this on Apple gets a stable baseline to compare against. Otherwise it stays unseeded and the snapshot drift assertion is just skipped on Emily, same way it is today on CUDA when no --snapshot is passed.
Re ausimian’s FastKernels follow-up commit: yes please, but as a separate :emily_fast profile rather than wired into :emily’s Coordinator.load/1 directly. That keeps bare-Emily as the reference research lane (matches Paulo’s bare-EMLX baseline) and makes the FastKernels effect attributable to its own dependency version — important if a future Emily release shifts behaviour and we have to bisect. The 15% wall-clock win on a single forward pass is real and should be considerably bigger on a decode loop. Detailed reply on the PR itself.
Thanks both — the diagnosis-to-merge round-trip on this was faster than I had any right to expect for an Apple-Silicon lane I can’t run myself.
I want to zoom out a bit and write down where I think this is heading architecturally, because the recent progress changes the nature of the problem.
The thread started as “can Sakana AI’s TRINITY/Qwen router be ported to Elixir/Bumblebee/Nx/Axon at all?” The answer is now much closer to yes than it was a few weeks ago. The thin-SVD work in Nx PR #1753 removes the worst memory pressure from export. We have CUDA as the original reference lane, EMLX getting a clean 37/37 decision-stable pass, and Emily now close enough that it has a real runtime profile with its own empirical margins. The remaining differences are the kind I would expect from backend-specific numeric behavior: route hashes drift, but the actual routing decisions can be made stable with per-profile fixtures and margin floors.
That is a big deal. The “can Nx express this pipeline?” question is mostly behind us. The harder question now is where this code should live before the bring-up repo hardens into the permanent monolith.
Right now trinity_coordinator is doing too many jobs. That was the right way to get the port working, but it is the wrong long-term shape.
It currently contains, in one application:
the TRINITY coordination semantics
the Sakana/Qwen manifest and routing-head invariants
Bumblebee model loading
Axon routing-head execution
Nx/SVD export and reconstruction mechanics
safetensors slicing
artifact fetching and pinning
provider-agent pool wiring
trace plumbing
operator Mix tasks
runtime profile and XLA/EXLA/EMLX/Emily bring-up logic
As a lab bench, that is useful. As a library boundary, it is too heavy.
The concrete split I am converging on is:
Product / AppKit
|
| governed product boundary
v
Mezzanine / OuterBrain / Citadel
|
| semantic, workflow, authority, review, provenance
v
trinity_framework
|
| Trinity.* contracts and coordinator semantics
|
+--> trinity_bridge_inference
| |
| v
| inference
| provider LLM calls
|
+--> trinity_bridge_self_hosted_inference
|
v
self_hosted_inference_core
|
v
self_hosted_inference_bumblebee
|
| Qwen load + Sakana adapter patch + routing head projection
v
RouteLogits
Reusable ML substrate underneath:
crucible_safetensors
crucible_tensor_patch
crucible_factorization
crucible_model_registry
The important point is that trinity_framework already exists and already has real Trinity.* contract modules. There is also already a consumer using it in stack_lab’s examples/trinity_platform_roundtrip. The workspace scaffold is there: core/, bridges/, apps/, tools/. Most of those package slots are stubs today, but the shape is right.
So I would avoid turning trinity_coordinator into a bigger framework. The cleaner next step is to finish the framework that already exists, move the reusable pieces to their real owners, and let trinity_coordinator become a temporary reference deployment while the migration is happening.
The main boundary correction is that the coordinator should never see the hidden vector.
An earlier version of the design had this rough flow:
Qwen runtime extracts hidden state
-> returns HiddenVectorRef or hidden-state payload
trinity_sakana_pipeline applies routing head
-> route decision
That split puts pressure in the wrong place. It either transfers a large tensor over a boundary or makes the coordinator orchestrate a runtime-local tensor operation through an opaque handle. Both are awkward. The extraction and the routing-head projection belong together inside the model runtime.
The cleaner flow is:
trinity_sakana_pipeline builds a plan
-> artifact pin
-> adapter ref
-> selected tensor/head spec
-> TRINITY shape invariants
self_hosted_inference_bumblebee executes the runtime work
-> load Qwen
-> apply/cache Sakana adapter
-> extract penultimate state
-> run Axon routing head
-> return small RouteLogits struct
trinity_coordinator_core applies semantics
-> role/agent partition
-> Worker/Thinker/Verifier choice
-> RouteDecision
That keeps tensors where tensors belong. The model runtime owns Bumblebee, Axon, Nx backend placement, and the routing-head math. The coordinator core owns what the logits mean.
The same idea applies to adapter identity. Sakana’s patch differs from LoRA in details, but operationally it behaves like an adapter: a content-addressed parameter-efficient modification that should be reused if the same patch is requested again. I would rather make that visible in the runtime lease model than hide it inside the Bumblebee backend as an ad hoc cache. self_hosted_inference_core should understand it at the lease level.
Concretely, SelfHostedInferenceCore.InstanceSpec should grow an optional adapter_ref, and RuntimeRegistry should be able to route by something like:
(backend_id, adapter_ref)
instead of only by backend/profile. That lets an already-warmed Qwen+Sakana-adapter runtime handle the next request with the same adapter hash. Without that, every distinct TRINITY patch plan risks becoming a fresh GPU process or a contested in-place mutation.
This is the part of the migration I expect to be hardest. Adding adapter_ref may be additive, but it may also expose assumptions in the existing Ollama backend or runtime registry about what “instance identity” means. I would wait for the real CUDA head_route gate to pass through the new backend before calling this a casual field addition.
The ML utility split is also important. The Nx PR #1753 point is a good example. One operation, Nx.LinAlg.svd, currently forces a git-SHA pin through the whole coordinator because export needs the better thin-SVD memory behavior. That tradeoff is fine in a bring-up repo. It becomes a problem when downstream consumers only want TRINITY routing semantics.
The SVD/SVF code should move to crucible_factorization. That one package can carry the Nx pin while it needs to. When the relevant Nx release lands on Hex, that package can relax the dependency. Other packages should be able to depend on a RouteDecision struct without inheriting a custom Nx commit.
One correction to my earlier thinking: I would fold this into crucible_model_registry instead of creating a new crucible_artifacts package. The registry already exists in the North-Shore-AI namespace and already has the richer storage-backend concept. The coordinator’s artifact pin schema should be merged into that. But that needs a real pre-flight audit. If the registry’s notion of artifact identity, SHA scope, revision refs, or cache layout differs from TrinityCoordinator.ArtifactFetch.Pin, then the work becomes schema reconciliation instead of a module rename.
Inside trinity_framework, I would split the TRINITY pieces roughly like this:
core/trinity_contracts
DTOs and behaviours:
RouteDecision, RouteLogits, AgentCallIntent, ProviderPool contracts,
trace events, model-runtime behaviour, agent-caller behaviour.
No Nx, no Axon, no Bumblebee, no Req, no HF.
core/trinity_sakana_contracts
Sakana/Qwen schemas and invariants:
manifest shape, router-head dimensions, export spec,
selected tensor keys, profile specs.
Still no runtime deps.
core/trinity_sakana_pipeline
TRINITY-specific plan generation:
what adapter/head/tensor plan should be executed.
It returns a plan rather than a hidden vector or model handle.
core/trinity_coordinator_core
the pure coordination state machine:
run loop, role injector, thinker/verifier policy,
route-logit interpretation, budget enforcement.
It should be testable with mock behaviours and no real model.
bridges/trinity_bridge_self_hosted_inference
translates Trinity.ModelRuntime calls into self_hosted_inference_core leases
and self_hosted_inference_bumblebee.route_with_head/3.
bridges/trinity_bridge_inference
provider LLM calls through the existing :inference package.
bridges/trinity_bridge_trace
trace sinks and redaction policy.
tools/trinity_ops
the Mix tasks currently under mix trinity.*
apps/trinity_single_node
the production-shaped local deployment app.
That last package matters because the long-lived deployment surface should move out of trinity_coordinator. The app that wires the single-node experience should live under the trinity_framework workspace as apps/trinity_single_node. It should own the runtime config reads: XLA_TARGET, HF_TOKEN, provider keys, runtime profile selection, artifact dirs, and so on. Those values should be materialized at the app boundary and passed down as config, so lower libraries receive config instead of reaching into env directly.
During migration I would keep the old coordinator config authoritative until the new app proves equivalence. In practice that means running the same HITL head-route gate first against trinity_coordinator’s config/runtime.exs, then against trinity_single_node’s config/runtime.exs. Only after both pass should the original config be retired.
There are also some boring but necessary enforcement rules:
No System.get_env under lib/**
Runtime/deployment env reads stay in config/runtime.exs or app bring-up.
No Nx/Axon/Bumblebee in contracts
Contract packages must remain lightweight and CPU/node friendly.
No direct SelfHostedInferenceCore/Bumblebee/Crucible imports in core packages
Those imports belong in bridges, runtimes, or ML utility packages.
No product bypass
Product code should go through AppKit/Mezzanine/Trinity surfaces,
not directly into lower runtime or ML packages.
The last point ties this back to the older governed execution platform thread:
That thread is out of date in details, but the boundary argument still applies. The larger stack has separate places for semantic reasoning, authority, workflow, lower facts, traces, review, and execution because those are different kinds of truth. TRINITY should plug into that shape rather than bypass it.
So the decomposition goes beyond moving files into smaller repos. It makes the TRINITY port obey the same truth boundaries as the rest of the platform.
The proposed phase order is intentionally conservative:
0. Freeze the monolith baseline.
1. Extract crucible_safetensors first, as the smallest proof of the pattern.
2. Extract crucible_factorization and move the Nx #1753 pin there.
3. Extract crucible_tensor_patch and merge artifact pins into crucible_model_registry.
4. Build self_hosted_inference_bumblebee and prove route_with_head/3 on CUDA.
5. Fill out trinity_framework core and bridges.
6. Move operator Mix tasks into tools/trinity_ops.
7. Make apps/trinity_single_node the deployment app.
8. Reduce trinity_coordinator to a compatibility shim.
9. Archive it after a release window.
The file audit gives this some teeth. The coordinator has about 60 .ex files under lib/, about 10.5 KLOC total. Roughly 40% belongs in trinity_framework, 30% in self_hosted_inference_bumblebee plus the North-Shore-AI crucible_* packages, 25% in ops/tasks, and a small amount should be deleted because provider transport already belongs in inference.
Two examples show the logic:
TrinityCoordinator.Sakana.Artifact is currently three owners in one file: manifest validation, artifact fetch/pin semantics, and model-state patching. Those should become:
TrinityCoordinator.CoordinationHead is also split in two:
role/agent/head invariants -> trinity_sakana_contracts
Axon model and projection -> self_hosted_inference_bumblebee
That split is the whole design in miniature: contracts define what is valid, runtime executes tensor work, coordinator interprets small semantic results.
I also want to preserve the git history when this moves. The history on files like sakana/svd.ex, coordination_head.ex, and the Sakana loader/patch code contains a lot of architectural decision-making that will not be obvious from the final file layout. Before trinity_coordinator is retired, I would subtree-merge or otherwise import the relevant history into trinity_framework.
My current summary would be:
trinity_coordinator was the right place to get the port working. The permanent library boundary should move elsewhere.
The reusable ML pieces should become North-Shore-AI crucible_* packages, with the existing crucible_model_registry extended rather than duplicated.
The single-node app should live as trinity_framework’s apps/trinity_single_node.
And the old coordinator repo should become a temporary compatibility shim, then retire.
If this sounds like a lot of decomposition for one model port, I think the recent backend work is exactly why it is justified. The port now spans CUDA, EMLX, and Emily lanes; it carries backend-specific profile behavior; it depends on a very specific Nx SVD improvement; and it is starting to look useful beyond the initial experiment. That is the point where “just keep it in the bring-up repo” starts creating future debt.
The goal is to avoid abstraction for its own sake and give each part one honest owner:
contracts describe
coordinator semantics decide
runtime executes tensor work
crucible packages do reusable ML mechanics
ops tasks operate
single-node app configures
AppKit/Mezzanine/OuterBrain/Citadel/Jido keep the governed platform honest
I can release that PR on a patch release soon (e.g. this weekend) instead of holding it off til 1.0. Also, Hex won’t allow git dependencies at all, so splitting the dependency off doesn’t help much.
Our custom Qwen3 implementation now reaches 300 tok/s on 0.6B 4bit and the emlx_axon bumblebee rewrites can reach 120 tok/s on the non-quatized 0.6B! Tomorrow I’ll release a new emlx/emlx_axon version with these improvements. Improvements are available in emlx/emlx_axon 0.4!
