Sovereign Decisions: Open-weight Jev Alternatives in Solace Agent Mesh

Your data. Your models. Your network.

TL;DR: Solace Agent Mesh decision nodes work with any model that speaks TypeSafe’s decision API, so they can run on hardware you control. We pointed the same workflow at two open-weight decision models on one Mac and measured them against Jev on 40 support tickets. Escalating the answers the open models were unsure about to a self-hosted large language model brought both to 95.0% accuracy.

In a few weeks, decision models have gone from one product to a category. A decision model does not write text: it reads some state, answers typed questions about it, and returns a probability for each answer, so software can branch on the result directly. TypeSafe introduced Jev on September 15. OpenAI released its Decisions API in beta on October 6, and by October 13 Microsoft had announced Decision-1, a post-trained Qwen3.5-9B. Open-weight models followed on Hugging Face, among them H2O-Lightning-4B on October 2 and Perplexity’s Decider 1.2 on October 15.

The appeal is how much of an agent’s work is deciding rather than writing. Which team owns this ticket? Is this tool call safe to run? Did the task finish? TypeSafe pitches decision models as smart if-statements in workflows, a judgment you can run over every item in a large batch, a fit for real-time paths, and a check on everything an agent does. Microsoft lists routing, classification and verification. In the blog post where we introduced decision nodes in Agent Mesh, our benchmark showed Jev routing a support ticket in about 200 milliseconds for about $20 per million tickets. The large language models (LLMs) Claude Haiku and Sonnet took 1.3 to 1.5 seconds for the same call and cost $73 to $1,391. The model’s confidence also tells the workflow which answers to send to a stronger one.

A decision node puts one of these models inside a Solace Agent Mesh workflow. The decision node asks the model typed questions about the workflow’s data, reads the answers and their confidence, and sends the answers the model is unsure about to a stronger node. That launch post ran decision nodes on hosted models: TypeSafe’s Jev and OpenAI’s GPT-6 Luna Decisions.

Some teams cannot send that data anywhere. Case notes in a bank, referrals in a hospital, and logs in a defence program often have to stay inside the organization’s own boundary, sometimes on a network with no route to the internet. For those teams a hosted decision API is ruled out however cheap it is. An open-weight decision model solves this: it runs next to the workflow, and the decision input stays on infrastructure you control. It also removes the per-call fee and the dependency on one vendor, though you pay for the hardware instead.

Why Can Decision Nodes Run on Your Own Models?

The TypeSafe provider in Agent Mesh is a protocol rather than a single service. It sends each decision to POST <api base>/v1/systemone and reads back typed answers. Any server that answers that request is a decision model to Agent Mesh: TypeSafe’s own API, a pass-through such as OpenRouter, or a server you run. A workflow names its decision model by alias, and the alias holds the endpoint, so moving decisions onto your own hardware is one edit to a model configuration.

The workflow never names a provider. The alias decides where each decision goes, and only a server you run keeps the decision input inside your network.

Which Open Models Did We Try?

We used two open-weight models as examples: H2O-Lightning-4B and Perplexity’s pplx-decider-flash-v1.2. Both are Apache-2.0 models built on Qwen3.5-4B, both produce the same typed answers that TypeSafe’s /v1/systemone request asks for, and both read images as well as text. Neither is a Solace product, and any open model that answers the same request works the same way.

How Do You Serve the Model?

In production you serve the model on your own GPU infrastructure, like any other inference service. H2O.ai’s model card serves H2O-Lightning-4B with vLLM behind H2O’s decision server, a small shim that speaks TypeSafe’s API. Perplexity ships its model as a Python package with an optional CUDA fast path and no HTTP server, so you put a thin server around it. Throughput and concurrency then come from the GPUs and the serving stack you choose.

Proving the integration needs far less. We ran both models on a single Apple-silicon Mac. For H2O-Lightning-4B that meant NetProjects’ community 8-bit MLX build of release v1.2.3, served by NetProjects’ h2o-lightning-mlx, which runs H2O’s own shim unchanged. Perplexity’s reference path runs on the Mac’s GPU in float32.

# H2O-Lightning-4B: H2O's shim in front of the community 8-bit MLX build
git clone https://github.com/internet-projects/h2o-lightning-mlx && cd h2o-lightning-mlx
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt
.venv/bin/hf download h2oai/h2o-lightning-4b --revision v1.2.3 --local-dir model \
    --exclude "adapter/*" --exclude "assets/*" --exclude "model.safetensors"
.venv/bin/hf download netprojects/h2o-lightning-4b-mlx-q8 --local-dir mlx-q8
MODEL=mlx-q8 SHIM_NOUL_FLOOR=0 ./serve.sh   # http://127.0.0.1:8741/v1/systemone

# Perplexity flash: the official checkpoint and package, then the wrapper below
uv venv --python 3.12 .venv
hf download perplexity-ai/pplx-decider-flash-v1.2 --local-dir pplx-decider-flash-v1.2
uv pip install --python .venv/bin/python ./pplx-decider-flash-v1.2/source
.venv/bin/python serve_pplx.py                 # http://127.0.0.1:8780/v1/systemone

The commands we used for the Mac test. On GPUs, follow each publisher’s serving instructions. SHIM_NOUL_FLOOR=0 turns off H2O’s yes-or-no floor, explained below.

Perplexity’s package returns answers in the same shape Agent Mesh reads, so the wrapper only has to carry the request and response over HTTP. Ours passes all of a request’s questions to the model together, in one pass. This is the whole wrapper:

import json
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

from pplx_decider import Decider

decider = Decider.from_pretrained("pplx-decider-flash-v1.2", device="mps")  # "cuda" on a GPU server
lock = threading.Lock()


class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        with lock:
            out = decider.decide({"state": body["state"], "questions": body["questions"]})
        raw = json.dumps({"model": "pplx-decider-flash-v1.2", **out}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw)

ThreadingHTTPServer(("127.0.0.1", 8780), Handler).serve_forever()

serve_pplx.py, as we ran it. It answers one request at a time and checks no key, so treat it as a test harness, not a production server.

Then register the server on the Solace Agent Mesh Models page: Add Model, the Decision model type, provider TypeSafe, the server’s URL as the endpoint, and a key. Agent Mesh requires a key on every decision model, even though neither of our servers checks one. Here is the same model as declarative configuration, for sam config apply:

kind: model
name: ticket-router
spec:
  provider: typesafe
  modelType: decision
  modelName: pplx-decider-flash-v1.2
  apiBase: http://decisions.internal:8780     # your server
  authConfig:
    type: apikey
    api_key: ${DECISION_API_KEY}

Every model in the run is self-hosted: the agents’ LLM and the ticket-router decision model, shown here pointed at pplx-decider-flash-v1.2.

What Happens When You Swap the Model?

We kept the launch blog post’s ticket-triage workflow exactly as it was and pointed its ticket-router alias at H2O-Lightning-4B. The desk agents and the reviewer ran on a self-hosted LLM as well, Qwen3.5-35B-A3B on the same Mac, so no model in the run was hosted by anyone else. A clear ticket routes in one decision call:

“I was charged twice for my October invoice…” H2O-Lightning-4B answers billing at 95% confidence, and the billing desk runs.

The ambiguous ticket escalates, as it did with Jev:

“We’re downsizing and need to remove 30 seats before renewal…” H2O-Lightning-4B’s best answer is other at 66% confidence, below the workflow’s min_confidence, so the reviewer agent takes the ticket.

Switching to Perplexity’s flash model changed the endpoint and model name in the ticket-router configuration and nothing in the workflow. It sent the clear ticket to billing at 99.6% confidence. On the ambiguous ticket it answered other at 57% confidence, below the threshold, so the reviewer took that ticket too. Its confidence on ambiguous tickets is lower than H2O-Lightning-4B’s, 57% against 66% on this one, so it hands more of them to the reviewer, which is why min_confidence is worth tuning for each model:

The same ambiguous ticket on Perplexity’s flash model: other at 57% confidence, escalated to the reviewer.

Does Everything Stay Inside One Boundary?

In a sovereign deployment, everything the workflow touches sits inside your network: Agent Mesh, the decision model’s server, the LLMs the agents use, and the data they access. The escalation target stays inside too, as an agent on a self-hosted LLM or a human-review step, never a hosted frontier model.

The decision input, the agents’ prompts and the escalations all stay inside the boundary. The one step that needs the outside world is getting the software in: you download the model weights and the decision server’s container image once, outside the boundary, and copy them into an internal registry.

What Do the Numbers Show?

We reran the launch post’s benchmark against both models: the same 40 tickets (32 clear, 8 ambiguous), the same three questions (department, urgent, severity), three runs each, on the same Mac (Apple M4 Pro, 48 GB). The Jev rows come from hosted runs: Jev 1.13 is the launch post’s run, with its escalation figures recomputed at 0.7 to match (the launch post used 0.8), and Jev 1.14, released October 14, we ran the same way on October 21. Accuracy and confidence describe the models; the latency columns describe this one Mac, not a production deployment. They also cannot be compared directly across rows: Jev ran hosted, with every request crossing the public internet, while the two open models ran on the Mac itself, with no network in between.

Both open models trail Jev 1.13 overall, as you would expect of 4B open models against a hosted one: 87.5% for H2O-Lightning-4B and 85.0% for Perplexity’s flash model, against 92.5%. Jev 1.14 scored 88.3%, 3.3 points above Perplexity, about one ticket in 40, which a sample this small cannot separate from noise. Most of the gap is on the ambiguous tickets. H2O-Lightning-4B got 30 of the 32 clear tickets right and Perplexity’s flash model 31, and both gave the same answers on all three runs of every ticket. Both are less confident than Jev: 15% of each open model’s answers fall below 0.7, against 12.5% for both Jev versions. Perplexity’s flash model is the less confident of the two on the ambiguous tickets, 0.58 on average against H2O-Lightning-4B’s 0.71. On the answers they kept, both open models were right 94.1% of the time.

Open weights cost a few points. Escalation inside the boundary wins back most of them.

We sent every answer below 0.7 to the self-hosted Qwen3.5-35B-A3B, playing the reviewer, and kept the decision model’s answer otherwise:

With escalation, every model finished between 95.0% and 97.5%, against 85.0% to 92.5% without it. Both open models reached 95.0%. In every case the LLM saw only 12.5% to 15% of tickets, the ones the decision models were unsure about.

On the Mac, H2O-Lightning-4B answered all three questions about a ticket in 0.67 seconds at the median and Perplexity’s flash model in 0.83 seconds, enough to prove the path end to end. Production speed comes from the GPUs you deploy on. H2O.ai reports about 30 milliseconds per decision on one H100, and 29 to 82 decisions a second on one H100 with 64 requests in flight, depending on its vLLM build (model card, v1.2.3, rechecked October 10, 2026). Perplexity’s card for the larger 27B model reports 9.1 requests a second on the reference path and 19.8 on the fast path on one GB200; we did not measure the flash model on a GPU. Those are the publishers’ own measurements, so measure on the hardware you choose.

What Should You Know Before You Rely on It?

• Plan concurrency per model. H2O-Lightning-4B’s vLLM server batches requests on the GPU. Perplexity’s package batches the questions inside a request and has a CUDA fast path, but it ships no server, so concurrency depends on the server you build around it. Our wrapper answers one request at a time.
• Size the timeout to your hardware. Agent Mesh gives a decision call 10 seconds by default. A cold server can exceed that: on our Mac, the first Perplexity request after the model had been paged out of memory took 41 seconds, and the decision node timed out. Perplexity’s card recommends calling decider.warmup() at startup on the fast path. Keep servers warm, or raise timeout_seconds in the workflow’s decision block.
• Check the cutoff for yes-or-no questions. A decision model does not answer a yes-or-no question with a plain yes or no. It returns how likely the answer is to be yes, such as 0.62, and Agent Mesh counts anything from 0.5 up as yes. That works when a model’s numbers are centred on 0.5, but some models lean one way. H2O-Lightning-4B agreed with our urgent labels on 82.5% of tickets at 0.5, and on 92.5% with the cutoff at 0.55. Before you route on a yes-or-no answer, look at the likelihoods your model gives on a sample of your own data, pick the cutoff that matches your labels, and have the workflow compare answers.urgent.probability with that cutoff instead of using the yes-or-no value.
• Test how you send Perplexity’s questions. Perplexity’s card recommends one question per request for this model. We sent all of a ticket’s questions in one pass, which on our tickets was faster and matched our urgent labels better. Try both on your own data.
• Turn off H2O’s yes-or-no floor. As shipped, H2O’s shim reports every yes-or-no answer as at least 80.1% likely on its chosen side. That hides exactly the uncertainty a workflow wants to see, so start the shim with SHIM_NOUL_FLOOR=0, as we did.
• The key is not checked. Agent Mesh requires a key on every decision model, but H2O’s shim ignores it, and Perplexity’s package has no server at all, so a wrapper like ours checks nothing. Put a proxy that checks the key in front of the server, and keep the server off networks that do not need it.
• Text decisions for now. Both models read images, but Agent Mesh sends images only to Jev and GPT-6 Luna Decisions today.

Ready to Try Open Decision Models in Your Own Workflows?

The decision model behind a workflow is a configuration choice. Hosted models are the convenient default, and Jev 1.13 was the most accurate model in our tests. Open weights are how a workflow’s decisions stay on your own network, and escalation to a self-hosted LLM closes most of the accuracy gap. We proved the path on a single Mac; in production, the same model configuration points at the GPUs you run.

Start from the ticket-triage example in our launch post and the Agent Mesh workflow documentation, watch the decision node demo, and download Agent Mesh. See H2O-Lightning-4B and Perplexity’s flash model for each model’s own documentation.

Frequently Asked Questions

What is a decision model?

A decision model reads some state, answers typed questions about it, and returns a probability for each answer instead of writing text. A workflow can branch on the answer and use the confidence to decide what to escalate.

Can decision nodes run on my own hardware?

Yes. The decision node sends each decision to /v1/systemone on the endpoint in the model configuration, and any server that answers that request works. Moving decisions onto your own hardware is one edit to a model configuration.

How much accuracy do open models give up?

In our 40-ticket test, the open models scored 85.0% to 87.5% on their own, against 88.3% to 92.5% for Jev. Escalating answers below 0.7 confidence to a self-hosted LLM brought both open models to 95.0%.

Does escalation send data outside my network?

Not when the escalation target is an agent on a self-hosted LLM or a human-review step. In our end-to-end runs, Agent Mesh made no outbound request.

Do I need a GPU?

We proved the path on a single Mac, which is enough to test the integration. For production throughput and concurrency, serve the model on GPUs and measure on that hardware.