The speed is not
in the LoRA.

The adapter changes what the model learns. The inference objective changes how much work you ask it to do.

Field guide · Decision experiments · Adaptation and training

Execution paths: Generative vs. Kev vs. Laya-LiteRT

Generative language model	Kev (ONNX pointer head)	Laya (LiteRT dual graph)
Prefill the input once.	Prefill state and question branches once (block-causal mask).	Main graph encodes prompt with right-padded inputs and [MASK] markers.
Decode token 1, then token 2 conditioned on token 1, and so on.	Read scores at option/decision positions with pointer head.	Host gathers token_logits at [MASK] markers; separate act head runs after.
With a KV cache, prior keys/values are reused. It does not recompute the whole prompt every step.	In-graph delimiter detection and pointer head; softmax in app.	Decision softmax computed host-side over gathered mask logits; act head emits act_logits [1,2].
Time depends on input prefill plus output length, decode throughput, batching and hardware.	Single forward pass per request. Input length, options, questions, attention/memory costs and infrastructure still matter.	Single forward pass per question. Input length, options, attention and memory costs still matter.

A hundred generated tokens involve repeated sequential decoding steps. Eliminating that loop can materially reduce latency. But a decision request is not constant-time with respect to its input, and neither “one pass” nor “LoRA” mathematically implies 200× speedup.

What LoRA and encoder decision layers contribute

Low-rank adaptation learns a small weight update rather than training every base weight. It primarily reduces training parameters and optimizer state. At inference, an adapter may be merged into the base weights or applied alongside them. It does not delete the base transformer or make a 4B model into an 11M-parameter forward pass.

Kev’s specialized training objective and readout repurpose that transformer: instead of repeatedly sampling vocabulary tokens, it computes scores over the caller’s declared options. The small trained head is critical, but most of the inference work is still running the backbone on the packed input.

Dual-graph LiteRT vs. single-graph ONNX

On-device decision models differ in their underlying backbone class and execution packaging:

Kev (causal decoder, single-graph ONNX): builds on a causal decoder (Qwen bases + LoRA + pointer head) in a single ONNX graph. It identifies option boundaries via custom delimiter token IDs in-graph, applying block-causal attention masking so candidate questions in a batch do not attend to each other.
Laya-LiteRT (bidirectional encoder, dual-graph LiteRT): builds on a bidirectional encoder (ModernBERT-large for English, mmBERT-base for multilingual; originating from convaiinnovations/laya and packaged by litert-community/laya-LiteRT) with two decision layers and a [MASK] option scorer directly inside the main graph (main.tflite). It takes right-padded input_ids (or inputs_embeds), attention_mask, and qtype_onehot. The separate act-head graph (act_head.tflite) is an auxiliary post-decision network that consumes pooled_cls plus host-derived probability features to output an escalation signal (act_logits [1,2]). Because decision scoring is embedded in the main graph, task adaptation requires fine-tuning and exporting the encoder backbone, rather than swapping an isolated act head. Browser execution via Google LiteRT.js (@litertjs/core) with Wasm/XNNPack and WebGPU acceleration is prospective; the 644 MB to 1.69 GB graphs are currently validated on Android and desktop CPU.

A cost-shape calculator, not a benchmark

Change the assumed prefill and per-token times. The arithmetic illustrates why output length matters; it does not model an actual provider or account for all runtime costs.

Assumed prefill millisecondsGenerated output tokensAssumed milliseconds per decoded token

Formula: generative = prefill + tokens × decode; decision = prefill + illustrative 2ms readout. Shared prefill is an explicit simplifying assumption, not measured parity.

Where the evidence ends

TypeSafe announcement: describes parallel sampling and workflow-specific comparisons, with important benchmark caveats.
Archer Hume’s reconstruction: infers a plausible architecture from observations; not an official disclosure of Jev.
Kev source and 0.8B model card: inspectable masks, heads, LoRA and base model choices.
litert-community/laya-LiteRT and Laya-English-LiteRT: classic Google LiteRT (.tflite) dual-graph architecture (main graph + act head graph, right-padded input_ids and attention_mask), ported from convaiinnovations/laya.
Google LiteRT and @litertjs/core (LiteRT.js): Google’s runtime for on-device machine learning with Wasm/XNNPack and WebGPU acceleration (WebNN experimental, behind flags).
SemIf / OpenJev (openjev.com): an open-weights in-browser experiment comparing single-pass direct option logit readout against token-by-token JSON generation across Qwen and MiniCPM GGUF checkpoints using wllama WebAssembly/WebGPU.
The smart if-statement: another typed decision architecture; its measured latency is not a benchmark of this page.