What if my labels
are new?

New option names are easy. New meanings need evidence. Private business conventions do not become learnable just because the response is typed.

Back to the field guide · How the architecture changes the cost

Start with the failure, not the pitch

“Product-shaped questions with no training analogue are not guaranteed: on the TypeSafe docs example (‘two charges on my card’ → Is there a billing problem?) this checkpoint answers 0.22 while kev-0.6b answers 0.97…”

Source observation, not our experiment: Kev-4B card at revision 6b332b5. Later cards and checkpoints changed this example; the older Qwen3 tag reports 0.48. Do not attribute the 0.22 to today’s weights. Size alone did not guarantee useful transfer.

Three ways to adapt—and what each establishes

1. Call-time labels and descriptions. Choice options are supplied at runtime, not a permanently fixed classifier head. This permits a new label vocabulary without retraining. A nonsense label with a useful description may work. A nonsense label with no explanation cannot reveal an arbitrary private rule. Expressibility is documented; accuracy on your task must be measured.
2. Context and examples. Jev/Kev still consume text through state, instructions and criteria. Put labelled examples in that text. TypeSafe’s structured criteria guide explicitly demonstrates examples and boundaries. Whether examples in state improve your workload is empirical, not guaranteed by that API. No text generation is required for contextual conditioning.
3. Training or fine-tuning. Kev publishes training code and LoRA/head checkpoints; you can train on your own cases. This is one possible path, not automatically the best or only answer. Hosted Jev does not offer per-account fine-tuning in the inspected documentation. For Jev, use request context, rules, decomposition or a downstream model.

A paired experiment you can actually inspect

The labels veln, sova and tarn are invented workshop categories. Their exact names were invented for this page; we cannot prove a string never appeared in pretraining. Keep the test input and expected answer fixed. Compare a request without examples to a request with them, then try descriptive, opaque and deliberately swapped definitions. Any confusion is a result, not something the UI smooths away.

No real-model measurements recorded yet. Select Jev with a key or a local Kev server to measure. Illustrative mode demonstrates the table only; both arms receive the same synthetic distribution. A probability moving is not proof of calibration. One labelled case is not an evaluation set.

Not measured. Choose a provider, then run.

No trial yet.

What to conclude—and what not to

Correctness is the selected label versus your expected label. Expected-label probability shows whether the distribution moves toward the answer. Multiclass Brier score here is the sum of squared errors across all options (0 is perfect, maximum 2), not a calibration estimate. The gate reports confidence at your current threshold. It can accept a confidently wrong answer.

Opaque labels with no examples are an intentionally under-specified task: do not count a wrong guess as failure to recover information never supplied. Swapping descriptions creates a different policy; either update your ground truth or explicitly study contradictory instructions. Keep separate training, few-shot and held-out cases. A single successful pair is not generalization.

A business with unpublished rules

Start by separating exact rules from semantic judgments. If eligibility is a hard threshold, keep it in code. Use a decision model only where meaning is ambiguous. Proprietary rules can be supplied at request time, kept on a local server or learned from labelled historical cases; none requires publishing the rules. But sending them to a hosted API is still disclosure to that provider, governed by its terms.

For credit, mortgage or fraud workflows, this is research guidance—not a recommendation to automate consequential decisions. Historical outcomes can encode discrimination or outdated policy. Data minimization, authorization, legal review, fairness testing, appeal and human review requirements belong in the system from the start. The original Kev-0.5B card expressly excludes production decisions affecting people.

A realistic Kev fine-tuning path

1. Define a rubric and a baseline rule system. Obtain authorized, de-identified labelled decisions, disagreement labels and representative abstention cases. There is no universal sufficient number N; choose sample size through learning curves and risk bounds. A toy set of 100–500 invented cases can exercise plumbing, not establish business fitness.
2. Inspect upstream training code and fine-tuning guide. Match its state/question/options/target format, tokenizer and checkpoint. Keep documents and near-duplicates in the same split.
3. Train the adapter and pointer head or initialize from a compatible checkpoint. Retain base model licensing, hardware and memory requirements. Replaying old training cases can reduce forgetting.
4. Calibrate on development data, then compare against your original rules and a simple classifier on a untouched test partition: false positives, false negatives, coverage, subgroup behavior and cost—not accuracy alone.
5. Run in shadow mode, record revisions and rollback criteria, then add bounded human-reviewed actions if justified. Never let confidence override hard business constraints.

Not run here: no adapter was trained. The current Kev-4B card reports a 9-minute delta fine-tune on an H100 for its specific data and setup, and a roughly 9GB bf16 serving footprint. Those are upstream measurements, not promises for your laptop. The card includes the --init_from recipe and data/replay settings. A local training demo requires that environment, weight download, an authorized labelled set and a measured holdout; this static site cannot train it.