Class ChatPresets

Namespace

Qavren.Edge.Chat

Assembly

Qavren.Edge.Chat.Onnx.dll

The shipped catalogue, generated by chat/tools/model-hashes/fetch_chat_model_hashes.py into ChatPresets.g.cs with literal SHA-256s and a pinned commit revision, exactly as sub-project 2's EmbeddingPresets.g.cs is. The script reads each repo's own genai_config.json and emits the ChatModelShape from it, so no geometry value in this catalogue is hand-typed.

There is deliberately no default preset. The two entries carry different licences, and a silently-chosen 1.24 GB download under the Llama 3.2 Community Licence is not a default anyone should inherit. AddOnnxChat takes a preset as a required argument.

Each preset's sampling defaults are the model's own shipped search values, not this type's defaults (plan adjustment 15): Llama ships 0.6 / 0.9 / 50 and Qwen ships 0.6 / 0.95 / 20, and a preset that silently sampled differently from the publisher's own genai_config.json would be a quality regression nobody would attribute. ChatPreset's own defaults stay as declared, for a preset somebody writes by hand.

public static class ChatPresets

Inheritance

object

ChatPresets

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Properties

All

Every shipped preset, in catalogue order.

Llama32_1BInstructInt4

Arm/llama-3-2-1b-instruct-onnx-genai-int4-kquantlast-emb-int8-vivo-x300. 1.241 GB decimal on disk across six files; 1167.52 MiB of resident weights; context 4096 as shipped (the publisher patched it down from Llama's stock 131072 so the KV cache fits a phone). Geometry: 16 layers, 8 KV heads, head size 64, vocab 128256 - 32 KiB of KV per token. Measured on a vivo X300 (Android 16, ORT 1.27.0 CPU EP, 4 threads): 30.564 tok/s decode, TTFT 950 ms, peak RSS 1280 MiB, model load 1.61 s, MMLU 5-shot 44.2%. No iOS figure exists for this or any other ORT GenAI model.

Qwen3_600MInt4

Arm/qwen3-0-6b-onnx-genai-int4-kquantlast-emb-int4. 495 MB decimal on disk across six files; 461.25 MiB of resident weights; Apache-2.0 - the preset for an app that cannot take the Llama terms, and the model the nightly lane uses.

Smaller on disk is not smaller in memory, and here are the numbers. 28 layers, 8 KV heads, head size 128 - 112 KiB of KV per token, 3.5x this catalogue's larger preset, against a third of the weights. At 4096 tokens its KV cache alone is 448 MiB. Its declared context_length is 40960, so a generator built with no max_length would allocate 4480 MiB of KV cache - which is exactly the jetsam scenario the memory budget and the belt-and-braces search.max_length exist to prevent.

Its only published throughput is AWS Graviton (70.4 tok/s, peak 838 MB decimal), and that figure is deliberately not in MeasuredPeakBytes: the budget reads that field alone and has no way to discount a measurement by where it was taken, so a server peak entering it would silently become a phone budget.

Methods

ById(string)

Looks a preset up by Id.