Table of Contents

Class ChatModelShape

Namespace
Qavren.Edge.Chat
Assembly
Qavren.Edge.Chat.Onnx.dll

The decoder geometry the memory budget is computed from. A preset declares it - the budget has to refuse before 1.24 GB is mapped, and reading the shape at load time is too late - and the loader cross-checks the declaration against the provisioned genai_config.json field by field, so a republished model folder is ChatModelShapeMismatch rather than a wrong budget.

public sealed record ChatModelShape : IEquatable<ChatModelShape>
Inheritance
ChatModelShape
Implements
Inherited Members

Properties

ContextLength

model.context_length. The hard ceiling on search.max_length.

DecoderFileName

model.decoder.filename.

HeadSize

model.decoder.head_size.

KvCacheBytesPerElement

Bytes per KV cache element. Default 2 (fp16).

This is the one field the cross-check cannot validate. genai_config.json states layers, KV heads and head size but says nothing about the KV dtype, so a disagreement here is undetectable at load and shows up only as a budget that is wrong by a factor of two. 2 is the calibrated default, not a guess: for Llama32_1BInstructInt4 it puts the arithmetic within 1.2% of a measured 1280 MiB peak, where 4 would put it 11% over. Spec section 19 item 4 is the nightly measurement that answers it per preset, and MeasuredPeakBytes is what absorbs the error meanwhile.

KvCacheBytesPerToken

layers x 2 (K and V) x kvHeads x headSize x bytesPerElement.

MeasuredOn

What device and runtime MeasuredPeakBytes came from. Never a guess.

It is a diagnostics string and nothing else: spec section 9.3's formula reads MeasuredPeakBytes alone and cannot discount a measurement by provenance. So the rule is on the way in, not on the way out - MeasuredPeakBytes is set only from a measurement taken on a device of the class the preset is meant to run on. A server or laptop peak is recorded here with MeasuredPeakBytes left null, which is what Qwen3_600MInt4 does with its published Graviton figure.

MeasuredPeakBytes

A peak RSS measured on a real device at ContextLength, or null. When present and PreferMeasuredPeak is set, the budget takes max(arithmetic, measured + ReserveBytes) - spec section 9.3 is the normative formula - and records which term bound in UsedMeasuredPeak. The reserve is added to the measured term because a measured peak is the model's own resident footprint and carries no headroom for the rest of the app. Pure arithmetic underestimates large-vocabulary models badly - Gemma-3-1b computes ~929 MiB and measures 1502 MiB - so a preset without a measurement is running on arithmetic alone and the diagnostics block says so.

For both shipped presets this term is inert today, and that is stated rather than implied. Llama32_1BInstructInt4's measured 1280 MiB plus a 192 MiB reserve is 1472 MiB, below its own arithmetic of 1679 MiB at 4096 tokens, so the arithmetic binds; Qwen3_600MInt4 has no device measurement at all. The machinery exists for the case it was built for - a large-vocabulary preset whose arithmetic under-reports, which is exactly Gemma-3-1b - and spec section 19 item 3 is what would make it bind here.

ModelType

model.type: "llama", "qwen3", "phi3". Diagnostics only.

NumHiddenLayers

model.decoder.num_hidden_layers.

NumKeyValueHeads

model.decoder.num_key_value_heads.

SlidingWindow

model.decoder.sliding_window when the model declares one; null otherwise.

VocabSize

model.vocab_size.

WeightsBytes

Resident weight estimate: the sum of the manifest's Graph and GraphExternalData file sizes, and not TotalSizeBytes - a 17 MB tokenizer.json is not resident weights, and counting it would over-report by tens of megabytes inside a user-visible refusal message.

Methods

FromGenAiConfig(string, long)

Reads the geometry out of a genai_config.json body.

KvCacheBytes(int)

KV bytes for a context, clamped by SlidingWindow when the model declares one.

TryFromGenAiConfig(string, long, out ChatModelShape?, out string?)

Reads the geometry out of a genai_config.json body without throwing.

Edit this page