Table of Contents

Class ChatMemoryBudgetOptions

Namespace
Qavren.Edge.Chat
Assembly
Qavren.Edge.Chat.Onnx.dll

Everything the memory gate reads, and the one hook that replaces it.

public sealed class ChatMemoryBudgetOptions
Inheritance
ChatMemoryBudgetOptions
Inherited Members

Properties

ContextLadder

Contexts tried in order until one fits. The first entry at or below the requested length is the starting rung. An explicit ladder rather than repeated halving, so the diagnostics block reports a value a reader recognises.

MinContextTokens

Below this, refuse rather than degrade further.

MinTotalMemoryBytes

Refuse outright on a device whose total memory is below this, whatever it claims is free. Return null to disable the gate for a preset. Applied only when TotalMemoryBytes is known and TotalMemorySource is a real physical-memory reading - see spec section 9.3, which skips the floor entirely for GcMemoryInfo.

The floor is compared against a number the platforms do not agree on, so it is set below the nominal figure on purpose. Android's ActivityManager.MemoryInfo.TotalMem excludes memory the kernel reserved before userspace saw it and reports roughly 5.4-5.7 GiB on a device marketed as 6 GB; Apple's NSProcessInfo.PhysicalMemory reports the full nominal 6,442,450,944. A floor of exactly 6 GiB would therefore refuse the default preset on essentially every nominal-6 GB Android device while admitting every nominal-6 GB iPhone - the precise opposite of the intended calibration. The defaults are set at about 0.85x nominal so the same constant means the same device class on both platforms:

  • weights over 1 GiB -> 5.0 GiB (a nominal-6 GB device, by either reading)
  • weights over 200 MiB -> 3.4 GiB (a nominal-4 GB device)
  • weights at or below 200 MiB -> null, no floor at all

The third band is not a courtesy. It is what keeps the gate from refusing the tier-2 test fixture: a ~500 KB random-weight model has no device floor worth enforcing, and a default Android emulator on a hosted runner reports about 2 GiB of TotalMem, so a blanket floor would specify the sub-project's highest-value device assertion into a guaranteed 7006. A floor is a property of the preset's weight class, and below 200 MiB there is no class to gate.

Spec section 19 item 10 is the measurement that calibrates all three numbers against real TotalMem and PhysicalMemory readings from the device lanes, because until then the 0.85x is an engineering estimate like every other constant here.

Override

Replaces the whole computation. The last word belongs to whoever measured the device.

PreferMeasuredPeak

Take max(arithmetic, ChatModelShape.MeasuredPeakBytes + ReserveBytes) when a measurement exists. Spec section 9.3 is the normative formula; the reserve is added to the measured term because a measured peak is the model's resident footprint and owes the rest of the app no headroom.

RefuseWhenUnknown

Default false, and that is sub-project 2's rule restated: a reading nobody can make is never a refusal. Apple documents os_proc_available_memory() returning 0 for "unknown or already over", and inverting this would make the feature unusable on desktop. True makes an unknown reading fatal, for a kiosk build that would rather not boot.

ReserveBytes

Headroom left to the rest of the app: UI, images, the SQLite page cache. Default 192 MiB.

SystemWideMemoryFraction

What fraction of a SystemWide reading to believe. Default 0.60. Ignored for PerProcess.

WorkspaceBytes

Transient allocation beyond weights and KV: the logits buffer, the sampler copy, ORT arenas, the tokenizer. Default 192 MiB. An engineering estimate, and the refusal message says so in the same words sub-project 2's pre-flight already uses.

Edit this page