Class EdgeChatOptions

Namespace: Qavren.Edge.Chat

Assembly: Qavren.Edge.Chat.Onnx.dll

Everything one registered chat client reads.

public sealed class EdgeChatOptions

Inheritance: object

EdgeChatOptions

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

ConfigOverlayJson: Config.Overlay(json), applied last before load. The whole execution-provider escape hatch, and deliberately the only one: GenAI names providers as strings inside genai_config.json and has no CoreML and no NNAPI provider, so a typed policy object would model choices that do not exist on any platform this suite ships to. Setting any provider other than CPU on a mobile TFM is refused with ChatExecutionProviderUnsupported naming the provider, rather than being silently ignored by the native config parser.

DropConversationCacheOnSleep: Drop the cached Generator - not the model - on Sleeping. Default true: the KV cache is the half that is cheap to rebuild and expensive to be killed for.

DropOnMemoryPressure: Dispose the model itself on MemoryPressure(Critical). Default true.

EnableConversationCache

Keep ONE Generator alive between turns, keyed by ChatOptions.ConversationId, so a follow-up skips prefill.

A null id never hits the cache. Not "matches the cached null" - never hits. Two unrelated conversations that both leave the field null would otherwise share one KV cache and therefore one history, which is a correctness bug wearing a performance optimisation's clothes. Null means "build me a fresh generator and tell me its id".

A cached generator is built once with max_length = resolvedContext - the budget's answer, and the KV memory cap - while the per-turn output cap is a managed generated-token counter in the decode loop. SetSearchOption exists only on GeneratorParams, which is consumed at Generator construction, so "set max_length on every turn including a cached one" is not implementable and sub-project 4 does not pretend to.

Guidance: How a ChatResponseFormatJson request is treated. Default Disabled.

History: History reduction.

LoadTimeout: Bounds new Model(config) plus tokenizer construction. Default 2 minutes.

MaxContextTokens: Null derives the value from the budget, capped by the preset's default and by Shape.ContextLength. Setting it turns the budget into a refusal rather than a cap: a value that does not fit throws ChatInsufficientMemory carrying the largest context that would have fit.

MaxOutputTokens: Null falls through to Preset.DefaultMaxOutputTokens. A per-call ChatOptions.MaxOutputTokens beats both.

MaxQueuedTurns: Turns beyond this waiting for the model gate are refused with ChatBusy. Default 4.

Memory: The memory gate's constants.

ModelDirectoryOverride: An already-staged model directory, bypassing provisioning. Tests and the nightly lane.

Preset: Set by AddOnnxChat(preset, …). The callback may read it; replacing it is unsupported.

PromptFormatter: Overrides Tokenizer.ApplyChatTemplate. The escape hatch for a model whose Jinja template minja cannot parse - which surfaces on the FIRST ApplyChatTemplate call and not at load, which is why the warm-up task probes it.

Provisioning: Provisioning pre-flight.

RequireChatTemplate: Throw ChatTemplateUnsupported rather than falling back. Default true.

ReservedPromptTokens: Tokens held back from the prompt so a reply always has room. Default 64.

SearchOptions

Raw GeneratorParams.SetSearchOption entries, applied last so they win over everything mapped from ChatOptions.

The native API has exactly two overloads - SetSearchOption(string, double) and SetSearchOption(string, bool) - so this dictionary's values are converted or refused, never quietly dropped. That is deliberately the opposite of the rule for ChatOptions.AdditionalProperties, where an unrecognised value is ignored without error: that one is an open bag shared with the rest of the MEAI pipeline, while this one exists for exactly one purpose and nothing else writes to it, so a "0.7" typed as a string is a typo - and a typo in the escape hatch must not become a temperature that was never applied.

max_length is refused whatever its type: it is the memory cap, and the whole gate depends on the generator getting the value the budget chose.

StopSequences: Added to Preset.StopSequences, never replacing them: stop sequences are a property of the model's template as much as of the caller's intent, and a consumer adding "\nUser:" must not thereby delete the preset's <|eot_id|>. The effective set is the ordinal-distinct union of the preset's, these, and ChatOptions.StopSequences, sorted longest-first.

SystemPrompt: Becomes a leading ChatRole.System message; nothing is dropped.

Temperature: Null falls through to Preset.DefaultTemperature; a per-call ChatOptions.Temperature beats both.

Thermal: Thermal pacing and refusal.

TopK: Null falls through to Preset.DefaultTopK; ChatOptions.TopK beats both.

TopP: Null falls through to Preset.DefaultTopP; ChatOptions.TopP beats both.

TurnQueueTimeout: How long a queued turn waits for the gate. Default 30 s.

UnloadOnSleeping: Unload on Sleeping. Default false: an app backgrounded for two seconds should not pay a 1.6 s reload, and Sleeping already stops the in-flight turn and drops the KV cache either way.

Table of Contents

Class EdgeChatOptions

Properties