Class EdgeChatClient

Namespace

Qavren.Edge.Chat

Assembly

Qavren.Edge.Chat.Onnx.dll

MEAI IChatClient over ORT GenAI.

public sealed class EdgeChatClient : IChatClient, IDisposable

Inheritance

object

EdgeChatClient

Implements

IChatClient

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Thread safety. src/ort_genai_c.h states flatly "This API is not thread safe", while IChatClient's own contract requires every member to be safe for concurrent use. Both are honoured by SERIALISATION, not parallelism: one async gate per model holds for a whole turn, and a second concurrent caller queues rather than allocating a second Generator and therefore a second full KV cache. On a phone that is the point; on a server it is a throughput ceiling, and the README says so rather than letting someone read it as a bug.

One implementation, not two. GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken) is literally GetStreamingResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken) aggregated, because two independent implementations is how the streaming and non-streaming paths drift. Production runs on one Task.Run writing a bounded channel of 64 updates; the iterator reads it, so the caller's UI thread never enters native code and backpressure is real.

Nothing native is touched until the first turn. Resolving this client constructs no GenAI object; the model is borrowed from IChatModelHost per turn, or held for as long as the conversation cache keeps a generator alive.

Properties

Model

What the host last knew about the model, or null if it has never loaded one.

ModelId

The preset manifest's model id, which is also ChatClientMetadata.DefaultModelId.

Statistics

What this client has done since it was resolved.

Methods

Dispose()

Releases this client's lease and conversation cache. Does not dispose the shared model - the host owns it.

GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken)

Implemented as GetStreamingResponseAsync(...).ToChatResponseAsync(ct), so the two paths cannot diverge, plus one hoist: MEAI's aggregation lands an update's AdditionalProperties on the aggregated assistant message, and spec section 6.6 puts the turn record on the ChatResponse, so the two documented keys are copied up. Nothing else differs between the two paths.

GetService(Type, object?)

Resolves, in order: this when serviceKey is null && serviceType.IsInstanceOfType(this); ChatClientMetadata with ProviderName "onnxruntime-genai" and DefaultModelId = ModelId; ChatModelInfo; ChatClientStatistics; IChatModelHost; and, while this client holds the model, Model, Tokenizer and Config.

This method is the entire integration surface for three downstream consumers and is a contract, not an implementation detail: Semantic Kernel's GetModelId() is literally GetService<ChatClientMetadata>()?.DefaultModelId, Agent Framework's ChatClientAgent avoids double-wrapping by calling GetService<FunctionInvokingChatClient>(), and MEAI's own GetRequiredService<T>() goes through it. Returning the raw Model is also what lets a consumer build OnnxRuntimeGenAIChatClient, MultiModalProcessor or LoRA Adapters themselves without sub-project 4 wrapping any of it.

The three native objects are returned only while this client holds a lease - which, with the conversation cache on, is from the first turn until the cache is dropped. A model the host may dispose under the caller is never handed out.

GetStreamingResponseAsync(IEnumerable<ChatMessage>, ChatOptions?, CancellationToken)

Sends chat messages and streams the response.