Class EdgeChatTokenBudgetReducer

Namespace

Qavren.Edge.Chat

Assembly

Qavren.Edge.Chat.Onnx.dll

Implements MEAI's stable IChatReducer. System messages are preserved, pinned messages are preserved, then whole groups are evicted oldest-first until the token budget or MaxTurns binds, with MinimumPreservedMessages as the floor. Counts with the model's own tokenizer, not an estimate. Never throws.

public sealed class EdgeChatTokenBudgetReducer : IChatReducer

Inheritance

object

EdgeChatTokenBudgetReducer

Implements

IChatReducer

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

"Whole turns" needs a definition once the tail is not turn-shaped, so here it is. An eviction group is a non-system, non-pinned message together with every message after it up to but excluding the next ChatRole.User message. On an ordinary history that is exactly a user/assistant pair, which is the property the phrase is reaching for: an assistant message is never evicted without the user message it answered. On a history whose tail is user-then-user - which is what the RAG recipe produces, because it injects a pinned context message immediately before the real question - the pinned message is not in any group at all, and the real user message opens the newest group. Nothing is left dangling and nothing needs a special case.

The floor is system messages + pinned messages + the newest MinimumPreservedMessages messages, and the reducer stops there even if the budget is still exceeded. What happens next is the turn's job: it fails with ChatPromptTooLong (7102). That is the deliberate trade - an answer that silently lost its sources is worse than a turn that says the context did not fit and names RagOptions.MaxContextTokens as the knob.

Public, and applied internally. ReducingChatClient, UseChatReducer, MessageCountingChatReducer and SummarizingChatReducer are all [Experimental], and sub-project 4 does not put an experimental attribute on a shipped public surface - but IChatReducer itself is stable, so a consumer who accepts that warning can hand this instance straight to UseChatReducer.

Constructors

EdgeChatTokenBudgetReducer(Func<string, int>, ChatHistoryOptions)

Creates the reducer.

Methods

ReduceAsync(IEnumerable<ChatMessage>, CancellationToken)

Reduces a history to fit the budget, preserving system and pinned messages.