Method KvCacheBytes

Namespace

Qavren.Edge.Chat

Assembly

Qavren.Edge.Chat.Onnx.dll

KvCacheBytes(int)

KV bytes for a context, clamped by SlidingWindow when the model declares one.

public long KvCacheBytes(int contextTokens)

Parameters

contextTokens int

The context length in tokens.

Returns

long

The KV cache size in bytes at that context.