Property EnableConversationCache
EnableConversationCache
Keep ONE Generator alive between turns, keyed by ChatOptions.ConversationId,
so a follow-up skips prefill.
A null id never hits the cache. Not "matches the cached null" - never hits. Two unrelated conversations that both leave the field null would otherwise share one KV cache and therefore one history, which is a correctness bug wearing a performance optimisation's clothes. Null means "build me a fresh generator and tell me its id".
A cached generator is built once with max_length = resolvedContext - the
budget's answer, and the KV memory cap - while the per-turn output cap is a managed
generated-token counter in the decode loop. SetSearchOption exists only on
GeneratorParams, which is consumed at Generator construction, so "set
max_length on every turn including a cached one" is not implementable and sub-project 4
does not pretend to.
public bool EnableConversationCache { get; set; }