Property KvCacheBytesPerToken
KvCacheBytesPerToken
layers x 2 (K and V) x kvHeads x headSize x bytesPerElement.
public long KvCacheBytesPerToken { get; }layers x 2 (K and V) x kvHeads x headSize x bytesPerElement.
public long KvCacheBytesPerToken { get; }