Property TokenCounts

Namespace

Qavren.Edge.Embeddings.Onnx

Assembly

Qavren.Edge.Embeddings.Onnx.dll

TokenCounts

Per input, pre-padding, INCLUDING [CLS] and [SEP]. The only source for GeneratedEmbeddings.Usage's token count: summing AttentionMask gives the same number today and would silently stop doing so the moment anything masked a non-padding position.

public int[] TokenCounts { get; init; }

Property Value

int[]