Table of Contents

Property SpecialTokenOverhead

Namespace
Qavren.Edge.Ingestion.Onnx
Assembly
Qavren.Edge.Ingestion.Onnx.dll

SpecialTokenOverhead

Tokens the encoder adds that CountTokens(ReadOnlySpan<char>) does not report.

public int SpecialTokenOverhead { get; }

Property Value

int