Method CountTokens

Namespace

Qavren.Edge.Ingestion

Assembly

Qavren.Edge.Ingestion.dll

CountTokens(ReadOnlySpan<char>)

The WordPiece (or equivalent) cost of text, specials EXCLUDED.

int CountTokens(ReadOnlySpan<char> text)

Parameters

text ReadOnlySpan<char>

Returns

int