Method CountTokens

Namespace

Qavren.Edge.Embeddings.Onnx

Assembly

Qavren.Edge.Embeddings.Onnx.dll

CountTokens(ReadOnlySpan<char>)

Counts the tokens in text.

int CountTokens(ReadOnlySpan<char> text)

Parameters

text ReadOnlySpan<char>

The text to count.

Returns

int

The token count.