Class TokenizedBatch

Namespace

Qavren.Edge.Embeddings.Onnx

Assembly

Qavren.Edge.Embeddings.Onnx.dll

One padded batch, row-major, ready to wrap in OrtValues.

The three long[] are RENTED from ArrayPool<T>.Shared by EncodeBatch(IReadOnlyList<string>, int, IReadOnlyList<int>), so each one may be LONGER than Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength. Spec 11 pools them because an ingest batch would otherwise allocate 3 * BatchSize * SequenceLength longs per batch on the one path that runs once per document. Every layout contract below is therefore over the FIRST Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength elements: array.AsMemory(0, TensorLength) is what reaches OrtValue.CreateTensorValueFromMemory, and array.Length carries no meaning. Anything past that window is whatever the previous renter left behind.

public sealed record TokenizedBatch : IEquatable<TokenizedBatch>

Inheritance

object

TokenizedBatch

Implements

IEquatable<TokenizedBatch>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

TokenizedBatch(long[], long[], long[], int, int, int[], bool[])

One padded batch, row-major, ready to wrap in OrtValues.

The three long[] are RENTED from ArrayPool<T>.Shared by EncodeBatch(IReadOnlyList<string>, int, IReadOnlyList<int>), so each one may be LONGER than Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength. Spec 11 pools them because an ingest batch would otherwise allocate 3 * BatchSize * SequenceLength longs per batch on the one path that runs once per document. Every layout contract below is therefore over the FIRST Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength elements: array.AsMemory(0, TensorLength) is what reaches OrtValue.CreateTensorValueFromMemory, and array.Length carries no meaning. Anything past that window is whatever the previous renter left behind.

Properties

AttentionMask

1 for a real token, 0 for padding, same layout and window as InputIds. This is the mask this library SYNTHESISES - never anything Microsoft.ML.Tokenizers returned. There is no attention-mask API in that library, and GetSpecialTokensMask is not one: on its default path it ignores the ids and emits 1, 0...0, 1, and on the other it marks special tokens. Either way it is the inverse of what ONNX wants.

BatchSize

How many inputs this batch carries.

InputIds

Row-major and flat over its first BatchSize * SequenceLength elements: row b occupies [b * SequenceLength, (b+1) * SequenceLength). That is exactly the layout OrtValue.CreateTensorValueFromMemory expects, so no reshape happens anywhere.

The array may be LONGER than BatchSize * SequenceLength. The three buffers a batch from EncodeBatch(IReadOnlyList<string>, int, IReadOnlyList<int>) carries are rented from ArrayPool<T>.Shared, which returns a buffer AT LEAST the requested size. Read BatchSize * SequenceLength elements, never InputIds.Length. Spec 7's declaration of this record annotates the three arrays [BatchSize * SequenceLength], which describes the WINDOW rather than the allocation; that sentence needs the edit, and the pooling it describes away is mandated by the plan's Step 6.

SequenceLength

The padded width every row was padded to.

TokenCounts

Per input, pre-padding, INCLUDING [CLS] and [SEP]. The only source for GeneratedEmbeddings.Usage's token count: summing AttentionMask gives the same number today and would silently stop doing so the moment anything masked a non-padding position.

TokenTypeIds

All zeros for a single-sequence encoder; same layout and window.

Truncated

Per input. Raises EmbeddingInputTooLong (5105) when Truncation is Throw, and logs EmbeddingInputTruncated (701) when it is Truncate

• which is the default, so the out-of-the-box behaviour on an over-long input is a logged truncation, never an exception.