Constructor TokenizedBatch

Namespace

Qavren.Edge.Embeddings.Onnx

Assembly

Qavren.Edge.Embeddings.Onnx.dll

TokenizedBatch(long[], long[], long[], int, int, int[], bool[])

One padded batch, row-major, ready to wrap in OrtValues.

The three long[] are RENTED from ArrayPool<T>.Shared by EncodeBatch(IReadOnlyList<string>, int, IReadOnlyList<int>), so each one may be LONGER than Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength. Spec 11 pools them because an ingest batch would otherwise allocate 3 * BatchSize * SequenceLength longs per batch on the one path that runs once per document. Every layout contract below is therefore over the FIRST Qavren.Edge.Embeddings.Onnx.TokenizedBatch.TensorLength elements: array.AsMemory(0, TensorLength) is what reaches OrtValue.CreateTensorValueFromMemory, and array.Length carries no meaning. Anything past that window is whatever the previous renter left behind.

public TokenizedBatch(long[] InputIds, long[] AttentionMask, long[] TokenTypeIds, int BatchSize, int SequenceLength, int[] TokenCounts, bool[] Truncated)

Parameters

InputIds long[]

Row-major and flat over its first BatchSize * SequenceLength elements: row b occupies [b * SequenceLength, (b+1) * SequenceLength). That is exactly the layout OrtValue.CreateTensorValueFromMemory expects, so no reshape happens anywhere.

The array may be LONGER than BatchSize * SequenceLength. The three buffers a batch from EncodeBatch(IReadOnlyList<string>, int, IReadOnlyList<int>) carries are rented from ArrayPool<T>.Shared, which returns a buffer AT LEAST the requested size. Read BatchSize * SequenceLength elements, never InputIds.Length. Spec 7's declaration of this record annotates the three arrays [BatchSize * SequenceLength], which describes the WINDOW rather than the allocation; that sentence needs the edit, and the pooling it describes away is mandated by the plan's Step 6.

AttentionMask long[]

1 for a real token, 0 for padding, same layout and window as InputIds. This is the mask this library SYNTHESISES - never anything Microsoft.ML.Tokenizers returned. There is no attention-mask API in that library, and GetSpecialTokensMask is not one: on its default path it ignores the ids and emits 1, 0...0, 1, and on the other it marks special tokens. Either way it is the inverse of what ONNX wants.

TokenTypeIds long[]

All zeros for a single-sequence encoder; same layout and window.

BatchSize int

How many inputs this batch carries.

SequenceLength int

The padded width every row was padded to.

TokenCounts int[]

Per input, pre-padding, INCLUDING [CLS] and [SEP]. The only source for GeneratedEmbeddings.Usage's token count: summing AttentionMask gives the same number today and would silently stop doing so the moment anything masked a non-padding position.

Truncated bool[]

Per input. Raises EmbeddingInputTooLong (5105) when Truncation is Throw, and logs EmbeddingInputTruncated (701) when it is Truncate

• which is the default, so the out-of-the-box behaviour on an over-long input is a logged truncation, never an exception.