Class EdgeDocumentConverter
- Namespace
- Qavren.Edge.Ingestion.DataIngestion
- Assembly
- Qavren.Edge.Ingestion.DataIngestion.dll
Converts between SP3's ExtractedDocument and MEDI's IngestionDocument, in both directions (spec 11, plan task 6.3 step 1).
Lossless in both directions, except for two things, both by design.
-
Spans. MEDI's model has no character offsets: an element is its text. ToMedi(ExtractedDocument)
slices each block out of Text and FromMedi(IngestionDocument, ILogger?) rebuilds
a text buffer by joining the element texts with a blank line, assigning fresh
[Start, End)offsets into that buffer. Every block's text, kind, heading level and page number survive the round trip; the original offsets do not. - Images. DocumentBlockKind has no image kind (spec 6, suite decision 9), so an IngestionDocumentImage converts to nothing. FromMedi(IngestionDocument, ILogger?) logs event MediImagesDropped (917) once per document, with the count, never once per image.
What MEDI's vocabulary actually is. Spec 6 describes DocumentBlockKind as "MEDI's vocabulary". The shipped abstractions carry five concrete element types — paragraph, header, footer, table and image — so Heading, Footer and Paragraph map to a native element, a run of TableRow blocks becomes one IngestionDocumentTable (one row per block, one column), and the four kinds MEDI has no element for — ListItem, Code, Caption, Quote — ride on an IngestionDocumentParagraph carrying BlockKindKey in its metadata. The mapping is one switch over all eight members with no default bucket, so a ninth kind fails to compile here rather than silently becoming a paragraph.
Document-level facts (extractor id and version, media type, page count, text-layer flag,
warnings and Metadata) have no home on IngestionDocument
either, so they travel on the root IngestionDocumentSection's metadata under the
qedge. keys declared below. A MEDI document that did not come from SP3 simply has none of
them and gets the documented defaults.
public static class EdgeDocumentConverter- Inheritance
-
EdgeDocumentConverter
- Inherited Members
Fields
- BlockKindKey
Element metadata: the DocumentBlockKind name, on the four kinds MEDI has no element type for. Absent on every element whose type already says what it is.
- BlockSeparator
The separator FromMedi(IngestionDocument, ILogger?) places between two element texts.
- CellSeparator
The separator FromMedi(IngestionDocument, ILogger?) places between two cells of one table row.
- DefaultExtractorId
The extractor id a MEDI document that carries no SP3 facts reports.
- DefaultMediaType
The media type a MEDI document that carries no SP3 facts reports.
- DocumentMetadataPrefix
Root-section metadata prefix: every Metadata entry is stored as
qedge.meta.<key>.
- ExtractorIdKey
Root-section metadata: ExtractorId.
- ExtractorVersionKey
Root-section metadata: ExtractorVersion.
- HasTextLayerKey
Root-section metadata: HasTextLayer.
- MediaTypeKey
Root-section metadata: MediaType.
- PageCountKey
Root-section metadata: PageCount. Absent when null.
- WarningsKey
Root-section metadata: Warnings, carried as the list object itself. Absent when null.
Methods
- FromMedi(IngestionDocument, ILogger?)
MEDI to SP3. Walks every section in order, descending into nested sections, and rebuilds one text buffer with fresh offsets. A table yields one TableRow per row, its cells joined with CellSeparator. Images are dropped and counted; a non-zero count is event 917, once.
- ToMedi(ExtractedDocument)
SP3 to MEDI. One root section holds the document facts and one element per block, in block order. Zero-length blocks are skipped, because a MEDI element cannot be empty.