MA
← Selected systemsENGINEERING CASE STUDY

Evidence-grounded analysis · multi-format ingestion

Universal AI Analyzer

A domain-neutral Python service that snapshots heterogeneous sources, extracts and profiles them deterministically, runs hierarchical source-grounded analysis, validates every cited source/chunk relationship, and renders bounded Telegram reports.

SystemDomain-neutral multi-format source-grounded analysis engine
Core boundaryDeterministic extraction + strict evidence references around LLM analysis
InterfacesTelegram commands · local files · Google Docs / Sheets

Architecture

Treat every source as untrusted evidence, not prompt context.

The system separates owner objective from source payload, gives every snapshot and chunk stable local identity, and validates source ownership before any referenced conclusion can leave the analysis boundary.

Source coverage

One analysis boundary across heterogeneous evidence.

The source matrix is deliberately explicit: supported formats are accepted through bounded parsers; unsupported visual/scanned inputs fail rather than being silently treated as readable evidence.

01

Text & code

Telegram text · local text/code/config files

02

Tables

CSV · TSV · XLS · XLSX · ODS

03

Documents

DOCX · PPTX · ODT · text-based PDF

04

Structured

JSON · JSONL · YAML · XML · HTML

05

Extended

EPUB · EML · Jupyter notebooks

06

Google

Sheets · Docs

EXPECTED REJECTIONScanned PDF without extractable text

No OCR/Vision claim. The source is rejected instead of hallucinating content from an unreadable file.

Deterministic layer

Before the model reasons, the system computes what it can prove.

Table profiling is domain-neutral and deterministic, creating inspectable statistics that do not depend on generation quality.

STRUCTURE

Shape & completeness

Row and column counts, duplicates, missingness, and distinct-value counts.

NUMERIC

Distribution signals

Ranges, mean, median, and IQR-based outlier detection after Western/Persian/Arabic numeric normalization.

FORMULAS

Unknown stays unknown

Unresolved formulas are represented explicitly instead of being evaluated or fabricated.

PROVENANCE

Local identity

Stable source and chunk identifiers plus bounded snapshot byte counts and SHA-256 provenance.

LLM boundary

Generation is permitted one repair attempt, then the system fails closed.

Strict schemas reject extra fields; cited chunks must exist, belong to the cited source, and collectively cover the reviewed sources. One bounded semantic repair is allowed before rejection.

MODEL ALLOWED

Hierarchical source-grounded synthesis

Map/reduce analysis can recursively reduce evidence while the owner objective remains serialized separately from untrusted source payload.

MODEL NOT AUTHORITY

Source identity and report validity

The model cannot invent source IDs, cross source/chunk ownership boundaries, bypass full reviewed-source coverage, or emit unsafe Telegram markup.

Benchmark surface

What the system is designed to measure.

MEASUREMENT CONTRACT

Analysis quality starts with source fidelity: unreadable, uncited, or ownership-invalid evidence must fail before synthesis is accepted.

Format matrices, malformed-input suites, source/chunk mutation tests, deterministic profiling fixtures, renderer fuzzing, and provider-failure replay.
01Format coverageaccepted + expected rejection matrix
02Extraction fidelitycontent preserved through parsing
03Citation validitysource/chunk ownership checks
04Profile accuracydeterministic table statistics
05Renderer safetybounded output under fuzz
06Failure recoveryprovider/schema/repair behavior

Evidence

Release evidence with promotion boundaries.

SignalEvidenceInterpretation boundary
Targeted regression/security/fidelity tests64 passedVerifiedDependency-available release checks; not represented as the full pytest/coverage suite.
Source matrix25 fixtures · 24 accepted · 1 expected scanned-PDF rejectionVerifiedOCR/Vision for scanned PDFs/images is not supported.
Compact offline E2E24 sources · 34 chunks · 5 simulated LLM calls · 2 Telegram pagesSyntheticSimulated provider path; not live semantic-quality proof.
Long offline E2E24 sources · 34 chunks · 5 simulated LLM calls · 26 Telegram pagesSyntheticOffline evidence only.
Renderer fuzz500 reports · 3,816 pages · max 3,638 UTF-16 unitsVerifiedRenderer boundary, not end-to-end delivery proof.
Wheel structure48 RECORD entries verifiedVerifiedPackage structure verified; production deployment not implied.

Failure contract

When evidence becomes unreadable or invalid, the analyzer stops before synthesis becomes fiction.

Unreadable source → reject. Scanned or unsupported content is never silently treated as extracted evidence.

Invalid source/chunk reference → reject. Citations must resolve to a real chunk owned by the cited source.

Malformed model output → one bounded repair. Schema repair is finite; repeated invalid generation fails closed.

Provider failure → no fabricated report. Analysis availability is allowed to fail without weakening source fidelity.