How we test rendering fidelity in SharePoint Server SE

When you replace Office Online Server, the first question from your users is simple: does my document still look right? This post describes how we answer that question for Office Render Engine, what we count as a defect, and which results we publish only once they are reproducible.

Faithful, not pixel-identical

Our fidelity bar is written down before any test runs, so a result cannot be argued into a pass afterwards.

A preview has to be faithful: the same text, the same tables and images in the same places, and the same page breaks. It does not have to be pixel-identical. No viewer in this category renders every document identically, Office Online Server included, as soon as the fonts on the server differ from the fonts on the author's laptop.

So we split differences into two groups.

Accepted:

• Substituted fonts that keep line breaks and page counts.
• Small differences in glyph shapes, kerning and spacing.
• Sub-line offsets of an element by a few pixels.

Defects we fix:

• Missing, truncated or unreadable content.
• Pagination that differs from the original, because it breaks page references in contracts, policies and audit reports.
• Objects placed so far off that the page no longer reads as intended.

The same bar is printed on our known limitations page, so you can hold us to it.

The test corpus

We test against a fixed, synthetic corpus rather than a pile of random files. Every document is generated deterministically and listed in a manifest with its hash, size, page or slide count and the features it exercises. A run therefore always tests exactly the same bytes, and two runs can be compared line by line.

The principal documents include:

• a 20-page Word document with headings, tables and footers,
• a large, image-heavy Word document,
• an Excel workbook with conditional formatting (colour scales and data bars), plus formulas, merged cells, charts, wide sheets and multi-sheet files,
• a 30-slide PowerPoint deck, plus decks with shapes, tables, speaker notes and sparse images,
• Word documents with nested tables, and with Arabic, Hebrew, Chinese and Japanese text.

Small and broken files belong in the corpus too: a one-page document, a single slide, malformed files and archive bombs, because a viewer that hangs on a bad file is as much a problem as one that draws a table wrong.

Customer documents never enter the corpus. When a partner provides samples, they are anonymised and kept out of the repository.

How a comparison works

Each document is rendered by the engine under test and by a reference, and a comparison tool checks the pair in four steps.

1. Pagination, checked exactly. Word pages and PowerPoint slides must match the reference count. One page more or less is a failure, not a tolerance.
2. Missing content. For every page the tool measures how much ink is on it. A page with far less ink than its reference is flagged as possibly missing content and goes straight to human review.
3. Ranking, not scoring. A block-wise structural similarity score (SSIM) orders the pages so that the most different ones are reviewed first. It is deliberately not a pass mark: a uniform shift of a few pixels lowers SSIM a lot and changes nothing for a reader, while a missing word barely moves it.
4. Visual review. People look at side-by-side pairs at 96 and 192 dpi, at overview grids of every page, and at the worst-ranked pages, and record a verdict per document.

Spreadsheets are the exception. A printed worksheet is cut into pages for paper, while a sheet on screen is not, so page counts mean little. Excel files are compared by eye, with colour scales, data bars, merged cells, number formats and charts checked one by one.

Fonts: substituted, and reported

Most visible differences between viewers come from fonts. Microsoft fonts such as Calibri and Cambria may not be redistributed, so a server without them uses metric-compatible substitutes: Carlito for Calibri and Caladea for Cambria. They have the same character widths, so lines break in the same places and page counts stay the same, even where the letterforms differ slightly.

We do not hide this. The service reports at start-up which expected fonts are missing, and every .docx, .xlsx and .pptx preview that needed a substitute records which font families it replaced. If your organisation licenses the original fonts, you install them on the server and previews are rendered again with them.

What an internal run looks like

In September 2026 we ran the comparison on 33 documents covering every must-render Word, Excel and PowerPoint file in the corpus. This was an internal engineering run: our native preview renderer was compared with our own PDF rendering path as the reference, on our lab machine.

• Pagination: all 14 Word documents and all 11 decks matched their reference page and slide counts.
• Missing content: no page was flagged.
• Visual review: every document was accepted. The lowest similarity score came from a nested-table document that was shifted uniformly by a few pixels, which the bar accepts.
• One real defect: an earlier run showed Chinese and Japanese text as empty boxes. We added the missing fallback font, and the text now renders on the same lines and pages. Arabic and Hebrew were correct throughout.

That last point is why we review pages by eye instead of trusting a score: the defect was obvious to a person and easy to miss for a metric.

Comparing with Office Online Server

An internal reference can only show where two of our own rendering paths disagree. A defect that both share stays invisible. The comparison that matters to you is against Office Online Server, so we run the same corpus and the same tool with Office Online Server as the reference, on reference hardware, with the fonts and settings recorded for each run.

We publish those results here once they are reproducible: the same corpus, the same version, the same numbers on a second run. Until then you will not find a fidelity percentage on our website.

Measuring speed

Speed is measured in two places, because users feel both:

• On the server: time waiting for a renderer, then load, layout and export of the document, and storage in the preview cache.
• In the browser: from the moment the page starts loading until the first part of the document is visible on screen.

Every run records the build, the renderer version, the operating system, CPU, memory, installed fonts, cache settings and limits, because a timing without its environment is not a result. Our current timings come from a development machine and are not representative of a production server, so we do not publish them. Production figures follow on reference hardware, together with the sizing they apply to.

Test it on your own documents

The most convincing corpus is your own. During a pilot you run Office Render Engine next to Office Online Server on a test farm or an extra zone, open the documents your users actually work with, and compare them side by side. We help you set up the comparison and treat every missing element or broken page you find as a defect.

How documents are processed and protected during that test is described on our security page. If you are still deciding between your options after the end of Office Online Server support, start with our end-of-life guide.