Understanding quality scores

Enriched products come back with a quality score per factor and a list of the specific problems behind it. The score tells you where to look; the issues tell you what to change.

The three factors

Every enriched product is scored on three factors:

  • Accuracy — does the generated content reflect your product data? Unsupported claims are the most serious issue type.
  • Consistency — do the generated fields agree with each other? A description and a Q&A answer stating different materials is a consistency issue.
  • Brand adherence — does the content follow the brand guidelines you supplied? Scored only when you provide brand guidelines.

Brand adherence is scored only when you supply brand guidelines. Without them, output is assessed on accuracy and consistency alone — one more reason to write them properly. See Writing brand guidelines.

Scoring is issue-based

The score is not a judgment handed down as a number. An automated reviewer checks each generated field and records issues — each naming the field, describing the problem, and carrying a severity. The score is derived from what it found.

Diagram: accuracy, consistency, and brand adherence are each checked per field. Problems are recorded as issues naming the field, the problem, and its severity, with no issues recorded when nothing is found. This produces a score per factor and an issue list showing what to fix.
Each field is checked against the three factors. Problems become issues, and the score follows from them.

Typical issues:

  • A claim the product record does not support — an accuracy issue, and the most serious kind.
  • A description saying “set of 4” while a Q&A answer says “set of 6” — a consistency issue.
  • Copy using a word your guidelines rule out — a brand adherence issue.

Because every deduction traces to a named issue in a named field, a low score is actionable rather than mysterious.

Score panel for a product showing an overall score of 9.3 out of 10. Accuracy scores 8.5 against a threshold of 7 and is marked passed, with a low severity issue explaining that the content frames the product for audiences the source does not explicitly support, and listing the affected fields. Consistency and brand adherence both score 10 out of 10 and pass.
A product's scores, with the issue behind the accuracy result and the fields it affects.

Setting thresholds

You set a minimum score per factor before the enrichment runs, each with a suggested starting value.

Products scoring below a threshold are held for your review, not discarded and not regenerated. The threshold decides what needs your attention, not what gets written.

Setting them per factor is useful because the factors carry different risk. A store with regulatory exposure can hold accuracy high while staying relaxed about tone. A store with a distinctive voice can do the reverse.

Score panel showing an overall score of 10 out of 10, with accuracy, consistency, and brand adherence each scoring 10 against thresholds of 6, 9, and 7 respectively, all marked passed.
Each factor is scored against its own threshold.

Start strict on your first enrichment. You will learn quickly what a given score means for your catalog, and you can loosen from there.

A good score is not a guarantee

A high score means the reviewer found no issues it was looking for. It does not mean the copy is right for your brand, your customers, or that particular product. Scores narrow where you look — they do not replace looking. See Reviewing at scale.

The reverse also holds: a product can score well and still read poorly to you, which is usually a signal that your brand guidelines need to be more specific rather than that the score is wrong.

Using scores to decide what to change

Which factor is failing points at which input to fix.

Factor scoring lowUsually means
AccuracyThe product record is thin, so the copy reached beyond it
ConsistencyThe record contains contradictions, or the batch mixes product types
Brand adherenceYour brand guidelines are too general to apply

See Deciding what to change.