Quality assurance for AI-assisted content: gates, independent checks and quarantine

Large language models can turn a topic into lesson text and exam questions in a few minutes. For a team that prepares learning material that speed is very tempting, but not every generated text is fit to publish. A model can mark a wrong answer as correct in perfectly fluent prose, write two of four options so that they mean the same thing, or, without anyone noticing, make the right option the longest and most detailed one every time, which quietly gives the answer away. Because the writing reads so smoothly, readers often miss these errors.

This article explains how we check model output before it is published in the content pipeline we built for the ScholaMundus learning platform. The pipeline turns a topic into an eight-lesson course and a ten-question exam in which every question comes with a rationale. Our approach rests on one simple principle: model output is a draft, and it only goes live after passing a series of automatic gates and after a person has reviewed anything that looks doubtful. The example code is written in Python and uses only the standard library.

Principle: model output is a draft

The first step of quality assurance is a mindset rather than a technique. Treating model output as a draft that must be checked, instead of finished content, changes the design of the whole pipeline. You deliberately leave a gap between the generation step and the publishing step, and in that gap the content passes through measurable rules. Because the rules are written down and automated, they are applied the same way to every draft; fatigue, haste or a "let it through this once" decision never comes into play.

The second important decision is to ask the model for structured data rather than free text. When the lesson title, lesson text, question, options, index of the correct option and rationale arrive as separate fields, you can check each field on its own. The examples below assume the model is asked for JSON and that the incoming data is first converted into a data class. If that conversion fails, the draft is rejected at the very first step.

from __future__ import annotations

import ast
import json
import operator
import shutil
from collections import Counter
from dataclasses import dataclass
from fractions import Fraction
from pathlib import Path


@dataclass
class Question:
    qid: str
    text: str
    options: list[str]
    correct: int               # zero-based option index
    rationale: str
    expr: str | None = None    # for numeric questions, e.g. "3/4 + 1/6"

Structural validation: form comes first

The first gate ignores meaning and looks only at form. Is the question text a reasonable length? Are there four or five options, as expected? Has the same option been written twice? Does the index of the correct option actually point at an option that exists? Is the rationale a throwaway sentence, or long enough to explain to a learner why the answer is right? Each of these is a check of a few lines, yet together they catch a whole class of errors.

The same logic applies to lesson texts: a very short lesson usually means the model skimmed the topic, so we flag short lessons with a separate rule. Thresholds have to be tuned to the type of content; the numbers below are only examples.

Next to the structure gate we add a "longest option" gate. When writing questions, people and models alike tend to phrase the correct answer more carefully and in more detail. The result is a correct option that is clearly longer than the others, which lets someone who does not know the topic guess it. The gate flags a draft when the correct option is both the longest and longer than the second-longest option by a given ratio.

def structure_gate(s: Question) -> list[str]:
    problems = []
    if not 20 <= len(s.text) <= 600:
        problems.append("question text length out of range")
    if len(s.options) not in (4, 5):
        problems.append("there must be 4 or 5 options")
    if len({o.strip().lower() for o in s.options}) != len(s.options):
        problems.append("duplicate option")
    if not 0 <= s.correct < len(s.options):
        problems.append("invalid correct index")
    if len(s.rationale) < 40:
        problems.append("rationale too short")
    return problems


def longest_option_gate(s: Question, ratio: float = 1.3) -> list[str]:
    lengths = sorted((len(o) for o in s.options), reverse=True)
    correct_len = len(s.options[s.correct])
    if correct_len == lengths[0] and correct_len >= ratio * lengths[1]:
        return ["correct option is clearly the longest"]
    return []

Option balance: where the correct answers sit

Some problems are invisible in a single question and only show up across an entire exam. Models can tend to put the correct answer under the same letter; if half the correct answers in a ten-question exam are under one letter, an attentive learner will notice the pattern. That is why we add a distribution gate that looks at the whole exam: if any letter holds more than a set share of the correct answers, the exam is flagged.

The fix is usually not regeneration. If the questions themselves are correct, shuffling the options and updating the correct index accordingly is enough. The distribution gate still has to run again after shuffling, because random shuffles can also produce unbalanced results in small exams.

Independent solution: checking numeric answers a second way

The most dangerous error is a wrong answer backed by a fluent rationale. In verbal questions this is hard to catch automatically; in numeric questions it is possible. The method is this: alongside the question, we ask the model for the mathematical expression that yields the answer, in a separate field. We then evaluate that expression with deterministic code that has nothing to do with the model and compare the result with the marked option.

Two details matter here. First, running the expression with eval is not safe; whatever the model returns must never be executed as code. So we parse the expression with Python's ast module and use a small evaluator that only allows numbers and the four basic operations. Second, floating point numbers produce small rounding differences in operations such as 0.1 + 0.2. Computing fractional answers as exact ratios with fractions.Fraction makes the comparison precise. In fields that work with decimal notation, decimal.Decimal gives a similar guarantee.

_OPS = {ast.Add: operator.add, ast.Sub: operator.sub,
        ast.Mult: operator.mul, ast.Div: operator.truediv}


def _evaluate(node: ast.AST) -> Fraction:
    if isinstance(node, ast.Expression):
        return _evaluate(node.body)
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return Fraction(str(node.value))
    if isinstance(node, ast.BinOp) and type(node.op) in _OPS:
        return _OPS[type(node.op)](_evaluate(node.left), _evaluate(node.right))
    if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.USub):
        return -_evaluate(node.operand)
    raise ValueError("unsupported expression")


def independent_solution_gate(s: Question) -> list[str]:
    if s.expr is None:
        return []
    try:
        expected = _evaluate(ast.parse(s.expr, mode="eval"))
        given = Fraction(s.options[s.correct].replace(",", ".").strip())
    except (ValueError, ZeroDivisionError, SyntaxError):
        return ["numeric answer could not be verified independently"]
    if expected != given:
        return [f"independent solution {expected}, marked {given}"]
    matches = 0
    for opt in s.options:
        try:
            matches += Fraction(opt.replace(",", ".").strip()) == expected
        except ValueError:
            pass
    return [] if matches == 1 else ["more than one option equals the answer"]

The gate checks not only that the marked option is correct but also that no other option equals the same value. "3/4" and "0.75" are the same number; if two options are written that way, the question has two correct answers. That is an error that slips past a visual review very easily.

Chaining the gates

Keeping the gates in a list turns adding a new rule into a one-line change. The check function runs the gates in order and collects the problems it finds. If the structure gate fails there is no point running the others; with an invalid option index, the length comparison would itself fail. So the chain stops early when it finds a structural problem.

GATES = [structure_gate, longest_option_gate, independent_solution_gate]


def check(s: Question) -> list[str]:
    problems = []
    for gate in GATES:
        problems += gate(s)
        if gate is structure_gate and problems:
            break                     # other gates are meaningless if the structure is broken
    return problems


def distribution_gate(questions: list[Question], ceiling: float = 0.4) -> list[str]:
    counter = Counter("ABCDE"[s.correct] for s in questions)
    return [f"{count} of {len(questions)} correct answers are option {letter}"
            for letter, count in counter.items() if count / len(questions) > ceiling]

Quarantine and human review

A draft that fails the gates is not deleted; it goes into quarantine. Deleting it would make it impossible to understand why the same error keeps coming back. The quarantine folder keeps the rejected draft and the reasons for rejection side by side. A person reviews the folder regularly and makes one of three decisions for each draft: fix it and publish, regenerate it, or drop it completely.

def quarantine(path: Path, problems: list[str], folder: Path) -> Path:
    folder.mkdir(parents=True, exist_ok=True)
    target = folder / path.name
    shutil.move(str(path), target)
    target.with_suffix(".problems.json").write_text(
        json.dumps(problems, ensure_ascii=False, indent=2), encoding="utf-8")
    return target

Quarantine has a side benefit: it shows how often each gate rejects drafts. If a gate keeps catching the same kind of error, the problem is most likely in the generation instructions, and that is the place to fix it. The gates thus become not only a filter but also a feedback channel for the generation process.

The small example below shows how two versions of the same question, one marked correctly and one marked wrongly, pass through the gates. In the second version the rationale describes the right result, but the marked option is different; the independent solution gate catches that contradiction.

s = Question("mat-001", "What is the sum of 3/4 and 1/6?",
         ["11/12", "4/10", "5/6", "1"], 0,
         "Use the common denominator 12: 9/12 + 2/12 = 11/12.", "3/4 + 1/6")
print(check(s))
s2 = Question("mat-002", "What is the sum of 3/4 and 1/6?",
          ["11/12", "4/10", "5/6", "1"], 2,
          "Use the common denominator 12: 9/12 + 2/12 = 11/12.", "3/4 + 1/6")
print(check(s2))
print(distribution_gate([s, s, s2]))
import tempfile
d = Path(tempfile.mkdtemp()); f = d / "mat-002.json"; f.write_text("{}", encoding="utf-8")
print(quarantine(f, check(s2), d / "quarantine"))
print(Fraction("0.75") == Fraction(3, 4))

Source and copyright discipline

The easiest shortcut in exam preparation content is to reuse questions from past exams. The copyright of those questions, however, belongs to the institution that runs the exam. So our pipeline never pulls in past exam questions; the model writes original questions starting from the topic and the learning objective. The same principle applies to lesson texts: the model is never asked to reproduce the text of a specific book or website.

Originality cannot be fully verified automatically, but a rough check is possible. Comparing a new question with your existing question bank by text similarity catches questions that are near copies of each other. That helps you notice both repeated content and external text that somehow made its way into the bank.

After publishing: feedback and transparency

However good the gates are, some errors will reach readers. Two things are needed for that. First, a learner should be able to flag a question as "possibly wrong" with a single click, and that flag should land directly in the quarantine queue. Second, whenever possible every reported error should become a new gate or a new threshold on an existing one, so that the same kind of mistake does not reach readers twice.

Finally, transparency: telling readers plainly that the content was produced with AI assistance and reviewed by people is honest and sets a healthy expectation. When readers spot a mistake, they know they are looking at a draft that went through a process, and they do not hesitate to report it.

Conclusion

In AI-assisted content production, quality comes less from the model itself than from the process built around it. Structured output, form gates, option balance, independent verification of numeric answers, quarantine instead of deletion and human review: each one is simple, but together they form a strong safeguard. None of these gates is perfect; the goal is to catch most errors before publication and to make the rest quick to fix. If you would like a similar review setup for your own content pipeline, write to us.