Guard tests with pytest in Flask projects: locking rules into code

Even a small website gradually collects rules that nobody ever writes down: a certain name must never appear on the site, every page needs its own title, the cookie consent default has to run before any Google tag, and a contact form that arrives with its hidden field filled in must not be stored. These rules are usually agreed in a meeting or a chat thread, and a few weeks later, with no bad intent from anyone, a template edit quietly breaks one of them.

This article describes the method we use to lock such decisions into code: guard tests. A guard test is less about proving that a feature works and more about checking, over and over, that a decision that was already made has not been undone. All examples are simplified from this site's own test folder. The site is written in Flask, the tests run under pytest, and the repository holds roughly 200 tests today.

The foundation: an app factory and a temporary database

Guard tests only stay fast and trustworthy if every test starts from a clean application. The simplest way to get there is Flask's application factory pattern: instead of building the app once at module level, you write a create_app function that accepts a settings dictionary. Each test passes its own settings, and most importantly points the database path at the temporary directory pytest creates for every test (tmp_path).

import pytest
from app import create_app

@pytest.fixture
def client(tmp_path):
    app = create_app({"TESTING": True, "DATABASE": str(tmp_path / "t.db")})
    return app.test_client()

def test_home_page(client):
    r = client.get("/")
    assert r.status_code == 200

This setup pays off in three ways. First, tests never touch real form submissions; the SQLite file is created from scratch for each test and removed afterwards. Second, tests do not depend on data left behind by other tests, so running them in a different order gives the same result. Third, the TESTING flag tells the application not to start background jobs (in our case, the daily backup scheduler), so the suite finishes quickly and leaves no stray threads running.

Turning a decision into a test

The core of a guard test is a habit: on the day a rule is agreed, a test that catches a violation of that rule is written as well. If you decide, for example, that the name of a client or of an old brand must never appear on the site, you add a test that walks through every important page and searches for that name. The docstring records the date of the decision and a one-line reason, so whoever reads the test a year later knows why it exists.

FORBIDDEN_NAMES = ["old brand", "client name"]

def test_forbidden_names_never_appear(client):
    """Decision (date): these names never appear on any visitor-facing page."""
    paths = ["/", "/iletisim", "/gizlilik", "/hakkinda"]
    for path in paths:
        html = client.get(path).get_data(as_text=True).lower()
        for name in FORBIDDEN_NAMES:
            assert name not in html, (path, name)

A small but important detail is the tuple attached to the assert. When the test fails, pytest does not just report "False"; it prints which path contained which name. Guard tests usually break weeks later, during an unrelated change, so an error message that explains itself is worth a lot.

The same idea protects other decisions on this site: a full street address is never printed, founder names appear only in the legal notice, and the internal name of the repository folder never reaches visitors. Each of them is a test of a few lines, and each one reflects a decision that was actually taken at some point.

Tests that walk the sitemap

Hard-coding a list of pages inside a test works for a while, but when a new page is added someone forgets to extend the list and the guard never sees it. A sturdier approach is to let the test read the addresses from the site's own sitemap. Every new page that enters the sitemap then joins the checks automatically.

Title uniqueness is the best example. Search engines can treat pages that share the same <title> as possible duplicates. On a multilingual site this happens easily: one translation is missing and the page falls back to the default language's title.

import re

def test_titles_are_unique(client):
    sm = client.get("/sitemap.xml").get_data(as_text=True)
    paths = sorted({re.sub(r"https?://[^/]+", "", u)
                    for u in re.findall(r"<loc>([^<]+)</loc>", sm)})
    titles = {}
    for path in paths:
        html = client.get(path).get_data(as_text=True)
        title = re.search(r"<title>(.*?)</title>", html, re.S).group(1)
        assert title not in titles, (path, titles.get(title))
        titles[title] = path

Sitemap entries carry the live domain, while the test client only works with paths, so the regular expression strips the host. The same loop can be extended to check that every page returns 200, that the canonical link is correct, or that a forbidden word is absent. We cover the multilingual structure itself in our article on hreflang and the sitemap.

Complete translation keys

If you keep interface strings in one dictionary per language, the most common mistake is adding a new string only to the Turkish dictionary. The application keeps working by falling back to the default language for the missing key, which means the bug shows up as a Turkish sentence in the middle of a German page, and nobody may notice. The test that catches this is tiny:

import ceviri, ceviri_en, ceviri_de, ceviri_es

def test_translation_keys_complete():
    for module in (ceviri_en, ceviri_de, ceviri_es):
        missing = set(ceviri.UI_TR) - set(module.UI)
        assert not missing, (module.__name__, missing)

The set difference yields the full list of missing keys, so the failure message is the to-do list. This test does not judge the quality of a translation, only its presence. Someone who knows the language still has to read it. But closing the gap at least stops a missing string from going live unnoticed.

Imitating production: monkeypatch

Some behaviour is only switched on in production. On this site, Google tags and the ad loader are printed only when FLASK_ENV=production, so local visits during development never pollute measurement. That is a sensible default, but the tests still need to exercise this branch. pytest's monkeypatch fixture sets an environment variable for the duration of one test and restores the previous state when the test ends.

def make_client(monkeypatch, tmp_path, **env):
    for name in ("FLASK_ENV", "GA4_ID", "GTM_ID", "ADSENSE_CLIENT"):
        monkeypatch.delenv(name, raising=False)
    for name, value in env.items():
        monkeypatch.setenv(name, value)
    app = create_app({"TESTING": True, "DATABASE": str(tmp_path / "t.db")})
    return app.test_client()

def test_no_tag_in_development(monkeypatch, tmp_path):
    html = make_client(monkeypatch, tmp_path, GA4_ID="G-TEST123").get("/").get_data(as_text=True)
    assert "google-etiket.js" not in html

Deleting the relevant variables first matters: a variable that happens to be set on a developer's machine can silently change the outcome. The pattern also assumes the application reads its environment inside create_app, at call time. A value read once when the module is imported will not react to changes made inside the test.

Testing order: scripts inside the head

Some rules are about order rather than presence. With Consent Mode v2, the consent default has to run before any Google tag. We test this by comparing positions in the raw HTML: the default comes before the tag file, the tag file before the ad loader, and all of them before the <title> element.

def test_consent_default_first(monkeypatch, tmp_path):
    c = make_client(monkeypatch, tmp_path, FLASK_ENV="production", GA4_ID="G-TEST123")
    for path in ("/", "/en/", "/iletisim"):
        html = c.get(path).get_data(as_text=True)
        i = html.find("gtag('consent','default'")
        assert 0 < i < html.find("google-etiket.js") < html.find("<title>"), path

Python's chained comparisons keep this kind of test readable. Because find returns -1 when nothing is found, the leading 0 < i also guarantees that the searched text really exists.

Security guards: open redirects and the honeypot

Forms carry small security decisions, and those deserve tests too. Our newsletter form accepts a geri ("back") field so it can send the visitor back to the page they came from. Without validation, an attacker could craft links that carry the site's name but redirect to another domain. The rule is simple: only relative paths that start with a single slash are accepted. The test checks it by submitting an external address.

def test_no_open_redirect(client):
    r = client.post("/bulten", data={"eposta": "x@y.co", "geri": "https://evil.example/"})
    assert r.status_code == 302
    assert r.headers["Location"].endswith("/guncellemeler")

def test_honeypot(client):
    r = client.post("/iletisim", data={"ad": "Bot", "eposta": "b@x.com",
                                       "mesaj": "automated message", "web": "filled"})
    assert r.status_code == 302

A honeypot is a hidden field that people never see but bots tend to fill in. When it arrives filled, the form pretends to succeed and stores nothing. The test above only checks the status code; a stricter version would also confirm that no row was written to the database. Addresses that start with two slashes, such as //evil.example, and backslash forms such as /\evil.example that browsers treat the same way, also lead to another host in the browser, so validation must reject them and a test should try that case separately.

Practical tips

• Write the test on the day of the decision. A guard postponed is usually a guard never written.
• Put the date and the reason in the docstring. Whoever decides whether to delete the test will need them.
• Do not maintain lists by hand. Read paths from the sitemap, keys from the dictionary and products from a single source.
• Make failure messages rich. Attach the path, key or value to every assert.
• Keep the suite fast. The whole run should finish in a few tens of seconds; slow tests stop being run.

Once running the suite before every commit becomes a habit, the guards remember things for you. Every release on this site has an entry on the updates page, and behind each entry sits the requirement that tests like these pass.

Conclusion

Guard tests are not complicated; most of them are five to ten lines long. Their value comes from moving rules out of people's memory and into code. A clean start with an app factory and a temporary database, address lists read from the sitemap, a production environment imitated with monkeypatch and failure messages that explain themselves: together they let even a small team keep the decisions it has made on a growing site. If you would like a similar setup in your own project, write to us.