A multilingual Flask site: URL prefixes, hreflang and the sitemap

Publishing a site in several languages looks like a translation job at first: move the strings into a dictionary, call the right key from the template and you are done. In practice the hard part is not the text but the addresses. Every language needs its own permanent URL, a first-time visitor should land in their own language but still be able to switch, search engines need to understand which page is a translation of which, and a page that has no translation must not look like a copy of another language.

This article walks through the setup we use in the Flask code of the very site you are reading. The site is published in Turkish, English, German and Spanish; Turkish sits at the root (/) and the other languages live under a prefix (/en/, /de/, /es/). The code samples are simplified, but each of them runs on its own.

Choosing the URL scheme up front

A multilingual site has three common options: a separate domain per language, a subdomain per language, or a path prefix. We chose the path prefix because one application, one certificate and one Search Console property is the lowest-maintenance route for a small team. Leaving one language unprefixed as the default was also deliberate: Turkish is the site's first language and we did not want any existing link to change.

One more detail: we did not translate the path segments. The English version of /hakkinda lives at /en/hakkinda. Translated paths such as /en/about can read better, but they require a per-language mapping table for every route and a lookup in that table whenever a link is generated. Keeping the same path lets us compute the counterpart of any page in any language in one line, and that convenience is the foundation of the hreflang and sitemap code below.

One view, two URL rules

We register every page view twice: once without a prefix (with Turkish as the default) and once with a /<dil> prefix (dil is Turkish for "language"). A small decorator saves us from writing this by hand for every route. The endpoint name of the prefixed rule ends in _dil, so a template can choose explicitly which rule it wants to build.

from flask import Flask, abort, g

DILLER = ("tr", "en", "de", "es")
VARSAYILAN = "tr"
app = Flask(__name__)


def rota(yol, **opts):
    """Register the same view unprefixed (Turkish) and with a /<dil> prefix."""
    def sarmal(f):
        app.add_url_rule(yol, view_func=f, defaults={"dil": VARSAYILAN}, **opts)
        onekli = "/<dil>" + ("" if yol == "/" else yol)
        app.add_url_rule(onekli + ("/" if yol == "/" else ""), view_func=f,
                         endpoint=f.__name__ + "_dil", **opts)
        return f
    return sarmal


@app.url_value_preprocessor
def dil_al(endpoint, values):
    if values and "dil" in values:
        dil = values.pop("dil")
        if dil not in DILLER:
            abort(404)
        g.dil = dil


@app.url_defaults
def dil_ver(endpoint, values):
    if endpoint and endpoint.endswith("_dil"):
        values.setdefault("dil", getattr(g, "dil", VARSAYILAN))


@rota("/hakkinda")
def hakkinda():
    return f"Page language: {g.dil}"

The url_value_preprocessor pulls the language out of the URL values and stores it in g.dil, so none of the view functions has to accept a language argument. An unsupported prefix such as /xx/hakkinda returns a 404 straight away; otherwise an unlimited number of fake language addresses could be generated. The url_defaults hook works in the opposite direction: when a template calls url_for('hakkinda_dil'), the current language is filled in automatically.

With this setup /tr/hakkinda technically opens too, because tr is a valid language code. We resolve that with the canonical tag: the canonical address of the page is always printed as the unprefixed Turkish version. If you prefer, you can also send the prefixed default language to the unprefixed address with a 301.

Picking the visitor's language: browser, cookie and the root redirect

When a visitor first arrives at the root address, we look at the browser's Accept-Language header. The header lists several languages with weights (the q value), so looking only at the first entry can give the wrong answer. In en-GB;q=0.7,en;q=0.8,de;q=0.9, for example, German carries the highest weight.

def tarayici_dili(accept_language):
    """Highest-weighted supported language from Accept-Language, else the default."""
    if not accept_language:
        return VARSAYILAN
    adaylar = []
    for parca in accept_language.split(","):
        p = parca.strip().split(";")
        kod = p[0].strip().lower()[:2]
        q = 1.0
        for ek in p[1:]:
            ek = ek.strip()
            if ek.startswith("q="):
                try:
                    q = float(ek[2:])
                except ValueError:
                    q = 0.0
        if kod in DILLER:
            adaylar.append((q, kod))
    if not adaylar:
        return VARSAYILAN
    adaylar.sort(key=lambda x: -x[0])
    return adaylar[0][1]


assert tarayici_dili("en-GB;q=0.7,en;q=0.8,de;q=0.9") == "de"
assert tarayici_dili("en-GB,en;q=0.8,de;q=0.9") == "en"

Because Python's sort is stable, the first language in the header wins when weights are equal. The browser preference is only a first impression: if the visitor picks another language from the language switcher, we store it in a dil cookie, and from then on the cookie takes precedence over the browser header.

We redirect only at the root and only with a 302. When a visitor opens an inner address such as /en/hakkinda, they see that page whatever their language is; someone sharing a link should never have the recipient shown a different page. The root redirect is not permanent because its result depends on the visitor, which is also why the response must carry Vary: Accept-Language, Cookie, so that shared caches do not serve one visitor's redirect to another.

from flask import redirect, request, url_for


@app.before_request
def kok_yonlendir():
    if request.path != "/" or request.method != "GET" or "dil" in request.args:
        return None
    tercih = request.cookies.get("dil")
    if tercih not in DILLER:
        tercih = tarayici_dili(request.headers.get("Accept-Language"))
    if tercih != VARSAYILAN:
        yanit = redirect(url_for("ana_dil", dil=tercih), code=302)
        yanit.headers["Vary"] = "Accept-Language, Cookie"
        return yanit
    return None


@app.after_request
def dil_cerezi(yanit):
    secim = request.args.get("dil")
    if secim not in DILLER:
        uc = request.endpoint or ""
        secim = g.get("dil") if uc.endswith("_dil") else None
    if secim and request.cookies.get("dil") != secim:
        yanit.set_cookie("dil", secim, max_age=60 * 60 * 24 * 365, samesite="Lax")
    return yanit


@rota("/")
def ana():
    return f"Home page: {g.dil}"

A small but real problem shows up here: a visitor with an English cookie clicks "Türkçe" in the switcher, lands on the root, and because the cookie still says English, gets redirected straight back to /en/. We solve this by adding ?dil=tr to Turkish links in the switcher. With the query parameter present the root redirect does not run, and the after_request hook flips the cookie to Turkish. We never print that parameter in canonical or hreflang addresses, of course.

Search engine crawlers usually send no Accept-Language header and keep no cookies, so they see the default language at the root. For them to reach the other languages we do not rely on the redirect but on the hreflang tags on every page and on plain links in the language switcher.

hreflang, x-default and the canonical tag

hreflang tags tell search engines at which addresses a page exists in which language. Google expects every language version to list all counterparts, itself included, and expects the lists to be reciprocal: if the Turkish page points to the English one, the English page must point back to the Turkish one. x-default names the address to use when the visitor's language matches none of them; for us that is the unprefixed Turkish version. The canonical tag, in turn, points to each language version's own address. Giving one language version a canonical in another language effectively removes it from search results.

<link rel="canonical" href="{{ SITE_URL }}{{ kanonik_yol }}">
{% for d in hreflang_baglantilari %}
<link rel="alternate" hreflang="{{ d.hreflang }}" href="{{ SITE_URL }}{{ d.url }}">
{% endfor %}
<link rel="alternate" hreflang="x-default" href="{{ SITE_URL }}{{ varsayilan_url }}">

There are two deliberate exceptions to this rule. The first is the privacy notice. We keep the legal text in Turkish only, because a translation of a legal text is only trustworthy once it has been reviewed legally. The privacy addresses in the other languages show the same Turkish text, even though the menu and footer are in that language. On those pages we point the canonical at the Turkish address and print no hreflang at all; presenting the same text as four translations would be misleading. The sitemap lists that page once.

The second is the articles section you are reading now. Articles are published in Turkish and English only. When a German or Spanish article address is opened we redirect it with a 301 to the English version, and the hreflang list contains only the two translations that exist. On these pages the language switcher sends German and Spanish to that language's home page. The rule is simple: hreflang is only given to a translation that actually exists. A tag pointing at a translation that does not exist draws search engines a false map.

Language alternates in the sitemap

hreflang information can be provided in the sitemap instead of the HTML. Using both is not required, but it does no harm as long as they agree; we generate both from the same source, so the risk of contradiction is low. In the sitemap every language version is a separate <url> entry, and every entry lists all alternates with xhtml:link elements.

from flask import Response


def sitemap_uret(kok, yollar, diller=DILLER):
    satirlar = ['<?xml version="1.0" encoding="UTF-8"?>',
                '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" '
                'xmlns:xhtml="http://www.w3.org/1999/xhtml">']

    def adres(yol, kod):
        if kod == VARSAYILAN:
            return yol
        return "/" + kod + ("/" if yol == "/" else yol)

    for yol in yollar:
        for kod in diller:
            satirlar.append(f"  <url><loc>{kok}{adres(yol, kod)}</loc>")
            for k2 in diller:
                satirlar.append(f'    <xhtml:link rel="alternate" hreflang="{k2}" '
                                f'href="{kok}{adres(yol, k2)}"/>')
            satirlar.append(f'    <xhtml:link rel="alternate" hreflang="x-default" '
                            f'href="{kok}{yol}"/>')
            satirlar.append("  </url>")
    satirlar.append("</urlset>")
    return "\n".join(satirlar) + "\n"


@app.get("/sitemap.xml")
def sitemap():
    xml = sitemap_uret("https://example.com", ["/", "/hakkinda"])
    return Response(xml, mimetype="application/xml")

For the articles section it is enough to call the same function with diller=("tr", "en"). We also recommend adding a lastmod to every entry, but that date must reflect the day the content actually changed. Printing today's date on every request teaches search engines to ignore the field. To tell engines such as Bing and Yandex about new or changed addresses right away, we additionally use the approach described in our IndexNow article.

Catching missing translations with a test

The sneakiest bug on a multilingual site is the silent missing translation. A new key is added to the Turkish template, nobody adds it to the other languages, and the page ships in that language with a Turkish fragment in it. In our setup a missing key falls back to Turkish, so the page does not crash; that is a good safety net, but it also makes the bug invisible. So we wrote a test that checks every language file carries every key from the Turkish dictionary. It also checks that the translated product, module and update lists have the same length as the source lists; those lists are matched by position, so one missing row would shift every translation below it.

import ceviri
import ceviri_de
import ceviri_en
import ceviri_es


def test_ceviri_anahtarlari_tam():
    for modul in (ceviri_en, ceviri_de, ceviri_es):
        eksik = set(ceviri.UI_TR) - set(modul.UI)
        assert not eksik, (modul.__name__, sorted(eksik))


def test_tum_sayfalar_dort_dilde(client):
    for yol in ["/", "/hakkinda", "/iletisim"]:
        for dil in ["en", "de", "es"]:
            adres = f"/{dil}/" if yol == "/" else f"/{dil}{yol}"
            yanit = client.get(adres)
            assert yanit.status_code == 200, adres
            assert f'<html lang="{dil}">' in yanit.get_data(as_text=True)

These tests force a developer who adds a new string to update three more files. It feels like overhead at first, but it is far cheaper than discovering a half-translated page in production and fixing it afterwards. The same tests also guarantee that each page opens in every language and that the lang attribute is printed correctly.

Conclusion

The skeleton of a multilingual Flask site is made of a few small parts: a decorator that registers one view under two URL rules, two hooks that take the language out of the URL and put it back, a redirect that runs only at the root and carries a Vary header, a cookie that remembers the visitor's choice, template code that builds hreflang tags and the sitemap from one source, and tests that catch missing translations. Each part is simple on its own. The real care is needed in the exceptions: never present an untranslated page as if a translation existed, give the canonical to the right language, and tell search engines only about alternates that actually exist. If you are planning a similar setup, you can follow how ours evolved on the updates page or write to us.