Yandex

Yandex is a Russian search engine and webmaster platform. If you target Russian-speaking users, do not treat it as “Google, but in another market.” Validate crawl rules, index-removal methods, canonical hints, localized pages, and reindexing inside Yandex Webmaster instead of assuming Google defaults map one to one.

Direct answer

Yandex matters when Russian-language demand is material to your site. Its own documentation says robots.txt can still leave a page eligible for Yandex Search, recommends meta noindex or X-Robots-Tag for removal, warns that canonical targets can be ignored if they redirect or are blocked, and says it no longer supports using Sitemap files for language versions. Those are operational checks, not trivia.

What changes for SEO teams

1. `robots.txt` is not your only control

Yandex’s robots.txt documentation explicitly says a page restricted in robots.txt can still participate in Yandex Search. If your goal is removal rather than crawl reduction, Yandex recommends the robots meta tag or X-Robots-Tag and says not to block that URL in robots.txt, because the bot must fetch the page to detect the instruction. Google gives the same practical warning in its noindex documentation.

Use robots.txt for crawl control and server-load management. Use noindex, X-Robots-Tag, or a real 404 / 410 when the page should disappear from search.

2. Canonical still needs verification

Yandex’s canonical guidance warns that a canonical target may be ignored if it redirects, is blocked from indexing, points across domains incorrectly, or forms a canonical chain. Treat canonical as a hint that must be checked after deployment, not as a one-line duplicate-content fix.

At minimum, verify:

the live canonical in page source;
the final HTTP status of the canonical target;
whether duplicates are still searchable;
whether internal links and XML sitemaps point to the same preferred URL.

If you need a quick page-level check, run the Canonical Checker after you confirm the intended URL map.

3. Language versions need page markup

Yandex says it no longer supports using Sitemap files for language versions and recommends localized page markup with alternative links instead. Google’s localized-versions documentation still supports hreflang in HTML, HTTP headers, or sitemaps. The overlap is simple: if you want both engines to understand language versions reliably, put reciprocal hreflang links in the page source and keep the alternates crawlable.

<link rel="canonical" href="https://example.com/ru/page/" />
<link rel="alternate" hreflang="ru" href="https://example.com/ru/page/" />
<link rel="alternate" hreflang="en" href="https://example.com/en/page/" />
<link rel="alternate" hreflang="x-default" href="https://example.com/" />

Use x-default when users may land on a language selector or locale-adaptive page.

4. Yandex Webmaster is part of the workflow

Yandex documents important-page monitoring, page checks, searchable pages, and query statistics inside Webmaster. That means a basic Yandex workflow should include:

adding and verifying the site in Yandex Webmaster;
checking robots.txt with the built-in analyzer;
checking whether important URLs are searchable or excluded;
requesting removal or reindexing only after the underlying rule is correct;
comparing query statistics before and after a change.

Practical checklist

Confirm that the Russian-language page returns 200, renders the important copy in HTML, and is not blocked accidentally.
Use robots.txt only for crawl control. For removal, switch to noindex, X-Robots-Tag, or 404 / 410.
Keep one self-canonical URL per language version and avoid canonical chains.
Put reciprocal hreflang links in the source of every localized version.
Validate the result in Yandex Webmaster before you declare the page fixed.

Common mistakes

Treating Yandex as a generic “other search engine” without checking its own documentation.
Using robots.txt when the real goal is removal from search.
Shipping canonical tags that point to redirects or blocked URLs.
Relying on alternate-language sitemap markup alone.
Comparing Google and Yandex only by ranking, without checking crawler and index state first.