Ecommerce Faceted Navigation and SEO - Which Filters to Index and Which to Block (noindex, canonical, robots.txt)

Faceted navigation in an online store raises two questions at once: which filter combinations should be pages in Google, and which ones the crawler should not visit at all. The decision rule: a filter that has demand (it appears in Google's autocomplete or has at least about 50 searches a month) and at least about 10 products gets a fixed, friendly URL with an HTML link, its own title and a self-referencing canonical. Everything else (further combinations, size, price, sorting, products per page) should not be HTML links, and if it creates URLs, you block them in robots.txt. Use noindex and canonical to clean up URLs that are already indexed. Never combine them with a robots.txt block, because Google will then see neither.

Which ecommerce filters to index and which to block: a rule with thresholds, noindex vs canonical vs robots.txt vs nofollow in one table, a check of filters in 11 Polish stores, Shopify, WooCommerce, PrestaShop and IdoSell defaults, URL counts at two scales, robots.txt rules, a script and two pitfalls.

This post covers what each method does to the crawler (robots.txt, noindex, canonical, nofollow, # URLs, filters without links), the results of my check of filters in 11 Polish online stores, how Shopify, WooCommerce, PrestaShop and IdoSell handle filters by default, how many URLs filters create at two store sizes, a script for checking how filters are handled and two pitfalls I found in large stores.

Can filters hurt SEO at all - the first decision

They can, but you do not always need to act. Google says plainly that faceted navigation is "by far the most common source of overcrawl issues" site owners report. Every filter combination is a new URL, so with a few hundred filter values the number of possible URLs runs into the tens of thousands.

Robots.txt, noindex, canonical, nofollow - what affects crawling and what affects indexing

This is the most important table in the post. Most filter mistakes come from confusing two things: blocking crawling (the crawler does not fetch the URL) and blocking indexing (the URL does not appear in results).

Three Google rules worth knowing: robots.txt is not for canonicalisation (a blocked URL can stay indexed), noindex is not for choosing the canonical version within a site, and canonical is a strong signal but takes time. If you do index filters, Google recommends the standard & separator in parameters, a fixed filter order in the path and a 404 status for combinations with no products.

Which filters to index - the rule and examples

A filter becomes a landing page when it meets three conditions at once: it has demand, it has at least about 10 products and it answers a different intent than the parent category. I showed how to collect demand and map queries to filters in the post on ecommerce keyword research.

How 11 Polish online stores handle filters - my check

How I checked: on 15 October 2026 a script in a Chromium browser opened one category in each of 17 Polish online stores, counted filter controls and a href links to filters, then clicked filters (one, then a second) to see which URL appeared. For every filter URL it checked the HTTP status, meta robots, canonical and whether robots.txt blocks it for Googlebot. I could establish filter handling in 11 stores. In the other 6, filters did not change the URL after a click or the script could not click them, so I do not assess them.

What this shows:

• The most common method is a canonical to the category (5 of 11 stores), and 4 stores give filters noindex. 4 stores index filters as pages with their own path URL, one of them even with just a few products.
• 6 of 11 stores do not link filters with a href, so their URLs only appear after a click. That effectively limits crawling without any rules.
• 2 stores combine noindex or canonical with a robots.txt block. That is the pitfall described below.
• The number of filter values is striking. On the tablets category page of morele.net I counted 459 filter controls, and 282 in the trousers category of smyk.com. With 459 values, combinations of just two filters give up to about 105,000 URLs - which is why indexing everything is out of the question.

Filters on Shopify, WooCommerce, PrestaShop and IdoSell - default settings

The platform decides your starting point. I checked the default rules in store robots.txt files and in the PrestaShop source code.

The most common platform problem: the default block works, so nobody creates pages for filters with demand. In PrestaShop and in the IdoSell store from my check, all filters are blocked, so a query like "salomon hiking boots" has no chance of landing on a filter page - you need a separate category or landing page for it. I build such pages for filters with demand in store templates as part of my ecommerce SEO work.

How many URLs filters create - two scales

An estimate assuming filters create URLs and are links. The number of two-filter combinations is n × (n - 1) / 2, where n is the number of filter values in a category.

The takeaway: in a small store the problem often does not appear on its own, but it is worth blocking sorting from the start and adding a few dozen filter pages with demand. In a large store without rules the crawler can face hundreds of thousands of filter URLs for a few thousand products, and new products wait longer to be indexed - I explain how Google allocates resources in the post about crawl budget.

An example robots.txt for filters

Rules for a typical store with filters in parameters. Replace the parameter names with yours and test them in the robots.txt report in Search Console before deploying.

User-agent: *# sorting and products per pageDisallow: /*?*sort=Disallow: /*?*orderby=Disallow: /*?*limit=# filters in parameters (indexable filters have their own path URLs)Disallow: /*?*filter_Disallow: /*?*price_from=Disallow: /*?*price_to=# internal search resultsDisallow: /searchSitemap: https://yourstore.com/sitemap.xml

Add these rules only after filter URLs with parameters have dropped out of the index (see the pitfall below). Filter pages with demand have path URLs, e.g. /hiking-boots/brand-salomon, so the rules do not block them.

Script: check how a store handles filter URLs

The script takes a list of filter URLs (e.g. from a crawl, logs or the Search Console Pages report) and checks, for each, the HTTP status, meta robots, the X-Robots-Tag header, canonical and the robots.txt block. At the end it lists URLs with conflicting signals.

# pip install requestsimport csv, re, sysfrom urllib.parse import urlparse, urljoinfrom urllib import robotparserimport requestsUA = "Mozilla/5.0 (compatible; filter-audit)"urls = [u.strip() for u in open(sys.argv[1]) if u.strip()]host = urlparse(urls[0]).scheme + "://" + urlparse(urls[0]).netlocrp = robotparser.RobotFileParser(host + "/robots.txt"); rp.read()def check(url):    r = requests.get(url, headers={"User-Agent": UA}, timeout=20, allow_redirects=False)    html = r.text if "html" in r.headers.get("content-type", "") else ""    meta = re.search(r'<meta[^>]+name=["\']robots["\'][^>]*content=["\']([^"\']+)', html, re.I)    canon = re.search(r'<link[^>]+rel=["\']canonical["\'][^>]*href=["\']([^"\']+)', html, re.I)    robots = ((meta.group(1) if meta else "") + " " + r.headers.get("x-robots-tag", "")).lower()    canonical = urljoin(url, canon.group(1)) if canon else ""    return {"url": url, "status": r.status_code, "robots_txt_blocked": not rp.can_fetch("Googlebot", url),            "noindex": "noindex" in robots, "canonical": canonical,            "canonical_self": canonical.rstrip("/") == url.rstrip("/")}rows = [check(u) for u in urls]with open("filters.csv", "w", newline="") as f:    w = csv.DictWriter(f, fieldnames=rows[0].keys()); w.writeheader(); w.writerows(rows)for r in rows:    if r["robots_txt_blocked"] and (r["noindex"] or (r["canonical"] and not r["canonical_self"])):        print("CONFLICT: robots.txt block hides noindex/canonical:", r["url"])    elif r["noindex"] and r["canonical"] and not r["canonical_self"]:        print("CONFLICT: noindex plus a canonical to another page:", r["url"])    elif r["status"] == 200 and not r["noindex"] and r["canonical_self"]:        print("INDEXABLE filter - check demand and product count:", r["url"])

Note: the robotparser in Python's standard library does not fully handle * and $ in robots.txt rules the way Google does. With wildcard rules, check the result in the Search Console robots.txt report or use a Google-compatible parser.

Pitfall 1: a robots.txt block hides noindex and canonical

The most common mistake I found: the store gives filters noindex or a canonical to the category and blocks them in robots.txt at the same time. That was the case at taniaksiazka.pl (noindex, nofollow, canonical and the rule Disallow: /*?params*=*) and kontri.pl (canonical and the rule Disallow: *?filter_*). Google does not fetch a blocked URL, so it sees neither noindex nor canonical. URLs indexed earlier can stay in the index as "Indexed, though blocked by robots.txt", with no content and no signals passed to the category.

The correct order for a clean-up:

1. 1.Keep filter URLs crawlable and give them noindex (filters with no value) or a canonical to the category (filters almost identical to the category). Do not use both at once - Google does not recommend noindex for choosing the canonical version.
2. 2.Remove a href links to filters without demand from product listings.
3. 3.Wait until the number of these URLs in the Search Console Pages report drops clearly.
4. 4.Only then add a robots.txt block, if the filters still use up crawling.

Pitfall 2: indexed filters with a few products

The other extreme: the store indexes every brand filter because it has a mechanism for it. At modivo.pl, brand filter pages in the leggings category had index and a self canonical even with 2, 4 or 6 products. There are thousands of such pages, each with an almost empty listing, and after a few models sell out they become empty. The fix: a product-count threshold in the template. Below the threshold the filter gets noindex and disappears from the links, above it returns to the index. An empty filter returns 404, as eobuwie.pl does and as Google recommends.

What not to do

• Do not block in robots.txt the URLs you have just given noindex or a canonical.
• Do not give one page both noindex and a canonical to the category.
• Do not index every filter combination - only those with demand and at least about 10 products.
• Do not link every filter value with a href in the product listing.
• Do not leave sorting and products per page indexable.
• Do not redirect empty filter combinations to an error page - return 404 at the same URL.
• Do not rely on nofollow - it only works when every link to the URL has it, including external ones.

FAQ

I describe the order of all store work in the ecommerce SEO strategy. If you want me to review the filters in your store and implement the rules in its templates, see ecommerce SEO.