62% of Top-Site XML Sitemaps Have Flaws, a Vendor Study Reports

Of the top 1,000 Chrome UX Report origins, 400 (43.7% of the 916 that responded as websites) exposed an XML sitemap, and 248 of those 400 (62%) had at least one measurable flaw, according to a study lintlab published after a crawl on 1 October 2026. lintlab sells a sitemap checker, so this is vendor research that Google has not reviewed, and this article has not been able to verify it independently. A separate check of the same origins on 2 October 2026 found a valid llms.txt file on 66 of the 1,000 (6.6%). The details below are from the lintlab XML sitemap study page and from PPC Land's report of it.

Who ran the study and what the vendor disclosure says

lintlab sells an XML Sitemap Checker on the Apify marketplace, priced at $0.001 per file parsed, $0.0003 per URL extracted and $0.0005 per status check. PPC Land reported that the study includes four checks that the product does not, among them the identical lastmod test that produced the second most common finding. The study page states that lintlab publishes aggregates only and names no site, and that Google has not reviewed or endorsed the study.

Those are the reasons to treat the numbers as a snapshot from an interested party. They are not reasons to dismiss them. The methodology is published in detail, which allows a reader to judge how far the figures stretch.

How the crawl worked

The sample is the top-1,000 origins bucket of the August 2026 Chrome UX Report global snapshot. The crawl ran on 1 October 2026 between 22:15:40 and 22:47:14 UTC, with a re-fetch of blocked origins completed by 23:48 UTC. It checked robots.txt first, following RFC 9309, then located sitemaps from robots.txt declarations and the default paths /sitemap.xml and /sitemap_index.xml. It parsed the XML against the sitemaps.org protocol and sampled up to 20 URLs per origin with HEAD requests. It did not render pages, log in or bypass challenges.

The study examined 1,677 sitemap files containing 13,894,476 URL entries and ran 7,456 status checks. From each sitemap index it fetched at most 5 child sitemaps, chosen by hash, with a ceiling of 8 files and depth 3 per origin. The study says 248,513 index children went unfetched, so its file-level issue counts are floors, not totals.

How many top sites expose a sitemap

Of 1,000 origins, 916 responded as websites. A sitemap was found on 400 of them, or 43.7%. The study reports that 73% of those began with a sitemap index file and that 69.5% served at least one gzip-compressed file. Another 256 origins (27.9% of the 916) had no sitemap found, 165 (18.0%) blocked the crawler with a 403, a 429 or a challenge page, and 77 (8.4%) disallowed fetching through robots.txt. The table lists every outcome the study reports.

The seven most common sitemap problems

Among the 400 sitemaps, 112 (28%) had no measured issue, 248 (62%) had at least one and 40 (10%) could not be fully checked. The next table gives the seven most frequent issues, each as a count of sites and a share of the 400.

Less frequent problems were scheme mismatches (2.0%), malformed XML (1.0%), server errors (1.0%), invalid dates (0.8%), URL fragments (0.5%) and oversized files (0.3%). PPC Land quotes the study as saying the XML itself was rarely the problem and that the URLs and dates inside it were.

What each flagged issue means

The study defines each check, and the reasoning it gives is useful for anyone auditing their own file.

Redirects: a URL counts when its first response was a 3xx. The study notes that Google asks site owners to list the URLs they want shown in search results and that Google generally shows canonical URLs, so a sitemap full of redirecting URLs lists the wrong addresses.

Identical lastmod: every URL entry carries the same value. The study cites Google's position that lastmod is used only when it is consistently and verifiably accurate, and observes that a single shared value may record when the file was generated, not when each page changed.

Different hosts: the hostname in a loc entry differs from the sitemap's origin. The study cites sitemaps.org, which says all URLs in a sitemap must be from a single host. It also notes that this check does not look at whether cross-site submission was set up legitimately through robots.txt or Search Console verification.

Duplicates: the same loc appears more than once. A repeated entry adds no new URL but still counts toward the 50,000-entry limit per file.

Namespace, dates, scheme and XML: the study ties an incorrect namespace and a parse failure to the matching Search Console messages, and says future dates cannot describe the last change. The study applies stricter date parsing than the W3C Datetime format and accepts only full dates or timestamps with seconds and a timezone, so some of its date flags may be stricter than Google's own handling.

What PPC Land adds about lastmod and llms.txt

PPC Land's report adds context that this article has not verified against the original sources. It says Google (Gary Illyes, May 2024) and Bing (Fabrice Canel and Krishna Madhavan, July 2025) have both identified lastmod accuracy as a recrawl signal. It also reports Ahrefs data from May 2026 from server logs of 137,000 domains showing 97% of llms.txt files received no requests at all that month.

On llms.txt, the study's own figure is narrow: 66 of 1,000 origins (6.6%) answered /llms.txt with HTTP 200 and a plain-text or Markdown content type. PPC Land states that 417 origins (45.5%) declared sitemaps in robots.txt, which is a useful comparison for how common each file is among large sites.

What the sources did not say

The study does not name any site. It does not say whether the flawed sitemaps hurt crawling, indexing or rankings for those sites, and it does not measure traffic. It does not say how many of the redirect or lastmod problems come from one CMS or plugin. It reports a single crawl from a single network location, and says live responses change. The authors acknowledge that 165 origins blocked the crawler and 77 disallowed access, which means the picture covers only the sites that allowed a check. Nothing in the study concerns Thailand or Thai websites; its sample is a global top-1,000 list.

The study also notes Google's view that a site of about 500 pages or fewer with good internal links may not need a sitemap at all. That qualifies the headline figure of 43.7%: lacking a sitemap is not itself a defect for every site.

What this means for Thai marketers

The sample is the largest sites by CrUX, not Thai small and mid-sized business sites, so the percentages should not be applied to Thai sites as a benchmark. What carries over is the checklist. The four most frequent problems, redirects, identical lastmod, cross-host URLs and duplicates, are common outputs of CMS plugins and migrations, and each can be checked in an afternoon. That is analysis; the study does not say where the faults came from.

Thai sites that moved from HTTP to HTTPS, added a www or non-www redirect, or switched language folders can check whether their sitemap still lists the old addresses. A routine SEO audit would cover this alongside crawl and indexation checks, and the sitemap belongs inside wider SEO work for Thai sites. On llms.txt, the 6.6% figure suggests the file is still rare among large sites, and its effect is unproven according to the context PPC Land reports; teams working on AI visibility can treat it as optional while putting effort into content and structured data, topics covered under generative engine optimization.

How to check your own sitemap

The study's own recommendations map onto a short audit that anyone with a crawler or a spreadsheet can run.

• Add a Sitemap line to robots.txt with the full absolute URL, and make sure robots.txt returns 200 or a plain 404.
• Serve sitemap URLs with an XML content type and a 200 status, not an HTML page.
• Use valid XML with the correct sitemaps.org namespace.
• Keep each file under 50,000 URLs and 50 MB uncompressed, and split larger sets with an index file.
• Make every loc an absolute URL with one consistent host and scheme.
• List each URL once across all files.
• Write lastmod in W3C Datetime format, never stamp future dates, and avoid updating every value on each site rebuild.
• Sample your listed URLs and check that each returns 200 directly. Replace redirects with their destinations and remove URLs that return 404 or 410.

Frequently asked questions about the sitemap study

How many top websites have an XML sitemap?

The study found one on 400 of 916 responding top-1,000 origins, which is 43.7%. The figure comes from a vendor crawl on 1 October 2026 and covers a global list of large sites.

How many sitemaps have problems?

248 of the 400 sitemaps (62%) had at least one measured issue, while 112 (28%) had none and 40 (10%) could not be fully checked. Redirecting URLs were the most common issue, found on 92 sites (23.0%).

Is the study independent?

No, lintlab sells an XML sitemap checker, and the study page states that Google has not reviewed or endorsed it. The methodology is published, but this article has not repeated the crawl.

Do I need llms.txt?

The study does not say you need it; it found a valid file on 66 of 1,000 origins (6.6%). PPC Land reports that Google said in June 2026 the files neither help nor hurt rankings, which this article has not verified directly.

Does every site need a sitemap?

No, the study notes Google's view that a site of about 500 pages or fewer with good internal links may not need one. Larger or frequently changing sites gain more from a clean file.

If you would like a second pair of eyes on a sitemap, robots.txt and indexation set-up, Relevant Audience can run a technical review. The figures above are lintlab's, reported as published on 1 to 2 October 2026.