Skip to content
SEO & Websites

Technical SEO Checklist for Tool Websites

A practical technical SEO checklist from a live tools site: canonical URLs, honest sitemaps, noindex for thin pages, titles, schema, speed and IndexNow.

Written byAI Online Tools editorial teamUpdated 8 min readPublished

A tool website needs the same technical SEO as any site, plus extra care with duplicate and thin pages, because tool catalogues grow fast. The essentials: one canonical URL per page, and a sitemap of indexable pages only, with truthful dates. Add noindex on anything unfinished, unique titles and descriptions, connected structured data and fast pages. Finally, tell search engines when things change. This checklist is drawn from what this site actually does, with the reason behind each step.

Tools used in this guide

All free, with no sign-up. They run in your browser.

The checklist at a glance

CheckWhat to doWhy
One host301 redirect www to non-www (or the reverse)Links and signals collect on one URL
Canonical tagEvery page points to its own clean absolute URLStops parameters and copies competing
Real 404sUnknown URLs return HTTP 404No "soft 404" pages in the index
SitemapOnly indexable, canonical URLs, with honest lastmodSearch engines trust it and crawl what matters
Thin pagesnoindex, follow until the page is realKeeps weak pages from dragging the site down
robots.txtBlock private areas only; link the sitemapCrawlers waste no time on login or admin pages
Titles & descriptionsUnique, title ≤ 60 characters, description about 120–158Readable search snippets, no duplicates
Structured dataOne Organization and WebSite, referenced by every pageSearch engines see one connected site
Social imagesA 1200 × 630 card per page typeShared links look specific, not generic
Internal linksRelated tools, breadcrumbs, guides that link to toolsCrawlers find pages; users find the next step
Core Web VitalsLCP ≤ 2.5 s, INP ≤ 200 ms, CLS ≤ 0.1A fast, stable page on real phones
Change pingsIndexNow after deploys; sitemap in Search ConsoleFaster recrawls on Bing and others

One URL per page: hosts, canonicals and real 404s

Pick one host and redirect the other

https://www.example.com/tool and https://example.com/tool are two different URLs to a search engine. This site sends every www request to the bare domain with a permanent (301) redirect that keeps the path and query string. Google's canonicalisation guide lists redirects as the strongest signal for which URL should count.

Add a canonical tag to every page

Each page carries <link rel="canonical"> pointing to its own absolute URL: https://, the chosen host, no tracking parameters, no #fragment. That way a link shared with ?utm_source=whatsapp or ?gclid=… still points back to the one clean URL.

Tip: The canonical tag generator and checker cleans a URL the same way. It forces https, applies your www choice and strips tracking parameters such as utm_*, gclid and fbclid. It also removes index.html, fixes trailing slashes and explains each change.

Redirect duplicate tools to the real one

Tool sites often end up with two pages for the same job, such as an old stand-alone URL and a newer one inside a category. Pick the better page and 301 the other to it. Two half-ranked pages rarely beat one strong one.

Return a real 404 for unknown URLs

If /tools/minifier/anything-random shows a friendly page with HTTP 200, search engines may index endless empty pages. Make sure unknown tool slugs return a true 404 status, and give the 404 page useful links back into the catalogue.

Sitemap, robots.txt and noindex

List only indexable pages in the sitemap

A sitemap is a list of pages you want indexed. On this site it's built from the same list that produces canonical URLs, and anything marked noindex or redirected is filtered out before the XML is written. So the sitemap can never advertise a page the site itself says not to index.

Keep within the limits: 50,000 URLs or 50 MB (uncompressed) per file. Larger sites split into several sitemaps under a sitemap index. The XML sitemap generator builds a file from a list of URLs, and flags relative paths, duplicates and mixed hosts.

Make lastmod truthful

Google's sitemap documentation says it ignores <priority> and <changefreq>, and uses <lastmod> only if it's consistently and verifiably accurate. A common mistake is stamping every URL with the build time on each deploy. That makes every date meaningless.

This site keeps a single "content last updated" date for tool pages, changed by hand only when page content changes. Each blog post carries its own date. Copy that discipline: update lastmod when the content changes, not when the footer or a dependency does.

Noindex thin and placeholder pages

A large catalogue usually contains pages that aren't finished: a tool that's planned but not built, or a demo with nothing behind it. Here, those pages get noindex, follow. They stay out of the index, but the links on them still pass crawlers along to real tools. When a tool starts working, it's removed from the placeholder list and joins the sitemap automatically.

Warning: Don't block noindex pages in robots.txt. If Google can't crawl a page, it can't see the noindex tag, and the bare URL can still appear in results from links. Use one method: crawlable plus noindex.

Keep robots.txt short

This site's robots.txt allows everything except the admin panel and login page, and lists the sitemap URL. That's enough for most sites. robots.txt controls crawling, not indexing, and it's public, so never use it to hide anything sensitive. The robots.txt generator builds a valid file with the sitemap line included.

Titles, descriptions and social previews

Unique titles of 60 characters or fewer

Google usually shows about 50–60 characters of a title on desktop. Rather than cutting titles with "…", this site's metadata helper first drops the brand suffix, then shortens at a whole word. Put the main search phrase first so it survives any cut.

Descriptions that end on a full sentence

Aim for roughly 120–158 characters. This site's helper trims a long description back to the last complete sentence that fits. It pads a very short one with a plain fact about the tool. It also writes category-specific default descriptions, so 30 converters don't share one identical sentence.

Write each title and description with the meta tag generator, then check pixel width in the SERP preview tool, since Google cuts by width, not character count.

Give each page type its own social image

When someone shares a link on WhatsApp, LinkedIn or X, the Open Graph image is the first thing people see. Here, each tool, category and blog post generates its own 1200 × 630 card with the page's name. Other pages fall back to a site-wide card. Pair it with twitter:card set to summary_large_image. The Open Graph generator writes all the tags.

Structured data as one connected graph

This site doesn't repeat the organisation's details on every page. It outputs Organization and WebSite once, in the root layout, each with a stable @id such as https://example.com/#organization. Every page-level item then points to them by @id:

  • tool pages: WebApplication (free offer, any operating system) plus BreadcrumbList;
  • blog posts: BlogPosting with published and modified dates;
  • hub pages: CollectionPage with an ItemList of the tools on it.

The result is one connected description of the site rather than dozens of disconnected fragments. Two details matter in practice. First, escape < in the JSON (as <) so text containing </script> can't break the page. Second, keep markup honest: no ratings you don't collect, no FAQ markup expecting dropdowns, since Google stopped showing FAQ rich results in 2026.

Build and test individual blocks with the schema markup generator, then confirm them in Google's Rich Results Test.

Internal linking that helps people and crawlers

Search engines find pages by following links, and people find the next tool the same way. Three link types cover most of it:

  1. Related tools at the bottom of each tool page, drawn from the same category. Someone formatting JSON may want JSON to CSV next.
  2. Breadcrumbs (Home › Category › Tool), both as visible links and as BreadcrumbList markup.
  3. Guides that link to tools, and tools that link to guides. A guide on compressing a PDF links to the PDF compressor. That's a topic cluster, and it gives thin-looking tool pages real context.

Use descriptive anchor text ("XML to JSON converter"), not "click here". Check that no indexable page is orphaned, meaning nothing links to it.

Speed and Core Web Vitals

Google's Core Web Vitals targets, measured at the 75th percentile of real visits on mobile and desktop, are:

MetricMeasuresGood
LCP (Largest Contentful Paint)Loading2.5 seconds or less
INP (Interaction to Next Paint)Responsiveness200 ms or less
CLS (Cumulative Layout Shift)Visual stability0.1 or less

Tool pages have particular risks: heavy libraries, big inputs and ad slots that push content around. Habits that help:

  • Load heavy code on demand. This site's JavaScript minifier imports its minifying library only when you press the button, not on page load.
  • Move long work off the main thread. The diff checker compares large texts in a Web Worker, so typing stays responsive.
  • Reserve space for ads, images and results panels so the page doesn't jump (CLS).
  • Scope special headers. The background remover's multi-threaded model needs cross-origin isolation. So it's set on that page only, where it can't break ad iframes elsewhere.
  • Cache hashed files for a year. Content-hashed model files are served as immutable.

Google says Core Web Vitals are used by its ranking systems, but good scores don't make up for a less helpful page. Treat speed as part of the product. Check field data in Search Console's Core Web Vitals report and PageSpeed Insights.

Telling search engines about changes: IndexNow and llms.txt

IndexNow for Bing and others

IndexNow lets you ping participating search engines (Bing, Yandex, Seznam, Naver and others) when URLs change. Setup takes minutes:

  1. Generate a key and host it as a text file at https://yoursite.com/<key>.txt.
  2. After each deploy, POST the changed URLs (or every URL in the sitemap) to https://api.indexnow.org/indexnow with your host, key and key location.
  3. A 200 or 202 response means the batch was accepted.

This site runs a small script after deploys that checks the key file is live, then submits URLs in batches. Google doesn't use IndexNow; for Google, the sitemap in Search Console is the equivalent.

llms.txt as an optional extra

llms.txt is a proposed plain-text file that lists a site's key pages for AI assistants. This site generates one from the same catalogue as its sitemap, so it never goes stale. Be realistic about it: Google's documentation says llms.txt isn't needed for Google Search and won't help or hurt rankings. It's cheap to maintain, and other AI services may read it.

Work through the table at the top once, then repeat it whenever you add a batch of new tools. The errors that hurt tool sites most are quiet ones. Think of a placeholder page left indexable, a sitemap date that changes on every deploy, or two URLs for one tool.

Frequently asked questions

Should I block placeholder pages in robots.txt or use noindex?

Use noindex. If robots.txt blocks a page, Google can't crawl it to see the noindex tag, and the URL can still be indexed from links. Keep it crawlable with a noindex robots meta tag.

Do priority and changefreq in a sitemap matter?

Not for Google, which says it ignores both. It does use lastmod, but only when the dates are consistently accurate, so set lastmod to the real date of the last meaningful change.

Does Google use IndexNow?

No. IndexNow is used by Bing, Yandex, Seznam, Naver and others. For Google, submit your sitemap in Search Console and keep lastmod accurate.

Will an llms.txt file help my Google rankings?

Google has said llms.txt files are not needed for Google Search and have no positive or negative effect on rankings. It is a cheap, optional extra for other AI tools that may read it.

All guides