The technical SEO that actually matters

Technical SEO is a small subject wearing a large hat. There are perhaps six things that genuinely matter, most of them are one line of markup, and the difficulty is almost entirely in knowing which instruction does what — because several of them look interchangeable and are not.

Crawling and indexing are different

This distinction explains most technical SEO confusion. Crawling is a search engine fetching your page. Indexing is deciding to store it and show it in results. A page can be crawled and not indexed, and — importantly — indexed without ever being crawled.

robots.txt controls crawling, not indexing

A Disallow rule says “do not fetch this”. It does not say “do not list this”. If other sites link to a blocked page, Google can still index the URL — showing it in results with no description, because it was never allowed to look.

To keep a page out of results, use noindex, and do not block it in robots.txt. The crawler has to fetch the page to see the noindex instruction; blocking it means the instruction is never read. Doing both is the single most common way people fail to hide a page.

Generate a robots.txt and test a URL against one before deploying it — a stray Disallow: / has taken entire sites out of search.

Canonical tags

The same content is often reachable at several URLs — with and without tracking parameters, with and without a trailing slash, HTTP and HTTPS. A canonical tag names which one is the real one:

<link rel="canonical" href="https://example.com/page">

Rankings then consolidate on that URL instead of splitting between duplicates. Three rules worth following: make it absolute, make every page canonical to itself unless it genuinely is a duplicate, and never point every page at your homepage — which is a surprisingly common mistake and tells Google the rest of your site is worthless.

It is a hint rather than a command. Google can ignore it if the pages differ enough.

Titles and descriptions

The title tag is still one of the strongest on-page signals and the main thing a person reads in results. Put the distinguishing words first — “Subnet Calculator” before your brand name, not after.

The meta description is not a ranking factor, and has not been for years. It is advertising copy: it affects whether someone clicks. Google rewrites it more often than not, usually picking text from the page that matches the query, so writing a good page matters more than writing a good description.

The meta tag generator covers those plus the Open Graph tags that decide how a link looks when shared.

Structured data

JSON-LD in a script tag, telling search engines what a page is rather than what it says — this is a recipe, this is a product, these are its FAQs. It does not improve rankings directly. It makes a page eligible for rich results: star ratings, FAQ dropdowns, recipe cards.

Two things stop it working:

• Marking up things that are not on the page. The structured data must describe visible content. FAQ markup for questions a visitor cannot see is a policy violation, not a shortcut.
• Missing required properties. A type with a required field absent is simply ignored, silently, which is why so much markup does nothing.

Eligibility is not a promise. Google shows rich results when it feels like it. Generate valid JSON-LD for the common types rather than writing it by hand.

Sitemaps

An XML sitemap is a list of URLs you would like crawled. It helps most on large sites and on pages with few internal links; it will not rescue a page nothing links to and nobody wants.

• Only include URLs you actually want indexed — no redirects, no noindex pages, no 404s.
• Give honest lastmod dates. Stamping every URL with the build time claims the whole site changed on every deploy, which is false and a signal crawlers learn to ignore.
• Reference it from robots.txt and submit it in Search Console.

Build one here if your platform does not generate it for you.

Multiple languages

hreflang tells Google which version of a page suits which language or region. It has three rules that trip everyone:

1. It must be reciprocal. If A points to B, B must point back to A. One-directional hreflang is ignored entirely.
2. Every version must reference itself as well as the others.
3. Include an x-default for anyone who matches none of them.

The hreflang generator produces a reciprocal set, which is most of the difficulty.

Why a page is not indexed

In rough order of likelihood:

1. It is too new. Days to weeks is normal for a new site.
2. Nothing links to it — internally or externally.
3. It is blocked by robots.txt or carries a noindex.
4. It canonicalises elsewhere, so Google indexed that instead.
5. It is judged not worth indexing. Thin, duplicated, or near-identical to a page already indexed. This is the real reason more often than people would like.

Search Console’s Page Indexing report tells you which. It is worth reading before changing anything, because four of those five need completely different fixes.

Measuring what arrives

UTM parameters tag a link so analytics can attribute the visit. Keep them consistent — Facebook and facebook are two different sources in most tools — and never put them on internal links, which resets the original attribution and rewrites your own traffic as coming from yourself.

What to ignore

• Keyword meta tags. Ignored by every major search engine for well over a decade.
• Keyword density. Not a thing. Write for the reader.
• Submitting to search engines. Unnecessary. A link is enough.
• Chasing a perfect performance score. Core Web Vitals are a real but small signal. A page that loads in 2 seconds and answers the question beats one that loads in 0.8 and does not.