Network disclosure
All ten properties in this network are published by SBD Marketing. Their links improve navigation and topic coverage. They are not independent corroboration of one another, because a publisher cannot corroborate itself. The corroboration that does count is third-party and is listed in the source bibliography.
Check what is indexed
Run a site-restricted search for your domain and count what comes back. Compare that against the number of pages you believe you have. A large gap is the finding.
Google Search Console gives the authoritative version, including the reason a page was excluded. Every defense firm should have it connected, and reading the index coverage report once a quarter catches most problems before they cost anything.
Why pages go missing
| Cause | How it happens | Fix |
|---|---|---|
| Blocked in robots.txt | A staging rule copied to production | Remove the disallow; check after deployment |
| Noindex tag left in place | A page built as a draft and published | Remove the tag, then request indexing |
| Reachable only via JavaScript menu | A navigation built as a script | Add real links in the markup |
| Orphaned page | Published but linked from nowhere | Link it from the practice index |
| Judged thin | A templated city page with no substance | Consolidate or remove |
| Duplicate of another page | Two pages targeting one query | Consolidate and redirect |
The robots file
Read your own robots.txt. It is at the root of the domain and takes ten seconds to check, and a disallow rule left over from a staging site is a genuinely common way for a firm to be invisible without knowing why.
While you are there, confirm the AI crawlers are not blocked. That is covered in chapter 36, and it is the same file.
Sitemaps
An XML sitemap listing every page you want indexed, submitted in Search Console, is worth having and is not a substitute for internal links. A page that appears only in the sitemap and is linked from nowhere is still being presented as unimportant.
Keep it accurate. A sitemap listing pages that no longer exist, or omitting half the charge pages, is a maintenance failure that suggests others.
The quarterly check
Once a quarter: read robots.txt, open the index coverage report, and count indexed pages against pages you meant to publish. Fifteen minutes, and it catches the problems that silently cost the most.
Chapter assignment
The assignment for Crawlability and indexing is to turn the chapter's advice into a testable site component with an owner and rollback path. Begin with the question that caused the reader to open this chapter. Write it in one sentence and name the person who can approve the answer. The chapter premise is: An unindexed page ranks for nothing regardless of how good it is. This is the cheapest problem in the handbook to find and one of the most common.
For Crawlability and indexing, define completion before doing the work. A task is complete when the firm can inspect the changed asset, recover the starting state and repeat the check without relying on memory. Activity counts and dashboard labels do not satisfy that rule. The retained deliverable is the test result, changed files and acceptance record.
Set up the working file
Create the Crawlability and indexing folder under the market and observation date. Put the baseline or source copy first. Add a short manifest that names the asset owner, operator, approver, measurement unit and next review date. Link to firm-controlled credentials through the approved credential system; never place a password in the workbook.
The Crawlability and indexing file records unknown fields as unknown. An empty value is a prompt for the owner, not permission to infer. If two sources disagree, save both and open a conflict row before choosing one. For public legal text, identify the attorney who will check the statement. For measurement, identify the instrument and settings. For vendor work, identify the export and exit right.
Execution pass
- Observe. Save the present state and note why it matters to the firm's criminal defense work.
- Diagnose. Separate the visible symptom from its possible causes. Choose the smallest change that tests the diagnosis.
- Approve. Obtain factual, attorney or commercial approval at the point required by the asset.
- Change. Make one attributable revision and record the operator and time.
- Verify. Repeat the original check under the same scope. Capture the result even when it failed.
- Hand off. Store the files under firm control and assign the next review.
The Crawlability and indexing failure to guard against is shipping a technical change without checking the rendered public page. If it occurs, stop the release. Preserve the failed state and decide whether the work, source or instrument caused the problem. A new method needs a new baseline and version; it should not be spliced into the previous series.
Decision points
Continue the Crawlability and indexing assignment when the starting record is complete, the owner is known and the acceptance rule can be tested. Pause when access is disputed, a legal statement awaits attorney review, the measured unit has changed or a source conflicts with the proposed wording. Cancel the task when it does not match a real charge, market, reader question or business decision.
Prioritize Crawlability and indexing by dependency. Ownership comes before optimization. A baseline comes before a change report. A charge brief comes before prose. Crawl access comes before answer testing. A defined intake event comes before a return calculation. This order prevents later work from resting on an unknown or disputed input.
Quality review
For Crawlability and indexing: Test crawl access and the contact path after deployment, then retain the result. Then ask whether a new operator could reproduce the check from the handoff. Confirm that page headings name the reader's question, local claims name the correct market, and measurement statements retain their units. Verify that no confidential matter fact or unsupported outcome entered a public file.
Check source control for Crawlability and indexing as well. A statement on the firm's own page is first-party. A publisher-owned sibling domain is still the same origin. An award citation must keep the issuer's qualification. A vendor screenshot is useful evidence of a recorded state, but the firm should also retain an export when one is available.
Field exercise
Choose one live example for Crawlability and indexing from the firm's current operation. Use a real market and a real service the firm accepts. Save the present public view before discussing improvements. Ask the asset owner to describe the problem without offering a solution. That account becomes the problem statement and gives the later review something concrete to test.
Now complete the Crawlability and indexing assignment on that single example. Keep a side log of assumptions and evidence requests. When an assumption is confirmed, replace it with the source. When it is rejected, note the effect on the proposed work. Do not expand to a second asset until the first has passed its acceptance rule. This narrow trial exposes access problems and approval delays while the recovery cost is low.
Have a second person review the Crawlability and indexing handoff without verbal explanation. Ask that reviewer to locate the baseline, identify the change, repeat the test and find the approval. Any answer that depends on the original operator's memory is a documentation gap. Repair the file and repeat the review.
Finally, write two Crawlability and indexing report sentences. The first states the observable result in its native unit. The second states the limit or next decision. Remove any causal language that the record cannot support. Store the approved pair with the deliverable so the monthly report does not have to reconstruct meaning from a chart.
Handoff record
The Crawlability and indexing handoff contains the starting record, work completed, approval, final check, open limitations and next action. Give each item a stable filename. The receiving owner should be able to say what changed, why it changed, who approved it and where the proof sits.
End the Crawlability and indexing file with a plain decision: accepted, accepted with a named limit, returned for repair or stopped. A partially completed task should remain open. Carry the unresolved field into the next work cycle instead of hiding it inside a broad progress label.
Acceptance record for Crawlability and indexing
Reopen Crawlability and indexing with the test result, changed files and acceptance record in view. Apply this review: Test crawl access and the contact path after deployment, then retain the result. Record whether the file answers “An unindexed page ranks for nothing regardless of how good it is. This is the cheapest problem in the handbook to find and one of the most common.” at its stated market and date. A different question gets a new assignment and cannot borrow the approval attached to Crawlability and indexing.
Return Crawlability and indexing when it shows shipping a technical change without checking the rendered public page. Keep the rejected copy and name the record needed for another pass. Acceptance means the firm controls a testable site component with an owner and rollback path, can recover the earlier state and has dated the next check.
Related chapters36. Opening the AI crawlers · 28. Site structure · 31. Internal linking