1.0
Map
The crawler walks the site the way a search engine would, following links until it has the full list of pages a visitor can actually reach.
Most problems do not live on the homepage. A broken booking form, a page served without HTTPS, a template that collapses on mobile: these hide two or three clicks in, on the pages that convert.