The Navigator and the Dredge: Two Ways of Seeing a Website

I’ve been thinking about how we approach the vast, uncharted territory of our own websites, especially when trying to make them known to the wider web. It often feels like there are two fundamentally different mindsets at play. One belongs to the Navigator, the other to the Dredge. Both have the same ultimate goal—getting pages discovered—but their methods, their priorities, and even their understanding of what a website *is* are worlds apart.

The Navigator views a website as a landscape. It's a territory with features, contours, and natural pathways. A user might enter through a high peak—the homepage—and follow a sloping trail down to a specific valley—a deep article. Another might arrive via a river—a search query—that deposits them at a precise bend in the waterway. The Navigator’s primary tool is the sitemap, which is less a construction blueprint and more a topographical map. It charts the existing landscape, showing the connections between canyons, forests, and streams. The Navigator trusts that the crawlers, like skilled explorers, will use this map to understand the relationships and intrinsic value of the terrain. They spend their time improving the paths, adding clear signposts (internal links), and ensuring the landscape is rewarding to traverse. Discovery, for the Navigator, is a natural consequence of a well-ordered, meaningful place.

The Dredge, conversely, sees a website as an open-pit mine. The site is a resource to be extracted from, and every page is a potential nugget of traffic. The Dredge’s primary tool is the crawl budget, a finite amount of excavation equipment that must be deployed with ruthless efficiency. Their focus is on maximizing yield. They build vast networks of low-grade tunnels (thin category pages, filtered lists) to expose as much ore as possible to the surface. They worry about orphaned pages not because they represent a broken user journey, but because they are wasted assets, veins of material that the machinery cannot reach. Discovery, for the Dredge, is a direct result of logistical calculation and brute-force exposure.

The distinction becomes sharpest when something goes wrong. When the Navigator’s sitemap goes missing, it’s like a ship losing its charts in a familiar sea. The landscape is still there, beautiful and coherent, but the most detailed overview of its connections is gone. Crawlers must find their way by coasting the shorelines of internal links, which is slower but can still reveal the true shape of the domain. When the Dredge mismanages its crawl budget, it’s a mechanical failure. Machinery is wasted on barren rock (low-value pages) while rich seams (critical content) lie fallow. The problem isn’t a loss of understanding, but a critical inefficiency in the extraction process.

Neither approach is purely right or wrong, of course. Most of us operate with a mixture of both. The best strategies often involve the Dredge’s analytical eye for ensuring key pages aren’t overlooked, guided by the Navigator’s holistic sense of creating a place worth visiting. The danger lies in leaning too heavily into one. A site built only for extraction feels hollow and transactional, while a site without any consideration for the mechanics of discovery can remain a beautiful secret, forever hidden in the fog. The balance lies in knowing when you are mapping a territory, and when you are simply digging for gold.

Notes & further reading

A few pages I came back to while writing this: