The Cartographer's First Draft: On Sketching the Known World Before the Expedition
Before the age of satellite imagery, a map was not a definitive record but a hypothesis. A cartographer would begin with portolan charts, sketching coastlines from the accounts of sailors, marking where the known world faded into "Here be dragons." This initial draft, for all its inaccuracies, was essential. It wasn't the final word on geography; it was a template for exploration, a guide for where to point the ships. In our efforts to get our web content discovered, we work in much the same way, and the sitemap is our modern portolan chart.
We often think of a sitemap as a definitive list, an invoice of pages we demand a search engine crawl. But this view is flawed, placing the cart before the horse. A better approach is to see it as the cartographer’s first draft of our own territory. It is our internal sketch of what we believe exists and how it is connected. By submitting this sketch, we are not issuing a command but inviting a collaboration. We are saying, "Here is my understanding of my domain. Can your crawler, with its vast and impartial perspective, help me verify it?"
The process illuminates the gaps in our own knowledge more clearly than any server log. A crawler, acting as the returning explorer, reports back discrepancies. It finds the dead-end streets we drew as thoroughfares—the pages linked to nowhere, the internal broken links we were blind to. It discovers the phantom islands—URLs in the sitemap that return a 404, places we thought were solid ground but are merely legend. It corrects the scale, revealing that we’ve overestimated the importance of some minor pages while leaving vast, valuable continents of content uncharted because we failed to link to them properly.
The Value of an Imperfect Map
This is the true, humbling power of the exercise. The goal is not to produce a perfect document on the first try. The goal is to start a dialogue. Each crawl is a voyage of discovery that refines our map. When we see which pages are actually indexed and how they are connected by internal linking, we are receiving data to redraw our charts. We realize that the ornate, hand-illustrated corner of our map detailing a complex interactive feature might be a cartographic flourish the crawler’s vessel simply cannot dock at, while the simple, well-signposted blog post is a bustling port of entry.
Ultimately, the most successful websites are not those with a flawless initial plan, but those managed by humble cartographers. They understand that their sitemap is a living document, a draft constantly being amended by the feedback from mechanical explorers. They listen to what the crawler finds, and what it misses, to better understand the true topography of their own site. They learn that discovery is not just about being found, but about first understanding what is actually there to find. It’s a continuous process of sketching, exploring, and correcting the map until the dragons retreat and the true, navigable coastline emerges.
Notes & further reading
A few pages I came back to while writing this: