The Unindexed Garden: On the Fallacy of 'If You Build It, They Will Come'
There’s a persistent, almost folksy piece of wisdom that floats around the edges of web development and content strategy. It’s a comforting notion, a digital-age twist on an old baseball movie line: “If you build it, they will come.” The implication is that quality, in and of itself, is a beacon. That a well-structured page, a beautifully written article, or a genuinely useful tool possesses an intrinsic magnetism that will inevitably draw the attention of search engine crawlers and, by extension, human visitors. It’s a seductive idea, because it places the creator’s virtue at the center of the universe. But it is, in the stark light of the server log, a profound fallacy.
This belief mistakes the nature of the web. The web is not a curated gallery where every new exhibit is immediately spotlighted; it is a vast, chaotic, and deeply indifferent wilderness. A new page is a single seed dropped in a forest of trillions. It has no inherent light. It simply exists. To believe otherwise is to misunderstand the fundamental mechanics of discovery. Crawlers are not omniscient curators of quality. They are finite, logical, and bound by the constraints of their own programming and the sheer physical limits of processing power—a concept we neatly box up and call ‘crawl budget.’
The truth is far less romantic. Discovery is not an automatic reward for quality but the result of a deliberate and often unglamorous process of connection. A page is found because another page, already within the indexed garden, points to it. It is the digital equivalent of a whispered introduction at a crowded party. Without that anchor text, that hyperlinked handshake, the new page remains a stranger in the room, unheard and unseen. A sitemap can serve as a formal invitation, but it’s no guarantee the busy host will have time to read it, let alone seek out every guest it mentions.
This isn’t to devalue quality. Once a crawler arrives, quality determines whether it stays, understands, and deems the page worthy of serving to others. But quality does not summon the crawler. That initial spark of discovery is almost always procedural, not qualitative. It’s a matter of topology, not talent. To assume otherwise is to cultivate a beautiful garden behind a high wall, wondering why no one ever visits. The work isn’t finished when the last seed is planted. The work is in building the gate and the path that leads to it, ensuring that the explorers who already walk the known trails have a reason, and a way, to turn down yours.
Notes & further reading
A few pages I came back to while writing this: