The Gardener's Folly: On the Overzealous Pruning of the 'Broken' Branch
There is a piece of advice so common in our field it has become dogma: find your broken links and fix them. The 404 error is treated as a kind of digital blight, a failure in the garden of your website that must be eradicated. We are told that these errors squander crawl budget, frustrate users, and signal neglect to the all-seeing engines. We deploy armies of crawlers to hunt them down, and we prune them with zealous efficiency. But what if this relentless pursuit of perfection is, in some small way, a mistake?
Consider the link that points to a page that no longer exists. We see a dead end. But to a curious crawler, it is something else entirely: a question. A 404 is not a wall; it is a door that has been closed. And the act of closing that door—of serving a proper, gentle 404 page—is itself a powerful signal. It tells the engine, with absolute clarity, "This path ends here. Turn around." This is not wasted effort; it is definitive wayfinding. By cleaning up every single broken internal link, we are not just removing dead ends; we are removing the signposts that clearly mark them.
Our obsession with a pristine, error-free site creates a different, more insidious problem: the silent redirect chain. In our panic to "fix" a broken link, we often point it to the nearest related page. The product page for a discontinued item is redirected to the category page. The old blog post is redirected to the blog homepage. We create a web of gentle, polite detours. To a user, this might seem helpful. To a crawler, it is a maze of diluted meaning.
Each redirect, however well-intentioned, asks the engine to do more work. It says, "This isn't the place, but maybe try over there?" It consumes crawl budget not with a definitive answer, but with a suggestion. The crawler must follow the suggestion, only to often find a page that is a vague approximation of the original intent. The authority that might have been concentrated on a single, precise URL is now scattered across a chain of guesses.
This is the folly. We fear the blunt honesty of the 404, so we replace it with the ambiguous whisper of the redirect. We trade a clear, momentary stop sign for a long, meandering path that leads to a place of diluted relevance. Sometimes, the most efficient map is one that clearly marks where the roads end, not one that sends you on a scenic tour of somewhere almost—but not quite—like your destination. A broken link, properly handled, is not a failure. It is the most honest form of directions you can give.
Notes & further reading
A few pages I came back to while writing this: