The internet has at least 4.46 billion indexed pages currently active (this is most likely a gross underestimate). Millions of new pages are published every day, and millions of old pages are removed.
So what happens to the URL when a page is removed?
When a page is removed from a server, any links to that page usually become 404 errors (this is the default response for most servers).
There are many good reasons to make sure 404 errors are fixed, but what might surprise you is that there are also good reasons to let some remain.
We’re here to help you understand the best ways to handle a 404 error and whether you should fix it or forget it.
How to Find 404 Errors
Before you can decide what to do about 404 errors, you need to know that they exist. There are several ways to check your site for these errors, and even ways to see if other sites are linking to non-existent pages on your site (ok, that seems like a bit of an oxymoron, but bear with us and we’ll explain).
Check Your Site For Broken Links
Don’t let visitors to your website see something like this broken image!
There are several tools available that will crawl your entire site and show you every 404 error. Screaming Frog sets the bar in this regard, and their SEO Spider is a great tool for finding these broken links.
Their free tool will crawl 500 URLs looking for 404 errors, showing you the link source and destination, canonical errors, and even lets you search by URL if you don’t want to crawl the whole site.
They also offer a paid version that has more functionality, which we recommend for more advanced sites.
There are also free Google Chrome plugins like Check My Links that let you do this page-by-page if you don’t have a large site and want to keep it simple.
The information isn’t as detailed and it lacks much of the functionality of Screaming Frog, but if you just want to find a broken link or two, plugins like this will get the job done.
Google Webmaster Tools will provide a complete list of broken internal and outbound links on your site, but also includes a way to see broken links that come from other websites.
Check For Broken Inbound Links
Google Webmaster Tools can show you all of the inbound links that point to a URL on your domain, even if the URL is incorrect (only your domain name needs to be correct to show up in GWT). This can happen if a site is trying to link to you but there is a typo in the URL, which results in a 404 error (to avoid doing this yourself, we recommend copying and pasting URLs).
These 404s can be ignored as they will not affect your SEO, but there may be a way for you to do something about them.
Finding them is easy. From the Google Webmaster Tools Search Console Dashboard, go to the left column navigation menu. Select Crawl and then Crawl Errors. Google will show you a list of all of the errors on your site.
If you click on one of the URLs you will be shown a box with error details, and on the Linked From tab you will see a list of all of the pages containing inbound links.
If the linking site is reputable and the link juice from the inbound link is worth the time, then you can try to contact the site and let them know about the error. This may also be a good way to open a channel of communication for any future content you produce that might be a good fit for their site.
If the broken URLs are the result of site scraping then it may be possible to redirect the URLs, but since there is no SEO penalty for you it may not be worth the trouble.
You can also send Google a takedown request if you believe that a site is using your content illegally.
Are 404 Errors Bad For SEO?
404 errors are a normal and expected part of any website. There are going to be times when pages are removed or content is made unavailable, resulting in a 404 error.
Google (et al.) knows this, and they don’t penalize you for it as long as you use a proper 404 error instead of a “soft 404”.
A “soft 404” is not a proper 404 page, but a regular page showing a 200 error. Google reads these “soft 404” pages as regular pages with no content, which is bad for your site’s SEO.
Make sure your 404 response code is set up properly in the HTML when you remove a page from your server to avoid any penalty from search engines.
404 errors can be a problem for your site even if they aren’t a problem for Google. These errors stop traffic from reaching your site, which reduces your traffic and provides a poor user experience.
Both of these negative effects can be mitigated through proper 404 page design, which we will discuss in detail below.
Some 404 Errors Can Help SEO
Broken links on other websites can actually help your site’s SEO! Part of your link building strategy should involve creating a list of sites you would like to be published on or receive links from.
If you are regularly monitoring these sites for broken links (using the tools we discussed above), then you could notice broken links on these sites before they do.
Sending an email informing them of the broken link along with the URL to your similar (but superior) content can generate good will and a point of contact for the future along with a link for your site.
Everybody wins!
What To Do About 404 Errors
Once you’ve found all of the internal and external 404 errors on your site, you need to decide what to do about them.
We’ve already talked about how and when to do something about broken inbound links (in the Check for Broken Inbound Links section above), so we’ll only talk about how to handle internal and outbound links here.
Internal Links
If an internal link on your site is showing a 404 error, your best option is to link to other similar content, if available. You may have created multiple pieces of content that cover the topic, so all you need to do in that case is fix the link and set up a 404 error response code for the old URL.
Problem solved!
If you don’t have any similar content, then we’re going to assume that the content wasn’t driving traffic to your site anyway and probably isn’t worth saving or recreating.
If this is the case, just remove the link, edit the text as necessary, and set up a 404 error response code for the old URL.
404 pages are necessary even after you remove the links, since search engines may still index the pages and other sources may also link to the removed content.
You should design your 404 error pages to provide visitors with more information, like links to other content. You don’t want that page to be an exit point for them, just another stop on their way to conversion.
We’ll talk more in a minute about 404 page design best practices.
Outbound Links
If you find broken links to other sites you have linked to, the process is simple. All you need to do in this case is to find a different source that covers the topic.
If you were referencing general information, or a product or event that other sites have covered, this shouldn’t be a problem.
If you were referencing specific content and an alternative won’t work or isn’t available, you’ll need to remove the link and then edit your content to allow for the removal. Simply removing the link won’t be enough since your text references the missing content.
You do not want to leave links on your site that send visitors to another site’s 404 error page – this provides a poor user experience and makes you look unprofessional.
Be the First to Notice Broken Links
You should be using the tools we discussed at the beginning to monitor your site for these errors on a regular basis. Make sure you notice any 404 errors before your visitors do!
How to Optimize a 404 Error Page
A 404 error page should be treated just like any other page of your site. The design should be uniform, compelling content should exist, and it should help to move visitors further down the funnel.
This page should look identical to the other pages of your site. The header and footer should be the same, but the navigation bar can be removed from the page.
The content should be more than just a simple “Page not found” message. Give your visitors more information or somewhere else to go. You can provide engaging (i.e. funny or informative) text along with links to related content or to other pages of your site.
Whatever content you put on this page, you want to be encouraging visitors to continue deeper into your site. This page should not be an exit door!
Conclusion
Knowing how to find 404 errors and your options for dealing with them is crucial to running any website.
By taking the appropriate action, you can improve your user experience, build links to your site, and keep visitors on your site and on their way to converting into customers.
