Published on February 8th, 2015 | by Igal Stolpner
The impact of 404 errors on a new sub domain
A few months ago I wrote about our experience with the rel=”alternate” hreflang=”x” attributes for a new edition we opened for the UK. Then, earlier in 2015, we opened our 3rd edition in English, this time for India.
For our English version of Investing.com, India has been an important country for years. And since we knew exactly what the users from India were looking for, we wanted to provide them with the best possible local experience.
On top of that, thanks to our successful, previous experience with Google’s International tags, we felt quite confident opening an additional edition in English.
So far so good, but unfortunately — this time the implementation of the international attributes didn’t go as smooth as it was supposed to, and one of the programmers who was implementing the attributes did a very small mistake which made Google index over 35,000 errors on a completely new sub domain.
How could this happen?
In short? easily. One of the problems for SEOs with programmers is that when we ask for something, we usually check if the specific request was done properly. But when working on a code of a site with millions of pages, sometimes something very small can go very wrong. It usually doesn’t happen, but this time it did.
In the end of the day, the programmer implemented the attributes on pages which he wasn’t supposed to. As the main site is being indexed VERY often by the Googlebot. The bot followed the attributes’ URLs and indexed hundreds of thousands of pages within a couple of days. Including some pages which never existed on the new sub domain.
How soon did we see a drop in organic traffic?
It was a matter of a couple of days. And it was a big one.
A few days old site which already gained over 2,000 daily sessions from Google dropped by over 80% overnight.
As we track our rankings daily, we also saw that the new edition simply disappeared from the local search engine.
And what about Google Webmaster Tools?
Google Webmaster Tools did notify us on an increase in “not found errors”, but it was before the main increase itself. This means that when we looked into the not found pages on the day of the message, the number of 404 errors was in its hundreds, and we decided to give it a few more days (which of course was a mistake).
So we fixed the errors, and then what?
It took us about two days to figure it out and another day to fix the problem.
A few colleagues didn’t think that the drop in traffic could be so significant because of the 404 errors and they thought that it’s something else. But we simply couldn’t think of anything else.
And the good news is..
A single day after the errors were fixed, the organic traffic (and the rankings) were back.
Google launched the rel=”alternate” hreflang=”x” attributes to both understand better which version of an international site to serve in each country, but also to make a connection between domains and sub domains of the same entity. We tried the rel=”alternate” tags already a few times with new language editions, and I can surely say that the attributes help significantly with the rankings of a completely new site.
At the same time, Google gave webmasters a tool which can be easily misused. And I’m still surprised how strong it is.
From this short case study I understand three things:
1. Google gives trust and authority to completely new language editions with the rel=”alternate” hreflang=”x” attribute. Even if the edition has absolutely no other positive indicators, such as mentions, shares, content or links.
2. Google’s algorithm is so sensitive to a significant increase in 404 errors that it completely ignores what I said a second ago because it’s so afraid to provide a bad experience.
3. In July 2014, Google announced the International Targeting Errors feature in Google Webmaster Tools. But the algorithm still doesn’t know how to differentiate between an increase in not found pages that used to exist, and an innocent bug in the rel=”alternate” hreflang=”x” attribute which points to a page that was never there.