A local home décor business with an e-commerce website built with Magento signed with Boomtown hoping to improve their presence in Google’s search results. They had previously been with another SEO agency, but after several consecutive months of declining traffic decided to move on.
Within Boomtown’s first month (April 2015) of working with the client, we discovered a major technical issue with the website and how search engines were indexing it.
The previous SEO agency had submitted a Robots.txt file requesting search engines not to index any URL containing /index.php/
Many sites require there to be /index.php/ URLs in addition to the clean URLs and it is common to block search engines with Robots.txt
However, in addition to the Robots.txt file, the previous SEO agency had also added canonical tags on every clean URL specifying that the URL with /index.php/ was the preferred version.
Canonical tags are used when a website has 2 identical, or very similar, webpages, which is common with e-commerce sites. The tag tells the search engine which of the 2 pages is the preferred page to show up in search results.
Essentially, the website was telling search engines that they preferred the /index.php/ URLs instead of the clean URLs, but at the same time were telling search engines NOT to index those /index.php/ URLs.
Most of the product and product category pages were not being indexed by the search engines. In fact, Google had only 14 pages indexed for the entire site even though the site actually had over 1,200 pages. This resulted in the vast majority of their web pages being left out of search results, meaning that they received zero visits from Google searches. So, if you were to conduct a Google search for a very specific product, while you were located in the business’s hometown, their product page would not be listed in the results. The only page that was ever included in the search results was the homepage.
Boomtown changed the canonical tags so that the clean URL was the preferred version, rather than the /index.php/ version. For example, we set http://www.example.com/products/ as the preferred version instead of http://www.example.com/index.php/products which was not being indexed due to the Robots.txt file.
The next time search engines crawled the site they indexed all of the product and product category pages. Soon, these pages began to show up in search results and receive traffic.
If you’re finding that your website is not showing up in search results, please contact us and we’ll see if we can fix the problem and get your business back on track.
