Checking a ‘Noindex’ tag

Published on 8th February 2017

Sometimes a ‘Noindex’ tag needs to be added to certain pages of a website. In the case below, a client had a set of redundant pages within the site, this was due to them using a Shopify platform, where a ‘vendor’ page link, which was used to simply display further content on the main product pages, was in fact unintentionally taking consumers to a page displaying just an image.

Examples of the redundant pages are listed below:

This potentially meant Google was unintentionally indexing these pages and may become confused when a client searched for example, “Emmy wedding court shoes”, as rather than displaying the core page we indented, it could display one of the redundant ones. Having many of these “Low quality” or redundant pages can lead to dilution of the overall quality of the website in Google’s eyes and as a result negatively impact the entire website.

To solve this issue, we firstly unlinked these webpages in the site to no longer link to the ‘Vendor’ page but the intended product pages. This meant that there was no longer any way Google (or any other search engine or user for that matter) could find these pages. Secondly, we contacted the developers to set up a ‘Noindex’ tag, specifically for the products containing ‘vendor’. This would allow us to address the oages that were already in Google’s index and have them removed.

A noindex tag is used to instruct search engines that the page being viewed, should not be included in their index.

How to check if a noindex tag is in place

To ensure this had been done by the developer correctly, I searched a selection of the vendor pages, right clicked for ‘View page source’ and searched for ‘Noindex’. As you can see below this ensured this was in place:

To double check, I then used the tool ‘Screaming Frog’ to search for the main site URL, then selecting ‘Directives’, and filtering ‘Noindex’:

Clearly showing here, that the ‘Noindex’ has been successfully put in place.

What to do next

As our objective here is to clean up Google’s index (records) of a website and it’s pages, the next step would be to monitor Google to see when the redundant pages are removed.

This can be done using a “site:” search in Google:

site:http://www.emmylondon.com/collections/vendors?q=white+lacup+baby+shoes

The above would ask Google to return whether it has a record of the specific page in its index.

To look at the bigger picture and try and establish what “vendors” urls are in Google’s index, you could use a more advanced site search which includes the “inurl” query:

site:http://www.emmylondon.com/ inurl:vendors

How long will it take to remove a page from Google’s index

Great question. Unfortunately there is no set time in which it will take for a page to be removed from Google’s index as this depends on when the next time Google decides to crawl/look at the page, identify the noindex and then take action.

With a regularly updated, top-level or well linked page, Google could pick up on the noindex directive in a few days, hours, even minutes! However, if the page you are trying to remove is poorly linked and hasn’t been updated for years the rest of web, then it could take months before Google sees your changes!

How can i speed up the process of getting a page removed from Google’s index

The best thing you can do in this instance is to try ad tell Google directly to reindex the page. This can be achieved by:

  • Have just a handful of pages?

    Use Google Search Console’s “Fetch and render” tool  and then use “request index”

    Google Search Console Fetch and Render

    Google Search Console Fetch and Render

    Google Search Console Request Indexing Tool

    Google Search Console Request Indexing Tool

  • Have lots of pages?

    Use Google Search Console to submit a sitemap containing these links and then remove it after everything has been recrawlled.

    Google Search Console Add / Test Sitemap

    Google Search Console Add / Test Sitemap