How to NOINDEX, FOLLOW product attributes links

#672271
  • Resolved Anonymous
    Rank Math free

    Hi there,

    I’m asking for help regarding an issue I have on several websites I monitor and it seems like that we, using Google Merchant Center very early in the website life can lead to important issues with indexing our pages.

    For each website, we upload a feed of our catalog that includes all the variants. This means that each variant has its own link.

    Example:

    The product is “Rank math hoodie”
    Canonical link is: domain.com/product/rank-math-hoodie
    And each variant has its like this: domain.com/product/rank-math-hoodie/?attribute_pa_color=black&attribute_pa_size=xl

    These attribute links redirect to the same canonical page, there is no difference, and that is what may create the problem.

    The main issue is that these attribute links are indexed instead of the canonical because of the merchant center feed cannibalizes the indexing. There is no issue with the canonical tag, it’s just that Google stupidly prefers to index the wrong link for nothing, so we need to force the noindex setting on the attributes pages.

    What would be the best solution that automatically noindex these attribute links but still lets Google craw and follow them?

    I tried to use a PHP or javascript code but I cannot see the meta tag change to noindex as it keeps the tag of the canonical page instead.

    If you could help me out with this, that would be great as we will be lots of people suffering from this problem soon, so managing to solve this now is very important.

    Thank you very much to your time and hopefully we can manage this together.

Viewing 7 replies - 1 through 7 (of 7 total)
  • Hello,

    We apologize for the issue you’re facing with the indexing of the attribute links on your website. We understand that it’s important to ensure that Google indexes the correct canonical page and not the attribute links.

    To address this issue, you can use Rank Math to set the noindex tag on the attribute links while still allowing Google to crawl and follow them. Here’s how you can do this:

    1. In your WordPress Dashboard, navigate to Rank Math SEO > Titles & Meta > Misc. Pages.

    2. In the Misc. Pages settings, you will find the Noindex Paginated Single Pages option. You can enable this option to set noindex on paginated pages, including the attribute links, in order to prevent them from showing up in the search results.

    NoIndex-Paginated-Single-Pages

    By enabling this option, you can ensure that the attribute links are set to noindex while still allowing Google to crawl and follow them.

    If you have any further questions or need additional assistance, please feel free to reach out to our support team, who are always more than happy to help.

    Thank you for reaching out, and we hope this solution helps resolve the issue you’re experiencing.

    Sources:
    Advanced Tab in Rank Math SEO’s Meta Box
    How to NoIndex Paginated Pages » Rank Math
    Setting Canonical URLs in Rank Math » Rank Math
    How to Fix “Crawled – Currently Not Indexed” Error in Google Search Console » Rank Math

    Anonymous
    Rank Math free

    I have activated the option in the bot answer, but the meta tag of the page is still not changed to noindex, thus I can’t know for use if the situation is solved or not. Would it be possible for human support to check with me if the situation is solved?

    Thank you in advance

    Hello,

    Thank you for contacting Rank Math support.

    Please try adding this code to your website to programmatically set any page/URL that contains that query parameter to bounded, this would make sure Google does not index those URLs.

    Here’s the code to add:

    // Get the requested URL
    $url = $_SERVER['REQUEST_URI'];
    
    // Define the forbidden query
    $forbidden_query = '?attribute_pa_';
    
    // Check if the URL contains the forbidden query
    if (strpos($url, $forbidden_query) !== false) {
      // Set the robots meta to noindex
        unset( $robots['index'] );
        $robots['noindex'] = 'noindex';
      }
      // Return the robots meta
      return $robots;
    }

    Please add this code to your website and let us know if it works for you.

    Thank you!

    Anonymous
    Rank Math free

    Hi, where do I put this please?

    I’ve added this code in the functions.php:

    function customize_robots_meta_tag($robots) {
    // Get the requested URL
    $url = $_SERVER[‘REQUEST_URI’];

    // Define the forbidden query
    $forbidden_query = ‘/?attribute_pa’;

    // Check if the URL contains the forbidden query
    if (strpos($url, $forbidden_query) !== false) {
    // Set the robots meta to noindex
    unset($robots[‘index’]);
    $robots[‘noindex’] = ‘noindex’;
    }

    // Return the robots meta
    return $robots;
    }

    // Hook the custom function into the wpseo_robots filter
    add_filter(‘wpseo_robots’, ‘customize_robots_meta_tag’);

    But it still doesn’t change the tag to noindex, can you check as well using the URL in the sensitive data?

    Hello,

    The filter you have added above is not for Rank Math’s meta robots.

    Please add this filter instead to change those filtered URLs to noindex and follow:

    add_filter( 'rank_math/frontend/robots', function( $robots ) {
    	$url = home_url( $_SERVER['REQUEST_URI'] );
    	if (strpos($url,'?attribute_pa') !== false)) {
                    $robots['index'] = "noindex";
                    $robots["follow"] = 'follow';
    	        return $robots;
            }
    	return $robots;
    });

    You can add the filter to your theme’s functions.php file or you can follow any of the methods mentioned in the guide below to add this filter to your website:
    https://rankmath.com/kb/wordpress-hooks-actions-filters/

    Hope that helps.

    Anonymous
    Rank Math free

    Hi there,

    This one worked, thank you. However, it has removed the canonical tag from the page, I don’t know if it’s problematic, what do you think?

    Hello,

    That is normal since the canonical URLs are automatically removed if a page is set to noindex.

    Here’s a link for more information:
    https://rankmath.com/blog/canonical-urls/#common_canonicalization_issues

    Hope that helps.

    Thank you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 7 replies - 1 through 7 (of 7 total)

The ticket ‘How to NOINDEX, FOLLOW product attributes links’ is closed to new replies.