High number of pages blocked by Robots.txt

#806147
  • Resolved Anonymous

    7 months, 3 weeks ago

    Rank Math free

    Hello,

    I’m writing because I noticed a very high number of page for my website myproroofing.com are currently flagged on Google Search Console as not indexed because due to robots.txt blocking them. An example is https://www.myproroofing.com/roof-replacement/ for which I ran a live test on GSC directly as well as other 3rd party validator tools and got a message that the URL is not available to Google.

    I tried to follow some of the available guides in your knowledge base to try to troubleshoot the issue myself but it does not seem to have solved it. I don’t know what else to do.

Viewing 1 replies (of 1 total)
  • Rakibuzzaman Siam

    7 months, 3 weeks ago

    Hello,

    Thank you for contacting us and sorry for any inconvenience that might have been caused due to that.

    We checked your robots.txt and we can see the disallow rule is set for all the user agents which is why Googlebot is unable to craw your pages:

    Please remove this disallow rule and add the following default robots.txt rules on your website. 

    User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://yoursite.com/sitemap_index.xml

    Please make sure to remove the https://yoursite.com with your actual domain name.

    However, it looks like you’re currently using the Rank Math PRO plugin on your website, but the account you have used to open this ticket is the Rank Math free version.

    Can you please search and forward a receipt from FastSpring.com so we can locate your account and verify it?

    Looking forward to helping you.

    Thank you.

    RankBot

    6 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 1 replies (of 1 total)

The ticket ‘High number of pages blocked by Robots.txt’ is closed to new replies.