Hello,
I’m sorry to hear that you’re facing issues editing your robots.txt file. It seems like you’ve encountered some difficulties with the process. I’d be happy to help you with that.
To assist you further, please assign this ticket to a human agent by clicking on the “Assign to me” button below. Our dedicated support team is available 24/7 and will be more than happy to help you resolve this issue.
Thank you for reaching out to us, and we appreciate your patience as we work to resolve this matter for you.
Sources:
I Can’t Edit My Website’s Robots.txt » Rank Math
How to Edit Your Robots.txt With Rank Math SEO » Rank Math
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
Hello,
Thank you for your patience.
Please share the rules you have added in Rank Math and the robots.txt URL so we can check as well.
Meanwhile, ensure that you don’t have any existing robots.txt file in your website’s root directory. You may get in touch with your web host as well to confirm that.
Looking forward to helping you.
On Sat, Dec 9, 2023 at 3:00 PM UTC, Mohammed Ismail Anwar <a************@g********> wrote:
The following is the rules that I have done on Rankmath
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /cgi-bin/
Allow: /page/
Allow: /blog/page/*
Allow: /blog-articles-2/page/*
Disallow: /amp/page/*
Disallow: /dgd_scrollbox/
Disallow: /?s=*
Disallow: /go/
Disallow: /recommended/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /index.php
Disallow: /xmlrpc.php
Disallow: /?comments=*
Disallow: /search?
Disallow: /?p=*
Disallow: *?replytocom
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /tag/
Disallow: /author/
I have checked that I dont have the file on website’s root directory.
I have also checked hosting dose not serve robots.txt
Hoping to hear from you soon.
Thankin you,
Best regards,
Ibrahim
Hello,
It seems like you didn’t add the URL of your robots.txt in the last reply. We will need it to determine if the robots.txt is being generated by Rank Math dynamically.
You can flush your permalinks by going to WordPress Dashboard > Settings > Permalinks and clicking Save Changes without making any changes.
Then clear any site/browser cache on your site and check the issue again.
Let us know how this goes. Looking forward to hearing back from you.
Thanks.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
