robots

Hello, I’m sorry to hear that you’re experiencing issues with your robots.txt file. If you’re using Nginx and have added the configuration to the .htaccess file, it’s possible that the issue is related to the Nginx server configuration.

To resolve this, you’ll need to add the appropriate rewrite rules to your Nginx configuration file. Here are the rewrite rules you need to add:

# START Nginx Rewrites for Rank Math Robots.txt File
rewrite ^/robots.txt$ /?robots=1 last;
rewrite ^/([^/]+?)-robots([0-9]+)?.txt$ /?robots=$1&robots_n=$2 last;
# END Nginx Rewrites for Rank Math Robots.txt File

Once you’ve added these rewrite rules to your Nginx configuration file, be sure to restart your Nginx server to apply the changes.

If you need further assistance with this, please feel free to reach out to our support team. We’re here to help!

Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix the ‘Blocked Due to Other 4xx Issue’ Error » Rank Math
Rank Math Sitemap Issues and Their Fixes » Rank Math

Hello,

Thank you for contacting us and sorry for any inconvenience that this issue might have caused.

Please note that Rank Math does not generate or create a physical robots.txt file in your website files as it is created virtually.

In this case, please check your Nginx config file and ensure there are no directives blocking robots.txt on your site or serving .txt files if they are missing on your site.

For Rank Math to create a virtual/dynamic robots.txt, there shouldn’t be any existing robots.txt file in your website files (root directory).

Alternatively, you may create a static robots.txt file on your root directory and include our recommended rules here:
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/.#default-rules

Hope that helps.

Thank you.