Hello,
Thank you for contacting Rank Math today.
I have checked your site and the robots.txt file is editable as shown below:
Please note that the file is empty by default in order to allow the plugin to control the settings dynamically depending on how you have configured the robots meta of your pages.
Looking forward to helping you. Thank you.
Hi there,
Thanks for the reply.
In that status, you only need to set the cursor in the text box and start typing and then hit save.
Let us know how this goes.
Eh? I could have sworn I tried to add text in the input box….
Are these settings recommended (the greyed out ones):
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://reclaimdesign.org/sitemap_index.xml
Hi Michael,
Thanks for the reply.
Yeah you should keep it blank with the default settings since at most times custom robots settings will conflict with the ones set in the plugin.
Rank Math will then update the robots.txt file dynamically.
I hope this info helps. Thank you.
Thanks Michael. Earlier I added this – can you confirm it’s OK please?
User-Agent: *
Allow: /wp-admin/admin-ajax.php
Allow: /wp-content/uploads/
Disallow: /wp-admin/
Disallow: /wp-content/uploads/wpallimport/
Disallow: /wp-content/uploads/sucuri/
Disallow: /wp-content/uploads/screaming-frog/
Disallow: /wp-content/uploads/js/
Disallow: /wp-content/uploads/fonts/
Disallow: /wp-content/uploads/ao_ccss/
Disallow: /*?reviews-page=*
Disallow: /*?*reviews-page=*
Sitemap: https://reclaimdesign.org/sitemap_index.xml
Hello,
Thank you for contacting Rank Math today.
It seems OK. Always keep in mind to only allow sections of your site you wish Search Engines to crawl and Disallow if you don’t wish to.
Looking forward to helping you. Thank you.
Thanks Michael. As long as I’m not breaking anything 🙂
Hi Michael,
It is always a good idea to test your changes:
https://www.google.com/webmasters/tools/robots-testing-tool
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
