-
I need help fixing a noindex crawl error, I have removed or reset the robots.txt file to default settings and also checked all the rankmath settings to make sure tags, categories, and posts are set as index and follow…please help, I need these pages to be crawled asap. I have also cleared my cache and can’t figure out why it still won’t allow crawling and indexing of any of my pages, posts, tags, or categories sitemaps.
-
Hello,
I’m sorry to hear that you’re experiencing issues with the noindex crawl error. It’s great that you’ve already checked the robots.txt file and the Rank Math settings to ensure that tags, categories, and posts are set as index and follow.
To further troubleshoot this issue, you can start by checking whether the page’s header includes the “noindex” robots meta tag. You can open the page source of the URL that has been classified by this error to confirm this.
If you find that the page includes a noindex meta tag, you can follow the relevant steps provided in our guide on how to fix the submitted URL marked ‘noindex’ error. You can access the guide here.
If you’ve already confirmed that the page is set to noindex correctly, you should also check if Google is able to access your website’s sitemap. You can use a tool like httpstatus.io to check if your sitemap is accessible or if it is throwing an error.
If your sitemap is not accessible for any other reason, you should be able to figure it out by now, so you can sort it out accordingly.
I hope this helps! If you have any further questions or need additional assistance, please feel free to ask.
Sources:
How to Fix Submitted URL Marked ‘NoIndex’ Error » Rank Math
Troubleshooting the “Couldn’t Fetch” Error in Google Search Console » Rank Math
Why Google Hasn’t Indexed Your Post, Page, and/or Website? » Rank MathHello,
We’ve checked your site and robots.txt, but they seem to be working fine.
Also, your URLs are already appearing on the search results page. Please refer to the screenshot in the sensitive data section.
Moreover, please note that it is completely normal for a newly published post/page/website to take time before it gets crawled or indexed by Google. It depends on a lot of factors. Your posting frequency + the domain authority are just two of the many factors Google considers when indexing some new URLs.
Google assigns a crawl budget to your website depending on these factors (especially these two) and that has a direct effect on how soon or how late your content can get indexed:
https://developers.google.com/search/blog/2017/01/what-crawl-budget-means-for-googlebotHere’s a link for more information to ensure that your site is set to index:
https://rankmath.com/kb/google-index/Hope that helps.
Thank you for the reply and the screenshots. However, for example when I do a very specific search for our blog post “Travel Vaccinations UK: Essential Precautions for Safe Travels ” it doesn’t show up at all on search…We also use Brightlocal and we have 100 keywords being tracked and still show zero results, which I have never seen before, and have submitted all our pages and sitemaps and made sure all of them are set to index and follow. So any information to help us get this cleared up would be greatly appreciated! Thank you
Hello,
We checked your site on the Google SERPs using the
site:
operator and we can confirm that your pages are being indexed on Google as expected.We can also confirm that the page that you are concerned about is also indexed in Google SERPs :
Based on my review of the webpage “Travel Vaccinations UK: Essential Precautions for Safe Travels” on Pinpoint Travel Health, here are some specific SEO improvement suggestions:
1 . The title of the page is “Best Travel Vaccinations UK: 18 Essential Travel Precautions.” Consider revising it to more closely match the target search query “Travel Vaccinations UK: Essential Precautions for Safe Travels.” Additionally, ensure the meta description is engaging and includes the primary keyword.
2 . While the page contains relevant information, it should consistently incorporate the target keywords and related phrases throughout the content, headings, and subheadings.
3 . Strengthen the internal linking structure by linking this page to and from other relevant content on your website. This strategy helps in distributing page authority and improving site navigation.
4 . The content is informative but could be enhanced with unique insights, more detailed information about specific vaccinations, or data on travel health risks in the UK. Adding case studies or testimonials could also improve user engagement.
5 . Analyze what competitors ranking for your target keywords are doing and apply any relevant strategies to your page.
Consistent efforts in these areas should improve the visibility of your page for specific search queries. SEO is an ongoing process, so continuous optimization and monitoring are essential.
We hope that helps, and please don’t hesitate to get in touch if you have any other questions.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
The ticket ‘Noindex Error for sitemap and all urls’ is closed to new replies.