The fact of “asking search engines not to index this site” will no longer have the same consequences.
WordPress blocks crawl (not indexing) to 5.2
The developers of WordPress have decided to change their method to tell search engines not to index sites. Previously, ticking the box “Ask search engines not to index this site” in the WordPress settings had the effect of adding a line Disallow: / in the robots.txt file . The robots, including Googlebot, were therefore invited not to crawl the site, while the user wanted to tell the robots that they should not index the site, a significant nuance. It could happen that Google finds a link to your site and decides to index it. He was not crawling your site, so he could not identify a possible meta tag name = ‘robots’ set to noindex, nofollow .
What WordPress 5.3 will change for SEO
Starting from version 5.3, WordPress will no longer modify the robots.txt when the “Ask search engines not to index this site” box will be checked. This will allow robots, such as Googlebot, to browse the site. On the other hand, the tag META name = ‘robots’ will be added, with the parameters noindex, nofollow . Google’s bot will arrive on your site, see the tag and understand that the site should not be indexed – even, that pages should be deindexed. The crawl will stop quickly, Google being logically sparing savings for budget crawl sites specifically asking not to appear in the search results.
WordPress sites impacted as of November 12, 2019
Version 5.3 of WordPress is under development: the first beta is expected for September 23, the Release Candidate for October 15, while the final version should be available to users on November 12, 2019. The most popular sites Responsive WordPress updates will therefore be impacted from this date. For more info, you can check out this WordPress note.
Add Comment