It is said that Google always indexes pages blocked by the robots.txt file. But Google knows what kind of questions they rank these pages?

Do you want to contact a digital marketing agency in Pakistan? Why not contact A One Sol?

This is a question that is born in Google Webmaster Central Hangout:

“Now, everyone talks about the intent of the user. If the page blocks and robots. txt files are filed by how Google determines the relevance of the query to the content of the page?”

In response, Google’s John Mueller said that Google could not be examined that the content had been stopped.

Google is looking for other ways to compare URLs with others, which blocks the robot-txt file, which is more difficult.

In most cases, Google prefers indexes of other pages on this site to become more accessible and inactive.

Occasionally, if Google thinks they are useful, pages blocked by the robot-txt file will be ranked as search results. This is fixed from the page link.

How does Google calculate ranked pages? The answer link has been reduced.

Get our SEO services in Pakistan to outrank your competitors.

Lastly, this cannot be suggested to prevent content with robots.txt, and Google is expected to know what to do.

However, if a robots.txt file blocks your content, Google will do everything possible to determine its rating.

You can hear the full answer from Google’s John Mueller at 9:49.

“Itis then blocked by robots.txt, apparently we cannot authenticate the content. Therefore, we have improvised and other URLs that can solve those issues which are very difficult. The URL will need to find a way to compare. “

Because this is because of this so much more difficult; if you have great content and an index to crawl, then we are usually trying to use a random robot page instead of something.

So, from this perspective, it’s not so small. Sometimes we show bad pages in our search results because we know they work very well. For example, when people come in touch with them, we can say it might be worth it.

Read also: MicrosoftAdvertising Releases Great Tool That Generates Hundreds of Ad Variations.

Therefore, as a website owner, I do not recommend using the robot-txt to stop its content. I hope everything is fine. However, if a robots.txt file blocks your content, we will try to show it somehow in the search results. “