Some Factors regarding Google Crawling

Posted on:November 15, 2017


Category:SEO, Uncategorized

Share: / /


In this article, I will introduce some attractive rarefied SEO facts. Although it may not be an immediate effect, it will acknowledge strengthen SEO knowledge, and it will alleviate happening to boost WEB site ranking within search results.

Let’s take a quick look at the list.


1. Page speed is important

A lot of people are thinking that the page loading speed is slow to the extent that it is annoying to the user, but the result that it brings is not only that. Page speed has long been a search rank element, and Google is said to use mobile page speed as a rank element of mobile search results soon. A lot of people use Google’s page speed insight (tool) to obtain suggestions of site speed analysis and improvement measures. For those who are particularly concerned about improving mobile performance, Google has released a new page speed tool dedicated to mobile. John Muller of Google’s Webmaster Trend Analyst recommends that loading time be limited to 2 to 3 seconds. (Even if it is longer than this, it does not necessarily affect the ranking)


2. The robots.txt file must be placed in the main directory of the site.

This file must be named in lowercase (robots.txt) in order to be recognized by the crawler. Furthermore, the crawler refers only to the main directory of the site. If not found there, it presumes that the file does not exist and continues crawling. The order at robots.txt cannot stop (perfectly) ranking within Google

There are many misunderstandings about the “Disallow” command in robot.txt. The robot.txt file simply tells Google not to crawl disallowed pages, folders, and parameters, and it does not mean that these pages are not indexed. The Google’s Search Console Help documentation is as follows. You should not use robots.txt for the purpose of hiding from Google search results. Because it is thought that you get index by avoiding robots.txt file because other page shows that page.


3. Google does not mind how to build a site map.

As long as it is an XML file, a site map can be constructed in any way. That is, breakdown of categories and the overall structure are all up to you, and it does not affect how Google crawls.

There are many things to know about the site map

  • The XML sitemap must be encoded in UTF-8
  • Must not contain the session ID of the URL
  • The number of URLs is 50,000 or less, and it should not exceed 50 MB
  • The Sitemap index file contains multiple it is recommended over the site map.
  • You can use another site map according to the type of media, such as video, images and news.


4. Usually, Google crawls the home page first

This is not a fixed matter, but usually Google sees the homepage first. The exception is when there are many links to a specific page in the page. Google is scoring internal and external links separately. Links from third party sites to your content or website are evaluated as separate from links from your own site.


5. Of the first pages of Google search results, half are HTTPS

Web site security is increasingly important. In addition to a safe site ranking boost, Chrome now emits a warning when a user encounters a site that is not secure. And the webmaster seems to respond to this update. According to MOZ, it seems that more than half of the websites on the first page of search results are HTTPS. If you want to block your page against search results, you can use other methods such as password protection or no index tags and instructions.


Leave a comment