Meta tags (meta tags) is a very useful label, it is located in the head of the document storage, some pages of information, the most common is description and keywords, they were told what is the search engine, page description and keywords. Use the following:
: you can crawl the page.
NOFOLLOW command: tell search engines to find links, refused to allow access from this page.
HTTP-EQUIV is used to provide some information to the browser, according to these instructions accordingly. HTTP-EQUIV is not only a function of this character encoding ", commonly used HTTP-EQUIV type also includes:" the expiration time, the default script language, the default page style language, auto refresh time.
INDEX command: tell search engines crawl the page
How to limit the spider
NOINDEX command: tell the search engines are not allowed to crawl the page
: don’t grab the page, but this page can crawl along the other link
, its role is to specify the character encoding of the current document is GB2312, which is Chinese simplified characters. According to this line of code, the browser will be able to identify the page should display Chinese simplified characters. Similarly, if we replace "GB2312" with "Big5", known as the traditional character of the Chinese.
1. prohibited search engines crawl the page.
also has some other such as:
here, META NAME=" ROBOTS" refers to all search engines, here we can also refer to a search engine, such as META NAME=" BOT (Google spider baby; noble) " META, NAME=" Baiduspider (love Shanghai spiders) & quot; etc.. Content has four commands: index, Noindex, follow, nofollow, command to English "," separated.
Meta tags in the crawl (crawl)
FOLLOW command: tell the search engine can find links from this page, and then continue to crawl down access.
in order to make search engines banned grab this page, it is our general practice to add the following code in the meta tag page:
according to the above command, the combination of four kinds of
: you can grab the page, but also can follow this page to link other index