

So if you are using the following code in the head section of a webpage, you are actually telling search engines not to index the content of the page and not to show the page in Google search results The date/time must be specified in the RFC 850 format. A static thumbnail (if available) will still be visible.ĭo not offer the translation of this page in search results.ĭo not show this page in search results after the specified date/time. Directiveĭo not index this page and do not show this page in search results for user queriesĭo not follow any links placed on this page (both internal as well as external links)ĭo not show a “Cached” link in search results.ĭo not show a text snippet or video preview in the search results for this page. But before we discuss that, let’s see all the directives which are supported by the robots meta directive. Depending on your requirements, you can modify the content of this meta tag and achieve desired objectives. This is just a simple example of the usage of Robots meta tag in a webpage.

Like any tag it should be placed in the section of an HTML page on your website, before the section. Think of Robots meta tag as a common directive given to all search engine bots about a specific page on the website which should be treated in a special way. In all these situations, a robots meta tag comes in extremely handy as it can be used to tell search engines not to index a specific page or not to crawl any link on a given page on your website. There are situations when you would not want search engines to crawl any link on a specific webpage. There are situations when you would want to prevent search engines from indexing a page (or a bunch of pages) on your website. If your website has lots of pages, it is not necessary that every page on the website has to be indexed by search engines.
META ROBOTS HOW TO
They seem to forget how to use robots meta tag for SEO and to provide specific directives to search engines on which pages should be indexed and which pages should not be indexed.

When it comes to search engine optimization, most website owners only pay attention to rel=nofollow attribute for links.
