Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robots.Txt generator tool is used to generate a file which is very much opposite of the sitemap which indicates the pages to be included. Robots.Txt is a text file which is created by web administrators to instruct search engine robots on how to crawl and index pages that are the part of their website. Robots.Txt file is very powerful and a single text file which could destroy your entire website.

SEOToolsLand Robots.Txt Generator tool allows you to create new robots. Txt file, or edit an existing file for your website. The user can specify as to which search engines you want to include in your personalized criteria. Depending on your new robots. Txt file which user will generate, google or other specified search engines will be pointed to as which pages of your website should or should not be shown in searches. The Tool will give you proper recommendations when you decide to add the new directive, file or file path to either the new or existing robots.Txt file.