#Bot Control #Put this document only in root directory #(Search engine) spiders will not crawl these pages User-agent: * # means ALL bots will be affected Disallow: /lib/