Q) What will the following robots.txt file do?
User-agent:Googlebot
Disallow:/*?
User-agent:Scooter
Disallow:
Answers:
• It will allow Google to crawl any of the
dynamically generated pages. It will also allow the altavista scooter
bot to access every page
• It will disallow
Google from crawling any of the dynamically generated pages. It will
also disallow the altavista scooter bot from accessing any page
•It will disallow Google from crawling any of the dynamically generated
pages. It will allow the altavista scooter bot to access every page
• None of the above
Post Views:
135
