Apache robots.txt

debu debajit_kataki at rediffmail.com
Wed Jul 11 02:52:05 UTC 2007


  
Hi Group,

I was just trying to find all the tags that can be used inside apache robots.txt, any link, or any addition to the following?

i have these,

User-agent: 
Disallow:
Crawl-delay:
Request-rate:
Visit-time: 

Thanks
Debajit kataki


More information about the redhat-list mailing list