[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Apache robots.txt



  
Hi Group,

I was just trying to find all the tags that can be used inside apache robots.txt, any link, or any addition to the following?

i have these,

User-agent: 
Disallow:
Crawl-delay:
Request-rate:
Visit-time: 

Thanks
Debajit kataki

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]