meta tags?
meta tags?
Out of context: Reply #3
- Started
- Last post
- 5 Responses
- maximillion_0
http://www.webmasterworld.com some search engines till use them and robots.txt. you can also use robots.txt (which should be read by any crawler) to detect he bad ones..they dont read it. you can also use it to set traps for bad bots by setting up pages that shouldnt be read in the robots.txt file and then blocking ones that robots.txt and then read the pages you dont allow