Quote:
Originally Posted by binkuang
if you use that robots.txt. is that the spider can crawl the newthread and post or not if i put this robots. txt? my forum is new not so many index page. thanks
|
They will still crawl the showthread page which has all the posts on it. The showpost page just shows a single post and the newthread page is used to make new pages - there is no content on it.