robots.txt

5 posts in this topic

Posted

Has anybody got any good robots.txt tips to share? I have set up my forum with google webmaster tools ect ect and need a robots.txt (not just for google as apparently it doesnt listen) but to stop bots from bombing the sites pages? Or would someone be willing to share theirs with me as a template :)

Con and symbian07 like this

Share this post


Link to post
Share on other sites

Posted

There's a robots.txt file in the board download, I use that and works well for me.

Kurisu87 likes this

Share this post


Link to post
Share on other sites

Posted

thanks to the both of you, i was unaware of the robots.txt, as it was named robotstxt.txt so google was saying it did not exist and i read the other post, some pretty useful stuff which i have implemented! thank you

Share this post


Link to post
Share on other sites

Posted

Its just named that by default so if you already have one present you do not overwrite it :) , in a similar way the config file with a new board is called .dist.php so its more or less ignored (and you don't overwrite the "real" one)

The Guy and Kurisu87 like this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Who's Browsing   0 members

    No registered users viewing this page.