Block Google and bots using htaccess and robots.txt | HTML Remix: "Solution 1 : Password protection
Protecting site with htaccess password is the best way to block anyone else accessing the site. But that is not possible all the time when you have demo audience test.
Solution 2 : Robots.txt
Another Solution Google is providing is to use Robots.txt file to tell Bots not to crawl or list pages in results. But that’s not always a solution. Google’s Matt Cuts has confirmed that Google may include pages from such sites if Google think is relevant."
'via Blog this'
No comments:
Post a Comment