Robots Text File Required says Google

Are you keeping your site or pages from being indexed?

Google’s Eric Kuan stated, in an online discussion that Google won’t index a website if they can’t access the robots.txt file of that site.

He said, “When Google encounters trouble crawling your robots.txt file, it will stop crawling the rest of your site to prevent it from crawling pages that have been blocked by the robots.txt file.”

If you are using WordPress, it includes a simple robots.txt file that’s dynamically generated from within the WP application.

But what happens if you want to change it to keep certain pages or even folders from being indexed? Enter the WP Robots Txt Plugin, it allows you to do that as needed. Here is a link to the plugin (copy / paste).

Late Breaking News:

Google webspam team actively monitoring your Gmail account

In a recent post on his site Charles Floate, a UK based Search Engine Optimisation and Search Marketing Consultant performed a study to check if Google is reading Gmail users mail for webspam.

Here is an excerpt of what Charles has to report.

“This promoted me to do my own study. I did a fake correspondence with myself and apparently another client, in which we did a black hat site. I said I’d been doing things like buying links and other monitoring based keywords I thought the NSA.. I mean Google Webspam team would be monitoring.

The fact that Google is using underhanded tactics to begin targeting individual people and not just websites is a testament to how the company has changed from the ‘Do no evil’ stance they once had. I don’t believe we can trust Google with our privacy, our security or even to have good morals and do the right thing.”

Read more (copy / paste).:

9 Responses to Robots Text File Required says Google

  1. Gabriele says:

    Is there an easy method to create one of these robots text files on my blog?
    Thank you!

  2. Renato says:

    Awesome post.

  3. vilmak says:

    You could possibly have heard about write-up rewriters.
    Many people have no concept iin regards to what rewriters are why theey may be required.

    • admin says:

      Yes, that is true.

      It is very difficult to write massive fresh content for your site on a schedule, but that is what you must do if your goal is to rank your site or drive traffic to it using the search engines. The more you need to drive traffic the more relevant content is needed.

      There are people on microjob sites that can take PLR articles and rewrite them, but the cost is high. It’s best to use a rewriter that has ENL capability to rewrite. Such a product is Spin Rewriter which we have as an advertiser. Try them I think they will impress you.

  4. zarabeaman says:

    My family members every time say that I am wasting my
    time here at net, however I know I am getting experience all the time by reading these fastidious articles.

  5. UK says:

    Every weekend i used to go to see this web site, sluggish loading instances times will
    very frequently affect your placement in google and could damage your quality score if ads and marketing with Adwords.

    Anyway I’m adding this RSS to my e-mail and could look out for a lot more of your respective interesting content.

Leave a Reply

Your email address will not be published. Required fields are marked *