Publishers were always welcome to exclude their content from Google News by filling out a contact form in the help center, but now the search giant is making the process automated.
Publishers can block any content — whether it be a single page or whole site — from Google’s index via a Robots Exclusion Protocol (REP), a text file that the web crawler searches for to make sure it has permission to include on the index. Publishers can now include directions to opt out of Google News in the REP while staying on the regular search index or vice versa. Of course, the publisher could forbid any content from appearing on any part of Google.
Automating this process gives publishers far more control, something that should please. The day before, Google updated First Click Free to allow news sites to limit the number of free clicks from Google News to five a day.
These developments come a week after word leaked that News Corp. was chatting with Microsoft about de-indexing its content from Google for cold hard cash. Instead, Wall Street Journal and other News Corp. content would appear in Bing’s search. Apparently Microsoft has been propositioning the same idea to several European news outlets.
So you could call Google’s move a flinch, but at the same time the Internet behemoth may be letting these news organizations dig their own graves instead of giving their sinking revenue away to Microsoft.