My Technical Notes & Others...: Google

Showing posts with label Google. Show all posts

Saturday, 7 September 2013

Sep 4th, 2013 - Sudden Drop In Traffic - A Thin Or Lack Of Original Content Ratio Issue?

Many people have reported a sudden drop in traffic to their websites since September 4th, 2013.

Google Webmaster forum is full of related posts. A Webmaster World thread has been started. Search Engine Roundtable mentions 'early signs of a possible major Google update'. A group spreadsheet has been created. No one seems to make sense of what is happening. There is a lot of confusion and speculation, without any definitive conclusion.

I have seen a major drop in traffic on a new website I am working on since then. However, the traffic for this blog has remained the same. No impact.

I am going to post relevant information and facts here as I find them. If you have any relevant or conclusive information to contribute, please do so in the comments and I will include them here. Let's try to understand what has happened.

Symptoms

Sudden drop in traffic, 50% or more since September 4th, reports of 90% drops too.
Site used to rank in top 5 results and is not ranking in top 100 results anymore.
No one is reporting new manual action, messages or warnings in Google Webmaster Tools.

Facts & Observations

Many owners claim no black hat techniques, no keyword stuffing, only original content, legitimate incoming links.
Many owners say they have not performed any (or significant) modifications to their website.
All keywords and niche are impacted.
Both old and new websites are impacted.
Both high and low traffic websites are impacted.
Some blogs are also impacted.
It is an international issue, not specific to a country, region or language.
Site with few backlinks are also penalized, not only those with many backlinks.
Nothing changed from a Yahoo or Bing ranking perspective.
One person mentions a site with thin content still ranking well.
At least one website with valuable content has been penalized.
Several sites acting as download repositories have been impacted.
Some brands have been impacted too.
So far, Google has nothing to announce.

Also:

In May 2013, Matt Cutts announced that Panda 2.0 is aiming at getting better at fighting blackhat techniques. He also announced that upcoming changes include better identification of websites with higher quality content, higher authority and higher trust. Google wants to know if you are 'an authority in a specific space'. Link analysis will be more sophisticated too.

Example Of Impacted Websites

Hypotheses

Websites with content duplicated on other pirate websites are penalized.
Websites with little or no original or badly written content are penalized (thin content vs plain content ratio).
Websites with aggregated content have been penalized.
Sites having a bad backlink profile have been penalized.
Sites having outbound links to murky site or link farms have been penalized.
Ad density is an issue.
Google has decided to promote brand websites.
This is a follow-up update to the August 21st/22nd update, at a broader scale or deeper level.
An update has been posted and contains a bug (or a complex, unanticipated and undesirable side effect).

Analysis

Using collected information and data gathered in the group spreadsheet:

Average drop in traffic is around 72%
No one reports use of black hat techniques
12,8% report use of grey hat techniques
23,1% report impact before 3rd/4th September
7,7% have an EMD
17,9% had a couple of 404 or server errors
17,9% are not using AdSense
30,8% admit thin content
38,5% admit duplicate content
25,6% admit aggregate content
15,4% admit automatically generated content
64,1% admit thin or duplicate or aggregate or automatically generated content
The range of backlinks is 10 to 5.9 millions
The range of indexed pages is 45 to 12 millions

The spreadsheet sample contains only 39 entries, which is small.

The broad range for the number of backlinks seems to rule out a pure backlink (quality or amount) issue.
The broad range of indexed pages points at a quality issue, rather than a quantity issue.
More than 92% do not have an EMD, so this rules out a pure exact domain name issue.
More than 82% did not have server or 404 issues, so this rules out them as the main cause for a quality issue.
17,9% are not using AdSense, meaning this cannot be a 'thin content above the fold' or 'too many ads above the fold' issue only.
Some brand websites have been impacted. Therefore, it does not seem like Google tries to promote them over non-brand websites.
Domain age, country or language are not discriminating factors.

Best Guess

By taking a look at the list of impacted websites and the information gathered so far, it seems like we are dealing with a Panda update where sites are delisted or very severely penalized in search rankings because of quality issues.

These are likely due to thin content, lack of original content, duplicate content, aggregate content or automatically generated content, or a combination of these. It seems like a threshold may have been reached for these sites, triggering the penalty or demotion.

Regarding duplicate content, there is no evidence confirming for sure that penalties have been triggered because a 3rd party website stole one's content. More than 60% do not report duplicate content issues.

To summarize it, the September 4th culprit seems to be a high thin or lack of original content ratio issue, leading to an overall lack of high quality content, leading to a lack of trust and authority in one's specific space.

Unfortunately, Google has a long history of applying harsh mechanical decisions on websites without providing any specific explanation. This leaves people guessing what is wrong with their websites. Obviously, many of the impacted websites are not products of hackers or ill willed people looking for a 'I win - Google looses' relationship.

Some notifications could be be sent in advance to webmasters who have registered to Google Webmaster Tools. If webmasters do so, it can only mean they are interested in being informed (and not after the facts). This would also give them an opportunity to solve their website issues and work hand-in-hand with Google. So far, there is no opportunity or reward system to do so.

Possible Solutions

Someone from the Network Empire claims that Panda is purely algorithmic and that it is run from time to time. If this is true, then this might explain why no one received any notifications or manual penalty in Google Webmaster Tools, and why no one will.

Google might just be waiting for people to correct issues on their websites and will 'restore' these sites when they pass the Panda filter again. The up side is that this update may not be as fatal as it seems to be.

Assuming the best guess is correct, the following would help solving or mitigating the impact of this September 4th update:

Re-read Dr. Meyers' post about Fat Panda & Thin content.
Thin content pages should be marked as noindex (or removed from one's website) or merged into plain/useful/high quality content pages for users.
Low quality content (lots of useless text) pages should preferably be removed from the website, or at least be marked as noindex.
Internal duplicate content should be eliminated by removing duplicate pages or by using rel="canonical" (canonical pages).
Content aggregated from other websites is not original content. Hence, removing these pages can only help (or at least, these page should be marked as noindex).
Not enough valuable content above the fold should be solved by removing excessive ads, if any.
Old pages not generating traffic should be marked as noindex (or removed).
Outbound links to bad pages should be removed (or at least marked as nofollow), especially if they do not contribute to good user experience. This helps restore credibility and authority.
Disavow incoming links from dodgy or bad quality websites (if any). One will loose all PageRank benefit from those links, but it will improve their reputation.
Regarding Panda, it is known (and I'll post the link when I find it again) that one bad quality page can impact a whole website. So being diligent is a requirement.
Lorel says that he has seen improvement on his client websites after de-optimizing and removing excessive $$$ keywords.

Something to remember:

Matt Cutts has confirmed that noindex pages can accumulate and pass PageRank. Therefore, using noindex may be more interesting than removing a page, especially if it has accumulated PageRank and if it has links to other internal pages.

Tuesday, 8 May 2012

How To Perform A Keyword Traffic Analysis

This post is about learning how to perform a keyword analysis for traffic using Google AdWord's Keyword Tool. You will need an AdWord account for this. If necessary, apply for one. Then log into AdWord and select the Tools & Analysis tab.

The keyword tool is a mean to estimate how much traffic and user searches is generated according to some words carefully selected by you. Using a metaphor, such traffic can be compared to water flowing in a pipe. The end destination are web pages. The valves and the pipe connections are the key influential words you have selected. They influence the debit and the type of user flow delivered to those web pages.

Performing a keyword analysis is about identifying and estimating the quality and the debit of user flow (traffic) to web pages containing those keywords. It is not about implementing a keyword strategy. This comes after the analysis. The product is this analysis is a set of traffic generating keywords.

What is at Stake?

As a website publisher, you want to make sure the user traffic you receive matches your website content. If you are discussing jam recipes, car sellers will not be interested. It is even more important if you plan to put advertising on your pages. As an advertiser, you want to make sure your ads are displayed to users interested in your products or services. Therefore, you need to be positioned on proper web pages made available by publishers willing to accept advertising.

From a publisher's perspective, even though the keyword tool is primarily developed for publishers planning to put ads on their web pages, it can also be used by those who don't plan to put ads on their pages. For example, if you need users to register to your website to access a service, you have the same goal as an advertiser willing to attract relevant traffic to its ads.

Google AdWord (or any other ad network) acts as facilitator for advertisers willing to find locations for their ad inventory. Advertisers specify the keywords they have selected for their ads and AdWord is in charge of displaying these on suitable web page made available by publishers.

Google AdSense (or any other ad network) acts as facilitator for publishers searching for advertisements for their web page inventory. They make sure their pages are properly identifiable with keywords. AdSense is then responsible for displaying the best suitable ads from advertisers to these pages.

Of course, all this happens for a fee that advertisers pay to Google for publishing ads. A percentage of that fee is paid to the publisher by Google. The difference is a commission Google keeps for its matching (facilitator) role. An advertiser and an publisher could talk directly to each other, but they would have to organize the connection between ads and web pages themselves, which has a cost in time and resources.

How is Efficiency Measured?

From an advertiser's perspective, you want the maximum number of user to click on your ad per ad display (or impression). This is called the CTR for click through rate. It is measured as a percentage. The higher the better for the advertiser.

One way to reward the publisher is to pay a commission per ad click. This is called PPC or (CPC) for pay per click (or cost per click). In this case, it is also in the publisher's interest to maximize the CTR to maximize its commission. If Google does not match ads and pages correctly, the advertiser will be disappointed by the results of its campaign and Google will loose their future ad inventory. If this happens, then it does not have anything to offer to publishers, which will be then disappointed too and reduce their web page inventory offer to Google. Hence, it is also in Google's interest to maximize the CTR.

The publisher may not be satisfied with being paid only when there is an ad click. On the other side, the advertiser may not be interested in paying for each click and will prefer to pay per impressions. The price to pay is then called CPM (or CPI) for cost per mille (or cost per impression). CPM is a cost per thousand impressions (or ads display on publisher's web pages). In this case, it is also in everyone's interest to maximize the CTR.

This explains why advertisers are ready to pay more for higher CTR and why it is in the publisher's interest to match its content to proper keywords. Cheating or failing will only reduce the number of clicking users and therefore, commission.

Now, some advertisers only want to pay when there is a real sell (or an action), eventually after a user has clicked on an ad. This is called CPA (or PPA) for cost per action (or pay per action). They are ready to pay a much higher commission when this happens. In this case, it is also in everyone's interest to maximize CTR, since if there is no click, there is no possibility for subsequent actions and sales.

But CTR is not the only efficiency factor to be taken into account, volume (or traffic) is also as important.

From a PPC perspective, everyone remains happy to see a high CTR (the higher the traffic), since advertisers get more customers and publishers get more commission. From a CPM perspective, advertisers will be interested in the quality of the traffic. If it does not match their keywords, CTR will plummet and they don't want that (low bang for the buck). So they need a mean to control this. From a CPA perspective, the publisher is interested in the quality of the traffic, because it wants users to click on the ad in order to trigger a possible sell (or action) behind this. So they need a mean to control this too.

How to Control Traffic Quality and Quantity

Back to the keyword tool screen. On the left column, there is a Match Type box. It controls the connection between keywords and the users' queries. There are three types of "valves".

Broad means that if keywords appear at least once in a user search request, it counts as traffic. This means more traffic in general and more flow to web pages, but not necessarily of quality. If the keywords are 'Victorian Furniture', the 'Victorian wood furniture' search will count as traffic as well as 'furniture'. If you are an advertiser for Victorian era furniture, you will not be interested in all users querying about 'furniture'. It can be a low bang for the buck situation.

Exact means the keywords must match the user query exactly, in the same order (not more, not less) to be considered as traffic. The advertiser for Victorian furniture will be interested in this option, because it is obvious that the user shows interested when typing 'Victorian furniture'. It is a high bang for the buck situation. However, such option will miss 'Victorian wood furniture' which is interesting traffic too.

Phrase means that if keywords appears in that order in a phrase, it counts as traffic. If a user types 'Victorian furniture book', the advertiser will get this traffic, but he won't get 'Victorian wood furniture' traffic. He will also get 'Victorian furniture reparation' traffic as well as 'Victorian furniture destruction' too.

A publisher will mostly be interested in phrase traffic, unless he develops a niche website about selling and buying Victorian furniture. In this case, the exact option will be more appropriate to evaluate traffic.

How to Use the Keyword Tool

After selecting the match type (broad, exact or phrase), type in a word or phrase such as orange paint for example.
You can control additional search options such a location and language, etc...
Google AdWords displays results for orange paint for example: around 14000 monthly searches globally.
But, it also displays keywords ideas which could be related to orange paint too. This where publishers can find extra keyword ideas for their pages, together with an estimation of users searches (i.e., potential traffic)

Publisher should use this tool iteratively, to search for new keywords, earlier suggestions, or using synonyms. Each valuation combination should be written down with potential traffic. This collection of keywords can then be used in web pages to capture corresponding traffic.

There is another tool for advertisers called the traffic estimator. For a given keyword set and given budget, it provides an estimation of daily clicks, impression, CTC and average CPC. We will not cover this here, but it is easy to use and well documented.

Conclusion

Performing a keyword traffic analysis is pretty easy and only requires understanding a couple of simple principles. It is a huge traffic differentiator between websites implementing a proper keyword strategy and those who don't.

Pages