The Paid Search Impact of Google Instant – Some Initial Data
The arrival of Google Instant created great excitement in the search community. There has been a lot of speculation about the impact that Google Instant will have on paid and natural search. Will it help or hurt the long tail? What will its impact be on conversion rates? How will click-through dynamics change on the page? Will big brands benefit from an increased traffic share? Some initial data started surfacing at SMX East and some blogs. The consensus seems to be that there are no dramatic changes as yet. Here we will present our own data on the paid search impact we have observed so far.
IMPRESSIONS AND AVERAGE CPC
We looked at data for a set of our US retail clients and compared metrics after the launch of Google Instant on September 8th, up to October 6th, against a similar length period prior to the launch. In figure 1 we show the distribution of percentage changes in overall impressions and average CPCs before and after the launch of Google Instant at a campaign level. We focus on a reasonably large set of campaigns where the overall spend has stayed fairly stable over the period. In each boxplot the horizontal black line represents the median change for each metric. An earlier post explain how to interpret the boxplots in figure 2. For impressions there was a median 2.9% decrease before and after the launch of Google Instant, and for CPCs a median 3.2% increase. Given the variability in the data, it would be very premature to get too excited about the changes we see below. It really suggests that nothing has changed significantly in terms of impression volumes and CPCs. A change may come when people will change their search behavior over a longer period of time and get familiar with Google Instant.
THE EFFECT ON THE KEYWORD TAIL
Many pundits argued that the arrival of Google Instant would spell the end for the long tail. A searcher may start a search with the intention of using a long-tail search term but may end up getting good enough results when they have only typed a part of the original intended query. Others argue that most people would finish typing their original query anyway. We analyzed a large set of over 20,000 paid search keywords across several US advertisers to see if we see any drop in the revenue contribution from tail terms. We compare the 3 weeks before Google Instant to the 3 weeks thereafter. There were 22,031 keywords with an impression in the 3 weeks before Google Instant. This dropped to 20,911 keywords after Google Instant. This is not significant, as this level of variation was not unexpected for such a large keyword set even before Google Instant. What we are more interested in, is to see if there has been a shift in the contribution to total revenue between head and tail terms. In figure 2 we rank all keywords into percentiles according to their traffic contribution, starting with the highest traffic keywords on the left. As an example, we see that the top 10% of keywords in terms of traffic account for about 80% of total revenue. We can consider these as head terms. Total revenue refers to click date revenue only, as there will be very little data with full cookie revenue. If there has been a substantial shift in traffic towards the head keywords after Google Instant, we would have expected the red line in figure 2 to have shifted above the black curve on the left of the plot; however, it is clear that there has been very little change in the relative contribution of head terms and tail terms. This suggests that we are not seeing a dramatic decrease in the contribution of tail terms to total revenue yet.
CHANGING CLICK-THROUGH DYNAMICS
An interesting potential impact of Google Instant is how it will affect the click-through dynamics on the first and subsequent pages of search results. There are some theories suggesting that relatively more clicks may occur on higher positions than before. The data for a group of our US clients in figure 3 suggest that there may have been a subtle shift in click-through rates before and after Google Instant. We plot the square root of the click-through rate in figure 3, as it results in better visualization due to the heavily skewed click-through rate distributions. We focus on a subset of keywords with at least 1000 impressions in a 4 week periods before and after the launch of Google Instant. The median click-through rate (represented by the black dots) seem to have shifted slightly higher at the top positions and slightly lower at the lower positions.
Subsequently, we fit a logistic regression model that enable us to model the click-through rate as a function of position and match type before and after Google Instant. The regression fits are shown in figure 4 for 2 match types. They also suggest that there has been a subtle increase in click-through rates after Google Instant at the higher positions and a subtle decrease at lower positions. In order to evaluate the statistical significance of these changes in the light of the inherent variability in the data, we fitted another regression model for three position ranges: top (positions 1-2), middle (positions 3-6) and bottom (positions 7-12) instead of individual positions. We then compute click-through odds ratios (and their 95% confidence intervals) for comparing the odds of a click before and after Google Instant for a specific position range. For a refresher on click-through odds refer to this earlier post. The results are summarized in table 1 below. If there has not been a significant change in the odds of a click we would expect the confidence interval to include 1. Table 1 suggest that the odds for a click on the top positions has increased by about 6.6% after Google Instant, while the odds for a click has decreased by about 13.1% after Google Instant at the lower positions. The change for the middle positions is much smaller and barely significant, reflected by the fact that the corresponding confidence interval almost includes 1. The lower positions here also include some page 2 positions, which may suggest that Google Instant is reducing the importance of page 2 results as well. The relatively thin data on second page positions prevent confident inference about the effect on page 2 at this stage.
Below we summarize our findings:
- There does not seem to be a significant shift in overall impressions and CPCs at this stage.
- Our data does not provide any evidence at this stage that we should start preparing for the funeral of the long tail of paid search just yet.
- A subtle shift seems to be happening in terms of click-through dynamics on the first page (and possibly the second page). There seems to be a slight increase in click-through for higher positions and slight decrease for lower positions, resulting in relatively more traffic from higher positions than before. If this results in higher competition amongst advertisers for higher positions in order to maintain volume, it is certainly not something Google will be too unhappy about. The best strategy remains to bid traffic to its estimated value rather than focusing too much on position.
- More time is needed to see how Google Instant affects longer terms search behaviour, as people become more familiar with it.
- Google Instant could also affect conversion rates, which was not investigated here. We will investigate this once we have gathered some more conversion data.