Do Human Raters Have A Direct Influence on Google’s Search Results?


Not surprisingly, Google takes search result evaluation rather seriously. Exact evaluation enables Google’s teams to find out "which way is up". One of Google’s principles in search quality is to rely on data in its decision-making. Google tries hard not to count on anecdotal fictions, which are mostly misleading in search and where hundreds of millions of searches a day can be affected. Statistically-meaningful evaluation gives Google the data needed to create real improvements.
Matt Cutts about how Google uses human raters
 
Evaluating search algorithms is difficult for several reasons:


1.    Comparing Search Engines (whether versus its competitors or over time) is never black and white. No change ever will be 100% positive. Any change in algorithms will result in some searches getting better and some worse.

2.    It’s not always obvious what a user wants when they type a query into the search box. E.g. what is a user looking for with the query “soccer”? For a soccer game, for tickets, for a local soccer club, for news, for soccer accessories?

3.    In the past it was sufficient if a search engine delivered relevant results. But nowadays users ask for more. Of course, the search results still have to be relevant. But they must be free of spam, comprehensive, fresh, timely and should origin from authoritative sources.

4.    Google is covering an enormous breadth. More than hundred country and language pairs with in-depth evaluation. Google supports also spelling suggestions, universal search results, video searches, image searches, related query suggestions, and many many more.
To ensure and increase the quality of all the above aspects of search results Google uses a whole bunch of methods and ways to achieve best search results. One of those methods is to employ human evaluators. These evaluators are carefully trained people and they are asked to check the quality of search results. Sometimes Google shows to the evaluator whole sets of results, sometimes two different result sets side by side with alternatives to compare. May be, Google shows only one single result at the time and the evaluator has to rate it.

One of the key competencies of Google’s evaluation team is experimental design. For each proposed search enhancement they create an experiment plan to check all major aspects of that suggestion. Often they use human evaluation in combination with some other methods. When applying human evaluation, Google asks the raters to rate sample queries for relevance, usefulness and appropriateness of each particular search result. Human raters have definitely not a direct influence on Google’s Page Rank of a specific website. The influence of human raters is limited on the estimation, if a certain proposed change will have a mostly positive impact on the quality of search results in general or not. 

Read here, how to create a huge avalanche of web traffic over time:

Comments

Popular posts from this blog

Unique Keyword Rich Content Is Best SEO

Why do you think, you can make money online?