Do Human Raters Have A Direct Influence on Google’s Search Results?
Not surprisingly, Google takes search result evaluation rather
seriously. Exact evaluation enables Google’s
teams to find out "which way is up". One of Google’s principles in
search quality is to rely on data in its decision-making. Google tries hard
not to count on anecdotal fictions, which are mostly misleading in search and
where hundreds of millions of searches a day can be affected. Statistically-meaningful
evaluation gives Google the data needed to create real improvements.
Matt Cutts about how
Google uses human raters
2. It’s not always obvious what a user wants when they type a query into the search box. E.g. what is a user looking for with the query “soccer”? For a soccer game, for tickets, for a local soccer club, for news, for soccer accessories?
3. In the past it was sufficient if a search engine delivered relevant results. But nowadays users ask for more. Of course, the search results still have to be relevant. But they must be free of spam, comprehensive, fresh, timely and should origin from authoritative sources.
4. Google is covering an enormous breadth. More than hundred country and language pairs with in-depth evaluation. Google supports also spelling suggestions, universal search results, video searches, image searches, related query suggestions, and many many more.
To ensure and increase the quality of all the above aspects
of search results Google uses a whole bunch of methods and ways to achieve best
search results. One of those methods is to employ human evaluators. These
evaluators are carefully trained people and they are asked to check the quality
of search results. Sometimes Google shows to the evaluator whole sets of results,
sometimes two different result sets side by side with alternatives to compare.
May be, Google shows only one single result at the time and the evaluator has
to rate it.
One of the key competencies of Google’s evaluation team is experimental design. For each proposed search enhancement they create an experiment plan to check all major aspects of that suggestion. Often they use human evaluation in combination with some other methods. When applying human evaluation, Google asks the raters to rate sample queries for relevance, usefulness and appropriateness of each particular search result. Human raters have definitely not a direct influence on Google’s Page Rank of a specific website. The influence of human raters is limited on the estimation, if a certain proposed change will have a mostly positive impact on the quality of search results in general or not.
One of the key competencies of Google’s evaluation team is experimental design. For each proposed search enhancement they create an experiment plan to check all major aspects of that suggestion. Often they use human evaluation in combination with some other methods. When applying human evaluation, Google asks the raters to rate sample queries for relevance, usefulness and appropriateness of each particular search result. Human raters have definitely not a direct influence on Google’s Page Rank of a specific website. The influence of human raters is limited on the estimation, if a certain proposed change will have a mostly positive impact on the quality of search results in general or not.
Read here, how to create a huge avalanche of web traffic over time:
Comments
Post a Comment