For website publishers, a poor web rating can be disastrous. Bad television ratings mean television shows get canceled, bad web ratings mean websites go out of business. (http://www.digitalhome.ca/content/view/1818/1/)

I read recently that there are 800 of those tv boxes in the GTA, so I don't trust tv ratings. How do web ratings work? By hits? Are hits trustworthy?

The truth is that no one is entirely sure hence why the IAB is demanding an audit of their methodologies.

From what I have learned, Comscore uses various giveaways and contests to get users to download and install an application on their computer. this application, which I believe is called relevant knowledge, then sets up a proxy server to monitor you web surfing activity.

Here is what PestPatrol has to say about the software

Overview

Vendor Description
RelevantKnowledge is part of an online market research community with over 2 million members worldwide. RelevantKnowledge relies on its members to gain valuable insight into Internet trends and behavior. In exchange for participating in periodic surveys on topics of interest to the Internet community, and for having their Internet browsing and purchasing activity monitored, RelevantKnowledge sponsors select software that its members can enjoy for free.
Alias
Spyware.Marketscore [Symantec], Proxy-OSS [McAfee],
Category
Spyware : Any product that employs a user's Internet connection in the background without their knowledge, and gathers/transmits info on the user or their behavior. Many spyware products will collect referrer info (information from your web browser which reveals what URL you linked from), your IP address (a number that is used by computers on the network to identify your computer), system information (such as time of visit, type of browser used, the operating system and platform, and CPU speed.) Spyware products sometimes wrap other commercial products, and are introduced to machines when those commercial products are installed. See also Adware.


Origins

Author
MarketScore
Others By This Author
RelevantKnowledge A · RelevantKnowledge E ·
Date of Origin
April, 2006


Privacy Policy
http://www.relevantknowledge.com/RKPrivacy.aspx

The problem with that methodology is that you've got a biased sample, those people who are interested in the giveaways etc, and don't mind installing the app.

Any time that you've got a biased sample then it's very hard, or impossible, to use your biased data to get unbiased results.

I agree but comscore says they use statistical analysis to get rid of the bias.

From their methodology page

comScore has developed a statistical methodology to enhance the accuracy and reliability of projections to the total population based on its network.

I'd love to see it. The only ways I know of correcting for sample bias need to know the pattern of bias. For example, if you have too many people under 25 and not enough people over 50, then you can weight the under 25's lower and the over 50's higher. But this sort of bias, where it's a behaviour bias and not a simple demographic bias doesn't give you any easy way of compensating.

From Today's Globe:

The 'Page View" is so last year... (http://www.theglobeandmail.com/servlet/story/LAC.20070426.TWINGRAM26/TPStory/?query=page+view)

If you run a website and are trying to impress someone -- an advertiser, an executive in your company, or just your Aunt Agnes -- the term "page view" is a handy shortcut.

As many have pointed out, these kinds of figures can be easily boosted. MySpace has been criticized for making it impossible to do anything without clicking through four or five pages, and Facebook is similar.

Nielsen said in a news release that it is going to focus on what it calls "time-spent" data, which will track how long a visitor spends on a site, and comScore also said that it is working on data about "visits," which would aggregate the number of visits and total time spent by a user over a month.

Interesting article but the reality is that page views, unique visitors, and visits are all relevant ways of measuring the success of a website (just like ratings are of television shows) but they have to be accurate.

The IAB sent its letter to the two firms because it found its sites were grossly underreported by the two measuring services.

In other words a company like Coke knows from its web analytics package and server logs that it gets 20 million visitors a month but maybe comscore says the number is 2 million and NNR says its 3 million (I'm making the numbers up for illustration so don't quote me on them) so then Coke says, if comscore and NNR are so wrong about my site's traffic, how can I believe anything else they say.

These companies methodologies are not open to scrutiny so no one really has a clue why they are so different from reality