Data trumps intuition. You can use web analytics tools to measure the difference.

See the below screenshot of a report for my blog's javascript status as measured using Yahoo! Web Analytics:

You don't need to believe what anyone else says, you can measure your websites and your own customers.

Comparing log files and javascript driven data is like comparing apples and horses.

At some point they were related (early in the formation of planet earth :)) but they have evolved so much just in the last few years that it would be nearly impossible to come close with both methods.

But if you do want to reconcile your data, here is a guide you can use:

The Ultimate Web Analytics Data Reconciliation Checklist

Good luck!

Avinash.

Still, I'm seeing a huge difference between visits in GA and visits in the log server analysis software I'm using called Summary (www.summary.net).

I've already turned off known robots in the report, and I'm filtering all images, multimedia, stylesheets, XML feeds, and JS.

What other factors do I need to consider to reconcile the difference in visits between the two software?

Steve comments that “something like 50%…of my current users either can’t, won’t or don’t run JS.”

Edward Vielmetti commented that the percentage is more 2-3%.

Has there been any research that offers a more empirically substantiated percentage of users that don't use JS?

Ultimately it depends on what you are using the data for and the complexity of your site. Moreover for the vast majority js solutuions are more than sufficient, after all the majority of affiliate/marketing business has been built on the back of this kind of tagging and they've been paying cash off the back of them.

The overstated dream if you like, is to easily integrate transactional data to behavioural, with maximum flexibility to manipulate the raw data and import into other BI systems (so also as accurate as possible); in addition maintenance and ease of implementation are part of the consideration (though that is clearly debatable also and depends on your setup if your dev is outsourced etc). I know Unica, webtrends, etc are also focusing on these areas too and having open systems seems to becoming the general trend now which is great.

Thanks for the link, will definately have a look into the various hardware solutions in this space too.

I guess with more standardisation, is it too much to wish for vendors to seamlessly combine W3C standard logs, JS, and any other methods (packet sniffing etc), and extract one state of the truth to wherever you want to use that data?

Both statements are "fundamentally flawed". :)

PION is an improvement on how packet sniffing used to be done, it still suffers from the limitations that would have to be overcome with tagging / log filing / data combining with a existing tagging solution, though half of one's life and resources will go towards creating that "perfect" (let me emphasize in words the quotes) system.

There is no system on Planet Earth on Jan 11 2010 that will collect EVERY (emphasis yours) transaction. Sad, but absolutely true.

Every data collection mechanism today comes with a set of pro's and a set of con's. The duty we have as stewards of spending money for our companies wisely is: Which data collection system provides the best balance of investment and ROI (insights)? In an imperfect world.

The answer will be different for each company, and there is no blanket right answer for everyone.

I'll fail if I also did not add that perfection is vastly overrated.

-Avinash.

PS: For your data collection investigation here's one more: From Coradiant the Analytics In A Box solution: http://goo.gl/AEJL No javascript tagging required.

PPS: If you want to learn more about pro's and con's about each data collection mechanism (in short blurb's) here is a video that might be of value (the first part):

http://www.kaushik.net/avinash/2007/08/web-analytics-vendor-tools-comparison-and-one-challenge.html

In my mind tagging is fundementally flawed (javascript can be easily switched off; not available on all platforms especially mobile; and often fails to be called due to tag positioning or load times and internet imperfections).

Log analysis is old hat and shows only half the story.

I've been looking at some new tools on the markets such as PION (Atomic Labs), who use packet sniffing and export the data into 3rd party web analytic tools. Is anyone aware of any others who do this kind of thing, as this seems to be the only logical way forward (unless your happy with google style high level analytics – javascript is fine. If you need EVERY transaction, you have to look elsewhere).

mackaycs