{"id":176,"date":"2007-01-23T17:58:23","date_gmt":"2007-01-24T00:58:23","guid":{"rendered":"https:\/\/www.kaushik.net\/avinash\/web-analytics-tool-selection-10-questions-to-ask-vendors\/"},"modified":"2026-09-20T09:30:34","modified_gmt":"2026-09-20T16:30:34","slug":"web-analytics-tool-selection-10-questions-to-ask-vendors","status":"publish","type":"post","link":"https:\/\/www.kaushik.net\/avinash\/web-analytics-tool-selection-10-questions-to-ask-vendors\/","title":{"rendered":"Web Analytics Tool Selection: 10 Questions to ask Vendors"},"content":{"rendered":"

\"WhiteJerome N, a reader from the UK, asked me to write a post on the questions that should be presented to a Web Analytics Vendor during tool selection process. My instinctive reply to Jerome was \u201cI think it is more important to know what questions to ask of yourself (your company) before you go into tool selection because the answers to those questions are a lot more critical and will tell you what you are ready for<\/em>\u201d.<\/p>\n

My reaction was that because I feel companies don\u2019t do a good enough job of understanding themselves (in a deeply self critical way) and hence pick a sub optimal solution. But it was also sourced from the fact that in vendor pitches all the answers are \u201cyes we can do it\u201d and it is very hard to discern reality. (If you are\u00a0a Vendor web analytics tool selection Process<\/a>).<\/p>\n

But since Jerome was so kind to ask I wanted to respect his request. It is important to note that most of my suggestions below you would ask the for-fee (paid) Vendors, but you should also evaluate the free vendors in this context. I\u2019ll also fess up that in a vendor pitch or conversation it is hard to get tease out the nuances but hopefully there is enough meat in rest of the post to empower you to tease out some reality.<\/p>\n

10 Questions to ask Web Analytics Vendors: Summary:<\/h2>\n

1) Main differences with free tools.
\n2) Types of versions and flexibility.
\n3) Types of data collection options and entrenchment.
\n4) TCO (Total Cost of Ownership).
\n5) Type of Support (options, pricing, technical or business).
\n6) Segmentation\u00a0awesomeness (post data capture).
\n7) Exporting data (options, history, data ownership).
\n8) Integration with other sources of data.
\n9) What\u2019s up next, the competitive edge.
\n10) Types of business lost, why.
\n11) Bonus Question.<\/p>\n

Jerome here are ten questions you should ask a Web Analytics Vendor before<\/em> you get too deep into selection\u2026\u2026..<\/p>\n

1) What is the difference between your tool \/ solution and Gatineau<\/a> analytics package from Microsoft, or if you want log file based solutions then the free 10\/90 rule for web analytics success<\/a>?<\/strong><\/p>\n

The answer you are looking for is not that Google and Microsoft are big and bad and your privacy is under threat. That\u2019s a cop out if there ever was one. You are also not looking for a answer that you won\u2019t get support for a free tool or that they will die and wither away.<\/p>\n

You are looking for specific and tangible examples of reports and metrics that your for-fee vendor will provide that Google (and later Microsoft) don\u2019t provide. Any analytics Vendor worth paying for will have a crisp answer that focuses on differential features, reports, metrics, integration points etc and not scare tactics.<\/p>\n

I want to stress that GA and Gatineau are not right for everyone and don\u2019t have every feature you want.\u00a0You should carefully consider fit of those tools for your company. But it should be based on facts and not some of the FUD (Five \u201cEcosystem\u201d Challenges for Web Analytics Vendors<\/a>.\u00a0 Currently most vendors are ASP based with no software based offerings (except for WebTrends and ClickTracks, I believe). That is ok for now, might not be so in a year or two.<\/p>\n

With this question you are trying to probe for how ready the Vendor is for the future in terms of having differentiating offerings for you should you want them (and I think you will, not tomorrow but in the near future). You are also looking for the intangible: how they react to this question, just as much as the content of their answer.<\/p>\n

You can also ask them about first part and third party cookies, which one they use and how much pain, and cost, would it be to use first party. You should almost always use first party cookies, most Vendors enable this. What you are looking for when you ask this question is their reaction. Did they proactively advice you to have first party cookies? Did they insist on it? It shows the mindset of the Vendor.<\/p>\n

3) What data capture mechanisms do you use? Javascript only? Javascript and web logs? Web logs only? Others?<\/strong><\/p>\n

There are many different ways to capture data, javascript tags and web logs and packet sniffers and \u201csensors\u201d and so on and so forth. Javascript tags and web logs are the most common, each have their merits (if you are stuck between making a choice between them, after having considered all options, flash, flex, RIA\u2019s, RSS<\/a>, mash-ups and things we don\u2019t even know). No current methodology will survive for a long time,\u00a0is the Vendor you are considering ready for the near future data capture challenges?<\/p>\n

You are not looking for them to brainwash you that javascript tags are God\u2019s answer to all your prayers (or web logs or packet sniffers or \u201csensors\u201d). If they try that chalk it up\u00a0in the sub optimal sales choice column.<\/p>\n

<\/a>4) Please help me calculate the TCO (Total Cost of Ownership) for your tool. <\/strong><\/p>\n

Most Vendors will tell you that that the cost of their tool will be $30 per month for any site of any size and it is a all you can eat buffet<\/a>. Ok so I am stretching that a bit, but by not that much. :)<\/p>\n

The Total Cost of Ownership of a web analytics tool can be massively different depending on who you are as a company, what you already have it in place and, mostly, who your Vendor is and what their pricing strategies are. I encourage you to poke and dig for data to get a clear understanding of what the TCO is for each Vendor you are selecting.<\/p>\n

Here are elements or TCO that you\u00a0would have to consider:<\/p>\n

=<\/span><\/strong> Cost Per Page View <\/em>(since most ASP based vendors charge per page view)<\/p>\n

+<\/strong><\/span> Incremental costs beyond initial lump-some <\/em>(charge if you go over your allocated page views, are there any \u201cadvanced\u201d features – say RIA tracking or RSS or whatever in terms of extra modules that would cost more, what else could we buy from the vendor that we might need later\u00a0\u2013 for example Pay Per Click integration with Google \/ YSM or keyword bidding feature or whatever, better to know now)<\/p>\n

+<\/span><\/strong> Annual support costs after year one<\/em><\/p>\n

+<\/span><\/strong> Cost of professional services<\/em> (initial install and then post launch trouble shooting or customizations)<\/p>\n

+<\/span><\/strong> Any additional hardware you need at your end<\/em> (pc\u2019s, laptops, web servers, data storage drives etc, could vary by Vendor)<\/p>\n

+<\/span><\/strong> Cost of \u201cadministration\u201d<\/em> (manage vendor relationship \u2013 could be partial head count, someone to create all the reports and publish them, someone to coordinate between vendor and IT and marketers \u2013 all these could be one person, better to know now)<\/p>\n

+<\/span><\/strong> Cost of analysts needed to draw insights<\/em> (you can lump this with the one above if you want but it is important to be aware of the 10\/90 rule and realize that you can\u2019t just buy the tool, you also have to hire a relatively intelligent brain to interpret the data \u2013 there are vendors who have stated that their tools are so smart that you don\u2019t need Analysts, this would work in your favor but make sure you buy that)<\/p>\n

+<\/span><\/strong> Additional head count (partial or full) to maintain the tags, liaise with IT, update pages on the site etc<\/em><\/p>\n

Total that up across Vendors and make a informed choice. The only thing above that it not totally in \u201cunique to each Vendor\u201d camp are the Analysts you\u2019ll need, stress test that it is ok to break the 10\/90 rule because the tool inherently is so smart.<\/p>\n

Also notice that while some of the above are eliminated with a free tool other things above will still apply even in the case of a free vendor.<\/p>\n

5) What kind of Support do you offer? What is included for free and what would cost more? Is free 24×7? What are the limits? How far will you go to help me solve my technical problems and answer \u201csilly\u201d questions from my business users?<\/strong><\/p>\n

During Vendor pitches you\u2019ll hear\u00a0that\u00a0everything is free (and\u00a0some web\u00a0analytics Vendors do indeed offer loads and loads of absolutely free support as long as you stay with them). But often there are limits and caveats that are not explicit and you\u2019ll have to dig them out.<\/p>\n

Signing and contract and implementing the solution signifies the start of your problems and not the end of your data problems. It is critical that you understand exactly what you\u2019ll get and exactly how much it will cost to get what you need (say if a Vendor will only provide business hour support during week days then what would 24×7 cost, or if they will only answer questions about the tool and not why the tool is not working with your site then what would that cost \u2013 just to stress these are just suggestions to get your juices flowing, you\u2019ll have to make up your own unique questions).<\/p>\n

You will need support and professional services, understand what the Vendor will provide or what the Vendor\u2019s \u201cAuthorized Consultants\u201d will provide etc.<\/p>\n

6) What features do you have in your tool that allow me to segment the data?<\/strong><\/p>\n

Ian posted recently asking if we need Power or Simplicity (please massive segmentation<\/a>, in fact every tool should\u00a0allow for Trinity model<\/a> and multiple sources of data and how to get a complete picture and all that\u00a0nice stuff. Bottom-line your clickstream data, no matter what Vendor you use, will feel limiting after a while when you want real insights and you\u2019ll be forced to integrate it with other sources of data. Since the export route (#7 above) is so hard you\u2019ll have to bring that data in. This question goes to the heart of figuring out how easy it is to do that for each Vendor you are considering.<\/p>\n

Examples of data you might want to bring into your tool are:\u00a0meta-data from other sources in your company,\u00a0or you CRM data, or data from\u00a0you ad \/ search agency, or\u00a0data from \u00a0surveys that has the primary key in there (like cookie values), or a\/b \/ multivariate testing integration so you can measure conversion from the MVT tool but also analyze Site Overlay (Click Density) for each recipe, etc. You have to be able to import data efficiently (without needing humans if possible) and then use it for segmentation or reporting.<\/p>\n

Some Vendors can automatically pick up data from Google AdWords and do direct integration without you having\u00a0to do anything. Others will require you to do a daily download yourself and upload a text or csv file into your tool. Others still can\u2019t do anything. Figure out what the line in the sand is for the Vendor you are considering for the kinds of data you want to integrate.<\/p>\n

9) What are two new features \/ tools \/ analytical advancements \/ acquisitions is your company cooking up that would keep you ahead of your competition for the next three years?<\/strong><\/p>\n

At the core of this question you are trying to see if your Vendor is good at the today or if they are worried about the tomorrow and have plans to deal with tomorrow.<\/p>\n

It also gives you a sense at how much they know about their own position and that of their competitors (hence we are not asking what are two things you are doing that\u00a0are good, the framing is in context of competition). As our E-metrics 2006 DC summit experiment proved some Vendors are much much better at having a good reality check about themselves (read: Hello, My Name is Avinash. What is Unique About You?<\/a>).<\/p>\n

You want to be impressed by atleast one of the two things you hear being a complete surprise to you (which means they are ahead of you, always a good thing), and you want to get the feeling that your Vendor has a good sense for themselves and their competitors.<\/p>\n

If you ask this exact same question across a few Vendors they will talk about each other and the differing perspectives are a source of valuable insights for you.<\/p>\n

10) Why did the last two clients you lost cancel their contracts with you? Who are they using now? Would you permit us to call one of them?<\/strong><\/p>\n

I have to admit this is not one of my questions, a Vendor taught me this\u00a0question and it is truly fantastic. \"PatriotsYou want to be confident that you are making\u00a0the right choice and there is no better way than to learn why each Vendor recently lost someone\u2019s\u00a0business.<\/p>\n

Now the reality is that you will probably hear\u00a0marketing \/ sales speak\u00a0and not\u00a0reality but even hearing the marketing \/ sales speak can be of value.\u00a0Of a whole\u00a0bunch of vendors only two have ever\u00a0answered\u00a0this question\u00a0honestly (both were VP of Sales answering the question) and the interesting thing is that we are\u00a0doing\u00a0business with both today, even though in both cases they are not the most awesome\u00a0Vendor\u00a0technologically.<\/p>\n

Remember with any Vendor you are actually buying a relationship\u00a0and not\u00a0just the tool<\/u> and on the\u00a0long run (even near short\u00a0term)\u00a0people you can do business with will far outstrip\u00a0value with than working with the most advanced tool on earth whose people you\u00a0can\u2019t have a relationships with.<\/p>\n

Bonus question to ask:<\/u><\/p>\n

11) Do you read Occam\u2019s Razor by Avinash Kaushik and what do you think of his \u201cradical\u201d postings that tend to have a anti-vendor<\/em> slant?<\/strong><\/p>\n

This one is only partly fun.\u00a0 A\u00a0friend who is a respected industry authority and works at a vendor did say to me that he and the other Vendors got together and the sentiment that was expressed was \u201cwhat does Avinash have against us\u201d, <\/em>he actually used even stronger language.<\/p>\n

It was\u00a0a huge surprise to hear this because this is absolutely not true and some my industry best friends are Vendors. But since you might ask them some of the above questions it is fair for you to ask them question 11 so they can give you their point of view on my stated and unstated biases.<\/p>\n

Just for the record, I don\u2019t have anything against any Vendor (explicit or implicit). We are lucky to have a good crop of tools in our industry, some good, some better, some awesome. I am glad that we can pick and choose and I am glad that my friends at Omniture, ClickTracks, Google Analytics, WebTrends, CoreMetrics, Instadia, indexTools, HBX, Visual Sciences, Unica NetTracker, etc read this blog, subscribe to it and often make public comments and send me private emails. I thank you all and I promise you there is absolutely no anti-vendor slant on any vendor.<\/p>\n

What do you all think? Of #11 and all other questions? What am I missing in the list? Do you have your favorite question to ask any Vendor? Please share your feedback via comments.<\/p>\n

[Like this post? For more posts like this please click here<\/a>.]<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Jerome N, a reader from the UK, asked me to write a post on the questions that should be presented […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[11,2,6,14,8,9,12,10],"tags":[],"class_list":["post-176","post","type-post","status-publish","format-standard","hentry","category-advanced-analytics","category-analytics","category-marketing-tips","category-search-engine-marketing","category-voice-of-customer","category-web-analytics","category-web-insights","category-web-metrics"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/176","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/comments?post=176"}],"version-history":[{"count":4,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/176\/revisions"}],"predecessor-version":[{"id":10428,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/176\/revisions\/10428"}],"wp:attachment":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/media?parent=176"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/categories?post=176"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/tags?post=176"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}