Comments on: Got Surveys? Recommendations from the Trenches http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html Pluralitas non est ponenda sine neccesitate. Tue, 16 Mar 2010 15:01:56 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Scott Gilkesonhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-490403 Scott Gilkeson Sun, 06 Dec 2009 18:35:43 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-490403 Interesting and long-lived discussion. But I haven't seen anyone talk about what I perceive as the biggest problem with surveys--that they are from a highly self-selected sample. I know this is true about all sorts of surveys, not just online ones, but it seems like respondents are likely to be either fans, or have an ax to grind, or maybe be people with a lot of time. I like the idea of targeting regular visitors - that might help somewhat. I don't think any sort of incentive would help--may make things worse. I'm especially interested in non-commercial sites, where typical (target) users are looking for information. Pop-up surveys tend to interfere most with visitors who are on-task, and those are the ones whose feedback is most desirable.I get so fed up with ForeSee Results surveys popping up when I use government web sites that I've taken to dismissing them all in protest. Any thoughts on ways to get to the core, task-centered, busy visitors? Interesting and long-lived discussion. But I haven't seen anyone talk about what I perceive as the biggest problem with surveys–that they are from a highly self-selected sample. I know this is true about all sorts of surveys, not just online ones, but it seems like respondents are likely to be either fans, or have an ax to grind, or maybe be people with a lot of time. I like the idea of targeting regular visitors – that might help somewhat. I don't think any sort of incentive would help–may make things worse. I'm especially interested in non-commercial sites, where typical (target) users are looking for information. Pop-up surveys tend to interfere most with visitors who are on-task, and those are the ones whose feedback is most desirable.

I get so fed up with ForeSee Results surveys popping up when I use government web sites that I've taken to dismissing them all in protest. Any thoughts on ways to get to the core, task-centered, busy visitors?

]]> By: E. David Zotterhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-488500 E. David Zotter Sun, 23 Aug 2009 15:48:24 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-488500 I ended up using RelevantView for several large projects with great success.It handles the merging of both survey and web analytics automatically. I ended up using RelevantView for several large projects with great success.

It handles the merging of both survey and web analytics automatically.

]]> By: Avinash Kaushikhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-488487 Avinash Kaushik Sat, 22 Aug 2009 22:05:21 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-488487 <font color=blue><b>Marianne: </b></font>The problem with integrating survey data with analytics data (from any tool) is that your visitor session happens and causes two different sources of data. Your survey data (with your survey provider) and your clickstream data (with your WA tool).The challenge is to merge these two things together.Here are your choices:<strong>1.</strong> Data must come into the WA tool from the survey tool <strong>2.</strong> The reverse of above. <strong>3.</strong> Come out of both tools and into a new place (spreadsheet, database etc).All of the above allow you to analyze the merged dataset of clickstream and survey data, IF you have a primary key.The most common method to have a primary key is to<strong>1.</strong> Pass the unique survey id into the analytics (Google Analytics or whatever) <strong>2.</strong> Pass the session or persistent cookie id into your survey tool.All that is context.With the first set of choices you can do #2 and #3 with GA.For the primary key you can do #1 or #2 with GA.In either scenario there is some technical work you (or your hired consultant) have to do to make this work, with Google Analytics or any other tool.-Avinash. Marianne: The problem with integrating survey data with analytics data (from any tool) is that your visitor session happens and causes two different sources of data. Your survey data (with your survey provider) and your clickstream data (with your WA tool).

The challenge is to merge these two things together.

Here are your choices:

1. Data must come into the WA tool from the survey tool
2. The reverse of above.
3. Come out of both tools and into a new place (spreadsheet, database etc).

All of the above allow you to analyze the merged dataset of clickstream and survey data, IF you have a primary key.

The most common method to have a primary key is to

1. Pass the unique survey id into the analytics (Google Analytics or whatever)
2. Pass the session or persistent cookie id into your survey tool.

All that is context.

With the first set of choices you can do #2 and #3 with GA.

For the primary key you can do #1 or #2 with GA.

In either scenario there is some technical work you (or your hired consultant) have to do to make this work, with Google Analytics or any other tool.

-Avinash.

]]> By: Marianne Conklinhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-488486 Marianne Conklin Sat, 22 Aug 2009 20:11:36 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-488486 Hi - WA and Market Research are a great mix to completely understand a customer's experience. My question: how do you merge collected survey data with analytics data? Can this be done with GA? Hi – WA and Market Research are a great mix to completely understand a customer's experience. My question: how do you merge collected survey data with analytics data? Can this be done with GA?

]]> By: Holgerhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-486900 Holger Tue, 02 Jun 2009 15:20:36 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-486900 Hi Avinash,I ran into this older article and found it interesting and I agree with a few points, and disagree with others.However, what made me wondering the most was this: > Usually around 1,300 survey responses > are statistically significant.I was wondering how you calculated those 1300 responses and on which basis this is all calculated? Because statistical significance not only depends on the pure number of respondents. Usually you apply it to a difference in results (e.g. among subgroups) and see whether the differences among e.g. male and female respondents are significant (high chance that the difference has a reason behind it) or we can not tell whether there is a reason behind it or it was just and deviation due to chances...So I am wondering on what you are basing the affirmation that 1300 responses are statistically significant and especially for what? Hi Avinash,

I ran into this older article and found it interesting and I agree with a few points, and disagree with others.

However, what made me wondering the most was this:
> Usually around 1,300 survey responses
> are statistically significant.

I was wondering how you calculated those 1300 responses and on which basis this is all calculated? Because statistical significance not only depends on the pure number of respondents. Usually you apply it to a difference in results (e.g. among subgroups) and see whether the differences among e.g. male and female respondents are significant (high chance that the difference has a reason behind it) or we can not tell whether there is a reason behind it or it was just and deviation due to chances…

So I am wondering on what you are basing the affirmation that 1300 responses are statistically significant and especially for what?

]]> By: Damianhttp://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html/comment-page-1#comment-482746 Damian Fri, 13 Feb 2009 22:51:47 +0000 http://www.kaushik.net/avinash/2006/05/got-surveys-recommendations-from-the-trenches.html#comment-482746 Hello Avinash and everybody!Currently we are running a feedback survey at http://www.ilovephotos.com/help/feedback/survey/ that is powered through Google Docs.Using Google Analytics I can track how many people visit the page, how long they stay, etc. Using Google Docs I see how many people have actually finished the survey.However...I have no idea how to measure they people that start the survey but don't finish it. Any ideas? How can I track the number of people who start the survey and then finish it?I know that the first recommendation would it be to create a "thank you" page but Google Docs does that already. (If you take the survey, you would see that once you complete it.) How do I add a tracking tool to the message from Google Docs?Thank you everybody!Cheers,Damian Hello Avinash and everybody!

Currently we are running a feedback survey at http://www.ilovephotos.com/help/feedback/survey/ that is powered through Google Docs.

Using Google Analytics I can track how many people visit the page, how long they stay, etc. Using Google Docs I see how many people have actually finished the survey.

However…I have no idea how to measure they people that start the survey but don't finish it. Any ideas? How can I track the number of people who start the survey and then finish it?

I know that the first recommendation would it be to create a "thank you" page but Google Docs does that already. (If you take the survey, you would see that once you complete it.) How do I add a tracking tool to the message from Google Docs?

Thank you everybody!

Cheers,

Damian

]]>