Level 5 jbenson2 2,147 Posted January 18, 2015 Level 5 Share Posted January 18, 2015 Evernote sent me an email yesterday: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Subject: We would love to hear from you!From: The Evernote Team Hello! Thank you for using Evernote! We would appreciate your help to make Evernote better by answering a few quick questions in our short survey. Take the survey Thanks,The Evernote Team - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -So I tried to answer it today and received the following response: Thank you for your taking the time out of your day to take our survey, we greatly appreciate your desire to give us feedback. However, at this time the survey is now closed because of high response volume. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I'm glad the Marketing Team is looking for customer feedback. Perhaps CEO Libin is putting some pressure on the group. I wonder what type of questions they asked. Instead of a quick 20-hour snap-shot of responses, I wanted to tell them to there is a wealth of information in the comments in this Evernote forum. Link to comment
Level 5* CalS 5,280 Posted January 18, 2015 Level 5* Share Posted January 18, 2015 Wonder why a company would turn off a feedback stream that they had requested. Statistically large enough sample set and analysis done in 20 hours so as to know the answers? Not bad in 20 hours. Link to comment
Level 5* JMichaelTX 4,117 Posted January 18, 2015 Level 5* Share Posted January 18, 2015 Wonder why a company would turn off a feedback stream that they had requested. Statistically large enough sample set and analysis done in 20 hours so as to know the answers? Not bad in 20 hours. Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results. Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies. Link to comment
Level 5 jbenson2 2,147 Posted January 18, 2015 Author Level 5 Share Posted January 18, 2015 The reason I wondered what type of survey questions were asked was to get an idea on what Evernote considers important. If they are in the process of a corporate take over or IPO, they would be asking the usual mundane questions such as: education level?, income bracket?, how often is Evernote used?, etc. But if the survey was meant to uncover actual user unrest (as seen in the forum over the past year), the questions would be more specific: sync problems?, ease of use?, graphic interface?, scalability problems?, etc, Link to comment
Level 5* CalS 5,280 Posted January 18, 2015 Level 5* Share Posted January 18, 2015 Wonder why a company would turn off a feedback stream that they had requested. Statistically large enough sample set and analysis done in 20 hours so as to know the answers? Not bad in 20 hours. Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results. Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies. Kind of the point of my post.... Sarcasm is becoming a lost art I guess. Link to comment
lykoz 147 Posted January 20, 2015 Share Posted January 20, 2015 Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results. Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies. Completely not true. If you had any experience in research, you would know that the type of research being done, and what the researchers are trying to achieve plays a much more important role, than just numbers in bulk. We did not see the survey, so we don't even know if it was qualitative or quantitative in nature.. There are also practical restrictions... Research information takes time and effort to make sense off.. As more information comes in there is diminishing returns as you start saturating the representative population while costs increase in procuring the data. Most people work in a budget, with regard to time and resources. They may have gotten close to a million people responding as a hypothetical situation... Would you really need more numbers if that was the case?Again it depends on the goals of researchers. And the nature of the survey, which you did not take into acount or consider. You cant make generalised statements. Link to comment
Level 5 jbenson2 2,147 Posted January 20, 2015 Author Level 5 Share Posted January 20, 2015 My generalized statement and opinion is that someone screwed up the survey, yanked it down the next day, then pasted in the apology. If thousands of users responded, then I am confident someone else in this forum would have received it as well.If just a hundred users responded, then I question the validity of the survey results. Just my opinion and Occam's razor. Link to comment
ScottLougheed 1,316 Posted January 20, 2015 Share Posted January 20, 2015 Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results. Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies. Completely not true. If you had any experience in research, you would know that the type of research being done, and what the researchers are trying to achieve plays a much more important role, than just numbers in bulk. We did not see the survey, so we don't even know if it was qualitative or quantitative in nature.. There are also practical restrictions... Research information takes time and effort to make sense off.. As more information comes in there is diminishing returns as you start saturating the representative population while costs increase in procuring the data. Most people work in a budget, with regard to time and resources. They may have gotten close to a million people responding as a hypothetical situation... Would you really need more numbers if that was the case?Again it depends on the goals of researchers. And the nature of the survey, which you did not take into acount or consider. You cant make generalised statements. Indeed, not only does a larger sample not necessarily imply more reliable results, it can actually result in biased results and increases the potential of making a type 1 error. While you want a sufficient sample size, the reliability of the inferences being made does not increase infinitely and in a linear fashion with sample size. There are some generic formula for determining optimum sample size, and many statistical analyses also have formula that can assist in determining what the optimum sample size is for that particular analysis given a particular model. Nevertheless, closing the survey after what appears to be a very short period of time is odd. It is either a bad decision, or they are trying to save face in light of some other reason for closing the survey such as:1) It got flooded with bad responses from a bot or from sort of other malicious source2) They reached their limit on responses available to them given the survey software license they purchased (e.g., many survey services offer different tiers of service that allow only a certain number of responses)3) There was some other issue with the survey itself. We can really only speculate, but hey, at least they're trying to get some more insight from their customers... Link to comment
Level 5* JMichaelTX 4,117 Posted January 20, 2015 Level 5* Share Posted January 20, 2015 This is getting a bit off-topic, but I'll reassert my statement that (all other factors being the same), it is generally accepted in the field of statistics that a larger sample size usually provides better correlations, better results. Clearly there can be a cost with obtaining more samples, and I did NOT argue for an unlimited number of samples, nor did I say that the reliability increases linearly with sample size. My only point was that stopping a survey after just a few days seems premature when the cost for keeping it open for a longer period would most likely have not added much cost. Whether or not Evernote has collected sufficient sample we have no way of knowing. Here are two references: 1. Statsoft.com Why Significance of a Relation between Variables Depends on the Size of the SampleIf there are very few observations, then there are also respectively few possible combinations of the values of the variables and, thus, the probability of obtaining by chance a combination of those values indicative of a strong relation is relatively high.. . .The smaller the sample size in each experiment, the more likely it is that we will obtain such erroneous results, . . . 2. http://en.wikipedia.org/wiki/Statistical_power The sample size determines the amount of sampling error inherent in a test result. Other things being equal, effects are harder to detect in smaller samples. Increasing sample size is often the easiest way to boost the statistical power of a test. Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results. Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies. Completely not true. If you had any experience in research, you would know that the type of research being done, and what the researchers are trying to achieve plays a much more important role, than just numbers in bulk. Indeed, not only does a larger sample not necessarily imply more reliable results, it can actually result in biased results and increases the potential of making a type 1 error. While you want a sufficient sample size, the reliability of the inferences being made does not increase infinitely and in a linear fashion with sample size. Link to comment
gbarry 2,658 Posted January 20, 2015 Share Posted January 20, 2015 All--we received the statistically significant number we needed from the survey recipients, and the survey was closed for analysis. Link to comment
lykoz 147 Posted January 20, 2015 Share Posted January 20, 2015 Firstly I don't think any of this is off topic... The thread is at least partly based on the relevance of why a survey was apparently "pulled early". The only other thing to discuss is what the questions were... Which so far we don't know... In fact we know nothing about the survey... How they pulled the data, its relevance or what they wanted to achieve. so making definitive statements like: "Anyone who really knows about statistics, knows that the greater the sample set the more reliable the results.Yes, you do have to cut off a survey after some reasonable deadline, but NOT after a number of replies."Is simply just wrong... JMichael you are making big assumptions on the most basic level, that the research (as an example) was quantitative in nature... Among other things...In a hypothetical Qualitative study, information could be completely saturated by as little as 30 participants... (Just one example of your flawed reasoning).Quantitative studies similarly have more to consider than raw sample size... (As mentioned Earlier.) As longhead stated there are many considerations... And a basic wikipedia article is hardly a cumulative view, not to mention that you quickly googled "statistical power"... (Also I didn't know that wikipedia was now used as a reliable and exhaustive reference on such a large subject matter) Research is about more than just raw statistical power... Having done research assignments both qualitative and quantitative myself.. I am the first to say I am a newbie in the subject. Sometimes I think the more you learn about anything the less we make generalizations and assumptions about anything. Ever heard the phrase: The more I learn, the less I know? I think that applies here. Often people who just dont understand a subject, just simply think they know 'more' than ingrained professionals. Also its incredibly easy for Evernote with about 100 millions users... To quickly reach ideal sample sizes... Even in only a few hours. Furthermore 'extra costs' has much less to do with 'how long' the survey is up... and more to do with how they extract and interpret the data once they have it. JMichael... I get constructive criticism... I get fielding your complaints.. But I get a feeling that all your posts are always negative towards Evernote these days, and how people should move on and use other products, and that you will be leaving if "this carries on"... I wonder why you constantly engage in these forums and continue using the product if you are so dissatisfied in everything the company and its employees do. I have started sticking around in these forums because although I field my views and what can improve in my personal layman's opinion... In totality I believe in what I am being offered. I believe the guys here are miles ahead of the competition. I am constantly surprised at how the employees in these forums always engage us.. How they are quick to help and solve the issues I have... And although I will complain when I need to.. I try be as constructive as possible. I truly and honestly believe Evernote is on the right path.. Today more than ever... And I see a bright future. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.