mytest
[poll id=”2″ type=”result”]
COMMENT: Clearly, the majority (62%) felt that Japan was an easy or very easy place to live. Well and good.
Next poll: Is Japan an easy place to WORK? Vote early, vote often! Debito
mytest
[poll id=”2″ type=”result”]
COMMENT: Clearly, the majority (62%) felt that Japan was an easy or very easy place to live. Well and good.
Next poll: Is Japan an easy place to WORK? Vote early, vote often! Debito
6 comments on “Results of our first poll: In your opinion, is Japan an easy place to live?”
Well, whether it’s an easy place to work is a tougher question because:
1) Working conditions for Japanese and [most] foreigners are quite different, and
2) There are many kinds of work environments out there.
And is the question about work load? Is it about office politics? Does it include the commuting? How about dealing with immigration?
–Quite. That’s why I gave this question second. It’s a much more difficult issue, with fewer things under the individual’s control. I’ll let people take whatever they want into account and arrive at their own conclusions.
Debito, I find the question hard to answer. Is Japan an easy place to WORK, compared to,,, what? Compared to where you came from, compared to the world average, compared to the OECD average, or compared to an ideal place?
If people answering the question have different yardsticks, the result of your survey may be hard to interpret. If I may suggest, I hope you change the question more specific next time.
–The same possible comparisons could have applied to the “…live?” question. Don’t overthink it (and don’t fear a possibly negative outcome). It’s a simple but important question that must be asked. People can qualify their yardsticks in the Comments section if they like. Or they can just choose the “can’t say either way” option, of course.
I participated in the poll. However, I so wish that I could qualify my response.
–Go right ahead. That’s why we have a Comments section. 🙂
I chose “on balance Japan is a difficult place to work.” My particular situation is good (I coordinate youth exchange programs at a Japanese company), but I know that in many ways I am lucky. My main complaint against Japanese corporate enviroments is that western NJ (I’m not even going to try to speak for NJ Asians, I know their situation is far different from mine) are usually put into special categories and kept on short-term contracts. It can be extremely difficult to become a “normal” employee, and that’s before we even start to talk about differences in culture or communication style.
shouldnt we only vote once, otherwise it’ll be skewed?
I put that it was an easy place to work (on balance)…but this is because I work an NGO where I dont get paid↓↓and later I will work as an expat and have an expat package
…if I tried to get a “legit” job maybe that’s a different story
–“Vote early, vote often”… is an old joke, FYI.
“Don’t overthink it (and don’t fear a possibly negative outcome). It’s a simple but important question that must be asked.”
HO makes a valid point, Debito. You should not dismiss his comment without acknowledging the problems of internet polls. If done properly, yes, opinion surveys can be a meaningful measure of public opinion. If done improperly (as is often the case with internet polls), the results are either (1) methodologically questionable or worse (2) completely meaningless.
The top five reasons why internet polls *usually* fall into the second (i.e., “meaningless”) category include (in no order of importance):
(1) Selection bias: your poll is tapping into a self-selected group of opt-in respondents on the internet. Already the poll becomes suspect because it is *not* a random selection of respondents across Japan, but rather the result of opinions from those who already exhibit a high interest in the subject matter—either for or against—and who actively read your website on a daily basis. Consequently, the poll overvalues those who are internet-literate, aware of your website, and had the time and inclination to vote.
(2) Poll stacking: the original problem (now largely solved) was that the same IP address would vote multiple times in the same survey, skewing the results. Internet poll software *partially* corrected the problem by preventing the same IP addresses from voting more than once. What they have *not* solved is the problem of “sockpuppets” and “meatpuppets”. A “sockpuppet” is the same person voting the same way from different computer IP addresses in order to distort the results. A “meatpuppet” is someone who votes at the request of a like-minded friend after having his or her attention called to the internet poll. Both undermine the scientific validity of internet polls.
(3) Sample size: there are roughly 2 million non-Japanese living in Japan. Your poll had 147 respondents. You are a smart guy, Debito – even if we assume that poll stacking and selection bias were not at issue here, I doubt you (or anyone else) honestly believe that 147 respondents is representative of what the other 1.999 million people in Japan think.
What would a scientifically valid sample size be? Creative Research Systems offers a free “sample size calculator” at their website. If we assume a population of 2 million foreigners in Japan, with a 95% confidence level, and a confidence interval of .5, a *meaningful* poll would need to have surveyed a random selection of over 37,000 individual respondents. (See: http://www.surveysystem.com/sscalc.htm for further details.)
(4) Question formation and leading: HO et al. are right. How one poses the question determines what kind of answers one gets. Vague questions get vague answers. Clear questions get clear answers. Misleading questions (because they’re vague), get misleading answers (because the answers are vague).
(5) Survey duration: for an internet poll, yours seemed to close very fast. Had you kept it alive for a longer duration you might have been able to improve on problems (1) and (3), but you would have increased the risks to problem (2). That said, I am not sure what you could have done to improve on these serious methodological problems short of what professional pollsters already do – use telephone calling banks to contact a random selection of respondents across the country over a limited period of time. Provided the sample size is meaningful, one can avoid all five methodological problems at the same time.
In short, I am not suggesting that it is wrong to care about what “the people” think. I’m suggesting the opposite, actually: I applaud that you are taking an interest in opinion polls. Just be mindful of these results – in either direction – when writing newspaper articles and (especially) academic articles. In fact, if I were you, I would not use internet polls at all. They are usually taken with a grain of salt…and rightfully so.
–Of course. I was never going to make this a poll I would cite with any scientific certainty. For all the reasons you iterate above. Thank you very much for a thoughtful post. Debito
PS: I’m planning to leave polls open for ten days at a time from now on. Sorry for the short duration of the last poll. I’m still getting used to the polling settings and functions.