Caitlan - yes please do this review that you are planning - it's a very important issue, and the key is to assess what extent public policy is misled by the omissions and biases. There has been a lot statistical work on missing data that you'll have to search for, though it might be easier if you limit yourself to opinion polls. Cathie Marsh did critical work on quota versus probability sampling for opinion polls in the 1980s, which you will find useful for its clear perspectives but there must be much more uptodate work too.
You might find useful a review of how data become missing in Radical Statistics 71: 45-60. http://www.radstats.org.uk/no071/article5.htm
Although censuses gain much higher response rates than surveys and opinion polls, the UK censuses are special in that their post-enumeration Coverage Survey does estimate the types of people missed according to some key characteristics including age, sex, ethnic group and tenure, for each local authority of Britain. The analysts then have no option but to assume that the remaining characteristics of those who were missed are like those from the same groups who were enumerated. But at least the outputs tell you the difference in response rates for each characteristic. For England and Wales the different response rates are here, http://www.ons.gov.uk/ons/guide-method/census/2011/census-data/2011-census-user-guide/quality-and-methods/quality/quality-measures/response-and-imputation-rates/index.html. Search for Scotland if that's your focus.
Good luck Ludi
------------------------------
Date: Fri, 24 Jan 2014 13:00:47 +0000
From: "BYRNE D.S." <[log in to unmask]>
Subject: Re: Request for help on information re. opinion polls and low-income participants
Although it doesn't deal with sample surveys as such, a lot of relevant issues are discussed in:
Nick Emmel and Kahryn Hughes 'Small N Access cases to refine theories of social exclusion and access to socially excluded individuals and groups' which is Chapter 18 of the Sage Handbook of Case Based Methods edited by Byrne and Ragin.
Nick Emmel's book on 'Sampling and Choosing Cases in Qualitative Research' is helpful - see:
http://www.uk.sagepub.com/books/Book235588?subject=L00&classification=%22Academic%20Books%22&sortBy=defaultPubDate%20desc&fs=1
The key issue for me is that low income marginalized people (and the very affluent as well) are under-represented in all sample surveys and the classic approach to dealing with this is weighting, usually against census data, which never adequately deals with the fundamental problem: the people we contact are different and probably very different from those we do contact. At least government surveys weight but I am not at sure that private polls do so??? Anybody know.
This is a big issue. Response rates, e.g. in the survey which is used to construct an account of the impacts of taxes and benefits on household incomes, are often low - in that case as I recall only in the 40-50% range for the sample drawn from a working universe frame, and god knows what in relation to the real universe.
David Byrne
________________________________________
From: email list for Radical Statistics [[log in to unmask]] on behalf of Kait Laughlin [[log in to unmask]]
Sent: 24 January 2014 11:57
To: [log in to unmask]
Subject: Request for help on information re. opinion polls and low-income participants
Dear Radstats Folk,
I'm working on issues around the inclusion/exclusion of people living in poverty in opinion polls and public attitude-measurement surveys, and could do with a bit of help please.
I'm familiar with the main issues around the unrepresentative nature of telephone and internet based samples, and have my own thoughts around those as well as on issues relating to doorstep polls, which I want to draw together in some form - but I cannot find anything that gives me the level of detail that need to do a proper analysis.
- So... does anyone know if an in-depth analysis of methodological issues around low-income participation (preferably in the UK) is available anywhere? I've found a lot of general stuff, but I'm thinking beyond the usual structured sampling and weighting formulas - more along the lines of how representative are those who do participate from low-income groups (particularly those on benefits) of those who don't - and how robust any assumptions on that have been found to be and what issues are identified as significant/relevant in making that judgement.? (I think there may be a gap, but want to make sure I'm not missing anything I should know about.)
- Also, I've been trying to find a source that quantifies what percentage of polls, say in a given period, are telephone, internet or doorstep based.
- And also whether there is a summary table or something that lists pollsters and what methods they typically use.
I've done some preliminary research that says this is worth pursuing - particularly relevant at this time in Scotland in the lead up to the Independence Referendum - but not being a statistician it is hard to be absolutely sure that I'm looking in all the right places (and not just the obvious), and at times it's also hard to evaluate the reliability, etc. of some sources. It would be helpful therefore (also in terms of saving time and not re-inventing the whee)l, if I could tap into member's knowledge and expertise on this?
Also, this is my first post here, so I want to take the opportunity to say - I love this list! It's one of the few where there are people who are not afraid to be openly angry and rightfully outraged at the manipulation and ideologically-based distortions of the public discourse and cultural narratives around wealth and poverty and the 'othering' of those on benefits. It's no exaggeration to say that there have been occasions where some of the posts have made my day, and I have certainly followed up several relevant to my research interests, so thanks very much!
Best wishes,
Kait Laughlin
'At any given moment, there is a sort of all pervading orthodoxy, a general tacit agreement not to discuss large and uncomfortable facts.' -- George Orwell
'I am not part of that agreement.' -- Kait Laughlin.
Kait Laughlin,
PhD Researcher on Poverty & Human Rights in Scotland, Associate Lecturer in Sociology, Chair, Modern Universities Research Group, Post-Graduate Committee, School of Social Sciences,, University of the West of Scotland, Hamilton Campus,
ML3 0JB.
01698-283100, ext. 8550
[log in to unmask]
Please consider the environment and think before you print
***************************************************************************************************************
University of the West of Scotland aims to have a transformational influence on the economic, social and cultural development of the West of Scotland and beyond by providing relevant, high quality, inclusive higher education and innovative and useful research.
Visit www.uws.ac.uk for more details
University of the West of Scotland is a registered Scottish charity. Charity number SC002520.
***************************************************************************************************************
Legal disclaimer
--------------------------
The information transmitted is the property of the University of the West of Scotland and is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Statements and opinions expressed in this e-mail may not represent those of the company. Any review, retransmission, dissemination and other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender immediately and delete the material from any computer.
--------------------------
******************************************************
Please note that if you press the 'Reply' button your message will go only to the sender of this message.
If you want to reply to the whole list, use your mailer's 'Reply-to-All' button to send your message automatically to [log in to unmask]
Disclaimer: The messages sent to this list are the views of the sender and cannot be assumed to be representative of the range of views held by subscribers to the Radical Statistics Group. To find out more about Radical Statistics and its aims and activities and read current and past issues of our newsletter you are invited to visit our web site www.radstats.org.uk.
*******************************************************
******************************************************
Please note that if you press the 'Reply' button your message will go only to the sender of this message.
If you want to reply to the whole list, use your mailer's 'Reply-to-All' button to send your message automatically to [log in to unmask]
Disclaimer: The messages sent to this list are the views of the sender and cannot be assumed to be representative of the range of views held by subscribers to the Radical Statistics Group. To find out more about Radical Statistics and its aims and activities and read current and past issues of our newsletter you are invited to visit our web site www.radstats.org.uk.
*******************************************************
------------------------------
End of RADSTATS Digest - 23 Jan 2014 to 24 Jan 2014 (#2014-11)
**************************************************************
******************************************************
Please note that if you press the 'Reply' button your
message will go only to the sender of this message.
If you want to reply to the whole list, use your mailer's
'Reply-to-All' button to send your message automatically
to [log in to unmask]
Disclaimer: The messages sent to this list are the views of the sender and cannot be assumed to be representative of the range of views held by subscribers to the Radical Statistics Group. To find out more about Radical Statistics and its aims and activities and read current and past issues of our newsletter you are invited to visit our web site www.radstats.org.uk.
*******************************************************
|