Excerpts from National Household Survey: Data quality, Statistics Canada
The National Household Survey (NHS) contains all of the questions that Statistics Canada contemplated for inclusion in a 2011 Census long-form. The NHS is therefore identical in content to what would have been collected in a 2011 Census long-form.
Data quality
Response rates
In its initial planning, Statistics Canada assumed a response rate for a mandatory 2011 Census long-form of 94%, identical to that achieved for the 2006 Census.
Statistics Canada has assumed a response rate of 50% for the voluntary National Household Survey.
...
Comparability of data over time
Any significant change in the methods of a survey can affect the comparability of data over time. There is a real risk that this will be the case for the National Household Survey. There will always and inevitably be an element of uncertainty as to whether and to what extent a change in a variable reflects real change or an artefact arising from the change in methodology from the mandatory long-form census to the voluntary National Household Survey.
Change in survey processes, however, is inevitable and has precedents even in the Census of Population. In 1971, for example, two major changes were introduced: selfenumeration in the place of interviewer enumeration and asking some questions of a subsample (then 1/3) of the population rather than the entire population (there had been some sampling in previous censuses, beginning in 1941, on a much more limited scale).
Conclusion
We have never previously conducted a survey on the scale of the voluntary National Household Survey, nor are we aware of any other country that has. The new methodology has been introduced relatively rapidly with limited testing. The effectiveness of our mitigation strategies to offset non-response bias and other quality limiting effects is largely unknown. For these reasons, it is difficult to anticipate the quality level of the final outcome.
The significance of any quality shortcomings depends, to some extent, on the intended use of the data. Given that, and our mitigation strategies, we are confident that the National Household Survey will produce usable and useful data that will meet the needs of many users. It will not, however, provide a level of quality that would have been achieved through a mandatory long-form census.