Stats NZ has a new website.

For new releases go to

www.stats.govt.nz

As we transition to our new site, you'll still find some Stats NZ information here on this archive site.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Post-enumeration Survey: 2013
Embargoed until 10:45am  –  27 May 2014
Data quality

About the PES

The 2013 Post-enumeration Survey (PES) was the fourth to be undertaken in New Zealand. The first was in 1996. The 2013 PES, like its predecessors, was a sample survey of individuals in private dwellings. The survey was carried out between 9 April and 10 May 2013 and included just over 14,900 dwellings and nearly 32,700 individuals.

The main objective of the 2013 PES was to measure the level of national coverage (undercount and overcount) in the 2013 Census. Counting more than 4 million people throughout New Zealand is a major undertaking, and inevitably some people are missed or counted more than once by the census.

Many countries conduct surveys to estimate the number of people who were not enumerated or were overcounted by their censuses. Experience from previous PESs in New Zealand, access to new information and international developments, and advances in methodology help us improve the accuracy and robustness of these surveys.

A post-enumeration survey is one way of examining the effectiveness of the census data collection and any implications for quality of census output. Results from the 2006 PES contributed to 2013 Census development, and similarly the 2013 PES will help us develop collection strategies for future censuses.

The environment for collecting data is becoming more difficult. The New Zealand population has become more diverse in terms of ethnicity, living arrangements, and lifestyles, and people are becoming more difficult to contact. We have continued to develop and enhance initiatives to encourage people to participate in census. These included public communication campaigns via advertising and community liaison, and developing the option (first provided in 2006) for people to complete their forms online. Thirty-four percent of people completed their forms online in 2013. Results from the 2013 PES provide assurance that through initiatives such as these, the census can be carried out effectively and efficiently in an increasingly complex environment.

A PES provides the census coverage information required to calculate a reliable population base for deriving post-censal population estimates and projections. The 2013 Census usually resident population counts will be used with the estimated net undercount (undercount less overcount), as part of the adjustments necessary to derive the estimated resident population of New Zealand at 30 June 2013. This forms the base population for estimating changes in the resident population until the next census and for the 2013-based population projections.

This first release provides an overview of the results and the methods used by the survey. A full report on the results and supporting information on the 2013 PES will be released in August 2014.

Objectives

The Post-enumeration Survey (PES) measures the level of coverage (overcount and undercount) for both people and private dwellings in the census. Coverage measures are the key performance indicators for the 2013 Census of Population and Dwellings.

The PES produces estimates of net undercount (ie, gross undercount minus gross overcount) for the following key population groups:

  • total population
  • sex (male, female)
  • age (0–14 years, 15–29 years, 30–44 years, 45–64 years, and 65+ years)
  • ethnicity (European, Māori, Pacific, Asian and 'other', which combines both remaining two high level groupings of ethnicities: Middle Eastern/Latin American/African (=MELAA) and Other)
  • geographical area (in two groups: (1) northern North Island (ie, Northland, Auckland, Waikato, Bay of Plenty regional council areas), southern North Island, South Island, and (2) Auckland region, Wellington region, rest of North Island, Canterbury region, rest of South Island).

The survey period was chosen to avoid census collectors and PES interviewers being in the field at the same time, yet remaining close enough to census date (5 March 2013) to assist accurate respondent recall.

Source of data

The 2013 PES was based on a stratified, multi-stage sample of approximately 15,000 permanent private dwellings. The survey population consisted of New Zealand residents, either usually resident in a New Zealand private dwelling, or staying at one during the survey period. Overseas visitors who were resident in a private dwelling at the time of the PES and were somewhere in New Zealand on census night were also included. Overseas visitors were surveyed to investigate the consistency of identification of person type (resident or overseas visitor) between the census and the PES.

For practical reasons, non-private and other private dwellings were excluded from the survey, as were dwellings in remote areas.

Sample design

The sampling process was complex. The geographical framework of New Zealand consists of 46,637 meshblocks. A meshblock in an urban area is usually a residential area containing about 40 dwellings surrounded by streets. In rural areas a meshblock covers a much wider area because dwellings are sparsely spread.

For sampling purposes, these meshblocks are aggregated into 21,813 primary sampling units (PSUs). To improve the sampling efficiency, we put these units into 127 groups (or strata) based on region, urban/rural mix, ethnic population, and other socioeconomic variables (income, employment status, population aged 65 years and over).

We randomly selected PSUs across the 127 strata, with oversampling in strata containing people who were deemed more difficult to count in census. The 2013 PES sample was made up of 1,299 PSUs, with a total of almost 15,000 dwellings (or about 0.9 percent of total permanent private dwellings in New Zealand).

Survey form and interview

The 2013 PES visited almost 15,000 households in areas throughout New Zealand, from 9 April to 10 May 2013. Our field interviewers asked each respondent demographic questions (date of birth or age, ethnicity, sex) and questions about their usual address and their address on census night. Interviewing usually took between 10 and 15 minutes per household. Respondent information was recorded on a paper questionnaire.

This information is used to locate a census record for each respondent to determine whether or not they were counted (or double-counted) in the census. The PES does not evaluate the quality of census responses. Under the Statistics Act 1975, respondents must provide the information requested for the PES.

Methodology

We matched respondents in the PES to census forms by comparing the information given by PES respondents with the information given on census forms. We estimate measures of undercount and overcount from the results obtained in the matching procedure.

Two major changes from the 2006 PES are the increased sample size (15,000 dwellings, up from 11,000) and the introduction of automatic matching of records to supplement clerical matching between PES and census. The net undercount estimates produced from the PES are subject to both sampling and non-sampling errors. The enhanced sample size aimed to reduce the sampling error of coverage estimates. Introducing automatic matching significantly improved the quality of matching.

In 2013, a new automatic matching procedure used data integration software to electronically link PES records to census records. Previous matching was done manually only. This reduced dependence on the quality of address information and meant we could cast our net wider in searching for PES respondents in census.

Matching was done to determine if a PES respondent was counted in the census at the address at which they were interviewed in the PES, or at each address where a census form may have been completed for them. If an address given on the census form was different from the PES address, we performed a search to find the address.

The estimation methodology for the 2013 PES includes several improvements over that used in 2006, 2001, and 1996. The main change was to include a dwelling weighting step to adjust for dwelling-level non-response in the PES. This enables us to take into account the extra information we have about census attributes of dwellings found in the census.

Estimation and adjustment

Each person in the sample has a selection weight that reflects their probability of selection into the sample. We adjust these selection weights to compensate for non-response in the PES and to reduce bias that may result from sample imbalance. If the PES sample does not accurately reflect the characteristics of all New Zealanders, and some subgroups of people are more likely to be undercounted than others, then undercount estimates will be biased unless some adjustment is made. We combine the information collected from the PES sample using these adjusted weights to provide an estimated undercount for the whole population.

Substitute records

Substitute individual records are individual census records we create if there is sufficient evidence that a person exists but we have no corresponding individual form. 

Understanding substitution and imputation in the 2013 Census has more information about substitutes and imputation of key variables.

Sampling error

Net undercount estimates produced from the PES are subject to sampling error. Since only a sample of dwellings was included in the PES, our estimates may differ from figures that would be obtained if all dwellings had been included. The sampling error indicates the extent to which an estimate from the PES might have varied by chance because only a sample of dwellings was included.

Because of the limited sample size, we can only provide reliable estimates of undercount for broad groups of the population. Some estimates have high sampling errors, so be cautious when using these. In general, the sampling errors associated with subnational estimates (eg breakdowns by area or ethnic group) are larger than for the national estimate. When comparing undercount estimates, keep both sampling error and non-sampling error in mind.

Non-sampling error

Non-sampling errors in surveys typically arise from a number of sources including non-response, imperfections in reporting by respondents, data collection, and data processing.

The impact of these non-sampling errors can be reduced by careful form design, training and supervision of interviewers, and efficient operating procedures. The matching of PES forms to census is an additional source of non-sampling error encountered by this survey.

The requirement for independence between census and the PES – where people are randomly missed by census or the PES – may not always be met and can therefore contribute to non-sampling error. Lack of independence can occur when a person missed in the census is more likely to be missed in the PES, for example if they are deliberately avoiding contact with government agencies. As a result, the PES estimates of undercoverage are likely to be lower than the true undercount.

Matching impact study

A matching impact study (MIS) was done on a subsample of 2,000 PES dwellings. This study evaluated the new method of predominantly automated electronic matching against the manual-only method that had been used in 2006 to measure the effect of the change in method.

The MIS verified that the 2013 PES results improved the quality of matching. The overall impact is a reduction in the net census undercount from the new method because we could find more people who gave vague or inaccurate census-night addresses in the PES. In the MIS, the national net undercount rate was estimated to be 3.9 percent (+/- 0.6 percent) using the 2006 method, compared with 2.4 percent (+/- 0.5 percent) using the new method.

More information

See more information about Post-enumeration Survey

Liability

While all care and diligence has been used in processing, analysing, and extracting data and information in this publication, Statistics NZ gives no warranty it is error-free and will not be liable for any loss or damage suffered by the use directly, or indirectly, of the information in this publication.

Timing

Our information releases are delivered electronically by third parties. Delivery may be delayed by circumstances outside our control. Statistics NZ does not accept responsibility for any such delay.

Crown copyright©

Creative Commons logo.
This work is licensed under the Creative Commons Attribution 3.0 New Zealand licence. You are free to copy, distribute, and adapt the work, as long as you attribute the work to Statistics NZ and abide by the other licence terms. Please note you may not use any departmental or governmental emblem, logo, or coat of arms in any way that infringes any provision of the Flags, Emblems, and Names Protection Act 1981. Use the wording 'Statistics New Zealand' in your attribution, not the Statistics NZ logo.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Top
  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+