Stats NZ has a new website.

For new releases go to

www.stats.govt.nz

As we transition to our new site, you'll still find some Stats NZ information here on this archive site.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
New quality measures for the Household Labour Force Survey

This page explains the quality measures added to the ‘Data quality’ section of Household Labour Force Survey (HLFS) information releases from the June 2013 quarter onwards.

Before June 2013, the HLFS provided both response rate and sampling errors as indicators of quality. The new measures will complement these and provide more insight into the quality of the HLFS. These measures are part of Statistics NZ’s quality standards and guidance framework and follow international best practice for household survey quality measures.

Current quality measures:

New quality measures:

Household Labour Force Survey (HLFS)

The HLFS measures the number of people employed, unemployed, and not in the labour force. This is the most comprehensive measure of people employed.

HLFS results are based on a representative sample of 15,000 households and about 30,000 individuals throughout New Zealand. The survey is designed to produce robust estimates of the numbers of people employed, unemployed, and not in the labour force. It uses international best practice and accepted guidelines and definitions of employment and unemployment as set out by the International Labour Organisation (ILO). The HLFS is the official measure employment and unemployment in New Zealand.

Current quality measures

The HLFS currently releases the national response rate and the high-level sampling errors as indicators of quality.

Sampling errors

Sampling errors are released for key HLFS estimates in the tables of each information release. They measure the variability that occurs by chance because a sample rather than an entire population is surveyed. The high-level sampling errors in the HLFS are consistent over time. This indicates that the variability of estimates is similar from quarter to quarter, making the estimates consistent over time.

See more about sampling errors

Response rate

The national response rate is released in the ‘Data quality’ section of the HLFS information release. It indicates what percentage of eligible households responded to the survey. The response rate is a survey estimate and is generally consistent over time; however, this measure can be affected from time to time by changes in coding practices.

See more about response rates
See more about eligibility to respond to the surveytop

New quality measures

Achieved sample size

The HLFS sample contains about 15,000 private households and about 30,000 individuals each quarter. We sample households throughout New Zealand on a statistically representative basis, from just under 2,000 primary sampling units (small geographic areas containing an average of 67 permanent dwellings). We get information from each member of the household. The sample is arranged by geographic region, urban and rural areas, ethnicity, and socio-economic characteristics.
The achieved sample size measure is the number of eligible households and individuals that responded to the HLFS in the quarter. The achieved sample size typically increases over time as the population grows and more dwellings are added to the survey sample.

In the March 2013 quarter, 30,212 people in 15,434 households responded to the HLFS. This compares with 28,139 people in 14,776 households for the December 2012 quarter and 29,737 people in 15,171 households for the March 2012 quarter.

Achieved sample rate

The achieved sample rate is calculated as the number of eligible households that responded divided by the total number of dwellings sampled. Essentially, it tells you what percentage of the sample responded to the survey. Expressing the achieved sample as a rate controls for population growth.

Achieved sample rate  =

 Eligible responding
__________________________________

 

 Ineligible + eligible responding + eligible non-responding

 

Achieved sample rate compared with the response rate

The response rate is calculated as the number of eligible households that responded to the survey as a proportion of the estimated number of total eligible households in the sample.

Response rate  =

 Eligible responding
__________________________

 

 Eligible responding + eligible non-responding

 

The achieved sample rate differs from the response rate because it includes the ineligible dwellings in the denominator. This difference means that the response rate is particularly sensitive to the classification of household eligibility. As a result, the achieved sample rate is more stable over time than the response rate. Table 1 depicts the response rate and achieved sample rate over the last five quarters. top

Table 1

Response rate and achieved sample rate, and components used to make the rates
March 2012 quarter to March 2013 quarter
Quarter National response rate Achieved sample rate
Ineligible rate(1)
Eligible non-responding rate(1)
 Percent
March 2012  85.7  73.5 14.2 12.3
June 2012  86.3  73.4  14.9  11.7
September 2012  82.6  70.2  15.0  14.8
December 2012  84.4  71.6  15.2  13.2
March 2013  85.8  74.9  12.6  12.4
1. Ineligible and eligible non-responding rates are calculated using the same procedure as the achieved sample rate.

Changes to the response rate

The response rate estimates reported in table 1 have been revised from those previously published. Revisions have been made to the period from the June 2011 quarter to the December 2012 quarter.

In late 2012 we identified that we had coded a greater number of ineligible households over 2011 and 2012, when some of these should have been coded as 'eligible non-responding'. This resulted in an artificially high HLFS response rate over this time, but had no impact on the number of eligible responding households or the overall quality of key HLFS measures.

The elevated level of ineligible households was highlighted due to an increased focus around the response rate and other quality measures. The coding practice has since been changed and we are continuing to monitor the new level of ineligible households to ensure current practice continues.

To ensure that the response rate measure is an accurate and consistent reflection of the quality of the HLFS, we have revised down the number of ineligible respondents for the June 2011 quarter to the December 2012 quarter. This has lowered the response rate over this time, as the rate of ineligibility for this period was clearly higher than it was in the past. We have not revised the most recent period (ie the March 2013 quarter).

We recommend that the achieved sample rate be viewed alongside and given the same importance as the response rate. The achieved sample rate is not affected by the coding of ineligible households and will provide a more stable picture of response. Table 2 gives the revisions to response rates over the seven revised quarters.

Table 2

Revisions to response rates in the HLFS
June 2011 quarter to December 2012 quarter
Quarter

Response rate
(Percent)

Previously published  Revised 
June 2011 87.2 85.7
September 2011 88.2 88.2
December 2011  88.8 87.3
March 2012 87.3 85.7
June 2012  87.7 86.3
September 2012  85.0 82.6
December 2012  86.7 84.4

Sample characteristics

Obtaining a sample that represents the population is essential when it comes to producing reliable labour force estimates. The HLFS goes through three stages of weighting to achieve this.

First, every household in the HLFS sample is given an initial weight. This is based on the probability of the household being selected for the survey.

Second, this weight is then adjusted, by month and region, for households that did not respond. This results in a ‘non-response-adjusted’ weight.

Third, the sample weights are adjusted to known population benchmark totals (calibration process). The HLFS benchmarks are: overall sex by five-year age groups, and Māori by sex by age group. This process results in a final weight for each household.

In figure 1 below you can see that while the distribution of the pre- and post-calibration weights differs within a quarter, the difference between the weights typically does not change from quarter to quarter. top

Figure 1

Graph, Age distribution in the HLFS, by age group, March 2012 quarter, March 2013 quarter, and December 2013 quarter.

Undercoverage rate

The difference in the survey estimates before and after calibration is an estimate of coverage. Typically, estimates before calibration are too small. This represents undercoverage. For example, in figure 1, even though the 15–24-year-olds are correctly represented in the post-calibration estimates, this age group is under-represented in the sample before calibration.

The undercoverage rate gives an indication of how representative the pre-calibrated sample is. The higher the undercoverage rate, the less representative the pre-calibrated sample. Usually the undercoverage rate in the HLFS is around 20 percent.

The overall undercoverage rate for the HLFS in the March 2013 quarter was 16.9 percent. This compares with 20.2 percent in the December 2012 quarter and 19.9 percent in the March 2012 quarter.

Proxy rates

Where practical, the HLFS gets information directly from each household member. Otherwise, a proxy interview is conducted, in which details are given by another adult in the household.

The quality of data from proxy responses is affected by two factors: what type of information is being asked for, and the relationship between the proxy (the person that the survey questions are being answered for) and the proxy respondent (the person replying to the questionnaire on behalf of the proxy).

In general, around 97 percent of proxy respondents in the HLFS are related to the person they are answering the survey for. When there is a relationship between the proxy and the proxy respondent, the quality of response is high. More than 90 percent of related people answer correctly for key variables. When the proxy and proxy respondent are unrelated there is still a high quality of response.

The proxy rate is calculated as the percentage of respondents who had someone else respond on their behalf divided by the total number of respondents. A typical proxy rate in the HLFS is around 30–35 percent. This excludes quarters when a supplement was attached to the HLFS. When a supplement is attached to the HLFS the proxy rate typically falls. This is because supplements often have different proxy rules, which have a small effect on how HLFS responses are collected. Table 3 shows the HLFS proxy rates for the past five quarters. top

Table 3

HLFS proxy rates
March 2012 quarter to March 2013 quarter

Quarter

Proxy rate
(Percent)
March 2012 36.4
June 2012 28.4(1)
September 2012  24.6(1)
December 2012 23.6(1)
March 2013 32.5

1. A supplement was attached to the HLFS for this quarter.

A full list of supplements to the HLFS is available in table 4.

Table 4

Supplements to the HLFS
Supplement Quarter
Childcare September 1998 and September 2009
Cultural experiences March 2002
Dynamics and motivations of migration March 2007
Education and training September 1996
Health June 1992 to March 1993
Household use of Information Communications Technology December 2006, December 2009, and September 2012 
Iwi affiliation survey June 1990
Marine recreational fishing survey June 1987
New Zealand income survey June quarters of 1997 to present
Older people  March 2000
Retirement income  March 1992
Survey for the Royal Commission on Social Policy  December 1987
Survey of working life  March 2008 and December 2012

More information

Each of the new quality measures will be updated every quarter and can be found in the ‘Data quality’ section of HLFS information releases from the June 2013 quarter onwards.

For longer time series information for response rate and achieved sample characteristics, see the table in the ‘Available files’ at the top of this page. If you have problems viewing the file, see opening files and PDFs.

For more information contact:
Daniel Griffiths or Conrad MacCormick
Wellington: 04 931 4600
Email: info@stats.govt.nz  top

Published 24 July 2013

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Top
  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+