I was very pleased to be invited to speak at this conference today. At Statistics New Zealand our whole reason for being is the creation of an informed society; a society that uses information to inform its decisions, monitor its progress, create knowledge, inform its debates, and anticipate its future. We are therefore very interested in data being used to inform the development and implementation of policy that works and improves the well-being of New Zealand, its communities, and its people.

 

I have been asked to talk about how smarter data can be a policy enabler. To do this I will discuss how policy making can benefit from smarter data; provide examples of where smart data been applied successfully; and talk about what do we need to realise this vision.

 

1.      The Case for Evidence-Based Policy

 

Good policy requires a good understanding of what the problem or opportunity is, options for dealing with it based on a good knowledge of what works and what doesn’t, along with effective monitoring of the results and impacts. Evidence is critical at all parts of the policy process. Without it we are reliant on instinct, anecdote, intuition and theory, all of which have a place in the process but which on their own can be a bit like flying a plane without any instruments  – possible but risky.

 

New Zealand, along with many other countries, is facing significant challenges. We have complex problems to address and we’re operating in a challenging fiscal environment. In this environment it is more important than ever that we direct our effort to things that will make a difference to our social, economic and environmental performance. To do this we need to know what works and what doesn’t.

 

I’m sure that everyone in this room understands the importance of evidence-based policy. It isn’t new. History is full of fantastic examples of where policy making has benefited from the use of data. One that I was introduced to at university, that has stuck with me ever since, concerns the work of Dr. John Snow in London in the 1850s.

 

In the nineteenth century, there were several outbreaks of cholera in London. These outbreaks were devastating and caused many deaths. In 1854, there was one such outbreak in Soho. At this time there were two major theories about the transmission of cholera. Dr. John Snow used techniques which would later be known as medical geography to confirm that the transmission of the disease occurred by swallowing contaminated water or food and not by inhaling infected air.

 

The distribution of deaths was one of the primary factors that proved this.  Dr. Snow plotted the distribution of deaths in London on a map to illustrate how cases of cholera clustered around one particular water pump. He also used statistics to illustrate the connection between the quality of the source of water and cholera cases. As a result of this work, the local authority removed the handle of the pump so that it could no longer be used, and cases of cholera immediately began to reduce. This is an early example of statistics and evidence-based approaches being used to inform policy and action.

 

However, I’m sure that we all know of times when we haven’t taken an evidence-based approach. This isn’t a new either as illustrated by the following quote from Florence Nightingale to the English Parliament over a hundred years ago.[1]

 

“You change your laws so fast and without inquiring after results past or present that it is all experiment, seesaw, doctrinaire; a shuttlecock between battledores.

 

There are many reasons why we don’t always take an evidence-based approach. What is important is that we work together to overcome these challenges so that we produce effective policy that benefits New Zealand.

 

2.      The Evidence-Based Policy Equation

 

Since I am from Statistics New Zealand I thought it was time to introduce an equation – the evidence-based policy equation. This equation says that the development of good policy requires good analysis and good process. My particular interest in this presentation is the ‘good analysis’ part of the equation. This part of the equation requires three things:

• A receptive policy environment
• Good people
• Smart data 

 

Although my presentation is about smarter data as a policy enabler, the smartest data in the world isn’t enough. Without all three parts of the equation, it just won’t add up.

 A receptive policy environment

Good analysis requires an environment that values data; a policy community that is receptive to using evidence at every stage of the policy process from problem definition, through options analysis, to implementation, and evaluation; a policy community that values scrutiny and transparency and is prepared to have its views challenged and tested by new evidence[2].

 

This can be challenging. The timeframe within which advice is needed, the desire to act quickly and sometimes behind closed doors, and the fact that clear or consistent information doesn’t always exist, can all work against an evidence-based approach. As such, developing an evidence-based culture and the capability it needs requires strong leadership. The recent review of policy advice led by Graham Scott found that what seems to matter most in the development of quality policy advice is the culture created and reinforced by leaders.

 

The pressure to improve public access to information has grown in recent years. The Government has expressed a desire to see much greater dissemination of public good information to improve decision-making throughout the economy. More sharing of data will increase transparency and accountability in government decision-making; help informed participation by the public in government decision-making; and support the innovative application of data collected for one purpose to other tasks.

 

Currently, large amounts of government-held data are not easily accessible to users either because they aren’t made available or because of legal restrictions. Increasing the availability of statistics across government and the ability to re-use data for a variety of purposes requires government agencies to release their information in a clear, timely, accessible manner. This requires a culture of access across government supported by appropriate tools, products, services, and legal framework. This does not feel comfortable for everyone. Government agencies can sometimes be disinterested in the release of information and data, unaware or unconvinced of its benefits, or actively opposed to its release. I acknowledge that there may be barriers to sharing data but none of them are insurmountable and making this happen is the challenge for those of us in government. Statistics New Zealand works actively across government with producers and users of official statistics to support a culture of access to these statistics and to government data more generally.

 

Good people

As we have seen, there is an expectation that advice given to government will be based on accurate quantitative evidence and robust analysis.  This requires staff with strong quantitative skills – skills in analysis, research, and evaluation. Policy analysts must be equipped to understand and critically assess the evidence and its reliability. Smarter data also provides capability challenges for information providers such as Statistics New Zealand as it requires new and different skills sets that are in short supply.

 

In New Zealand, as in many other countries, the education system has tended to produce graduates in either literacy-based or numeracy-based disciplines. Very few have both sets of skills[3]. In many cases, government policy makers have high literacy, but only limited numeracy skills. A pilot study by Victoria University in 2004, found a lack of quantitative skills in the New Zealand state sector[4]. Statistics New Zealand’s own consultation with statisticians and policy managers in 18 state sector agencies reinforced this finding3. 

 

Policy agencies face the challenge of building the necessary capability. This can be particularly challenging in tough economic times when spending on research and evaluation is often the first to be cut. This undermines the ability of the sector to provide evidence-based advice and as Graham Scott points out, the assumption that research and evaluation capability can be rapidly expanded is flawed as the tools, skills and expertise needed are relatively scarce.

 

A key part of Statistics New Zealand’s strategy is to help build the statistical capability of the state sector. Statistics New Zealand has developed a basic qualification to give policy analysts the skills to critically evaluate statistical releases and research reports, as well as published policy and media documents for their appropriateness; the quality of the data used; their survey design; the analysis undertaken; and the conclusions made. 

 

Supporting this growth of capability is a real challenge. Levels of expertise and the statistical techniques used are diverse and so a one-size-fits-all solution won’t work. We are currently doing work to consider how best to support the development of the statistical capability of policy analysts to interpret official statistics and use these meaningfully in their work. On a practical level, we are also able to help people use statistics appropriately by helping them to think through the questions they are trying to answer, find out what data exists, and how this might be used.

 Smart data

The third element of the equation is smart data. Smart data has a number of characteristics – it is relevant, coherent, timely, accurate, trusted, accessible and efficient. I will talk a little about each of these.

 

Smart data is relevant to the needs of current and prospective users. It covers the topics that people are interested in, in ways that enable them to answer the questions they are seeking to answer. For some this may be high level tables showing broad trends but for others it will be access to detailed data about individuals, households or firms to enable assessment of individual responses to specific policy settings. In conjunction with a number of policy agencies, Statistics New Zealand is making exciting advancements in integrating data to create innovative databases to meet these needs and inform research and policy.

 

Data is most useful if it is coherent, painting an understandable picture of the real world. We must work to ensure that we produce the information that is needed most. This requires us to identify what is important and to make prioritisation decisions. This can be difficult. To be useful for social and economic research and decision-making, frameworks and classifications used in statistics have to be relevant for key users. The adoption of common standards and classifications across the vast pool of data available from administrative and survey databases allows separate datasets to be related and enable more comprehensive statistics to be produced.

 

Statistics New Zealand leads a number of cross-government processes to identify priorities, plan statistical developments, and encourage the use of common standards. For example, in each domain of activity (e.g. knowledge and skills, agriculture, crime and criminal justice) we lead planning across government to identify the enduring research and policy needs relating to that domain; determine the extent to which current statistics in the area are adequate for current and prospective information needs; and identify actions required to address any significant shortcomings or gaps. We also work across government to identify and agree the list of New Zealand’s most important statistics, known as our Tier 1 statistics.

 

Smart data is also timely and accurate. It is available when it is needed and correctly describes the phenomena it was designed to measure. It is also accessible, easy to find and understand. It is presented in formats that suit the needs of those who want to use it, and is well-documented so that users can understand it and judge the quality of its fit for their needs.  In today’s society, this means providing access to data through the channels and services most appropriate in an increasingly technologically advanced environment. Our strategy at Statistics New Zealand includes work to increase access to our data and that of other agencies. One example is our Infoshare product launched a couple of years ago which provides free access via our website to a wide range of time-series data.

 

As noted earlier, smart data is trusted and accepted as impartial. Achieving objectivity and transparency involves using well-established frameworks and methodologies, stating assumptions,  making the methods and procedures used publicly available, and highlighting major findings.

 

Finally, smart data is efficient. It is produced in a coherent, timely and efficient way. It is only collected when the expected benefits of doing so exceed the imposition on the providers of the information. It is not collected if suitable data already exists. Costs can be contained by technical measures such as standardising and harmonising surveys; better exploitation of existing data; and shared used of data, particularly administrative data.

 

At Statistics New Zealand we are working to develop a statistical and IT infrastructure that will facilitate the linking and integration of data, enabling data to be re-used and re-packaged to create new information. This will enable us to provide more meaningful information and to do this more efficiently.

 

 

 

3.      Using Smart Data in Policy Making

 

The policy process requires lots of different sorts of data at different stages. At the agenda setting/problem definition phase, data and statistics are needed to help define the nature, size and distribution of the problem.  The policy design phase involves determining the most efficient and effective way to address the issues identified. There are often a number of ways of addressing a problem and choices must be made. Every combination of choices has different effects on the outcomes for individuals, families, firms, and sectors according to their situation. Statistical information is needed to illuminate the likely effects of alternative choices on different groups.

 

Early on in the policy process we need high level data at a fairly high level of aggregation. For example, national output is lower than that of our contemporaries or is growing at a slower rate, unemployment is high or rising, savings or exports are too low, or trends in crime rates. However, these data only hint at evidence of a potential economic or social problem. They don’t tell anything about the nature or cause of the problem, or whether government should intervene or regulate, and if so, how they should do so.

 

Answering these more detailed questions requires different sorts of information that tells us more about cause and effect; provides insight into the persistence of outcomes such as unemployment, low income, or low productivity; provides information about transitions from one state to another; and the factors that influence these transitions. This typically involves using data about individual people, households, and firms (microdata) to assess how individuals in different circumstances interact with the programme or policy in question. As technology and techniques to do this have improved, so the demand for microdata, longitudinal if possible, has increased. 

 

The implementation phase of the process requires us to monitor the way in which the programme is being administered or managed so that we can improve the implementation process if necessary.

 

At the final stage in the process policymakers need data to evaluate the impact of their policy. The ultimate measure of the effectiveness of policy is the impact on high-level outcomes such as gross domestic product (GDP) or unemployment. However, because of the time lags between getting data and measuring the effect of the policy, the many other influences on these high-level outcomes, and the lack of a control group, such aggregate statistics are often blunt measures of policy effectiveness. To overcome these problems, longitudinal microdata is increasingly being used in policy evaluations as it can be used to study individuals or firms affected by the policy, as well as a control group of others who not affected, before, during and after the policy’s introduction.

 

4.      Examples of Policy Making and Evaluation with Statistics New Zealand Data

 

The final part of my presentation looks at examples of how policy has been analysed and evaluated using new, innovative datasets built by Statistics New Zealand in collaboration with policy agencies. These datasets take two forms: the first are those that have been created by integrating data from a variety of existing data sources; and the second are datasets created from longitudinal household surveys.

 

Linking data to create integrated databases

For a number of years Statistics New Zealand has been working with policy agencies to develop datasets that combine information about the same individual or firm from a range of administrative sources and surveys. These datasets enable a wide range of policy questions that we couldn’t previously answer to be explored.

 

I have three examples that I’d like to talk about. The Linked Employer-Employee Dataset (LEED), the Longitudinal Business Database (LBD),  and the New Zealand Census-Mortality Study.

 Employment and income outcomes – LEED

Many areas of policy aim to improve the employment and income outcomes of people in New Zealand. Traditionally policy analysts and evaluators have used administrative records, tabulations from surveys such as the Household Labour Force Survey, and sometimes ad-hoc surveys to explore these issues. Although suitable for some purposes, these data sources have a number of limitations. The Linked Employer-Employee Dataset (LEED) was constructed to improve information about labour market dynamics and the factors influencing achievement of improved employment and income outcomes.

 

LEED brings together existing sources of data about employers from Statistics New Zealand’s Business Frame (which is a register of private and public sector organisations in New Zealand) with tax data about employees from Inland Revenue. It is based on monthly administrative data collected by Inland Revenue. It includes all individuals employed, the amount of income they received from all sources, and the tax that was deducted at source. Annual self-employment tax returns are also integrated into LEED.

 

LEED has subsequently been linked in separate databases with information on benefit data; tertiary education outcomes; and school achievement. Migration data is currently being linked to tax data within our latest linked administrative data project that we call iLEED (intergrated Longitudinal Employment and Education Data). More about this later.

 

A number of research initiatives have been undertaken using LEED:

• Outcomes of benefit receipt - In 2008, LEED was integrated with benefit records from the Ministry of Social Development. This allows income-tested benefits to be broken down by benefit type (e.g. unemployment, sickness). Recent research on this topic for the Welfare Working Group has looked at beneficiaries’ current employment, and beneficiary transition to employment. Other research has examined benefit-to-work transitions for different demographic groups. Previously pre- and post-benefit relationships could not be examined using administrative benefit records.
• Outcomes of education - The Employment Outcomes of Tertiary Education dataset links LEED with tertiary education information and provides information on the labour market outcomes of those who complete tertiary education, including industry training and modern apprenticeships. Various research initiatives have been undertaken using this data including investigating the labour market outcomes of young tertiary students after study; looking at the post-study outcomes of doctoral students; investigating the gender pay gap after study; and assessing the impact of workplace-based training on employment outcomes compared to other types of tertiary education. This integration also has the potential to provide information on the contribution of education to firm performance and productivity.

 Business performance and the impacts of business assistance - LBD

Traditionally, industry analyses and supporting data disaggregations have informed much microeconomic policy-making. In recent years, new sectoral and production-component surveys (e.g. research and development, and information and communication technology) have added to understanding of the performance of particular sectors and productivity drivers. An increasing trend in New Zealand and worldwide is to analyse performance and productivity drivers at the firm level.

 

The prototype Longitudinal Business Database (LBD) is a longitudinal database containing business-related data from 2000 to 2009. It is constructed by a linking a longitudinal version of the Business Frame with cross-sectional business surveys and administrative data sources. These data sources and others are integrated to produce a database that is annual and enterprise-based.

 

The LBD is currently being used to analyse business practices, performance, productivity, hedging behaviour, merchandise trade, and finance. Researchers in the Data Laboratory and researchers seconded to Statistics New Zealand are carrying out this work. Examples include research on:

• Government assistance to businesses - the LBD provides a unique opportunity to evaluate business assistance schemes by comparing the performance of assisted firms to the population of unassisted firms. An evaluation of the Growth Services Range, a programme designed to accelerate the development of high-growth firms, showed that the programme had a positive impact on business growth that was greater than the cost of the programme.
• Emission pricing policy - In 2009, the LBD was used to examine relationships between emissions intensity, firm performance, and trade exposure. This research has contributed to our understanding of the short-term impacts of emission pricing on firm performance and has informed policy development about emissions pricing and trading in New Zealand.

 Life and death - The New Zealand Census-Mortality Study

The New Zealand Census-Mortality Study is the principal instrument by which the Ministry of Health monitors social inequalities in health. It links administrative death registration records with census records. The study is a collaborative project between the Ministry of Health, the University of Otago, and Statistics New Zealand. Census adds socio-demographic variables to the deaths information, which then allows measurement of mortality differences by variables such as ethnicity, occupation, and cigarette smoking status. Similarly, the Cancer Trends project creates links between the census and the Cancer Register.

 

Starting with the 1981 Census, the study has linked mortality records to census records for the three years following each census. Unlike LEED and the LBD that link on IRD number there is no common unique identifier that would allow direct matching of records from the two data sources. For this reason, the records were matched using probabilistic methods that use information such as date of birth, sex, country of birth, ethnicity, and area of usual residence. Names were not used in the matching.

 

The creation of the integrated census mortality database has enabled the evaluation and monitoring of ethnic and socio-economic inequalities in mortality in New Zealand. It has resulted in over 50 studies that have addressed a wide range of questions such as:

• What is the relationship between individual socio-economic factors and mortality in New Zealand?
• Is unemployment associated with suicide?
• What is the contribution of smoking to health inequalities?

 Longitudinal surveys

The second form of smart data that I will talk about is created from longitudinal household surveys. Over the last decade, Statistics New Zealand has run two such surveys, the Survey of Family, Income and Employment (SoFIE) and the Longitudinal Immigration Survey: New Zealand (LisNZ).

 Income, saving, and family dynamics - SoFIE

The Survey of Family, Income and Employment (SoFIE) was New Zealand's first national longitudinal survey designed to study income, family type, and employment, and how these change over time. It covers major influences on income, including employment and education experiences, household and family status and changes, demographic factors, and health status. SoFIE began in 2002 and ran for eight years. Survey participants were visited each year to build a picture of how their circumstances change over time. In each alternate year there were rotating modules on health and wealth.

 

The Treasury has used SoFIE to investigate whether New Zealanders are saving enough for their retirement. They found that while large parts of the population aged 45–64 have made adequate provision for their retirement about one-third of the population may not be saving enough. The Treasury is continuing work to identify the groups who may not be saving enough and to consider what kind of policies might encourage those groups to save. Questions specific to KiwiSaver were incorporated into the SoFIE questionnaire for wave 8 to assist evaluation of this scheme.

 Immigration policy and settlement services - LisNZ

The Longitudinal Immigration Survey: New Zealand (LisNZ) was a partnership between Statistics New Zealand and the Department of Labour. This survey was designed to trace the pathways of migrants and to produce a detailed, ongoing information base of their experiences and settlement outcomes. Migrants were interviewed at 6, 18, and 36 months after they had taken up permanent residence in New Zealand.

 

The findings from the LisNZ are used to inform and evaluate immigration policy, particularly in the areas of immigration selection and settlement policies. The findings also help a variety of agencies and community groups to develop services for migrants.

 Benefits and challenges of integrated databases and longitudinal surveysBenefits

As we have already seen in this presentation there are considerable benefits for policy makers in having longitudinal data about individuals and firms. This sort of data allows policy makers to follow individuals and households over time and understand more about transitions, cause and effect, and connections between polices and outcomes.

 

In most cases the integrated research databases provide longitudinal data for much of the population of interest which means that research can be done on very small groups without the introduction of sample error. The ability to support outputs to finer levels than currently published, and the opportunity to be able to create control groups over a common time period means these datasets have huge potential for evidence-based policymaking.

 

Longitudinal surveys enable us to collect information that is not available from administrative sources, such as family structure and personal characteristics as well as information about attitudes, behaviours, and particularly personal topics.

 

Challenges

While there are obvious benefits with this sort of data, there are challenges associated with it. These challenges cover its production and use.

 

Production

A major issue with the data integration projects related to how they would be perceived externally and whether it would adversely affect public confidence in Statistics New Zealand. This is particularly true when integrating census records with other data.  In the case of the Census-Mortality Study, Statistics New Zealand decided that the benefits of the data integration for health research and policy outweighed the risks. The risks were managed by being open and transparent about the linkage of records, by not using direct identifiers in the integration process, and by restricting access to approved researchers within Statistics New Zealand’s Data Laboratory. This restriction currently applies to all our longitudinal microdata in order to protect the confidentiality of individuals.

 

A number of challenges arose as LEED was being developed. One was the understanding of legal and relationship issues to do with accessing datasets from other agencies. In general, source agencies saw the benefits of sharing their data, however, in some cases there was a reluctance to allow access to politically sensitive administrative data. This challenge was overcome through a slow process of building trust. There is also a need to be continually aware of the potential concern of individuals that unrelated information about them might be collected in an ever-growing database for non-specific purposes. For this reason transparency has been a feature of all of these projects as has the focus on data security and confidentiality in the outputs produced.

 

Last but not least there are a series of technical and capability challenges facing projects of this nature. It was challenging to develop the statistical tools and information systems needed to integrate these large complex datasets and turn them into information. Capability issues were overcome by contracting in data warehousing expertise and seconding other government researchers to do policy-relevant research, while we built and continue to build our capability.  Longitudinal surveys are also challenging both in terms of data collection and methodology. Statistics New Zealand’s experience with SoFIE has demonstrated methodological and technical challenges. We have learnt a lot from this experience, and many of the lessons learnt in running SoFIE were then taken on board in designing LiSNZ - which ran much more smoothly as a result

 

Use

As well as challenges in producing these datasets, there are challenges in using them. Really exploiting their potential needs people with strong quantitative skills which as we have already seen can be in short supply. The relatively small pool of people in New Zealand with the skills to analyse longitudinal data limits the value that can be gained from it.

 

The issues surrounding access to this sort of data can also create barriers for some. Legally, the Government Statistician is currently unable to grant access to microdata to non-government researchers, and in some cases our processes have been slow. At Statistics New Zealand we have work underway to improve access to microdata. We have streamlined our processes to make access quicker and easier for those we are legally allowed to provide access to. We have worked with the Ministry of Economic Development to include a change to the Statistics Act in their Regulatory Reform Bill to allow non-government researchers access to the data at the Government Statistician’s discretion; and we are exploring offsite and remote access possibilities. As always we need to balance our desire to provide access to this powerful data for the benefit of New Zealand with the need to ensure that we protect the information provided to us by New Zealand firms, households, and individuals.

 Where to next with integrated databases and longitudinal surveys

In this paper I’ve shown the value of longitudinal microdata for evidence-based policy. This sort of data is both complex and costly to produce. Longitudinal surveys in particular are very expensive to conduct and analyse. They also place a significant burden on survey participants. For these reasons it is important to ensure that we meet our longitudinal information needs in the most cost effective way.

 

Our current priority is to maximise the use of existing data from administrative sources and surveys. There is huge potential in linking this data to create exciting new datasets for researchers and policymakers to use. We do not have plans to run a SoFIE equivalent in the near future.

 

As mentioned earlier, we are creating a new flexible data integration environment (iLEED) which will allow a more systematic approach to longitudinal data linkage across government. It will enable links between existing and new administrative and survey datasets at the person and business level while ensuring that this is managed in a way that respects privacy and security protocols.

 

The extension of LEED to iLEED will enable much greater insight into complex policy questions. Further development of the LBD will enable a number of sets of statistics to be brought together to provide real insights into issues such as the impact of government intervention on firm and employment dynamics and productivity. Linking the two together will open up a world of possibilities for the research and policy communities.

 

5.      Conclusion: realising the vision

 

Realising the potential of smarter data in policy making requires data providers and policy agencies to work collaboratively to create and use evidence. As we have seen it requires a receptive policy environment, good people, and smart data. It requires a partnership between those of us that provide the data and those of you that use it.  Statistics New Zealand often finds itself talking to research and evaluation staff in agencies but less so with policy staff. At our most recent User Conference[5] a number of public sector Chief Executives spoke about the critical importance of statistics in developing and evaluating policy. Yet while the forum was well-attended by research and evaluation staff from a range of departments, there was a notable lack of policy staff there. This is a problem if we are serious about the importance of evidence-based policy.

 

Over the past decade, policy development and evaluation has been significantly advantaged by new, policy-relevant, official statistics. In building these datasets, Statistics New Zealand has benefited from collaboration with policy agencies. Policy agency sponsorship through the budgetary process was instrumental in establishing SOFIE, LisNZ, and LEED, while data quality and design has benefited from the insights of users. Data will only improve if we sustain strong partnerships between data users and providers, especially in an economic environment where value for money is closely scrutinised.

 

At Statistics New Zealand we have just embarked on a ten year strategy, Stats 2020 – Te Kāpehū Whetu. This will transform us into a modern statistical office that ensures that New Zealand gets and uses the information it needs to grow and prosper. We look forward to working in collaboration with the research and policy communities to create the evidence base the country needs and to see it used to develop innovative policy that benefits New Zealand and its people.

 

 

 

 

References

 

Banks, G. (2009) Evidence-based policy making: what is it? How do we get it?

 

Bascand, G. (2009) The role and challenges for Official Statistics in policy evaluation, paper presented at the 2009 Australasian Evaluation Society International Conference, Canberra

 

Brown D. (2011) Social and population statistics information needs for New Zealand: towards 2020, Available from www.stats.govt.nz

 

Bycroft C. (2011) Social and population statistics architecture for New Zealand, Available from www.stats.govt.nz

 

Forbes, S., Bucknall, P., and Pihama, N. (2010) Helping make government policy analysts statistically literate

 

Macky, R. and Saffron, K. (2004). A qualitative study of training needs of statistical analysts in the public sector. Victoria University, Wellington, New Zealand.

 

Statistics New Zealand (2007) Principles and Protocols for producers of tier 1 statistics, available from www.statisphere.govt.nz

 

Committee Appointed by the Government to Review Expenditure on Policy Advice (2010), Improving the Quality and Value of Policy Advice