There are three particular services which supply access to official statistics:

Statisphere: Portal to NZ’s Official Statistics System.
NZ Health Information Service (NZHIS)
Education Counts

Other major data-providers include:

Labourforce Data
Sports/Leisure Data
Housing Data- Ministry of Housing or Housing New Zealand
Work-related injury Data
Injury Data

Within the broader social sciences community the following are relevant:

The New Zealand Social Statistics Network (NZSSN) aims to assist in the development of quantitative social science research in the academic, government and private research sectors. NZSSN's intent and activities are based on those of the Australian Consortium for Social and Political Research Incorporated (ACSPRI).
Social Statistics Reseach Group, University of Auckland
New Zealand Social Statistics Network

New Zealand Social Science Data Service: NZSSDS provides access to New Zealand social science survey data and metadata, with online analysis and download available through registration. There is also a wiki-style environment for collaborative documents, which currently has its own registration system, and there is a set of analytical tools in development.

A Community profiling service with advice and also downloadable data is available here.