Skip to main content

Log in with RealMe

To access the Procurement online service, you need a RealMe login. If you've used a RealMe login somewhere else, you can use it here too. If you don't already have a username and password, just select "Log in" and choose to create one.

What's RealMe?

To log in to this service you need a RealMe login.

This service uses RealMe login to secure and protect your personal information.

RealMe login is a service from the New Zealand government that includes a single login, letting you use one username and password to access a wide range of services online.

Find out more at www.realme.govt.nz.

How to measure outcomes and outputs

​In the social sector, it's important to measure outcomes as well as outputs.

Outcomes are what you want or expect to happen as a consequence of a service. Outputs are the things or activities produced.

Example

In a programme to improve the mental health of the long-term unemployed, the improved wellbeing of the clients after treatment would be the desired outcome and the number of sessions held or clients treated would be the output.

Specifying and measuring outcomes, rather than just outputs:

  • focuses on the difference a service makes for people
  • gives providers (together with funders) flexibility to develop different and innovative approaches to test and change according to the evidence about the difference a service makes
  • supports evidence gathering about what works.

As Results Based Accountability (RBA) is the outcomes framework that is most widely used by government agencies and NGO providers, this guide uses the structure and language of the RBA framework when discussing outcomes.

Under RBA there are two levels of outcome:

  • population outcomes
  • client outcomes.

Measuring population-level outcomes

Population-level outcomes are the outcomes or results for an entire population. A 'population' could be:

  • a specific geographic region (eg a country or region), or
  • a particular demographic, eg an ethnicity or age group.

No single provider or programme is responsible for achieving a population-level outcome. However, the services they deliver should contribute to achieving a population level outcome.

When you begin a procurement activity the population-level outcomes will generally already be known.

Measuring client-level outcomes

Client-level outcomes are the outcomes achieved by the clients of a particular programme, service or intervention. Sectors, agencies, and providers are accountable for client level outcomes, through accountability documents such as Agency Statements of Intent, approved business cases or contracts.

Examples of client-level outcomes:

  • # and % of service recipients who have reduced debt.
  • # and % of service recipients who enrol in early childhood education.
  • # and % of service recipients who do not reoffend in the first year after receiving the service.

How to determine client outcomes

Client level outcomes can be determined through the earlier stages of the commissioning process, or in the planning phase of the procurement lifecycle using one of three approaches, often in conjunction with development of other performance measures:

  • as part of a co-design process with sector representatives, potentially including clients
  • by the agency, with sector review and input, or
  • by the agency.

The risk profile and strategic importance of the programme or contract, and how much it will cost, generally influence the decision about the approach.

As part of the client outcomes development process you should be clear about:

  • the needs to be met and the outcomes to be delivered by the service
  • how clients and providers will be involved in defining their needs, expressing their wishes and choices, and influencing the design of the service
  • how the service will contribute to the defined population level outcomes and organisational objectives
  • how the outcomes will be measured.

Specifying outcomes

Outcome-based specifications should be written in performance terms, which focus on the outcome required. Describe what needs to be achieved rather than exactly how it should be done.

A key consideration when drafting the specification is whether you are the subject matter expert and able to develop a detailed service specification to be delivered to achieve the client outcomes. If the purchasing agency is not the subject matter expert you should provide flexibility within the specification around how the programme or service is delivered to achieve the required outcomes. For example, you might specify that the programme must be delivered in a way that is culturally appropriate, rather than detailing precisely what that means.

As a buyer, you should not fall into the trap of being overly prescriptive and specifying detailed requirements. For example, do not require the provider to employ two staff with X qualifications to undertake Y duties, when it is the outcome for the client that is most important.

Specifying outputs

In most cases, you will be specifying requirements for outcomes. However, you will also need to consider whether specifying and measuring outputs is going to be required for your procurement activity. This will depend on what the outcomes are and what the service is likely to be. You should ask yourself the following questions:

  • Is information on outputs and quality necessary to measure achievement of your goals?
  • How flexible should the delivery of services be? For example, is it necessary to be prescriptive about the number of services/sessions/treatments that are required?
  • If so, what do you need done and in what quantity?

Examples of outputs include:

  • 1,000 referred individuals a year undergo anger management training.
  • 15 employment skills training courses for the long-term unemployed are held each month over three years.

Resources

For additional information on developing outcomes refer to:

The Compass: Your Guide to Social Impact Measurement

Results-Based Accountability Implementation Guide

Results-based accountability videos

Top