Appendix D: Two representations of hierarchy of outcomes

Process EvaluationOutcome Evaluation
Low levelMedium levelHigh level
Implementation /effectiveness measuresPotential data sourcesOutcomePotential data sourcesOutcomePotential data sourcesOutcomePotential data sources
How effectively has NIM been administered and implemented?Short term changes to expenditure patterns (ie less expenditure on excluded goods, more expenditure on priority items).Take-up of referrals to money management and financial counselling services and improved educational attendance.Acquisition of money management skills, improved employment opportunities and improved educational attainment.
What have been the resource implications of implementing the program?

Number of service providers (Centrelink staff, MM&FC, Child Protection Case workers etc).

Expenditure on:

  • communication products
  • staff  and
  • BasicsCard.
Has there been a change in spending patterns on food among people on NIM?Allocation of funds and expenditure data from Centrelink.Has there been a reduction in harmful behaviours among people on NIM (e.g. child neglect)?

Crime data

Child health and development (e.g. incidence of low birth weight; stunting, wasting, underweight, anaemia in children under 5, child protection substantiations etc).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.

Crime data.

Has children's wellbeing (in relation to their basic priorities being met) improved?

Child health and development (e.g. incidence of low birth weight; stunting, wasting, underweight, anaemia in children under 5, child protection substantiations etc).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
Have suitable individuals and groups been targeted by NIM?

Centrelink administrative data (demographics).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink staff
  • MM&FC service providers and
  • Child Protection Case workers.
Has there been a change in expenditure on alcohol?

Allocation of funds and expenditure data from Centrelink.

Store based data (Coles, other stores).

Has there been an increase in the take-up of early childhood education?Department of Education data (school enrolment and attendance data, National Assessment Program – Literacy and Numeracy (NAPLAN) data).Has there been increased financial literacy?

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
Have people been able to transfer into and out of NIM appropriately (e.g. choosing to transfer from income management under NTER to VIM etc)?

Administrative data (all relevant sources).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink staff
  • MM&FC service providers and
  • Child Protection Case workers.
Has there been a change in expenditure on cigarettes?
  • Allocation of funds and expenditure data from Centrelink.
  • store based data (Coles, other stores).
Are people better able to manage and maintain their budgeting skills?

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers
  • Child Protection Case workers and
  • longitudinal case studies.
Are people able to set aside money for unexpected expenses (e.g. car repairs, illness)?

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
What is the profile of people on the different income management streams?Administrative demographic data (all relevant sources).Has school attendance improved?Department of Education data (school enrolment and attendance data, National Assessment Program – Literacy and Numeracy (NAPLAN) data).Are people better able to plan for unexpected expenses (e.g. car registration)?

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers
  • Child Protection Case workers and
  • longitudinal case studies.
Has NIM contributed to increased choices and opportunities for people (e.g. set aside money for holidays, access to better jobs, education)?

Matched savings data

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.

Matched savings data

Have there been any initial 'process teething issues' that need to be addressed?

Data from focus groups with community leaders, Government Business Managers (GBMs) and Non-Government Organisations (NGOs).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
Has there been a change in mobility patterns?Centrelink administrative data.Has food security improved?

Health data

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
  
What are the views of participants in the NIM model and their families? Client interviews  Has housing security improved (ie reduction in evictions, paying rent on time)?

Incidence of late payment/non-payment of utilities bills.

Housing data.

  
Has the measure been implemented in a non-discriminatory manner?Ombudsman and appeals data  Has there been a reduction in pressure to give money to others (including humbugging).

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.
  

BasicsCard usage:

  • appropriate number of merchants signed up
  • people understand how to operate the card and how to check balances.

Data on BasicsCard including the number and percentage of:

  • declined transactions
  • successful transactions and
  • usage of BasicsCard.

Quantitative survey of merchants accepting BasicsCards.*

  Has take-up of money management and financial counselling services increased?

Data on MM&FC services referrals and completions.

In-depth interviews or surveys with:

  • NIM customers
  • Centrelink social workers
  • MM&FC service providers and
  • Child Protection Case workers.

Data on MM&FC services referrals and completions.

  
    Has school attendance improved?Department of Education data (school enrolment and attendance data, National Assessment Program – Literacy and Numeracy (NAPLAN) data).  

* While Centrelink does not provide a break down BC data by each stream of IM, it may be possible to obtain approximate distribution by stream by using 'imputation' – a statistical technique used in such cases.

Greater Self-Reliance And Economic, Social And Community Engagement And Increased Choices

Greater Self-Reliance And Economic, Social And Community Engagement And Increased Choices
This chart outlines how greater self-reliance and economic, social and community engagement and increased choices result from:

  • increased financial literacy,
  • Improved Children’s wellbeing (in relation to priority needs)
  • Income management is no longer needed

These result from

  • Ability to set aside money for unexpected expenses (e.g. illness, car repairs) and to develop basic household budgeting practices
  • Increased take up of MM/FC services.
  • Reduction in harmful behaviours and pressure to give money to family and friends (humbugging).
  • Increased take up of education services, including school, early childhood education and age appropriate, social, learning or physical activities.
  • Responsible parenting activities undertaken.
  • Improved food and housing security
  • Widespread ability to use BasicsCard (stores are available, understanding operation of card)
  • Money is spent on priority needs

This page is part of: Evaluation Framework for New Income Management (NIM)

  • Print
  • Email
DSS2877 | Permalink: www.dss.gov.au/node/2877