7. Assessment tool comparison
7.1 Assessment tool comparison matrix
To evaluate the relative strengths and weaknesses of existing wage assessment processes being employed by Business Services, a matrix incorporating a range of criteria defining elements of each process was developed. The matrix addresses each tool/system's compliance with relevant legislation and standards, wage outcomes for workers, and the extent to which the process meets social and political expectations. In addition, the scientific rigour and practical applicability of each tool/process has been considered in the matrix.
The key components of the matrix are:
- compliance with relevant legislation and standards;
- validity - the extent to which the assessment is assessing what it claims to be assessing (that is, competence or productivity);
- reliability - the extent to which different assessors would achieve the same result when assessing the same person (inter-rater) and the extent to which the same assessor achieves the same result for similar situations or for the same person at different times (intra-rater);
- wage outcome - an assessment of the 'fairness' and appropriateness of the wage outcome from the assessment in accordance with assessed 'scores';
- practical application of the tool; and
- administration and cost implications of instituting the system.
The types of assessment tools/processes have been separated in accordance with the rationale upon which they are based. As discussed previously, a range of assessment processes (with varying degrees of structure and formality) are currently used by Business Services. The assessment types described below represent the main categories observed during the research project.
Whilst the matrix overleaf attempts to present an objective comparison of assessment tools, some subjectivity was required in the absence of detailed comparison studies. In addition, it should be recognised that the consideration of reliability, validity and legislative compliance was based on brief site visits, responses to stakeholder surveys and review of documentation. Whilst we acknowledge the limitations this places on the rigour of the arguments presented in the matrix, it represents the research team's 'best guess' on the basis of the information presented.
| ELEMENT OF COMPLIANCE | ASSESSMENT TOOL/PROCESS | |||||
|---|---|---|---|---|---|---|
| SWAT | Greenacres | Other productivity | Competency only | Hybrid | Ad hoc/Historical | |
| Disability Services Standards | Complies | Complies | Some questionable | Would comply | Most comply | Many questionable |
| Disability Services Act | Complies | Complies | Most comply | Would comply | Most comply | Some questionable |
| Disability Discrimination Act | Complies | Complies | Most comply - more detailed assessment required. | Likely to comply | Comply – fosters workplace integration. | Some questionable, particularly those continuing to pay historical (usually very low) wage rates |
| Workplace/Industrial Relations Act(s) | Complies | Complies | Usually comply - have been included in some enterprise agreements. | Likely to comply | Complies – may be linked directly to award and industry standards. | Questionable compliance. |
| Validity | Very good – independent accredited assessors using standardised and structured assessment tool. Applied across various settings/ industries. | Good – assessors trained but internal to organisation, potentially limiting objectivity. Being tested in other settings/ industries (generic application). | Average – most use internal assessors with a range of qualifications/ experience. Systems sometimes not structured. | Variable – range from average to very good, depending on training, experience and objectivity of assessors and structure of assessment process. | Average – systems not always formalised or documented. | Poor – large degree of variability, process not transparent. |
| Reliability | Strong – standardised tool and independent assessors enhance reliability. | Reliable within the organisation, and system is only endorsed in other Business Services when sold with training. | Internal reliability likely to be variable unless assessors are trained and monitored. External reliability not applicable. | Variable - depends on qualifications, training, monitoring of assessors and supporting QA mechanisms. | Variable – depends on assessor training and supporting QA mechanisms. | Poor - usually limited supporting documentation, may have no formal assessment structure. |
| Acceptability to workers/carers | Well understood, due to application throughout open employment setting. | Clearly articulated to employees, but a complex system. Processes in place to enhance employee's understanding. | Usually quite clear and simple. | May become complex, consumer/advocate training required. | May become complex, consumer/advocate training required. | Usually clear, often based on rationale that all workers earn the same. |
| Administration/ cost | Commonwealth subsidised for some services, not s13 Business Services. | Significant commitment to the Training Unit (to cover costs of assessment) is required. | Assessments can be tailored to fit resource availability. | Application of endorsed Training Package competencies may reduce pre-assessment workload. | Reasonably costly if conducted with all QA processes, or if not subsidised. | Low cost option. |
| Links to training/ development | Average – does not apply assessment to training/development needs. | Very good – training closely linked to findings of assessment. | Not necessarily linked to formal training. | Likely to enhance access to training with transferable qualifications. | Has potential to link with variety of training structures, including onthe- job, in-house or external (eg TAFE) programs delivering transferable qualifications. | Limited - training usually limited to personal development/daily living skills training eg transport to work. |
| Calibration of wage determinant | Good - linked to coworker benchmark, but may be manipulated. | Average - wage outcome does not adequately reflect assessed capacity, as it is not linked to industry endorsed Training Packages. | Average – good. If linked to SWAT increments or productivity benchmarks. | Likely to be good if linked to endorsed Training Package competencies. | Good - can link to endorsed competencies and productivity benchmarks. Should ensure that employees are not 'doubly-penalised'. | Poor - usually paid based on arbitrary determinations. |
| Payment outcome | Good | Average – can be used to match capacity to pay.* | Poor – Average – can be used to match capacity to pay. | Average – can be used to match capacity to pay. | Average – can be used to match capacity to pay. | Variable – usually poor |
| Incorporation into certified agreements | Yes – numerous agreements in place. | Yes – agreement in place. | Occasionally – likely to be placed under increased scrutiny. | Possible | Occasionally, otherwise use 'slow worker permit' applications | Some have had wage processes included in enterprise agreements by linking to awards (eg as a proportion of award). |
| Advantages | Accepted by open employment sector and understood by services. Promotes links between Business Services and open employment. Clear link to award wages, model clause already part of many awards. | Enables transferability of skills. Accreditation and registration of courses/assessors enhances credibility. Cost-effective if linked to ongoing training models. | Assessment can be adapted to meet needs of individual and/or job. Cost-effective. Does not necessarily on co-worker, may be linked to other benchmarks. | Potential to link with industry-endorsed competencies. Potential to link with formally recognised training. | Has potential to consider both productivity and industry-endorsed competencies. Enhances links to formally recognised training. | Assessment (if conducted) is opportunistic, therefore low cost Employees not differentiated on basis of productivity etc. |
| Limitations | Designed for open employment settings. May be difficult to compare with nondisabled co-worker due to nature of job. Potentially costly. Assessed level reflects comparative performance over short term only. | In-house assessment may limit objectivity. Potentially costly to services without significant commitment to training and assessment. | Assessors usually inhouse, may limit objectivity. Assessment not transferable between industries or jobs. | Does not consider productivity, which would limit acceptance within the sector. | May increase wages beyond capacity to pay. May 'doubly penalise' employees if not appropriately structured. May be costly if not planned/implemented effectively. | Usually based on historical or subjective processes. Demonstrates lack of sophistication in business/admin processes. Usually associated with low wages. |
* The assessment of the Greenacres tool recognises that wage outcome is better than in many other Business Services, but the research team considers that improvements may be made.
DSS3061 | Permalink: www.dss.gov.au/node/3061