1. Essence | 2.
Aproaches | 3. Process | 4.
Measuring outcomes | 5. Paradoxes
Examples | 7.
Jargon | 8.
Checklist | 9.
Practice tips | 10.
Some questions to consider in measuring outcomes are:
does measuring outcomes fit (in evaluation)?
in theory can we measure outcomes?
are some of the paradoxes and dilemmas in practice?
is realistic? Who can do what?
Where does measuring outcomes fit?
essence, in measuring outcomes in human services we
want to describe:
outcomes we are trying to achieve (and any unintended outcomes)
The extent to which we are achieving these outcomes (including
showing a cause and effect link between the services provided
and the outcomes
outcomes in human services is one small part of evaluating
outcomes in human services does not replace other essential approaches
to evaluation, for example:
- Program monitoring and review
- Service network capacity evaluation
if we are able to measure outcomes we also need to know the answers
questions such as:I
Why would we want to measure outcomes in human
If human services outcomes could be measured:
Clients will have a more accurate and reliable picture of what
has been achieved by a particular service. This is likely to allow
them to make better judgments
about the value of the service and also make better choices about
Practitioners will be better able to monitor and
reflect on their work because they will have measures of what has
Services that want to continuously improve the quality of their services
will have information about the effectiveness of the services provided. This
information can be used to monitor the effects of improvements to service
Services that want to reflect on the different kinds of services will be
able to see the relative effectiveness of the different kinds of programs
they are offering. For example is family work in home more or less effective
than group work? Under what circumstances?
Peak organisations will have the evidence
- develop standards
of practice based on what standards actually get better results
- develop suggested
tools and databases for services
in policy discussions and debates with evidence
about the costs and benefits of services.
Government will be able to determine the costs and benefits of human
services and make better policy decisions about prevention
For example if
family support can be shown to achieve outcomes and those outcomes
lead to less child abuse it may be far more cost effective to pay
services than for services that have to work with problems after
they have arisen.
How “in theory” can we measure outcomes
in human services?
There is a difference between measuring outcomes in human services
theory” and “in practice”.
It is useful to ask the question: How could we measure outcomes in
human services “in theory” if adequate
resources were given to the task and the practice paradoxes and dilemmas
there to complicate the
The answer can provide a starting point for thinking about measuring
outcomes in human services.
“In theory” we can measure outcomes in human services
outcomes we are trying to achieve
The extent to which we are achieving these outcomes(and unintended
To do this we
A. Agree on the specific outcomes
to be achieved. For
example in family support services outcomes may include: improved
self esteem, improved family and friends networks, better parenting
B. Develop valid and reliable
measurement tools for these outcomes,
eg, for self-esteem, family and friends networks and parenting
practices. (Valid meaning that the measurement tool measures
what it says it measures and reliable meaning that if we
made the same measurement twice we get the same result.)
Develop valid and reliable measurement tools for unintended
D. Use the tools to measure
change in clients over time (including
collecting and analysing the data). For example
by measuring the clients situation at the beginning of
service, at the
end of service and at a point in time after the
completion of service and analysing the results.
E. Show a cause and
effect connection between the service provision and the outcomes. This
could be done in one or more of several ways, for example:
such as random assignment of clients to different types of service
research findings with practice measures. For example
in measuring health outcomes we can monitor the number of people
who are smoking
or non smoking because there is separate research to
show the connections between smoking and health outcomes. If
adequate research into family support outcomes we could
connect our service measures with the findings of that research.
4. What are the paradoxes and dilemmas in practice?
practice there are numerous paradoxes and dilemmas in evaluating
human services and community development, and in particular measuring
outcomes in human services. These paradoxes and dilemmas create
enormous difficulties in practice.
of the key paradoxes and dilemmas are:
The outcomes to be achieved may not be known in advance.
outcomes to be achieved may be difficult to precisely define.
specific outcomes we are wanting to achieve may differ depending
on the values we hold (clients, worker, funding body)
involved with the service processes may have different views
about what outcomes have been
achieved and the extent
to which they have been achieved.
in the service process are part of families, friends, neighbourhoods,
work teams and communities (and so there multiple causes
and multiple effects on clients)
are provided short term. We are hoping for long term outcomes.
service is part of a service network (and so is only one part
of the human services system)
There are different approaches
to how to measure outcomes.
There are conflicting
demands between good service delivery
and good measurement
are conflicting demands between
resource use for
and resource use
and data collection.
There are conflicting
services work with clients with low literacy levels, physical
disabilities and mental health issues.
tools are often not suited for use with these people.
There are characteristics of being human that mean that we
always know that we think we know.
full details see Paradoxes and dilemmas in practice.
overall consequences are that:
it is often difficult to measure outcomes in human services
it is difficult to show cause and effect
services provided and outcomes for clients or communities.
What is realistic? Who can do what?
the context of all the paradoxes and dilemmas if outcomes in
human services are to be measured then everyone needs to play
Clients and practitioners will be explicit
about the client’s situation
and what they are working on. This involves practitioners
- the service
models they are using
judgments they make about the client and the client’s situation.
will require practitioners
- using appropriate
assessment tools and
on the analysis of the data to improve service practice.
It will require
an assessment tool or survey
follow up surveys or other tools
on the results.
Services will gather data on who receives what services.
They will systematically review the evidence for change taking place with
clients and their situations.
In addition to the points above for practitioners and clients,
service staff will:
on all clients
Report on the analyses
Use the analysis in reflection on service practice to improve service practice.
monitoring and review data
Coordinate the development of the measurement tools
Develop a research agenda in collaboration with other relevant
organisations to work specifically on showing cause and effect
links between services
Government will have program monitoring and review information
available at a regional and state level. This will require support
for services programs:
Data collation tools
Data collation and analysis systems.
establish frameworks and systems for data to be gathered from the
services network about clients, not just about individual services.
Government will fund services for the time and resources they spend on data
collection, collation and analysis.
Government will fund research into human services, particularly
research to show cause and effect linkages between
organisations and agencies
Universities and other academic organisations will undertake
the necessary research to provide a sound basis for the use and
interpretation of evaluation
tools in human services.
6. Practice tips
the intrinsic appeal of measuring outcomes in human services
with the enormous difficulties cause by the
complexity, paradoxes and dilemmas a few practical tips are:
it is not possible to prove cause and effect relationships do
not use outcome measure to judge your performance. Rather use
to help you ask good questions.
just focus on the outcomes to be achieved, have processes in
place to identify and document unintended