Contents | 1. Essence | 2. Aproaches | 3. Process | 4. Measuring outcomes | 5. Paradoxes
6. Examples | 7. Jargon | 8. Checklist | 9. Practice tips | 10. Resources

6. Examples

Evaluating “small projects”

The extent of an evaluation usually needs to be proportional to the size of the project.

A small project probably requires a small evaluation.

Example 1
Develop a community event to allow participants to contribute to their community

Evaluation questions and strategies
Who came to the community event? Observation.
What new initiatives have emerged? Six months after the event do phone interviews with key people to identify what new initiatives for people to contribute to their community have started to emerge.

Example 2
To develop a web site for information on local community services

Evaluation questions and strategies
Is the web site operational? Observation
What are the usage statistics? Web server statistics.
Is the site useful? Phone interviews with a group of users to see if it is meeting their needs? Focus group of service providers to see if it is meeting their needs.

Example 3
To provide computer equipment to a Neighbourhood Centre to allow the Centre to increase services.

Evaluation questions and strategies
Has the computer equipment been purchased? Observation.
What has it been used for? Staff to record.
How have these uses impacted on the services operated by the Neighbourhood Centre. Interviews with staff.

Evaluating “large projects”

Evaluating a large project is usually more complex than smaller projects.

Each large project may need to have an evaluation plan which will include a statement of:

1. Evaluation policy
2. Unique characteristics of the project that impact on evaluation
3. Essential evaluation strategies
4. Evaluation tools to be used

1. Evaluation Policy
An example of an evaluation policy is:

Our project will have strategies in place so the staff and the advisory committee are able to:

• Identify the extent to which the project is achieving its aims and objectives (consistent with its core values)
• Continually improve the work of the project and
• Minimise the room for self-delusion of the staff and committee members (i.e. staff and committee members thinking they are doing a good job when they are not).

2. Unique characteristics
Some projects have unique characteristics that must be taken into account in designing evaluation strategies.

For example if a project were to provide recreational opportunities to people with acquired brain injuries then some of unique characteristics would be:

• People with acquired brain injuries experience various levels of cognitive, social, emotional and physical deficits.
• The project’s philosophy is based on meeting clients individual needs - the services provided to each client will be designed to meet their individual needs
• Determining whether services are of an appropriate quality requires complex judgements to be made.
• Residents, family and significant others, and staff may make those judgements differently.

These characteristics mean that:

• It will be difficult to make definitive judgements about the worth of the project.
• It will be important to hear from clients, staff, family members and other service providers in reviewing the worth of the project.

Many projects will have unique characteristics that will affect how evaluations can be undertaken. It is useful to make these characteristics explicit.

3. Essential evaluation strategies
Larger projects will require a range of essential evaluation strategies.

For example a project to employ a transport development worker to coordinate transport options and to establish transport information points would need to include at least the following in their essential evaluation strategies list:

1. Explicit statement of values
2. Project plan
3. Explicit community development model and outcomes hierarchy
4. Focus groups of service providers
5. Key numbers. For the Advisory Committee and key staff to regularly review key (e.g. transport usage information)
6. Questionnaires to users of transport services dealing with questions of the quality, availability and access
7. Peer Review - the project officer dialoging with others to review the work in the project
8. Supervision - regular ongoing formal supervision.
4. Evaluation tools

Projects may require specific evaluation tools. For example, in the above evaluation tools will include:

1. A community development model
2. A peer review process model
3. A database for transport usage statistics

These tools will either need to be developed specifically for this project or existing tools can be modified.

Large project evaluation - an example
Preventing family violence

Here are suggested evaluation strategies and tools that could be used in the following project - outlined under needs, aims and objectives

The project

Needs - People in families need to:

• be accepted and respected
• be empowered to make their own decision
• understand the dynamics of family violence
• understand the myths surrounding family violence
• know what to do if there is family violence
• be free from violence.

Aims
The project aims are:

• For 12 to 18 year olds living in our community to:
  - recognise what family violence is
  - develop positive no-violence approaches to resolving conflict
  - gain self confidence and self esteem
• For other family members (of the 12 to 18 year olds)
  - become more aware of family violence issues
  - become less tolerant of family violence

Objectives
The project objectives are:

• For all 12 to 18 year olds attending three high schools in our area to work on preventing abuse in relationships
• For their families to be aware of the program and the issues their children are working on.
  

Strategies
The project strategies are:

• For the Preventing Abuse in Relationships program to be run in three high schools in our Shire in Term 1.
  

Evaluation questions, strategies and tools

Who participated in the program?
School records of the number of students who participated.

What difference did the project make for them?
Students to complete a questionnaire before the program and again three months after the program.
Discussion with a small group of teachers at each school to reflect on the changes they have observed within the school.

What difference did the program make for parents?
Parents to complete a questionnaire two months after the program.
Discussion with a small group of parents to reflect on the changes they have observed for their children and within their families.

VIP question: Have we sufficient evaluation strategies in place to convince a reasonable person about the worth of what we are doing?

Evaluating services

Example 1 - Casework

Evaluation strategies could include:

Telephone follow-up by an independent person of a small number of cases once every three months to see whether the service was useful and what happened.

Peer review, eg, a colleague sitting in on cases for half a day once a quarter and then discussion and review of the casework process.

Casework supervision by a supervisor.

Data collection on who receives what services and what the issues were for these people.

Population/community data - who is in the community? Are those coming to casework representative of those in the community of newly arrived people?

Example 2 - Groups

Evaluation strategies could include:

Satisfaction questionnaire at the end of the session / series of sessions? Were the groups useful? Did participants enjoy themselves?

Telephone follow-up by an independent person of a small number of participants once every three months to see whether the groups were useful and what happened.

For groups that are run often, a focus group of participants for half an hour at the end of a series- run by an independent facilitator

Data on who participates? Are they representative of the people in the community/ newly arrived migrants?

Example 3 - Community development to establish networks to reduce social isolation

Evaluation strategies could include:

Development of outcomes hierarchy for the community development process.

Peer review of the community development process, i.e., discussion of the proposed community development process - values, outcomes hierarchy, strategies,

Review of the process by an Advisory Committee

Monitoring use of staff time in the community development process? How much time is used? Is it the most effective use of time? Is it worth the effort?

Data on who becomes involved - who is involved? How many people are involved relative to the needs in the community?

Telephone follow-up by an independent person of a small number of participants three months after the network is operating to see what happened.

Questionnaire to network participants after 12 months. What have been the benefits of the network?