WHAT DO YOU WANT TO EVALUATE?
This may look like a simple question, but it can sometimes be surprisingly complex. As you move through the steps of planning an evaluation, you may find yourself having to come back to this section to think (or re-think) about the important characteristics of your service. For example, you may decide on an evaluation question and design but then realise you do not have the resources to complete that task. Having a clear, accurate idea of what your service is meant to be achieving can help you identify smaller evaluation projects or alternative ways to achieve the same goals.
Before diving into an evaluation plan, take time to think about the characteristics of the service you want to evaluate. This influences the types of evaluation questions you will be able to ask and answer. Develop a program description that includes:
These are important questions because it makes no sense to judge the effectiveness of your service against criteria that it cannot logically meet. For example, services that support carers are unlikely to be trying to improve cognitive function in people with dementia in the same way a clinical trial of a new 'dementia pill' might be.
Optional activity: Writing a program description
Before diving into an evaluation plan, take time to think about the characteristics of the service you want to evaluate. This influences the types of evaluation questions you will be able to ask and answer. Develop a program description that includes:
- The problem/issue the program is addressing
- What the program is expected to do (short-term, mid-/long-term)
- What the program does to address the problem
- What resources are used by the program (people, financial, equipment, etc.)
- The context of the program (rural/regional vs urban, can it be adapted for other settings?)
These are important questions because it makes no sense to judge the effectiveness of your service against criteria that it cannot logically meet. For example, services that support carers are unlikely to be trying to improve cognitive function in people with dementia in the same way a clinical trial of a new 'dementia pill' might be.
Optional activity: Writing a program description
LOGIC MODELS
The information from your program description can be brought together in a logic model.
A logic model is a summary of a program and can change as the program develops over time. It is a plan for how your service is addressing the identified problem and what outputs and outcomes are expected. This information can aid evaluation planning because it clearly states what you expect to happen as a result of your service. In its simplest form, a logic model can be thought of as a flowchart that shows the connection between a problem/need, what your service/program does, and how this changes things for those affected by the problem.
For example, to solve the problem of feeling hungry:
A logic model is a summary of a program and can change as the program develops over time. It is a plan for how your service is addressing the identified problem and what outputs and outcomes are expected. This information can aid evaluation planning because it clearly states what you expect to happen as a result of your service. In its simplest form, a logic model can be thought of as a flowchart that shows the connection between a problem/need, what your service/program does, and how this changes things for those affected by the problem.
For example, to solve the problem of feeling hungry:
And using the Dementia SPEAKE Program as an example:
A common style of logic model is that developed by the University of Wisconsin-Extension (see figure below). Assumptions and external factors can also be included. Logic models are read from left to right; however, they do not have to be created in this way. They can also be changed and revised as you gain a better understanding of your service and what it aims to achieve.