WHEN SHOULD YOU COLLECT YOUR DATA?
Generally speaking, if you are evaluating an existing program, you will want to know whether the expected outcomes have been achieved and whether those outcomes are due to your program. To be able to show that something has changed as a result of your program, you will need a comparison group. How this comparison group is selected impacts on how certain you can be that the differences observed are due to the program and not some other factor.
COLLECTING THE DATA
Before you start your data collection, set up a timeline and a clear protocol so everyone involved understands who is response for each part of the evaluation. Depending on the complexity of your evaluation, you may need to devote time to train people in the evaluation process, or receive training yourself. A protocol could be the evaluation plan itself, or the evaluation plan can be used to produce a flowchart. If this protocol is adhered to, you can be confident that the data is reliable because it has been collected in a standardised way.
Example: Organisation X will be evaluating its carer program by conducting focus groups with previous and current clients. A simple flowchart of what needs to happen before someone signed up for a focus group is shown below.
Example: Organisation X will be evaluating its carer program by conducting focus groups with previous and current clients. A simple flowchart of what needs to happen before someone signed up for a focus group is shown below.
You should not assume that all the people involved in the evaluation process understand what you are doing, or why something is important. For example, if you are relying on colleagues to distribute a survey as part of their usual duties, be specific about your needs. They may assume that it does not really matter when the information is collected, as long as it is collected at some point. This could result in you having an incomplete dataset, or data that does not meet your needs (e.g., missing baseline measures or baseline measures collected after clients have received the service). Anticipating problems and adapting methods to minimise their potential impact can help your evaluation run smoothly. Providing information about the evaluation to colleagues and other stakeholders can help them understand what is happening and why.
It is important to establish a baseline by collecting data at the start of your program so that you can compare the outcomes against this at the end of your program to determine its effectiveness. The tools you choose will depend on the type of information that you wish to gather, and the timing of your data collection time points will be determined by the goals and objectives of your evaluation.
It is important to establish a baseline by collecting data at the start of your program so that you can compare the outcomes against this at the end of your program to determine its effectiveness. The tools you choose will depend on the type of information that you wish to gather, and the timing of your data collection time points will be determined by the goals and objectives of your evaluation.