Evaluating the impact of projects
Our impact evaluation framework (IEF) is a mechanism to help answer this question. The IEF was first developed in 2010, then trialled in 2011. This work has been shared with the Office of Teaching and Learning in Australia1. From 2012 it has been implemented across all Ako Aotearoa project funding streams.
How projects are evaluated
Ako Aotearoa’s IEF examines a project’s impact in four key dimensions:
- Reach (generation and dissemination of project outputs).
- Impact on teaching practice
- Impact on learners
- Impact on the project teams themselves
In each of these dimensions several themes are explored.
Ako Aotearoa staff visit the project teams and conduct a series of evaluative interviews at six, 12 and 24 months after a project has been completed. The four impact dimensions (reach, impact on practice, impact on learners, and impact on project team) are explored in a process that seeks to distinguish between independent evidence and that attributed by practitioners.
Interview conversations are conducted in the context of the original project goals, and results from the conversation are developed into a project ‘story’ by Ako Aotearoa staff. The story summarises the key achievements of the project to date, and the future focus of activities. These ‘stories’ are updated at each interview point, and act as a comprehensive account of the impact of an individual project over the two years after the work been completed.
The IEF is not just an opportunity to measure change it is also an opportunity to:
- extend project work and outcomes, and to further support the project team
- collect qualitative stories about the transformative potential of improved individual, group and learner experience
- be able to show both our government funders and our co-fundees (our tertiary sector partners) their return on their investment
- better understand how we can most effectively lever off the good work that has already been done and identify when and how it can be applied in different contexts.
Keesing-Styles, L (2017 March) Contributing to better tertiary teaching – The summary report
Chauvel, F (2017 March) Contributing to better tertiary teaching – The full reports
Coolbear., P. and Hinton, T. (2013 July). Evaluating the impact of research projects in tertiary learning and teaching: exploring the geography of change . Plenary address HERDSA Conference, Auckland, New Zealand
Weir, K. (2014). Enhancing tertiary teaching and learning through Ako Aotearoa-funded project work: Part 1. Collated results of Impact Evaluation Framework conversations about National Project Fund projects completed from November 2009 to November 2013. Wellington: Ako Aotearoa
Weir, K., Thomson, R. and Coolbear, P. (2014). Enhancing tertiary teaching and learning through Ako Aotearoa-funded project work: Part 2. Collated results of Impact Evaluation Framework conversations about Regional Hub Project Fund projects completed from June 2008 to November 2013. Wellington: Ako Aotearoa
1 Refer to the Coolbear, P. and Hinton, T. (2013 July) presentation available on this page.