Evaluating the impact of projects

Ako Aotearoa has started an impact evaluation of the projects funded under its Regional Hub Project Funding Scheme and National Project Fund. This will help us understand the sustainable impact these projects have on improving the teaching and learning process and learner outcomes.

The evaluation includes interviews that are conducted with the teams of all completed projects at 6-, 12- and 24-month intervals. The data from these interviews will be used to evaluate Ako Aotearoa’s funding initiatives. This will also inform how we support the project teams in disseminating their work more widely. The following examples from my recent interviews with completed Central Hub project teams give an indication of how projects are impacting positively on learners and teachers.

The success and impact of early career academics in New Zealand tertiary institutions

This project, by Dr Kathryn Sutherland, Victoria University of Wellington, and Lesley Petersen, Eastern Institute of Technology, set out to uncover the experiences of early career academics at two New Zealand tertiary institutions. It aimed to identify key recruitment, appointment, induction, training, and development processes that result in well-prepared academics capable of enhancing their students’ educational outcomes and their own careers.

The findings have been incorporated into a printed resource used in the orientation programme for all new teaching staff. Dr Sutherland has also started running “Developing Scholarly Habits” retreats for early career academics at VUW. By the end of this year, 80 teaching staff will have used the resource and 30 will have participated in the retreats. Feedback from the new teachers indicates they are more organised, managing their time more efficiently, preparing more productively, and enjoying their work more.

> find out more about this project

The utility of learning packages and their suitability for building apprentices

The Building and Construction Industry Training Organisation (BCITO) wanted to review all aspects of the training of building apprentices and accept recommendations for the improvement of their learning experiences. With Ako Aotearoa funding, the BCITO commissioned Dr Amanda Gilbert to produce a comprehensive report that has enabled the BCITO to make changes to their training initiatives. These changes included: the case management of apprentices’ learning, a focus on the assessment of integrated tasks rather than on unit standards, providing professional development in assessment and moderation techniques for a cohort of training advisors, and removing identified barriers to apprentice learning.

To date, over 6,000 trainees have benefited from the improved methods, with 2,573 apprentices successfully completing their qualifications under the new system during 2010.

The BCITO chief executive Ruma Karaitiana explained the different ways that learners have benefited from the project, citing: increased support and more regular formative assessment, clearer expectations supported by goal setting, better identification and addressing of barriers to learning, and steadier progress with improved completion.

> find out more about this project