'Opening our eyes': Transferring learning to enable better person-centred support services
The publication focuses on a specific piece of evaluation carried out for the organisation which aimed to discover how effectively the organisation is using training to support ‘training transfer’, that is, the application of the knowledge and skills acquired from training in the workplace.
Dr Nicky Murray, Careerforce
Careerforce, Spectrum Care and Te Pou
The Kaiāwhina (or non-regulated) health and disability workforce adds value to the health and wellbeing of New Zealanders by being competent, adaptable and an integral part of service provision. A fundamental component of workforce development for this workforce is to ensure that the learning that trainees participate in is successfully transferred to better job performance, resulting in sustainable improvements to the health and wellbeing of New Zealanders. There are two aspects to understanding how this transfer of learning occurs; the first revolves around the trainee – the pedagogical processes that support and enhance transfer, and the second revolves around the organisation – the workplace environment and ‘affordances’ that allow the learning to be fully utilised.
This publication describes the practice of systematic and in-depth evaluation of the effectiveness of training undertaken by a disability support service organisation, Spectrum Care. The publication focuses on a specific piece of evaluation carried out for the organisation by Te Pou, which aimed to discover how effectively the organisation is using training to support ‘training transfer’, that is, the application of the knowledge and skills acquired from training in the workplace.
As far as we are aware, this is the first time that this sort of in-depth evaluation (of workplace-based learning for non-regulated staff) has been carried out in the health and disability sector. Te Pou, the organisation who conducted the evaluation, is building a body of knowledge in this area. The partnership between this workforce development agency and Spectrum Care has provided a model and set a benchmark for other organisations in the health and disability sector, should they wish to better understand and reflect on the value and effectiveness of the training that they offer. The model could also be extended to other sectors or industries where workplace learning is predominant.
Spectrum Care is an independent charitable trust that provides services for children, young people and adults with disabilities, and their families. Services include 24-hour support for people living in residential homes throughout the Auckland and Waikato regions, respite care for adults in the Waikato and Bay of Plenty, and respite care for children in Auckland.
As an organisation, they believe in providing person-centred services and options that focus on individual needs. All services support people to identify their personal goals and aspirations. These are developed into a personalised and achievable ‘Outcomes’ plan, which supports service users to achieve their immediate and lifelong objectives - ‘Enabling Good Lives’. Spectrum Care supports the principles of the New Zealand Disability Strategy and places great emphasis on the worth of the individual, personal growth and the provision of holistic support for people with disabilities.
Spectrum Care invests heavily in workforce development. Staff training comprises a significant portion of the organisation’s development budget. All workforce development undertaken aligns to the organisation’s vision and is expected to enhance organisational performance in relation to this. Spectrum had some existing understanding of the quality of the content and delivery of the training they provide for staff. However, they were less clear about how well the organisation used the training to support enhancements in practice and performance that aligned to the organisational vision.
The purpose of the evaluation was to identify how effectively Spectrum used training to meet its existing and future business objectives. The focus was on how well training was utilised to support ‘training transfer’: the application of the knowledge and skills acquired from training in the workplace. They sought to understand the training and non-training factors (see Table 1 below) that enable or constrain the effective use of training by Spectrum and how the organisation could strengthen or address these.
Table 1: Training and non-training factors that support the use of training
The research literature on effective training supports the focus of the evaluation. Multiple factors influence performance outcomes from training and many of these are non-training factors that occur before or after training itself (Brinkerhoff, 2005; Roche, 2009; Salas, et al, 2012). For example, was training initially confirmed as the correct solution to the problem? Were trainees ready and motivated to learn and did they have a need for training? After training, did trainees return to a workplace that supports training transfer? The evidence is particularly strong that if the post-training environment is not supportive, sustained performance improvements from training are unlikely (Arthur, et al, 1998; Baer et al, 2004; Bennett et al, 2007; Miller & Mount, 2001; Roche, 2002; Walters, et al, 2005). Effective training is a whole of organisation challenge and evaluating effectiveness requires attention to the entire ‘training to performance’ process (Brinkerhoff, 2005).
The primary intended audience for this evaluation was the Spectrum board and Spectrum managers. Secondary audiences were Spectrum staff and wider stakeholders within the disability sector with an interest in the effective use of training. The Spectrum board has used the evaluation to inform strategic decision making regarding the future design and implementation of Spectrum’s training programme. Senior managers and leaders within Spectrum have used the evaluation to guide operational decisions regarding the training programme.
A mixed methodology was employed. An online survey examined use of training, and enablers and constraints to its use. The survey data was primarily quantitative. Group and individual interviews provided deeper understanding of enablers and constraints. The interview data was primarily qualitative. Both forms of data collection complemented and strengthened the other.
The evaluation methodology drew from Success Case Method, theory-driven evaluation, and utilisation-focused evaluation. Success Case Method (Brinkerhoff, 2005) examines how well an organisation is using training to achieve performance outcomes. Opportunities for strengthening factors contributing to success are identified by focusing on trainees who are using the training well. Constraints to training transfer are identified by focusing on trainees who appear less successful in their use. The approach is learning-orientated and focused on what an organisation can do to enhance performance outcomes from training.
The survey was distributed to 263 people, undertaking three different types of training (Support Worker NZQA Training Level Two and Three, Outcomes and Person Centred Active Support Training, and Applied Leadership Training). Responses were received from 126 people, giving a response rate of nearly 50%.
Thirty telephone interviews were then conducted with respondents selected from the completed surveys. The selection was based upon choosing a number of respondents who appeared to be using the training well and a number who appeared to be less successful, in line with Success Case methodology. Interview participants represented the diversity of trainees on key demographic characteristics.
The evaluation showed good performance across the three types of training on pre-training factors known to enhance the likelihood of use. Trainees had good understanding of how training was expected to be of benefit to them and commonly saw it as an opportunity. There was consistent evidence that the training was providing the knowledge and skills required by trainees to meet practice and performance expectations. Reflecting this, trainees frequently used the training in their work and described many examples of how it was relevant and useful to them. These are experiences and conditions that will support the motivation of trainees to engage in and use training.
Many trainees described the training as a link between theory and their practical experience. The training provided frameworks for applying existing knowledge and experience within a theoretical and Spectrum-specific context. Trainees gained a better understanding of the person-centred approach and outcomes philosophy and other policies and procedures (e.g. code of rights, ethical considerations). This helped to clarify the ‘why’ and ‘how’ of their work: …“it makes me reflect in so many ways of how to do things…”. The training also helped to confirm to trainees that they were “doing the right thing” or refreshed aspects of practice known and practiced for years. The training “took away assumptions” through providing guidelines and instructions for practice (e.g. dealing with difficult situations, documentation). It enhanced core skills (e.g. communications, cross-cultural competency) and clarified expected performance and quality standards.
The training set a standard for the work and basically your eyes are more open…you are more aware of what tasks you are undertaking…
The value of group work, where trainees learn from each other, was noted by trainees across all three types of training. Trainees also valued how the training process provided the opportunity to apply new knowledge and to rehearse new skills on-the-job and to learn from this through follow-up reflection in groups or in the classroom. Trainees considered the mix of complementary delivery methods used in the training as a key strength.
The evaluation findings show that Spectrum performed well on developing organisational and workplace environments that support the use of training. Spectrum’s commitment to training is shaping an environment of shared understanding and expectation where staff and management are increasingly ‘working from the same page’. Trainees described this environment as supporting their use of training in a variety of ways. These findings support the overall training strategy being implemented by Spectrum and the organisational commitment to learning and development. These inputs are contributing to the development of a learning-orientated culture within the organisation and an expectation and belief in performance enhancements.
The evaluation identified that a range of supports are available to trainees. However, findings across all three types of training raise questions about trainees’ access to continuous post-training supports, including coaching and mentoring and opportunities for on-going learning on-the-job. Time and workload pressures on staff expected to undertake coaching and mentoring roles may help to explain the findings. Staff in these roles may also require more specific training and support themselves. Spectrum is also in a transitional phase under the ‘outcome and active support’ philosophy. Because of this, some staff may not have sufficient opportunities to practice core skills while receiving the supports necessary to ensure on-going learning and development through this.
Trainees regarded the practice and reflection on practice learning cycle commonly used during training as key to their learning. The findings on training follow-up seem to be challenging Spectrum as to how this learning cycle can be more explicitly supported and continued. The applied use of most training requires the ability to adapt skills and knowledge in context. Follow-ups such as coaching and mentoring play a core role in building this capability.
Spectrum has further questions about whether the training is leading to intended practice and performance outcomes for trainees. Addressing these questions will require methodologies beyond the resources and scope of this evaluation. It will require a mix of methods (e.g. on-the-job assessments, observations) and ideally the integration of the assessment into routine performance appraisal.
As has been mentioned, Spectrum Care is now engaged in the next steps of its training effectiveness evaluation cycle. They have piloted a series of simple pre- and post- workshop tools designed to maximise ‘return on learning’ by supporting a partnership approach to learning and development between staff, managers, and the learning and research team. They are currently in the process of evaluating the value of these tools for initial learning goal setting, reflection on learning, and, application of learning within the workplace.
The organisation has a strong commitment to excellence and to upholding its person-centred vision and values. This suggests that the next steps in supporting training effectiveness may include more work around the development of a reflective disposition. As Vaughan et.al. (2014) state:
Reflection on personal performance has long been recognised as an important strategy to promote high-quality, deep learning and improve practice (Brockbank &McGill, 2007) – the kind now required for contemporary professional or occupational practice in a huge range of occupations. The process begins with critical reflection on existing perspectives to create new perspectives, discourse to validate critically reflective insights, and then action leading to learning (Segers & De Greef, 2011).
- Arthur, W., Jr., Bennett, W., Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11, 57 101.
- Baer, J.S., Rosengren, D.R., Dunn, C., Wells, E.A., Ogle, R., & Hartzler, B. (2004). An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence, 73, 99-106.
- Bennett, G. A., Moore, J., Vaughan, T., Rouse, L., Gibbins, J.A., Thomas, P., James, K., & Gower, P. (2007). Strengthening motivational interviewing skills following initial training: A randomised trial of workplace based reflective practice. Addictive Behaviors, 32, 2963-2975.
- Brinkerhoff (2005). The Success Case Method: A Strategic Evaluation Approach to Increasing the Value and Effect of Training. Advances in Developing Human Resources, 7 (1), 86-101.
- Brockbank, A., & McGill, I. (2007). Facilitating reflective learning in higher education (2nd ed.). Maidenhead, England Open University Press.
- Miller, W.R., & Mount, K.A. (2001). A small study of training in motivational interviewing. Does one workshop change clinician and client behaviour? Behavioural and Cognitive Psychotherapy, 29, 457-471.
- Roche, A.M., Pidd, K., Freeman, T. (2009). Achieving professional practice change: From training to workforce development. Drug and Alcohol Review, 21, 550-557.
- Roche, A. M. (2009). New horizons in AOD workforce development. Drugs: education, prevention and policy.16 (3), 193 - 204.
- Roche, A. M. (2002). Workforce Development Issues in the AOD Field: A Briefing Paper for the Inter-Governmental Committee on Drugs. Unpublished Report. National Centre for Education and Training on Addiction.
- Salas, E., Tannenbaum, S., Kraiger, K., & Smith-Jentsch, K. (2012). The Science of Training and Development in Organizations: What Matters in Practice. Psychological Science in the Public Interest, 13(2) 74 101.
- Segers, M., & De Greef, M. (2011). Transformational learning: the perspective of J. Mezirow. In F. Dochy,D. Gijbels,M. Segers, & P. v. d. Bossche (Eds.), Theories of Learning for the Workplace. Building Blocks for Training and Professional Development Programs (pp. 37-51). Abdingdon: Routledge.
- Vaughan, K., Kear, A., & MacKenzie, H. (2014). Mate, you should know this! Re-negotiating practice after a critical incident in the assessment of on-job learning. Vocations and Learning, 7(3), 331-344.
- Walters, S. T., Matson, S. A., Baer, J.S., & Ziedonis, D. M. (2005). Effectiveness of workshop training for psychosocial addiction treatments: A systematic review. Journal of Substance Abuse Treatment, 29, 283-293.
 See Spectrum Care’s Annual Report for more detail: http://www.spectrumcare.org.nz/media/other/SC-Annual_Report-2014-Condens...
|This work is published under the Creative Commons 3.0 New Zealand Attribution Non-commercial Share Alike Licence (BY-NC-SA). Under this licence you are free to copy, distribute, display and perform the work as well as to remix, tweak, and build upon this work noncommercially, as long as you credit the author/s and license your new creations under the identical terms.|