Assessment of Industry-Based Learning: A Self-Assessed Approach to Learning

Authors

Diana Ayling, Dave Hodges, and Prue Cruickshank, Unitec New Zealand

In this publication, a process to assess industry-based learning is described. The process consists of 8-stages and is guided by four principles: Self assessment, teacher validation, learning outcomes and student portfolio. This process utilises practice-based learning (including a portfolio approach) and has been shown to result in effective assessment of work-based learning and enhance learner self-assessment.

Introduction

Perhaps you are interested in developing learning or a version of practice-based learning and Portfolio assessment. Unitec is committed to "real world" learning, therefore practice-based learning, or ‘cooperative education’, is a key component of many Unitec programmes. In Unitec’s Bachelor of Business programme this commitment was provided through the Industry Based Learning (IBL) course.

The IBL course involved up to 50 students undertaking a work placement each semester. Students were required to spend approximately 140 hours in the workplace. Students reflected a wide range of ages, ethnicities and cultures, with a significant proportion having English as an additional language. The student group included new migrants, international, and domestic students. Students studied majors including accounting, finance, human resource management, marketing and operations management.

The key aspect to the IBL course was the approach to assessment. A Portfolio assessment model was introduced in 2007 not only to improve student learning, but also to ensure assessment issues of validity, reliability and fairness were adequately addressed. A unique feature of the Portfolio model was the incorporation of the perspectives of all stakeholders – the host (employer), the academic and the student – in the assessment process. The Portfolio was evidence-based, requiring students to self-assess their own workplace performance, learning and future development needs. The role of the academic staff member changed from assessment marker to assessment moderator. The self-determined and self-created nature of this was ideally suited for IBL students, who came from diverse backgrounds and cultures, as it enabled them to individually express their learning and achievement.

Industry Based Learning builds a bridge from student to worker.

Jasmin, Student

The eight stages in Figure 1, which describes the process of the Portfolio development based on workplace preparation and achievements, are explained below the figure.

Figure 1: The Industry Based Learning Process

Stages in Placement and Portfolio Development

Stage 1

The students are asked to write three personal and professional goals, and prepare a CV based on their current knowledge, experience, and skills. Their CV may be provided to a potential host employer to gain a placement.

Stage 2

Students must seek and find an appropriate workplace and supervisor who will oversee either their negotiated tasks or a project. The parameters and expectations of the work and the responsibilities of each person are clarified at an on-site meeting with the student, employer, and academic supervisor.

Stage 3

Students perform as agreed in the workplace and collect evidence of this for their Portfolios, including seeking feedback from their work colleagues, to prove the effectiveness of their efforts, while maintaining workplace confidentiality. Such evidence may be in the form of emails, notes, cards or photos. Evidence is required to support the achievement of the Learning Outcomes.

Stage 4

Learning Outcome (3) is about developing students as reflective practitioners who can evaluate and critically reflect on the project/work processes and outcomes within the context of the workplace environment. The first stage of developing the students as reflective practitioners involves the writing of 10 weekly reflective journals during the placement, which are emailed to their academic supervisor for feedback. The journals are included in the Portfolios as evidence of reflection-in-practice.

Stage 5

Learning Outcome (1) requires students to successfully complete an approved project or placement in the workplace; and (2) to participate effectively in the workplace. This requires evidence to demonstrate that the student has successfully completed the negotiated project or task, and has also participated effectively in the workplace. A collaborative meeting is held with the student, host, and academic supervisor to provide feedback on each aspect of the negotiated work. Each participant assesses the quality of work achieved before the meeting, then shares and discusses the outcomes. Evidence from this collaborative meeting is included in the Portfolios.

Stage 6

The second stage of developing students as reflective practitioners requires identification of what they have learned from their placement experiences. Students were expected to draw on the information they collected in their learning journals, together with the feedback from the collaborative assessment meeting, in order to ‘mull over’ and critically reflect on their workplace experiences. These further reflections also inform students’ future personal and professional development plans (see Stage 7). These reflections-on-practice provide further evidence of meeting Learning Outcome 3 for their Portfolios.

Stage 7

Learning Outcome (4) requires students to identify, implement, evaluate and critically reflect on a personal and professional development plan. Once their workplace experience has ended students are asked to update their CVs, incorporating the new knowledge and skills they have acquired, and setting new personal and professional goals. They must also demonstrate the achievement of the personal and professional goals they set at the beginning of their placement. All are included in the Portfolios.

Stage 8

Self-assessment, which was initiated at the beginning of the placement process by setting goals, continued through the Learning Journals, which culminated in reflection on practice, which in turn led to the re-setting of goals for the future, underpins the Portfolio philosophy. Students also had to self-assess the evidence produced for their portfolios to show whether they had met the requirements for a Pass or a Merit Pass. This was then subject to academic validation (moderation)

Learning Outcomes

1. Complete successfully an approved project or placement in the workplace.
2. Participate effectively in the workplace.
3. Evaluate and critically reflect on the project/work processes and outcomes in the context of the workplace environment.
4. Identify, implement, evaluate and critically reflect on a personal and professional development plan.

Support Provided to Students

Student Placements

Students obtained placements in a range of organisations in the greater Auckland area. The Unitec Career Centre supported students to find work placements. These placements ranged from small businesses to large organizations, and included private, public and community sectors. In the workplace, students were guided by a workplace mentor. This person had at least five years work experience and was prepared to foster and support the student at work. There was no expectation that students are paid for the work while on placement.

Academic Supervision

In addition to the supervision provided by the host mentor, Unitec provided an academic supervisor for each student. All academic supervisors were provided with training on course requirements and mentoring.

I got a lot of support from my academic supervisor…who would go through my journals with me, help me develop my goals and help me develop as a person.

- Adela, Student.

Workshops

In the previous semester of study preparatory workshops were held to help students seek and gain placements. In the semester of study four workshops were held to support student learning for the workplace, and learning for the requirements of the course and the assessment. The four learning outcomes provided the framework for the workshops. Additional workshops were held towards the end of the course to provide critical peer feedback on students’ draft portfolios.

Blackboard

An extensive Blackboard course supported this course, as students were off campus for the majority of the semester. Students had online access to support information for each learning outcome, assessment requirements, and further links to useful online resources. The course coordinator used the digital drop box, announcement, and email functions available on Blackboard to further support students while on placement.

The Portfolio has been a valuable assessment tool as a supervisor, because it has given me a fantastic overview of what the student has experienced in the workplace.

- Denisa, Academic Supervisor

Purpose and Benefits of Assessment

Portfolios are becoming a popular assessment tool in cooperative education and practice-based learning courses because they are able to capture the complex learning and personal development students experience. The Industry Based Learning course (IBL) was developed to embrace these complexities and to recognize the variable influences on learning students encounter in the workplace. These variables included: the length, structure and work objectives of the placement; the quality of the support and mentoring provided by academia and the workplace; the contextual and socio-cultural influences of the workplace; and the background, culture, experience, competencies, motivation, goals and aspirations of the student.

Portfolios combine theory and practice, and are used in diverse disciplines to foster student responsibility for learning and professional development. While portfolios are useful at advanced study levels, they can be used anywhere in a learning programme. Portfolio models encourage deep learning through reflection and self-evaluation; they underpin the autonomous learning required by self-regulating professionals.

Portfolios can provide students with:

• A powerful sense of achievement
• A greater understanding of their strengths and areas for future development
• The freedom to express what they believe they have learned in their own way
• A vehicle to capture and document their unique learning experiences
• A framework for the collection of evidence of their learning from multiple sources
• An opportunity to use their creativity and originality to present evidence of their performance and learning.

The IBL Assessment Model

The Portfolio is evidence-based and takes a holistic approach to assessment and learning by making overt connections between each of the course’s four learning outcomes and also between formative assessment and summative assessment. It requires students to take responsibility for their learning, development and performance, with host employers and academics taking mentoring and moderating roles.

The theoretical framework and basis for the Portfolios, is described in more detail by Hodges and Ayling (2007). A graphical overview of the model is shown in Figure 2, followed by a brief explanation of each of the model’s four phases. 

 Figure 2: The Model

Phase One - Determine Goals, Objectives and Stakeholder Responsibilities

The four-phase model (Figure 2) recognizes the experiential nature of students’ workplace learning and can be viewed as cyclical. The cycle started before the placement, in the planning phase, when a standard ‘learning agreement’ was drawn up. This agreement specified the work objectives and the responsibilities of each of the three key parties involved in the placement (the workplace supervisor, the academic supervisor and the student). In addition, students were required to produce an updated curriculum vitae (identifying perceived strengths and competencies) and set personal and professional learning goals for their placement.

Phase Two – Learn by Doing and Reflecting

When participating in their placement students were expected to clarify employer performance expectations, linked to their work objectives, and collect evidence to demonstrate achievement of these. Typically, one of the key strategies students adopted was to set up regular, structured meetings with their workplace supervisor. Gaining regular feedback and support was intended to help students to know how they were progressing and allowed them to make any adjustments needed during their placement.

Students kept a weekly learning journal to capture key learning experiences and enable regular critical reflection. The learning journal was used as the basis for the regular meetings students have with their academic supervisors. This encouraged further reflection, and ‘deeper learning’. In addition, academic supervisors had periodic conversations with the host mentors. These ‘long conversations’ provide on-going support and formative feedback to students on their progress, development and achievement. It also ensured there were ‘no surprises’ in the evaluation phase.

It has made me reflect on the things that went wrong, the things that went right, and overall it has given me a broader perspective of where I am currently in this stage of my career.

- Stevan, Student.

Phase Three – Review Performance

At the end of the work placement each of the three parties involved completed a standard ‘collaborative assessment’ form. The form has two components – student achievement and areas of focus for future development. Comments were made for each component within four broad competency dimensions:

• Professional (e.g. client focus, use of initiative, standard of presentation)
• Interpersonal (e.g. ability to work with others, express ideas, communicate effectively)
• Intellectual (e.g. ability to apply theory to practice, find solutions to work problems)
• Project/time management (e.g. self-organization, meeting deadlines).

The basis of these four dimensions of competency originates from the work of Hodges (2006) and Hodges and Burchell (2003). A fifth dimension of the form covers the ‘value of the work completed’ (i.e. the value of the work to the organization).

The completed forms were the foundation for a three-way dialogue, between the student, host and mentor. The process produced a broad consensus on the student’s performance and future development needs. The performance and development outcome was largely formative, as all students were expected to meet a minimum satisfactory level of performance.

Phase Four – Achieve Learning Outcomes

The fourth phase had two interconnecting components – critical reflection and personal and professional learning goals.

Students identified what they have learned from their placement experiences through critical reflection. Students were expected to draw on the information they collected in their learning journals and feedback from the collaborative assessment meeting, and also to ‘mull over’ and reflect critically on their workplace experiences. The latter overall reflections were expected to address three components of their work experiences:

• General learning experiences
• Personal learning experiences
• Workplace culture, values, and practices.

Students explicitly identified the extent to which their workplace experiences helped inform their future personal and professional development needs. Future career preferences and directions may have been clarified. This future-focus contributed to learning and included:

1. Personal and professional learning goals set for the work placement. Students self-assessed their achievement of their initial self-set goals.
2. Summary of key skills and competencies demonstrated and developed during placement. Students briefly explained with supporting evidence, of how, where and when relevant skills/competencies were developed and demonstrated. This information was integrated into a new CV.
3. Student reflections. Students provided reflections of these experiences, to identify a new set of learning goals. For each goal students provide a set of strategies they intended to adopt in order to achieve it, together with ways of identifying achievement.

Assessment and Grading

As the portfolios are evidence-based, responsibility for demonstrating achievement of the four learning outcomes rested with the student. This followed a self-assessment model, with academics taking a moderation role rather than a marking role, by moderating (or validating) student’s self-assessment. Academics validated the portfolios of students they did not mentor, ensuring separation of roles and avoiding any potential conflicts of interest.

Before submitting their portfolios for moderation students were encouraged to use a peer-critiquing workshop process. By submitting their portfolios students were indicating they had self-assessed their portfolio and minimally considered they had met the criteria for a ‘pass’, i.e. they have produced sufficient evidence to demonstrate achievement of the four learning outcomes. Criteria for a ‘merit pass’ required evidence of outstanding performance as well as a higher level of critical reflection.

Success of the IBL Portfolio Assessment

To gauge the success of the portfolio assessment model in terms of the benefits to student learning, stakeholder feedback was collected through quantitative and qualitative means over a two semester period. The mixed methods approach to the data collection included:

• A questionnaire survey of 28 host employers at the end of Semester One (71% response rate)
• A questionnaire survey of 28 students at the end of Semester One (79% response rate) and 33 students at the end of Semester Two (61% response rate)
• In-depth interviews with six students
• Three focus groups involving nine academic supervisors
• A review of a sample of six completed student portfolios.

The findings showed that the implementation of the portfolio assessment had strong support from all parties involved – the students, the academics, and the host employers. While learner benefits were diversified and broad in scope, some common themes were apparent. These are briefly summarised below – for full details of the findings see Hodges (2010):

• The formative feedback provided through the ‘long conversations’ during the placement period enabled students to gain a good understanding of their workplace performance. Students felt this enhanced their confidence in self-assessing their performance before the collaborative assessment meeting;
• Previously, marks were allocated to student workplace performance at the collaborative assessment meeting. Removal of such marks in the portfolio model forced greater emphasis on qualitative feedback. Evidence suggests this enabled students to gain a more in-depth understanding of their performance and professional development needs;
• The portfolio model was developed to empower students to take responsibility for their own learning and to determine achievement of outcomes and professional development needs. This mirrors the reality of being a self-regulating professional in the workplace. In relation to the self-assessed grades, a review of the final grades awarded following moderation showed that nearly 80% of students were reliably able to self-assess the adequacy of their portfolio evidence to demonstrate achievement of the learning outcomes (either at a Pass or Merit Pass level);
• A likely contributor to students’ ability to reliably self-assess their performance reliably was their firm belief that they had increased their self-confidence. Given their increased awareness of those competencies considered important in the workplace, it is likely that students also increased their self-efficacy;
• By reflecting on their experiences students were able to identify a range of important learning experiences, both personal and professional. For example, the weekly learning journals enabled Aija* to express her feelings and emotions that she found helpful when later identifying important learning experiences. A common feature of students’ learning journals was their gradual coming to know. They moved from descriptive writing in the early weeks, to reflective writing in the later weeks, as they began to question their experiences and take greater notice of things around them;
• The portfolio enabled diversely experienced students, many of whom have English as an additional language, to express their views, feelings, and thoughts. A common feature of such expressions was the noticeably informal aspects of workplace culture, particularly the communicative aspects of daily work life. For Jiao*, the informality of how the ‘boss’ and the staff interact was a surprise: “In China the boss is like [at a] higher level than the employee and they just really sit in [their] office and not really talk to the staff outside”. The main surprise for Jing* was the importance of humour at work: “They make lots of jokes and things, and they would laugh and laugh ... and I don’t think it’s really funny. I understand the words, I understand what they’re saying, but maybe there’s something behind the language itself. Whether it’s a culture thing I don’t know, but yeah, I think it’s a barrier between me and these guys”.

  * A pseudonym

Student and Staff Perspectives

Students and staff speak about their experiences in the course and their perspectives on self-assessment of portfolios:

Assessment of Industry Based Learning Final Edit
Uploaded by emckee. - Explore lifestyle, fashion, and DIY videos.

References

• Hodges, D. (2006). A qualitative pilot study of New Zealand employers' views on the importance and measurement of business graduate competencies. In R.K. Coll (Ed.), Proceedings of the Ninth Annual Conference of the New Zealand Association for Cooperative Education: Using cooperative education and work-integrated learning to develop lifelong learning skills (pp. 45–53). Hamilton, New Zealand: New Zealand Association for Cooperative Education.
• Hodges, D. (2010). Assessment of student learning in a business internship. Unpublished PhD thesis, University of Waikato, Hamilton, New Zealand.
• Hodges, D., & Ayling, D. (2007). A model of learning: Reframing assessment practices in a business co-operative education course. In R. K. Coll (Ed.), Proceedings of 10th New Zealand Association of Cooperative Education Conference: Developing Work-integrated Learning in Cooperative Education and Internship Programmes (pp. 49–56). Hamilton, New Zealand: New Zealand Association for Cooperative Education.
• Hodges, D. & Burchell, N. (2003). Business graduate competencies: employers’ views on importance and performance. Asia-Pacific Journal of Cooperative Education, 4(2), 16–22.