Student engagement in New Zealand’s universities

Edited by Ali Radloff - Australian Council for Educational Research (ACER)

Published by Ako Aotearoa and the Australian Council for Educational Research.

Student Engagement in New Zealand Universities was launched in November 2011 at the Tertiary Education Research in New Zealand (TERNZ) Conference at Victoria University of Wellington. Dr Peter Coolbear, Director of Ako Aotearoa launched the report before an audience which included a number of the authors.

Left to right: Keith Comer of University of Canterbury, Stephen Marshall of Victoria University, Erik Brogt of University of Canterbury, Ineke Kranenburg of AUT University and Malcolm Rees of Massey University.

Introduction

• The construct of student engagement
• AUSSE background and methodology
• Students at New Zealand’s universities
• Overall findings for New Zealand universities
• Recommendations based on the findings presented

The number of students commencing undergraduate study at New Zealand’s universities has been steadily increasing over the last decades (Ministry of Education, 2010a). While in 1991 only six per cent of New Zealanders aged 15 or older held a bachelors degree or higher qualification,
by 2009 this had increased to 17 per cent (Ministry of Education, 2010b). Over 100,000 domestic and international students are currently enrolled in bachelor degree study at one of New Zealand’s eight universities and the numbers of students enrolled has increased 17 per cent between 2002 and 2009 (Ministry of Education, 2010a).

Although the numbers of students entering bachelor level study in New Zealand is growing and is high relative to OECD averages, the number of students leaving with a qualification is low compared to many other countries (Scott & Gini, 2010). Data from the Ministry of Education show that completion rates are relatively low, particularly among certain groups of students. Around one-third of students who began a bachelor degree in 2002 had not completed the degree, or a degree at the same or a higher level, eight years after starting (Ministry of Education, 2010c).  Completion rates are higher among Asian New Zealand students, but much lower among Māori and Pasifika students (Ministry of Education, 2010c). Eight-year completion rates are also much higher among students studying full-time (80%) than students studying part-time (52%) (Ministry of Education, 2010d).

It is clear that holding a bachelor or higher degree is valuable for an individual. New Zealanders with a bachelor or higher degree are more likely to be employed, with 82 per cent of the population holding a bachelor or higher qualification employed either full-time or part-time, compared with 63 per cent of the overall New Zealand population (Ministry of Education, 2010e). Only 2.5 per cent of New Zealanders with a bachelor or higher degree are unemployed, compared with 3.4 per cent of the overall population (Ministry of Education, 2010e). New Zealanders with a bachelor degree level qualification also earn on average around 60 per cent more than those with only a school-level qualification (Ministry of Education, 2010f). Increasing the proportion of the population that holds a bachelor or higher degree is not just beneficial for the individuals with the qualification, but also contributes to the overall economy of New Zealand by providing highly skilled workers.

As well as concerns surrounding the high numbers of students entering tertiary education but leaving without completing a qualification, concerns exist about low labour productivity relative to the qualification levels of the New Zealand population (Earle, 2010). As demand for highly skilled workers is increasing in New Zealand and the economy requires more people to have better skills, there are concerns about the quality of education people undertaking tertiary instruction receive (Earle, 2010). Enhancing the quality of tertiary education in New Zealand will address potential future skilled-worker shortages by reducing student attrition, improving the education students are receiving, and helping students graduate ready for employment. To improve the quality of education received by students studying at New Zealand’s eight universities, there is a need to understand not only students’ attrition and completion rates but also the way in which students are learning and engaging in their study – aspects of students’ experience at university that are intrinsic to their success.

The Australasian Survey of Student Engagement (AUSSE) provides data that higher education providers throughout New Zealand and Australia can use to attract, engage and retain their students. Through measuring the time and effort students devote to educationally purposeful activities and other aspects of their experience at their institution the AUSSE provides a greater understanding of students’ engagement with study and their learning. Instead of focusing on retention and completion rates, or upon student ratings of satisfaction with their education, the AUSSE focuses on the way in which students learn and on the outcomes they achieve. Having information about the way in which students are learning and their self-perceived outcomes
allows institutions to gain a better understanding of the quality of education students are getting. Collecting this sort of information allows higher education institutions to understand what really counts in terms of quality.

The AUSSE is an annual survey managed by the Australian Council for Educational Research (ACER) in cooperation with participating tertiary education providers in Australia and New Zealand. The AUSSE builds upon a decade of development that has been done by the North American National Survey of Student Engagement (NSSE), which has been run for over a decade in the USA and Canada. The NSSE has been administered at more than 1,300 institutions throughout North America, and methodologies and research foundations developed in the NSSE have laid the foundations for the AUSSE.

The AUSSE was first run in 2007 within 25 institutions, and participation has grown each following year. In 2008, 29 institutions participated; in 2009, 35 institutions in Australasia; and in 2010 that increased to 55 institutions – including universities, TAFEs, private higher education providers, and institutes of technology and polytechnics. By providing information that is generalisable and sensitive to institutional diversity, and with multiple points of reference, the AUSSE generates information that institutions can use to monitor and enhance the quality of education. The AUSSE surveys students who are currently at two points of their higher education journey – in their first year of study and during a later-year of study (usually third-year of a bachelor level qualification).

Although in recent years more and more research has focused on student engagement  worldwide, little focus has been given to the engagement of students studying at New Zealand’s universities. Information on the engagement and outcomes of students studying at New Zealand universities can be used to better understand the sector, identify areas where improvements can be made and to celebrate students’ successes.

This particular report explores student engagement among students studying at New Zealand’s eight universities, and focuses on student groups that are of particular interest to the New Zealand higher-education sector, such as Māori and Pasifika students, students studying via non-traditional modes (such as part-time or extramurally), and international students. Other chapters in this report focus on student workload, differences in engagement between male and female students, students studying in different fields, and students’ departure intentions.

Using the most recent results available at each of the eight New Zealand universities participating in the AUSSE from 2007 to 2009, this report provides an overview of the university sector and some answers to questions about students’ experience of university and how they are learning.

The construct of student engagement

‘Student engagement’, which can be defined as students’ involvement with activities and conditions that are likely to generate high-quality learning, is increasingly seen as important for positive learning outcomes. The concept of student engagement provides a practical lens for assessing and responding to the significant dynamics, constraints and opportunities facing tertiary education institutions. Measuring student engagement provides key insights into what students are doing, which helps provide information that can be used to enhance students’ experience and generate continued improvement in school systems.

While student engagement is now seen as vital to quality tertiary education, information on student engagement has not been readily available to Australasian tertiary education providers until very recently. Prior to 2007, when the AUSSE was first run in New Zealand and Australia, existing data collections and surveys tended to focus on student satisfaction, quality of teaching and other aspects of students’ experience at their institution. Now that the AUSSE is being used by many institutions, there is an increased ability to understand students’ engagement, and  institutions have more information on what matters for their students’ experience.

Student engagement is an idea that specifically focuses on students and their interactions with their institution. While the concept has previously been considered behaviourally in terms of ‘time on task’, contemporary perspectives now touch on aspects of teaching, the broader student experience, learners’ lives beyond the classroom, and institutional support. Students lie at the heart of conversations about student engagement – conversations that focus squarely on enhancing individual learning and development.

In short, measures of student engagement provide information about individuals’ intrinsic involvement with their learning, and the extent to which they are making use of available educational opportunities. Such information enhances knowledge about learning processes, can be a reliable proxy for understanding students’ learning outcomes and provides excellent diagnostic measures for learning enhancement activities.

The AUSSE explores six areas of student engagement. These include things that are related to students’ institutional support as well as their involvement in certain types of educational activities. Table 1 details these six scales.

Table 1: AUSSE engagement scales

In addition to measuring student engagement, the AUSSE also measures several general and learning outcomes. The seven outcome measures in the AUSSE focus on broader forms of learning and development. These outcome measures are described in Table 2.

Table 2: AUSSE outcomes measures

AUSSE background and methodology

The AUSSE measures student engagement through administration of the Student Engagement Questionnaire (SEQ) to a representative sample of students at each institution. With formative links to the NSSE, the AUSSE provides data that complement and extend current collections that focus on satisfaction with teaching and support. It makes available to higher education institutions a new means for measuring and monitoring the effectiveness of learning and teaching.

The SEQ is based on the College Student Report, the instrument used at over 1,300 North American institutions that participated in the NSSE. The SEQ is designed for administration to undergraduate students in under 15 minutes, either online or in paper form. The same SEQ content is provided to all students. To manage and reduce levels of item-level non-response, sampled students were randomly distributed one of three different online versions, each containing different rotated orderings of the items. All students who submit an online form are presented with an overview of student engagement, a summary of key findings, and information about what institutions have done with the results.

ACER further developed and validated the College Student Report before deploying it in Australia and New Zealand. Validation included item design and development, focus groups, cognitive interviews, pilot testing and expert review. A range of psychometric and conceptual analyses was conducted. This work builds on the extensive validation undertaken in the USA. The SEQ will further develop with ongoing development of the AUSSE. Evolution of the instrument depends on evidence of the kinds of engagement that are linked with high-quality learning outcomes.

The cross-national comparisons facilitated by the AUSSE are important. While tertiary education is an increasingly internationalised activity, data limitations have to date constrained comparative analyses. Specifically, very little student-level and process- or outcomes-focused data is available. Through its links with the NSSE, the AUSSE represents a trend towards developing more educationally nuanced cross-national collections and interpretations.

When analysing the AUSSE item and scale statistics, various different technical perspectives could be and have been used in this report. Statistical significance, correlations and effect size are among some of the statistical techniques employed by authors in the chapters to interpret the data. Given the relatively large size of the sample and the magnitude of the scale standard deviations, using statistical significance alone can be somewhat misleading. With large samples, such as those used in this particular report, even small differences between groups can be statistically different. In these cases, a statistical difference does not necessarily indicate that a difference between two groups is meaningful or is of practical significance.

In order to determine the practical significance of differences between groups when using the AUSSE data, a ‘rule of thumb’ can be adopted and is utilised by many of the authors in this report to pin-point meaningful differences between groups. A scale score or percentage difference of five or more points on the reporting metric is likely to be both ‘statistically significant’ and indicate there is a meaningful difference between two or more groups’ results.

Including different types of analysis in this report provides different perspectives. The types of statistical analyses and figures presented by the authors of the chapters in this report are varied, but reflect the number of different ways in which the data can be analysed. Analyses presented in this report include frequencies and mean responses for particular groups of students, significance testing, and effect–size calculations; however, there are many other types of analyses that could be used to interpret and explore the findings from the AUSSE.

Students at New Zealand’s universities

For this particular report, data from each university’s most recent AUSSE administration were merged into a single data file to provide an overall New Zealand AUSSE data file, which represents all universities in New Zealand rather than all New Zealand universities that participated during a particular year. This data file included the data from all eight New Zealand universities and includes data from the 2007, 2008 and 2009 administrations of the AUSSE. The administration year of the data included in the file for each New Zealand university is summarised in Table 3.

Table 3: New Zealand universities results included in report

To ensure confidentiality of university responses, only staff at ACER had access to the combined New Zealand universities AUSSE data file. All analyses involving the use of this data file were conducted by ACER, and no analyses identified individual universities. Overall statistics, which included scale and item level statistics for particular student subgroups, were provided to all authors during a two-day meeting in Wellington in September 2010. Additional analyses requested by authors were also conducted by ACER.

The data file used for the analyses in the following chapters includes responses from 8,378 undergraduate first- and later-year students who completed the AUSSE survey at their university in 2007, 2008 or 2009. These include 4,223 first-year students and 4,155 later-year students.

Table 4 summarises the individual demographic characteristics of students at New Zealand universities and Table 5 provides a summary of these students’ educational contexts and backgrounds.

Table 4: Demographic characteristics of secured New Zealand response

Table 5: Educational characteristics of secured New Zealand response

Post-stratification weighting of AUSSE responses is used to ensure that responses represent the target population. As far as possible, given available information, AUSSE data are weighted within institutions for year level, attendance type, and sex.

Overall findings for New Zealand universities

The AUSSE findings provide information that New Zealand universities can use to better understand what their students are doing, and where improvements could be made to better ways in which students engage with their studies. Findings from the AUSSE can also be benchmarked internationally with responses from Australian university students who also took part in the AUSSE, South African university students who participated in the South African Survey of Student Engagement (SASSE), and USA undergraduate students who participated in the National Survey of Student Engagement (NSSE).

Comparing responses from New Zealand undergraduate university students with those of students in other countries reveals that compared to the USA, New Zealand students are far less engaged in their studies during both their first year and later year of study. Looking closer to home, it appears that when compared with undergraduate students from Australian universities, New Zealand students are doing a little better. As shown in Figure 1 and Figure 2, New Zealand students report very similar levels of engagement as their Australian peers. Two areas where there are significant and meaningful differences are between first-year students’ engagement in active forms of learning (d=0.31) and involvement in work-integrated forms of learning (d=0.26). Although New Zealand students’ involvement in work-integrated learning increases significantly between first- and later-years of study, later-year students studying at Australian universities continue to report significantly higher involvement in these types of activities (d=0.30).

A total of 12.8 per cent of New Zealand first-year students say that they ‘never’ ask questions or contribute to discussions in class or online. This is more than twice the proportion of Australian first-year students (5.6%). By later year, 40.7 per cent of New Zealand students report asking questions or contributing to discussions frequently, however this is still significantly lower than among Australian students (55.2%). New Zealand students are also far less likely to make a presentation. While 26.4 per cent of first-year and 16.4 per cent of later-year Australian students say that they have ‘never’ given a presentation in class or online, nearly half of New Zealand first-year students (48.4%) and a quarter of New Zealand later-year students (23.7%) report that they have ‘never’ made a presentation.

New Zealand students are also less likely to work with students during class, and to a lesser extent outside of class. However, New Zealand and Australian undergraduate students are just as likely as each other to have tutored other students or participated in a community-based project as part of their study. New Zealand students are slightly more likely to frequently discuss ideas from their classes with others.

While New Zealand students’ involvement in work-integrated learning activities increases significantly from first- to later-years of study (d=0.42), Australian students are far more engaged in work-integrated types of learning than New Zealand students. By later years of study, 22.1 per cent of New Zealand students have participated in an industry placement or work experience; among Australian later-year students, 31.4 per cent have done this. Australian students are also more likely to feel that their experience at university has contributed at least quite a bit to their development of job- or work-related knowledge and skills (73.0%) by their later years of study than New Zealand later-year students (67.1%). Australian students are also more likely to frequently explore how to apply their learning in the workforce, to develop discipline-relevant communication skills and to improve knowledge and skills that will contribute to their future employability.

Another aspect of the student experience that is measured by the AUSSE is students’ perceived outcomes from their university experience, including students’ average grade, the types of thinking that their coursework emphasises, development of general learning skills, personal development, career readiness, satisfaction with their experience and non-graduating students’ departure intentions.

Unsurprisingly, there appears to be a relationship between the length of time a student has been at university and their perceived outcomes of study. Later-year students generally report better outcomes than first-year students, although they are less satisfied than first-year students. This is particularly the case for higher order thinking (d=0.24), general learning outcomes (d=0.32) and career readiness (d=0.28).

There are fewer differences between New Zealand and Australian university students’ perceived outcomes; however, New Zealand students report significantly lower levels of career readiness than Australian students during both first-year (d=0.24) and later-year (d=0.26) study. Furthermore, quite large proportions of both Australian and New Zealand students do not feel fully prepared for future careers. Rather high proportions of Australian (37.9%) and New Zealand (47.6%) students report never keeping their resume up-to-date. A total of 40.1 per cent of New Zealand students reported never having networked to find job opportunities, 30.1 per cent have never set career development goals or plans, 28.6 per cent have never thought about how best to present themselves to potential employers, and 26.0 per cent have never explored where to look for jobs.

One area of great concern in New Zealand particularly is students’ departure intentions. Given the relatively high rate of attrition from tertiary education and low completion rates internationally, exploring students’ departure intentions can provide an interesting insight into the reasons why many students leave, and can help universities increase retention and subsequently student success. Australian and New Zealand students report similar levels of departure intentions, suggesting that the issue of retaining students in study is one that is not unique to New Zealand universities. Overall, 29.4 per cent of New Zealand university students have seriously considered or plan to leave their current institution prior to completing their qualification. Among New Zealand students with departure intentions, reasons given for considering leaving included convenience or practical reasons (27.2%), for academic reasons (26.8%), to improve career prospects (25.9%), for financial reasons (24.4%) and to obtain a better quality education (17.2%).

Although quite a number of New Zealand university students have seriously considered leaving, most of these students plan to stay on at university next year and continue with their current study (72.7%) or leave university having completed their qualification (16.1%). A smaller proportion of students plan to shift to a different qualification (14.2%) or shift to another university (17.8%), while only a small number of students who have seriously considered leaving plan to move to vocational education and training (3.2%), or leave university before finishing their qualification (7.3%).

While nearly one-third of New Zealand’s university students have seriously considered leaving their university before completing their study, students are generally very satisfied with their experience at university. A total of 78.8 per cent of first-year and 74.8 per cent of later-year students rated the quality of academic advice received as ‘good’ or ‘excellent’. A further 85.0 per cent of first-year and 82.3 per cent of later-year students were satisfied with their overall educational experience and rated it as ‘good’ or ‘excellent’. The vast majority of New Zealand university students also indicated that given the chance to start over, they would attend the same university again (89.1%).

Figure 1: Student engagement scale scores among first-year students

Figure 2: Outcomes measures scale scores among first-year students

Figure 3: Student engagement scale scores among later-year students

Figure 4: Outcomes measures scale scores among later-year students

Recommendations based on the findings presented

The chapters presented in this report provide a more in-depth look at student engagement in New Zealand universities, focusing on specific student groups of interest and certain aspects of the student experience. Through sharing information and results with other universities and learning about different ways to improve the student experience, New Zealand universities will be able to effectively enhance students’ engagement with learning, and increase students’ success.
Overall, the chapters suggest that providing university students with support in their studies is vital to ensuring student success.

Māori and Pasifika students

Māori and Pasifika students are of particular interest in the New Zealand university sector, and while the numbers of students enrolling in university study is increasing there are still widespread concerns about their high level of attrition and low completion rate relative to other students. Results from the AUSSE show that Māori and Pasifika students have a similar level of engagement with their studies to other students and also report similar outcomes overall; however, they are more likely to have difficulty keeping up to date with their study and more likely to have seriously considered leaving than other students.

For both Māori and Pasifika students, a key relationship emerges between support provided by their university, frequent and high-quality interactions with academic staff, and positive student outcomes including satisfaction with their educational experience, departure intentions, general development and learning skills. Although this relationship appears for all New Zealand university students, among Māori and Pasifika students this relationship is stronger. This suggests that providing greater support through programmes such as Peer Assisted Study Sessions (PASS) may help Māori and Pasifika students feel greater support from their institutions and peers, and may also assist them in keeping up to date with their studies.

Field of study

There is a great amount of variation in the way in which students in different fields of education are engaging in their study. Overall, students in many fields are reporting low levels of engagement with work-integrated forms of learning and low career readiness. Also, students in many fields of study reported relatively low levels of involvement in active forms of learning,  particularly in terms of giving presentations.

While engineering students reported the lowest departure intentions, they were also the least likely to contribute to discussions in class or online, but at the same time reported frequently working with other students during and outside of class. Architecture students on the other hand, while strongly engaged in active forms of learning, reported the highest departure intentions of all fields of study. These examples show just some of the findings that reflect traditional academic disciplinary and curricular differences between the disciplines.

Sex

Although male students are less successful than female students in terms of their pass rates and successful and timely completion of qualifications, with a few exceptions, there are very few meaningful differences in the way in which male and female students engage in learning. Female students are somewhat more engaged than male students in academically challenging activities and enriching educational experiences, while male students are more engaged in certain active forms of learning such as working with other students during and outside of class and tutoring other students.

International students

International students are coming to New Zealand to study in increasing numbers. A lower proportion of international students drop out of study, and more complete within eight years. There are no large differences in engagement and outcomes between international and domestic students; however, international students report slightly higher interactions with staff, a greater focus on work integrated learning, and career readiness. International students are also slightly more likely to be involved in enriching educational experiences, and to feel that their experience at university has contributed to their personal development.

International students are slightly more engaged with learning than domestic students but also have somewhat higher departure intentions and lower rates of satisfaction. International students are also significantly less likely to have frequent interactions with students from a different background or of a different ethnic group and rate their relationships with other students more poorly than domestic students. A clear link emerges between international students’ relationships and interactions with other students and their departure intentions and satisfaction.

Work and study balance

It is clear that university students in New Zealand, like those in other countries, are not spending sufficient time on their studies or in classes. Students who spend no time preparing for class are far less engaged in many areas of learning than students who spend a substantial amount of time studying. As more and more students balance work with university study, there are concerns that employment is interfering with students’ success at university. Results from the AUSSE suggest that students who are working for pay for 30 or more hours in a typical week are significantly less engaged with their studies; however, no negative effect appears for students who report working for pay for up to 25 hours a week.

Departure intentions

With 17 per cent of university bachelor degree students dropping out of their study during or immediately following their first-year of university, and nearly one-third of students not completing their bachelor degree within eight years of first enrolling, it is vitally important that we understand the various reasons students leave their study and how attrition can be mitigated.

Around 29 per cent of New Zealand university students have seriously considered leaving their current institution or plan to leave by the following year. The primary reason given by these students centres on practical reasons or reasons to do with convenience. Although many practical reasons may be outside the control of a university, some relationships between departure intentions and students’ engagement and outcomes suggest that more could be done to mitigate student departure intentions and therefore their attrition from study.

A strong relationship emerges between students’ satisfaction with their educational experience, academic advice received and students’ departure intentions. There also appears to be a relationship between departure intentions, student grade, support provided by the university, and general learning skill development.

Part-time students

Part-time students have much lower completion rates than full-time students. For this reason it is interesting to explore whether the way in which part-time and full-time students are engaging with their study and university is different and whether this may be affecting their intentions to depart and actual attrition from university. While data from the Ministry of Education show that part-time students are more likely to drop out of study, no meaningful difference emerges between part-time and full-time students’ departure intentions from the AUSSE data. However, part-time students are much more likely to cite financial reasons as being one reason for seriously considering leaving.

Part-time students have fewer opportunities to interact with other students, and have lower ratings of the quality of their relationships with other students than full-time students.

Extramural students

An increasing number of students are studying extramurally, but little is known about the differences in how these students are learning due to their different location of attendance.  Although extramural students have fewer opportunities to engage in certain learning activities, such as interacting with fellow students and academic staff in traditional ways, students studying extramurally are actually more engaged overall than their peers studying on-campus.

References

• Earle, D. (2010). How can tertiary education deliver better value to the economy? Wellington: Ministry of Education, Tertiary Sector Performance Analysis and Reporting Division.
• Ministry of Education (2010a). Domestic and international students enrolled by qualification level and sub-sector 2002–2009, ENR.10, Ministry of Education, Wellington, viewed 10 March 2011, Education Counts, <http://www.educationcounts.govt.nz/statistics/tertiary_education/participation>.
• Ministry of Education (2010b). New Zealanders aged 15 and over by highest tertiary qualification level 1991–2009, EAP.12, Ministry of Education, Wellington, viewed 10 March
  2011, Education Counts, <http://www.educationcounts.govt.nz/statistics/tertiary_education/retention_and_achievement>.
• Ministry of Education (2010c). Eight-year qualification completion rates for domestic students by sub-sector, ethnic group, full- or part-time, period of study, and qualification level, COM.35, Ministry of Education, Wellington, viewed 10 March 2011, Education Counts, <http://www.
  educationcounts.govt.nz/statistics/tertiary_education/retention_and_achievement>.
• Ministry of Education (2010d). Eight-year qualification completion rates, LNR.6, Ministry of Education, Wellington, viewed 28 March 2011, Education Counts, <http://www.educationcounts.govt.nz/statistics/tertiary_education/provider_summary>.
• Ministry of Education (2010e). Distribution of the working age population by labour force status, highest qualification and ethnic group 2006 – counts, PSE.1, Ministry of Education,
  Wellington, viewed 10 March 2011, Education Counts, <http://www.educationcounts.govt.nz/statistics/tertiary_education/life_after_study>.
• Ministry of Education (2010f). Region-wise distribution of weighted average of earnings of New Zealanders aged 15 and over by highest qualification 2006, PSI.1, Ministry of Education, Wellington, viewed 10 March 2011, Education Counts, <http://www.educationcounts.govt.nz/statistics/tertiary_education/life_after_study>.
• Scott, D. & Gini, P. (2010). How does New Zealand’s education system compare? OECD’s Education at a Glance 2010. Wellington: Ministry of Education.