methodology.png
Tracking the secondary to tertiary education transition, methodology
Introduction

This article provides an overview of the methodology used in the analysis of secondary to tertiary education transitions, including how we track students who took part in secondary to tertiary programs, and how we determine the effect of those programs on their post-school choices.

Defining populations of interest

The main population of interest is those who took part in a secondary to tertiary program while at secondary school. For our analysis, we also need an appropriate comparison group of students who did not participate in the program. There are often unit standards associated with each secondary to tertiary program, and students who attempted these standards can be identified in Ministry of Education (MoE) data. For our analyses we considered only those who successfully completed one of these unit standards.

To find a suitable comparison group, we used MoE data for school leavers. This gives us information for everyone who left school each year since 2009. We only consider domestic students. We divide this group into those who have completed a unit standard relating to a specific secondary to tertiary program, and those who have not. We also track students who completed relevant unit standards and left school before 2009, however we do not use a comparison group for these students as there is no school leaver data available.

Determining outcomes

The main outcome we are interested in is whether a student who participated in a secondary to tertiary program went on to enrol with an industry training organisation (ITO). We use ITO data to determine which students were enrolled with an ITO each year after leaving school. The ITO enrolment data gives training start and end dates as well as whether the course was completed and a qualification obtained. We consider someone to be currently in training each year between their start and end dates (inclusive). Most of the time, those who are recorded as successfully completing a course are also recorded as having received a qualification and vice versa, however in a few cases one is recorded without the other. We consider someone to have successfully completed a course if either they are recorded as completing the course or receiving a qualification.

Training end dates are not always recorded in the ITO data. For recent data, this likely means that the training is not yet complete. There are however much older records with no end dates that need to be accounted for. We consider any record with no end date, a start date of more than three years ago, and no more recent start date to be incomplete training.

Measuring a program's effect

To accurately measure the effect a secondary to tertiary program has on a student's propensity to begin ITO training, we need to control for other factors that may influence both of these decisions. For example, male students are much more likely to both participate in the BCATS program and start an apprenticeship with BCITO. This will make it look like students who participate in BCATS are far more likely to start an apprenticeship. Without controlling for gender, we would therefore overestimate the effect of the program. To mitigate this problem we use a logistic regression model to determine which factors had a significant effect on starting training with an ITO.

We used enrolment with the ITO providing training related to the secondary to tertiary program as the dependent variable of the regression. We also limited the sample to those who went on to any kind of ITO training as we wanted to determine the specific effect on enrolment with the relevant ITO. The independent variable of interest was whether or not a student completed a unit standard from a particular program while at school.

BCATS-square.jpg

Evaluating BCATS

Analyse the effectiveness of BCATS in guiding students towards trades training

factory-factory-worker-girl-1108101.jpg

Evaluating the mechanical engineering gateway

Analyse the effectiveness of engineering gateway unit standards in guiding students towards trades training

capacitor-chip-circuit-163100.jpg

Evaluating the electrotechnology gateway

Analyse the effectiveness of electrotechnology unit standards in guiding students toward training