Good morning. Good morning. Let's just see. Welcome Sevilla and Anna-Marie to workshop number four, the economic and management science at UFS, mapping out the student life cycle and evidence-based decisions. It's 10 o'clock. It seems like everybody's here and I'm sure we can go ahead. It's all yours Anna-Marie. Thank you so much Carmelita and Elisabeth. Sorry, just wanted to stop my video this side. Yes, there I am. I just want to say yes, thank you. Thank you for this wonderful opportunity. My name is Anna-Marie Miller. I'm from the faculty of economic and management sciences from the University of the Free State. I'm the teaching and learning manager here. But we want to thank you for this opportunity to just share with you some of the data analytics, learner analytics that we are currently busy with. And you know, to have you join in this conversation and have us even ask the opportunity to join in the conversation is such a privilege. So as you saw, we are presenting on mapping out the student life cycle and making evidence-based decisions. So there we go. All right. So the facilitators, the workshop today will be presented by myself. I'm the teaching and learning manager at the EMES faculty. But also our data analyst in the faculty, Mr. C. E. Lane-Zamini, you will be joining me and being a co-presenter in this workshop. So just to give some context, we are basically the teaching and learning office. We have some other colleagues here as well in our office. But we are a subset of the dean's office in the faculty. And we are very much concerned or tossed with. We are passionate about student success. So student support, academic support. We provide academic support fostering and promoting the scholarship of teaching and learning. And we also assist the dean and the faculty manager, our faculty marketer with elements of the student life cycle. Things like admissions policy, you know, curriculum design, PQM, those type of things. So I think let us jump into it. Sigo, I don't know if you want to quickly switch on your camera. Just to say hi. There you go. All right. Fantastic. Thank you. I think I'm going to stop my video now and get into this. Okay. Fantastic. So we embarked on this analysis or process of mapping out the student life cycle. But to start off with what is the student life cycle? What is the student life cycle management? So if we are in office of the dean, so this is our context, we are here in middle management. So what is it that we want to manage and why do we want to manage it? So just here from the California State University cited that the student life cycle framework is used by educational leaders to help with support service analysis, prediction and planning, and can also be used to help design new and creative ways to improve first year retention and student success, overall student success. And the analysis that we will be sharing here today is very much focused on retention and graduation and student success. In line with this, we've also seen that making use of a student life cycle framework is a big drive. Nationally, internationally, nationally, I know there's this big drive at VITS under Prof Diane Grayston to improve student success through a holistic student life cycle framework. Now, student life cycle that of course is from entry, recruitment, looking at data and analytical aspects of our students and information from recruitment all the way up until graduation and even post graduation. So Pravee Govinda in 2020 posted an article indicating that student life cycle management is a data driven approach of an educational institution that focuses on the entire student journey right from the very first time the student becomes aware of the institution through to admissions and then on to alumni. So we do we as a teaching and learning office subset of the Dean's office fit in with the student life cycle. I think all of us sitting here are involved in the student life cycle. But as I mentioned, we assist with admission criteria, so designing policy and assisting with policy around admissions. We assist academics with curriculum development and PQM. And then also student support at identifying and providing designing interventions for at risk students, for example. And then marketing, not so much in our office specifically, but the marketing of our programs is very much a priority for the Dean's office as well. We do have faculty specific marketer addressing that specific part of the life cycle. Today, what we will be sharing with you is just a focused look on the life cycle, specifically from where students enter the higher education environment, specifically the US and an EMS program up until graduation. So what I want to share with you here is a typical life cycle at the EMS faculty at the University of the Free State. So maybe just starting from the back there, I think it's easier to to to go from the back. So for a student to ultimately graduate with a bachelor's degree at the University of the Free State at the EMS faculty, they can enroll for a mainstream program. Typically, we this is a three year qualification, a BCOM or the admin qualification. So we've got two streams then the EMS faculty, a three year program. Our admission requirements is an AP scores, a minimum AP score of 28. And then of course, also we have a minimum NSC math requirement. Should students not meet that initial or qualification admission requirements for the mainstream program directly, we do provide access to students through our extended program. So either they didn't meet the AP or they didn't meet the math, then they can enter the the EMS faculty or one of our qualifications through the extended program. The minimum time here is four years. So the first year of the curriculum design there is very much focused on development, developmental modules that we include, you know, for the development of skills and supporting our students to be successful in the mainstream programs once they progress to their chosen mainstream program. And of course, before that, we have our recruitment and our prospective students that then either enter through the extended program or the mainstream qualification. Like I mentioned, we've got two two streams, we've got our BCOM stream in the EMS faculty, which we have subdivided actually into a financial stream and a non financial stream. So the financial stream is our qualifications that has a higher math requirement, NSC math requirement in the admission for admission, and then the non financial and then of course the the admin stream that has no math requirement and students can enter without any math. So this is the typical life cycle of a student, but what we've seen and what we've experienced at the EMS faculty at UFS is that students can or may or are possibly using our highest certificate. So this is our one year qualification, our highest certificate, we've got one in commerce, and we've got one in admin. So for both streams, we do have a highest certificate that we offer. And it seems to us that students are using the highest certificate as a mechanism to access a mainstream qualification. So this of course asks or lead us to ask a couple of questions. Is this in fact the case? What is the trend there? If the students are entering through a high certificate, then progressing to extended program, then progressing to a mainstream qualification, are they successful? And if they are successful, are they successful in minimum time? And if they are not successful, what extra support measures can we provide to support our students? Further, not only to student success, but also to curriculum design, you know, is our highest certificate, if this is the trend, is our highest certificate as it is currently designed fit for purpose? So these are a couple of questions that we would like to address. So the first thing we did is to go and look whether in fact the data substantiates this hunch. So what we have on the screen here is basically some descriptives of four cohorts of high certificate students. So the green bars represent our students entering through the high certificate in admin. And the blue bars represent the students entering the high certificate in the commerce field. So students that enters the high certificate graduate graduating from the high certificate office one year, they do have in the design, they do have the opportunity to articulate or progress to the extended program, ultimately following a mainstream program. So in 2017, in our 2017 cohort, 61% of our admin students did come back after the higher certificate and enrolled into the extended program. In the BCOM route, that is 88% of our commerce higher certificate students came back the following year and enrolled in the BCOM extended program. And this screened, we do see and exists across all four cohorts here. So we do see that students are moving from a high certificate to an extended program, ultimately with the aim of graduating in one of our mainstream programs. And that's not where I come and ask the question. Is our high certificate, if this is what the students are doing, is our higher certificate fit for purpose? Now according to the HQSF, our high certificate is primarily vocational with a strong industry orientated focus. It typically includes aspects such as will work, integrated learning, simulated work experience. But we do see that and it is possible for a student to then use this higher certificate as a method of entry into our mainstream programs. So back to basics. We have to go back to the growing board. Let's see, is our curriculum design fit for purpose, are our students successful? So our problem statement basically is if the higher certificate is being used as an access mechanism to a formal bachelor's degree. Are we providing this opportunity to our students, they are doing this or are we providing this to our students with the purpose of having access with success? Or is it access versus success? So we embarked on this learning analytics exercise that we will share with you to determine whether students entering the high certificate programs successfully progress to and graduate with a relevant mainstream qualification from our faculty to inform our decision making. In terms of admission requirements, there's a lot of decisions that such an analysis could be useful for. In terms of our enrollment planning, like I mentioned, our curriculum design, graduation targets. What are graduation rates are looking for at and what do we want to target in future? If students are successful, what pathway do they follow? If students are not successful, how can we identify them timeously and provide a focused support to our at-risk students? And then of course, the resource allocation decisions around resource allocation in terms of our high certificate and the financial viability. So with this background and context in mind of why we embarked on our analysis, we would like to ask or to now have a group discussion. So we are going to give 10 minutes group discussion time. I don't know whether it will be Elizabeth or Kamalita that will be assisting us, but we will be dividing the forum into five groups with 10 minutes discussion time. And we ask that you please assign a leader, a scribe and a time caper so that we can manage the time efficiently. And then afterwards, provide five minutes feedback per group from your discussions and your views and the how you might go about such an analysis. But we would like to also provide you with some prompts. So we're not just leaving you there in the woods to facilitate this discussion. So we've got two prompts, two things that we would like you to please address and think about. The first is which indicators are important in measuring the success of the higher certificate of the students and the higher certificate in light of this problem statement of students going and progressing to a mainstream program are they equipped? What analytics and tools would you need to convince decision makers that the higher certificate is indeed or is not a mechanism for access to success or access with success in higher education? So we will be providing you with a document, a Google Docs document with these prompts to assist you to facilitate the discussion. Siwu, if you can maybe just pop the links into the chat. I would really appreciate that. So after they are in the chat, I'm just going to give a minute for everyone to please copy the links because you will be randomly assigned. We don't know who's going to be in group one or who's going to be in group four. So if you can please copy all five links and then once you have been assigned randomly assigned to your group, please use the link for your specific group in guiding your discussions and jotting down your thoughts around these two questions.  So if everyone can maybe just give them, I'm just giving a minute for everyone to download that or copy that. Elizabeth, just want to ask about the logistics. Will you be assisting with the groups? Yes, I've already created the five groups. There will be people helping with you as well or the people in the groups as well. So yeah, it's sorted. Okay, so it's cheetah, lion, elephant. Is that for our group? Yes, so automatically people will go into their group when I open it. Okay, fantastic. Just before we do that, any questions from the platform? Can you please just put the links again? Sure, sure. I think maybe I'm going to copy them outside of the word document. There we go. Thank you, everyone. We're really looking forward to, in light of this situation that we're faced with and the questions that we answer, we're really excited about having your inputs and your thoughts and your views on this. So thank you so much. Thank you, Elizabeth. I think we can then move to the group discussions after which we will come back to the main room for the five minutes feedback per group by the leader. Thank you so much. Elizabeth? Hi. Hi, can we do another take of the screenshots again, please? Maybe just after this session at the end of that, if you can, please. Can we do it before break? Yes, before the next break. Yeah. Yeah, Takenlani and I just chatted now and we see that we need a better screenshot. We have one, but there were so many screens and we missed the others and there are more people attending now. Okay, thank you so much. Welcome back. I think we are all back. That team minutes really flew by. I was so engaged in the discussion in the group that I sat in. I can't believe how quickly it went, but I'm very excited. I really want to thank you for taking the time to address those questions. So just in the interest of time, I'm just going to jump in with the feedback sessions. I will nominate the groups and if the leader can then maybe just give us a maximum of five minute feedback from your group in terms of the discussions you had, that could be fantastic. If we can please start maybe with the Lions. I'm not sure if it's group three that was the Lions. And I see we were cheat does. So I'm not sure if I should just continue. Please, I think I'm just going to call on on the animal names. Thanks, John. We don't have that description, so it was kind of confusing. So we just briefly discussed some points that came up in for question one. We would quantify their admission point score range. I'm on the right document now. We wanted to know if they are completing the higher education certificate and how is their engagement before, while completing the higher certificate. And afterwards, what does their engagement look like? When are they doing this? What achievements have they obtained? And how does their marks look? Well, another suggestion was to evaluate the curriculum, observe the requirements versus the requirements of if they want to apply for a different degree versus the skills or knowledge they obtained from the higher certificate. So maybe have stricter admission requirements. And then we also mentioned to quantify the admission point score and measured up to the NQF level that is already pointed out in South Africa. And then we didn't really get time for the second question. It was like the last minute, so we just really quickly said the number of students that complete the higher certificate. Where do the students want to move inside the university? Once they've completed the higher certificate and then again quantify the admission point score just to get an overall broad view of admissions. That's it from group three three slash cheetahs. I don't know if some of my group members want to mention anything else? Brilliant. Thank you. Thank you so much to you and your group. I'm very excited. I think there's some interesting notes that we are taking. I'm just making notes. But the whole idea of looking at the admission policy, I think this is where it is really critical for us as well. The indicators that we will need, those that you mentioned, the entry requirements, are they successful? What does the curriculum look like? All of this will influence our admissions policy. And quantifying the admission requirements is definitely very, very important. And I actually want to speak to you after we've done our presentation of what we've done as a possible next step to go into that quantification, even in more depth. So thank you so much for these valuable contributions. We really appreciate it. Okay, so if I can then, so you were your group three the cheetahs, John. Yes, we were group three. Cheetahs, yes. Fantastic. Sorry. Okay. Great stuff. So if we can then maybe move on, let's go to group one, the elephants. We only just got the time to talk about the first question. So one of the, there's a couple of indicators that we thought were important for measuring whether a student was going to transition from the highest certificates into the university and be a success at university. So you have the throughput rate through the program, the one year program, but more importantly, it's the time to completion that would be of interest. If it's a one year program and the students taking two years, it's not likely that they are going to be able to adapt and cope to the university environment. We also felt that there was an ability to, I mean, the need to understand writing capability. So having a measure of literacy and writing and articulation in that sense would be very important for university. Then someone was also mentioning that there is a need to have, and this is not necessarily an indicator, but it's an interesting thing to reflect on, that the highest certificate and the way that that program is designed needs to have a relevance or a continuation into the university so that there's no disconnect. Those programs should be very carefully designed to ensure that the learner is being prepared for the university. And then as we were leaving the discussion room, we had this brief talk about the age of the learner that is either entering into the higher certificates or university. And I think this is very interesting to me because I mean, I did fieldwork with metric students and some of them are 21 years old. And if they are leaving and graduating metric at 21, it means that they've likely struggled through their high school education. It will likely mean that it's going to be harder for them to excel in the higher certificate and university. But it's also interesting, some students may have matriculated when they were 18, but taken time to get and prepare themselves and get the necessary support in place for them to do the higher certificate, in which case they are a more mature student and may have more drive to succeed within their higher certificate and their degree. So I think that the age could have two kind of polar opposite indications of students drive for their future success, one looking at the matriculation age and one looking at the age of the entry into the high certificate. So that's what we were able to discuss. Fantastic. Thank you so much from the elephant. So I can maybe just mention just on the disconnect. And I know the first group also mentioned that in terms of the curriculum design at EMS, a student after they completed the highest certificate, they moved to the extended program, but they do get so there are similarities between the highest certificate and the extended program. So they do enter the extended program with some of the skills that is required that would and and recognitions that would allow them to get this mainstream qualification that they're wanting to do, let's say become economics within minimum time. But then of course, that's the very big question that the elephants also asked in terms of success. Do they do this in minimum time? So that is that is something that we really wanted to look into. We didn't include anything around age, which I find absolutely so invaluable, including that in future, maybe to expand on this to see what student behaviour is specifically in terms of taking that gap and what impact that has on student success. So thank you so much for that input as well. Then I think let us move on to the next group. If I can maybe ask for group two, the lions. Hi, sorry, I think we're the lions. We pretty much we haven't added anything in addition to what's already been covered by the other two groups. Our conversations are very similar, just looking mainly at the graduation rate, the throughput rate and the success rates. And unfortunately, we didn't have time to get on to the second question. Fantastic. So then we're all in the same mind and on the same page. But if at any stage you feel that there's something that you want to add, please feel free to do so. Thank you so much, then to the lions as well. And then group four, the rhinos. Hoping you can see the screen. I must say, so it's Ashton here from the rhino groups from DUT. And I found the problem statement to be quite interesting because at DUT, our policy is that high certificates are not a stepping stone to the institution. And we actually had students hold the Dean of Management Sciences hostage in their offices as well as senior management because the students had this expectation in the beginning of the year that having completed the highest certificate last year, they would automatically be accepted into the formal undergraduate degree programs. So very interesting high certificates at the moment. And also noting that your high certificates are self funded compared to the Nesfa students who would receive funding for their undergraduate degrees. So it was made more complex by the students having self funded themselves for the high certificates, thinking that they would then be able to get the Nesfa funding after they had been admitted into the formal degrees. So that's a little bit on the side notes. In terms of the breakaway questions, so some of the data points we looked at was around figuring or calculating their success rates in the next degree. So after they completed the highest certificates and they move on to their next degree, what are their success rates? And then linked to the success rates would be the ability to graduate on time as well as how many years did it take them to complete. So I think that has been mentioned before. An additional point we brought in was access to employment after graduation. So all the students who've gone and completed a higher certificate more employable and more more easily able to get employment as a result of the highest certificate. And then linked to that is the high certificates able to build those soft skills better or your graduates attributes. So that the students have a more holistic students experience and education. And again, that's kind of linked to the employability and the learning outcomes. We briefly had a look at the question two. So we said, well, if students are not graduating or if they're taking too long to graduate or if they drop out, then you're not improving their education. And then again, we raised the issue of employability of the student of, you know, we'll speak to the relevance of the qualifications that they've done at the institution. So I think the additional things we brought in there is around the employability of the students as well as looking more at the softer skills and the graduates attributes that should have been developed more if they're doing more qualifications. Thank you. Thank you so much, Ashton, and to the Lions team. I really appreciate that. Just in terms of employment, then I'll get back to the I want to say the discussion, I think, around how high a certificate. But in terms of employment, 100% agree with you. So our graduation or our alumni offices don't track our high certificate graduates. We have got no information in terms of the employability or, you know, what happens after they did graduate with a higher certificate, even less so from those that do not. And I think that is something an area of interest that we identified that we also want to follow up with. So thank you so much, you know, for that qualitative enrichment feedback from the students, from the graduates themselves. So thank you so much for bringing that up as well. But in terms of, and I think that's that's what you started off with, the national use of a high certificate. Absolutely. So some universities or whatever that are using a high certificate as a stepping stone as as we are seeing happening at EMS in the University of the Free State. And others do have this policy that say no, it's not allowed. And I think for us the question back to basics. So we've been doing this for so long. Is this the right thing to do? Is this stepping stone that we are creating the right thing to do and to be able to answer that and to inform our policy we need to see as you as you mentioned, are they successful? Do they have the necessary graduate attributes? Are we contributing to the education? Because if they're dropping out, if they're not doing this minimum time or taking too long, then we're not helping them at all through the stepping stone policy. So thank you so much for contributing in that way. And then I think it's the last group that we have left group five, the giraffes. Thank you. I'm sorry, that was my error when I jumped on with the alliance, just to say that the giraffe had covered pretty much what was covered by the other group. Okay, brilliant. Thanks. Thanks. So that was the giraffe's feedback then. Anyone else from the lion group or the rhinos? Have we missed a group? Okay, guys. So thank you so much for your time, for adding to our conversation and giving us so much to work with. I think we're very excited. I know I am after this session to expand on what we have done. But if we're mapping out the student life cycle, it is from entry up until graduation. So we in our analytics, how we've approached it is we're looking at admission requirements, so students admission requirements, then their performance throughout. So the performance, the success in their modules, the academic average year on year on year per student up until that point of graduation in a mainstream program. So the jumping highest certificate extended extended to mainstream program. And so up until the point of graduation there in, in minimum time. So the data and the analytics that we're going to share that I'm going to hand over to Sivu, our data analyst now is following that pathway. And then absolutely in line with the discussions in minimum time, graduation in minimum time. So thank you so much for your input. Sivu, it's over to you. Right. Thanks, Anari. I see most groups didn't get an opportunity to discuss around the analytical tools that we would use in order to do this analysis. Can I just ask from the group, it doesn't have to be according to your big five educations in terms of groups from the people in the auditorium, essentially online auditorium, which tools would you use? And by tools, I include programming languages, I include statistical software, which tools would you use to sort of get started with this analysis? And the reason I'm asking this is because some of the data tool selection that we use can have an influence in terms of how much data we share with decision makers, how that data is formatted, etc. So can I just see from the group, it can be anyone, which sort of tools would be your starting point in terms of this analysis? I can go. So in my case, I would use, so I started from Excel, Stata, R, Power BI. That would be my starting point. Any other person? Thank you very much for that feedback. From my side, Sivu, SQL for sure is starting best as a programming language, if you want to call it that. Thanks. I can also comment from my side. Oh, sorry about that. Sorry, I spoke over here. I see that you're muted. It's Melanie from the North East University. So obviously, depending on your warehouse and how you're storing your data, I mean, programming languages like SQL to try and collect all your data, using maybe R or SAS for statistical analysis, depending on what you're familiar with and what you want to do and then possibly Power BI if you want to display this data in dashboards and so on. But I'm sure there's many other options. I think it just depends on the data structuring at your university. Thanks. There we go. So I like the feedback that we've received so far. The reason I asked that question is because as I mentioned earlier, there's a myriad of tools that one can use that can inform some of the decision making that we do and some of the analysis that we do and the results there often how we share that as well. Now within the faculty of economic and management sciences, when I got this analysis from the teaching and learning manager and the dean, the very first thing I had to answer was which tool am I going to use? Who am I going to rely on to get this analysis going? And how do I report it as well? Now I just want to see, can everybody see my screen? I should have data analysis and results. Yes. Thank you, Steve. Perfect. Right. So the first step we took care in order for us to collect this data was to contact our directorate and directorate institute for rather the directorate for institutional research and academic planning. It would be rather unfortunate for us to come to SAA and not refer to our directorate that's responsible for data. So we send a request to them and they respond to us with a Microsoft Excel dataset and this dataset contains quite a lot of information that we use. This includes the NSC results of the students you would have seen in the feedback from the groups that was an important matter. It also contained the academic progress of the students per module over the years. It also contained information on which programs the students are registered for in each respective year. And beyond that, we also have an indicator of whether the students have firstly obtained a higher certificate qualification. And if so, have they enrolled further in subsequent years and whether or not they have graduated in a mainstream program afterwards. So this is sort of the baseline data that we have to work with. And in order for us to handle this data, we import it into R. And now I go back to the question I raised earlier about which tools would you use. R is the program language we went with, but it doesn't have to be R. I know you can do this type of analysis as well in SDSS. You can do this type of analysis in SAS and so forth. The reason we chose R is because it's this programming language that R most familiar with and it's the programming language that we use to do this type of analysis. And in R we have quite a number of packages that we can use to do some of our operations in terms of preprocessing of the data. We have the read excel package which helps us with importing excel formatted data. We have the tidy models framework, which contains quite a number of steps that you can use to process our data further. Things like normalizing numeric variables, creating W variables and so forth. And in addition to that, we have the mice package. That's an acronym for multiple imputation by chain equations methods package. Now that package became extremely important to us because I don't know if you would remember earlier, Anare mentioned that we have students in our admin cohort. So they go into the higher certificate in administration route that do not have a math requirement. So as a result, our data set had some missing values in between. Now the first benchmark we had to check there was, are these missing values comprising a value less than 95% of the sample that we're working with? In our case, it was well below that. And from there, we had to decide on an imputation method. And here we used a random forest equation in order for us to fill in those missing values. If you want to find out more about that, there is a book by Fund Proven 2018, flexible imputation methods using R, I believe is the title of that book. But eventually, once we're done with the missing value imputation, we then export the data into three tables. Now, I heard earlier, I mentioned of SQL and we're basically using that approach in terms of separating the data into three different streams. The first data set contains the students as they enter the university. So at entry to the university. And then we have a second table that contains the students academic progress through the year that they registered at the university. And then beyond that we have graduation status, which is an indicator of whether or not the students has graduated in a mainstream program. Because remember, our primary question was around, firstly, is there a path where students are going from higher certificate to a mainstream graduation? And if there is, what are the sort of predictors that are leading into that? Now, there's a whole host of analysis that one can do. I mean, if you look at the type of data we have here, you could look at panel regression methods, you could look at logistic regression. But in our case, we went with survival analysis. And the reason for that is survival analysis is analogous to how you would analyze the time to an event. So if you think of graduation, you can think of graduation as the event, and the interval between entry and that time as where our data would be. And the reason we went to survival analysis is it started off in the medical domain, but it's going on to many other fields. I believe there's an article in the Journal of Learning Analytics that actually made use of survival analysis as well. I will drop a link on that later in the presentation, that used survival analysis to try and understand outcomes for students. Now, the workflow that we have in terms of our analysis as follows, we started off with the Kaplan-Meier plot, which provides an overview of the outcomes for students. And within that Kaplan-Meier plot in R, you can actually extract some of the backend data that will give you a live table, which will indicate to you things like how many people are enrolled at a given time, how many of those people subsequently unentroll or we don't know the status at any given point, and the probabilities of success. And then we go on to cox proportional hazards regression, which is a bit more in-depth. So here, we're taking these outcomes that we can see from the Kaplan-Meier plot and live tables. And we're trying to look at whether the odds of graduation based on some of the variables that we have on hand. So to start off with, I saw in the groups that we were working with, we had quite a number of aspects that we could look at in terms of how students are progressing through the university. And as I mentioned, the Kaplan, Meier plot and the live tables, you can actually see from entry. So we start off with in the administration route. For example, here we have a cohort of 2018 in higher administration route. We start off with 128 students in 2018. In the following year, we have some accretion in the number of students that enrolled to 94. And then another student unenrolls in the following year to 93 and so on until we have as often in 250 people that were still observing in terms of their academic progress. And in that time from the table, we can also see how many people are graduating. Now, for the first three years, you're not going to see any graduation because that's below minimum time. But at year four, we start seeing some graduation. So we start off with 33 in 2021 and then further eight graduated in this year for the higher certificate in administration route. Now we can display this information in a Kaplan Meier plot. In summary, I don't want to get into the esoterics and nomenclature when it comes to survival analysis. But using the Kaplan Meier plot, we can look at the probabilities. The probability of graduating remains one for the first three years. Remember, we are counting for essential data as well. And then at year four, we start to see this difference between the higher certificate students and the higher certificate in administration and the commerce route of the higher certificate as well. Now, the next question one might have is, okay, now that you've seen these differences, what attributes or what variables are leading to these differences? Now, as I mentioned earlier, we get our data from the institutional research office and we are provided with quite a number of data points on students' academic performance at entry throughout their academic period at the university. And then we also have an indicator for the graduating or not. And then the next step that we have to decide on is which variables do we select to put into our model. Now we're working with a sample of 66 graduations in total across both routes in our higher certificate for the 2017 cohort. And then we're also working with 45 graduations for the 2018 cohort. And in our variable selection methods, there's quite a number of methods that one can use. You can use stepwise regression to select modules and variables rather and compare those models against each other. You can also use penalized regression for cox proportional hazards, which is what to be used, which is a shrinkage method that you can use for our survival data. So we have to consider the structure of our data is going to be fundamentally different if you were trying to do linear regression, for example. But ultimately what we end up with for both cohorts is pool models. The first is where we take every variable that we have and we throw it into our model and see what the results look like. The second is the more appropriate approach, I would say, where we are looking at the outcomes of our penalized regression for cox proportional hazards model and we are only using those variables which are statistically significant in that model and carrying them forward to our final model, which will be in model two. So for each cohort in the results I'm about to show you, for each cohort you will see model one and model two. So this is for the 2017 cohort. What you find here is in model one only quintile four schools. So the type of school that the student went to is a statistically significant protector and you can see the confidence intervals and so forth below here and you can see the notes in terms of the hazard ratio as well. But if you work with model two, we have three variables that are statistically significant. That is the AP score, the students as they enter and then we have the route that they take. So did they go through the administration commerce route and then we have the academic year or rather the academic performance in 2021. Now a note on the academic performance, the academic performance data is based on an aggregate using a mean of the academic performance of the year that we are looking at and then we normalize that data as well as mentioned earlier. We normalize that data and we fit it into our model in terms of the interaction about here in terms of the interaction we had to try and model that mathematics mark is not equal to a mathematical literacy model. So we have a dummy variable that indicates which mathematics a student took and we try and account for that and in each of our models we don't see that as statistically significant. Now there are probably ways for us to try and input or rather add additional variables into our larger data set. But the summary from the 2017 cohort is these are the top three variables that come out. Now if you compare these using ANOVA, we see that model two is the better predictor and we can actually see in terms of the estimation in the hazard ratios that model two is a bit more conservative which would help us avoid overfitting if we were to test this against a test data set. We do the same analysis for the 2018 cohort where we're looking at which variables are statistically significant given our data set. Here we have a slightly different picture for one the academic progress and AP score are statistically significant in both models and we have the route that the students are taking. Are they in the admin route or are they in the commerce route is statistically significant in the model two as well. Now when we test these two models in terms of their performance using ANOVA we don't see a statistically significant difference between the two. However again I want us to look at the hazard ratios here. If we are to if we were to test these two models against a test data set what we'd look at there is how is it performing at predicting those outcomes for students over time. But overall I just wanted to highlight that these are some of the variables that we're working with and the next step for us is remember we're trying to look at the full path that students are doing. So from enrollment in higher certificate in which programs are they ending up in. So we go and zoom in a bit more on our cohorts. So let's start here with the 2017 graduation rates for the 2017 cohort. Here we have 61 graduates that we're observing 45 of these graduates initially started in the administration route and the remainder were in the commerce route. What you can see is 25 of the 45 graduates graduated in an administration program. So BA administration, B administration which is offered by our faculty and then the remainder of those students are actually graduating in other programs. So we have one student that has graduated in Bachelor of Administration honors others have graduated Bachelor of Arts honors in film and visual media. And when we look at our higher certificate in commerce there's a smaller cohort remember we're working with only 16 majority of whom graduated in our big home investment management and banking. So what we're looking at in terms of our time range here is minimum time plus one. So as as of minimum time plus one this is what our graduation rate looks like among higher certificate students. If we move on to cohort of the 2018 cohort we see a similar trend where our higher certificate administration 32 of the 44 students graduated in a Bachelor of Administration. A smaller proportion of them graduated in qualifications such as the BISOC business management in psychology, BISOC criminology and psychology and BISOC industrial psychology and psychology. And if we look at the higher certificate through the commerce route so far observed two students that have graduated in big home investment and banking others in big home marketing and BSC statistics and economics. Now with all this analysis that we've done with the request that we send to the institutional research office what are the sort of main takeaways that we can look at. AP score is appears statistically significant across our models. The stream that the student is taking appears as significant as well and then the academic performance. Now the difference here is in the 2018 cohort the first year academic average is important while in the 2017 cohort the final year academic progress is important. And that in itself opens up a whole set of questions around why are these differences are their differences in our in the predictor of variables. In terms of qualifications in the higher certificate admin route we see that most of them are graduating in the Bachelor of Administration. In commerce we see a bit more diversity in terms of which qualifications they're graduating in so we see students graduating in the Bachelor of Arts, students graduating in the Bachelor of Social Sciences and the final group is the big home group. I'll hand over to Anneri for the next steps. There we go. Thanks Sivu. Thank you so much. So I think in summary what Sivu shared there and if we want to the next steps and the questions that we want to have answered and the decisions that we want to make I think from from the regression analysis we clearly see that the whole student life cycle so the AP score which is part of their admission requirements is statistically significant in terms of predicting the student's probability to graduate. So admissions policy as we spoke about earlier admissions policy decisions can be informed here. Then we saw that the different streams that they take so and this comes to my to my first question there is our higher certificate for purpose. So with the admin group we saw that the students that are graduating following a high certificate in B admin the majority of the students that are successful do graduate in the mainstream B admin which is how it is designed what it is designed to do but with a high certificate in commerce it's a different picture. We see that although it's a smaller cohort again asking the question about admission requirements but we see that the students that graduate and successfully progress and graduate in a mainstream program the students are not necessarily serially graduating in an EMS program. Yes there are portions but if you count and add up all the the graduations from the from the humanities faculty then they exceed those in the BICOM program. So we see a trend where our students enter the highest certificate in commerce extended in commerce and somewhere move and a very large proportion of our commerce students are moving to the faculty of humanities. Now this begs the question I think a big difference there is I'm not saying this is the reason but this is a next step that we can take is definitely quantifying those admission requirements but in terms of the mathematics our BICOM programs have a higher mathematics admission requirements than the programs of the humanities. So is that a factor is that the type of qualitative feedback that we can get from our students? Is that something that is discouraging being from going to an EMS BICOM program? So interesting things that that we see so is our highest certificate fit for purpose I think when it comes to our highest certificate in commerce specifically we need to we need to go back to the drawing board there we see that our students that those are that are successful are not necessarily pursuing a degree within our faculty although at our institution but then we need to address the curriculum to either prepare them more intensively if you look at curriculum design there that I have at the bottom prepare them more intensively with quantitative skills to improve the the success within the academic progression ultimately to graduation or maybe we should consider interdisciplinary higher certificates this is something that maybe we we need to explore talk to the faculty of humanities say listen this is what we are seeing we are seeing that our students are preferring so maybe we should pull resources maybe put these things together and design something that can have a benefit for this particular pathway that the students are following benefit for the students then of course another type of decision that this could inform us in terms of our enrollment targets where do we want to grow again is is should should we shift targets and resources to our B admin or should we shift it to our extended enrollment targets or should we shift it to the mainstream should we keep it as is but address curriculum challenges so all of these things that that we are working through I think one very important aspect that we that did come from this analysis is the importance of academic advising we really need to as a next step to intensify the academic advising that we provide to our students specifically in the commerce route so to help students to identify their appropriate academic career then in terms of our graduation rates how can we improve on our graduation rates as you see they are not they're not that bad but they're not that great either so how do we improve on our graduation rates how do I identify these students earlier and provide focus support so having focus support measures their resources allocation for those support measures as I mentioned I think as a next step we would really like to enrich this analysis with qualitative data definitely with the inputs that you gave us from the workshop as well but some qualitative data as well targeted at our current students but also the alumni as I mentioned this the alumni in the highest certificates is an area that is not specifically tracked and that we have very little information on so to expand on that we want to provide some measures of engagement or include some measures of engagements that one of the groups also mentioned you know add some online activity data for example that's just one and then of course the funding that that the colleague from DUT also mentioned so what is the funding status of the students ultimately going to mainstream and how is that assisting or not in terms of their success and I think in a nutshell that is about that is our story from our side and thank you for listening and adding to our discussions in trying to address this this question that we have we really really appreciate your inputs we do have a full report of this analysis that we would love to share so if you are interested I'm going to drop my and see this email in the chat box in a moment but we will share that with you or if there's any other feedback or discussion that you would like to have maybe some additional critique we would love to hear from you to to take this to take this further so thank you for your time I think Elizabeth we can maybe open up for some discussion thank you Camilita I think Camilita was chairing the meeting yes are there any questions comments for Annemarie and I see Ayanda's hand is raised hi hi colleagues thank you very much for such wonderful presentations really UFS you've really touched on a serious point of concern and it really impacts us at the most especially our universities where we get high influx of undergrads are mostly dominated with undergrads I'm from Mangosu to University sorry for for getting the intro and we highly dominant on the undergrads section and we do get you know a sense of this higher certificate having an effect on you know your input your entry level students so my question to you I know you touched on it on the on the last part the financial viability I'm just interested to hear how do you now with the results that you've obtained and obviously evidence to back it up for presenting to management how do you you know sway your management into buying into because there's a little bit of an element of difficulty in convincing people that you know certain departments or certain programs are not viable to to be kept on board simply because they don't maybe have a good effect for the business or the organization in the long run but not to say that they they don't exist in the academic space in terms of HEQSF approval yes they are there and recognized as a form of qualification in higher education South Africa but in a sense for the organization itself or institution it may not be viable to keep them on board so just an interest how do you get to sway your management into buying into that that feedback or that results thank you very good question I under and let me tell you to be frank that's something that still is in the cards for me so this is one of my next steps um so let me tell you I think the big question here is financial viability you know if we want to talk financial viability that's not my area of expertise per se but I think I would definitely go back to our directorates for institutional research and planning to assist us with the funding model and then to determine listen if we if we compare our inputs our resource allocations to our high certificates we've mentioned we've got two streams but we've got many satellite campuses many facilitators many lecturers that has to be appointed because of of the distribution all over the country versus what we get at the end of the day in the mainstream program a subsidy back is that is that worthwhile and I think that is a very relevant as you mentioned topic currently I think we did show and I think the evidence is there for for both cases I think it is worthwhile but then of course I think we need the financial financials to to either substantiate that or not and that is where I will pull in my our institutional research um division to assist us with such modeling but yes I think you are raising a very uh you know the the question on our minds as well thank you very much thank you very much that's quite quite helpful understood thank you I think from a modeling perspective we um though we have the models that we are working with I think we can still expand on it so one of the examples I can make here is we still don't know who is funding the students from throughout the period so you might have a student that might have started out with the bursary funding in the first few years um or a student that didn't have funding for the first few years and then their funding status changes now we need to find a way of modeling for that one way we could is rather than focusing on a yearly basis so the results I shared with you are based on academic here how about we look at a semester where is the accretion happening are we losing students in the first semester or second semester and that question also leads to does that have an impact on the viability of the program if you are creating students let's say after three semesters versus four semesters so those sort of questions would I would assume would be asked by management and other stakeholders in terms of viability so from an analytic standpoint I think we could actually do more modeling looking at obtaining more data one of the other feedback we got in the groups was around how engaged are the students another data point we don't have in terms of the students performance over the years if they access on the learning management system so how often is the activity in line with the graduates are there any statistically significant differences between the activity of the graduates and the non-graduates what is their current status and so on and so forth so I think more data would be useful for us to investigate in order to feed into the financial viability analysis team definitely thank you colleagues it certainly gives food for thought as well thank you so much for that elaborative answer I'm covered thank you and any other questions or comments Elizabeth it seems like we've come to the end of the session I'm landing over to you thank you very much animerian