Hello, everyone. Thank you for joining us. Welcome to the Moodle Academy webinar, improving Moodle through UX research and data analysis. I am Sandra Mads, Moodle Academy Educational Events Coordinator here at Moodle HQ. My colleague, Mary Kud, Education Manager from the Moodle Academy team, is also joining me today and will help facilitate the webinar. Today, I would like to welcome Elliot Hovey, Senior Learning Technologist and Stuart Lamour, UX researcher and developer, both joining us from UCL. UCL Moodle is one of the biggest Moodle installations in Europe. It provides an essential arena for the teaching and learning of UCLs, 51,000 students and 16,000 staff. It is also an incredibly complex Moodle that combines third-party cloud infrastructure, third-party development, around 200 plugins, some local and some from the community, and internal agile-based development. Today, our invited speakers will outline the methods of UX and data analysis used at UCL in order to upgrade to Moodle 4.2 in 2023 and the results. They will further show how these methods can be adapted at other institutions looking to improve their Moodle. So over to you, Elliot Hovey. Thank you. Thank you very much, Sandra. Yeah, thanks for that. And hi, everyone. Bring up my last slides. So I hope you can all see them on the screen now. So, yeah, as Sandra said, so myself, Elliot and Stuart, both at UCL. And today, this is kind of a run overview of what we're going to run through. So we'll talk a bit about the UCL context that we're working in, a bit about the design system that we used in our building our Moodle theme and then the kind of research that we did. So top task analysis and benchmarking, testing the initial designs we'd learn as running design, focusing on user needs and then how we're kind of evaluating this going forward. It's going to cover similar territory to our Moodle Mood talk, but we're going to update you on some of, I guess, some of the progress. And then at the end, we can have Q&A. But as we go, if people have questions or thoughts, do put them in the chat and then we can answer them at the end, probably, and just respond to them. So hopefully that sounds OK. So the UCL context, if you haven't heard of UCL, essentially, we're a massive kind of blended learning, some online learning university. We need to update our Moodle to 4.2 in July 2023 from 311. And it was a good chance to kind of update the theme, the landing page and the dashboard of my courses. And this is kind of what we concentrated on. We've also just upgraded to Moodle 4.3 in January. So it's part of our kind of continual development. The business requirements that were kind of major when we started this project was to use the UCL design system. And we say it's in beta because it's still like an evolving document or resource. If people have their own design systems that are using, that'd be really interesting. So do share that in the chat if you have one or maybe even share it if you have the link. Or if you don't know what a design system is, we'll kind of explain that in a second as well. We also had a requirement to have marketing spots, like to advertise events and news to students on Moodle and to alert users when things are broken. So kind of basic requirements that we wanted to include in our upgrade to Moodle 4 and in our redesign of the theme. OK, so I said Moodle, you said Moodle is big. So these are some kind of, you can see a sense of the sale. So about close to 40,000 active users per day, 70,000 sessions per day, 5,000 courses in each kind of up to a year, and 150 plus plugins. So the UCL design system, hand over to Stuart. Cool. So this is a UCL design system. It's UCL's design language to use across all of its websites. Design systems are really helpful. You have one basis for your code, for your HTML, for your JavaScript, for your CSS, so that every time you build something, you're not starting from scratch. It means that everything you build is consistent, which is really useful for users. So that when they see a button on one page, as they see a button on another page, they recognize it's a button. They can see it has cards and a grid. It means that accessibility is built in whenever you build a component or an element on your page. And this was the UCL design system designed by Clear Left. It's in beta at the moment because UCL's design systems evolve to incorporate different needs. But we thought with UCL Moodle, we kind of want it to look like UCL. We want it to look like the students are still in the same system. So we wanted to apply this design system, this design language to Moodle as best we could. Next slide, please, Elliot. So on the left-hand side, you can see what the Moodle did look like that we started with. That was Moodle in 3.11 or 3.9. I'm not too sure. It's been three something, possibly two something, for a very long time. And as you can tell from the juxtaposition of what it looks like on the left-hand side of the Moodle and what it looks like on the right-hand side, which is a page from the UCL website, they're very different. Next slide, please, Elliot. Top task analysis and benchmarking. OK, am I doing this one or are you doing it, Elliot? I've forgotten. Are you? OK, cool. Top task analysis and benchmarking. So this is an industry standard process you kind of do. Basically, you ask people what their top tasks are. Why do they come to the website?  You can also do data analysis and see where people are going in the click paths if you've got kind of that kind of recorded. And the idea is to improve those top tasks if people find that they struggle doing them. If something's difficult, let's fix it. But the top task analysis allows you to focus on the big problems, the ones that the most users have difficulty with and go, you know what? Those ones are our priority, which makes sense because you've got to spend money on doing these things. So what you do is you choose what to measure these top tasks. You find out what the main problems are for the majority of your users. You choose how you're going to measure the successiveness of your design. So it might be people can see the information clearer, people have less difficulty finding things, or it takes them less clicks. And then we rinse and repeat. You just re-measure it with your new designs all the time and keep testing and iterating. Hopefully that makes sense to everybody what top task analysis is, and benchmarking. Next slide, please, Elliot. So yeah, we surveyed 100 users. We asked them how often they visited UCL Moodle, what the main task they did when they visited UCL Moodle, how easy it was to do this task, what their favorite thing about UCL Moodle is. That's a bit of a general question, but just for fun, really, and what the worst thing about UCL Moodle was. I don't know if anybody's done the same with their users, but it'd be interesting to see. So this is what we found. Yeah, on the form you could see from the responses that the students love UCL Moodle. It was really nice to see. And this is with the old design even. So hats off to Moodle generally. Learners have Moodle open all day. So with everybody we interviewed, we found that basically they always had a tab or two or three with Moodle opening it throughout their entire day when they were learning or when they're doing work. We found out about the top tasks. So they're all based around course content, finding course content, doing activities and feedback, looking for feedback and grades. Here's some quotes, what people liked. It has all the information I need, ability to locate all information, and tasks in one place. And then we asked them how easy it was to use with a rating one to five. And we got a 3.96, which isn't too bad. I mean, that's pretty amazing considering lots of the content is user-generated content where busy lecturers are having to go into Moodle and build things that they might not be experts. But what they didn't like was how it looks and feels. So Climkey, I think that's a general thing that Moodle gets quite often. I think it's got bear with four. So we'll see. Old and inconsistent was the other one. Yeah. I don't know if that's the kind of feedback that other people get about their Moodles at all, but that's what we got. Next slide, please, Elliot. Testing the initial designs with learners. So we had some initial designs for what it could look like. And we wanted to test them with learners to see if the designs fitted the needs of our users. We did once one moderated distance user testing similar to this over Teams, though, because UCL uses Teams. Quick sessions, 30 to 45 minutes, conducted remotely through Teams, they were recorded. Ideally, you also want people observing the call who aren't necessarily on it to make notes for you. That's always really good. And we want to capture qualitative and quantitative data. There's a graph here on the right-hand side. What it shows is the amount of usability problems versus the number of users you're testing with. And you'll find when you're doing user testing that after you've done maybe three, four, or five, you start to get deja vu because the users are telling you exactly the same thing as all the other users. Every person struggles with the same things. So it's kind of like the curve. You can see here, once you get to about five, you've got 80%, a few, 90%, 80% maybe, of usability problems documented for every user that you will ever encounter. So I think at one point, we were testing with about seven people, but we were just seeing the same results, really, to be honest. So you don't have to test with massive groups to do usability testing. Five is enough. Next slide, please, Elliot. Here's what it looked like. So this is the initial design that we started with, the UCL Moodle. It was a theme that UCL had bought and it had lots of different bits in it. As you can see, there's me in the top right on my screen. There's our user who we've masked and there's Elliot observing. And we recorded all the sessions and watching the user click around and seeing what they struggled with, seeing their facial expression, hearing their tone of voice is really useful for getting empathy with your learners. Next slide, please, Elliot. So yeah, what we did was that we had a facilitation guide and you write this down and you could go through the same script pretty much with every person that you're testing. We tested three most common tasks from the top task analysis. We did, designed to go through the main pages, landing page, dashboard page, my courses page, course pages. We didn't delve too much into activity pages. We looked at the elements on site, so blocks, navigation, another familiar part. So the system to allow us to of where participants struggled or endured using it. And at each page, we asked participants to stop and not click on anywhere. Tell us what they could see and think out and we asked them, whereabouts would they go to complete the task? What would they expect to see there? And then they would click on it and would say, is that what you expected to see? See if their expectations match reality. Next slide, please, Elliot. Afterwards, we did a debrief survey with the participants. A series of quantitative and qualitative questions in Microsoft Forms. For each Moodle page, we asked participants to rank the elements on the page in order of importance or usefulness for them. Did they find it useful? How often would they use it? And we also asked participants, what was missing? Next slide, please, Elliot. So this was our landing page. As you can see, it contains a login link. That's the thing at the top right there. It's got massive carousel. Course library link. That's the one on the left. Free course link, help and FAQ. New courses and then there's also a bit underneath about site announcements somewhere. What did users think of this? Should we have a look? Next slide, please, Elliot. Jump in for the next slide. Okay, go for it. So this is part of the exit survey on the landing page. And so we found that users, so the students we tested were primarily looking for their courses and focused on elements which may lead to this, which often lead them, led them to the wrong place. Which kind of makes sense, what are students looking for? And overall, so the landing page content did not seem useful to the demographic we tested. In that sense, the design kind of had all these other elements in front of the student, that they were kind of either getting confused by or just expressing that that didn't make sense. So the other thing is like this course library, so these are the rankings in which people ordered how useful it was. And you can see course libraries number two. But that course library was like a library of every course that UCL, so thousands of courses. So the students were going there, but what they really wanted to obviously just to log in to get to their own courses, rather than see thousands of courses which isn't gonna be helpful. Similarly with free courses, UCL doesn't really have, well it has some free courses but not in the way that it was promoted on the front page. So again, that's why students kind of favored that. And then if you go down the page, these kind of other elements like a carousel of images, which is quite common in different Moodle themes or a Moodle that was ranked less useful. So this is like the login page, so it's quite clear that students want a login button and a way to get to their courses is kind of the key or the top task they're trying to complete. We then looked at the dashboard. So once you log on, brought to the dashboard a Moodle four and this was the kind of design that we tested and our initial design had a kind of site announcements at the top, items accessed by peers in kind of the drawer on the right, which would show like, here are some activities or items that your fellow students have accessed. Maybe you wanna access them too, so that was the kind of idea. We had last access courses, a timeline of deadlines, which is just the Moodle core kind of block. My courses updates, which would be like a my courses grid, but also showing when a course has been updated, some kind of indicator there, and then a searching course catalog. So these are kind of just the initial ideas that we had and we did kind of a basic mockup. Not everything was interactive. Some of it was just pictures on the dashboard that we tested with users. So how did this dashboard perform? So students ranked it in this order. So last access courses was the main thing, my course updates timeline. Items accessed by peers was kind of, we thought it was, well, I thought it was an interesting idea, but many students kind of fed back that they weren't sure how useful that would be and like if everyone accesses the same kind of assignment detail at the start of the year, does that just always show as the most like recommended? Item. So they kind of expressed that they didn't think that would maybe be that helpful. And some of the other elements going down and it's quite clear that again, like accessing courses is quite the key demand here. Just to summarize that. So yeah, for the dashboard, so overall participants just wanted to see their courses as the primary requirement. Anything that hinted at showing courses was rated highly. The second most useful element was the timeline, which we thought was great because that's something Moodle provides, so we could just use that. And then participants were able to find their feedback grades from the dashboard. So that was, again, we were thinking about in terms of whether we could serve as feedback or grades on the dashboard and whether users were familiar with this or would favor that. I think we'll talk about that a bit more. I'll hand back to Stuart on this one. Yeah, so this was a quote from one of the users. Okay, so I don't think this is for me. Maybe it's for someone else. So thinking about all those elements that we'd put on the dashboard and kind of like when students were going through it, they'd quite often skim an item. They'd go look at it and go, okay, this isn't for me. And then just skip over to the next one and go, maybe this is for me. And then skip over it and go, maybe this is for me to try and find the information they were looking for. And I think when you do that to anybody on your website, you kind of, the trust degrade as you go down almost. If people seeing irrelevant data, the next bit of data they're less likely to think is relevant or trust it. I think that's really important. Quite often we might overload people with data and choices and information just because they might need it. Whereas actually it's not good. It doesn't create a good user experience for them. Next slide, please, Elliot. So yeah, for the tasks we asked participants to show us the user journey they showed us was always exactly the same. They went to log in as quickly as possible, painlessly as possible, pure as clicks as possible, find their course, open the course, find the activity and open the activity and resource. And that was it, very simple. Next slide, please, Elliot. So these were our recommendations from the user testing. Focus on the primary tasks, users need to do. Focus on the elements, user need, making them more prominent. That are suited for the user's needs if they don't work very well, improve them. Be careful of adding content that makes users feel the page is not relevant for them. Be careful of adding miscellaneous content which makes the user journey harder. I think it's everywhere a choice paralysis where when somebody's given two choices, three choices, etc. the more choices or the more links you have, the more information on your page, the more likely it is that people will not make any choice at all or will just leave. I mean, with Moodle, they have to use it, but we are still doing the same thing to them if we just fill the page with loads and loads of redundant information. And next slide, please, Elliot. And yeah, this is Brad Frost. So he's a really great developer and he did, he works on lots of huge, huge websites and kind of this is what he found quite often. So this is from one of his talks called Death to BS. And yeah, this is something we see quite often. That's the content that people are looking for, which is quite small on the web page. And then the web page tends to get filled with lots of business requirements or those kind of political arguments that people have. This has to be really prominent. And instead of asking the users what they need, do you end up with this kind of thing? Yeah, just if you can control your content on your Moodle landing page, the sign in and the dashboard in my courses, don't do this to your users. Next slide, please, Elliot. OK, yeah, so. We'd seen what users thought of the initial designs. We'd evaluated it and we needed to redesign focusing on those user needs. So this is kind of what we did. Should we have a look? OK, so this is or was a few weeks ago, the homepage of UCL Moodle. Yeah, we've got the business requirement of using events at the bottom. And apart from that, there's kind of just one big call to action, UCL login and no carousels. There is a photo underneath. Next slide, please, Elliot. On the left hand side, you can see what the dashboard design looked like. We'll go into each of the elements in detail and how they help users achieve their tasks. And on the right hand side, you can see what my courses page looks like, which is probably similar to most peoples. Next slide, please, Elliot. What we're looking at here, we'll find out in a minute. Might be the pick up where you left off block. Let's find out. Yeah, so OK, this was a block we designed called pick up where you left off. It's kind of a replacement for the recent courses block. We found that the recent courses block has weird things with a carousel. Don't even want to go there. It didn't show people courses that were hidden. So if you weren't enrolled in the course as a newer teacher, it wouldn't show here. And that wasn't very useful. And we just wanted to make it kind of responsive, feel like it was part of UCL's brands because this just exactly the same as the UCL brand system and provide the information that students needed. Next slide, please, Elliot. So we had two options on the pick up where you left off block. We've got courses at the top, as you can see, shows the image from the course and your progress on it. And then there was a second link underneath called activities, which allowed you to carry on with an activity or if you were a cheater, just find very quickly what you were last editing or whatever. Next slide, please, Elliot. So it's a pick up where you left off block. We've got two other elements on the page. So on the left-hand side, you'll see what we did to the timeline. This is the middle timeline block. Students actually commented this should be called deadlines, not timeline. But there you go. We took out lots of the information that people didn't need and just kind of... I think everybody's familiar with what this is. It looks like a timeline. And we wanted to make sure that it looked like a timeline to students. Yeah, I don't know what more to say. And then on the right-hand side here is a custom block we made called feedback. And it says feedback for Stuart because this is a screenshot from Stuart's page, which is me. What it does is it goes off and it gives you a link to the activity that you've received any feedback from. It would show you the name of the cheater in the picture if that's allowed by permissions because students really like getting personal feedback from cheaters and having meant deference to their teachers and things. And it just makes it feel really personal as well, which is important. And then underneath that, you've got a link to your full grades page and things like that. But clicking on each of these links just takes you straight to the activity. Yeah, so we wanted to solve the issue that students couldn't find the feedback. We think this might do it. Next slide, please, Elliot. And using events. Skip, go. It looks like UCL's using events, that's fine. We did it. It looks like a design system. That's all we need to do. Okay, so once you get to a course page, we haven't done much with the course pages at the moment, but well, we've just done a massive research project into formats, which was pretty interesting. And that's a whole another presentation. And maybe next year at Middlemoot, we'll see.  We show the category, the name of the course, the course description, and the progress. And then kind of tidied up some of the other bits to make it match UCL's brand a bit more. I think that was about it, really, on that one. Next slide, please, Elliot. Evaluation. Yeah, so I'll jump back in. So just on evaluations. So as Stuart said, so we've got this kind of, we're trying to really embed this benchmarking process. So to run these user interviews with at least five students regularly in our, I guess, in our year. And that's part of our agile approach to the development. So we're constantly iterating and releasing, hopefully improved and new kind of developments on Moodle. So we ran our initial session with the kind of mock-ups of what the theme could be. Then we kind of did the theme work and the dashboard work. And then we ran a second round of interviews in November last year. And so instead of the kind of mock data, we had the live Moodle site with the new login page, the dashboard page and the course page headers that Stuart has taken us through. So how did we do? So this is an interesting one. So this is like a poll. So similarly we polled about 100 students and we asked them a series of questions, one of which was like, what's the worst thing on Moodle? So in March, this is kind of a word cloud of their free text responses. And then we re-ran that thing in the same question in November. And then you get a different word cloud. And you can see in the first one, there's kind of slowness being logged out and time is kind of a key issue. And then there are things about kind of lesser things about maybe the old thing being a bit ugly, for instance, or inconsistency. When we re-run that survey, the slowness and logging out kind of disappears, which is good. Now, I think that was a suggestion that Stuart maybe kind of led on about increasing the session time out on Moodle. I think we had it at two hours and we increased it to 48 hours, something like that. So kind of soft. When we were doing the interviews, there's a student with three tabs open or with Moodle open and then they're on a call with us and they go back and they're like, oh, it's logged me out again. And it's kind of like, wow, we're really doing that to students. That's bad. Let's stop that. Yeah. So part of these interviews, we were focused on kind of the theme, but then other issues kind of emerged. So it was really valuable to just sit down with users and talk to them. And you can see the issues in November. So again, the worst things are around pages and courses. And that's hardly why we have, or fits very well with our course format work. So how can we, you know, improve course formats so that they're more organized, easy to navigate, more usable to students. So this, I guess it shows we still, we still got challenges that we're working on. But this has given us key feedback that we can then put into our, you know, development and our projects and things. People mentioned that we left off block. We had quite a bit of feedback from students in our November interviews. The main one was that we initially showed three courses, but students mostly have four courses. So they wanted to increase. So this is like the new design that we're going to launch very soon and then evaluate it. And then now the feedback was maybe that the activities people didn't necessarily realize the activities was there, so we made it more prominent as these kinds of buttons at the top. So yeah, again, we're trying to reiterate on this approach and then evaluate and see how it goes. In the, so the exit surveys for our new rounds of students, we had some quite positive feedback. So obviously the visual appeal, which a lot of the work we did kind of related to kind of increased in their score that they gave it. So 3.75 in the March interviews to 4.25. And so the content designer, the front page and dashboard page had been updated since the previous test. Navigation and findability remained the same at 3.25 and we'd like to improve this. So while we're looking at course formats and other little things around moodle. And then the ease of use for elements increased from 3.25 to four. This could relate to the participants' increased familiarity with moodle four, given they had more time to use it. But yeah, that's pretty promising. We also did this kind of visual design, I guess adjectives that the students rank, whether the design is more or less creative, calm, busy or all these different adjectives that you got up on the page. We're happy to find that the highest move was on inspiring, which is good. So from 2.75 in the old surveys to four. And then this table kind of breaks down. So the items on the left, the adjectives went up. So people kind of ranked them. The new design is more creative, more cutting edge, more fresh. I think someone put in the chat fresh as well, which is good. We think it's fresh. And then down some interesting things. I mean, calm went down, which is interesting to interpret. But people didn't necessarily think it as cheap or old or unprofessional or inconsistent. So they thought the new design was, the opposite of these things, so it improved. And there was no change on busy and trustworthy. I guess this gives us a sense of the kind of how students are perceiving the visual design in terms of descriptive, but somewhat kind of emotional level. So it's quite helpful to see. Generally quite positive there. One of my projects at the moment is around data analysis. So we are using different tools to try and see how users are responding to the new Moodle design. So in addition to the interviews, we have Microsoft Clarity, which is a heat mapping tool and also does different web traffic analytics. So this is a heat map of the dashboard. You can see where the red is indicated is where most users are clicking. So we find that the first thing is that first card on the pickup block and the second card. And then our users are going to view all courses and then the third card. So we would suggest, it kind of confirms that the key task is to get to courses on this page. And we're gonna be evaluating this because it's clear that kind of students go into the initial courses they've accessed, but then also go to the view all courses. We just need to be careful that we're allowing them to access their courses as quick as possible. And that's why we're doing the four courses here to hopefully get the balance there, right? Also quite promising is that you can see the red further down the page. So some interactions with the timeline block. This is obviously gonna occur at a less frequent rate than courses because it's the timeline blocks gonna show the deadlines when students have them. And there's also some interaction with the feedback block on the right that you can see, which is also promising. So again, we've got evidence that when students are getting their grades, they're starting to go there and see content. So we're gonna keep an eye on that. This graph is just showing that, so in the theme we kind of promoted in the course page header that it showed an image, it showed a progress bar if completion tracking was enabled on a course and it also showed a bit of the course summary. So we're promoting that in the theme. And this graph just shows you that these elements are being used more for this academic year, which is good. So particularly that kind of orange or tan line that's gone up, which is more users using course images in their courses. And that's something we promote at the top of a course page. So that looks promising. So if you have any questions about how we're evaluating it, do put in the chat. I might hand over to Stuart if you wanna talk about course formats. Yeah, so we've been looking at, as said, we've just finished a big project on course formats. And this is kind of a preview of what our course format might look like in the feature. So that was it really. We might have two course formats because we might have one for those kind of courses that people put in middle that are really how to use the library and those kind of information things. But this is kind of gonna be probably what we do for blended learning. And is the next slide back to that to the UX workflow top testing thing, Elliot? It's Q and A. You put the UX testing workflow back one wherever that was. It's fun to meet. Yeah, so I just wanted to say, like on this one, I think we kind of missed it somehow and when we presented, but with each of those elements, the pick up where you left off, the timeline, the feedback, you don't just put them all live and then see what happens when you're developing each of those individual elements, you test it with users as you're going along. I hope that's clear to everybody from the diagram that you kind of start with something that's a simple wireframe of what you think is a good idea. You test it with users to see if it works, if it gives them all the data that they need if it's easy to use and if it isn't, you fix it. And so then when each of those elements is, yeah, they can all be improved iteratively and we test before we put it all just go, it's all live now. Yeah, you test during design as well, if that makes sense. All right, so. All right, thank you. Nice page. Thanks everyone. I hope you enjoyed the talk. We can go through the chat from the top or don't really mind. Maybe do that. If people want to jump in on the mic, we can, or what in the chat, we can answer those questions. So I'll just look back over the chat. And the first question was from Russ Ward, who asked, so when you say 100 users were surveyed, is that all roles or specifically undergrad students? I can answer that. So we put our survey just up on Moodle as a link to Microsoft Form, so just on the homepage. So anyone could kind of access it, but we advertise it as for students because we really wanted to target them. And then we asked them what kind of, what was their enrollment at UCL? I'll perhaps look up the stats of people interested, but it was mostly, yeah, undergrad students, but with a mix of post-grad, post-grad and some research students. And then we asked them as well, series of questions, and then if they wanted to attend a follow-up interview. And so we were targeting very much a different range of students. And then that goes well with the next question. So Xavier has asked, did you have different personas or just cleverly picked your student sample? Do you want to answer that, Stuart? Yeah, we just tried to be clever with picking our student sample. Cool, and then Fabian's asked, where are you using Boost or another theme? Would you create your own theme? That's a really good question actually. So we did start with a different theme. Might stop sharing, because maybe we don't need to. So we worked with Titus Learning on their Norse theme. That was the initial theme that we were using at UCL. And they were super helpful in working through our ideas and developing those initial mock-ups. So we worked with them to do that. And then we have now undertaken that Stuart can perhaps talk more about of building a UCL version of our theme. Just because we found that our requirements have kind of maybe diverged a bit from what the Norse theme does. So we're kind of very much in a lucky position to build our own UCL theme. Which we will open source. Yeah. And similar to most of the plugins, I think all the plugins actually you've seen today, will be open sourced pretty much. Certainly the pick-up block, for instance. The back one is, there's a few people using the pick-up block already, I've noticed. So yeah, that's good. So another question Vanessa, we might have answered it, but what theme do you have on your Moodle? Are you using a custom theme? I think we've answered that. What else we've got? What system did you use for analyzing your qualitative data? So we mainly looked at ourselves and I guess categorized it into different themes, top usability themes based on frequency. So we didn't use any other tools per se. I like using the Rice methodology. Google it. That's a really good one for prioritization. That means ask kinds of extending session time. None that I'm aware of. So far everyone's happy with us. Yeah, I guess if you leave your computer open and you're using like a computer in the library, a shared computer or something and you forget to log out. Xavier's asked, are you using Iotracker? I thought you would. No, we didn't use that. We might use it in future because some of our computer science departments would lend us their software and systems, but no, haven't used that yet. If you've used that, it'd be really interesting, maybe put in the chat what you've used before or found. That'd be helpful. Katya has asked, how many designers, developers worked on the new theme? That's a good question. So, I mean, we initially worked with Titus, but I would say most of the work's been done with myself from a kind of product owner of the theme, planning, user engagement side, and then our main designer developer is Stuart. And then we do have a development team, that they've kind of chipped in on different elements. But yeah, so here's the pickup. We are a third block available to the public. Yes, it is. Are you trialing the course format first for any particular faculties? Are you expecting a period where all courses switch to a new format over summer, automated or manual? Might of my own assuming that. So, we've done a research project where we again interviewed students on a series of course formats. We targeted the most used ones, which are topics, kind of the tabs format, and also kind of grid, more visual display formats. And this was more of a research project, which out of that, we produced a presentation and also options paper for senior leadership. That still has to go to senior leadership to approve. So, we haven't necessarily agreed at UCL. We very much hope they agreed to building a new course format because we can see how it'll improve a lot of the issues students are having. If we do, if that does get agreed, I think we would then make it part of our automatic course rollover process so that staff could convert their old courses to this new format. And we provide support around that. And we would be trialing it. We would make it available when people do that course rollover. So yeah, summer and then leading up to the new academic year. If people have moved to a course format, that would be awesome too. It's like a single course format, that institution that would be good to share. How you communicated that, because we very much haven't started that planning. We're still waiting for approval. Okay, Vanessa's asked, do students land on the dashboard after login is the home page before login? Login takes into the dashboard. Yeah. And the home page is before login. So what we called the landing page was before login. But yeah, once students are gone, they go to the dashboard. Leslie's asked, what were the students' thoughts on the course index and block drawer within course pages? We're finding that our students are not using these or they don't know about them as they are not open by default. You wanna answer that, Stuart? I think it's another talk, isn't it? We will show our research on course formats at a different time, if that's okay, Leslie. Yeah, I think in general, we found similar results to what you were saying. Maybe the other way around, because I think they are open by default for some reason in all moodle. So then students were getting tied up in them and not really paying any attention to the main content and going, I'm meant to use this. It's open and kind of failing. They're open by default, but then they change to the user's preference. So if a user closes them, they don't reopen when they go to the page. So I think we found many students, most students probably didn't use them, probably closed them when they went to a course and then hadn't, like in the interviews, certainly didn't reopen them. So yeah, it would suggest they're easily lost, but yeah, we could go into this a lot. The block drawer having like stuff for power users and admin in, I think is really good. You know, it's stuff you don't need all the time. You see how we've got loads of custom admin teacher, editor blocks and things and having those somewhere. So it's not just taking up room, distracting for the users is really good. And the course format page we showed briefly did have an idea of maybe having a menu, course menu on the left-hand side, which potentially it would always show. So students don't lose it and would only show kind of section headings rather than all the content on the course. I think a lot of the issues with the course index menu has been like we have lots of content on our courses, which it's perhaps not designed to do and it just shows them, shows all that content and becomes, some people have said like a second scroll of death. So yeah, we're definitely very aware and interested in that topic and trying to improve it, I think, these two elements. Amy's asked a good question. So I'd be interested to hear how you address accessibility and inclusivity in your design. Did you include a diverse range of students in terms of neurodiversity disability, for example? Stuart, do you want to access really this action of yours as it's- No, for the second part, we didn't test with any users with any range of neurodiversity at all for accessibility and I'm really passionate about accessibility and I've done talks at quite a few accessibility events in the United States. Used to work hand in hand with a developer who was blind and yeah, when you build things, you test for accessibility as you're building them. You don't make them inaccessible by design and then when you launch them, they're accessible and that's it, but you kind of just build. Accessibility, accessible by design is our solution generally to things, I think. There's also a really good team at UCL who do kind of will run the lighthouse checks, kind of that can run saying things like, the area role is different from the button name and the contrast isn't high enough for muted text on a dark background and I think UCL generally try and report those to Moodle Tracker or get them fixed by Catalyst if they're kind of, because there's lots of them and more core issues. But when we're building stuff, we just build it accessible by default. Yeah, so our accessibility team did audit the theme and then we either resolved those issues. Most of them we resolved or logged with Moodle and the issues we couldn't are in our accessibility statement on Moodle and we've tried to, through the accessibility team, I think, tend to liaise with students with different needs and kind of feedback to us and then we also have a staff user group that we've encouraged a range of different students, so staff, so in needs to join and provide feedback through there. I think it's always a challenge, we could definitely improve, but so far feedback's been good and our accessibility statement is pretty short, actually, there's not too many issues on there, which is good. So Xavier's asked more concurrent users more expensive to host, yes, about the session timeout. Not sure, we haven't heard anything back about that, we could look at it. But yeah, it's good, possibly. He's asked, is there any plans to create the course module visibility as a looped format? Students are able to view the course modules after the finished slash previous configured one and the accessibility to the modules is vaguely currently. Don't quite understand, don't know if anyone else does, but if you wanna ask your question and the mic, if that is a question, not sure. Tim's asked, was there a reason you concentrated your time on the initial part of the user journey rather than in course experience? I think so we wanted to, so we say with Moodle 4, we're quite kind of, users to get into the courses have a clear journey that we found, so that was the login page, the dashboard. So we concentrated on those initially, just because if you get those right, that's the first start of most students' journeys, most staff as well. And then now we're very much looking at the in-course experience, so this is what that whole course format interviews were and survey sessions that we ran. Hopefully we can report on them in another meeting or at Moodle Mood, because yeah, we followed a similar methodology, but also kind of streamed our surveys to a group of maybe 20 to 30 staff members that could watch and see students navigating through a course. So that was kind of really impactful and led to a lot of really healthy discussion with our different kind of digital education, e-learning support folks and staff around UCL. Yeah, formats are more difficult. One doesn't simply mandate to a university that every tutor must use a format or you'll get a riot going on, won't you? You have a rebellion. You've got to kind of manage it and I think one of the best ways to do that is to get the people who are building your courses, designing your courses, department heads, anybody else to actually watch how users use the Moodle courses, where they're struggling and understand the pains and complexities of their end users. And then you might get memento towards moving to better course formats, but until you've done that, maybe it's a bit difficult to kind of mandate everybody has to do one thing. Yeah, nine formats to three, that sounds great. Wow, more than nine formats, I didn't, but maybe we can answer in the chat now. Nine, oh my God. Again, she was paralysis. Did your tutors have a degree in Moodle course formats to be able to choose which ones to use? Yeah, I believe we have five formats, I'm right, but they used to be six. I'm not sure. I think we have to hand back to Sandra and Mary, but people can go to contact details. We'd love to continue the discussion. I might just answer two quick questions. Russ said, did you use a Moodle app? We don't really promote the app. We have our own UCO app where people use the browser on their phone. That's a whole nother talk as well. And then we do have a separate Moodle instance where we tested these things. So we have tested instances. Cool, thanks everyone for joining. Really, really good to hear feedback as well and to share our findings. Thank you so much both of you for being here. It's been great and thanks for all the interesting questions. I guess we could continue going on for a little bit, but it's good to know that, you know, you might have some more content to share with us, maybe at a future Moodle Mood Global. So that would be good to see. But if you have enjoyed this session, we'd love to, we'd love you to consider getting involved further and help us grow by contributing to the development of Moodle Academy. You can do this by visiting our Get Involved course, which you'll find on the front page of the Moodle Academy site. You can suggest ideas for new webinars and courses and you can vote on ideas that have been suggested by others already. We're always in the lookout for community members to help present webinars and co-create short online courses. And we'd love you and we'd love your help, basically making Moodle Academy more inclusive. So if you're able to please jump in or translate Moodle Academy course and get started with helping us translate our courses and webinars into other languages. And of course, please help spread the word about Moodle Academy by telling your colleagues about the courses we offer and the events we run. When you complete the course, you will earn a badge. You will also receive a badge if you present a Moodle Academy webinar like today's session. Educators might like to think about getting involved with the Moodle Educator Certificate. You can take the are you ready for the MEC quiz and one of our certified service providers will support you through the certification process. So that's all from my side. Thank you so much for joining us today and I hope you find this useful. And I also hope to see you on the Moodle Academy course forums and in our coming webinars. So thank you again for joining us today and thank you to Elliot and Stuart for being here today. And we hope to see you soon at future Moodle Academy webinars.


From the heart of the jungle comes a savage cry of victory. This is Tarzan, Lord of the Jungle. From the black core of dark Africa, land of enchantment, mystery and violence comes one of the most colorful figures of all time. Transcribed from the immortal pen of Edgar Rice Burroughs. Tarzan, the bronzed white son of the jungle. And now in the very words of Mr. Burroughs, the story of Tarzan in captivity. Tarzan had left the city of Grutendal several days before. He had traveled both night and day, anxious to be in his jungle once again. And now his goal reached, he rested in the crotch of a tree, slept as peacefully as you or I might sleep on a feather bed. He was completely relaxed because he was confident that even in sleep his sensitive nostrils would catch the scent of an approaching enemy. His acute hearing warned him of the coming of man or beast. And yet Tarzan did not stir as the gray bearded Arab walked noiselessly along the jungle path, climbed the tree where Tarzan rested and crouched on a nearby branch, looking at the jungle man curiously. Allah grants such sleep only to the innocent. Who are you? What do you want? You need not reach for your knife Tarzan. Had I meant you harm I would not have delayed my act until you awakened. Yes, I suppose that's true. As Allah is my witness it is true. The wise need not mask their words in falsehoods when truth conquers citadels. Your words are as strange to me as your actions. Perhaps it's because I am confused. Never before has anyone approached me without giving some hint of his coming. I heard no sound, I got no scent. In every age there lives one who surmounts the obstacles of the seven senses. The seven senses? You speak in the most fantastic riddles. But all riddles have answers if we know where to find them. The answers to those now uppermost in your mind lie with me. You heard me not because I made no sound. You failed to catch my scent because I did not imbue the air with one. Who are you? I am the Grand Wazir of the Caliph of Karadan. You have heard of the Caliph, my master? Very little, for as I understand it his province is surrounded by a wall so high no one can scale it and no one passes through its gate without his bidding. And now you are bid to enter this kingdom. You are invited to be the guest of the spiritual and civil leader of tens of thousands of people. The Caliph is inviting me to be his guest? Why? One does not question the motives of our potentate. Perhaps it was rude of me. Thank your master for me if you will tell him I have just returned to my jungle perhaps in a few months. The invitation is for now. The way you say invitation it sounds like command. The Lord of the jungle may make his own interpretation of my word. I have done so and whether I am invited or commanded to come to Karadan the answer is still the same. I am not leaving my jungle home at the moment. Tarzan is a good servant of the Caliph I attempted to use whatever graciousness Allah has bestowed upon me to persuade you to return. Since my humble efforts have failed I shall have to take other measures to ensure your appearance in our capital. You intend taking me there by force? I am the Grand Wazir. A wizard, huh? The term cannot be so easily translated as I have learned the secrets of moving noiselessly through your jungle of erasing every vestige of my body sent so have I mastered other secrets of nature. No no I shall not have to exert force at least not my physical force. In just a moment we shall return to our story of Tarzan in captivity. A Grand Wazir disappeared as silently as he had come seeming to melt into the darkest regions of the primordial jungle. Tarzan did not know fear in its ordinary sense but his brow was furrowed and his thoughts still remain with the emissary of the Caliph as he made his way cautiously through the jungle and even as his thoughts conjured possible dangers from the Wazir a sleek powerful black panther came toward him but as it sprang Tarzan turned twisted from its path pulled his great hunting knife from its sheath the beast was a clawing spitting fury of destruction as it lunged at Tarzan's throat but Tarzan moved to split second before the cruel fangs could find their mark and with one great stroke his life plunged deep into the flesh beneath a shiny black fur another black panther and another and shoot a whole pack of them but this is mad black panthers in this district in a pack of them they don't travel and back the Grand Wazir. Ah it's well that i am here and that these black panthers do my bidding another moment in you would have gone to your maker. It is true until you spoke there were snarling messengers of death but now they stand like household pets. Yet with another word i can have them rend you limb from limb now do you return with me to Karadan? I seem to have no choice take me to your caliph so i should have accepted your first invitation Grand Wazir. I have never seen such splendor it's something to remember. These halls out of Marble brought over 8,000 miles for the caliph's palace the walls of the throne room are encrusted with more jewels than you could find in the earth we're to have a thousand miners dig for a thousand years. Are we on our way to the throne room now? No the caliph has no desire to have his emirs viceroys or grandees of the realm learn of his business with you we are invited to his chambers. Invited you you still have a strange concept of the meaning of that word. This is the entrance to the caliph's chambers. Tarzan i would humbly offer you a bit of advice do not anger the caliph do not for example depreciate his invitation you have your choice of being treated as an honored guest or as a slave. During the few hours i've been here i have seen how your slaves are treated you witnessed but a few lashes of chastisement there are more ah serious punishments for more serious crimes this is the door to the caliph's sanctum. The Grand Wazir. It is indeed the Grand Wazir your excellency. Enter enter.  Now we have no need for such ceremony between friends welcome Tarzan i am extraordinarily happy that you have consented to come to Harada. I am happy to be here i desire to learn more of the people who can train Sheetah the panther to do their bidding. Oh you refer to the extraordinary pets of the Grand Wazir. They are dangerous animals despite his training i do not like having them above their presence in the courtyard surrounding the palace is most useful it keeps those who should not enter on the outside and those who should not leave within. It must be a pretty sight to see your pets prevent the escape of a guest. The Wazir is filled with an extraordinary sense of the dramatic the panthers are kept caged except at night. During the day there is no need for them not with an entire company of fierce Senegalese whose chief delight is in the use of their bayonet. You may save yourself the trouble of going into further particulars about your means of protection i uh i'm quite confident no one could gain access to the palace without consent or leave without permission.  If you do the bidding of his excellency really my faithful courtier i'm afraid i cannot conduct a normal conversation with Tarzan in your presence your excellency wishes me to leave no i should discuss my business with Tarzan later i'm now extraordinarily retired and surely Tarzan must be wary from his long trip with your excellency's permission i would discuss this business for which i've been brought here now. I realize your impatience Tarzan but i desire to delay our conversation until another can be present i know you will take my word that that is best all right we shall wait then thank you Tarzan i am sure that our hearts are at tune Allah has sent you to me as a brother he has provided me with a man to whom i can bear my soul you will be the instrument through which i can accomplish my great purpose mihil maktub it is written thus mihil maktub it is written thus strange enigmatic words that meant everything and nothing what business was there to be transacted between Tarzan and the caliph of karadan how was Tarzan to serve as the instrument through which the caliph was to accomplish some great purpose the questions crowded Tarzan's mind as the grand was here guided him through the endless corridors of the extravagant palace by a hundred gold encrusted doors passed a score of watchful sentries through an exquisite inner garden and to the gates of a special wing of the palace a section of the ornate structure even more luxurious than the rest if such a thing were possible your courtes are here Tarzan the slaves will do everything possible to make you comfortable at least my legs will not become cramped in these quarters i think this wing must be as large as the village of the punyas all of this is for my use oh no there are other guests in this wing you see our diplomatic visitors are quartered in the jade wing those we invite here for matters of trade relations are housed in the pearl wing this section is kept for our permanent guests permanent i shall leave you here the slaves have been instructed to do your bidding the caliph will let you know when he desires to communicate with you how will i know where to find i shall be happy to lead you to your chambers what this great hall leads to the sleeping rooms of those of us who reside here and also to the drawing rooms and libraries and other facilities the caliph provides for us i see then you are a guest also yes i am made up you thought me a slave girl well your clothes i have long since adopted the clothing of the harem they are more comfortable and they seem to blend more naturally with the surroundings i have been a guest here for almost 20 years for 20 years but you seem scarcely more than 20 years of age now i was but a child when i came here at the caliph's invitation my father was the ruler of a small province near here and protocol demanded that the invitation be accepted of course while i was here a revolution broke out in my country my father was killed i'm sorry perhaps the revolution was not instigated by the caliph but he did not desire that i should return to my people for he was anxious to annex my country to his and if you return your people might have elected you ruler women do not rule in this land but they sometimes have influence only were i to return now what could i tell them nothing save that i have been treated as a daughter that the caliph has showered me with gifts that every desire has been satisfied almost before the wish found voice i see and what about the other guests here a scientist who parted with many wonderful secrets and was rewarded richly but cannot be permitted to leave lest he also tell his secret to others some of the grand wasier's information comes from him then they meet often also the wasier has many conferences with the others the architect whose genius must design buildings for the caliph alone the goldsmith the lapidary the gunsmith the others but i am not an artisan what what can he want with me i do not know but i know this regardless of what the caliph tells you regardless of how charming he is to you he will never permit your departure even after you have served his purpose it is his inflexible rule and should you refuse to serve him yes what if i should refuse to do what he asks i have been here 20 years many nights have i been sleepless kept awake by the screams and many days have been filled with the horror i have seen in the courtyard the screams the the anguish cries of torment came from those who refuse the bidding of the caliph of caradan in just a moment we shall learn of the caliph's plans for tarzan although tarzan much preferred to eat the meat of freshly killed game and to drink water from a swiftly flowing jungle stream he could not help but admit that the repast served to him in the wing permanent guests was a delicious one an assortment of the orients richest delicacies and most delicate ambrosias and the company was equally delightful skilled artisans painters musicians scientists masters of everything save their own fate and when dinner was over tarzan was escorted to the caliph sanctum once more there to learn his fate pray be seated tarzan i wasn't sure i was permitted to sit in your presence oh you are as yet a stranger here we shall not be stern about matters of custom until you are used to us thank you for your leniency for your future guidance however i might point out that there you must be seated at once when i am seated and should i recline you must recline never must your head be higher than that of the caliph my height makes that difficult you will incline your head to compensate for the extraordinary difference in our stature oh not now there is time for such formal behavior later now i would talk to you as a brother i certainly have no desire to delay our talk but you did say something about waiting until someone else could be present my ibn will be present shortly ibn i'm i'm sorry my knowledge of arabic is very limited oh a thousand pardons ibn means son the amir of caradan will present himself when the time comes amir that means prince precisely and it is of the amir that i would speak to you it is for him that i have called you from your jungle i do not understand oh you will it will be an extraordinarily clear to you in a moment i hope so when you entered my province you observed a great wall so high that no one not even the lord of the jungle can scale it i saw the wall for many generations it has kept us secluded alone free from foreign influences away from the contamination of other peoples yes but we are living in a different world no longer can my people stay apart from others no longer can the walls serve as an effective barrier against progress soon my people will intermarry with others their ideas will become ours their customs will intermingle with those of caradan but what is this to do with the amir and me i was raised in the old manner my speech is that of another age because i was a prince i could not learn to play as other boys i could not attend school i could not learn the ways of the outside world my son must be raised different but you surely don't think of me as a teacher and extraordinary teacher i did much research concerning you i've ascertained that you excel at every form of athletics i've ascertained that you speak a dozen languages that you have knowledge of many subjects i want you to train the amir so that he can be prepared to face a modern word but i am not a teacher that there are many tutors you could engage with my son will be taught by no spectacle wearing mincing school teacher he shall be taught the ways of man by a man i the caliph of caradan have spoken i see i pull the silken cord that will bring my son here of course it is your privilege to turn down this assignment if you are not made happy by my request i am free to refuse your request but of course we give the extraordinary pleasure were you to accept my son something has caused you annoyance such a slave girl she failed to cast her eyes to the ground as i passed she will not repeat the offense i can promise you that it is well amir come close to this stranger in our midst he has tars on lord of the jungle you are a member of nobility my subject number thousands of savages and tens of thousands of animals ah i think you hold a position that entitles you to speak to me without kneeling thank you for the privilege i do not permit most people to converse with me except through an intermediary holds noble rank beloved son i have been persuading tazan to accept the role of your tutor would star be pleased with him as a guide to the workings of the modern word could he make me strong like he is strong he could and i will accept his teachings i desire a greater strength in the wielding of my whip tazan now that you have met the amir will you accept your assignment there are many things i should like to teach the noble prince i do accept such a change in anyone in my life tazan what have you done to the prince i have attempted to teach him humility sympathy and understanding of a few new values and he looks so different why during the few months you have been here he must have grown inches more important than that i've tried to instill in him the idea that the stronger one is the more restraint one must show in the display of strength it is a lesson his father might well learn it's too late to teach the caliph i'm afraid but perhaps the lessons may help his son to exist in a modern world what is the modern world like tazan what kind of clothing do women wear what are the latest fashions and carriages and is it true that now people can speak to one another at some distance by means of some invention but surely the scientists here have told you of the outer world one finds it hard to believe stories that might as well be fairy tales told to children mida i have about completed my mission here and when it is complete i shall be a a permanent guest who proves most impermanent if i can i shall free you also not enjoy one more series of exercises the mind must be developed too i'm here oh here is a slave with some cool drinks we will enjoy them during our french lesson i bow low and profit sweet ambrosia noble army her favorite son right slave in the modern world one does not bow before another but thank you very much for having brought the refreshments blessings upon the noble prince speak well tazan you spoke like a real prince among men what you must learn to speak so in many times for someday you will travel now the lesson vous voulez vous me dire quelque chose de la fête du 14 juillet eh bien c'est la fête nozionale or commemoration de la bestie very good very good tazan tell me what is bestial day really i mean the thought has been praying on my mind yes and well it might for in every land there comes a time when the subjects of an autocratic ruler turned to revolution on bestial day on july 14 1789 their french threw off the yoke of their ruler a few years earlier in 1776 america had escaped from england's tyranny this was the birth of democracy something that is not known here but your excellency tazen's presence threatens a very way of life perhaps you exaggerate grand wasi but i have heard these things with my own ears were they are mere to succeed to your throne he might well do away with a form of government extraordinary extraordinary obviously something must be done about tazan tazen by morning they will come to the wing of permanent guests a dozen slaves will pounce upon you as you sleep you'll be dragged to the dungeons it is time for me to leave now anyway mita my real purpose has been accomplished but the centuries and the panthers i shall have to take my chances do you wish to brave death for liberty no i am not afraid of death but i am unprepared for the world outside i shall wait here and hope that the seeds you have planted namir will someday bear fruit goodbye then mita perhaps i shall be the first permanent guest to make his escape and perhaps not tazen moved softly along the marble corridors he slid quietly through the many gold encrusted doors he leaped across the moat surrounding the palace and managed miraculously to avoid both the centuries and the panthers but the great obstacle was yet to be overcome the high wall that surrounded the city mile after mile tazen searched for some spot that might be scaled but he could find none and then suddenly almost directly at his side a section of the wall swung over you eat tazen here amir what is this how did you get here part of the great woe is hollow and disconnected with the palace by an underground passage well how did you know that i must hurry you can reach a jungle through this opening the grand marcius learned of your departure unless you pass through the wall here you will stand little chance hurry please and i'll be with you goodbye amir and may you someday rule with the love and affection of your people goodbye tazen you did well my son tazen don't even suspect i had anything to do with his escape do you think the grand marzia may yet cause tazen trouble it will be some time before the wasier escapes from the cell he himself designed for those who must be kept in captivity tazen will be safe in his jungle and likewise shall i tazen is truly an extraordinary person yes extraordinary we shall return in just a moment with a word about the next exciting story of tarsen 15 degrees north of the equator in the heart of the anglo-egyptian sudan the blue nile and the white nile meet forming a narrow hook of land strangely shaped like an elephant's the city that lies there has been given the arabic name for elephant's trunk cartoom and it is from cartoom that many many safaris have started in quest of the now almost forgotten gold of the sudan tarsen the transcribed creation of the famous edgar rice burrows is produced by walter white jr prepared for radio by bud lesser with original music by albert glasser this is a comodore production