Thank you. Oh, can you hear me? Yeah. Good. It's working. I'll try and entertain you before lunch as much as possible This is a slightly longer session. So I'll try and make it fun From Monash, I'm Cliff Ashford. I'm from the ICT function. So I'm not an educator but I came into Monash about a year and a half ago and my previous boss used to call me a Troublemaker and I sort of take that as a badgie of honour in sort of things I try and do at the university Let's see if the slides are working They are so Quick slide about Monash. This is the one that we also have to go Monash is quite big But there's a reason I'm putting it up here is because of the little Malaysia dot down here, which is Quite important in some of the things that we're trying to do with this space, but I'll get to that in a little bit Okay, and obviously we have lots of students so Does anyone know what this image is showing? So welding practice I put this slide in because I want people to understand where we're trying to go with what we're doing at Monash So this is a virtual welding simulator Which teaches and assesses welding better than using a welding torch Because you get much better feedback. You can see what's going on It can tell if you're going too fast if you're going too close all these various features the The technology is better than reality and it's something that you get great analytics back from my background is analytics I want to capture as much data about what's going on. This has very rich data I want you to bear that in mind as I go through the rest of the presentation By the way, that's been around for at least 10 years that technology We're still building enormous welding labs at universities that cost us Hundreds of thousands of dollars when we could be doing a lot more of this sort of thing You still do the practical in reality, but yeah, this is good to do too Okay I can't remember Monash administers 360,000 exams every year This is equivalent to 8.5 million sheets of A4 paper or a thousand trees Thousand trees from a plantation takes one acre and In virgin forest, it's even more each year We have thousands of students undertaking paper-based exams. It's the only time we ask students to sit down Silently in a huge room and just hand write for hours. The best thing about the whole platform is probably being able to see the Exam paper on one side and have my exam on the other side in the split screen function and just have everything in front of me as I work I thought that was really good in terms of setting up that that's how I set up my Physical desk for a handwritten exam and it's sort of it was really intuitive Typing something out is a lot more flexible than writing something out in terms of being able to change the order of my paragraphs Or go back and edit something without it giving too messy So I won't go through the whole video that's up there on YouTube if anyone wants to look at it and that was from last year so When I came into university a year and a half ago, I knew nothing about assessment I discovered obviously I went to university 30 some years ago and I remember doing some but not much about it lots of handwriting as I say what We're trying to do here is reimagine what we're doing with handwriting So you saw Chris Ryan in there who's the the head of the learning innovation department at Monash Chris and I have worked very closely on a variety of different things But one of the sort of most challenging things was the assessment program of work We originally sat down and just brainstormed this about little over a year ago about how to solve this The thing about assessment is there's lots of components And I couldn't have given you this list a year and a half ago Now I've been immersed in this for a long time There's huge complexity to what you do, especially when you're talking about the scale that's happening at Monash that 360,000 is about 250,000 take place every year on Prem if you like we take over Coldfield Racecourse, which we'll talk about in a bit and we invigilate all those students Literally thousands of students coming through the first time I saw the paper exam. I Almost sort of gave up the project. It was terrifying There's rooms with a thousand students sitting there scribbling way on sheets of paper and thought what on earth Do I think I'm doing trying to get into this space Obviously each of these stages that we have to address there's the creation the setting up of a new exam Finding questions novel ways of sort of assessing students Staging the exam setting up as a we take on Coldfield Racecourse and we have this mammoth procedure Twice a year primarily with some other sort of sessions outside of that Sitting the exam the advantage of Coldfield Racecourse is it's right by a railway station And we can have those thousands of students pouring through and then finally close to the hearts of an awful lot of our academics is marking improving the marking process That's far more deep and complicated than I ever imagined it could be and I've got a whole presentation just about marking Which I won't talk about today particularly This project has a lot of components to it In terms of what we're looking at here, there were a few sort of key things that we had about this So when I first discussed with the University what happened when we ran these huge paper exams Literally sacks of paper and I went and had a look at this and I couldn't believe this there's bin liners full of papers There are whiteboards with lots of things being checked out and checked back in if you've done marking at scale You'll be very familiar with this Shipping off these papers getting the mark shipping the back making sure we got them all making sure they're all marked There is a lot of error in that process We have one horror story at the University which I usually pull up when people are saying electronic assessment Isn't that dangerous? You end up in the news and all sorts of things go wrong One of our senior academics had two of these bin bags full of papers in the corner of his office Cleaners turned up thought it was recycling recycled them You can't see the bottom of realize the bottom of the screen has moodle moot in front of it Some of the advantages about getting things on to computerized system analytics again my background understanding What's going on with the analysis at fine-grained detail? We got a lot of data out of moodle We've got a lot more as a side effect of this Authenticity that bit at the start where I showed you the the welding simulator That's where we're trying to go but to do that We've got to have a platform to hook all this stuff into in the first place and student experience it's whenever you roll out a system at the university We have a lot of academics talking just about all the things that they want And we're going yeah, but there's a lot more students than academics So we'd quite like to make sure we got the student bit right First we're going to do lots of things but we're going to focus on student feedback and the student experience Sitting an exam is the most stressful thing. It's a horrible experience We're putting them on a new platform We want to try and make it as easy and pleasant and nice and helpful for the students as possible So we're actually assessing what they know rather than how they deal with a stressful situation Okay, so we started this really at the start of last year That very thin line at the left hand side Was our pilot group that we ran in semester one last year That was just 600 students We built this system I suppose it's no secret we built the system on moodle Why did we do that? I'll talk about that in a second, but in terms of the growth we have here 600 students We tried to grow it as fast as possible So our Vice Chancellor Margaret Gardner has made a couple of sort of statements about since you took over Well, lots of statements, but specifically about this at Monash Monash is one university Anything we roll out in terms of systems have to be rolled out enterprise-wide at the university There's no more pockets of this and pockets of that. We run stuff across the whole university Her Deputy Vice Chancellor in charge of education Sue Elliott We went to her and said we can do a couple of things with the system We can add lots and lots and lots of bells and whistles and lots of different ways of assessing just now and focus on that Or we can focus on pushing into as many students as possible first and then do the bells and whistles And she said I'll have more students please as soon as possible. We'll talk about the other stuff afterwards So we went on this rapid growth curve that you might be able to see so Semester one last year. We did 600 students semester one this year. We just completed last week. I think it was we did 37,000 students 37,000 students here in Australia And In the Malaysia campus combined with synchronized assessment taking place in both areas That's why the map was up earlier to tell you about that We we have another mandate at the university, which is equity We want to deliver exactly the same educational experience in Malaysia as we deliver in Australia And one of the things to prove this system would work in both those places flawlessly and have it synchronized We expect to do 40,000 seats in semester two this year That will give us about a third of our seats for the sort of on campus process that we run through Next year we want to double that we want to hit to about sort of 70 percent There's a lot of technical challenges in doing that But we'll get to those later But we are doing extremely ambitious growth curve because we want to stay ahead of Stay ahead of the wolves if you like people who try and drag us down One of the things about this project and what's worked so far is it isn't an it project It's not one coming from exams alone. It's not one coming just purely based on pedagogy From the very first stages Chris. So you saw earlier on the video Um Vern over there who runs exam services and runs these enormous exams and is thankfully the calmest man you've ever met in the world And me at the top we have this combination of the three of us working on this project together Any of the three of us can veto any decision at any point if we think we're pushing things too hard Raising the risk too much we can veto what's happening So we're trying to balance that rapid movement forward with good technology a good solution without having too much risk for the university That said we have a risk register of I imagine something at 100 different things each of those things has three or four at least mitigations on them If this thing happens then we fall back to this we fall back to this we fall back to this we fall back to this The last thing we want is to be on the cover of the newspapers on the tv for all the wrong reasons That happens around electronic assessment. We will not let that happen at monash For instance this last semester we still printed all the exams. We had them there if anything went wrong We've taken the paper copy roll it out to all the students in that and they'd run a paper exam instead We want to fall back.  Yes It's a much bigger team than Vern Chris and I of course This is the call field race course where you just finished the exams. I think that's the final day That's the team who were on site at the time and we did this big photograph of them all There's a lot of people involved in this project from an it point of view. It cuts across every discipline of it Okay, there's a timeline. I was talking about originally like everybody else. We looked at different systems off the market back in 2015 Those systems weren't providing what we needed Because one of the core things about a university or any educational establishment isn't so much the teaching It's the assessment. What we're what we're licensed to do is assess people It's good if we do the education that's tied into the assessment process and it should be tightly tied But what we're trying to do is assess We decided that a core function of the university as fundamental the future of Monash is possible We had to have complete control of Move forward to 2017 when I started at university We did this market scan. We looked at all different tools Chris and I sat down in a room and said how about we do this a different way We already have Moodle. We've had Moodle for six years. Why don't we look at using Moodle to be the basis of what we're going to do And I think is Andrew here somewhere We met in a coffee shop about the same time and said we've got this really weird idea. Do you think it's a good idea? And you went yeah, it'll be fine. No problem at all So with that level of confidence, we thought we'd go for it Okay 2018 We hammered into this so we hired a team. I imagine Darren is somewhere in the room as well Ah, there's Darren So Darren was the core of our team The good sort of he possibly convinced of it was a good idea as well because he'd previously been at the college of surgeons And had done a similar thing with the college of surgeons And we might think things are stressful at university, but the college of surgeons Significantly more stressful. So Having a couple of clear voices saying yeah, this is doable Yeah, we thought let's go for it Okay This is one of Chris Ryan's slides. Um, so I'll let you read a lot of it Pretty much it's saying that we want The unit cost of what we do when we run paper exams is very low Because we do so many students at the same time. We've ground it down over the years to be as efficient as possible What we want to do is do the same thing with electronic exams So we need to get scale as fast as possible because we get savings a side effect of scale Scales difficult with solutions like this But this is how we save the money Currently we have a fleet of people or a horrid of people that we pull in to read students writing And the writing is often A stream of consciousness that jumps up and down because it's oh, I forgot to mention that I'll put this I've got an arrow to go here. Oh, I need to write this a different way It's a hideous thing to mark if you tried to OCR that the the machine would go insane Just moving from the written to having a sensibly formatted long four months are ignoring all the multiple choice Ignoring all the auto marking stuff just that aspect of our exams If we hit 80 percent of our exams Compared to the statistics out there. It's twice as fast to mark a well typed exam At monash because the scale that saves us seven million bucks a year That pretty much pays for the project in one hit Getting that money back out of the faculties and into central front is another problem But we'll just deal with that one separately But it was a good it was a good argument to have with the vice chancellor She liked that number. She went with it We also my background is it obviously we want to run. This is an agile project The whole concept of those monolithic pieces of work that you do from start to finish and then roll out and everybody goes That isn't what I asked for that's dead Thankfully, so we worked through this project. We were trying to be agile We were trying to be best practice We were trying to make sure we addressed all the correct things that we had decent user interface and user experience That we made the system accessible for all the students. We have the 75,000 students Every variation on the theme you can possibly imagine with those numbers and the system has to support every single student We also wanted analytics at the far end of this because you want to understand what we're doing Most of our assessment process just now you can't see the detail of what's happening There's our architecture aws underneath Moodle on top catalyst on top lot making sure that whole process works For the first year we bought laptops The university doesn't have a bring your own device policy across the whole university So we couldn't just say bring your laptop to the exam and do this We needed to buy laptops in the first year. We bought 1500 laptops. You can do the arithmetic. That's a large number Sitting on the laptops we rolled out Sort of throughout last year octa, which is two factor authentication The idea of that is to try and make the university as secure as possible We want to ensure that the people logging into our network are the right people So we have octa in place and we have that's the symbol of safe exam browser I don't know if you've used it. It's a lockdown browser which takes control of your machine You can't do anything unless we configured it to happen. So we can whitelist websites We can whitelist applications, but by default it locks everything down That combination Is what we consider to be your assessment at the moment at the university Um, this is Is it clock counting? Yes, it is. This is actually a video. This is what it looks like So we've had various experience with Moodle over the years academics Some areas love it some hate anything that's in front of them What we want to do is just do some work which separated this from the the main Moodle So we have an enterprise Moodle, which may or may not be the biggest one in Australia If you went to an early presentation We have this one entirely separately hosted We've taken a lot of the features out what this is doing is running pretty much quiz We changed the look and feel of it At the time we were working on this we were still at version 3.1 I think we're at version 3.5 at the university on the enterprise Moodle Some of the things that we've done in this actually exist in the sort of core stream now So this should be familiar, but there are various features in here you can see You might have seen that one turn orange up that that's for the students who say I'm unsure about this question. I want to come back to it later Lots of things you've seen before, we can do various formats We've got six question types We know there's lots of question types in Moodle We don't want to do them all We wanted to keep it to six and make sure we did those really well And worked at the whole experience for the students So those types of questions are elegantly delivered I should be able to list them all But we've also got MCQ, we've got short form, we've got long form We've got sort of fill in the blanks We've got a drag and drop feature and possibly another one You can ask Darren later, he'll know these things But we tried to wrap a lot more around this to make sure it was a complete solution So you probably see at the top that little colored thing That's accessibility for color blindness So you click on that, it will give you different color sets So that you can discriminate what you're seeing on the screen We've got things up here like the highlighting mode Our first collaborators with the law faculty And what they do in law is they have a huge case study Is they want to be able to highlight bits of text Move them into the answer and then address what they're doing So essentially we replicated what they're currently doing With a large pile of highlighters when they're doing their exam Let me have a look at this We have a notes function which drops in So that another thing that came from the law faculty Is they're sort of essentially trained to take notes as much as possible Do all your notes and then turn it into your answer So that features there, it can collapse or expand as required It's everywhere just now, but it's primarily I think For use if you're doing the sort of long form answers of which they have a lot Left hand side, you'll recognize that navigation But we sort of pimped it up a bit from what was already in Moodle quiz So you can see the very situation is the one I've done The things that I'm unsure of We have some multiple sort of nested question types So you see that 15ABC in there We have the marks up there We have Pro, we got sort of strange feedbacks from academics Saying that you shouldn't be able to see the whole exam Because students will then game the exam And only answer the questions that they know Which I thought was a bit bizarre in terms of feedback Like again, we stopped listening to some of those academics And said what do the students want in terms of the experience As you can see it's quite a clean interface Now a lot of this stuff is taken away So it doesn't detract from what you're looking at We have the clock counting down at the top What I cut off as it started is the front page that you sit at And the exam comes live just before you're about to kick off Here is the long form answers So we put in a rich text entry box with various features It used to have icons all the way over here And the students said can you take away most of them Because they're confusing Can you just keep the ones that we want The other thing that was in there was this Which is an undo and a redo feature Which is really simple and something like word It's not simple when you get into something like this That was actually quite hard to implement And we had some interesting bugs associated with that But from that point of view we knew that the system Had to behave as much like a word experience as possible We don't want the students having to learn new behaviors Just when they do exams That's what we're trying to get away from So we put a lot of effort into this This is the widescreen view that we built for the students As well because when they're doing a case study They want a lot of room So this, you can have the case study over here And we have some very extensive ones With a notes area that will collapse down there And then they get this large editing point You saw in the video earlier they said This has had to lay out a desk during the exam So it felt familiar for them I'm probably going to skip You can see this, like I say, we'll release this whole Presentation so you can watch these videos to the end They are quite large What it eventually does at the end of this Is takes you to the summary page So the students can go in there And they can look through all the questions they've answered And then dive back to the ones that they want to look at Oh, word count We put that in because we didn't want a hard word limit In there because we don't want a student having to Reduce five words So they have exactly 3,000 words in their essay Okay A couple of words about the implementation That's our Caulfield campus That's Caulfield Racecourse Where we run the exams We run it because it's so big We had to take our network From there and stick it in there Never ever say to network engineers I don't care about the price Just triple gold plate it Because the original quote They gave me It was five times that when we installed it But we put multiple dedicated connections Between the campus and Caulfield Racecourse We put a complete new network in Caulfield Racecourse And the densest wi-fi implementation I believe in Australia And much broader than that potentially It took us a long time to design it Some of the stats When we built that We used 56 kilometers of network cable We put in video cameras We haven't done that before But we had several million dollars worth of laptop sitting on desk So we thought we should probably have some security We put those fiber optic links in $80,000 worth of patch leads I asked the guys to calculate the other day Because we went through and clipped off All these cable ties that we cabled up all those desks 850 meters just of cable ties And 1,500 laptops in 40 charging trolleys And again you can do the arithmetic for how much this cost Luckily we'll be saving money out of this Okay Every day the students were asked to do a survey At the end of the exam And we collated that as we went through This was the final one on the final day Very positive across the board from the students The interesting one is this question over here That said essentially e-exams are less stressful than paper exams Day one that was the other way round And throughout the two and a half weeks of the exams That slowly flipped until the students prefer the electronic to paper Okay We ran various focus groups And we had lots of feedback So we are trying to engage across the whole university as much as possible There's a very holistic piece of work Lots of people involved I'll run through some of these Because you might want to ask questions How are we doing for time Yeah we're pretty good Okay This was one of the ones that was our most prized feedback Because we had a senior academic It was very cynical about this And then chose to run some of his exams on it And then we got this feedback from him If you can read that My goodness this is absolutely wonderful Where has this been all my life We put this up every time Because it was a huge relief that we got this feedback We also have Some other interesting things we did So we got the students in If you've been to Monash campus That's our big learning in the round space In the learning and teaching building We just pulled students in and we said Tell us what's going on about this What do you think about the environment Lots of good feedback Students generally have been going Why do I have to do this writing thing For such a long time It's completely irrational that you asked me to do this You teach me through the whole process To use electronic techniques And then you make me write on a sheet of paper for three hours Okay, where next Harking back to that first slide This isn't about getting rid of paper What it was about was building a platform We chose Moodle as the platform Because Moodle is open source And extensible and solid and reliable And all our academics were familiar with it We found some good partners to work with us And some great developers to use that as our basis But we are taking it somewhere else Couple of things There's an awful lot of assessment at the university That needs some sort of free forum input It's not handwriting as such We want to get away from handwriting Because it's hard to read But what we do need to be able to do Is draw pictures and annotate things Also in things like pure maths We need to be able to do a mathematical formula And do it quickly You don't want someone to have to be Fighting with the experience to do that The other thing over here That's sort of a slightly ominous picture of a laptop That does two things One, we're going to bring your own device next year So we have a new policy at the university We're at limit to how big we can get I'm not buying another 1,500 laptops Could it get insane So what we're going to do is move to Primarily bring your own device at scale next year So to hit that 70% That'll be BYOD at the university We'll still have our 1,500 laptops Because we have to be able to swap laptops out For the students So if their laptop fails We bang another one in really fast So we need that process to work We also have a virtual delivery Of application program Which is one of the other things I've delivered this year And we want to be able to put the actual tools The students use into the application So rather than just a rich text box at the far side You would have Excel The entire faculty of accountancy said If you can do that We'll get everything on your system like that For IT We want to be able to put a full development IDE Into that So you code, you compile Everything takes place within the exam So it's authentic A couple of years time We want to be doing assessment of people's behavior We want to roll augmented and virtual reality into that So that's where we're going We're not about We're about Once we built the platform It's about what we do with it after the fact So this is running at speed And running quite ambitiously Okay Key takeaways There's lots of stats for you That you can take away from this Lots of those numbers Again, you get these slides afterwards But our focus was on students We're adding lots more functionality this afterwards The whole marketing experience Is like a project in its own right And then finally On the very last day We flew a drone down Caulfield Racecourse Just to have a look at the scale of it And it still horrifies me to some extent What we did We have another slide of the breakdown With time lapse But I didn't manage to get that for you in time But it is a large venue So that's our journey so far And we're hopefully going to go to in the future Any questions Presentation I did have a couple of questions Is there automatic saving in place Or are the students required to save As they're working through their assessment questions Good question One of the things When you originally set something like this That you can go two ways You can go with a client That has the entire assessment downloaded onto it And it's essentially local So that if anything happens to network It will still run We decided it's 2019 And networks are pretty good So we think we can rely on the network So from that We need to take advantage of the fact we have a network Every 30 seconds Whether the student does it or not So we're forever storing this back to the cloud Chink, chink, chink, chink, chink We've had situations where a student has Accidentally done the select all From a 3,000 word Sort of response that they've had in the long form And then press space And they've suddenly lost 3,000 words and panic We can catch them packing And we can go there And we can roll it straight back in for them And they have everything back in a few moments The other aspect of this Is we can see how they actually Write their assessment We can see how it builds up How the thinking process goes Because we have 30 second timestamps Of everything they've done through that whole period So the analytics about how people respond To long form answers We have sitting on the servers We haven't done that thing with it yet Because we've been so busy But we think there's educational advantages to this To see how people form an argument Does that cover that one off? Any more questions? Very good, thanks Cliff Students are bringing their own device How do you plan on locking down those devices And deploying per faculty or per assessment A machine, like you said, Excel or something So the lockdown will still be safe exam browser So one of our faculty is faculty of pharmacy They do a lot of in semester assessment As well as end of semester They volunteered Because they already had a de facto BYD policy To roll this out last year They were running slightly faster than we were They said the system works perfectly We'll just run it in the faculty rooms And we'll use that We will force the students To put a safe exam browser onto their own laptops And we will go through the lockdown process At the start of every one of these assessments Pharmacies quite bullish about these things They tell the students to bring laptops And they tell them how they're going to use them That is not the case of the whole university But they were a good place to start from Second part was Okay So currently we use We've got a program called Move Which uses Citrix to deliver All those sort of high powered apps to the students So they don't have to go to PC labs Because we think the PC labs are As they stand a waste of good flat floor space At the university We want to have fewer of them But the fewer labs have more sort of High tech equipment in them And larger screens and things like that So we get rid of those generic labs If you see the learning and teaching building To the new buildings at Monash The students sit everywhere Through these new sort of communal spaces With their laptops studying We want more of that And less of that formalized situation Did that cover that one off? Cool Anybody else? Good evening, Garon I'm really interested in their learning analytics Especially about learning styles Have you set into place the ability To judge the different type of question types? So kinesthetic learner might really enjoy A drag and drop question Have you put into place analytics To determine that as opposed to the paper based And the effectiveness of that teaching? So that's another really good one Because it's On the paper based side we have some analytics The whole trouble this is baseline The time to see how well things are going We expect to see some changes And we have seen some changes in assessment Since these things were done on paper This is an ongoing piece of work Because it's quite The devil's in the detail With this sort of analytics Initial position was to capture As much as possible And then see what we could do And understand where it's going We don't want to We want this to enhance The learning experience We want to roll this into semester We don't want to do end of semester exam We see the platform as a way of saying It's easy to run an invigilated exam It's no longer with paper You can do it in semester like pharmacy does Then with the other tools you already have In Moodle and some of the stuff That obviously open university put in As you're saying earlier That feature you do your multiple choice You get multiple hits at it Or you do your long form answer And you get the feedback And you've got all of that available to you So you can see you had to do things Better in the future That's I'm assured far better Educationally That's where we need to get to This stressful falling off of cliff At the end of semester is not the way We should be treating our students to my mind Any other question? Hi, have you started looking into Doing maybe advanced mathematics or physics And those sort of complicated formula On line assessments We have So we're trying to skin these Counts several different ways That's sort of the free hand approach And we know some things just need free hand You just need to build the right formula We do have various tools That will let you use an iPad Or a similar thing to write the formula in That's sort of working But it doesn't work very well for us at scale Again, so we have to We're actually attacking this problem In a variety of different ways There's nothing off the shelf You can buy that makes this work What we're doing for 2020 I suppose it is This year already we do some hybrid exams So if you've got some formula you've got to do Or you have a diagram you've got to do For example, sheets are still delivered Out to the students So they do most of it electronically That sheet is there That gets gathered up It gets scanned And then it gets given back to the academic for marking Next year we're moving to industrial scanners So it's not so manual We'll have essentially printed sheets With QR codes and various other things on there Those will be collected up And just dropped into the hopper on the scanners Darren's going to kill me about this in a bit We will script on the back end of that Gather up those scanned sheets And tie them straight back to the original Moodle quids So it's available in the same place for marking I am convinced it'll all be some sort of hybrid on this It's not like there is no single solution to any of this Some of the sort of drawing aspects That if you look in IT They're really pulling together flowcharts Or they're pulling together mind maps and things That stuff can be delivered via a graphics tool Really quite well It doesn't need to be freehand In fact it's nicer to do that I even hate drawing flowcharts by hand I'd rather do it in a tool So we're essentially trialling lots and lots Lots of different technologies To look at these really gnarly complicated things And not just saying We'll try one site's fits all for university That would be madness Anybody else? I've got a question in regards to You're bringing your own device policy How do you relate that with Or how do you associate that with your Wi-Fi policy? I appreciate that the assessment component You provide that But so I'm just wondering beyond the assessment Your Wi-Fi associated policies with it Do they provide their own? This is something we're looking at in our workspace Okay So Wi-Fi is obviously ubiquitous all over Monash What we had to do around assessment Is we couldn't have the two-factor authentication component Which requires you to have your phone When they were doing assessment So in that situation We create a separate Wi-Fi zone Around where it was taking place And we disable some of the two-factor authentication So you just go into your password The students are already connecting All the time to this And that process of coming in Through the Wi-Fi and having to go through our authentication area Locks the machine at that point We know which machine is being used We know where it's being used physically So it's been a sort of ongoing process With our Wi-Fi teams to work at How best to segment the spaces So we do it at Caulfield Racecourse We don't necessarily do it at the in-semester stuff Because you can see all the students In the room together using their laptop That said, we did have one student Trying to cheat with their mobile phone On our Wi-Fi In this last round of exams Where she would slide her phone out Take a photograph of the screen And then someone would text the results And she'd type the results in When we caught her, she denied doing it And we went, you see those six cameras right here We have you from every angle doing this And we found a lot of cheating from that point of view About as much as we get on paper Same sort of amount But we have clear evidence of it And it's one of those things We want to discourage people to even attempt to cheat Because we tend to catch it anyway But bringing that equipment around sort of dels it down Does that cover it? Or do you want more information about it? The only one Yeah Yeah Yeah So Monash has one giant Wi-Fi They have free Wi-Fi and logged on Wi-Fi Edge your own It's actually the same Wi-Fi We don't really separate it We spread the rumor that the free Wi-Fi is slower That the other one it's not really It's exactly the same We have time for one more very quick question Over here And then it'll be time for our lunch break Hold on for one more question Who writes the questions? And do you have any special custom tools to write the questions Or are there any templates for us Different types of questions Okay That comes to the question bank function So the academics are writing the questions As you know when you have quiz The sort of question bank is tied to the quiz What we've done is we torn the question bank away from quiz So question bank is freestanding That was a significant piece of development But we needed that so that we could We could have a bank of questions that can be used And reused over the years And refined and modified And we'd have all the analytics about the use of those questions Stored in the question bank And from that question bank we can then construct new exams We can produce randomized ones If we don't need to have an invigilated environment And also Monash College Obviously is a part of Monash And they do an awful lot of exams That are essentially built digitally But then delivered on paper Because the network conditions in various countries around the world Aren't good enough to do this So we can extract any given test as a PDF So we can pull the basic exam Or we can pull the answered exam out as a PDF Because we obviously have to store the exams In perpetuity and to our records management system as well So the academics do it They can either do it straight into the system And yes, following templates in the question bank Or a lot of our academics sort of scroll it out somewhere And give it to admin staff And say you go and put that in the system So we support both Thank you very much for the thoughtful questions And for attending this session And thank you very much for an incredible presentation Please join me in a round of applause


My name's Steve Grubb. I'm a security architect at Red Hat, and I work on Red Hat Enterprise Linux, primarily looking at security for certifications, such as common criteria, FIPS, SCAP, and other things we do. So this talk is going to be about application whitelisting, and we're going to dig into some interesting topics along the way. So this is the outline of what we're going to be talking about. So what we're going to talk about here is what is application whitelisting? Compare it to other solutions, how code executes. We're going to talk about the design of a solution, talk about sources of trust, and how this might fit into an overall system design. So a couple years ago, NIST released a special publication, 800-167, and it's useful because it defines a few terms and talks about different things that application whitelisting should take into account. And so the way that they define whitelist is it's the list of applications, libraries, or files that's authorized to be present or active based on a well-defined baseline. A blacklist is a list of discrete entities previously determined to be associated with malicious activity. So permitted activity corresponds to a whitelist and not permitted activity corresponds to a blacklist. Also, common criteria has an optional requirement. That's the software restriction policies where you may elect. One of several ways to restrict the execution of software. Some people use digital signatures. You can also use hash or you can use file paths. Antivirus is a blacklisting approach. It defines the malware, but the problem with it is there's much more out there that we don't know about. So mandatory access control usually restricts based on behavior. And subject-object rules around information flow and access. Providence of software is really not taken into account. Application whitelisting is a different approach where you tell it this is the things we know about, which is simpler to say because usually you know what's installed on a system. So if you take a look at the Lockheed Martin kill chain or the MITRE attack system, which describes the way that intrusions happen and what they try to do, the place where application whitelisting sits is right between compromise and execution. And that's the area that we're able to target. Coincidentally, should they get past execution, then we'd be targeting everything on the right-hand side also. OK, so one of the things we need to think about is how programs execute. Usually they start with an exec VE, which calls the kernel and it opens a file, loads it, passes it to the runtime linker, it resolves the libraries, and then it jumps to main and takes off. But there's also another trick. And that is you can execute ELF files directly from the runtime linker. And so if you have something that's mounted read-only, or no, with the no execute, this is a trick that people have used in the past where you can just download it as a read-only file, and then you can use the runtime linker to execute it. Because runtime linker just reads the file, it doesn't try to execute it, but later it does. But it doesn't do it through an exec VE. Other ways that you can force execution is by using LD preload. And there's several viruses that take advantage of this. And so once it gets loaded, it passes it on to every child process. And if it gets into the user session, then it can do things against your account. There's also another variable called LD audit, which is a cleaner version of LD preload. But the effect is the same, that you intercept the runtime linker. I mean, with LD audit, you can do a lot more damage, because it allows you to intercept all of the runtime linking and setting up, and you can redefine the functions. So it's a formidable tool to know about. Another way that things can execute is that somebody can go change the ELF interpreter. Embedded inside these files is the preferred interpreter. And usually it points to LDlinux.so. But it doesn't have to. It's something you can define at compile time. So what you can do is you can have a legal program, but the ELF interpreter's changed.  So it looks like you're executing one thing, but you're really executing something else. Another way that programs execute is through language interpreters, such as Python, Perl, Oc, other things like that. So that's the traditional ways. But there's also much more malicious ways of executing code. And that's the mobile code. And when you execute through mobile code, we're talking about piping things in to standard and to be executed. You can also pass programs as command line arguments. I think you have up to 4,096 characters on the command line, which that could be a small program that could bootstrap itself into other things. Then there's remote fetching, like what Python does. You can override and redefine the module import so that it pulls Python code across the network. And another thing that you can do with Python is that it can call arbitrary system calls. And so what you can do is you can open with MFD Create, download something into that, and then execute. And it never touches a disk. It's entirely in memory. Another thing is you can just paste the program straight into the shell. And here's an example of some of this mobile code. This is a function that's written in bash. And this is functionally equivalent to the W get command, but it's entirely in bash because somewhere along the line, somebody decided it was a good idea to have bash to do TCP IP. It can also do just IP. If you are an attacker and you get to a shell and you're in a container and it's mounted read only, they're still going to get you. And the reason why is because of this. If they have access to bash, they can pull down anything. As a matter of fact, I can show you something here. We can, this is a white hat talk, but we can do black hat for a little while. So here we got a bash shell on the one you see here. Just close. So here's the function that you can just paste right into the shell.  You get a shell, a reverse shell. And now you got the function. OK, so there's someone there. Let's just source it just to make sure. Now, I've already got something set up here. So entirely using bash, right there, it can pull down a Python script. And if you go and just pipe that right in the Python, and let's pull up another. And you can see right there on port 8080, we have a server. So you just never touch a disk. This came off of the internet. There's other things that you can do. Like, for example, here's another one with Wget. OK, so this just pulled another program off of the internet. And this is the bad one. What this one does, this is a program called SnakeEater. And you can find this out on GitHub. What this does is it creates an URL to a shared object. Next thing it does is it opens it right here. And it does that by making an arbitrary system call to create an MFD, or MFD create. So then what it does is it reads it. And then down here, it sets a path to this FD. And then it executes it. And so here's a quick demonstration, again, of a program in Python that pulls down a shared object and executes it. So you can run arbitrary ELF code using something like SnakeEater, and it never touches disk. OK, so let's go back to White Hat. OK, so this is just some of the objectives that an attacker may try to do. But we can narrow that down a little bit for this talk and say that without privilege is what you can probably do is download malicious escalation tools. You can change search paths for an account so that it's trying to resolve things out of an attacker-controlled directory. And you can ran somewhere the account. With privileges, you can modify and replace applications or libraries. You can install new applications, backdoors, rootkits, ransomware, crypto miners, everything. Or you can inject malware into a running process using ptrace. OK, so with all this, we're going to start thinking about a solution to try and catch these things. There is an API that the kernel has called File Access Notifications. And it's been available since the 2.6.37 kernel. And it allows recursive monitoring with an amount point. It allows the user space application to say yes or no on an access or execution. And the kernel hands an open descriptor of the file to the monitor program so they can read it. And this was originally designed for antivirus. It has a couple drawbacks that you don't get any notifications on deletes or renames or file moves. This is just to show you what you get when you do the FA notify and knit. And then you set a mark, is eventually you get a callback. And the data that comes to you is in this structure right here. So the two interesting things that we're interested in is that you get an FD to the file, and it's open, so that you can inspect the file. But you also get the PID of the program that's trying to open the file. So from the FD, we can gather some information about the file. We can take a look at what the file's full path is by using read link against proc self FD. We can also take a look at what the type of the file is by passing the descriptor to lib magic. So from this API, you can figure out, this is a Perl program, this is Python, this is Ruby, this is PHP, this is ELF, or just a text file. We can also figure out what device it's on through the UW library. We can also figure out a trust status by looking it up. Once we've got the path, we can look it up in a database and see if this is something that we know about. We can also calculate our SHA-256 hash of it using lib gcrypt. We can also get some other information about the subject. The FD, obviously, is information about the object. But we can get information about the subject by looking at the PID. We can go into the proc file system, and we can pull out what the command name is. We can pull out the executable. We can figure out what type it is by passing that into lib magic. We can also figure out the UID, the login UID, and the session ID, and the proc file system also. So with these primitives and attributes, we can start to fashion a policy along these lines, where we have a decision, some statements about the subject, and some statements about the object. And so for the decision, we can tell it if these things match to allow the access. Or we can also tell it to do that with auditing so that we get an audit event saying that this is allowed. We can also deny the access. And we can also create audit events based on the non-isles if we want them. The subject attributes is just what we talked about a second ago in the pictures. You can tell it all, a UID, UID, session ID, PID, COM, and also some patterns. On the object side, we can tell it paths, directories, devices, file types, and hashes. We can have multiple statements and they're ended together. This is a little bit of information about what is in these different things. For the UIDs and session IDs, these are numbers. Process IDs, COM, is a 15 character string. Executables are also strings. The exec dir also has some key words that we can tell it, executable dirs, which would be stuff like sbin, bin, lib, and lib64, and libexec. System dirs would also include Etsy and maybe one or two other things. Now, we also can do pattern detection because the way the programs start up is different depending on what's happening. I did have a pattern in here for LD preload, but then one day, I was looking at it on a system that held LD library for an NVIDIA graphics card, and so that causes the runtime linker to do completely different things. And I decided it was unreliable at the moment. There's another way that we can add this back in, but I pulled that one out for the moment. The object statements, we can just tell it all. We can have a path. We can tell it that it has to be trusted or untrusted, meaning that it lives in the trust database. We can also tell it that a device like dev at CD-ROM, and one of the main uses, though, is for the file type, which just because of libmagic having things based around MIME types, we list things in that format. So this is a sample policy, the point it looks like. And this is a first match wins kind of evaluation. So at the very top, we tell it that we don't want any execution straight from the runtime linker. So we tell it deny that with audit and trigger on the pattern of starting the program from the runtime linker. And that's for all objects. We also don't want to let untrusted executables run. And so we tell it we want to deny that with audit and that the execution dirs have to be the exact dirs that I mentioned before. And it has to be trusted. If it's untrusted, then we're going to deny. And that's against all objects. We also have a pattern here to allow all ELF applications. And this is a pattern where you have to tell it to allow the types that you want. That's the white list. And then to deny everything else. And so we do the same thing with ELF libs. And that is we tell it what we want to allow to execute. And then we tell it deny everything else. Same thing with Python. We can restrict it to the exact system directories so that it has to come from the system directory. But we also want it to be trusted. The design goals of this policy was to have no bypass of the security by starting a program from the runtime linker and only approved executables that's in the trust database can run. ELF and Python files have to come from the system directories. And this prevents LD library path and Python path redirection. Also, the other languages are disallowed by default. So if you have Perl on the system, you would want to go in and adjust the rules. Or if you're using Ruby or PHP or anything like that. The design of this application looks like this. We get the events in a reader thread. And the reason we do this is because the system will deadlock itself if this application tries to open anything. So what the reader thread does is it receives the event, looks at the PID, sees if it's the PID of the monitor program, FA policy D. And if it is, we go ahead and approve the access. Because why wouldn't we want to approve the accesses that we need? So if it's not that PID, it goes into an event queue and a decision thread gets it. And this way, the reader thread can continue getting events and putting them into the event queue. So the decision thread then needs to figure out, what are we looking at? What's the subject? What's the object? And the thing is that it takes several system calls to figure out who's what. And so what you really want to do is to cache this information. And so the first thing it does is it tries to figure out, what are we looking at? And is this already something we know about? And is it in the cache? Because if it is in the cache, then we don't need to open up all these proc things. We can just go ahead and shortcut to evaluating the rules based on the cache. And so the cache is designed as a least recently used cache so that it's self-cleaning. Things that are recently accessed stay at the top of the list and things that haven't been accessed for a while eventually get popped off of the cache. The decision thread also has a trust database. In this particular case, since I'm designing this on a Red Hat system, the trust database comes from RPM. So basically what the policy is saying that everything that we know about is packaged. The packaging information is trusted, and we're going to use that to make decisions. Oh, there's one other feature in this, and that's a watchdog timer. Because this is approving or denying access to things, I think it's likely to be a target for attack. And so I've tried to design this in a way so that if somebody does try to get execution control, that there's a watchdog timeout that both of the threads have to acknowledge periodically or the watchdog timeout's gonna kill the application. The program, since it might be an attack target, doesn't run as root, it retains capabilities. It also loads a sec-comp policy that prevents exec VE so that if somebody were lucky enough to exploit this program, that the one thing they probably want, which is a exec VE, is denied by the policy. And then they're also gonna have to deal with that watchdog timer. So I mentioned that the sources of trust, we can use a package database such as RPM. And out of that database, there's the path, there's permissions, there's ownership, there's a SHA-256 hash, and all of the entries are signed. So every package can be trusted in the database. There's another source of information that's new and on the horizon. And this is called SWID, which is an acronym for software identification. This is covered by an ISO standard, and this is also putting this, they've also got an information reference, 8060, which details their take on the ISO standard and they put their electives into it. SWID is also being driven to all of the common criteria, protection profiles. One by one, they're asking for manufacturers to include SWID information for all the software they're shipping. So today it's kind of sparse and you might not find many SWID tags, but over time, it's gonna be everywhere. Now to talk a little bit more about SWID tags, because this is really an up and coming standard, there's four kinds of tags. One is called the corpus tag. And what the corpus tag is, is it's like for a body of software, like a CD-ROM. So a CD-ROM would have a SWID tag in a specified directory, and it would give the information about what's on the CD-ROM. There's a primary tag, which allows you to describe the product. There's a patch tag, but this is really aimed more at the Microsoft world where you install something and then you have patch Tuesday and you keep updating. And so this is designed more for the Microsoft world. And then there's supplemental tags, which allow you to add information like linking to webpages for more information. The SWID tags are an XML file. They convey information about the publisher of licensing. And then there's an optional payload section, which details files, sizes, and hashes. And this can also be extended with information about permission and ownership. And then the whole thing is digitally signed using the X80s specification. On Fedora and in REL, you can find SWID tags in user lib SWID tag. And here's an example of what they look like. There's much more to this because this doesn't have the optional payload section, which is the part that's probably most interesting. Okay, so let's take a quick look. I'll do another live demo here. Okay, so it's coming up right now. It's running, it loaded its rules, it changed the UID, initialized the database, did an integrity check of the database, and it found a miscompare. So right now it's rebuilding its database. Now, one thing to mention is that what it's doing right now is that we use LDBM as the database. And the reason why is because it's way faster than the RPM database. So what we do is we create our own database out of the RPM database. And by the way, we don't have to worry about locking and other applications. Normally it works a lot faster than this, but this is an old laptop. And I was having trouble getting the PDF file to open. So there's something busy on this machine. It was taking way too long. Yeah, I'm wondering if RBM had something locked. Okay, now we got this thing up and running. So this is the debug mode, which is not the way you would normally run it, but this is a lot more informational so you can see what it's doing. What it's telling you is the rule that it triggered on, what the decision was, and then some information to help debug with, which is what executable and what file. It's trying to be accessed. Okay, so let's follow this demo script here I've got. Let's go over here to the temp. And you can see when I ran CD that it created a whole lot of access. So what we wanna do is copy the system LS into the temp directory. And let's also make a sim link. Let's try to run this program. Over here you can see the denials. Okay, so let's try it from a sim link. And you see it was allowed. And the reason why is because it resolved it to the system LS program way back here. So the sim link, you didn't get the full path, you get what the real object was. Okay, so let's try to run this program from the runtime linker. And so we got a denial. And just in case, well, we've already got a copy of it there. Let's try to run that one because just to show you that the temp directory is loaded with a permission so it doesn't allow execution. So you see here it denies it, even from the home directory. And you can see that the denial is over. Well, way passed down. Okay, so back to the demo. Let's take a look at a program that has the interpreter changed. And just to show you that the interpreter is changed, we'll take a look at it with read-elf. That's right. Oh, there it is, yeah. I think it was denied. Yeah, there was a denial. Well, let's try to run it. Actually, I'll show you this file in just a second. Let's run it. So there was a denial. And the one last. Okay, let's go try and run that. You can see that system Python files are allowed to run. And one of the things that I have it to do in debug mode is to output some statistics so that you can see how it's performing. In this case, there was about 1200 accesses allowed and 11 of them denied. The cache size is 1024, 43 slots were in use. We had a lot of hits. Same thing on the object side, that that's shut down. You can see right here in the elf that it's not requesting the normal runtime linker but a fake interpreter. Okay, so I did show you the statistics report, but it also leaves a breadcrumb trail. Over in VAR log, there's a FA policy D file that's dropped there when the program shuts down so that just in case you need it for forensics information, there's a breadcrumb trail of everything the system was doing right before it shut down. But that's also configurable because there may be cases when you don't want that information sitting on the disk drive. Okay, so looking back at our original code coverage, we can deny everything that's got a line through it. The things that are in red are within reach. Those I can cover with just a little bit more work. And then also this fetched remotely thing. This is being handled an entirely different way. I've been talking to people in the Python community and explaining the dangers of Python and mobile code and things like that. So what they've done is there's a PEP 578 that's been approved and is being worked on for Python 3.8 and what it will do is it will add audit hooks to the Python interpreter. And then also you can have a monitor that looks at these hooks and decides yes or no. And this will be inside the same binary. And so we will be able to have a policy inside the Python interpreter that says no code from standard in. You're not allowed to override the module resolution. You can't pull things from the net. You can't have programs that come from the command line. So this should be on 3.8 and it will solve the problem there but you've got the same problem in other languages but I don't have leverage or ways of influence communities for the other interpreters. Some things I really wish would make life a little easier is that if we get notification on exit. I suppose there's a way to do it but it wouldn't be serialized with the main event queue and that causes some problems. The other thing is if, because we have to figure out is this thing in the cache, you know, what are we looking at? The very first thing we need to do is get some stat information. So if that was passed with the event, then that would save one system call and we can make a decision a lot faster. And then the other thing is in the proc file system, everything is in different files. It would be awesome if some of this information was collated like for example, proc self status does not have the login UID. You have to go open proc self login UID to find that. So it would be good if some of this was consolidated so there's just like one open system call and then a read and then we can make a decision. Things that are in the near term is reinstating the LED audit, LED preload coverage, detecting statically linked applications, interpreters pulling code from standard in, even though this will be solved by 578, there's still old systems out there that need protecting and detecting code from standard in, from the command line, standalone shell usage and adding more threads to it so it can scale out. So how this might fit, I'll briefly go through this. The audit system has a bunch of event feeds. From the kernel, we can get promiscuous socket, core dumps, sim links, net filter, TTY, syscall and file watches. There's also trusted programs that send events such as PAM login, shadow utils, password, SE manage, Cups, Clevis, Libre Swan. And there's also policy engines, LSMs and Setcomp can also set cause events. And then there's integrity apps, aid, FA policy, the USB guard. So we can take these events and start to fashion a system together where events wind up coming to the audit daemon. Right here it just shows the application white listing daemon. And then the audit daemon can have an IDS plug-in to look at things. The audit system is really easy to use. You don't have to worry about parsing events. There's an audit parsing library which takes care of all the idiosacracies of the audit system and just makes some calls and it gives you the data in nice little chunks. And then what we would look at is putting this into an IDS system with an ensemble model, something like this that looks for bad events, it does pattern analysis, does burst analysis, looks at historical norms and does misuse detection. And then that all gets summed up and then there's a reaction to it. That concludes the presentation. Questions? So is there a revocation story for the trusted database? Yes, anytime that the database is updated we get signaled and we update the database. So if, for example, you do RPM-E to erase a package, then that would trigger an update to the database and then we would know that that's no longer there. So that's kind of how revocation would work. IDS model that you talked at the end about is more of a blacklisting model. I'm afraid I didn't understand the question. So the last thing that you talked about is not whitelisting. You said you are looking for patterns, blacklisting basically patterns. Yes, yes, it's doing more than whitelisting. Yes, you're absolutely right because whitelisting is just about the provenance and whether or not it's in the trusted database. But in a way you can stretch that to say that the runtime linker is what we trust. And if they're calling a different runtime linker then it's outside what's trusted. You mentioned SWITTAG, in case you're not a big fan of XML, just want to let you know there's an IETF draft for doing SWITTAG in concise binary called COSWID just in case you're not a XML fan. Also I just want to do a full reference. We at our three o'clock presentation are defining another type of SWITTAG for firmware. So just kind of underbashed. So how does this work along with other access control systems like IMA or SCLinux? How does it coexist with other ecosystems? Well, this, because we're in the file access path, I think that we're before the DAC permissions. And SCLinux is after the DAC permissions. So I suppose in a way we get first vote or this program gets first vote or something like that. But it co-exists, find their complementary. So as UEFI secure boot, this doesn't try to solve that problem. And so it's complementary on top of that because really what this is aimed at is solving the problem of somebody pops in a shell from a demon or gets access to somebody's system to their credentials. And that's way after boot. But that's really what this is designed for. So, okay, last question. So, two questions. Since it's a last question. So, okay, so do you have any plans to extend this to kernel module white listing? Would we extend this to be a kernel module? No, to white list kernel modules. Oh, to white list kernel modules. Yeah, I believe we can do that. You know, I would just have to double check to see if the FAA notifying system gets notified. And if it does, then yes, we can. The second question is, okay. Believe the trusted database or the trust model works efficiently when the software or the applications are installed from the package manager. If you have legitimate scenarios where you are not using a package manager or you are not installing package, rather you are directly dropping the applications, distributing by a different distribution mechanism. And you have a legitimate need to allow only certain binaries which are distributed by a different distribution system. How do you foresee this fading? Okay, I'm not entirely sure. I understood all the question, but what I believe I heard was if somebody has applications that's not packaged and you install it and you want that to be trusted, how do you handle that situation? And if that's the case, there's an admin defined list that you can also modify which would then grant trust to that. So there is a way to add your own whitelist to this so that you don't have to depend entirely on the package. So we're out of time for the questions but I'm sure Steve will be around later. Thank you, Steve. Yes, thanks.


On the last four videos that I put up on YouTube I was explaining, describing, showing our dogs protection dogs compared to the rest of the world and the prices and market value for the dogs and the differences between our dogs and everybody else's dogs for the money okay so I've been harping on at least for the last four videos about dogs being sold for anywhere from let's say 50,000 because the industry goes generally 50,000 65,000 85,000 100,000 150,000 up to 150 okay and I was showing a Doberman that was sold for $150,000 that all he does is just bounce on a leash and go crazy right with no control, no discipline, chaos and is considered a fully trained elite protection dog okay and that's what the world is believing that that's what a protection dog is and the dog was sold for $150,000 okay so and even if it sold for 50,000 65,000 85,000 all this bouncing on a leash is unacceptable again it's beginner stuff baby stuff not a controlled protection dog let's put it that way so I've been saying in the videos that anybody can handle my dogs that I build okay that I will never sell anybody a dog that bounces on a leash and pulls and goes crazy with his teeth and all that right that to me is not a trained dog okay so I will never sell that kind of dog not for 10,000 1550 70 100 150 doesn't matter it will never leave my facility being that way because that's an untrained dog to me that's a day one beginner dog so I was showing how controlled and disciplined our dogs are that anybody can handle them men women children handicap disabled okay because they do not go crazy on a leash when you go watch up and they go crazy right that owner of that Doberman has young teenage girls okay and they better pray that those girls if they're ever walking with that Doberman somewhere never needs that dog because if they do you know those girls gonna be scared to turn that dog on because that's gonna be a lot of dog to try to hold back all right so you know that in those particular cases they're gonna be thinking oh my god do I have to turn them on and even if two of the girls were holding on at the same time they're going for a ride they're probably gonna crash into each other fall on the floor go for a ride on the concrete that dog's gonna pull them all over the place okay if they have to turn them on and try to use him to protect them that just doesn't work functionally that's just not acceptable okay so again saying that anybody and I use children as an example because I showed women doing it examples of if you are in a wheelchair right it's the same thing and here show you a client of mine doing it with their eight-year-old grandson right with these two Malawas this is the way it's supposed to be okay so the the child is doing everything he's giving the commands to these dogs telling them to watch when he does they start barking and alert stay with him no chaos the boy doesn't have to hold on to the dog and try to control it because that would never happen right that would be a disaster so the dog would pretty much be useless because there's no way that child be able to hold a leash or anything with that dog if they turn them on okay so here in these clips here you'll see the boy just tell watch the dog starts barking at the person from him to go revere bark and hold which is to go to the person hold them where they are not to come any closer bark at them and if they move the dog will bite I'll send the dog to attack instead of a bark and hold the threat is a little too much so now we're going to put an end to it and the boy will send the dog to actually do an attack okay on command a little shuffling backwards sideways hits the floor does shuffling over to the side scooting around the dog stays with him you know and trying to explain to people because there's always been this thing I don't need the dog to walk backwards I don't need him to circle I don't need him to stay with right yeah you do you just don't understand it but yes you do so people always start as fancy hocus pocus movie magic nonsense doesn't apply to real life okay and nothing could be further from the truth and again I sit all the time I don't do anything that's not functional I'm not going to focus on anything that doesn't work in real life that includes obedience tracking whatever it's going to be I am not doing it unless it applies to real life scenarios so if the boy chose to leave that area right to try to escape this person he could have he could just walk backwards walk away and get himself out further and further and further the dog would just stay with them following him he's not going to dogs not going to go chaotic bouncing all over the place and just take off and go bite the person okay so notice that as well that the dogs are so controlled that you know they all protection dogs right they would love to leave that body and go bite they're not going to do it though they're holding back that impulse because of discipline understanding the game of control hold themselves in control mentally right so that we don't have chaos we don't have an accident happen the dog doesn't go and bite the wrong person right this is why all this control stuff is so important let's say that somebody was walking up to the boy or any child and was actually trying to help the child right are you lost or are you okay do you need help and they were coming up and the child accidentally like just out of here you know panic turns the dog on right because he doesn't know they don't know that person and the person's intentions are good right we don't need the dog blasting off and going to bite somebody for no reason right just because the child doesn't know so that's why we do not like that exploding on a leash and going crazy into chaos right at all it's not good stuff at all I know the world loves it that's what they know is protection so yes this is why we have all these skills that the public again has shunned and thought very little of not realizing that these are the highest level skilled dogs in the world right and that again anybody including women children eight year old could have been a six year old five year old handicapped person disabled person whatever it is can control and handle our dogs okay most of the other dogs out there that the companies are selling nobody can handle but men right that can that are strong enough to handle those kind of dogs when they turn them on so just one to show you an example of a young boy in my system being able to control these dogs with no issue whatsoever beautifully done and that is the way a protection dog should be


Hello everybody and welcome back to my V-LOG channel. Now, this is a little quick. You want to show you a couple of things when it comes to these boards. Now, this is, I've been calling them genuine, the Highland stuff, but as you can see now, one of the links I've put in, it's not actually genuine, but they are one of the retailers, manufacturers that actually produce these boards and they work. This is one of those boards. Now, normally when you connect up your meter, you wouldn't connect it to the output. And the reason why you wouldn't connect it directly to the output is because it has to be powered on by something. It has the separate power on and then it's got a sense wire. And the sense wire is to detect what voltage is actually needs to be displayed on the screen. And so you need a separate power supply for the power to actually power this up itself. So you can actually give this some power itself and you've got to give it the sense wire so it knows what to display on the screen. Right. Well, on these boards here, generally you could, if you wanted to, ideas on my build and it worked very well, thank you, you connect it up to the 24 volts coming up. Now, you can change these regulators. Help me put a 12-volt regulator in there if you want to and you'll have 12 volts coming down here. But these can go up to 28 volts. So I'm going to use a 24-volt regulator that came with it. So to keep in mind, if you were to use a 12-volt regulator and consider it like this, if you're using 24 volts and you're just pulling half an amp, you don't, with a linear regulator, whatever voltage is not being used, let's say you're pulling 24 volts in, you're using a 12-volt regulator and you're pulling half an amp. So that means it's got 12 volts difference between the input and the output. And that means it's got to dissipate the heat because whatever current you're pulling through, you've got to times that. So the half amp by the 12-volt difference on these would be 6 watts. So it's got to dissipate 6 watts of heat. All right. That's something to keep in mind. Right. So what you'd normally get when you power up, which I'm about to now, is what you should be able to see on your oscilloscope. This is set for 10 volts per division and it's in DC mode. So click that down and you'll see it shot straight up, 10, 20. And that would be 24 volts. And you can see there, 24 volts bang on. Beautiful. Nice, clean line. And that's what you expect to see. And that means I'll power that quite nicely. I've actually got this twiddled to 5 volts. Yeah, that's about there. That'll do. And that's what you should expect to see. Now, when it comes to this bore that I've just bought, and yes, it does work. Not a problem with it. But, well, I said not a problem because here's the butt. We do have a small issue. And that is when you try to connect to the fan here, there's a lot of noise. And I'm going to show you that. So just bear with me. I'll do a jump cut on this so we can just save time. This is just like a little thing in the middle of the jump cut. Just so you know, yeah, so if you're connecting this, you'd have your red and black. And that will connect to an external or a separate power supply. Because if this goes below 5 volts, it just switches off. So if you were trying to do a 3.3 volt or whatever voltage below 5 volts, you're not going to get any power to this. You won't be able to see whether it's 5 volts or below. That's why you're going to have this separate supply. So your yellow wire then will go on to the positive output. And this black wire, this is the current sense. So what you'd have is your load, whatever device you're having connected, would come out of the positive. And on this case, it's this resistor. It'll come back through the red into this, the positive into the current detect, and then come back out through this ground. And the ground also gives the ground for this to work. If you don't put the ground in and you're just trying to sense it, it's not going to work. You won't see your voltage on there. So you've got to make sure you have this grounding and the sense on there and you can actually see what the voltage is across there. Not all of these are the same. I'm just going to let you know now this is the one with the two little twiddlies in there for the voltage and the current. There's two ICs in there. And it also has the four digits up top, four digits down the bottom. Not all of them like that. You might have to connect it up slightly different for different ones. But for this particular one, that's how you connect that up, right? Back to the jump cap. One of them is I've just connected this up as it was connected on the other one. And I've left this because we're not trying to sense the current here. We don't need to. The issue is going to be seen straight away as soon as I put down the power. So I'll put down the power. And as you can see over there, our nice DC is not nice DC anymore. This is awful. And if we look here, we can see that it says 5.6 amps and it's at 12.3 volts. Now, if I just turn this up a little tiny bit, we can clear that. That says it's 8.2 volts and zero amps. But that's an old go. And even though it seems to clear the issue there, we've still got this horrible DC here. So what do we do about that? That's not acceptable because we're going to be able to run this down to the lower limit. So otherwise, you can only do things above 6 or 7 volts. So that's not good. So I'm going to set just playing around with it. I figured that we can clear up most of it and get rid of that issue just by sticking the little capacitor across. I don't really like the idea of this because now, in some respects, we've got to look at the setup of that voltage regulator. Just because if you put a capacitor across, that means there's charge stored in there. If we get a short circuit here, it means that charge could go back into the voltage regulator and kill the voltage regulator.  But if I just put this on here and you're going to have to do it polarity-wise, of course. So if I put this on here like this, you can see that even though, just using this, it's only 220 microfarads. Oh, there we go, that clamped there. A bit of a pickle. It seemed easier last night, then I wasn't trying to do it on the camera last night. Just turn that around so I can hook these both in together. So if I clamp that like that, OK, so we've got that on there, and I put that back down again, you can see that this, even though it's not a lovely DC line, it's a lot cleaner than what it was. And here at 8 volts, I just bend them up a bit so you can see that. Let's start turning this down. That's gone wrong way. Let's start turning it down. You can see that we can go all the way down. And we don't get that issue. But straight out the box, that is something that if you did buy one of these, you're going to have to resolve. I might end up swapping this out, but see if I can put a tantalum in there, just because it's not going to hold that amount of juice in there. Or it's going to be a case of sticking the diode on, like I say, and I'll get my head around that over the next few days. I've ordered another one of these boards, because I'm going to do a contrast and compare, do a contrast and compare with one of these boards. Because they seem very similar. When I look at the other side, I'm not going to do it now because it's all connected. I can see, not the difference here, I want to be able to go over them without any population on the board. And also then I can try to find a genuine highland so I can contrast and compare this board with that as well and see why this hasn't got the filtering on the output. It could be the regulator. I will take the regulator out and I'll try a different regulator. 15 minutes, no. Minus the jump cuts, so it'll be less than that. And I'll try a different regulator and I'll let yous know. Again, for connecting it, it's all the same way as I said earlier. If you want to know how to connect to meter up because you're going to want to connect to meter up with one of these boards, you get it working because you're going to want to see what your voltage is and what your current drawer is. And then we can start talking about transformers. And we're going to do another one on this. I'll talk to you about the transformer, how it affects the board and the differences between this transformer and one that I think will work much better with this board. If you got this far, thanks for watching. I hope that was helpful for any of you that may be interested in buying this and you get it in your phone that you can't wire up. There you go. There's the pattern to wire one of these up. And also, if you get that issue, you know you can drop a little capacitor across there and it will sort it out, but it's not the end of the issue because it could cause a problem in itself. All right. Catch you in the next one, guys.


U, or I, was written by Marcellus D. Reigns on the SCP Wiki, it is under a Creative Commons 3.0 share-alike attribution license, a link will be in the description. You can't remember the last time you ever saw the sunrise. You think about it for a while, but that last memory of sunlight is so old and faded by now that you can't even describe it accurately if you had to. It's been dark for so long, days and nights have just led together. And now? Well, now you can hardly tell when it's daytime or nighttime. So much has changed since the blackout. You remember waking up here, but not where you were the night prior, had the Wickerman got to you? You notice a flickering light in the distance. It's not that far from where you are, maybe a 10 or a 15 minute walk through the void if you speedwalk. You glance down at the match you have in your hand, such a tiny thing, that matchstick, but it was the best source of light you could find before you left home. The matchstick burns down to the nub, scorching your fingertips. You bite back a yelp, wouldn't want the grooves to find you. You fiddle with the matchbook in the dark for a minute before thumbing your finger over its contents. You count the matches slowly, one, two, five, eight, eight matches left. If they burn as slow as they have been over the last few days, you figure that you could be able to figure out whatever that light source in the distance is before your last match burns out. You shudder at the thought of traversing the void without a light. The screams of your old traveling crew is still echoing in the back of your mind. You shovel those thoughts down into the recesses of your mind and you start walking. You've gotten quite good at that walking. You chuckle. The thought of having to get good at something that was already second nature to you again was... is absurd. The match didn't do much to expand your field division, but you felt comfort in knowing that your chances of falling into a wickerman's trap or stepping on a shard of rusted metal were lower if you could see, if only a little bit. Not to mention the added benefit of keeping the grooves at bay. You scold yourself for thinking about the monsters again. The light in the distance was closer now. Only a few dozen paces away by your estimation. Maybe it's a generator. No. It's too faint to belong to one of those massive hunks of metal in industry, so it must be something else. But what? There's a growl coming from behind you. Right to your left. Are you right? It seems to be coming from all directions at the same time and yet coming from nowhere as well.  There's nothing around you that you can see. Nothing but dirt, half-eaten corpses, and claw marks on old cars and stones. Claw marks? You move closer to one of the ancient cars on the side of the road and you touch it. It's cold and dead. Nothing like the void that surrounds you. When you fill your way around the car's exterior, you come across those claw marks you saw a moment ago. You thumb over them and the metal around the marks feels warm. You lean in closer and bring the match right up to the surface. There's blood in the grooves the claw marks made. There's a roar that tears through the darkness, drawing your attention behind you as your match burns out. You feel your chest tighten as your eyes strained appear through the inky darkness. You cannot see anything but you know that something has seen you. The ground beneath you starts to shake.  You try to light another match but can't. The stench overpowers you and you double over and vomit. With trembling hands, you pry open the rest of the car door and climb inside. You apologize to the occupant, a lone skeleton with half of its body missing.  The thing yanks on your leg and you feel as if it'll tear the whole limb off if you don't do something. You try to hang on to the other car door but the thing is stronger than your grip and you're forced to let go. You clutch the center console now desperately kicking at the groove that has a hold of you.  More appendages slither around your torso from the darkness and work in tandem to release your grip and drag you away. The air rushing past your ears hurts. You can only barely hear the groove opening its mouth to take a bite of you. Your thoughts race at a thousand miles an hour as you smell the scent of rotten meat blasting up at you from the mouth of this predator. Adrenaline flows through your veins and time slows to a near halt. The heave you push the grooves tentacles wrapped around your waist far enough away from you to slip your hand through you take a match out but as you're about to light it the groove thrust you sharply to the side knocking the match book out of your hand. It roars in your ears deafening sound coupled with a stench that burns the hairs in your nose.   You realize that you still have a single match in your hand.   You could feel yourself slipping away into the darkness still you try again and again and again to light the match. You don't want to die here. The next thing you realize is that you're falling again.  You gasp gulping in all the oxygen you can as fast as you're able and you can see again.  You gasp as you hold the match close to your chest to avoid getting it wet. To your left you notice the matchbox to your right you see the car. An idea forms in between your ears as you scramble to pick up the matchbox. The creature attempts to grab you again but you hold the match between yourself and it as you run back to the car and pry open the gas tank. You hold the lit match in between your teeth as you light another.  The groove was one you'd come to identify as the profane. Most of the creature's body was obscured by the darkness. Limited range of your matchstick only revealed a small portion of your body. You gasp as you notice a beak massive and lined with teeth larger than you. Opening and closing faster than your eyes could track. In the mouth of the creature was a tongue long slender that thrashed about wildly as it shrieked. A conglomeration of tentacles spread out from all directions. You cannot tell where in the darkness they're extending from but more and more appear out in the void and attempt to grab you. You tear yourself from gawking at the massive creature and throw the lit match into the gas tank. You barely have time to scuttle away as the gasoline in the tank ignites and explodes into a raging inferno. The creature attacking you shrieks and withdraws into the darkness. You can feel the heat from the explosion burning the hairs in your arms. But you're safe. For now, at least. Thank you very much for watching. If you enjoyed the video, please hit the subscribe button and then hit the notification bell next to that so you're notified when I upload new videos. And then head on over to patreon.com forward slash decimarian and pledge at any level like everybody here on the screen already has, including mc. Keshe mill who has pledged at $40 and synderiki who has pledged at $100. It is nice to know that I'm not alone out here and I will see you all again on Tuesday.