Hello everyone, Ross here at Teaching Toolkit. Thank you for joining me live. Over 250 teachers have signed up to attend this event tonight and I've got some special guests that I'm going to introduce to you shortly. Just to make sure that you're in the right place and you know how the technology works. If you're watching live on any of the social platforms, particularly on YouTube, as long as you're logged in, you can leave a comment for our guests. I guess before we kick off procedures for this evening, let me know where you're watching from. This is an opportunity for me to give your town, village, city a shout out to social media. I'll give you a couple of seconds to get connected while everyone's logging in. What we're going to do this evening is talk about low-state quizzing and retrieval practice. We're going to cover a broad range of academic research, primary and secondary. Then we're going to steer slowly into primary in particular. Over the last few years, I've been really digging into the details of retrieval practice in primary and early years classrooms. There's not much out there, but there is if you search really hard. I'm going to share a couple of pieces of research with you in our session tonight. I can see people are starting to say hello from lots of different places. Let me just put one or two faces on the screen here at the bottom here. Claridge in St. Albans. Thank you for joining me. Meg, I'm going to say hello to Meg. I know Meg who's watching in Batley in West Yorkshire. Thank you for joining us, Meg. We've got a big up here from the Bromley Massive in South London. Karen over in North London. Let's see if we can find some international guests. Let's go pop over to Leslie who's watching in France. Waiting for one or two more comments to come in. While we do that, let me just push my mug shot out the way. Here are our special guests that are joining me tonight. In a moment, I'm going to get them to introduce themselves to you. Let me just show you where you've all decided to log in from. I've got a small audience as you know. We've got here, I think at the last count, a good 18 different countries on the map here. People watching recorded, I know one or two guests have got in touch with me throughout the day to say they can't join us live because of the time zone. That's fair enough. If we just zoom in on where we are broadcasting from, a closer map here across the UK. You can see, obviously Birmingham is a very popular hotspot for retrieval practice. I'm just going to give a shout out to Rebecca watching us from Arkansas. Thank you, Rebecca, for joining us. Any more shout outs, let know. I'll ask my admin team behind the scenes to give one or two nudges on the screen here so you can see where we're all watching from. Thank you for joining me. This broadcast will be live. In the slides, you can see the QR codes to scan as we go through. Make sure you've got your mobile phone to hand. Any questions for our guests, pop them into the chat box. They'll remain private and then we will display the relevant questions to our guests as we go through. So it's going to be, I guess, a bit of a virtual busy party in some respects. I got to squeeze five wonderful colleagues on the screen here with me in a moment. So let me just bring this slide over here and then I'm going to move this out the way. I'm going to let everyone work themselves out. We'll do surname alphabetical order and I'll actually each to introduce yourselves to our guests.  So let me move my slides out the way. A reminder to everyone again just to say hello and let us know where you're watching from, get connected with the technology. So let's bring in some of our guests. So we've got Caroline, Ben, Sarah, Connell and Sophie. Good evening, everybody. So have we worked out alphabetical? Is it you, Ben, first? Yeah, it will be. Always top of the register. So yeah, I'm Ben. I'm working at Primary School on Whirl, currently the Year 5 teacher, but I'm also a maths subject lead and part of the senior leadership team there. I enjoy doing retrieval because I think it really helps the children see how everything clinks together and how the learning is obviously in progression. Okay, fantastic. So just for international guests, people, the Whirl is in the northwest of England, just in case you want to get a geographical footprint. Okay, thank you, Ben, look forward to firing some questions your way. Who's next in the register? I think it might be me. Is it me? Nobody else is claiming second spot. I'll go for it. So hi, everybody. My name is Connell. My background is a 25 year career in primary education. I was a head teacher for 11 years. And most recently over the last 18 months or so I've become the founder of Primary Quiz and developed a resource to try to help teachers bring retrieval strategies and low stakes quizzing into their classroom, whilst hopefully saving themselves a little bit of time as well in terms of planning and preparation. Fantastic. And I've been working with Connell for on and off for the last six months, and I can safely tell everyone watching the primary quiz is a fantastic piece of software. So we'll have a little look at primary quiz later. Thank you, Connell. Who's next to register? Are we self regulators? Caroline, she's got a hand up. So it is me. I think I'm the K afterwards on I Caroline keep. I'm a teacher secondary teacher. And I specialise in digitalisation and education and also a researcher, the University of Central Lancaster, where I'm specialist in digitalisation and how you use digital. And I've been doing research in how you integrate data into schools and how we have that relationship. So always the teacher, some other researcher, so a bit of both of the moment it feels like. Is that your dog trying to get in the door there behind the door? No, I think it is. I was just having that normal. I think he's just happy just because I just came in from teaching. So it was like, oh, let's see you and let's make a noise. Thanks for joining us, Caroline. Nice to see you again. Who's next? Who's next on the register? Sarah? It's me, Al, after K, Jess. So hi, everyone. I'm Sarah Larson. I'm a geography teacher and at the moment, a bit of RE as well, helping out where I can. I've been teaching secondary for, I think it's 26 years now, most of which has been down in Surrey at Rygate School, but I've done a couple of years in East London and down in Kent as well. I'm also currently our skit link tutor. I travel around our partnership schools, assessing the trainees and quality assuring the provision that the partnership schools are providing for us. I've spoken at a few research ed conferences. I've written a few chapters of a few educational books, Michael Childs and one or two others as well. So incredibly busy, essentially. And just for context, international guests watching skit or SCIT stands for School Centred Initial Teacher Training. And Sarah and I go back a few years. Sarah took part in some verbal feedback research. We conducted at University College London 2019. And today, she's still no longer Mark's book. So that's a fantastic success. If you're interested, get in touch. Now Sophie, Miss Wilson, thank you for joining us. How are you Sophie? Thanks for joining me. Hi, yeah. Good. Thank you. I'm a year two teacher in a primary school in Liverpool. We've just signed up to primary careers and staff and children absolutely love it. Fantastic. And you've been teaching 10 plus years or so, haven't you? Okay, fantastic. So we've got a nice, busy panel for everyone taking part. And we're going to unpick retrieval practice research, talk about low state cuisine in the classroom, show you a few little bits of research and primary quizzes the software. And I guess before we start, I'm just going to bump you all out the virtual room, if you don't mind. And I'm going to move over to a short kind of introductory example of how we might conduct a kind of low state quizzes as as as we would have it. So let me just push everyone out the room, you'll be back shortly back here on the slide deck here. And we've got just some examples of what a quick quiz now I've not formatted the slides here, which is a beginner's error, but you can kind of see here, these questions would be universal, I guess, depending on irrelevant of which country that you're in. And when we pose a question, so whether it's the, what's the second planet in the solar system, we're activating prior knowledge to help strengthen those connections. So when we talk about retrieval practice, we need to be very clear about definitions. And that low state cuisine where we reduce the stress to increase the recall, and retrieval practice always comes without a grade. There's a few misconceptions in parts of the profession. So we're here to kind of clarify some of your questions also. So I'm going to just come back to some benefits of low state cuisine in a moment. But here's the first question I want to pose to our panelists, and I'm going to bring them back in in a moment. Here's the, here's the question, what are the some of the most significant obstacles you face as a teacher when implementing retrieval practice and low state's cuisine? So how have you overcome those challenges? So that's the question I'm going to pose to our guests. I'm going to bring everybody back in and for people watching, feel free to pose your questions on your side. And we'll put one or two questions on the screen for our guests to respond to. Now we've got an hour together here, so I'm going to do my best to keep things moving on swiftly. So I'll start with Sophie this time. Let's do reverse register order. Over to you Sophie, there's your question there. What's the challenges that you faced in your classroom? So one of the biggest challenges for trying to fit in quizzes is always time in our timetable. Like the curriculum to jump back. So trying to get everything in has always been like one of the biggest barriers and trying to incorporate quizzes in. That has been one of the best things about primary quiz is because it's it's all there. We can just get it up at the end of the lesson. It's so quick. Time is not an issue now. We can put it in any lesson and because the questions generate new questions every time you go on it. It just means that it's always something new. It's the children have the time to answer the questions and with the answers being up on the quiz as well. It means they can self-cess, they can peer-assess. It creates a lot more time in the classroom to do those retrieval questions that we might not have had time to do before. So Sophie just a quick question from me then. What were you doing before you were using primary quiz? So if we were doing retrieval before primary quiz we were always having to make something as a starter or as a plenary and then again sometimes at the end of the lesson children are always finishing at different times. This quiz is just so straightforward. They just know what they're doing. It's just as those children finish they can just access it from the whiteboard. They can do it independently. They can do it here. They can do it collaboratively. I don't want to put words into your math but in terms of your planning and work load? Absolutely. It just makes it so much easier because obviously as well retrieval it's not the main objective of your lesson although it's obviously there's a too scaffolding but because you've spent your time creating the main resource you don't always have that time to create a quiz, create answers and implement it in the lesson. Sometimes you do that and it doesn't get used. Yeah I guess the brilliance of primary quiz that easy quick regeneration is fabulous for teacher workload. Sarah I'm going to come over to you now as a secondary specialist are there any differences? Students obviously know a lot more stuff. The subject knowledge is much greater. Does that change the workload pressures? I think the workload pressure I mean I agree that finding time in the lesson to actually make sure that you include retrieval practice can be tricky but also the pressures of planning it in the first place. Certainly at secondary what we try to do is we try to have a range of questions where we've got an easy quick win as our first question for our do it now task and then the remainder or the bulk of the remainder of the questions underpin the learning that's to come in that lesson so the idea being that you kind of look at prior learning and you're looking at building schemas across that lesson but and then looking further back as well we try and get a question in from further back to activate prior learning from their long-term memories but all of that requires planning so it's not a quick five minute job it's looking back at the curriculum and thinking okay what do they need to know so that today's lesson makes sense but also what do I want them to learn over the long term bringing that into the into the do it now task through retrieval practice as well so so planning also can be can be a time constraint but often if you do it collaboratively as a department that can be that can be an easy win where you're sharing the workload slightly. Now I'm going to give you a spanner question what if you're the only member of staff in your department in a secondary school which we know does exist. Absolutely I mean it's not something I've ever you know I've always worked in fairly large departments RE and humanities but yeah for example music and food technology quite often can just be one person so I guess that's a prime example isn't it where it's even more onerous on those on those people's work loads so if there are off the peg resources that they can use in order to facilitate that then that can only be a good thing. Sure now I'm going to bring Connell in for some wisdom here Connell what are your you know the question we've got on the screen here about significant obstacles what do you hear teachers say when it comes to implementing low-state quizzes in their classrooms. Well I think Sophie and Sarah have hit the nail on the head already I think the key issues really which keep coming back are about pressures of time and also from the angle of planning you know planning in I think I talk to teachers all the time about you know do they realize the significance of building in opportunities for children to revisit previous learning at spaced intervals because I think if you can if you can find a way of planning that into your schemes of work you are going to hugely increase your children's chances of remembering the new learning that you're exposing them to but of course as as Sarah and Sophie both pointed out you know that has real time restraints and also you've got to you know finding less in time during the school day is challenging enough but but then to also find a way of planning that at appropriate points of your schemes that you're teaching I think is a real challenge for teachers. And in terms of your technology am I right I mean I know the answer so it's a bit of a leading question but I guess it's possibilities of regenerating quizzes is endless. Yeah yeah infinite I mean primary quiz the resources is designed so that teachers can just keep coming back for more you're essentially you're never going to have the same activity twice so you can use and it's not just quizzes it's investigations it's puzzles it's it will help to you know extend pupils vocabulary because you've got vocabulary wall so there's a whole diverse kind of range of different activities on the side and you know it is designed to to solve two problems which we which we've already kind of alluded to tonight you know one is what is saving time and what is helping helping teachers with their with their work but of course in terms of children and how they benefit and you know all the feedback that we're getting from from schools that are using this resource children are the activities are really engaging for the pupils and they they develop a familiarity with different types of activities and the resource itself and of course in terms of people outcomes we're getting some really good feedback from schools about it making making a real difference you know whether that's developing fluency skills or consolidating knowledge and how many skills are using it at the moment well we've only been going quite a short time but I'm pleased to say we're already up to 70 schools and it's a figure that keeps going up and up and over time. So we'll come back and have a look at primary quiz in a moment thank you Connell for your time now I'm going to bring Caroline here for your research perspective Caroline. So I'm kind of going to I'm going to I'm going to do that thing as a teacher where I'm going to challenge this and as a researcher I'm going to go really a bit through for thought here. I absolutely love quizzing as a teacher and I think the the challenges of time are absolutely valid I get that and obviously the tensions in the curriculum and how much time you've got to do this but some of the latest research that's out there right now especially a brilliant study by Yangertal in 2021 and actually is a lovely combination of real data analysis as a data scientist that I like and some really good research on teaching and it was on 50,000 just 49,982 I think it was students and it was 200 research papers on a data analysis on low stakes quizzing and retrieval practice and actually the time constraints are well worth the investment for low stakes quizzing okay so the the headline there is if you put in the time effort for that it actually really consolidates learning quizzing regular low states quizzing is really beneficial in comparison to the normal standard practice that you do so making the effort to get those low stakes quids in is really really powerful and a lot of teachers really think of low stakes quizzing as a and a retrieval practice in a way to fill the gap for the knowledge that's in there but actually in the data analysis it was very much shown that it's also a way to boost learning so it's a way to boost learning in and of itself to be able to use those concepts and come up with those ideas and link things together so it's it's well worth using low stakes quizzing regularly even if you feel that that's a time constraint now obviously because i'm a digitization there are plenty of ways you can digitize that primary and quiz being a perfect example of that of ways to make that a more easier process for you as a teacher but as for doing it for learning it's well worth the trade-off um can i shoot to put the researcher's name in that in there not sure with yesterday you've got it so there is a good number of students involved in the studies isn't it for 49 000 you said so it's yeah probably nine thousand students um so it was a massive massive study and because it's it sits in my realm in data science and education they looked at a big meta-analysis all the data of like 49 000 students and 200 studies um so it's a huge data set in your research to back it up thank you so much now let me bring ben in here ben can i um ask you to tell me before and uh before before primary quiz your life as a teacher and uh and i know you're a new user so kind of life now and uh going forwards in terms of your workload and what impact it has on kids yeah well i think as a whole school beforehand our retrieval it was us as teachers creating the activities but going back to your definition of retrieval earlier on i think looking back now it was a case of well basically giving mini assessments rather than retrieval activities so you still had the same children who had succeeded and who could maybe retain things anyway were the ones who were completing the slips who are retrieving everything anyway and the children who struggled with their retrieval were only managing to get one or two things down since obviously primary quizzes come in and they've opened up the opportunities for the word walls for the multiple choice it's opened up the opportunities for more paired work for more of a discussion element to it really um so you know as a teacher it's great because it's a lot better for that but obviously the main benefit is has been for the children and for those children that maybe sort of how they engage with that kind of quiz yeah definitely it's much better like even today when i used it today you see the children who give a little yes kind of thing when you put it on the screen when they see that it's that and so i have it on each morning when i have it on every morning something different primary quiz wise when they come into class and they just sit down and get on a bit straight away kind of thing and how was life just reminders of before primary quiz you know your planning time and headaches to keep kids engaged what was that like well we made everything for it because it was difficult to obviously find the things that you wanted and to even if you're doing multiple choice you know it takes a lot of time to come up with a question base and then to come up with three or four multiple possible answers you know so it was a lot of work that way right now isn't there totally different well that's good that's a great endorsement so um i suspect as more subjects come into the software um is going to be really transformative across your curriculum um so thanks ben for your comments let me just bring my slides back up here so what i just want to remind people of you know the benefits and some of the message you've heard from some of our guests is you know it's a learning tool not an assessment tool it's really important i guess where primary quiz fits in here is some of these kind of research recommendations where students can spot their own mistakes which is great for self-regulation and you can build in that kind of space practice approach that you choose when you want to do the quizzes um you know the short bursts also building in time for students to forget all these recommendations come time and time again from retrieval practice research one thing that i've been doing over a number of years now it's a couple of qr codes on the screen which will take you to my summaries of this paper here on the left which is i guess a screenshot of a meta analysis so this is research of published research and what i found very early on as much of the retrieval practice research on low-stick quiz tended to be referenced to high schools and college scenarios not primary classrooms or even early years and in particular in kind of creative subjects so i've spent three or four years now trying to unpick where the research is so i've got a few of these to show you but here's your first one so if you miss the qr codes now and you haven't got a chance to log into the site look back at the live video and you can see these qr codes again and you can watch the take a look at some of the research papers and recommendations as we go now i'm going to come back i'm going to bring all my guests back in here let me just show you my next question before i bring you all in i think we'll do a kind of six face kind of tet-tie-tet on the next screen and so this is with with technology i guess you know particularly from the pandemic an explosion of technology and now with artificial intelligence it's easy for us all to be overwhelmed so the question is how do you strike that balance between teaching your students and then also using great platforms to ensure your kids can learn so there's the question a big one there let me bring everybody back in or one at a time let's get you all in and let's pop back to ben so ben how do you how do you strike that kind of balance between so many different things that are available to you but how do you keep kids learning well when we look at it it's a case of power say powerpoint for example and just the technologies of presenting tools you don't want to see your teachers obviously just presenting a powerpoint it needs to be a lesson it needs to be an aid tool and with it so we just make sure obviously we do lots of whiteboard work lots of work being written down on paper alongside those digital tools so it is obviously it's it's about them working together it's not a case of one over power of the other it needs to work together okay and can I maybe unpick that a little bit further what kind of practical recommendations ben can you offer to teachers to make sure that happens I think it's making sure that interactivity is there you know when you walk into the classroom it's like that live discussion is happening it's not the children just sat with iPads it needs to make be making sure those discussions are happening amongst the children based on the technology and what you're using it for really it's not just the case of the children considering get on with an iPad which you do see obviously sometimes happening so without kind of putting words in your mouth would that be like an interesting object or an image on the screen or a provocative question as kids come through the door what's kind of your favourite approach yeah precisely so we have you know we have one of the activities on primary quiz is the a word ball where it'll give you a a wall of definitions for whichever subject you've selected and the initials and the children will come in and get on with that and but it'll be on whiteboards and it'll be discussion you know there's ones with images that can go up on there as well so it's again it's just making sure it links in with the discussion and the assessments using the technology for the right you're obviously the right purposes as and when Sarah now I'm going to pop to you now in my life as a secondary teacher I think I used to have the login to 15 different platforms before single sign-on was a thing what's the kind of digital technology eyes and lows in the life as a secondary teacher for you I think certainly at secondary we make certainly in geography and RE we probably make more use of tech for homework setting I don't know if I'm allowed to am I allowed to mention other platforms on here of course you can okay so we use things like satchel one show my homework to put homeworks up for our students so they can access it you know video links PowerPoint links and what have you to facilitate that we also use Neto for for quizzing and that's great because you can have a you know a really quick glance at how your class has done how individuals have done certainly from my teaching I am a bit more traditional and so you know a few PowerPoint slides plus my expertise plus of course the visualize it tends to be the limit for me mostly in lessons there's a children are quite big fans of things like lookit which I tend to reserve possibly for you know last days of term or or something where I've specifically planned I think the point I'd make about any tech whether it be at home or in the classroom is that is it activating that hard thinking that we know leads to learning or is it acting or serving as a distraction so whether it be the tech itself is it too tricky to use is the learning is the content actually being lost through using the tech or is it enhancing the learning that you want to take place so I think you know like with anything whether it be a worksheet that's got too many decorations on or too many instructions I think as long as what you're using facilitates that hard thinking and it's not essentially an extraneous load within the classroom then it's going to it's going to be a positive a positive thing for your students Amen to that and there's nothing I could argue with there Sophie if I pop over to you you know you're new to primary quiz I'm sure there's other type kind of technology solutions and tools that you use for different purposes in your classroom but how do you strike a balance and could you remind us of what year group you teach yeah so I'm in year two so we do do a lot of practical activities there's a lot of um we have a lot of equipment we have that out a lot like using the apparatus and things like that and the brilliance of primary quiz is that you can just work alongside doing practical activities I could do a practical activity in my math lesson each each day and still use primary quiz of the as a starter as a as a plenary at the end of it and you're able to still use the digital tools as well as practical activities we do a lot of active lessons as well we use teach active for some math lessons and and English lessons like the children are engaged from it we still still find that even though you know in the lesson they've been active they use and the apparatus they use and practical equipment and then they can still come back and do primary quiz because there's such a range of activities on the website yeah it's just they're always motivated by it it was like Ben said before when you put it up and they're excited that it's on so it's just as engaging as using the practical activities I've found um yeah and that sounds like a nice response as students get excited as soon as they see on the screen so that that's a good really can we can we put the quiz on can we do the quiz like yeah great Connell can I come back to you for any words of wisdom in terms of that big technology question and you know particularly in the kind of era of you know new AI and things up there how do we yeah how do we just strike that balance yeah I mean I think there's so much technology now available for teachers to to bring into their classrooms and I think you know I've used a fair amount of different technologies myself when I was when I was a teacher and and a headteacher I think um that the the the issue for me wouldn't necessarily be about getting the right balance between the technology and the traditional teaching methods I think for me it would be more to ensure that any technology you use is contributing to the the the the creating meaningful tasks for your pupils and add and add value to the task that you're doing I think for example you know without without I think I think it has its place but I think we've got to be wary of technology that perhaps over gamifies learning because I think you're putting the focus on the on the on the wrong thing you're you're motivating children potentially for the wrong reasons and I think any technology that creates a meaningful task and is is going to be of value so you know what would I describe as a meaningful task anything which helps pupils acquire new knowledge anything that allows pupils to revisit previous learning anything which extends vocabulary anything that allows them to to join in discussions or debates or or or think and draw conclusions those are all meaningful things you want pupils to be doing in the classroom and any technology that helps you to to achieve that for your pupils I think is is is a good value for money fantastic now let me remind everybody watching behind the scenes that on your preferred social platform you can leave a comment for our guests I can see a flurry of comments coming in telling me where you're watching from so let me just remind you to post some questions some general comments and we'll display those on the screen throughout let me just pop back to another piece of research for everybody so here's another primary academic research paper I've discovered so if you scan that QR code either now or after that and watch it back on the reporting this looks at the kind of effectiveness of traditional space practice and retrieval for developing vocabulary and we'll look at how you know we'll talk about some examples here in our classrooms in a moment and also look back at what primary quiz actually does so there's another paper for everybody now let me pop over to question three for our guests and I'm going to bring our teachers in first and I'm going to finish with you Connell before we look at some primary quiz screenshots I want some practical examples either with technology or preferably also without and how do you encourage that collaboration so maybe students working in teams or groups to kind of boost that engagement and motivation so Sophie can I come to you first for some words of wisdom kind of two or three prompt question there how do you bring you know that quiz into life at the moment I just feel like the use of primary quiz is the main tool in the classroom I think it's just it's become an essential for retrieval practice there's just so many aspects of the website that we feel that that's what encourages the children to work collaboratively as well like we do shut the box and it's it's like a group task there's class one like as a class game we can do it there's independent tasks that are encoded in retrieval it's the same as the peeling and they can go head to head there's just so many opportunities for retrieval practice in the primary quiz and can you give us a sense of how are you using it every lesson at the start of the lesson end the lessons give us a clap yeah because we have quite a staggered coming in time at the moment since COVID they can come in over about 10-15 minutes we always have it up and pasting in the morning and they'll come in and it's also it means that we can identify some gaps in learning accessing learning from you know other lessons we can see if there is any gaps that we maybe need to cover in less again and then again we can use it at the end of a math lesson we're using it in English if there's ever any gaps in the day it's just so quick and so easy to have like Connell said a meaningful task for them to do there's just no wasted opportunities no no more dead timing class where you might have packed away accidentally and had to let the kids out early you've got as long as you can pull back on that finish a little bit quicker than you'd realise as well and you can just primary quiz offered to be brilliant Caroline let me pop to you for some practical things and ideas that you use in your classroom and just to remind that everyone watching bring your ideas and the chat yeah I do I teach physics in secondary I teach E.T. at the moment I'm doing some D.T. work with British Aerospace and I I also go and do ed tech and specialist tech in primaries as well so for primaries I think you know when you kind of doing low stakes quizzing you had a question earlier that I was desperate to getting on on tech and how we use tech and what we decide to use with tech and there's some great things you can do that are practical you know you can get your micro rates out and computing physical kind of resources that great but I think some of the things that we've got from primary quizzing do actually represent some of what I'd say being in the research of this area is probably the better version of the kind of tech we should be using it's based on solid kind of research for retrieval which we know works and we've got really good backing for that a lot of technology does come in that doesn't necessarily work so when you're trying to pick things try and pick the things that are much more actually than the traditional comfort of teaching and good practice so if I'm doing kind of quizzes in and I'm trying to do them in a kind of collaborative way in in classes I tend to use space learning I tend to use you know traditional sort of retrieval practices that are there I tend to use tech to adapt them to make sure that they're inclusive so students can access them if I'm doing them with very young kids then I tend to make sure that we do quizzes if we can do them online great if we can access them really quickly and better and I've seen in one of the comments somebody else has just put up as well the idea that students can feel safe in their environment to do quizzes what if you're testing them all the time they get a bit anxious about it there's actually some really good research at the moment from Arwin that'll put in for us and from Yang and from I think I've got on my list Brewer and Owensworth they all say that actually those with a lower working memory capacity so those who are going to struggle to kind of do those kinds of tests benefit the most to doing kind of low sleep squeezing so if you can feel nerve wracking to give them because they may be anxious they will benefit greatest from regular kind of retrieval part you prefer the academic research to benefit the benefit of being able to share what it is particularly beneficial for disadvantage kids yeah and I think quite often you get nervous of that because obviously you don't want to feel like they're in a continuous testing but it's normalising making quizzing a regular aspect of your classroom life so for me I'm a regular with that let's try this let's do it quickly you know let's see what we know and that just holds upon their their idea and their concept within that learning that's going on as well so I've got a nice comment here from Cherise so I'm going to put Sarah on the spot here in terms of EAL learners how do you adapt your retrieval practice low-stake methods to suit you know students that might struggle to access your curriculum so not just EAL but all sorts of send issues do you mean I'm assuming yeah you know how do you how do you differentiate and adapt your resources yeah I was going to say in my context I don't have very many EAL at all actually where I happen to teach so yeah of course within my setting I have a whole range of send issues might be neurodiversity or cognition and learning so I think as I said before I would ensure and I do ensure that my do now tasks which tends to be where I do the bulk of my low-stakes quizzing always start off with a really quick easy win that everybody can answer and our kind of rule of thumb at Rygate school is that if somebody another member of staff who's not a specialist was passing by the door and happened to poke their nose in they too would be able to answer that question so I'm talking you know for example yesterday I started looking at urban issues with my year nine geographers and my first question was what is the capital city of England there still was one student that wasn't too sure on that but he needed a little bit of scaffolding but making sure you've got those really easy quick wins in there so that everybody feels success and that means they're more likely to give the remaining questions that are a little bit harder ago and it might be that they haven't got all of them right you know the idea is that actually everybody should be challenged but just having that quick easy win is one way that I would scaffold I also make sure that I'm circulating whilst they're quizzing I tend to do them on mini whiteboards so they're writing bigger I can see what they're writing I'll make sure that I pass by quite early on those students that might struggle just that little bit more and possibly give them private hints that maybe other students don't need so just trying to get all your students to meet that that top end goal some students might need a little bit more guidance than others absolutely and Ben you mentioned some practical strategies that you used earlier have you got any particular recommendations of switching it to support your EAL or SCND students yeah well we can obviously dependent speaking from a maths perspective as maths subject lead we'll encourage does a place value counters activity for example on primary quiz so it's actually getting those physical resources out for maybe children that need that's extra support so they can physically make what is represented on the screen and on the technology with them as well alongside that obviously we do mixed pair into mixing up so you might have a more confident learner the less confident learner together but again it's them being part of those discussions which I think the low stakes aspects of things and the discussion aspects you know as long as they're part of that discussion and they can they can discuss their learning then we've noticed the big improvement with our ability and that's a really important point to make Ben so thank you for your contribution there's a question here from Lays in terms of mini white boards using them for the first time so there's a free a free question to anyone who fancies answering that one either now go on Sarah I was going to say I don't mind because I so I actually have my own classroom for the first time in quite some time this year and essentially I've got one of those each pair of children so each each desk that seats two children has a zip up of all large plastic wallet and inside each of those two each of those wallets is too many whiteboards two whiteboard pens and two rubbers and they just sit on my desk the entire time on those desks the entire time they never get put away so essentially my students get very used to having the equipment out as soon as they walk in they know what the routine is they have the low stakes quiz the retrieval quiz the do now task on the board and it just shows the power of routines that after two or three lessons they know exactly what's expected of them no one's having to hand the whiteboards out or collect them back in just the logistics it really really speeds it up so there you go Lee's pouches and pens and everything in place to help the learning process now I'm going to bump you all off the room Connor I'm going to bring you back in in a moment if that's okay just to answer some questions on primary quiz so we've talked about primary quiz a lot so in summary you've got kind of five recommendations or explanations of what primary quiz is I'm going to bring Connell in here to explain three or four images that we'll see on this and the next three slides and we'll just return to some of these comments here so the first slide I've got for you Connell is here could you explain to everybody watching what we can see on the screen and just remind us of those core messages about what primary quiz is I've got three slides for you in five minutes if that's if that's possible and you're on mute here Connell so you just need to unmute yourself let's see if I can do it for you there we go hang on help the teachers solve a range of different types of problem in terms of retrieval in there in the classroom and some of the resources for example the ones you've got on the screen and there's also a set of key stage two resources as well I'm not sure whether you're going to show those Ross or not but these are just designed to help pupils in year two and also at the top end of year five and six come to terms with what they should be able to do in regards to arithmetic at both of the end of key stage one and at the end of key stage two the the end of key stage two the tasks are particularly useful because they tackle every single just go sorry Ross just go back to the previous one I'll talk about those in a second so so yeah these are the key stage one examples but there are also the equivalent for key stage two as well on the actual resource and if you know just just to talk for example in terms of you know even if we were to talk about stats preparation for example you know be really useful for that because they've got every single type of addition, subtraction multiplication, division question on them to really help prepare pupils thoroughly for their sat's paper so we've you know we've gone through all the all the papers and made made sure every single question type is reflected somewhere in those quizzes regular practice will soon have children up to speed and dealing with the challenges that the the paper can throw out them but sat aside you know it's just good good practice at different types of arithmetic question in difference in different formats this slide here yeah so what we've got here we've just taken an example of one of the sections of the maths resources and we've got some sort of place number of place value activities Ben actually alluded to the first task earlier on this this evening top left is a just a counters exercise where children have got to work out what number is represented within the value of the of the counters to the right of that we've got a sequencing activities you can change the settings you can choose different sequences you can even change the number of gaps the children have to fill entirely customizable and it's just a little bit of practice there is a national curriculum objective doesn't it so so just getting used to children completing number sequences bottom left hand corner is an opportunity to to ask children to order numbers from lowest to highest so again you can choose two digit three digit four digit oh yeah I forget what it goes up to actually but then you've got an instant activity there where the children have got to look at the numbers on the screen and work out what order they go in starts with the lowest bottom right is just an example of a partitioning activity so we can choose whether we want to partition two digit three digit four digit numbers that even goes up to kind of six and seven digit numbers now and we partition the numbers in different ways we throw out sort of different gaps for the children to fill and it's just we even shuffle up the numbers so they're not in you know the right order from largest to smallest and it really just consolidates pupils understanding of history ones of you here so yeah so we've got some really nice history resources on the site examples so top left we have it's just a quick six quiz designed to be used entirely where it suits the teacher so it could be an lesson start or a plenary but you have six questions on a particular topic all the answers are at the top of the screen and the children can be set a task of making sure and they're linking up the correct answers to to the questions all of these activities by the way will you know shuffle for you regenerate so you can keep coming back and using them even on the same even within the same topic area to the right of this we've got a little sort of timeline activity so it would be quite you know if you wanted to set children a research task or what have you or use a little bit of logical deduction so it can you can you can generate six events again from a given topic and the children have got to try to work together to put the events in the right order so you get various clues and what have you but it's just there to consolidate you know get getting enabling children to sort of organize things on things on timelines you know we've got little activities and we've got little pictures of the timelines as well associated with these activities those will enlarge on the screen for you so you can have a discussion about what happened when and the bottom left here it's a nice little team game actually I think you know there's the primary quiz activities do offer a bit of a variety and with this one you can put your class if you if you like into into two teams and choose a topic and bring up a a whiteboard with a grid with just letters on it and then the teams take it in turns so just choose a letter from the grid the letter generates a question and the the team we've got to answer the question and if they do they can turn the that particular square into their color and it's just you have a little competition to see who can color in the most squares on the screen and then finally in the bottom right we've got a a storyboarding activity so we can take an event and we can shuffle up the all the the sort of take I think one of the what's the example that we've got here it's the Great Fire of London the story of the Great Fire of London we tell that story we chop it up into into segments then we we help children with you know reading out the different segments and then they can work together on seeing whether they can put the put the different segments of the story back together in the correct order so it's just a bit of sort of chronological or you know ordering ordering events at key stage one or fantastic now I'm going to bring everyone back in the the room here now I've used primary quiz for about three or four months just from a kind of teacher training perspective you know what you see these purple boxes on each of these screens here are where you can reveal the answers and regenerate the answers and the possibilities are endless and I guess just some benefits here for people watching virtually before I pose maybe a final question to our guests is for a teacher workload a piece of software such as primary quiz I know and I'm not in the classroom front line anymore it's going to be transformational as well as that engagement for your students and that low state cuisine that really boosts confidence and motivation and then that fifth one there you know no more queuing at the photocopier must be fantastic for teachers let them have the break time back so the question I've got for you all is whether you use primary quiz or not how do you think this is informing so the data that you might get from other platforms or the responses you might get from students that might use primary quiz how does it how does it help you adapt your teaching in the moment so let I'm going to come to Caroline first if that's okay and I'll move the slide and I'm going to people watching virtually please post some questions in the chat box we've got about another seven or eight minutes to go so Caroline the data question to the data scientist yeah so I'll probably point out with this one that good information with the data is key I think all of you intrinsically know if you're a teacher that is the data we're collecting and does it actually make an impact or as Selwyn would say who's a great researcher to read and are we just messing around with Excel's spreadsheets so I think the key thing here is when you're using things like primary quiz is that this is key information that's low stakes that's you know you can just put out there that'll instantly give you the kind of feedback that you need to integrate into teaching so I'm all about having the data to inform teaching but being able to make the key decisions about whether that goes true and relevant is it really informing your teaching practice because I think a lot of teachers you see regularly have been collecting lots and lots of data and actually a lot of it doesn't really inform the teaching practice in the classroom even though a lot of people think it does actually you've got to be really careful with your data exactly what you collect so these kind of like small stakes informal quiz in and retrieval practice those kind of platforms great because they're informing you in that moment exactly data should be telling us what we don't know not regurgitating what we already do and I think there's something to be said sorry just while you escape there's something to be said as well as say for the type of quizzing that you're doing being corrective as well so if you have low stakes quiz in in combination with corrective feedback verbal feedback you double the output you get from learning and there's some lovely research on that I'll pop in the comments pop in another LinkedIn chat box to share that would be great so really important point Sophie over to you in terms of being a new user to primary quiz how's that kind of data that you collect and I guess it's worth clarifying that primary quiz doesn't collect numerical data from students how does that help change the way you teach in the moment Sophie and you're on mute here so need to just press a button on your side and unmute yourself yeah sorry just to say that as we use primary quiz like I mentioned before it's coming up in the lesson maybe where there's potential gaps that we might need to come back to and whether we need to teach a new lesson on an objective or whether that could also be spotting children that might need some interventions or small groups it's just about being flexible in the classroom and you've just got to adapt your teaching to how your children are engaging with the activity and what they're achieving in the lesson it's just being able to spot misconceptions so just to make sure that everyone's clear who are not familiar with primary quiz let's say you're doing a quiz with your students Sophie and maybe they find it too hard how easy is it to switch to a different activity oh yeah it's just so it's so easy and the website's so quick that you can't and because like we said before there's such a range of activities if you're yearning cohorts because sometimes we'll cohort for our maths groups and that means that we can use the different activities depending on the ability of the children in the class and as we said before it's about differentiating and putting the support where you need it and put through primary quiz we're identifying gaps in the learning maybe from prior years so that we think well we need to target that area it's just so I just think a primary quiz now and you know I won't go back in the day kind of phrases but you know if you've got that one photocopy sheet produced you can't you can't run to the photocopy and grab another one so primary quiz allows you literally to click the button to switch over yeah sorry you can change the levels as well sometimes I know another year group you can go back a year group if you needed to as well if you felt you needed a different level or a different ability you can always go to you can go to year one or you can go to year three yeah I am actually just for a quick insight a quick point of view from yourself as a school leader I'm assuming you're using primary quiz across the school so what insight has it given you in terms of seeing changes to your teachers in terms of their workload their habits their pedagogy and students across the school can you give us a quick summary yeah well I think obviously all teachers now we are we're bombarded with new resources new things that SL has seen you in saying we're asking teachers to do but with primary quiz it's so obvious even just walking around the school you can appear into classrooms and you just see it being used constantly in every year group so I think that's like one of the biggest endorsements because there's only things that as teachers you get given and you'll use for maybe a week and then it goes kind of back in the virtual cupboard so to speak but with primary quiz we're seeing it being used by teachers constantly throughout and it is it's being used to guide the discussions you know there's no point in using things like that and revealing answers and moving on teachers are using the answers to generate discussions to address those misconceptions to get that retrieval within there so we seem really good engagement it's actually genuinely informing their teaching and it's a resource that you know that kind of tick box or flash in the pan kind of thing it's not though any of those at all Sarah, any last words of wisdom and from a secondary perspective you know in terms of how your retrieval practice methods you know generate that data and insight to change the way you teach I was just going to say about you know kind of the bigger picture is that when you think about the teaching standards and the eight teaching standards that we kind of have to abide by one of the eight teaching standards is responsive teaching so you know tools like primary quiz or you know just your basic mini whiteboards or anything that's going to inform your teaching whether it be in the moment or possibly putting it you know on the back burner for another lesson to address a little bit later on anything that allows you to do that has got to be a good thing you know it's something we should be doing come what may regardless of how we're doing it fantastic now I'm going to wrap things up I'm conscious of time so let me just pop over to this slide here so there's a QR code here on the screen Connell could you give us a kind of signpost to this obviously takes you to the free trial but give us a bit more information for people watching what they can do when they land on your site so there's various there's various options at this point we can if you get the best thing to do is to go straight to the www.primaryquiz.com that will allow you to try and activities and activity on the on that particular page that you can you can use straight away you can sign up for your free trial or you can simply contact me and I'll share you I'll email directly to your inbox some examples of the activities that you can use with your class I'll email you five or six examples and you can just click on the links and you're good to go what I would really advise is that if you are interested in taking a closer look just send me an email just email me straight away and then I can find out what I can answer any questions that you've got find out what your particular needs are and then I can even set you up with a free trial from this end or as I say email your stuff directly to your inbox but just get in touch and we can have a conversation fantastic now I'm going to ask Georgie behind the scenes to put the free trial website link on the chat box at the bottom of the screen if you could so my final question to guests watching and also to people watching virtually you're this is your last call for any questions or statements I'm wanting your number one top tip or piece of research wisdom that's changed your classroom life and if you can summarize that in less than 30 seconds just so we end the kind of session with kind of five recommendations for people watching to consider or take away I'd be most grateful so we'll start in reverse register order with Sophie and end with Ben so Sophie your number one recommendation tip or thing to read it will be to download primary quiz because the impact going to have on children's learning and teacher wear is just going to be make such a difference in your school so right fantastic and she's not being paid for that so that's right I love the primary quiz so much it's just brilliant brilliant right here are we with Sarah next top tip Sarah and I think for me what got me really interested in cognitive science and educational research was having a read through of Rosenstein's principles I actually have it here weirdly so Tom Sherrington version is a really look how thin it is it's a really quick quick read and it's all it's common sense that it's based in in research and it's a great great place to start if you want to get involved in educational research fantastic tip I can strongly recommend that book Carol I know it's you that's so hard how'd you follow on Rosenstein let's face it I mean so it's like yeah Rosenstein explicit teaching dual coding all of the kind of retrieval practices that I know and love but one that we just put in the chat for you that's really recent is Young's 2021 that is on low stakes quizzing and that is the meta analysis of it that basically has probably made me rethink this thing regarding how frequently we should be doing it and how regularly we should be doing it and exactly where we'll get that link more regularly is better actually even if it is time you know heavy to do what's the second planet in the solar system something simple like that brilliant thank you Caroline Ben any words of wisdom and I think as a as a whole school the biggest change for our retrieval in the past year obviously primary quiz has been a big part of it and but it's been also that discussion discussion discussion it's a case of it's not many tests that's not retrieval it's a case of using it for discussion and just letting the children talk about the past learning that's been the biggest kind of change for our teachers really is not testing them is not testing them on it it's been a case of allow it as a platform for discussing the previous learning fantastic and Connell could you remind people watching of you at the website for people can sign up I've got a short code here in the screen but give us the official website name yeah www.primaryquiz.com all lower case am I allowed to share a word of wisdom of course you can so I think it would be just don't underestimate the value of using low stakes quizzing in your classroom but in a way to support learning rather than measure it I think as soon as we start quizzing you know we start thinking in terms of assessments and we're not we've not been discussing assessments tonight we've been discussing the the value of quizzing in a fun supportive inclusive environment and my second tip would be don't underestimate the value also of filling children's head with knowledge because it's with knowledge that we we do everything else that we do whether it's learning a new skill or building new knowledge and if you want a good read there's an American author called Hersch HIRS HIRSC he talks about the importance of building knowledge rich curriculums and and that's what I think we should all be doing fantastic that is a good way to conclude so let me just bring back on the slide here thank you to everybody so there'll be one or two people asleep that will be watching this when they wake up tomorrow but people at least in the UK and to the west one or two people in America and anyone else watching this after that finds it on the internet thanks for joining us please thanks for watching please get in touch with primary quiz it is a brilliant piece of software highly recommended from me and top value you've heard from our teachers and special guests tonight and the benefits of retrieval and also some platforms that can help transform learning for your kids and our guest teachers watching and just watching looking at the stats on my so there's a large number of you still watching at seven o'clock here at night in the evening so that's a good endorsement for the benefits of such a very important topic so wherever we are in the UK watching this please take a look and if I don't see you online and see you on my travels then thank you very much for watching thanks for all your comments I can see still coming through in the chat box and thanks to my special guests I hope you have a lovely evening thank you for all your words of wisdom and keep up all the wonderful work that you do in our classrooms with our young people bye for now everyone thank you cheers


it's not perfect okay we're at the tail end of the day two more presentations to go and the first of those is by Jens Freiman yes and on Vert.io 1.1 what's new in the next version okay everyone can you hear me okay so yes my name is Jens Freiman I work for Red Hat and this talk is about the reason what I work that we've been doing on extending the Vert.io specification so the goal is to extend Vert.io in a way to improve its performance and also to make it work better with workloads that it currently doesn't support very well so I'll give a very very brief explanation of Vert.io and just put it into context to NFV very shortly before talking about the main new topic and Vert.io 1.1 which is a feature called packed vertices and eventually if time allows I will talk about a few other developments in the space of Vert.io that I think are worth mentioning here to this crowd but first let's start with what Vert.io actually is so Vert.io is an abstraction layer for devices and hypervisors and the idea is to have a common framework for IO virtualization and at its core it defines virtual cues also a way to configure devices and mechanisms for backward compatibility Vert.io is quite good at that so there are many different devices there but on net, block, console and so on and to guess these devices appear as normal PCI devices or if you're on a mainframe as channel devices good so and to put this into context to NFV so where on a possible NFV setup is better you're used and I really just picked a random example let's say you're using a DPDK based application in your VM and then you have a Vert.io front end in the guest and you have a Vert.io back end in the host in this case Vert.io is part of the DPDK library and in the host it's part of your stuza in this case also part of DPDK and by the way I guess most people are familiar with DPDK but this is like a library and provides users based drivers and libraries to build fast package and package processing applications so yeah just a part talking about the Vert.io architecture at the top you see the device specific components this has everything that's specific to Vert.io and that block and so on at the bottom you have the transport specific parts as I said devices can be PCI devices they can be also CCW devices that's for mainframes but I want to talk about the Vert.io part today so we want to add a new kind of Vert.io's and so why do we want to do this the very Vert.io's are defined until Vert.io 1.0 and I'll call them split Vert.io's from now on as opposed to packed Vert.io's they're not really fit for a straightforward implementation hardware so I've been expecting hardware support for Vert.io for quite a while actually and it never happened and well I guess it's just because they're too complicated to implement I know that one company once tried it and stopped at some point and now there are several companies that make an effort to push forward in this space and actually try implementing Vert.io in hardware and make accelerators forward but not also hardware also pure software implementation should benefit from Vert.io 1.1 and the packed ring layout so what's the problem with the old way of Vert.io's with split Vert.io's so this is what the data structures of it look like so first each request is stored in a buffer and memory shared between the driver and the device the buffer address and lengths are stored in the 16 byte descriptors and they are stored in a descriptor table now if the buffer is not contiguous then multiple descriptors can be chained together by setting a next field in the descriptor so we're on the table to put each descriptor that's up to the driver and a request is identified by a head index which is a 16 bit offset into after first descriptor in the table now head indices are stored in the available ring and to signal to the device which indices are valid the driver also maintains an available index in shared memory and after executing the request the device will store the head index in this use string which is also in shared memory it also stores the length of the use part of the buffer and it then increments the used index value in the shared memory to signal to the driver that this descriptor has been used and so fundamentally what we see is that hosts and guests communicate by passing messages between CPUs using shared memory and how do CPUs do this so at a very low level when a CPU accesses memory that is also modified by another CPU this will cause the CPU to synchronize their caches and they do this by exchanging cache coherence protocol messages so basically they speak to each other for example for some architectures when a CPU accesses memory that was previously accessed by another CPU this will also cause a cache miss and while the number of these cache coherence messages exchanged and cache misses directly impact the communication latency now let's count cache misses when using rather one that the queue consists of five parts and as a buffer moves between hosts and guests each of these parts is read and written at least once so this is implying at least five cache misses per buffer if no batching is allowed if we batch then it gets a little bit better but not too much so what can we do to reduce the overhead so the issue is that the information for this request is sped over multiple data structures and this is causing multiple cache misses so to reduce the overhead we should pack as much information as possible into a single data structure and what follows is a proposal to do exactly that this is currently being discussed on the virtual mailing list and there's a prototype for DPTK also on the DPTK mailing list for working on QMU and Linux as well but DPTK prototype was there first we started with DPTK and so what we do is we get rid of the available and use string and the index structures completely and what we do instead is we make the guest write descriptors out in the script or ring order and in our new descriptor ring so within each descriptor we add three new fields or flags and available bit use bit and descriptor ID then to make buffers available to the host the guest will write them into the descriptors in our new ring and then it will flip the available bit this implies that it's now okay for the host to consume this descriptor and the host can then process them in any order each descriptor also has an ID field as I mentioned and after processing the host writes the process descriptor ID back into the ring and it flips the use bit and by this it's telling the guest that an entry has been used and then it can that the guest can use this again or can make it available again so device and driver are also expected to maintain internally not a single bit counter and so starting at one and changing value each time the last descriptor in the ring is accessed it has to be flipped and the last one is accessed and as I just mentioned these bits are passed as available and use bit and the descriptors so assuming that the ring is zero initialized on the first iteration over the ring to mark the script as available the available bit is set to one and then on the next iteration it is set to zero and so on similarly to mark the descriptors used the use bit is set by the device on the first iteration to one zero on the second one and so on so what do we get by these counters well flipping the counter values allows the host and guest to detect new available descriptors even after the ring has stopped because it could be that some descriptors were not touched in one pass of the ring because we skipped them and this way we will know that this descriptor was made available and is currently to be used so as I said we have an implementation and I was measuring performance according to RC 2544 to stand on Intel based servers host and guest on the device on a test we're running rail 74 at the time the patches were based on the PDK version 1711 and the patches are now available on GitHub and hopefully soon also upstream so apart from micro benchmarks this is the only real test run I managed to do before this talk so with 64 byte packets we measured we measure the boost from 18.8 to 22.6 MPPS and well this is using slow nicks so larger packets simply hit wire speed on the setup and I'm working on benchmarking with larger packets so if someone is here familiar with t-rex and in combination with excel 7010 Intel nicks then please talk to me I have some questions yeah so I think we can conclude that what use as defined in 1.0 are not an optimal data structure for host to guest communication and also not perfectly fine for an easy implementation hardware and we're trying to make this better with 1.1 on well by the way if you want to participate in whether or work on a specification or you have a new feature that you want to add you have a new device that you want to add this is actually quite easy and also fun to basically just download the specification source from GitHub you edit it it's a lattice so you have to build it compile it and look there to make sure it looks right and once you've done this you basically just subscribe to this virtual comment mailing list you have to reply to confirm your subscription and then you can send your actual sometimes it's a one-liner patch to reserve just a feature bit you send it to the mailing list and if everyone's okay with it then you send another mail and ask for a ballot this ballot will run for for a week and after two weeks if all goes well your feature is in the specification good so that's the part of my talk where I wanted to I was going to talk about whether or specification now I have some actually two things that I wanted to mention that I think are interesting and so one thing is a project initiated by Intel PDPA it's called a virtual data pace data data path accelerator and well so better as a para virtualized device you know decouples your VM and your physical devices and that's nice and cloud environments because you can easily live migrate them and so on but in terms of north south traffic that doesn't compare to for example an siov device obviously now this picture here which is probably not easy to read it shows how it looks without an accelerator device and Intel is now working on a framework to make it easier to use a virtual accelerator device so gently this will allow you to use hardware that can write and read the whether or viewing format and it will get you basically siov like performance and and the data path and the way they did this will also make it easier to basically switch from a setup where you just use normal whether or to one that support that's actually using an accelerator device so this picture includes the accelerator device and as you can see the control path is still emulated but this is not a bad thing because it's not performance critical but the data pass is now handled by this hardware device and that can write directly into the viewing um yeah with traditional you would usually do packet i o via shared memory and interrupts via iq of these and kicks double via i event of these um back end then would be usually be host kernel or user but with vdpa that's a bit different you have the double kick and this could be depending on your guest either part i o based or if you have a newer guest then would be mmyo mapped um interrupt notification here is done via fio interrupts uh lacking pass through devices and as i said nq dq to the reeling is done with the accelerator device um it's also using vfo m therefore address space access into dvm um and the control path is set up by vhost user protocol messages basically um also was mentioning that here it's currently not using a virtual iam view um but that's something we're looking into um that should be improved so while you might ask if you have a virtual capable device why don't you just pass through the entire device to the guest um and well there's several reasons why you might not want to do this and one is that well there was a growing spec and things evolve in step and the hardware implementation would have to keep up with the uh spec all the time um kind of unlikely also physical functions have a lot more um features and metadata and so on that a virtual function doesn't really need to care about you would inherit all the pass through uh properties and that includes also um it's harder to live migrate um without having vendor specific solutions um so yeah for example in brother 095 um you couldn't really pass through a device to the guest because all the register base access was um part IO based and eventually the major reason is um that the nice thing about whether that is that you have a front end and a back end and that's decoupled and you can combine things so the front end is often um not a static but in the back end it's more common to have a choice of um the actual vertical back end okay um this is another project that's currently uh in the works it's also um started by Intel and this is more about efficient east west communication between virtual machines um so imagine you have a set of virtual network functions like i don't know firewall intrusion detection system router whatever and they need to forward packets to each other you would usually put them into a virtual switch in the host and then packets gather there and transfer it one by one or to the target VM and the nuts that you said you have like a long code bus and um it doesn't always scale very well um but what if instead you could send packets directly from uh to the target VM without going having the data pass going through the host um code pass would be shorter and it would be also easier to scale um now the way this works here is that they have introduced a new set of VOS PCI devices that you have in the VMs and also we host PCI server and uh as a central component in one of the VMs um and they are connected via unix sockets and uh they speak the existing VOS user protocol over that um to set things up so there's a presentation um from kvm forum about this um there's also a recording video on YouTube if you're more interested in this um the current status is that there are patches on the mailing list and they are being discussed there are still some design discussions and um so the thing was that the way they wanted to do this would also introduce a lot of code duplication especially in the VOS user part so um Stefan Hainochi suggested that they could use something similar um and okay so where are your devices usually um where just you know you had your driver and a guest and you had an emulated device in QMU and a host user space process and then as it evolved you had um so Vhost user uh Vhost came along which kind of um allowed you to implement part of the where they are back end and the uh kernel and user space um and then the next thing was actually Vhost user which would allow you to implement this back end in a user space process in the host now whether Vhost user is kind of taking this even one step further by moving the Vhost device back end into the guest and it works by tunneling the Vhost user protocol over a new virtual device type um which is called virtual Vhost user um so yeah the final diagram will show this um so in this diagram VM2 sees a regular virtual net device um VM2's QMU uses the existing Vhost user feature like if it were talking to a host user space um Vhost user back end and then virtual machine one QMU um will tunnel the Vhost user protocol messages from VM1 QMU to the new word IO Vhost user device so that um guest software and VM1 can act as the Vhost user back end um now it's possible to reuse existing the Vhost user back end software um this for the user since they used the same protocol right um now a driver is required here for the Vhost whether Vhost user PCI device um so that code has to be written for that as well and the Vhost device back end V-rings they are accessed through shared memory and do not require Vhost user messages to be exchanged in a data pass so no VM exits are taken here uh when pole mode is used of course not at all but even when interrupts are used then QMU is not involved here because you can use the lightweight IO event fd um VM exits um yeah so Java can be implemented in the guest user space process um using linux vfio PCI but uh also guest kernel drive implementation would be possible um also the Vhost device back end V-rings um as I said they are um accessed just using shared memory um it's also worth pointing out that this works for um net devices, block devices, SCSI and so on um another thing I think I might skip this here how much time do we have left okay then I might just skip this um by one sentence so this is just a new idea to do um transparent bonding in the guest by without having complex configuration by the user so the idea is to just have a virtual device that has a special feature flag and if this feature flag is seen in the guest then it we can go and look for another sriov device that we can bond with and then we could have um always use the fast data pass over the sriov device and for migration we could just put the link of the sriov device down and then switch over to a virtual device to a migration and on the target system see is there another capable sriov device that we can use for data pass and if not we could just um use the virtual device so yeah conclusion of my talk um so virtual 101 will be a rather big release um there will be many changes um mostly for the packed word queues but also some other things required for um hardware implementations um it's worth taking a look um dpdk implementation is available I didn't put the link in here but it was in a previous slide so you can look it up there um also if you're interested in this and in participating we have like a monthly meeting a monthly call where we discuss these things very one at one but not further about your features you're welcome to join just contact me um just quite a few companies already involved and you once are always welcome so that's the end of my talk basically um are there any questions there's one um so the question was um the implementation of weather 101 will require changes to qmu and what's the timeline for that um so right now the next target is to get things some upstream and dpdk and we're at the moment already started to work on the virtual implementation so as a few months any other questions okay thank you okay thank you yes so last talk of the day coming up now uh in just a few minutes with Pablo Camarillo I hope I pronounce that right on segment routing v6 with vpp nice to meet you so we are actually two so it's not me that it is just plain minutes and I'm going to explain to you okay and when you plug it in just give it give it a lot of time I'm sure you're meeting it after you don't have that just for those leaving one last reminder that we have a meet-up a little bit later for the stnnfv dev room uh in the mannequin pizcafe um which is rude grand calm I think around 7 30 so you're welcome to join us thank you I don't want to take any questions but so whatever I tell you that's when the end of year 30 okay so good afternoon everyone last talk of the day and then you can go for beers um so this is my colleague Ahmed he's a PhD student from Italy and I'm a software engineer at Cisco working in the segment routing architecture team and today I'm here to talk about segment routing which is something quite different from all the talks that we've been seeing today because all of them were focusing on different platforms and this one what is focusing is on the actual protocols on the network actually on sp networks um so what I'll try to do is give you a brief overview of segment routing to deployment use cases and then um talk a bit about bbp linux and serra which which you will see later what what it actually is so what is segment routing the idea behind signal routing is that we actually leverage the parting of source routing so what does this mean that instead of programming all the routers in the network what we are actually doing is that on the head end we're actually adding the list of segments that a packet has to traverse through the network so what this means is that if I want to go from Madrid to Amsterdam by a Brussels what I just need to do is when my packet is out from Madrid I just add one little segment that is saying Brussels and then Amsterdam and the packet will follow the shortest path to Brussels and then it will go to Amsterdam that's simple and this is actually really scalable because what this means is that you can implement any traffic engineering policy that you want and you can actually um put this together with any nfb deployment that you want and actually one of the main benefits is that we can have policies end to end so starting from the data center and I'm going traversing the entire networks through the metro on one so we have two data plane instantiations one of them is npls and the other one is ipv6 so in npls what we are just doing is one segment is one npls label and that's it the second instantiation is ipv6 so in ipv6 what we are doing is we are using an ipv6 routing extension error which it was defined 15 years ago and we have one segment is one ipv6