So ladies and gentlemen, welcome to this second and the last day of this conference. I hope you had a Good stay in the night. There's a lot of work late evening, which I'm sure your teams have completed. I kept getting mails late night Without further ado, I Would like to introduce our keynote speaker today, Professor Mark Russell True to the tradition that I started yesterday. I'll not read out the full bio data Suffice it to say that professor Russell has been in higher education since 1996 He's from King's College London like many of us. He's an engineer. So we note that with pleasure He is the head of the Center for technology and us learning King's College But what struck me as most important is he's known not for the use of technology and us learning but a thoughtful use of technology and us learning And that I think is a is a great objective. We admire you for that. You're most welcome, sir Well, thank you very much professor and thank you very much for inviting me to offer a few thoughts from I Guess I guess those number of years that I've been working in higher education first as an engineering Academic and I was told to make the case that I am an engineering academic I'm told that would warm me slightly to your heart because engineering is so very prevalent here and I have to say Unfortunately, it's it's a dwindling profession back in the UK. So it's so heartening to see you talk so passionately about engineering here So thank you to Manjula and Jill for inviting me from the British Council and of course the ministry and colleagues at the IIT For making this such an outstanding event. I've been super looked after I've had a wonderful day yesterday And I've learned lots and I'll be taking lots back with me So thank you for that and congratulations on securing such a such a fab event I'm going to share with you some of my thinking around technology enhanced learning and I think some of the Opportunities that exist in front of you. I suspect you know some of these opportunities if not all of these opportunities already But also just add a cautionary tale as well to those opportunities and say that This quite frankly brave and bold venture that your country is is engaging in is not without challenge So I'm just underscoring some of those opportunities with some of what I see as some of the challenges around technology enhanced learning So Mark Russell engineered by background I've been working in higher education since 1996 and More recently the last five or ten years. I've been looking after technology enhanced learning in institutions And I'm currently at Kings College London Which is a large research intensive institution at the heart of London in the UK So let's start with some positives and think about what the opportunities are For technology enhanced learning and I'm going to march you through Some opportunities some challenges and then I will just conclude with a story of my journey here So the opportunities that technology and enhanced learning brings are clearly to harvest and share resources and That was obvious to me yesterday that you were tuned into much of this and a number of you spoke about the MIT courseware Open courseware project a Number of you spoke about the Khan Academy some fab resources that are there for you and I to engage with There are of course a whole host of other Repositories available for you in the UK. We have something called Joram where academics are invited to post any resources that they think other academics and other members of the academy might benefit from YouTube presents a wonderful opportunity to share our resources as well There's one of my engineering simulations in YouTube here with a UH logo And of course the iTunes University site iTunes you provides yet another opportunity for the busy academic to share and use other resources that other colleagues have considered and I Suspect that iTunes you presents another shop front for our institutions as well for colleagues looking in I suppose what I'd want you to do is not just think about providing resources, but also take the Opportunity to use some of those resources that others have used because I'm not sure We need to write too much more about the first law of thermodynamics quite frankly There's probably enough stuff already out there about the first law of thermodynamics Our job as engaging educators should be to draw on that stuff That's already out there and make sense for it and put it into context with the people that we are trying to teach So it's less about the creation of those resources and more about the thoughtful integration of those resources into our curricula So there's the first opportunity But again underscoring the introduction which was about my thoughtful use of technology is the fact that We must think about curriculum design too. So good curricula Obviously is not just about Collection of stuff or a bunch of resources It's the ways in which we weave those resources together It's the story that we Create around those resources to engage and infuse and of course assess our students through those resources So resources are a very helpful starting point in thinking about our curricula The second opportunity that technology enhanced learning brings to us is the significant potential it brings to us as educators to enhance and extend our more traditional classroom interactions And I saw that yesterday through colleagues talking about the virtual classrooms I saw that yesterday when I was hearing you talking about bringing Skype into the classroom I saw that yesterday when I heard you talking about using your managed learning environments for formative assessments for quizzing for engaging students in pre-reading for engaging students in dialogue both before and after the lecture So it gives us a wonderful opportunity to make more of the space we have together both beforehand and after and of course in different domains so it extends our spatial connections and It extends our connections in the time domain as well And I've just put another couple of examples up there, which actually I don't think I saw too much of yesterday Second life and other virtual worlds present some good opportunities for immersive education in simulated spaces and Some work that was supported by the Microsoft Research Foundation was some work at Washington State University on classroom presenter Now I saw the fab fabulous work going on with your tablets Here's something that was done in the mid 2000s I think 2005 classroom presenter at Washington State was allowing students to interact with each other and Their lecturer or their faculty member through digital ink So that's another really smart piece of free software that might be part of your armory and again No doubt with generous support from Microsoft Research Foundation. So it means that we can do more with what we have here and I Wonderfully heard a number of you talk about the flipped classroom something that I think is Taking over much of our thinking around education. So great to see some of that and no doubt you're already tuned into this potential One of the things that I'm particularly interested in doing though is making sense of some of those words and some some of those opportunities and with colleagues back at Kings College London what I'm working on now is Helping colleagues or help faculty members understand the consequences of their curriculum design on student engagement So what I have here is Just as a very simple axis so on the x-axis I have days of the week Monday through One week so I put it on to Tuesday and on the y-axis. I have something that I've been purposely vague about It's either engagement Interest or learning I can put no numbers to that. I'm afraid But I'm sharing this with you for the first time So I'm wanting to contemplate the notions of engagement interest or learning versus day of the week the arrows are Highlighting two lectures so they are pointing to two circles. They are purposely that small Because they represent on that time domain the attention that we can grab with our students. They are two Hour lectures there are two to our lectures. That's what those red arrows are highlighting the yellow circles And I might conjecture That if a student is exposed to that pedagogic pattern two to our lectures That's possibly what their engagement interest or learning profile is going to look like They're going to do nothing. I suspect or start thinking about the lecture then their profile is going to peak During the lecture hopefully Then it's going to dwindle there might be a little bit of noise or there might be a little bit of engagement as they think About things or talk to their peers and then it spikes again at the following week's lecture Now I know that there will be a different Profile for each student so essentially we can now start to create a Three-dimensional engagement interest learning landscape of different individuals So the blue Candidates at the front of that image represents somebody that actually only ever tunes in during the lectures and doesn't do any thinking outside Whereas the purple candidate towards the back of the image is actually Really constantly thinking about and working around that lecture But I'd like to conjecture that that pedagogic pattern to discreet and unconnected lectures Simply create two peaks with some noise or other during the during the course of those interactions And so what we're trying to do now at Kings College London is help people understand the consequences of blended learning design or flipped classroom design or just in-time teaching on students engagement interest and learning and so just simply by Adding a quiz that follows the lecture and The quiz is the blue dot with the red arrow looking back. So it's a quiz that reflects back on the lecture the following day The green let me just quickly point to it here. This is the original profile I think if students know they've got a quiz that follows the lecture no doubt they will tune in and gain more from that lecture experience So you immediately get more interest engagement and learning during the lecture and Then I suspect the quiz actually stimulates yet more engagement interest and learning and by simply Thoughtfully connecting those interactions We lift the area under the or the area under the curve becomes greater and so we have achieved more interest engagement and learning and Of course, it will be of little surprise that we are Developing some of those engagement interest learning landscapes for a whole series of different interactions And we're not doing them as discrete entities. We're actually helping colleagues make thoughtful choices about their whole curriculum design stimulated by Technology and just seeing how technology can add More to those classroom interactions So when you think about what I've just described now clearly, it's giving us an opportunity to either nudge colleagues to think about curricula or Better still in some cases force a complete rethink about their pedagogic values and beliefs and indeed their teaching strategies Somebody said yesterday the fact that we are teaching does not mean that students learn we should hang on to that thought So but what we can do as good Educators is try to strengthen the link between our teaching and our students learning and so it's important that we do come back to the literature around What we need to do to create meaningful learning spaces And if I encourage you to read one other thing if you haven't already come across the work of John Bransford How people learn it took the state by storm a few years back I'm offering that to you because that was a free book on the internet and I suspect it is free now and it has Warts Psychologists those from the learning scientists together as well as educators together to understand and Present a coherent argument of the things we need to do to create an environment in which learning can flourish So how people learn John Bransford from Washington University Washington State? I believe something around what good pedagogy might look like and then of course some other helpful text around technology enhanced learning of which there are numerous others and Noop pointed to the study of John hate of hate I beg your pardon and the work around the ways in which if we engage students differently in our lectures how much the learning gains Improven Eric Mazur has done some incredible work around that as well So the text on the far right-hand side peer instruction is helpful as is the work of Greg or Novak and just in time teaching I Actually think these texts of the forerunner to our thinking around the flipped classroom So they are well worth a skim if you haven't come across them already so I Understand it's important for me to stop from time to time and ask you or invite you to think about what I've been saying So here's that my one thing for you to do. I might only ask you to do one thing If you were to do one thing after this event And I might be greedy and frame it around my presentation, of course It might be to invite you to take a fresh look at what you're doing in the context of what you've been thinking about and what you've been hearing About yesterday and no doubt what you'll be hearing about today So the things that you are doing and have responsibility for and ask yourself is it working is It's supported by the literature and evidence and Have you suitably? And I put in brackets there, but gently sometimes we need to be gentle when we're encouraging others Have you gently shared it for others to benefit from? One thing that astounds me about higher education is our Immensibility to keep secrets we do some brilliant brilliant things in our classrooms But we just sometimes forget to tell our others about it And if we're serious about moving the academy and learning forward We do need to take those opportunities to share our work So there's my thing for you to think about and I would say as an underscore ps Make sure you have a good definition of is it working because if you're going to share that with your colleagues They'd want to know what that means and they'd want to hear your evidence for it and If you want to work outside of engineering what works for us is very different from colleagues in other disciplines as well I don't understand why they would have a different narrative around what it's working than us engineers But apparently they do and there are different epistemological beliefs. So their understanding of evidence is different from ours So that's my one thing for you Just marching on them with these opportunities Tele presents fantastic opportunities to build additional connections I mean, of course, we have connected here over the last day and of course, we will continue to connect But I'm now able to connect with technology and technology enhanced learning with other educators with other students that I'll never ever going to see in my lifetime and Tune into some of their thinking and share some conversations with and we can do that through Twitter through LinkedIn through Facebook through a whole host of social and professional media that is Emerging and continues to be pervasive and I know this came up on a number of occasions yesterday But I think this is one of the real benefits of technology enhanced learning is it's not actually asking or inquiring us to Technologize what we already do. I Think it's got the real potential to actually force us as an academy to look again at what and why we do things And that's certainly been what I think this MOOC initiatives and these spoky initiatives have all been about and I think in some institutions This is such shaking their very foundations Because it's now saying do we really need to be here? Is education going to be I guess or are we going to engage our students differently three or five years? and Here is our first MOOC on the future learn platform that is in its second week now, so Not while I'm talking of course But if you want to tune into future learn you'll see Dr. Kyle Dyer from the world-renowned Institute of Psychiatry at Kings College London talking about drugs and drugs addiction And what it does is these MOOC platforms just allow us to engage with experts that we would never have had an opportunity to engage with And I might just pause here and say so we've talked about technology enhanced learning sometimes we come back to ICT and I might now want to ask well, what is ICT I mean the forerunner to this was it of course it was simply information technology But is ICT information and communication technologies is it as I have implied although not explicitly stated information and collaborative technologies or Is it actually information and connective? Technologies is that what we are seeing now with ICT and where we're going I? Have a view on what I think it's doing and what and its potential and it's certainly towards the bottom of that list Actually, what about this for a model? Why don't let's stop ICT? Why don't we call it the PC squared t4p squared? That's much more catchy isn't it? Well, perhaps not maybe not and it is of course a tease, but it better helps us understand what I think we are trying to do with ICT I think it is about a personalized collaborative and connective technology for pedagogic purposes I Remind you again, that is a tease by the way the PC squared t4p squared model but I think it's helpful that we forget about information technologies being about transmission and we forget about Education as being something that's delivered and we always think about education as an engagement potential And we think about ICT as an extension of that engagement potential both with our students and between our students I don't think I'll patent or copyright that notion so feel free to use the PC squared model, but please do think about the its potential that's Now just change tack slightly and move away from some of the positivity and just Remind ourself that a tell endeavor is not without challenges. I think it's foolish to Suggest that this is an easy path to lead it clearly is not as I suspect everybody in this room can subscribe to So it is not without its challenges So I've been teaching 16 years. I know it's important for me to stop from time to time So here's just an opportunity for you now for two minutes It's with your neighbor What do you see as the biggest challenge? To ensure an excellent tell experience is reported by all of your students and Not just the lucky few that are sat in that room over there and taught by somebody But all of your students in your universities and in your colleges So what do you see as the biggest challenges? Just share them with your neighbor for two minutes Okay That that I know was a very short two minutes But I but I thought it I thought it appropriate just to pause and give you an opportunity that to connect with your neighbor Perhaps to talk about the meal last night or actually to think is what is this guy saying? How does it make sense to me and actually? These are the specific challenges that I am facing or these are the specific challenges that my program team are facing or These are the specific challenges that I think us either as an institution or as an academy are facing So I think if we if we go into this naively Then we're missing opportunities to make a really good job of it So who wants to give me their first challenge? I'm not going to go around the whole room. You don't need to worry about that Yes To the technology based environment That's what I consider as the biggest challenge in terms of the skill or in terms of the aptitude whatever Resistance to change by the teacher So so I think that I think you're absolutely right and that's been and that's been one of my I think Overriding observations that the biggest challenge No matter what we are seeking to do as an academy and no matter what we are I guess Envisioned by our vice chancellors around in terms of the direction the biggest challenge The observation was made was around our faculty members and either helping them Getting them inculcated into technology enhanced learning their motivation or just I guess they're in some cases lack of interest But needing support in other cases lack of interest That's a really nice example one here. Yeah lack of appreciation of these activities by most faculty members this was given as a suggestion that most people most faculty members because the students are ready to accept these but teachers take a lot more time and This brings down the overall level and makes it difficult for people who are of course they have to work harder and raise the level But that's one of the challenges Okay, so again coming back we're coming back with with staff members They're actually the convincing other staff that it's in their benefit to do it It's not going to overwhelm them and I guess again coming back to offering them some support And so I absolutely think you're right that we need to work harder with staff Maybe just one more and then I'll march on who's pressing to tell me their challenge Oh gosh, gosh, let's have someone we've not heard from is that is that okay? I hope that's okay I'm sorry that I'm sorry that I'm dodging people and I'm sorry that I'm dodging you and going over you So just at the back so that's you sir. Yes turning around. Yeah, the students are still used to blackboard teaching and they find it much More convenient. This is one thing and secondly the elder teachers They find it difficult to use technology so That's in fact a great challenge, which we face actually the students Students they find blackboard teaching still convenient Still still convenient, okay, and did you say older faculty members? Yes. Yes Okay, so let me take both of them I'll be I'll be more ruthless on the second and a little more gentle on the first How about that? Let's let's let's start at the beginning actually I Have a sense that I agree with you that our students In many cases they prefer what they have been exposed to and I suspect Their exposure to date has been much around chalk and talk. I talk you listen. I write you copy I ask you answer so if that's their model of education, they may not know too much Different from that. So I think it's our job as educators to say this is my understanding of good education Here's my evidence for my understanding of good education and I'm going to do things with you as my students I'm going to engage you in ways in which you might feel uncomfortable about But you need to trust me. I've been teaching for 20 30 40 years This is my evidence base and I am here to help you so just as we need to support faculty members I think we need to support our students and telling them why we are doing what we are doing So I think again. It's that that wonderfully secretive nature of higher education We just get on with our business and forgot forget to take people with us The second thing about elder faculty members. I'm not sure I agree. I Have seen some wonderfully engaged members of faculty that are coming back on a part-time basis and Have retired or our retirement age that are utterly engaged in technology enhanced learning And I've also seen some younger members of staff that think that chalk and talk is the best way to educate students So I think it's an eye for me. I think it's forgive me a little bit of an over characterisation I think we need to work with staff that are new to the academy and I think we need to work with staff that are Experienced in the academy, but I don't actually believe it's an age thing. I don't believe that am I am I wrong? Per I'm happy to accept perfectly right that works for me. Thank you. Thank you very much But I but I think it's really useful to air that question. So thank you so I'm so I again forgive me colleagues for profanity for Going over your head. I just thought we'd engage some others I can only start the conversations. I can't finish with them feel free to continue those conversations over lunch, of course Some other things that we might need to do. Here's something that came into my Twitter stream from a conference Someone was reminding of some work from Anisa Ramirez Of course, we need creativity and curiosity and problem-solving By the way, I think they've just defined an engineer there. Haven't they? Yes, they have But their closing point is that we need to make friends with failure And I think that's actually fairly analogous to the question. I was asked around students. We need to be comfortable being uncomfortable and So as we engage in technology enhanced learning it is not without risk and we should expect some failures on the way so we need to expect that and make friends with failure and As an underscore to some of this We need not just to be looking at our smiling and happy graduates that are leaving our institutions and leaving the academy But another tweet that came in from somebody that is of the age of our graduates Said right today's amazing technology is the worst technology Their children will ever see it is their Commodore 64. I Think that's staggering quite frankly and I thought I can't believe that But take your minds back to when the first digital watch came out and how Fascinated we were to press the button and get the lights going just as it was flipping over What wonderful technology that was and now look at what we have on our wrists So the technology we have today folks is going to be so out of date. I Mean YouTube hasn't been around for that long Facebook's just celebrated its 10th birthday Who thought billions of people would be watching somebody dance to Gangnam style that would connect to all of us That's something that no one none of us would have believed I suspect and the potential for those Connectivities with technology enhanced learning we should tap into them So the underscore here folks is that when we talk about technology We should not be talking about the stuff But I think we should be talking about the affordances of technology and possibly thinking about principles and philosophies For me the biggest challenge is actually not motivating innovation The biggest challenge is diffusing the innovation and so I Work hard and no doubt we all should work hard and keep asking ourselves What are we doing or what can we do to help ensure? any innovations that we see or any innovative behaviors Impact more widely within our degree programs within our institution or indeed within the academy The biggest challenge really is that diffusion of innovation and there's a lot written about that activity I'll come back to my first two Questions that I was presented with around teachers Because there is a risk here and the challenges are also Associated with our faculty members too because quite frankly it is not an easy ride for teachers When I joined the academy all I really thought was I needed to Understand my subject Actually, I was quickly disabused of that. I need also to understand what good education looks like So I needed to be an engineering educator as well as an engineer So we do need that subject knowledge and we do need that pedagogic knowledge And when we bolt them together we come up with this notion of the dual professional And this was mentioned yesterday at one of the discussion sessions That's probably good enough or it was good enough 10 or 15 years ago Unfortunately We need to include technical knowledge in that well in that Domain as well, and I think when we bring them together we probably have the ultimate teaching professional I shouldn't say we're going to have the ultimate teaching professional because there's going to be a fourth in a few years time Horizon scan is never been one of my great ventures But I think those things are indeed what we need to be thinking of as we are working and supporting our faculty member Knowing your subject in the 21st century is not good enough You need to be a good educator and you need to understand the affordances of technology Am I able to ask you to do one more thing after today? If I'm able to ask you to do one more thing Refresh your memory of the work of Lee Shorman and Mishra and Kola on their tea pack work. It describes beautifully that technological content and pedagogic knowledge and the different domains that might be helpful to frame Perhaps some professional development for faculty members So if great teachers need to have that tea pack What's left for the rest of us? Well, I think there's ten things we need to do to be expert in to make technology enhanced learning happen I won't read all of them out. You can read those ten things yourself clearly. It's being expert in cell clearly it's having an ability to create a vision and It is really important as well to understand change management The rest of the ten I'll leave behind I've purposely left ten blank because I Wouldn't want to suggest I have all of the answers You have your own tenth by the way, but that allows that list to be constrained beautifully to ten So there are a number of things that we need to do to make sure that tell is pervasive in our institutions And I think some of these are those things and I've already worked through some of that today It's important of course as we're engaging in this agenda to think about the stakeholders and the stakeholders Start right at the prevailing priorities of the government and they come right back through this mind map to students and faculty members And I wouldn't want to trivialize the importance of working with every single member on that chart And of course, it's just a single pathway. There are multiple pathways through So we need to work with our students and faculty We need to have a philosophy around the degree program and the department and the school and the institution and There's a number of things that we need to do in terms of engaging with them Can I ask you to do one more thing? Have I been terribly terribly greedy? I probably have How will you ensure you're learning you've gleaned from yesterday and hopefully from the rest of today Will positively impact on your network and your stakeholders Learning's brilliant and it's very personalized and I've got some stuff. I definitely have with me. I've learned lots in this day But that would be quite single-minded if I didn't take an opportunity to share that for greater benefit So let me encourage you to do the same I'll just close very very briefly with my journey here I know a couple of you know about my journey here But I'll just close briefly with my journey here. I am mindful of the time But just a couple of more minutes if you'll give me that luxury, please professor My journey here started out with a very encouraging and flattering letter from Manjula Ralph and the British Council asking me to come To this event and describing what you are trying to do. I was terribly excited I was flattered and I was expecting and indeed have had much hope out of today. I Needed to do a lot of preparatory work before I came here. I needed to understand what medical needs I would need I needed to get a visa I needed people to work for me to organize my travel arrangements. I engaged other people before I came here. I Got a I got a car ride to the airport I had to say goodbye and leave people behind that quite frankly I didn't want to say goodbye to but I did I Was greeted with some unfortunate news about a plane delay to our delay So I was given some lousy news that just made me to rethink what I was going to do I took the opportunity to do something different while I was waiting Because of the delay. I was stuck at Dubai Airport for too many hours quite frankly That's not that I can remember I was treated not as I would have expected and I was introduced to ways of working That were very different from mine. I Wasn't fed. I wasn't watered. I was standing in a queue for a very long time I took the opportunity to talk with people that I didn't know beforehand and learn some things from them and they kept Me going and kept me motivated and Eventually I got here and was greeted and was rested and were re-energized That was my story how I got here It started with that hope and excitement and it had a number of things that I've described here Ultimately and my penultimate punchline is it is a journey worth doing so again Thank you very much for your very generous invitation and for sharing your learning with me But that's not my punchline folks my punchline genuinely is The journey I've described very rapidly enlisted here is actually the same journey you are on with technology enhanced learning those things that I've put up there Will resonate with you as you think about moving technology enhanced learning on it will be full of excitement You'll be asked to pause you'll meet people along the way. You didn't know you may have to leave people behind I will leave that with you. Thank you very much for your attention and for allowing me to present. Thank you Thank you very much Indeed one of the most amazing talks. I have one page full of notes and I learned a lot during this talk I must admit specially to all of us The brave and bold venture that India is embarking upon is not without challenges and he has listed some and Curiously, I have always known for example that a teacher teaching in the normal style does not necessarily amount to learning by students But I did not realize in some simple words what I've been doing for 40 years. I Talk you listen. I write you copy. I ask you answer. I Think that describes the current teaching wonderful professor Russell. Let's give you a big hand for his beautiful It is my pleasure to invite Professor Ashok Junjinnwala from IIT Madras to deliver his talk again Those of you who are from India, of course know about him, but for my other friends. I would just mention that Like many of us, he has been an accomplished teacher But he is better known for his extraordinary penchant for innovation both in technology and teaching Recognizing his contributions Many many years ago government of India conferred on him the fourth highest civilian award of Padmasthi and that tells everything He has been recently engaged in defining a model for quality enhancement in Colleges other than IITs and NITs and that's a model which will roll out in some 500 colleges beginning with some hundred colleges It's a very exciting opportunity for all of us to participate in that venture and of course most of these colleges which are necessarily engineering colleges because that is what we all look at Belong to several state technical universities and that is where we believe that all of us would benefit from his exposition So, Professor Junjinnwala all yours First of all To understand the state of education in India Some very very simple questions Of course, most of the audience here is probably very knowledgeable But I'll ask start by asking a question Do you know what percentage of Indian children when they attain age of six today go to school? Can someone tell me? I know the ministry officials will know but Can someone can guess what percentage of Indian children when they attain the age of six enter school? Okay, so that's good. So the I am Talking to a very educated audience because if you go and ask in anywhere in any place, they will say 60 some will say 50 and the reason is that I Somewhere in 80s the number was closer to 40 So from there we have very quickly expanded and today it is believed to be 96 percent There is varying between 96 percent and 97 percent Now this is not just in school education Take engineering education when I joined IIT Madras in 1981 As a young faculty not so parents used to bring their children to me and so I say I want my child to become an engineer See my daughter is so bright. She would really Benefit by getting into engineering But she just cannot get in And I examined the number of seats that we had for engineering where 20,000 seats per year and Today it is 1.7 million Not all of them are getting filled And it is not about the school education or engineering education You can ask for look for MBA or look for pharmacy look for practically any professional course other than medical and you see huge huge Expansion has taken place in terms of quantity so the point that I am trying to make is that one thing that we have achieved in India is quantity Now that's not the only thing But I think it's a major thing to to have because where we used to deny most of the children education Opportunity at least we're able to get them to now schools and colleges and Along with such large Expansion in quantity comes equity today. I mean it is very common practice In fact, I don't have the actual number somebody from MHRD should sometime figure out the number But it's estimated that about 25% of children in engineering schools are come from below poverty line and 25% from rural areas So the second major gain has been in terms of equity So we seem to have get got two major gain. We don't talk about it We don't celebrate it any gain should be celebrated We often don't talk about we actually only talk about our short coming and there is very serious short coming of course the short coming is in terms of quality and the quality has gone Particularly in fast expansion the quality further deteriorates and we have a serious problem of quality Whether it's engineering education or whether it's a school education Everywhere there is a serious quality problem problem is that we in the last 10-15 years when you are growing 50 times or 100 times you can't even have so many teachers So you have just brought somebody and made them teachers not even necessarily So the we earlier we had a very very exclusive education where there were some very good teachers and some limited number of Good students and we used to call celebrate that as a quality, but frankly speaking. That's really not a celebration talk So I think the key issue for us going beyond All the different things is that how do we fix the quality and we see ICT is a important tool to fix That's what it is. We see that as an important tool We are not saying it's the end and all it's an important tool in our pursuit of quality So lack of quality is what we are really trying to overcome and seeing what extent ICT Will help us we do not know the answer But I think enough has been done everywhere to kind of believe that yes somewhere. It is important We don't have the clear answer That what works what does not work when he talks about how does a last person or everyone really get benefit? We don't even have answers but I'm sort of saying that if the quality is here can we start improving it And I think it is in this context there was a committee that was set up The committee and there has been of course a lot of things going on which I think all of you know so I'll not talk about it So in India itself there has been number of efforts I mean for example NPTEL was a great example large number of IIT lectures being put together in top class lectures We went around various NITs to talk to students What how do have they how do they view NPTEL? and we were invariably found found that Top two to three percent of the students have really liked NPTEL and have leveraged The remaining to a very limited extent Occasionally they have used it most of them have not used it So we have got some very good which Motivated students highly motivated students can find ways and we have created content for like NPTEL content Lot number of other experiments have been there uptick has been less than desirable and As a result all the attempts that we have been making has had only limited success It is in this context that MHR is set up a committee I have happened to be the chairman of a committee to sort of say Take 500 engineering colleges other than IITs and NITs and fix quality Do whatever it is And No, we don't expect it in a year, but have a 10-year program Fix quality and if we can do it for 500 engineering college, maybe we can expand it to 2000 Maybe we there will be learning From there for other disciplines. Maybe they're not be learning. I'm not saying everything will be usable and Do sufficient experimentation my MHRD additional secretary actually taught me a word called action research But go and try Get things tried out continuously take feedback See what works. What does work? If it's something works to a limited extent, that's good then see how Rest can be actually taken out and that is what the committee has been trying to do In fact several people out here are members of the committee and the committee has been for last Nine to ten months has been working to try to figure out what can be done We clearly recognize that we need to do three different things One is can we do or something direct to student where can we directly get some programs to the student Like where say a MOOCs does it But let's let's try to figure out what can be taken directly to the students But we very clearly recognize that unlike some people who believe oh now the days of Universities are over and you just can directly teach students believe that can have a limited impact at best The rest has to be done. The rest has to be done elsewhere. So we sort of said well strengthen the teachers strengthen the existing teacher And bring in quality teachers for example a very very simple thing that we kind of try to figure out is that is the top 15 percent or top 20 percent of Students in engineering graduating students in engineering taking up teaching career We're asking her whether the teachers do they belong to the top 15 to 20 percent if they belong to top 15 to 20 percent they basically Irrespective of where they are coming from they basically have the quality and We haven't done any great survey, but even a limited extent that we found no no longer that is so the top 35 40 percent no longer If top 35 40 percent do not come to the teaching there is a serious problem Now I remember this issue used to be raised in our early part of this century that well part of the reason Why this is not done is because the teaching is a very poor career doesn't pay enough And in India therefore one of the major thing that was done in the last pay commission was very significant increase in salaries for the teachers And I know it took four years And I remember as a member of Prime Minister's scientific advisory committee We actually spent an hour and half with the Prime Minister on this particular issue We say you cannot fix it unless you are able to attract high quality teachers and for that you need to fix and They say okay for teachers. I'll have a special thing And in fact the I think while one can still continue to complain and There is no amount of money can be enough, but certainly it is not as unattractive I know remember that my own salary almost overnight increased by four times As a faculty member with shock actually when it came Very pleasantly in fact I mean people like me don't even know what to do with that but that's a different issue but Many of youngsters say no, no you have settled for too little but maybe we come from different generation but can we get the top 15 percent of the students to become teachers and Can we take the existing teachers and strengthen them and I excellent I'm not going to talk much about it because there is excellent work. There is the being done by IID Bombay On training the teachers and we found that they have actually gone far ahead Then pretty much anywhere thing else that we have seen so pretty much I'll say that's a model. We may fine tune it. We may continue to get feedback That's that's a model that we do the third important thing. We sort of say we need to do is really look at the whole governance issue the governance issue regulations the issue of Autonomy rating And unless you fix that and even make the economic model work Because if you don't make the economic model work One of the things that was pointed out by some people sort of saying that well you can't charge the students enough fee and Can't pay the teachers enough money. So if that happens then Unless money comes from somewhere else The whole thing will falter or the student teacher ratio will go Very so can what can you actually do? So these are three buckets in which we are working What I'm going to present to you is very very briefly the first bucket direct to student What did we look at in terms of direct to student the idea is? we sort of said well We will try everything that is available We'll try try everything take hundred colleges to begin with and then it increase you to 500 colleges and try out See then what works? What doesn't work? See what? This positive results what does not what enthused students what enthused does not and we saw said we will try life classes Life classes we sort of said they will put the best teachers in front of the very best teachers So we went to the four IITs IIT Madras Bombay Delhi and Kanpur and we asked the director to give us the teacher rating of their Of last three years. We found that the some teachers were consistently in the top five or ten Within IITs for three consecutive years. So you sort of say pick them up Get them to try to teach Because let's say with the best teachers does it really have a good impact And the life classes is actually a life class In any subject we define the subject and we saw in this subject There is a life class going on the students instead of for local teacher teaching it is now being taught by the remotely Simultaneously teaching to ten thousand students almost not right now. It is being done. Well, yeah It is six seven thousand students six seven thousand students. There is some interaction limited interaction You know we use all the other technology things and we try to actually do that Fortunately, I'll tell you a little bit more about it. We've got we're getting very very good response Because what the students actually keep on telling us that the top quality teacher We do not see a top quality teacher Most of the students mean many of the engineering college say I go through the whole four years and do not see one teacher Which is a top quality not just in top quality in teaching but also somebody who can inspire So we are looking at somebody who can inspire besides being the top quality teacher and can they do life But we're right in the beginning. So I said well that does not mean that we will try to disempower the local teacher So we said only one third of the course is going to be taught by this life Two-third will be taught by the local teacher Maybe the local team and the local teacher has to also attend the class and it will be in a classroom It will not be in your homes, huh? So in a normal discipline manner the local teacher themselves can gain out of this And this has been one of the thing that is going on and this is the only thing that has actually gone on for now About one month little more than a month and I've seen very very positive results, but again very early So I don't want to sort of say that yes Everything is working great There is also tutorials. We are the second thing that we learned at least at IIT when we were our students And now when we are teaching that what this major part of learning also takes place in tutorials The tutorial sheet is given and a small group 20 30 students is held by a tutor in solving the problem and we Recreated the same situation except a virtual tutorial. We get some of the best students to now try to train Remotely connect simultaneously to 30. They're sitting in a class and The tutorial problems are getting solved the third thing that we do this virtual labs There are already a lot of work going on actually NPTEL program had actually created a lot of things We started sourcing. How can we leverage some of these things? So Can we create these remote labs? Where the equipment may be sitting elsewhere But you get fairly good front end and all that and you are able to connect to the labs and carry out the experiment The fourth we sort of say will try out all kinds of MOOCs that are available We were sort of said well MIT has top quality MOOCs So one of the course from there we picked up and say let's students register for it Then there was another MOOCs that was done by Microsoft research which was actually implemented in Karnataka colleges and got very positive responses I will try that out and in the third MOOCs that the NPTEL Team along with NASCOM is trying to bring it. We say whenever it is ready. We'll try that out Some of them it will work some of that will not work and to what extent each will work will figure out and then we sort of say We'll have bridge courses. What are the bridge courses bridge courses are non-curriculum courses Which are essentially useful to the students when they go to the industry and we'll say we'll introduce some bridge courses So this is what we try to we are in fact actually try to do the bridge courses along with British council we had actually decided one of the English learning course But unfortunately the British council came and sort of said yes, they're agreed to everything But they cannot let the content be put on our server and it says well the kind of bandwidth etc that we have We are all NK and connected. There is no way we can manage it without that hence therefore We had to drop it, but we are doing some other things. So today the live classes are going on with 100 Colleges and tutorial as I pointed out virtual last MOOC and some bridge courses and And we are using leveraging NK great network has some problems We need to fix that We need to fix that we need to also have been I've been telling them that we also need a High-quality broadcast at least one way the second way can actually continue to come So that also we have been talking about and we have today classes going on We have professor Jagannathan from IID Kanpur teaching in Monday every Monday eight to nine nine to ten is being professor Neela Nataraj IIT Bombay they are all different courses and two different classes linear algebra and Heat transfer course is being taught live from IIT Madras and this we are currently doing putting the best teachers first time in front of for the students For teachers Initially, we had to a little bit push them today. They are really thrilled Say I'm teaching seven thousand students. It's a different experience and they say enough questions are asked Yes, this is the beginning. We like to do more. So enough courses are asked and So that that's what and this is one of the Aditya Jagannathan. Is that correct? Aditya Jagannathan anyone remembers him I D Kanpur. I think he's from Aditya Jagannathan This is another faculty member. I think is Balaji from what I remember now One of the first feedback that we got very early to get feedback Here's the students like the courses where they were to start on blackboard the most and we have difficulty initially on the blackboard You did not get the same quality and Then Kamal he's here. I saw him. He helped us fix this kind of thing And today we are actually able to get as good a quality on blackboard. They just feel that it's more interactive Now I'm therefore not saying that's the only way there are multiple ways But different teachers are encouraged to try different things Same course being taught by this I won't say that we have the best as he pointed out that 20 years down the line We will sort of say we are working with Commodore 64 exactly. That's the way we feel but that's fine But we will start with that. We'll keep on progressing today We have two screens in every of these hundred colleges Broadcasted one one is the faculty member another is the blackboard or the content and there is live interaction There is a complete framework where all the colleges which are logged on are known and the you can raise hands and Do all those things has actually been done and fortunately there are enough people who have worked on technology I mean that I mean I'll sort of say that that kind of thing has actually happened and This is another course where students are kind of sitting sometimes somewhat sleeping and we are watching closely what is happening That's how that shows us What what are the early feedback? They are very early feedback overall transmission and technical content are really nice and helpful to students as well as the faculty members Overall audio video quality is excellent. We are opting for the 12 live classes and have registered 700 students It's experience is excellent audio and video quality is good without any lag students are very interested to attend the classes Fundamentals are taken or this is but there are some negative points We have been repeatedly told that noiseless eco cans less interactions should be actually done Occasionally we do find echoes and things like that. I think these are early days is the first time this being tried out There are what I have not put out here There are glitches that we occasionally find in NKM a minute two minute glitches are very off common three days back the NKM Suddenly for one hour was totally flooded And created a problem remember that we are broadcasting to hundred colleges We need to close it to 40 megabit per second bandwidth Dedicated bandwidth and became a big problem. There are variations in syllabus offered at IIT compared to the university, but Basics are anyway covered Then we went into live tutorials as I pointed out tutorials is actually happening where a Tutor is connected to the students. They can put whatever material they can put video material written material question answers and That's another thing our we have started only about a week back ten days back our Feedback yet was from tutorials is not very strong. It's something that we really need to worry about Then we have these labs for example, we have this DSP cloud that was developed at my place By the way, what we decided is the best way to do it even at IIT Madras the regular labs are done But they are also done in the same mode now. They are no longer done by students going and connecting to each board They are just logging on to that. They use the same e-books. They use the same kind of because every lab There is a background theory that is faculty member. Go ahead. There is a video by a faculty member. There is a for every Experiment what is the objective of the experiment is described by a graduate student by earn a video students read and we have started to try that and I'll sort of say it's very early. I Don't expect everything to work. I expect lots of problems But my feeling is that we should between what we are trying to do what IIT Bombay has already done for a year and a half Two years probably longer period of time some others are doing IIT Kharagpur, etc. I think we just need to learn I think two three years we will be able to start making some impact already making some impact I'm not trying to say and I my own feeling is India being a peculiar problem where we have 1.5 million engineering students Most of them you go beyond the top 50s colleges the quality becomes very poor We really need to fix the quality and I think we will certainly gain will continue to learn from whatever else happens But one thing that I want to want since there are people from outside Often we have heard people coming and sort of saying Google comes and presents. We have a solution. This is going to solve all the problems Sometimes somebody from MIT also will come We know exactly what has to be done and unfortunately what we very quickly find that don't know Things may work in certain environment in certain situation I think we need to be very very careful that we will probably see more failures than success and when we recognize and accept That 75% of things may fail 80% of things may fail, but the 20% that will succeed We will multiply that and keep on gaining. We will gain. Thank you very much


So welcome everyone, this year we will be having a talk about test levels and we will be also talking about some type of creators along the way like doppelgangers. For this talk I will do it with Frank, who is a former student of the Paul Sabaté University in Toulouse and he contributed to Maouk and K-people and now nowadays he is one of the main contributors on Zenshin, which as you know is kind of my pet project, and is the resident software classman at K-Lab. As for me, I'm Kevin Notence, I'm also a former student of the Paul Sabaté University and I keep teaching there part-time. I've been working for the KD community since 2003, so third of my life, and I got a PhD in artificial intelligence in 2007. I'm also a resident software classman and a trainer at K-Lab. So as you know for a few years now we've been banging the drum of TTD, so test-reven development, but really when you say that it's just a label, right, that's like a brand, we're selling soap or something. Let's still ask the question of how you write those tests properly, and there are several challenges with that, one of them and the main one being really how you insulate parts of your design during the test, and so we need solutions for that, those solutions, that's what we call test-dubbles. The name you have to think about movies, you know, we've stunt-dubbles where you have one person looking like the real actor who actually takes his place during a stunt was the same here, but that will be the double of a class which will take its place during a test. There are several types of test-dubbles which exist and that's what we are going to explore in this talk. Before that we'll have a definition which I expected from Wikipedia on what they are, so let's say it's automated unit testing, it may be necessary to use objects or procedure that look and behave like they're really intended counterparts but are actually simplified versions that reduce the complexity and facilitate testing. The emphasis is my own in that definition, I think that's really the three main characteristics of test-dubbles, they have to look like the original, they have to behave almost like the original, and then it's a question of complexity inside of your test which will guide you into picking one of the different types of test-dubbles. There's roughly four types of test-dubbles, it's kind of a continuum so you can't really categorize them crudely. On the frontiers it's kind of fusive and you get from one to the other type but that's mostly so dummies, thubs, mocks and fakes. You can sort them on a complexity axis because you generally pick one depending on the level of complexity you have in your test and they are basically tool to move that complexity outside of your test inside of the test-dubble so it has to match in some way and so all the questions for picking that which amounts of complexity I want to move from my test, which design aspect I want to validate with my test is that just verifying the state of my code under test or its behavior. And so in the title we've been talking about doppelganger so we still have to carry on with that. So what are doppelgangers? In fiction and folklore doppelganger is a lookalike or a doppel of a living person sometimes portrayed as a paranormal phenomenon. Well test-dubbles are deep like that, right? They are someone doppelgangers of a living class instead of being doppelgangers of a living person and they can be of different types of paranormal phenomenon depending on the complexity you have to deal with. So now we are going to see the different categories. First one being the name is so that the simplest form of test-dubble you can have in a test anything that basically is an idea where anything goes, right? You will pass something it's totally passive and opaque and no one really uses it that's more to satisfy some interface. So if we look at our paranormal phenomenon that would be something like the voodoo doll, right? You have something which is totally passive you can just pin it, you can torture it because you can't really torture the original and that won't really have any consequence, right? Well except if you have some beliefs and so on. So the definition of a dummy that would be this one, dummies are used when a parameter is needed for the testing method or constructor and so on but without actually needing to use the parameter, okay? Maybe when you have your test running, okay, you pass a parameter but you know that that particular method in that particular case is not going to use it. Then you just pass some dummy value instead and something which will be opaque and not really reactive or passive. That could be a new pointer or something equivalent like a dummy identifier because you know it won't be used. So I've been talking quite a bit already and so I will leave the mic to Frank so that he can show you an example of dummy. Okay, so can you hear me well? Okay, so I need the pointer. Okay, so I'm not going to show Zenshin but you know it's a to-do application to keep it simple. So we're going to mostly look at code and test code basically which is always having an explicit name and a structure with a given which is basically an initialization when which is the test and then the test checking making sure everything went right. Okay, so in order to see the dummy we're going to see the should add task because we're managing tasks in Zenshin. So we have a given part to initialize stuff. In the test we're going to basically add a task here you can see. Just look at the selection I'm doing. Just don't try to reach to read every line. It's not going to be helpful. So in order to do so we need to initialize an entity and here you can see we pass a dummy because in order to create a task we don't need a note entity. It's totally unrelated and will not be used. It will be passive during the test. So instead of passing a real object what we do here is we create a dummy as Kevin said and basically it's a new pointer, a Qshad pointer but initialize as null. So this example is quite trivial and after it will be way more complex and useful. So that's our first case. So all dummies they have pros and cons, right? So the good thing about dummies is extremely simple to use, right? As we've seen well basically you've got a pointer. That's it. And you probably already do that without even knowing. And that's a mean for in your test you avoid creating a lot of collaborator objects for nothing, right? Because you know they won't be used. On the other hand they just cut dependencies and that's it, right? They won't help you to enforce anything or any type of checking in your test and your code under test has to be able to cope with the fact that there is a dummy, right? It has to not die just because you passed a new pointer so you need to be ready for that.  So that's why we have more complex cases like stubs and that's because now you're in a case where for you see with the dummy you basically just ignore it, right? You just pass it around and that's it. Now you need something that you can beat, okay? That you can beat and which will react always in the same way every time you pocket it, right? So that's a bit like a zombie, right? You can beat the hell out of it and it will always carry on asking for more brains, right? That's the only thing it does. It's totally predictable in its behavior and that's what we're after here. So stubs are used for providing the tested code with indirect input because very often in your tested code you will call a method and you pass parameters to that method, okay? So that's a direct input for your code under test. But generally when you start to have a system, well some of the input you pass directly and some of the input of the code under test will be coming from some other collaborator object, okay? That would be a stub and that's a way for you to inject values inside of your code under test if that's not a direct parameter of what you're manipulating. The simplest of stubs is basically you have something which returns always the same value that can be a bit more refined. You could have something which returns value a then b then c always in the same way and predictable order. Sometimes I might record b2 then we get slightly into the mocking territory that we see later on and they are meant for state checking, okay? The idea is that you create your object which will be your code under test, you create your stubs, link everyone together, you manipulate the code under test and then you check the state of that code, the state of the different stubs and you declare that was fine or not fine. Okay, to make that a bit more concrete we'll have another example with friend. Okay, so we're going to look at this piece of code so again just follow the selection, otherwise you're going to have a headache. Okay, so the then she is using an editor basically when you select a task you can have the detail of a task like the title, the description and some test, some start date, is it done and everything. So the editor is basically a widget and we use a model so here we're going to try to test the editor but we're not going to use a real model so we're going to stub it, that's why you can see here that the editor model is stub. In order to do so what we want is to set it up so the test is about verifying that the task which is delegated to someone will display a certain label, well some text basically. So, get back to the stub. In the initialization what we do is create the stub and we set it up the way we want so basically we made the task being available we set up the task that will need to be displayed later and after we will really do the test by putting, setting up the model into the editor and we want to make sure after that that the text is displayed so it's visible here the text is visible here. So what we do basically as Kevin said is we set up a really predictable object a counterpart of a real object which is the editor model stub we set it up with this value, we do the right test here we pass basically the stub, not the real object and back there we make sure that everything went well and it is because it's a really simple version of an existing entity because you made it predictable he just said the indirect output Kevin was speaking about which is the state that's all. So all stubs they will allow us to factor out some of the test logic and they will help us to make the setup and state checking of all tests more reliable in general. That's something here we use our own class for stubs, okay that's something you could do by using a mocking tool as well if you use it partially, but you're basically you turn to a mocking tool instead of making your own class you're basically trading readability with the code size. If you use a mocking tool you have less code to write to create your stub, but then there's much more setup appearing again, so you kind of lose in readability with your own class that means that you will have to maintain this stub class if one of your interface changes for instance but then that's generally way more readable the problem you might have with the stubs is the fact that you have no clue how you got to the result because the only thing we know when we use stub is that we queries the code under test to check okay what your state now and we will verify that the state we expected we have no idea how that code under test got to the conclusion of that particular state which is sometimes what you might need. If that's what you need then we want you a real mox. That's when you need something which starts to be really more clever and basically that's more for behavior. So we want something which will confront you every time you have a wrong behavior, so that's a bit like the Frankenstein creator it basically will pursue you until the end of the world for you to realize that you've behaved wrongly in the past right something went wrong in your test. So mox are used for verifying the indirect output and not input anymore the indirect output of the tested code. With the stub we basically injected value inside or for code under test with mox we are looking at what the code under test is pushing out toward collaborator objects so that's different direction of the data that we are checking and so we are checking both the actual value and what got called on the collaborator objects. That's why for stuff we don't know how we got to the conclusion why for mox we know because we can examine which method got called exactly. Know that in the literature there's two types of mox. There's what's called mox. That's the one where you define the expectations of the behavior of your code under test before you run the test. There's files where you set the expectations after the execution of the tested code. Nowadays these distinctions tend to disappear and people just talk about mox. That's kind more of a technicality which is not necessarily very relevant nowadays. And because of that that's clearly for behavior checking and before you fall asleep we'll have yet another example with mox. I'm going to try to take a bit of time because it's more verbal than we're using a framework which basically you need to understand a little bit before starting. I'm going to grow up as well a little bit. Okay so this test is about creating a new item. New item into the engine. In order to do so we use the only backend of the engine is icon ID. So if you see icon ID all around it's normal. Okay so in order to do so you look at the one section what happened we try to create a task basically and in order to do that this whole bunch of initialization I'll go through right after. And after that there is some checking like always. Okay so let's have a look at the initialization part. So the point is to create an item and in order to do so we need to initialize the repository with a bunch of objects and we're not going to pass real object. As we said we're going to pass mox because we want to make some more in-depth testing than with the steps before.  Okay so I'll be mocking the engine. We use moquito framework. So basically let's have a look at the storage moch. Storage moch is basically declared here using moquito functions. And right after that what we can define is how it's going to be, what's the impute, what's the state of the moch. So here what this line is meaning, the one I'm going to try to select is when you call the default task collection on the moch which is mocking the interface, when you call it the default task collection function with this parameter which is void basically, then it returns this which is something you just prepared right before. So you set up a state here. You do the same with another function to create item when you pass with this parameter and this parameter it's supposed to give back this which is made before like again everything is fake, everything is mocked. So you just have a lot of control against the entity your test is picking with basically.  Okay that's fair enough. So when you arrive here when you make the test to create task this is going to call basically all this method with just moch. And the point of the framework is to crash it and just replace with what you pushed before. So it's going to be really predictable as a step and everything is going fine. Okay so now what's the difference with the step we were speaking before is if you look at the Zen section what we're going to do is we're going to make sure it behaves as we wanted which means that we want this function, okay this one is the we are speaking about the storage so let's keep it. Okay so we make sure that this function is called with this parameter exactly one. So we're making sure of the code pass under the test you know like how it not only just the result how it has behave internally and same with create item we make sure that it called with this item exactly and with this collection exactly, exactly one. So that's more in-depth testing that's why it's more complex and study and more verbose. Okay so what's nice is that we can completely simulate all the collaborator with our code under test and that's also a way for us to verify hidden protocols. Okay because then we start to see protocols appearing between our collaborator objects and so we make sure that we make a certain number of calls of that particular method behind the scene with that those particular parameters. So that allows to get the code under test in a vacuum. Okay it's completely in a simulated environment which pretends to be the real thing. But as we just seen it's generally very verbose to set up right because you have to declare every method where you expect a call with particular value and then you have to do all the checks to say okay that's the number of time was expected to be called with those parameters and so on. The other big problem here is that it's making refactoring more difficult later on because by doing that you're coupling your test, the code of your test with the actual implementation of the class right. Okay so you later down the line you will have more problem okay you will be less able to react to change because then you have to adjust all the tests just because you have an internal behavior would change which maybe led you to the same result okay but just it's done different reason your test will fail. Okay so sometimes we might be in a situation where mocks or steps are not enough. Generally the sign of that that's when you start to have not one mock in your test but two mocks, three mocks, four mocks okay then you start to have something like 40 lines of code just to prepare all the states for your mocks and all the checking well basically what you have at that point is you're trying to simulate a subsystem, a complete subsystem and not just a few collaborator objects so in that case if you want to simulate a subsystem you need to create a fake one and so that's especially interesting when you have big external dependencies and you wish to cut them for the time of the test so you need something almost exactly like the original system but not quite. So that's where we get the real doppelganger right that's the clone so you almost can't tell apart which one is real and it's basically running at you trying to steal your life. So fakes are used as simple implementation of a system so for instance in-memory database so instead of the real dependency which could be for instance a database you plug something else which will be equivalent feature wise but you lose something else which is not relevant for your test. So in-memory databases in my opinion are a good example here in that definition because I find that fakes that's very often related to storage because if you're having something which end up on database and you have to write plenty of tests just because of the runtime of your tests with the database that would be way too expensive and you want to cut on that and having it in-memory would just go faster. So that's very often linked to that so storage would go to the system you're running on. So that's why on Libsolid which we brought like years ago there's a fake back-end which allows you if you're a user of Libsolid to run your test on a virtual machine right Libsolid will pretend being on the machine with some particular devices that's how you can simulate that. So if you're in pain of having me to talk so now we're having our last example with Frank so enjoy it. Okay, so the nice thing is it's fresh new fake just being done like last week or two weeks ago. So we will, not all tests have been ported so we will be able to compare this one with the other tests right underneath. Okay, so the keep looking at the selection. The test is about reporting all tasks so basically having them all if you look at how we do that we just have a task query entity which is basically doing a find all to retrieve them all and we're going to have a look at the initialization which is done through a fake because in order to initialize the query first we pass a lot of things. Before here it was mock I don't know if you remember and here now it's some fake entity with like you can see data is a fake and create storage is a fake as well. So let's go to the initialization part. Okay, so the whole point here is just to see that the call is way simpler than before we just initialize a fake create some data okay here create two top level collection create a task create, well just read the comment and you will understand it's quite easier to read there is not all this framework, mock it or mocking stuff that you which is way more verbose so it's basically having like the real code the thing is it's not like the real entity but it's really well imitating it okay so if you compare with this initialization with the one right under it which is more or less same kind of idea well you can see that here you have way more initialization when you initialize the objective in a really different manner you still set the mock which is very verbose as well you set each function of the mock so your code gets way more simpler counter part of that is when you go into the akonadi fake data you can see it's full of functions that need to be implemented to mimic the real entity like akonadi storage you're faking which is complex. Okay, so just to give you an idea about that because we see that there's a small blob of code for the initialization with the fake and the big one with the mock I started to port more of the test to use the fake because that's the most recent stuff we have and very often that stuff which was something like 40 to 60 line of codes to initialize all the mock so that they are consistent between themselves as well compressing something like half a dozen or maybe 10 line of code and done. So that's the main point of the fake it will simplify your test code quite a lot and that's because also your closer to actual production code just initialize a state in the fake and then all the rest is at the same usual code you would have and often then the fake is way faster than the original especially if they are storage but that means potentially you have a lot of code to write and maintain because that means you have to create a fake with the relevant features so you have to write the test code for your fake and then you can use that in the test so that's potentially quite some code for the fake the in-numory icon that we have right now in the engine that 2,600 line of codes of course that's something you end up doing if the technology you're using has no ready-made equivalent which was the case for something like a code but if you use database just swap I don't know might equal for something in memory and done so in those cases that's much less work okay so we've seen several types of test doubles and now you might be wondering okay which one do we pick is there's one we should pick in every cases where we've seen that no there's no silver bullet because they all have pros and cons and none of them are ideal so instead we will leave you with kind of a proposed approach and that's broad guideline that we are not using when we have to deal with test double and of course that's a proposal and your mileage might vary so our opinion is that well you should use the means whenever you can okay that won't hurt just make sure you're not killing your code under test by adding exceptions everywhere like verifying okay that's a new pointer I have to deal with it but if you feel that's natural for your class I mean that's easy to actually deal with a new pointer or something just go for it and then use dummies so whenever you can but in practice they have a limited use so that whenever you can won't be that often so otherwise if you can't go for a dummy release at the stub that you should prefer in most cases 90% of the cases the most common testable you should have in your test code at the end of the day that's just small helper class that you really use throughout your tests right so that's reusable code between your tests that's generally a good investment because that's cheap enough and then you get quite an impact if really you want to spare one then you can use the mocking tool to create a stub as we've seen you just do the state creation you don't do the checking at the end that's basically using mock tool to make a stub but then you have to be very careful because it becomes very easy to unwillingly move away from that's just a stub to that becoming a mock what matters here that's not the actual tool that's the intent of the code you're writing as for mocks you have to use them temporarily so they're really great when you're guiding the design the early design phase of some part of your code that's what I call last year emergent design because you start working on one class which needs collaborators which don't exist yet right so you create the mock and as you make the mock more complete to be able to make your test you're basically in the mock what you're writing that's the beginning of the specification of the next class you will have to implement right so that guides you doing that before you have the actual class so that's very nice for that but beware of the verbosity and coupling to implementation you provide so that's why you should do that so that you get to the point of that specification and then throw them away when you can then you just remove them and favor or stub or build objects whenever possible as for fakes there clearly are an investment so that's stuff to keep for the frontier of your system your external dependencies because they might get very expensive if you don't have a ready made solution as we've just discussed before you jump on them only when you actually settled on those dependencies if you change very often then that's wasted time and that's most useful for integration tests or acceptance tests which are full task testing where you want to speed them up and simplify and that's it for us thank you for your attention and I'm not sure we have the time for questions thank you we will have the group photo so one question and then we go out towards Munch following the balloons and at some point we will be stopped and wait for a second to smile and then have lunch so everybody please after the question go on and follow the balloons and take a quick picture so one single question and then we need to be on our way we're not inherent from anything so actually this doesn't work with better methods does this mean this works with templates instead? yeah that might not be clear when you see on easy using site of the code but I can only affect data as in two methods I can only affect data that's like having the naked data that you can manipulate and so on so that just wrap around ashes and stuff like that to set your data okay so it doesn't really need to inherit from anything but then it has two it has two methods one which is create storage and one which is create monitor okay and that's the one where we return an object with the right interface where it's all virtual and everything okay and then in that one the implementation talk to the data that you just set in the effect data so grab them on the way to lunch after lunch I'm sure thank you so much you short of memories