We are at the top of the hour and this is the Future Transform welcome everybody. My name is Brian Alexander I'll be your host and guide to the next hour I'm the creator of the Future Transform and I'll be happy to introduce it I'm absolutely delighted to welcome Justin Reich Justin is a professor at MIT Where and he is the author of a really really important new book failure to disrupt a very very powerful book about Education and technology and that's gonna be the subject of our conversation today If you'd like to learn more about the book if you don't have a copy if you like to grab one directly Look at the bottom left corner of the screen. You'll see a kind of tan colored button That says failure to disrupt press that take you to the page and you can grab a copy And now let me bring Justin up on stage Welcome sir Very well, it's good to see you Thanks for having me Brian. Well a pleasure and I'm glad to see your beard is growing every day. Yep That's right. It's a key part of the forum. We're Openly biased towards beards. That's an important thing Where are you coming to us from today? Are you in Cambridge? No, I'm in Barnard, Vermont everybody who does MIT tried to de-densify very early on like everywhere else and so the people who are on campus do things with fume hoods and nuclear Reactors and wind towels and those of us who just do stuff with computers were told to bug off when we did So I'm actually in Barnard, Vermont, which is a town of about 800 people right in the middle of the state. That's a big town How how much snow is on the ground? finally zero We had our We had our thesis day maybe a week ago Friday and it was the last big storm of the season with about a foot of snow So I just turned zoom on my phone and hopped on in the mountain and skied down and listened to graduate student thesis presentations Everyone's in a while. I have to ski to the side to safely have to ask a question and then That is Vermont story On the internet, no one knows you're a skier Did you get any good selfies? I didn't maintain a very low profile see that'd be your next book check at cover the You there are a lot of ways that people in academia can introduce themselves and the way we do it here in the forum is We ask you to say what you're going to be working on for the next year So what are you going to be writing? What are you going to be researching? What are you going to be teaching? And what are you thinking about? There are a lot of answers to those questions because I'm lucky to work with the big team interdisciplinary team of cool folks But the main thing we're working on Is that as we've looked at the sort of instructional design palette that online educators have We feel like there's a real missing piece around simulation and practice This is particularly acute in teacher education, which is my main area of interest When teachers learn they listen to people talk about teaching and they talk with each other about teaching But they very rarely do teaching Um, so we built a tool called teacher moments Which you can look at at teacher moments dot mit dot edu And uh, it's a digital clinical simulation platform that immerses people in vignettes of classroom life Through text audio and video And then periodically a microphone icon pops up or a text box pops up And you have to say what you would say to a person in that moment and make some difficult decision teaching And there's all kinds of data capture that lets us, you know Learn about how people are authoring scenarios how they're participating in the platform We just added some functionality out of sort of intelligent coaching agents come in So it was sort of our effort to say okay Here was here was what we think was one missing building block in the instructional design palette that we have Let's see if we can try to to build something that fills that in And make it available, you know under an open source license So a big part of what we're doing now is that we have a community of practice It may be a hundred people in various places that have started to to adopt this tool And we're trying to figure out how to how to support them and have that to grow in scale It's fantastic. I hope everyone gets a chance to play with this And rocks and thank you for uh, I'll put that in the link. Do you uh, what's behind the All the content. Do you have an algorithm that just randomly generates this or do you have a kind of imperative structure It's all authored. Um, so the so the back end of teacher moments. It's like a combination between google slides and qualtrics um, in in fact You know, we might not have built it Except neither of those platforms has really good audio input That turns out to be like the killer app or the killer feature You know what what we want is you're you're in some scenario And a kid calls you a racist and we want the microphone icon to pop up And we want you to say not we don't want you to scribe what you would do in that circumstance We want you to actually use the words that you would say and have them recorded and hear them back We have we have some colleagues um in neuroscience medicine the a yell professor who runs the national neuroscience education center and uh, apparently there are some Drugs that can be prescribed for opioid abuse which are dramatically under prescribed It's the same kind of thing like don't tell us what you would say the exact words that you would say to a patient suffering from opioid Disorder when you're going to suggest to this patient that you're going to prescribe this particular medication that has the stigmas associated with it and stuff like that but anyway, the I I got the rail from your question, which is You know the way the content appears Is that we make partnerships with folks? One of our partnerships is with 15 computer science teacher educators across the country people who are starting new programs to help Middle and high school teachers be licensed as computer science teachers And they think to themselves like what are the situations that I want my students to be able to practice in a simulation before they practice them in real life And then the tools are very very simple to author And so You know there's a bunch of there are a bunch of these scenarios that we make but mostly people make scenarios for their own local context And you know a big part of some of this will connect into maybe some stuff we talk about later There are a whole bunch of people who who've built simulation tools And they're like, oh, we should build it with a vr headset and or in augmented reality or in all these sort of complicated ways And we want people to be fully immersed in everything that's happening and teaching And there's potentially some good reasons to do that, but there are two big problems One is most institutions that train teachers Don't have tons of extra resources and don't have huge technical it capacities So our thought is why don't we build things that like people can author on a laptop and do on a phone? You know, let's let's let's You know, basically bring the technology demands of the system down as low as possible As long as people still feel like they're having some kind of authentic experience And then the second is that when people make simulations, they often make things that try to get folks to practice the whole thing But it turns out that there's there's good cognitive science that suggests that when you ask novices to practice the whole thing It's actually very difficult for them to get better at individual components If you want people to get better at things have them practice parts of things This is why athletes don't do scrimmages all the time. They do drills This is why musicians don't just play the whole piece all the time. They do scales and repeat pieces and they do other kinds of skills So we're trying to build like low tech totally seamless front end experience totally simple front end experience that does these sort of Drill like simulations that might have tons of complexity on the back end that you're not exposed to And all kinds of interesting research capacity and things like that But the initial experience, you know is simple as straightforward is authorable is adaptable to different contexts And you know it does just as much as it needs to do Nice Nice, and this is just one project you're working on. I know. Yeah We have a big team That's fantastic. And are you working on the on your next book yet? You know, I'm not It was a lot to write one book. I I think I agreed to write the book in 2015. I told them I would do it in 18 months And I did the final proofs of march of 2020 The one the next book that I'm thinking the two books I'm thinking two kinds of projects. I'm thinking about One is in my lab. We make a bunch of online courses I don't think it would be very hard to turn some of them to books because we put a lot of work into them already like with peter sangly I wrote this we made this course called Launching innovation in schools There's a lot of good stuff that's embedded in there and I bet people would read it as a book You know, I sort of take a bunch of transcripts and sticks them together the second project There's a little bit of the second project sort of branches out the book a little bit, which is that When I teach I teach a bunch of classes No one is an education student that I teach People often don't know much about education and learning and usually the whole course is not just a primer of education and learning So it's learning technique, you know learning media technology introduction of media studies You know technology design workshop. So so most courses have something that that starts with like a brief introduction to learning science And they're basically just two ideas like to a first approximation. There are two ideas in learning science There are people who think that you learn by pouring stuff in their head And there are people who think that you learn through apprenticeship like practice experiences, you know in in the 19th century in the United States these sort of You know positions were epitomized by Edward Thorndike and John Dewey But they're also captured like when Plutarch says something along the lines of Education is not the filling of a pale but a lighting of a plane There are a bunch of people who like no we have good science. It is also like it is filling of a pale Or it is also like filling of a pale So I think sort of explicating those two positions is interesting But I also think those two positions are are not just grounded in cognitive science But they have a sort of moral philosophical dimension to them You know, so Rousseau who had these ideas about You know natural liberties and the noble savage and the dignity of primitivism and things like that You know was a big fan of apprenticeship kinds of models and You know folks folks who like Sort of pale filling learning they tend to really celebrate how learning is difficult And folks who like sort of natural Apprenticeship learning like to celebrate how learning is easy And of course, I'm sure everyone who's listening can think of a moment in which they learn something effortlessly and naturally You know the way they they they sat next to their grandmother and learned her recipe for making tamales And it was it was pleasant and it was enjoyable and it was seamless and they could think of other things They could learn, you know oftentimes in math class or something like that where it required practice and revision and drafts and was painful and uh, you know, there's there's there's those those kind of like Those affective dimensions get kind of woven into what are nominally sort of You know arguments about the structure of the brain and cognitive science and things like that So that's another project that i'm thinking of trying to trying to weave together, you know How these two perspectives are not just pedagogical perspectives, but they're they're philosophical they're effective they're Um, they're multi-dimensional Quite extensive. Yeah. Yeah. Yeah, I mean, I don't know them as well in other kinds of cultures but uh, you know, they uh, um, you know, particularly in the west these two ideas run very very deep I'd go back to uh frankenstein and take a look at the education of the monster as well as the education of um, but friends, I've I've got a couple of questions, uh for an extremely energetic guest To ask him about about his book and where he's headed But then I'd love to hear more from uh from you and we already have a couple of questions, but I just want to begin what That it's the uh the technology But not transform it Or that it's just it never scales up Uh, and you have a whole series the second half of your book is about all the all the reasons why the curse of data The familiar the problem the familiar and so on Um, I'm curious when you started working on this, uh, was that where you began or did your view get Darker and darker as you again Um, well, I I don't I don't my view isn't dark to me. Um, it's uh That technology has a role to play in improving schools Um There are two ways of thinking about that improvement there may be more but there are at least two one And my colleague morgan aims is a great book called um the charisma machine about one laptop per shot And she defines this view of technology Um as a sort of charismatic technology and charismatic technologists which says new technologies will come and they will sweep away the future And they will they will break down the errors and failings of the past and they will lead us into uh into a very different kind of place And then There are also people who are more like tinkers who say no, no, no, that's very silly Um, educational institutions are incredibly complex They balance this incredible diversity of competing incentives and stakeholders and needs they're in a constant Delicate political balancing act, you know our school systems k through 20 They teach people to factor polynomials and to tie their shoes and pay their taxes and to get a job and to throw a baseball And to criticize the government and to love your country and to not have sex But if you do have sex to do in certain kinds of ways and on and on and on and all these kinds of things that we ask schools to do um And so it's silly to think that you're going to build something That's going to make a huge improvement across all of that because when you tug too hard on one part of that system All the other parts of the system go wait, wait, wait, you're screwing up our parts of the system This was all delicately balanced to meet all these competing needs But that's not to say that the system can't be improved It's very unlikely to be improved simply through the introductions of new technologies Because every technology solution is a human capital problem Every time you introduce some new tool or resource It's not really useful for reasoning there into until people have the time to develop new routines and pedagogies new supports and not just teachers Or faculty instructors, but students and families and community members and it staff and custodians and you know The whole range of of people that make these systems work So to me, it's not dark in the sense of I just think our Our celebration and our optimism is misplaced. I I think I think the charismatic have just been wrong over and over and over again and It's not it's not neutral They architect these boom and bust cycles that have institutions over invest in new ideas And then they get you know, then the charismatic skip board once they're not the saviors anymore and for the most part they leave We're not all in do Some of them turn into tankers And you know those systems are left behind my my mit students right now Our members of the smart board generation every year students come into my class I asked them tell me tell me the story of the technology that you remember from your schools And we've sort of finally hit the wave of students where their districts But smart boards in middle school and high school And nobody ever used them and they sat in closets or they just use as whiteboards And there's millions and millions of dollars It could have been done set productively to all kinds of other things sort of flushed away Which you know remain only as a memory in our students minds But I'm quite optimistic that when people bring a kind of shoulder to the wheel mentality you say look human development is hard Um the systems we work in are complex. They can be improved. They they get better technology Can be one thing that helps them get better of almost never technology alone But this is good work to be doing so so for me It's not so it's not so much painting a dark future. It's painting a cautionary tale About charismatic technologists who seem to be wrong over and over and over again And trying to really explain in detailed ways why And then give readers, you know both the motivation to be tinkers And then a set of tools that I think are useful for tinkers and doing their work of making change Well, thank you. That's an excellent excellent answer And I hope well already we're getting some responses and I'd like to Put these out for everybody Just really quickly Fritz van Dover says that charismatic technology seems to be brittle You get some older folks with older technologies mentioned here ditto machines and microfiche for example But let me put up a couple of the first questions that we've got This is one directly into a certain part of your book This is from Jeff Alderson at mathworks. Hello, Jeff. Jeff asks I have a question for you about auto graders and computer programming What do you recommend in lieu of auto graders to assess students? Sure, so our graders are pretty important to a bunch of models of learning at scale because scaling Scaling the dissemination of knowledge is not that hard We could record this talk anybody could watch it And they do and they do but if people wanted to learn they would probably want some feedback on their comprehension and understanding They want some way of knowing that what they've taken from from the discussion Is being interpreted correctly or being reinterpreted in meaningful ways and so you gotta give learners some feedback There are some domains in which we're pretty good at giving learners feedback The problem is that those domains are things that computers are already good at so When we ask humans to act like computers when we ask them to do sort of routine Highly structured kinds of tasks computers can evaluate them pretty well They can evaluate computation and math They can evaluate the conjugation of verbs the pronunciation of words in early language acquisition We don't have good auto graders that can assess whether or not You understand complex texts in a foreign language We can tell whether or not you've pronounced poor for work correctly We can't tell whether or not you've had some meaningful interaction with Cervantes d'Antio de So And also, you know one of the I call this the trap of routine assessment Which is that technologies are good at assessing things that we don't need human beings to do anymore And technologies are not particularly good at assessing the things where humans have a comparative advantage where our education systems should be focused on So You know I I hope I you know I hope that sort of argument does two things One is I hope splashes a whole bunch of cold water on the notion that like right around the corner Are these extremely powerful AI driven Auto graders that are going to be able to reliably evaluate human performance in all kinds of different domains Very very smart people at wonderful institutions with millions and millions of dollars and the support of foundations And governments and the departments have worked on this problem for many years and have not made much progress The essay auto graders that you know Sort of I first encountered 10 years ago 20 years ago The ones we have right now are like a little bit better Like you know in the way that Grammarly is better than you know the 1995 microsoft word grammar check But not better in the sense that we still don't have machines that can tell whether or not a sentence that you've written You know conveys meaning in a powerful way Now you know however all that said we have auto graders that are good at some stuff You know lots and lots of young kids and probably lots of College students too though, I know a little bit less about this particular application Have used some kind of math tutoring software this year They haven't been able to go to physical buildings. They've had less access to their human teachers And fortunately people have made stuff like you know alex and dreambox and st math and all these other kinds of things Um, and we can ask students to do computation kinds of things. Um, and the computers can go Yep, you've got that right or no, you've got that wrong sometimes they can even go No, you got that wrong because you don't understand this thing go look here There's lots of things though in math that they can't evaluate really well You know one of the things one of the things we most want people to learn as they study mathematics throughout their career Is to be able to articulate why they chose to solve a problem in a certain way to be able to explain What the solution to a computation means in the context of the real world, you know Was it 37 kittens or 37 board feet or you know 37 hot dogs We have much less good tools for that. But if you think about again, what mathematicians actually do Um, you know, I I do this much less, but I still periodically, you know, conduct statistical analyses and write papers and things like that The computer does all the computation for me. I figure out what problem is interesting I figure out how to set up and frame the problem. I explain what the computations mean and what their significance is Um, but you know, I let the computers do the computer stuff um So there's you know, there's going to be a role for auto graders The thing to avoid is two things to avoid are being seduced into believing that the things that we can auto grade are the important things for people to learn Um, in many cases, they're good building blocks, but they're not the whole field I'll give you a second example, which some of you may enjoy We have really probably the best auto graders that we have are in computer programming So I could ask you brian to write a computer program that performs a task And then I could write a computer program that could evaluate whether or not you've met engineering requirements Whether or not the the syntax and formatting is right. Um, certain kinds of errors there's a professor at um At mit of computer science named howl abelson sure who wrote a beautiful line, which is something like computer programs are communications between humans about methodology that only incidentally can be run by computers um If you believe that to be the case about computer programs, then our auto graders don't measure that at all You know What what howl is talking about is like art You know, are the structures that you're choosing the way that you're naming variables the way that you're commenting code Um, are you really communicating something about methodology to another person? Um, and so, you know, even the domains which are auto graders are best We're still not teaching people the thing We're still not evaluating the things that people need to be a really great software engineer a really great collaborator on a software project um But Like these things are helpful You know, like the technologies that we build are useful for Certain parts of certain subjects for certain students in certain contexts And we should use them wherever they're useful and we should not be seduced into believing that they're you know sweeping solutions Hi brian Hi Jeff Hi, Justin. So first off, I don't know if you can see this. I've got your book. It's literally in my hand. So I'm not bluffing Um, but but I know but as brian knows I've been I've been asked to join the stage a couple of times Um, actually I'm looking at us now. I think I'm the covet after six months. Justin, you're the covet after one year brian You're the covet after 10 years god forbid um, so So I'm happy to join I've been joined the beard panel today. So let me give you the context for why I asked the question I did so I'm I'm currently the product manager for matlab grader, which is an auto grading solution for matlab code at mathworks MIT is our one of our largest users of that auto grading solution your colleagues swear by this product in their in their classes and I'm also enrolled as a graduate student full-time at university of illinois for my masters of education And i'm focusing on learning technologies and when I was in a discussion in my class someone says, hey, justin You know right has this great book That's challenging some of the assumptions that you're making about your product management And I said, uh-oh And so I read the book And I'm in the chapter and I get to this heading and where you're literally put me and my work in the crosshairs but the good news is I actually Want to when I give you an open invitation the next iteration you do on this book. I would love to invite you to Talk with the folks at MIT. They're using our auto grading solution Because I think we actually are on to something and I want to specifically focus on What you said, which was the other stuff that auto graders aren't Evaluating but have to do with the intent have to do with the learning objective What was the behavioral things you're trying to check? The the skill the mastery of skills about computer programming and software engineering That can't be just gleaned from the syntax But they could maybe be implied by the syntax and need follow-up and discussion And and so I guess the way I was the reason I asked you the question is in on top of auto grading Is there another potential for automation or something at scale that could get at the heart of that? What's what's missing and as a product manager? What should I be focused on to? Take the next step beyond auto grading and look to Automatically assess the behavioral intent and the learning objectives Yeah, um, that that's a great question Um, you know, and I and I hope that one thing that people take away from the from the book You know two things one is that they're all kinds of good uses of auto graders um, there are you know, one thing I focus on a bunch is uh Computational assessment of pronunciation for language learning, you know I mean I had this sort of vision in my head of when you know, a Spanish teacher would go around the room And we would all say por favor You know in the Spanish teacher would like tap me on the head and be like, I don't think so But would move on, you know, because she doesn't want to spend Five or ten minutes just having you pronounce it slightly more right over and over again But computers will do that We'll just sit there and listen to you over and over again mispronounce por favor, you know coach you again kind of infinitely And I also think that instructional designers can do what I would sound or project managers can do what it sounds like You're doing which is continuing to think about How are we going to expand the domains of what it's possible to assess? And you know the example that I use for that in the book is some colleagues in mit Who are getting ready to teach a calculus class and they said we're not going to teach this class until students can draw curves It's just too important to be able to you know to to do the sort of conceptual work that's involved in You know in drawing curves and where they cross the excesses and y-axis and things like that And we could do with multiple choice questions, but that's not good enough. We have to be able to draw curves When they built that tool that drew curves, which took a lot of time and was was complex But worked very well like they sort of slightly expanded They sort of you know, that's kind of a way through the trap of routine assessment They kind of slightly expanded What was possible, you know, you've got a big team full of smart engineers at math works I'm sure day after day or sort of slightly expanding the domain Of what's possible And and pushing, you know, every every assessment You by definition is sampling from a domain Um, and you're making that sample sort of brought you know My argument is that the things that we're sampling are often not the most important things And you're continuing to try to expand that sample, you know For someone who's sort of looking at the field of of education technology broadly Those expansions are unlikely to help students You know who are who are in In a writing class or in a law class or you know in a poetry class or in a biology class They're just very particularly like your team is going to you know, spend gazillions of dollars and do tons of great work And you're going to keep sort of slowly expanding what's possible You know, it's unlikely that you're going to develop the sort of generalized ai that solves all these problems for everything All of that is great and worthy work You know, I think of that as sort of great tinkering work And the important thing for institutions to do is to keep auditing where that Where that where that sample from the domain is and to not be lulled into thinking that the sample is the whole domain Is to say look if these are things that that the math works on a greater doesn't evaluate right now Where in our program are we teaching these really important things to students? And you know, how are we you know paying people fairly at reasonable wages to be able to help other students learn that stuff You know, I I I am optimistic about the notion that There is more time for educators to focus on the things that they do best if you're building us some auto graders that That captures some things that computers happen to be good at doing and even You know, I mean MIT has also adopted. I think it's grade scope or something like that which Pass an animal to each other through the instrument. We can do this through that I just I just saw Brian holding at one cat and it disappeared. That's how Jeff pick up another cat. Um, it'll look very much the same so, uh You know, it's a it's a tool that it was built for huge classes And it just lets you sort of stand in test answers so people can grade them a little bit more easily It's like those are all good things to do and we just have to we can't confuse those like pretty good tinkering incremental improvements For the disruption of higher education Yeah, I I appreciate that context and it's one thing we actually ask our customers to help us with is to Prove that our auto grading solution is as good or better than what they used to do to assess the same learning objectives and outcomes and if we're able to In a statistically valid way show that our auto grading solution is doing the same things they used to do but at scale Then and only then will we move on to the next thing? Right, like we we want to take those incremental improvements But we don't want to like jump into such new territory that they've never tried to assess before that's not You know, we can't auto grade some new concept that they've never tried to even understand Behaviorally or from a learning objective or a pre-work as a skills perspective We don't we don't want to go into those territories. We want to show that we can grade things Faster more officially than that and improve that it works as good as what they used to do Um, you know, I appreciate I appreciate that context and thank you for the the answers in the time No, yeah, thanks and if folks at math works when I talk about this stuff more I'd be happy to Thank you, Jeff for the really good problem And uh, Justin, thank you for the very very rich answer Friends if you're new to the forum, that's an example of a video question. It really is that easy to do We just now share some of the test questions as well And let's see one comes from another close reader This is from Kate Montgomery at SMU and Kate asks You say that institutions and investors often favor programs to scale up quickly but at the expense of true innovation How is true innovation defined? I'll put that up on the screen again so people can see it I don't If you're going directly to the book, I don't remember that sentence anymore Um, here's here's what I hope it means but then someone can go back to the text and tell me if I did it wrong. Um you know, one of the phenomena I talk about is uh Is I call the cursive familiar? um and the uh The you know one way to frame the curse of familiars If you look at the history of education technology The thing that people do most commonly is they use new technologies to extend existing practices So they use new technology to do whatever it is They were doing before like a little bit faster maybe a little bit more efficiently If you look at the technologies and most widely adopted, you know, one example I use is in pay 12 One of the most widely used technologies by like half of all high school students I'm sure it's used by lots of higher education students too Is this tool called quizlet, which was made by a by a mit dropout or an mit You know industrial graduate or whatever you want to call that You know, and if I most of you probably know what quizlet is if you don't it makes digital flashcards It has you know, as soon as you glance at it, you'll know exactly what it is the flashcard app You'll have questions on one side answers on the ever terms on one side definitions on the other you practice Adopted very very widely very very quickly Um Because it let people do a kind of thing they were doing before If we grabbed a whole bunch of educational experts and pulled them together around a table and said like What do we really need in k12 and higher education? What do we need for the future? What do we really need to move things forward if you if you had not if you had 60 years of Future trends forum I'm pretty sure that not a single expert would raise their hand and go. You know what we have a real flashcard deficit in our schools You know the thing that we're up against is there there are not enough index cards We can't write definitions on them fast enough. This is what's really holding us back So I I think what I mean By true innovation in that context Is that the tools that most readily spread and scale are the things that let us do what it is that we're already doing But the but the trap of that is if we simply do flashcards a little faster and memorization a little faster That's not going to reduce the yawning gaps and equity that we have that's not going to you know help our six-year graduation rates that much That's not going to you know rearrange the iron triangle of cost access and quality By contrast When we build technologies that seem like wow, this could be a really different way of teaching and learning Like there could be some real advantages that would prepare people for the world in the ways that's you know Universities anchored in the medieval times might not be able to most people find them confusing You know when you looked at like the original massive open online courses that came out in 2008 that sort of borrowed from these network learning environments Super cool super interesting Really exciting really neat technologies and most people look them and like what am I supposed to do here? This is weird And we're really hard for people to persist in the k12 space You know a similar sort of phenomenon happens around the scratch programming language Which resnick and his colleagues at mit Build this incredible new community this incredible new programming language for creative computing That asks for really different relationships between teachers and students really different use of time really different ways of evaluation And so most schools if they do adopt straps, they're like all right We're not going to do that part actually we're just going to give students a recipe and we're going to have them all make the same computer program So You know that that's that's the curse of the familiar It seems it seems to be difficult to navigate between what kate reference You know just generating things that do what we're already doing a little bit faster That probably won't make that much of a difference or generating really new things that are hard for people to adopt The pathway through by the way, there seem to be two parts of it as far as I can tell Building things that start in the familiar and then lead you to really new kinds of practices Desmos is an online graphing calculator that I think has done that really well When you first encounter it you're like, oh, this does everything my ti 82 does but for free And then as you get into it more you're like this is a completely different way for students to learn modeling and math There's sort of a pathway for the familiar to something More ambitious. Um, and then it's not about scaling distribution. It's about scaling community If you have a thing that's going to help people do teaching and learning really differently They're not going to learn those strategies just from your technology They're going to learn those strategies from being in community with other people Now everyone's going to be graphing way And and the questions will drop. So I have to be careful here That's a That's a really good distinction between the strange and the familiar here We have more questions come in that build on that's uh, one I think comes from keeldeutsch It's most colleges just Reused classroom lecture model for online learning and they ignore the real value of the internet It's rich trove of books articles videos podcasts That's not a question. That's a comment Yeah, I think that I think the challenge of that is that um If you treat the internet just as a rich trove Even in some of the way that like mooks and other things have done like look, here's a course come and take it It's you know, the same thing that we offered at mit or harvard or whatever other great university. You think it is Most people are really struggle with self-directed learning Maybe one way to qualify that is when people learn things that have an Immediately that are immediate concern to them and are of high interest. They're actually incredibly good at learning online so if you want to learn new ways to style your hair to do your makeup to Do a trick on a skateboard to beat a level in video game People from all kinds of different backgrounds all kinds of walks of life All kinds of ages are quite proficient at that kind of learning When we ask people to do things that look like school Where you sort of have to like grind through some stuff that seems kind of abstract for a while for a longer payoff Many folks are not nearly as good at doing that so You know And the people who are really good at that tend to be people who have had a great apprenticeship in the formal education system You know, I don't I won't I won't say I notice for sure from science But if you want my current best bet the best way to become a really good autodidact Is to have a really good apprenticeship in the formal education system Which means that if we treat the internet as a rich trove of resources The people who will benefit from it most are already educated already affluent folks So we do need models That recognize that people don't just need a rich trove. They need supports They need peers and teachers and structure They need guidance some of which traditional lecture kind of courses offer But certainly I think more of our higher education institutions should be Should be deliberate about building pathways, you know By the time you leave this program you shouldn't have just learned some stuff You should feel like you've had some practice with some some mentorship and support at learning more about this stuff on your own That's a good answer And it makes me think of quite a few things involving MOOCs, especially the x MOOCs that you address so well We're building on the autodidact aspect Let's bring up a question from adib sayid Who asked what are your thoughts on alternative education such as homeschooling Unschooling and we just heard self-directed education, etc Well, you know an amazing thing that we learned during the pandemic Is there are a lot of there are a lot more students who might be interested in some of these approaches than maybe we thought In particular Amongst our most vulnerable students, you know, there there are lots of reports But you know, this hasn't gotten the quantitative science I don't think but but there's lots of really strong journalistic and qualitative reports that say You know in k12 There's a bunch of students who went to schools every day where they experienced a bunch of racism And then they didn't have to go into those buildings anymore and they really liked learning better You know in the loving arms of their homes and their families but the you know These homeschooling independent learning self-directed like all you know all that stuff is good Our our public education systems are a gem and a treasure and you know And maybe we should supplement them with these more distributed options But there's something enormously powerful about having places where people go to build community and learn together You know, even if a lot of that building, you know, even for higher education institutions that are moving more online You know, I do I do think we want a kind of federated system that has different options for different people because different folks are different Um, but but I'm but I'm not at all enthusiastic to some extent. I've never been enthusiastic about these sort of like highly These sort of models of highly personalized learning where we're going to algorithmically optimize each student's individual pathway through learning experiences In part because I got my career started as a ninth grade history teacher And there, you know, what I wanted more than anything else was kids With as different brains as possible and you have there are lots of ways of getting different brains But having different backgrounds and different life experiences is one great way of having different brains Having different brains read the same stuff At the same time and be in the same space with each other to talk about them You know, I want them to read the mayflower compact or the letter from a birmingham jail And just sit with each other and wrestle with what it might have meant to the people at that time What it means to them now what it means differently to each other because we have different life experiences And there's no algorithmic way to optimize through that faster It's uh, you know, one way to think about it is that there's some parts of Learning which might feel like, you know, the joy of learning is maybe like a jazz musician or something like that Where you're kind of doing your own thing Maybe doing your own thing with a couple of other people But a lot of the learning experiences we value most are like an orchestra Um, and even if the second violin has mastered their part You still just sit there and you play your part over and over again So the tuba can get it the trombone can get it and whoever else needs it can get it Because you know, the transcendent experience is when you all play that together And uh, you know, um, so that that's that's where some of my concerns about Some of you know, I did has a great point in the chat that like many homeschoolers and unschoolers create vibrant local communities That are more natural than the weird things that we do. I think that's right I think there's a lot that we can learn from those institutions. There's sort of cross pollination But I also think that some of the most vociferous advocates Of these kinds of things are also sort of advocates of free market approaches to learning Who you see online learning who see homeschooling who see unschooling You see virtual schooling as ways of breaking the social contract that we've had for you know, in various ways higher education for 16 years and Um 150 years and k-12 education of communities Educating people together Well speaking of time we're coming close to the end of our hour and I want to make sure that everyone gets a chance to Way in with their questions and comments So now if you have one that's burning up in your mind friends, this is a great time to put it in We have one from Michael fried at the ethical s and r and they just bring this up Is there a risk of veering the curriculum or course content to other technology is good at supporting its scale Instead of what is important to learn If so, how can that risk be mitigated? Yes, that is that is an excellent summary if you turn the first turn the first question into a statement. Yes We over anchor on what we can assess on what we can deliver online rather than what's really important The answer to that is to have as our ongoing mantra over and over and over again That technology needs to be in the service of learning that we think about Learning first and where we want people to end up and then we look at the technologies that we have And if we're educators, we say which of these helps meet our learning goals If we're technology developers, we say what are really important learning goals That whose needs aren't being met by technologies and let's try to build those kinds of things Financially that's actually quite difficult to do because if there's something for which You know it's not happening as well as it should have it probably doesn't have a particularly big market Because that needs to be sort of generated at the same time but You know I um Yeah, the way we do it is by is by having Learning be in the service by having technology being the service of learning the one of the challenges with that is for whatever reason Technology like takes up a ton of logistical brain space in people's minds You can start it You can start a conversation where learning and technology are in balance and all of a sudden you're finding yourself talking about like Software licenses and logins and single sign-on and where the chargers go And all these other kind of logistical questions You just found sort of further and further down this path and then you kind of look around and be like How do we stop talking about learning? How do we you know institutionally? How do we totally lose track of where we are? And it's very very common in all kind of schools. They're extremely well resourced schools. They're poorly resourced Elementary schools higher education institutions. It's a very very common pattern And so part of it is a discipline Of you know constantly asking ourselves the questions like you know, what are the learning goals? We have the students. What are the students goals that they have for themselves? What's the technology infrastructure that we're buying that we're building? You know, is it aligned with those goals and how do we prevent our so when I ever I think of this I think of the old farside cartoon of Um where the neanderthals in class and raises his hand and said excuse me, ma'am I'm going to go home. My brain is full Technology somehow does the brain is full in the chat carolin coward at jpl says that Chief finance technology takes up a lot of emotional brain space And if we furiously agree about that then we approve the point Thank you for that. It's a really good question. Um, and thank you again for that, just We have another question that comes off of the technology needle. Uh, this is from carolin How is the ed tech space combating hidden but systematic bias? And the algorithms drive many of our automated systems It's not it's not um, you know, one of the things that I don't know my students and I have just been reading The age of surveillance capitalism by uh, And one of the things that um that she says You know over and over again is that You know, it's it's it's in the interest of technology companies to make what they're doing is Tooth and abstract as possible Because they're trying to do things in places in which there aren't cultural norms or laws or regulations And so the more quietly they do it. Um, the deeper their incursions can go I'll tell you what we think the solution is. So as I mentioned to you before We have uh You know, we have a project to build this digital clinical simulation tool and we're actually building an artificially intelligent coaching agents in it You know things that say like recognize when you sound confused or hesitant and say, hey Brian, you sound a little hesitant there. Do you want to try that one again? Here are three sentence stems that might help Um Part of what we want to do we haven't gotten there yet But part of what we wanted to do in the build is also just like expose that whole process to people To be like, hey, remember those little nudges you got like, you know If you want to find out more about where they came from here's how we program them Here's how we tested them on them. How did it feel to be Supported by an intelligent agent. What concerns do you have? A lot, you know in a lot of consumer technologies. We want the technology experience to be seamless I just got a new electric car when I sit down in the electric car. Like I don't want I don't want to confront it You know starting to use my phone as a sound system. I just want it to work and go away I think in education we do want to trip over and encounter our learning technology systems I do think we want to have rougher experiences there where we go. Oh We are using ai How did it get trained? Who are these people? Why is it doing it to me? How do I have an opportunity to make input on my choices and things like that? So that I know that doesn't give you all the answers I saw a couple of great colleagues at the mit who are you know, not to not to be an advertisement for my institution today But um, who are trying to do more stuff in ai and education and help people especially young folks start understanding better how it works There's a there's a thing in uh in german called uh, I believe stolperstein them Which means stumble stones And this is when you have a cobblestone road and you take one of the cobbles You rotate it 90 degrees so that when you're walking along you hit it and you immediately look down And people have used it for ads eat at a hansel But also for history This happened in a place. No wonder about technology like that. But we have a metaphor. It's a great metaphor brian It's a it's a wonderful idea What does lead to bruise toes, which is also part of We have a one question from michelle denise miller who was a great guest. We also have a wonderful book Michelle says even as a fan of ed tech generally, I'm still constantly struck with a gap between a promise and the reality. What's actually That's one of the major reasons why this gap persists Uh, you know, I mean people fund promise and reality is hard um, you know, I think one of the things that I articulate in the book is that Though many of the promise reality gaps that we experience are not for lack of effort You know in in in mooks. I made this one of the promises of mooks was that we were going to generate a new data driven science of education that just like the genome project revolutionized medicine That uh, that mooks would be a kind of genome project You know that would that would dramatically change teaching and learning and has it and it's But not because the promise was totally crazy. It's kind of to me It's still on its face seems like a reasonable analogy and might have been possible And not because we didn't try I mean, I feel like I've worked with like literally hundreds of colleagues at dozens of institutions of millions and millions of dollars Trying to tackle this problem. Um, and I don't think we made a lot of progress because the challenge is hard Um, uh, that's what that's the thing I come back to over and over again You know, you think about the complexity of our institutions You think about the granularity of the work that we do like today Somewhere in this country. There's a seventh grade earth science teacher who is starting a unit on plate tectonics Let's say you want to make a technology that makes formative assessment a little better Well, what formative assessment on the first day of a unit on plate tectonics in the seventh grade looks like Is different than what formative assessment looks like in a spanish four class to a A graduating senior at a four year institution Is different than what it looks like to a computer science student who's doing their first lab practicum And it's different, you know with Like within each of our units within each of the days of our units that the granularity of the work that we're doing in Teaching and learning is so fine Um, then it's very hard to build things that are useful when you're trying to help people Learn how to tie their shoes and to have sex safely And to criticize the government and to conjugate arabic verbs and everything else that we ask people to do That's why it's hard, but each of those things that i've just described is just so immensely Valuable that they're all worth doing it too. You know, it's worth keeping our shoulders or we'll try to work on this stuff But we have to be realistic about what can be accomplished and we have to recognize that there's just a bunch of Financial incentives in the system for people to promise us the moon and it's not helpful when they do that It's not helpful when we believe them And so, you know, I I joined the ongoing Quixotic effort Trying to get my colleagues to You know, my colleagues in the tech development side to stop doing that my colleagues on the education side to Bring a healthy skepticism combined with the fakes that like if we tinker together we can totally make these systems better If we tinker together I I hate to pause you on this, but we just shot past the end of our hour. That's a fantastic note to end on sounds great. We're kind of like a tinkering and uh and a Marriage counselor at the same time What are the best ways for people to keep up with you? Is it uh, uh, you're on twitter or should we uh, I'm at bjfr on twitter if you go to the our website tsl.mit.edu. We send out a newsletter Um, okay, and uh to sign up there Those those those are those are the places that we that we publish stuff these I don't know you could sort of a google alert for my name, but That's not what I'll do. That's what I do. I wouldn't say what you all would do Well, that that works very well and some people have been tweeting at you during during this event and you probably get more More twitter questions and comments as we go. Thank you for an incredibly nuanced Deep and very inspiring discussion. I really thought it was a pleasure But don't go away from that as well. We're going to bring you back next time But let me tell you what's happening for the next few weeks. Um looking ahead We're going to continue our exploration of technology at academia again We're going to be looking at sparking emerging ed tech conversations How to improve equity and education for black students federal policy changes and a lot more so just go to forum the future of education dot us Now if you'd like to continue discussing these issues Honestly, I think right now twitter is the best one. Uh, we haven't seen a lot of people using our facebook or linkedin pages Um, so I may just sunset those but please use the hashtag If you'd like to look back into the past and to including some of the sessions that we've already evoked today, uh, please go to Our archive on youtube. You can find that at tiny url.com slash ftf archive And uh, as you proceed as you wrap up this spring semester as you think about the summer as you think about the fall I hope all of these discussions have been of use to you all I really appreciate all your comments all your questions today It's been a pleasure to think through all this together with you And in the meantime, stay safe. Take care and we'll see you online Bye


Okay everyone, so please welcome Fridolin who's gonna tell us about TOTS, the recommendation engine for Python. Okay, so hello everyone, welcome to this presentation about project TOTS, about recommendation engine for Python applications. Before I start, let me introduce myself. My name is Fridolin, I work at Red Hat.  I had a talk about project Selenon. Right now I work on project TOTS. I'm one of the developers of TOTS and I would like to introduce that project to you. So what is TOTS? And why TOTS? You probably know the PyPI, the Python Package Index. That's Index that hosts open source projects. And when I wrote these slides, I found out that there is something like 200,000 projects available out there, free to use. And there are about 1.6 million releases. That's quite a huge number, I would say, and this number grows each and every day, especially with popularity that Python ecosystem has. So let's try to use some packages. So let's create some artificial example and use some packages that are hosted on PyPI. So let's say we would like to use TensorFlow, a machine learning library, and also Flask. So we install these dependencies using, for example, PIP, and we write our application that uses these two libraries. OK, but if you take a look at this scenario, it's not just your application with these two libraries. By installing these libraries, you also introduce most of the time transitive dependencies, and you use some Python interpreter that uses these packages. This Python interpreter then runs inside some operating system that provides some packages, such as glibc, that is, in our case, used by TensorFlow. And then there is that kernel space where operating system runs and provides some kernel modules. And this all runs on some hardware. You can imagine that if you have any issue in any layer of this stack, then your application simply misbehaves or provides wrong outputs or simply doesn't even run or doesn't even start. So let's take a look at this layer. Let's take a look at direct dependencies and transitive dependencies. When I was writing these slides, I took a look about its packages, and I found out that there are 33 releases of Flask. And when you do PIP install Flask, you introduce five additional packages, such as click, and so on. These all are also released in different versions, and now let's suppose that we find all the versions of these libraries, and we would like to know how many combinations are there to install Flask with its dependencies. And I got this huge number. This is estimation. So the actual number will vary based on actual resolution, but you see that there is something like 50 million possibilities how you can install Flask application with different versions of its dependencies. I did the same for TensorFlow, and at the time I was writing these slides, there were 85 releases of TensorFlow, and TensorFlow is much bigger project than Flask, and it introduces something like 30 additional packages. When I did estimation on how many combinations are there to install TensorFlow stack, I got this huge number. So it was something like 10 to the power of 20. If you take a look at some AI approaches that are today, you probably know AlphaGo, the famous Go AI that beats every Go player, and it uses machine learning to find the best possible move in a game. And AlphaGo or Go game in general has something like 10 to the power of 172 combinations, how you can place things on board. So it's much bigger number compared to our number, but in project thought we took similar approaches as AlphaGo, and we are trying to solve developers' issues with stacks. So this was this layer. So we had something like 10 to the power of 20 number of combinations, but there is much more layers. So if we take a look at the software layer, there are different Python interpreters in different versions that you can install, then there are different operating systems in different versions that you can install. Then you have packages available on PyPI such as Flask or TensorFlow, or you can host your own packages on your own index, and we do so in our team. We optimize TensorFlow by switching different compiler flex, and we provide a community index that hosts TensorFlow releases free to use. So if you use TensorFlow, you can use these releases and gain some performance. OK, so now the question is, what should I use out of these options that I have? Now let's simplify our use case, and let's say that we have some two libraries. One is called SimpleLip, and the other one is AnotherLip. So it's similar case as Flask and TensorFlow, but in this case SimpleLip will not introduce any transitive dependencies to our stack, and AnotherLip will not introduce any additional libraries to understand. In other words, we install just these two libraries when we run our Python application. Now we would like to find some function that describes how good our software stack is. So we would like to install SimpleLip and AnotherLip in different versions and evaluate how good the given software stack is. We can do so, and we can create such function, and we install different versions of SimpleLip, different versions of AnotherLip, and then we have some overall score. This function is discrete, so we have discrete values, and let's try to interpolate these values. What we get? We get a surface, and we see that in some cases our application is scored better in some cases worse. What this score can mean? It can mean whatever you can think of. It can be, for example, number of vulnerabilities present in your software. It can be also performance of your machine learning model, but it can be also a combination of these two or any other metric you could think about. What we are looking for? We are looking for these spikes, like values that are very high score, and we assume that the scoring function returns higher values for software that is good for our use case. That's what we are looking for, and that's basically TOT. In TOT we aggregate some knowledge about packages, such as if the application builds correctly, if it runs correctly, if it runs correctly on different Python versions in different operating systems, if it behaves correctly, what's the overall performance, and then there is written a resolver that takes into account these observations and can resolve software stakes that are high performing. So if you remember that slide with function, these spikes in the wall state space. So we say that our latest versions are not always greatest choices, and that's basically the idea behind TOT. TOT is a bigger project. It runs in OpenShift. There are components and different integration points. But the most interesting component for this slide, for this talk, is Advisor. It's a component inside TOT, and it's basically the implementation of Python resolver. So TOT is a service that provides endpoints that you can consume data from, and it's pure server-side resolution. We had multiple iterations on implementation. The very first implementation was pure Python implementation, and it loaded the wall dependency graph into memory, and then we had representation of these packages. That was basically an re-graph, and we did transactional operations. So we removed these three packages from dependency graph, and that transaction error proceeded or not. It was also possible to score these packages, and the dependency graph was adjusted in a way that resolution led to higher performing stakes sooner. This implementation was, however, memory quite hungry, so the main issue was memory consumption. And, as you know, objects in Python carry some additional metadata, such as reference counts. And, for example, for a TensorFlow stack, we required something like 32 gigabytes just to resolve a software stack. So we abandoned this solution, and we re-written the core part into C++. So we designed a protocol that effectively serialized the wall dependency graph, and we were able to keep nodes of dependency graph in something like 24 bytes, and we gained also memory consumption improvements. However, we reached another bottleneck, and then there were a number of queries that were hit to our knowledge base or to our database just to obtain the dependency graph. So there were something like two and a half thousand queries to our database just to obtain TensorFlow's dependency graph. And then there were subsequent queries to score the dependency graph, and that required a lot of pressure on databases, so we changed the database two times. So right now we run third database, and later on we also decided to abandon this solution because of these queries. That's quite a huge number. So we looked for some solutions that we could apply from theoretical informatics, and we did operations research. So we implemented different types of resolving, for example hill climbing or simulated annealing. And this rethinking of the wall resolver led into split, the wall resolver into two parts. One was resolver that can lazily resolve software stack based on Python's ecosystem specification. And then there is predictor that guides this resolver on which packages, in which versions, should be resolved in order to come up with some software stack. This implementation worked for us, but if you think about it, we were basically randomly sampling this wall state space of software stacks that can be resolved. So we randomly picked some, scored it and evaluated the score. That's not nice, so we tried to find another solutions. We know that there is this function that describes packages. But what about finding gradients? So if we are able to find the gradient of this function, then the gradient can lead us to these spikes, and we can find higher scored software stacks much faster. And that was basically the next approach we took. So the most interesting paper that we evaluated was neural combinatorial optimization with reinforcement learning, published by Google Brain, and we really tried to learn that gradient. Unfortunately, if you think about thought as a service that should be responsible to users, you don't want to learn some gradients and you don't want to spend time on learning some neural network and provide inputs to neural networks. So you don't want to spend two hours to learn neural network and another 30 minutes to query your database. So we also abandoned this solution and right now we use gradient free methods. So we implemented temporal difference and Monte Carlo research, where Monte Carlo research looks the most promising way how to resolve graphs. So how does it work? We basically tried to sample that state space, that function of all possible resolved software stacks, and we learned policy how to find the best software stack based on predictor and based on scoring mechanism in the resolver. The resolver itself is a reconfiguration pipeline, so you can write different pipeline units. I think there are five pipeline units in total and these pipeline units are dynamically constructed on each request and they score actual resolver steps. So this way we can, for example, plug a new pipeline unit if you are using convolutional neural network. We can plug a pipeline unit that is specified just to score convolutional neural network layers. Then a special component in our deployment is called the dependency monkey. And this component is able to gather observations for us such as if the application runs correctly, if it behaves correctly, what is the performance. And this dependency monkey can sample that state space of all the possible stacks and resolve software stacks that we don't have any observations about and submit them to our service that evaluates how good the given software stack is. So this is how we advisor in our stage environment. So we have totally deployed at Red Hat and I asked to resolve TensorFlow stack. TensorFlow stack together with Flask. If you can see the whole resolution to something like 8 seconds, I asked to resolve latest software stack. When I asked to resolve the best possible software stack, the resolution took something like 2.7 minutes or almost 3 minutes and we were able to score a half million of software stacks. If you compare it to P-Pen, the packaging tool that is recommended by PyPA, when I tried to install TensorFlow and Flask and asked to resolve its latest version, the resolution took something like 1 minute and I had turned on pip cache so it was using cached artifacts from PyPA. So we are really benefitting of that offline resolution that we have inside our resolver where we do not contact directly PyPA, but we have pre-computed data and we can resolve software stacks quite fast. There are also other parts of thoughts that I haven't spoke about. For example, we use bots to automate updates of dependencies, new releases of bots components, of thoughts components. You can find also our TensorFlow index that hosts these TensorFlow artifacts that are optimized for performance and we have integration with different tools such as OpenShift or Jupyter notebooks. So if you are data scientist and you use Jupyter notebooks, you can enable our extension to Jupyter notebooks and it will talk to TOT and TOT will resolve software stacks for you. Then there is command line interface that is similar to PIP or P-Pen. If you have a GitHub repository and you use Python applications, we soon will release developer preview for our application that is called KBahead and you can install it, you can enable it for your repository and your dependencies will be automatically managed by TOT. So if you want to give TOT a try, feel free to install this application and we can make TOT better. So it's a community project. You can find information about TOT on this site. We have Twitter, so feel free to follow us on TOT Station Twitter handle. We are open source, so we can find us on GitHub under TOT Station organization and if you are interested in BOT, there is a link to BOT and if you have any feedback for BOT, you can submit it to our feedback form. So this way I would like to thank you and that's it. We have plenty of time for questions. So the question was whether this is just for PyPI or if this solution will work also for your local packages. In TOT, we can register Python index if it follows PEP specification of Python index and if it is publicly hosted, we are able to analyze these packages and resolve and use them for resolver. When computing the number of possible configurations, did you take into account the actual version requirements or it was just based on the name of each package? Sorry, can you repeat? Is that better now? So when computing the number of possible configurations, did you take into account the actual version requirements or it was just based on the name of each package? It was based on actual version specification. So some packages were really discarded, like older versions were really discarded by these combinations. How trustworthy is the output of TOT? Say it tells you that the best score for your software stack is using a pretty old version of a simple lib or another lib and you may be missing some features that you need but TOT is telling you that your software stack is going to have a better score. So how do you handle that? So if you use some features of libraries that are released in recent versions, it should be stated in your requirements file or in your PEP file because you really rely on that newer version that has these features. Can we expect this to come to other languages as well, like Golang or C? So right now we are focused on Python, maybe follow us on Twitter, we will post updates. So yes. If I have a very custom score function, I can provide it and you recompute it for all the package I need and it sounds very expensive. Recomputing of what? If I have a custom score function. Can I provide my own score function? You can. So there are these pipeline units that you can implement and in that case you can basically create your pipeline unit and say this is score that I would like to expect and then the pipeline unit is plugged to resolver. Well, thank you for doing. OK, thank you very much.


You're gonna need something to entertain you. What on earth does that have to do with The Last of Us? That piece really is kind of annoying. It's not just zombies, it's mushroom zombies when The Last of Us is over. I, along with the rest of the world, have been watching The Last of Us. And in talking to other people who are also watching The Last of Us, I started noticing how often I was using comparisons to books and book characters as, like, why I like something or what it reminds me of or recommending things to them like, oh, well, if you like that part of it, then, you know, there's this book I read. So I decided I could do a video talking about some of those books because The Last of Us is about to end and you're gonna need something to entertain you when The Last of Us is over. So I have five books for you today. Some of them are only, like, a very specific part of The Last of Us that it reminds me of. Others are kind of more generally something about The Last of Us. So let's go through it and I'll explain why each of them is in some way reminiscent to me of The Last of Us. First up, I have The Road by Cormac McCarthy. This book is not for the faint of heart but neither is The Last of Us. This was also turned into a film but if you've never heard of The Road and you don't know what it's about, it is a post-apocalyptic story in which a man and a young boy are traveling along trying to make their way through this post-apocalyptic world trying to survive together. Do you see why I think this is reminiscent of The Last of Us? So it's not quite the same structure we don't have, you know, this like zombie plague going on like we do in The Last of Us but there is this kind of like man with a child trying to protect the child, trying to survive unsurvivable circumstances, push to their very limits. It's quite dark as anything post-apocalyptic is and I just feel like the tone of it is in many respects quite similar. So if you like that about The Last of Us, the tone and the general kind of situation, I think you'd probably enjoy The Road. Next I have two books for the same reason. So if what you really like about The Last of Us is this dynamic between Joel and Ellie, this like gruff dude that's been settled with a kid, doesn't really want to be settled with the kid, doesn't really know how to handle being settled with the kid, is probably not the ideal caretaker for the kid except for in some respects such as being able to protect them from violence. If you like just like that piece of The Last of Us, I have two book recommendations for you. First I have Vicious by V.E. Schwab. This was originally I believe intended to be a standalone or at least sold as one. She has written another book and I think there's going to be yet another one planned but you can definitely read Vicious as a standalone. So Vicious, the storyline, the like plot, the concept for the book is extremely different from The Last of Us. This is almost kind of like kind of a grittier take on an X-Men type situation. So like there are people who are developing these kind of extraordinary abilities and so our two main characters meet in college and they unlock some extraordinary abilities in themselves and the story takes place in sort of like flips back and forth between the present day and their days in college and jumps all around but it's clear that they have become like nemeses even though they were like besties in college. So you kind of figure out as the story jumps back and forth why that is and now why what they're doing about it. So the book kind of like counts down towards like zero hour where everything's going to converge. If you're like what on earth does that have to do with The Last of Us? Victor Vail who's like arguably the main character, certainly one of the two main characters but this is Victor Vail in the middle of the title here. So I would say he's the main character. He's saddled with the kid, an extraordinary kid and he is definitely not a good caretaker and not doesn't want to be one but is likewise, you know, he's able to protect her in some respects and there is this, you know, kind of like cute kid with like gruff, violent, not ideal dude looking after her and it is like, I don't want to say cute because it's quite sinister but it does very much remind me of the dynamic between Ellie and Joel except that I like this girl way more than Ellie. Ellie's kind of annoying. So Sydney and Victor in vicious are a lot like Ellie and Joel. And then again for the gruff man with the child the Witcher series by Andrzej Sapkowski which there's also a show of this and one of the main things that I'm really mad about in the show is that the character of Siri was introduced already almost like fully grown almost an adult like a young teen basically which means that the extended portions of these books where Siri is a little, little kid and Geralt the Witcher is taking care of little Siri are not present in the show and it's really, really charming in the books. It's just again, it's quite dark and sinister. Geralt is probably not an ideal caretaker. He is a Witcher and if you don't know what that is because you've never played the games or seen the show or the books he is a monster hunter basically. He's like genetically mutated to be like good at that.  So now he's saddled with this little girl who's a princess and has to take care of her and not only is she a princess but she's extremely precocious and so they're going around this dangerous world filled with monsters and there's this princess and he's not particularly excited about the idea of being exiled with her and she's kind of a lot to take on and the dynamic between them is a lot like Ellie and Joel except again, I like Siri better than Ellie because Ellie's kind of annoying. Next up, I have one of pretty much like the only book I like that has like zombies in it. Not just zombies but like kind of does this sort of like overview of kind of like society reacting to this how would society structure itself in response to this not like just having a story about a monster run away kind of looking at how humanity we kind of respond to that situation and that is Dread Nation by Justina Ireland. This is about zombies during the Civil War or like just after the Civil War. That's kind of how the Civil War ends with like a zombie uprising. So we have like that era of America suddenly like history alters because the sudden appearance of zombies and now the trajectory of history is like shifted because people are responding to the existence of zombies and what to do about that. So it does a really amazing job at just being an alternate history and like believably depicting what that would be like. You're like, okay, let's say zombies appeared during the Civil War. How would people respond to that? This book does an amazing job of like realistically depicting that and handling like zombies as a concept very well. So I just highly, highly recommend this book in general. But again, if you like that piece of The Last of Us kind of more just like outside of the narrow character story if you like the kind of the way that it's exploring how the world has responded to zombies what are the different ways people have reacted to this and like built up defenses for it or have thought is the most wise response to this then I highly recommend Dread Nation. And lastly, if what you love about The Last of Us is that it's not just zombies, it's mushroom zombies. It's fungus zombies. If that's your jam, then my like favorite instance that I admittedly have not seen a ton of this. I've seen it a few other times. My favorite iteration of this that I've seen is in What Moves the Dead by T. King Fisher. Again, this story is nothing at all like The Last of Us. This is a retelling of Edgar Allen Poe's The Fall of the House of Usher. Where we are and what's going on and like the types of characters we have is extremely different from The Last of Us. But we do have fungus zombies, mushroom zombies. It's done so well in this book in my opinion. It's subtle and it's eerie and it's disturbing. And I love this book so much.  And it will put you off mushrooms just as well as The Last of Us does, I promise. And those are the five books that I have for you to read once you're done with The Last of Us. Let me know in the comments down below if you've read any of these books, if you've never read these books, if you plan to read these books, if you never ever plan to read these books. I want everyone to let me know. I post videos on Saturdays at the random times as well, but I think Saturdays will like and subscribe. Join my Patreon if you feel so inclined and I'll see you when I see you.


Sam here, late in sports car, drippin' 21-22, Peninimo's Egg basketball, choice five box break, number 10, random team style. There's a look again at your RIN results. As always, you can find these on the websites under the results tab, as well as the link in chat. Good luck. Yeah, true, I mean it is. Like, I mean, Kawhi with the Raptors, that's like the number one, how well that works. Like, loadmen, they probably, if they didn't rest Kawhi as much as they did with like his past injury issues, they probably would not have won those finals. Raptors would not have won the finals that year. Like, it does make sense. Like, I mean, I see it from like a coaching point of view, for sure, absolutely. Just Jalen Green and Evan Mobley. Nice two exclusives there. We've got Dwight Howard Silver and Yuda Watanabe on the mosaic. Nice one here. Black gold, six of eight, Rookie Otto, Keon Johnson. There you go, Portland, going to Shawn M. Nice low numbered, Jersey numbered, Keon Johnson. Focus, there you go. Again, I think he's number six and that is numbered six of eight, so that is Jersey numbered, very nice. That's Portland. We've got Ray John Rondo to 88 for the Cavaliers. Jalen Suggs, debut to 88 for the Magic and Maxi Cleaver to 88 for the Mabs. Nice three, Franz Wagner. And Chris Dwarze, that is Isaiah Stewart Silver and Fred Van Bleet, mosaic for the Raptors. Nice one here. It's gonna be Rookie Otto, Usman Garuba, the goofy Garuber, going to the Rockets. Nice one there, H-Town. Followed by Kyle Lowry to 88 for the Heat. A Brandon Boston Jr. to 88 for the Clippers. And Desmond Bain to 88 for the Grizzlies. Very nice, very nice, and very nice. Bain's out. I think we're really gonna see how good Desmond Bain is. Like if Jog has suspended for an extended period of time, I think we're really gonna see, with no Brandon Clark, I think we're gonna see how good Desmond Bain is. Bones Highland for the Nuggets. A Jeremiah Robinson Earl for the Thunder. Silver Charles Bassie and Obi Toppin. Isaiah Jackson, that is Rookie Otto going to the Pacers. That is gonna be Damian Lillard to 88 for the Trailblazers. A Sharon Jackson Jr. to 88 for the Grizzlies. And Nicola Vucovic to 88 for the Bulls. Nice ones there, on to the next one. I'm pretty pumped for iMac, dude. For iMac basketball and football. I can see us pulling some crazy stuff tonight, though. Global Man, NFL Shield, Nike Swooshes, crazy patches. Helmets, all that kind of stuff. JT Thor for the Hornets and Usman Garuba. Fred Van Ville runs, covers 2.77 miles per game. Oh my God, that's insane. John Wall Silver, that's wild. That is Davion Mitchell on the Mosaic Rookie for the Kings. We've got Otto, it's gonna be Alexi Pokashevsky. That's going to the Thunder, nice one there, on the Poku, 2.7 McCall Bridges, wow. Fakundo Campazo to 88 for the Nuggets. Looks like we have a Black Gold at the end here. Jaden McDaniels to 88 for the Timberwolves and Black Gold. Cade Cunningham, boom. Six of eight on the National Pride. Who's got Detroit? Holy cow, man. That is John W. Congrats, man. Big time Cade.  Huge card right there, huge card, huge card. You got that mag for you for sure. That's awesome, huge Cade, dude, huge Cade. And that's up, Jonathan. Right, you got another Black Gold too. Cam Thomas, nice one there. And Io Desunio. Spencer Tinwiti, Silver for the Mavs. And Mosaic, Yanis, nice one there. You had me confused for a second there, Jonathan. Doing well, Jonathan, how are you, man? I am doing well. There's a Jalen Suggs for the Magic on the Auto. Nice one there. Rookie Auto Orlando, Jeff Ries. This one here for Memphis, Black Gold, Triple J. Jaren Jackson Jr., three of eight. We've got Fred Van Vliet to 88 for the Raptors. And a Dearon Sharp to 88 for the Nets on the Rookie. Let's do a quick recap here. What's up, Kevin? We've got a Jalen Suggs, Blacksie Pokeshevsky. That wasn't Black Gold right now. Blacksie Pokeshevsky, Isaiah Jackson, Usman Garuba, Black Gold, Keon Johnson to eight for the Timberwolves, and a Black Gold, Kate Cunningham, six of eight. That will do it for the break. Thank you guys again for the fill on that one. Next up, we've got I'm Actiored.


What we find in the philosophy of Michel de Montaigne is a profound prose refreshingly original and adorned with an artful discursiveness far removed from that speculative tempest which sieved throughout the medieval age. Here is no blind following devoid of personal judgment, but instead, as said best by Stefan Zweig, was a philosophy that will teach us how to stay free. How to preserve our inborn clear-mindedness in front of all the threats and dangers of fanaticism, and how to preserve the humanity of our hearts among the upsurge of bestiality. Pursuing knowledge by way of personal experience instead of unsubstantiated speculation, Montaigne claimed to expound upon only one subject, this being himself. He went about this scrupulous self-examination by exposing his own judgments in order to better understand not only their strength, but also their weaknesses. He sought to influence the use of critical judgment and detested the common practice of labeling mere popular opinion as absolute truth, because how many things we held yesterday as articles of faith which today we tell as fables. Instead of making these baseless claims, we should seek to gauge their strength. Throughout his essays, the echoes of Hellenistic philosophy, most notably Stoicism, Skepticism, and Epicureanism can be frequently heard in alternating intensities. Along with his views on Christianity and Education, this video will seek to explore how these various schools of thought influenced his philosophy. Montaigne's Educational Ideal Montaigne believed that education should be no vain attempt to mold one's mind to establish beliefs, but rather the sharpening of one's own judgment and conduct. Instead of strict uncompromising book learning, children should be exposed to physical and playful activities. This, he thought, would allow for conformity to social and political customs while avoiding that eager subservience characteristic of those involved in that unholy inquisition. In regards to the study of philosophy, he thought that it should not be done with ostentation as it could lead to unorthodox tendencies and an uncongenial personality. Believing philosophy to be the practice of free judgment, Montaigne urges his readers to adopt a thought process that would encourage the use of free inquiry. Without this, we are inevitably limited by what is taught to us during those credulous years of our youth. Montaigne reminds us that our ignorance, if realized, becomes philosophy. Montaigne's Stoic Ideal Early in his philosophic voyage, Montaigne was in search of any school of thought that would uphold his moral and ethical ideals. And with Christianity busy waging war with itself and burning those trees in his hair ticks at the stake, he would, for a time, look at the world through that stoic lens and strive to abstain from all passions that would disturb the decency of his conduct or the tranquility of his mind. He so venerated Seneca that he would credit him along with Plutarch for whatever value his essays possessed. Contemporaries would even go as far as to refer to him as the French Seneca. Montaigne said to study philosophy is to learn to die, and there could be no greater teachers in the art of death than those great Stoics of antiquity. From his Stoic mentors, he would learn to focus only on what he could control, and the only thing he could control were his own judgments. He said in regards to this, not being able to govern events, I govern myself. Along with this self-governance, he embraced that Stoic virtue of self-doubt, which would come to compliment his philosophy throughout his years. In his essays, he offered us this in consolation. If a man cannot attain to that noble Stoic impassibility, let him hide in the lap of this peasant insensitivity of mine. What Stoics did from virtue, I teach myself from temperance. Montaigne's Epicurean transition. Alas, his scintillating spirit would prove irreconcilable with the life of the Stoic sage. He decided instead to follow his own natural desires. This, of course, as long as no harm was done to those around him. It delighted him to find in that fluent eloquence of Lucretius a philosophy akin to his nature. Following this discovery, he would don the cloak of an Epicurean and claim pleasure as no mere vice, but instead as a legitimate virtue to be enjoyed as long as one goes about seeking it with temperance and moderation. Intemperance is the pestilence which killeth pleasure. Temperance is not the flail of pleasure, it is the seasoning thereof. Montaigne's Skeptic Transformation. With Christianity continuing on with its descent into depravity, Montaigne would come to adopt the skepticism which would, from thereon, enrich his philosophy into that which many remember him by. He would, nearly 100 years before Pascal and 200 before Kant, ask that contentious question, why should we trust reason? His stance on reason, like that of Locke, is based upon the foundation that all knowledge comes from the senses, and because our senses are so often deceptive, reason is therefore unreliable. This precariousness of reason was not Montaigne's only path to skepticism. By way of contemplation about the vastness of beliefs present in the world, he reminds us that the difference between individuals is at times more than that between beast and man. Thus we should avoid limiting our minds to that small corner of the earth that we call home. He avoided absolutes in anything, but took his truth what appeared so to him. He said, I fastened myself on that which I see and hold and go not far from the shore. Taking his stance with the likes of Socrates, he claimed as his motto, the phrase, what do I know? His skeptic ideal would differ from that of Piro by offering his readers this simple pleasure of dismissing knowledge and exercising their own natural judgment. Montaigne and Religion To understand Montaigne's religious views, I think we should first understand his stance on government. He thought that changing forms of government or attempting to overthrow popular custom would inevitably cause more chaos than it would order. Thus, a wise philosophy should make its peace with religion. Montaigne himself maintained a vague belief in a god throughout his life. He notes that he is a Christian only by geographical accident and claimed, if born elsewhere, I should have rather taken part of those who worship the sun. He would reject atheism and agnosticism, categorizing them as unnatural. And even though he often criticized the contemporary Christianity of France, he was as said by Will Durant, too much of a gentleman to set his neighbor's house on fire before he had any other shelter to give him. Montaigne Through Posterity On September 13, 1592, at the age of 59, Montaigne's life would reach its end, but his influence would carry on throughout the ages as perhaps the modern world's first example of a tolerant mind. His philosophy would prove to be the antecedent of that great age of reason, and from his ashes would arise many of history's most influential philosophers, of these, Bale, Bacon, Diderot, Descartes, Rousseau, and even the great Voltaire would feel his influence. Why are we drawn into his philosophy? Well, perhaps because we can relate to it. His faults are laid out with surprising honesty and his informal style stands out among the formal dogmatism of contemporary minds. He teaches us that while seeking to further our own knowledge, we should not get caught up in those metaphysical unobtainables, but should instead focus on what is within our reach, this being ourselves. He made up his own mind and lived his life exactly how he had planned it. He said, I study myself more than any other subject. That is my metaphysics. That is my physics. Since his death, his style may have been copied, but never has it been matched. If you enjoyed this video and would like to see more on Eastern and Western philosophy, consider subscribing to the channel. And as always, thank you for talking philosophy with me. Until next time.