How your current search algorithm works, if I type something like, and this all got shifted something else. A little tiger should be in the box. Well, I think it got shifted a little bit here. If I go and type in a Google box, something like Bengal Tiger, hit return. Your Google search engine goes off and it's already indexed all of the web pages, 10 million of them. It will be back to me, links to what it thinks are the most relevant, like to talk about Bengal Tigers. And that's all very well and good, but my new idea is substantially better because this new idea is based on Darwinism that we're going to call Google on steroids. It doesn't have to simply limit itself to existing web pages. It can actually search the total space of conceivable web pages, of possible web pages. And so these guys are looking a little bit suspicious across the board and table. And you say, yeah, this is what I'm talking about. It is not limited to existing web pages, maybe 10 million of them. It can look at all possible web pages. You type in Bengal Tiger here, you hit return. And even if there are no sites on planet Earth that talk about Bengal Tigers, this will construct one and send it back to you. And it will be an awesome site on the Bengal Tiger. And they're saying, really? And you say, yeah, but that's not all. That's not all. Well, they say if it can do that, then we shouldn't really call this a search engine. We should call it something like an engine of invention because sites doesn't have to exist and this will go up and create it. And you say, yeah, that's okay. But it's even better than that because this search engine does not limit itself to the space of possible web pages. It actually will reconstruct matters. So if you type in Bengal Tiger and hit return, out comes a Bengal Tiger. And at this point, they're calling security and trying to get you out of there. And you say, hang on. Okay, I'll leave, but it's better than this. Because actually, you don't have to put anything as specific as Bengal Tiger. You should just type optimal fitness. And again, it serves the entire space of material possibilities. And it will produce not only the Tiger, but 100 million other equally remarkable species, actual living things, all simply by typing optimal fitness. And we'll go out there and do the job of finding these different things because they're fit. Well, at this point, they've tied you up and taken you off their campus in disbelief. And I would suggest you that really is what you need to believe if you want to believe a Darwinian story because it is solving problems like that. Now, you might say, well, but it gets a billion years to do it whereas Google has to return its result in a third of a second. But it turns out that if you look at the size of these spaces, even that does not make up for how much more remarkable the Darwinian search engine is than the Google engine. Because these become phenomenally huge when you start looking at possibilities, not just actualities, okay? So what I want to do very quickly is look at the beginner search where we're going from one protein to another that performs a different function, but it's structurally very, very similar. And I want to explain why this is actually more challenging than it looks. And we actually tested this in the lab where we believe that this is more often than not possible even in a billion year time frame by the Darwinian engine. And I'll do a finality. Suppose this is a message that we need to convey. I'll read it. Okay. Read it. Tell Sarah to bring Jeff and meet me at the entrance to Martin Hall at 530. Martin Hall is here on campus in the southwest corner. Suppose that's a message that we need to convey to someone. And suppose that we have access to this message which is not quite right. This one says, how David to bring Jeff and meet me at the entrance to Payton Hall at 430 instead of 530. So different person, different place, and different time. And here are the two places. So in other words, if this is what I want, even though this sentence is structurally very similar, it's not going to do the job because the person will be, it'll be the wrong person in the wrong place at the wrong time, okay? Very similar, but they're significantly different. And in fact, if you line them up, this again got expressed. They're supposed to be totally lined up. But I've colored the characters that differ between these two sentences. And if you count them up, there's a total of 11 letters that are different between the two. Now, a Darwinian search, the equivalent of the Darwinian search for this beginner problem, given this, it will go out and search variations on that theme until it comes up with this. And then your whole priest got the message that you want. And most of it, our intuition tells us that shouldn't be that hard because you only have to change 11 letters the rest of them are correct. Anyone have a guess on how many sequences would need to be searched in order for this to happen? In other words, how many sequences differ from the starting one by the same amount of system? It's phenomenally large. You can calculate it like this, and there's the answer. Five times 10 to the 28th power. That is how many sentences differ from this one by the same amount of that one. And the vast majority of them are Jewish of course, and only a very few of them would do the job of this. So that's the size of the space you have to search to solve a problem like this. And that turns out to be prohibitively difficult, even though this is an easy problem, relatively speaking. If you look at the advanced search where you have to get a new structure, we're now making it vastly more difficult. Here is more akin to this. Suppose this is the sentence that we have and we need something radically different. We need directions to the campus. Here it doesn't even help us to have this because this differs so radically from this, not even a helpful starting point. So basically we have to go back to scratch, we have to reshuffle letters, and just come up with this de novo. It does no good to start from this. In that case, we're searching a phenomenally large search space. This is literally Google. Google is 10 to the 100th power. This is like trillions and trillions of Google. Needless to say, that's not feasible. It's not feasible even on a multi-billion-year time frame. So, I've given it in a non-technical way, but the bottom line is what is claimed here of Darwinism functioning as a search engine if true would be mind-blowing. It should be mind-boggling for anyone to believe it. And if you go and test very carefully the actual numbers and you go to the lab and look at the changes that are needed to accomplish these changes in proteins, you find very clearly that it does not work in the time frame level. And I will wrap up with that simply saying that if there are people here who want the technical material behind these arguments, that there is a growing body of literature now that has it. And here are some of the key references for what I've said here. And I don't want to leave it with that. Paul Nelson has come up next and talked about what we call the developmental mutation problem. And this is probably related not so much to genes and proteins, but a problem related to body plans. And we can define a body plan as a distinctive set and arrangement of body parts, of tissues and organisms. Some different body plans that arose in the Cambrian's examples on the slide. And I'll just hand it up to Otto Falk and he can take it from there with a new and separable independent challenge to neo-Darwinism that comes from the consideration of mutation and what it can do with the level of body language. Paul has been extraordinarily patient. So I'm going to try to move briskly so we can get to Jonathan's talk and then what is, for me, the best part of the evening, which is the questions and answers interacting with you. Because I know what I'm going to say. I've prepared my PowerPoint. You know, I'd like to hear from you. Right side advances. This is a picture. It looks like what you saw in the film. I want you to keep in mind the differences in architecture. All of you have been to the seashore, I imagine. Perhaps some of you here in Texas down in the Houston or Corpus Christi. And you go to each of them and you notice that there are a variety of forms there in the tidal pools at low tide that differ from each other fundamentally in terms of their overall form. Now these groups here, the actual species represented are extinct but we have modern representatives of those groups that we can study today and some of them have provided a tremendous amount of information to biology, to the science of biology. In particular groups are a species from this phylum, from the arthropods but also from the echinoderms and other body plans represented here. So I'm going to talk a little bit tonight about the model system from the phylum Arthropoda to fruit fly Drosophila about which biology has learned in the past 100, 110 years actually a tremendous amount. And I want you to keep in mind not just those different shapes but the neo-Darwinian explanation for how those different architectures arose. Now this is textbook evolutionary theory. My guess is if you went to a evolutionary biology class here on campus and opened the textbook and looked at the explanation there for the origin of the animal phylum. This is what you would see. That the differences in form that we see in the Cambrian explosion arose by the natural selection of randomly arising small scale variation and that these forms share a common ancestor. Now this cartoon here which I've labeled Urbilateria that's the name given to this entity does not represent a real living thing. This is a strictly conceptual entity. We don't actually have any representatives of this. It's what is postulated to lie at the base of the radiation of the animals in the Cambrian explosion.  Bilateria these forms to which are extinct all have bilateral symmetry. That's their common ancestor by hypothesis and it's represented here in the cartoon in two forms. This one on the bottom has a little more detail than the one on the top. It kind of looks like a worm but it has its guts and sense organs and so forth. Some complexity represented there. Now my thesis for tonight is that biological research done within the neo-Darwinian framework over the past 30 years has actually shown that framework to be false. And the challenge for the biological community is where do we go from here. So in the next 7 or 8 minutes or so I'll show you why this is the case. Now let's go back to one of the founding documents of what is now the most widely accepted theory of evolution neo-Darwinism. Namely the publication in 1937 by Theodosius Dubjansky a great geneticist who mentored a whole generation of evolutionary biologists including one of my advisers at the University of Chicago. In 1937 he published a book with Columbia called Genetics in the Origin of Species that was one of the founding documents of modern theory. And early in the book this is page 12 so he's laying out his program here and he says we have a fundamental problem with evolution. It seems to require spans of time to which we do not have access time on a geological scale. To bring about the changes that we see in the fossil record requires a lot of time. And we don't have access to that. How can we work on the problem with evolution given that we don't have these long spans of time to observe. Well he says we're going to have to make an equivalence to push the program of evolutionary theory forward and he describes that as a reluctant sign of equality. And the reluctance there stems from Dubjansky's knowledge that many evolutionary biologists even at that time would not have agreed with him. For instance his own mentor in Russian Yuri Fulupchenko a great geneticist who coined the terms micro and macro evolution which were then brought into English by Dubjansky did not think that the mechanisms of micro or small scale evolution could be scaled to a problem like the origin of animal body plans. Dubjansky says that we're going to make any progress we've got to put that sign of equality there and we'll see what we can do with it. So here's a little cartoon showing you in a schematic form what his argument entails. So here's our starting population here with variation distributed let's say for character A around this range. Time is running this way and disparity differences are arising on the horizontal axis. Dubjansky argues what we see in macro evolution is just micro evolution summed over time where the difference that arises is proportional to the amount of time available.


It's actually Christmas Eve day when you're watching this. Well, no, not when you're watching this when I'm filming this So I'm so excited and earlier Wendy and I were shopping like maniacs to get everything I didn't really like film that but we went to guitar center. We went to public Bed Bath and Beyond our store so yeah I still haven't even shattered because like we were in such a rush that like we had to like get out of the house So I literally did not like even pick up the camera or anything like that So now Wendy is going to wrap gifts and just like last year I want to get a time-lapse of that So cue the montage All right, so Sherry got an ER today, which is awesome. She's home early. She got off at like 10 30 She said and this is the dilemma. We are facing we have a problem here We have a way too much stuff in our refrigerator. I have a question. Yeah, so what's all the beer in here for? I have no idea. I was gonna suggest we take that out. There's a problem, right? All right, so Wendy's getting all the stockings and presents situated I swear she is like Santa Claus and we have even more gifts. I swear today.  No, that's the serious gap of the game. Oh Right Merry Christmas Eve to Valerie Anyway, I just wanted to get her on camera Wendy's on her second beer Sherry's on her first She's catching up though. I'm trying to catch up. We had no more room Next year we have to decide. Did you just like one dip? I did. We have to decide just like one dip. Listen, next year we need to decide. Food or drink because we don't have room for drinks. It has to be one or the other. Bro, that's what coolers are for. Then we should have gotten a cooler. We have a cooler. And they're here. They're pairing with all the gifts. There's Deegan And then Brock's probably the last one. We have added to the present collection for the game gift Oh my goodness. This is like the most presents we've had in a very long time underneath this tree. It looks great Harry brought over some donuts because she works at Dunkin Donuts. Look how cute. Oh my gosh. They're so festive though. I love them. They look better yesterday. They're cute. They all steal donuts. We have to vlog it. So I just thought Harry had this cute little outfit on. Look at her. I notice this pin and it's so cute. Apparently it was our grandma's. Alright, so here's the feast. We have these sandwiches that I'm not going to eat But we have cheese, nut things, chips, salt and vinegar chips, dip, olives, veggies, celery, and Jerry's famous spinach dip And this is how Blankenships do it, I guess. Deegan people. Here's Perry's plate and here's my plate. I know it's very nutritious. Oh, here's Deegan's plate Come on in. Camera Deegan. There's Deegan Someone who knows the game explain it because if no one has watched last year's vlog Let's explain it again. Watch the other video and the link is below Okay, fine I don't think so there Sherry This time I'm not oldest I know you're not. She already picked Sorry She was supposed to argue with everything Quit arguing and just pick the damn number Almost Oh, anybody would cry and say Wendy was a witch on Christmas Okay, I always get one Perry has one Two Who has three Four Me Five Me Seven Oh, Sherry No, because this really isn't fair You don't even know what the new rule is You do Explain the new rule That's what I was about to say Oh, okay Excuse me, excuse me, Wendy The new rule is whoever goes first Perry Actually gets to pick what she wants at the end She gets one more point Oh, yeah She gets one more Four I don't have to choose wisely now But this one's pretty Ooh, I actually really like these What is it? Salt rock plant Oh, yeah That's what I was talking about I know What is it? A multi charger It's a multi charger Nice Very cool Thank you It's a glow charger It's a glow charger Going big, rock And it comes with six beer bottles But we're afraid of breakage in the box I want that Ah Are you two picking again? Yep, yep Or you can steal I love that Is that good? Because I really love Sherry And I want to keep the game going I'm going to steal from Perry Oh, yes I'm going to steal this Bailey would like it This is a good egg scoop We can't show you But it's just like a It's a pillow thing It unravels into a long string And you can wrap it No one wants my gift I'm ready to get wiped What is it, Valerie? It is Sergeant Art Supreme Series And it's from Hobby Lobby For $19.99 It's a colored pencil Or a pencil set Yeah, it's very interesting A pencil set That's cool So you have pencils That you can do art projects with Actually I'm going to take a beer Oh, I knew Sherry would For my Christmas present I want the light Me? Yes, you I want to see the blanket in action So I'll steal from Brock And the sexy chocolate Hello And you got white You guys just know me To keep That is a good egg skill I want the pillow And the sexy chocolate Okay Oh my gosh I'll put Alex I know No, I honestly do I want that I've wanted it since I saw it In Bed Bath & Beyond I got my Nestle cookie We don't tag team in this game Alright, sorry We're going to open them out I guess I'll steal this Oh, so we have Oh, the charger We need to like end it Dad, that doesn't have a charger It's probably time I know Say for me I know Oh Okay, Wendy Explain what we're doing with the gag gifts Same as before Link below Yeah, it's the same rule So it's basically the white elephant But we got like cheap toys And presents Instead of like the nicer items I like it This is like an all day Are we going to like pass it around?  It's an ear cleaner Oh my gosh Mother would love this Valerie actually needs that I actually need that I got 20% off on that one I was happy Is it time for five? Yes Is it mine then? Yeah Isn't that funny? Yeah What is it? Oh my god What is it? Oh, right We can wash each other's back We can not to open any of our presents That were gag gifts Because you obviously did Pick up on the first time It's not a gag Oh my god I have a serious gag of a gag Sorry Oh, yeah, alright It's not that bad So it's a washcloth Washcloth?  Thank you Don't be hostile now You can now be careful opening this one I think you should probably steal Remember what happened last time Maybe a box Valerie's like dying over here More crayola crap It says bath Tonight you're the nail buddy This on the tree You have to explain this game This game we don't know And there's no link it down below I see he's so anxious That one, see that one You can write on the wall It's in the bathroom Okay y'all ready? The object of the game Is to unwrap this giant saran wrap ball By using oven mitts Okay This is gonna take a while So the person to my left will start And I being on the right Will take dice I don't know if you wanna do this On this table Oh yeah It doesn't matter You may have Because the table looks so nice So you're gonna roll the dice So I'm gonna roll the dice And if it takes me 5, 10, 15 However long to get a double The person to my left Has that amount of time To unwrap as many gifts in here as possible When I finish And I hit a double Then it just rotates So the next person to the left Will try to open Meet the eyes Hurry Digi, hurry buddy Hurry You're getting all the good shit Harvest sauce into the skin It's shaving cream Listen, look at every single thing We're gonna be here Until like 3 in the morning Hi Cherry Oh, she has She's the one that wrapped it Oh Ooh Oh Booze Booze I thought there was just one gift in the middle Stop, booze, stop Oh Money Oh, and moon It's gonna take the gloves off Ready whenever you are Laughing too hard Okay Action You also get a soccer line To scratch off Yeah, we're walking Okay, come on Cash them in Valerie's third attempt at nothingness There you go Oh And she can do it And she can do it Everyone took a turn Bailey Did you have a good day at work? No Oh, okay Here we go now All right Mitz, unrolling He goes to my roller double Okay And Valerie has been shut down Every round She hasn't won any The prizes are rolled up I've only gotten two things I'd rather work it Does it look nice? What? Your overalls Like two years ago No, I bought this a while ago Guess what No one's gonna hear Can't tear it Sorry, can't tear it Let's see if she'll do it Okay, so I got this wireless charger Because I won the middle prize Which was a paddle ball And a dollar And I got this Wireless charger Wireless charger You got us something?  That's awesome With a bottle over on the side Yeah Oh, yay That's so parry I had to say that Because Wendy texted me And said, what would parry like the best I really need a new wallet Yay, there's card slots Can I see the front? So cute No, I do like it I need a new wallet I'm really proud It's very parry Yay Thank you I was telling dad how I like all the card slots And how I needed one with more card slots Because mine doesn't have enough And they're stacked behind each other Deegan, I promise this one is better Than the other presents you've opened At least I hope so Here you go, Merry Christmas Bailey Oh, thank you Stitch fidgets for you And there's not the What is that? Can you see they like They light up They've been wrapped in there for a while So I'm not sure Whoa, you got the zero boards? Yeah Hey, that was all me right there Yeah, they're a good time That's what they are You're a good guest All truck kids This is what I think it is This is really cool This is really cool Can we pass it around? This is so cool And do you thank you so much A little bit over The guy that made it Don't look at that That's not the person He signed it Oh, really? And I have a picture on my phone And I'm going to print it off And give it to you But it's the guy in Columbia that makes it It's a cigar Guitar What'd you get, parry? And you're stalking He's a large person They're very cute This is one of my favorite ones Oh, that's your color It is my color The life savers Also a tradition That's my name That's from Papa Yeah, I don't have a slide with me I saw it You know our business cards Yes You said you wanted that Southpaw cigar box guitars This is so cool Anthony H. Walker Very cool I like that I really wish I would have known I wouldn't have brought my slide with me That's where I like Take a picture of my nephew with it And post it for him Thank you so much Yes It's a gas card Yes That's exactly what I needed Thank you so much This is really cool I know It's like a guitar Yeah, it's an explorer Ornament You see that, dad? It's a Gibson Explorer Well, it might be more like An Ivan as destroyer It's like It's a record It's made out of a Oh, is it really? Yeah, it's made out of a record Oh, what? That's awesome Yeah She's opening it Oh my gosh I like that That's really cool Wendy was telling me about that Wendy's done a great job this year Oh, the Sour Patch Kids We remembered that from last year And the strings Is this a string club? Yes Oh my god, really? Yeah They've been trying to call me in to get that for forever I haven't been in the boy yet Thank you so much Thank you I got your thumb first I love it Thank you It's like a rewards thingy Yeah, it's awesome I've been needing to buy it for forever And I just haven't yet That's awesome And the strings Basically everything like last year But upgraded I didn't see that That is going to be it for this vlog I really hope you guys enjoyed it I completely forgot to end it Like my family didn't leave until like 1 a.m. So anyway I really hope you guys enjoyed this Merry Christmas Eve It's now Christmas Day So Merry Christmas I hope you guys enjoyed this video Give it a big thumbs up If you did Don't forget to subscribe to my channel And enter my giveaway And I'll see you guys tomorrow For the second to last day of vlogmas Because it's kind of crazy I vlog like two days in advance So that's why it's kind of weird But see you guys tomorrow Bye


All right, so we're recording. Let me go back to sharing. Okay, so you can see the screen simulation metamodal, right? Okay, great. Thank you. Last time, we did a problem with a t-test, right? A two sample t-test. That was not, I don't think it was, it was presented any difficulty for you. It's a pretty basic introductory statistical technique. I'm sure you've done t-tests in the past. I don't know if anybody here, I should have created a poll for that, has done analysis of variants. Analysis of variants is where, whereas with a t, you might have two groups, two samples, with analysis of variants, you might have, or with analysis of variants, you might have several Xs, several factors, right? Sometimes you see the analysis of variants laid out on the regression analysis as part of the output. And you've done regression, most of you have, some of you have done more intense work in regression. If you have a regression equation with several Xs, each X has a coefficient and then there's a constant term, that kind of equation can really represent just about any statistical technique you've worked with. Just give a shout out, a quick shout out if you've used or heard of the term, the general linear model. Yeah. Yeah, okay, at least I got one, yeah. So I'll assume that that means yeah. But basically the general linear model is in the form of a regression model, but it can be specifically used to refer to just about any statistical technique that you've learned, most of them assume an underlying relationship of linearity in the data and in the world. Doesn't mean it's true. Very often we violate that assumption of linearity. And in fact, most of the time in simulation models that are based on queuing systems or inventory systems or productions is, I mean, most simulation models are engineering type of systems.  So even when we do simple statistical tests like t-test, ANOVA, regression, we're assuming linearity and maybe we shouldn't be. And that's one of the things that I want you to keep in mind throughout this lecture, but we're not gonna address it until, maybe we're two thirds of the way through. This you've seen before, I just grabbed this from the experimental design lecture notes and basically when you design an experiment, you wanna make sure that as much possible information is in there when you analyze the results, that's when you wanna pull that information out that you made sure was in there at the design phase. And this is exactly what the simulation metamodel is made for. The simulation metamodel you can, until you know more about what it is, which is gonna be the next slide, but you can really think of it like a regression model. All right, here we are. If you, if we look at this as, it should really be in the form of a cloud, right? If we look at this as a depiction of whatever the real system is, there's inputs, there's some transform, some transform function, something in the real world that transforms the inputs into the outputs, the response variables. All right, and the outputs are in terms of the parameters of the system, and the real system is something we don't know. That's why it's big. It's full of complexity. It's the largest icon, it's the largest box in this picture. So this Greek letter phi is a function. It's the function that transforms the set of inputs into a set of outputs. The simulation model is built in order to represent the real system. We take data from the real system, we observe behavior in the real system, and basically we're building the simulation model to represent whatever our real system is. And the system, the simulation model is also a transform, an input-output transformation where you have some input variables and you have some output variables. The input variables are control factors. These are values you set. Sometimes they might be values you measure because you have no control over them and you just attach them to the rest of the data.  This F over here, that's a program. Okay, this F is your simulation program or your simulation model. And E is just something that we throw in randomness to model the fact that the real system has much more complexity than the simulation model. There are a lot of things in the real system we're ignoring because that's the point of a model.  What are these M's? For those of you who have had or are taking the multivariate course, M is a response variable. So you'd have a separate simulation model, you might say, for each response variable, although we all know that we don't really have a separate model for each response variable. We use one multivariate function, the simulation program, input all the factors and output the various responses. So the simulation model is slimmer and has less complexity and less randomness than the real system, but it's still trying to represent the real system. Then the meta model, which in this case, we're showing it as a linear additive model, a regression model, in other words, could be taking the same Xs as before and just putting it into the form of a linear regression model with several models, really, one for each multivariate response, one for each response. And the linear additive meta model is, again, smaller than the simulation model, which was smaller than the real system because the meta model is an actual function, it's an equation, and it's not this big model that you can't really examine all together, you have to run it. The simulation model, you could put in your inputs, you could put in your outputs, but you can't really tell how one turned into the other. When you have a mathematical formula, you have your inputs and you have your outputs and you can actually talk about the coefficients, you can talk about the relationship between the inputs and the outputs. So it would be a valuable addition to the simulation. Now, I just wanna point out here that if we call this meta model, the GLM, the general linear model, then basically what it does is it describes, it's an umbrella, for every possible statistical technique we might have done on our data. We have the real system, which we can't get at, we have the simulation model, which we build to represent the real system, we generate data from the simulation model, we use that data in either a t-test or in a nova or regression, something that falls under the umbrella of the general linear model. And it helps us to understand what's going on in the simulation and indeed in addition, it helps us to understand what's going on in the real system overall. I just repeated this again, so I don't have to keep on going back and forth with the slides. The simulation meta model is not only convenient and powerful, it helps us interpret, I mean, when we do a t-test, it helps us interpret, but with a meta model, a formula, you can actually look at relationships more concretely and more exactly than you can if you're just doing it, let's say a t-test. And it's used very, very often. It makes the implicit models, every one of those statistical tools that you learn has a model that it's based on, an underlying linear model. I've been calling that the implicit model. It just takes that implicit model and makes it explicit. So it's been there all along, it's just where we'd be working with it now, assuming that we want to have a linear additive model here. Okay, each one of these is a way of taking input and turning it into output, right? And it helps us to better understand what we do at every stage of the game. All right, we're gonna look at how this is done using an example. This is an MM1 system, which could easily be changed to an MMS system, but right now we're keeping it to MM1. There's an order taker using an old-fashioned black mobile phone. I don't know why, but okay. This is a call center. Calls come in, completed calls go out. And there might be some waiting time. We simulate the system, we validate the system, we run the system. It's a very simple simulation of an MM1 system. And here's what output data we got. We have 10 replications in each system variant. Okay, how many system variants do we have? Well, here's the design matrix that we used. Arrival rate could either be 9, 12, 15, or 18, it would two nines. Service rate 10, 12, 16, or 20, there's two 20s. And you know the basic rule of thumb is that you want arrival rate to be less than the service rate. And so these design points were chosen very carefully. Servers is here as of X variable, but it can be ignored because it's the same all the way down. And so each one of these, one, two, three, four, five, each one of these five system variants had 10 replications. The simulation ran for 10, 10's not a lot, as you know. And still we found some interesting results, even with 10. We're looking at the three basic system variants measures of effectiveness. L, the number of, in this case calls, the number of calls in the system, W, the amount of time each call spends in the system from arrival to departure to when it goes out. And utilization of the server, it's only one server. With arrival rate nine, I'm thinking these are minutes. Haven't really, I don't remember, did it say? Let's say minutes. With arrival rate nine and service rate 10, we had 8.84 calls in the system on average. With arrival rate nine and service rate 12 that goes down dramatically. Well, that's to be expected, right? And these are the standard deviations. With arrival rate nine, service rate 10, the amount of time a call spends in the system is 0.988, I'm assuming I'm still using minutes. I'm gonna just be consistent, let's say minutes. I really should go back to the original data and check it out. And here are the standard deviations. And again, this drops the time in system, drops once the lambda of the system is more, you know, less crowded. Utilization doesn't change as much, but it starts out at almost 90% for the first system, the very crowded system. Okay, so this is our simulation data, you're used to it. You know, there was a lot you could do with this data. You could even really build a regression, right? Where arrival rate is X1, I'm sorry, not a, yeah, well, that's right, these are the X's. Arrival rate is X1, service rate is X2, we could probably ignore X3. And these are our three response variables, our three Y's. So yeah, this is what I was talking about. So let's say we do that. The regression model or what you might call the general linear model might have Y equal to B0, you know, some constant plus the sum of the factors times the coefficients, that slash is not supposed to be there, and some sort of an error term. And Y could be either L, let's say Y, that's Y1, W, or utilization. Those are the three output variables, the three response variables. What's the main assumption here? There's more than one, but what's the most critical assumption and the most troubling assumption if we do it this way? The most troubling assumption really is that we're assuming that this MM1 system is linear. It's linear, that a linear additive model can represent the data in the simulation, which means that it can also represent the data in the real world. Now, no one has ever said that an MM1 queuing system is a linear additive system because it's not. You might be able to fit a regression model to it because you can fit a regression model to anything. Of course, you'd wanna test it for fit to make sure the model is a good fit to the data. I'm not even gonna bother with that because the kind of system that we have, the typical queuing type of system, I know is not, just from research, if you look at the research, you know it's not linear. It's much more likely to be some kind of multiplicative model. Let's just look at the top row here for a minute. MOE is some kind of someone, each one of the Y's is an MOE, right? So I'm calling it MOE right now. And if alpha is the constant and we have our X, X1 arrival rate, X2 service rate, we're ignoring X3, and new is some kind of an error term, then basically we're creating a multiplicative model. We can hypothesize this model based on other people's research. And we could also try it. I mean, we can do this empirically and say, well, let's try this and see how it works. We should be able to test. And it turns out that a multiplicative model works on a lot of different levels. One way it works is we can actually turn it into a linear model and regression is so easy to do. We know how to create regression equations. We don't really know how to create non-linear equations. All right, so one way to do this is to say, here's my hypothesized model, the one on the top. If I take logs of both sides, I haven't changed anything, but what I've done is turned this into a linear model. The natural log of an MOE, the natural log of one of the Ys, is equal to each of these pieces. Then remember, when you have a multiplicative model and you apply logs, you add or subtract, depending on whether you're multiplying or dividing. So you end up with an alpha factor, a beta one times a something, instead of the something being X1, it's the log of X1, a beta two times the log of X2. And then the error term, call it error term, call it log of error term, really, it doesn't matter to us anymore. Here's our change of variables, just for the purposes of the regression. The log of L, we're calling Y1. The log of W, we'll call Y2, the log of the utilization, we'll call Y3. And this is what I was showing you before in the equation, the betas, we're gonna call the, oh, this thing, the log of alpha, we'll just call it beta zero. Okay. And there's a different one for every M. And then we have X1, the log of arrival X2, the log of service and the error term. So all of a sudden, we're looking at this resulting equation and we're going, oh, hey, I have an easy thing here. I know how to do regression. And that's exactly what I did. And that's exactly what you would do. This is like the simplest kind of meta-model imaginable, by the way. The only thing is you have to test to make sure you were right about your hypothesized relationship. And we'll take a look at that. All right, here's the result of the regression, the estimated coefficients B0, B1, B2 for the response variables Y1, Y2, Y3. And of course the standard errors of the coefficients. And each one of these was tested separately for significance. And one of them was basically not significantly different from zero. If you've taken a multivariate class, you know that you can set up your multivariate regression model on a higher level, on a more abstract level or on a more overall level, then just looking at, in this case, three individual regression models, Y1 equals, Y2 equals, Y3 equals. So the Wilkes-Lambda statistic, which is really something everybody has and they have much better ones now. This is, as you could see from the typeface, this date is a little old. The Wilkes-Lambda statistic tests the significance of the model itself on a multivariate level.  But if you do, it's definitely the p-value is way smaller than 0.01. Testing X1 in the multivariate model, same thing, X2, same thing. And for lack of fit, what do we want when we're testing for lack of fit? We want the model to fit, right? Okay, we're testing for, when you're doing a fit test, you wanna see that it fits. And what you want is a very small F and a probability greater than whatever you're working at 0.05 or whatever. So all this did was a different kind of test for the model. Is the model significant? And how's the fit of the model? Remember last time we talked about experiment-wise error? And this was it exactly just to illustrate that if you have three variables, three Y variables and you think you're working at 0.05, you're not, you're really working at 0.143. And that's why the Wilkes-Lambda was done. So assuming, I know I am doing this very, very quickly, but it's not exactly the part I wanted to focus on. All I wanna say from here to here is yes, the model was good. We tested it, it was good, it was significant. So we have our equation. We have our metamodel. And the linear additive metamodel that we have are these three. And instead of saying Y1, Y2 and Y3, I went back to the natural log of L, the natural log of W, the natural log of utilization. And instead of saying X1 and X2, we have the log of arrival and the log, excuse me, the log of service. Okay, so it kind of like, what did I accomplish here? Right, you kind of say, okay, fine, I have a nice little equation. And I managed with the logs to take something that was not linear and turn it into something linear just so that I could apply regression to it. Cause I know regression, very fine. Like what's the big deal? Well, we can't really use these. Like, you wanna use regression to predict, right? So let's say we want to take a different value of X1, one we didn't have necessarily in the experiment and a different value of X2, a different value of an arrival rate and a service rate and predict what L and W and utilization would be. It's very hard to do with these logs in the linear additive model. We wanna go back to the predictive model because that's the one we're actually working with. Okay, so this is just a repeat of what we saw before. So what do we do? We take antilogs. What's an antilog? It's just, you know, e to the power. All right, so when we take antilogs cause we look at these equations, if the antilog of the left-hand side would be e to the log of L, just left with L. And you apply e to the right side and do a little algebra and we get this. And it looks familiar, right? We're back to the multiplicative model that we had before, but now we have the coefficients, the, what used to be the coefficients in the linear model. The first equation turns out to be L. L can be computed as the arrival rate divided by the service rate, which by the way, we know was just lambda, right? The arrival rate divided by the service rate raised to a power of 5.8 times e to the 2.8. That's one equation. And of course, there's always a little wiggle room cause there should be an error term. With W, what we have is e to the 2.8 times the inverse of arrival and then times lambda raised to 5.8. Utilization looks very familiar, right? What's utilization in queuing in an MM1 system? If anybody is courageous enough to unmute themselves, what is utilization? It's just lambda over mu, right? By the way, over here, what I meant to say was lambda over mu. It's utilization, that's utilization, lambda over mu, arrival over service, that's utilization. 0.97 is how it turned out from the data. 0.97 is very close to one, but the data said 0.97 and regression is very much a database technique. You can take anything and create a regression from it. But we happen to know, you don't always know this, but we happen to know because we know the MM1 system very well. That arrival divided by service raised to the power of one is actually utilization. Another thing we have when we look at the equations is you can say, you can refine L, redo L, because if you say utilization is arrival over service, right? You could say that L is e to the 2.8 times the utilization raised to the power of six-ish. And you could also say that L is equal to lambda times W. Let's see, L, let's see, arrival, if you take W, okay. I know I did this, but I should have had the way I worked it out over here. Take my word for it, it works out. We should really do it outside of class. I'm not gonna spend time doing it again now unless somebody really wants me to. But basically you can show that lambda times W, it can compute L. There's a relationship here. We can relate L and W by lambda by the arrival rate. Isn't that interesting? Well, somebody else figured this out and it actually has a name. It's called Little's Law. Now, why am I so excited about this? Let me go back up a minute. You know why I'm so excited about this? Here's why I'm so excited about this. Because if the real system is our MM1Q, right? Then Little's Law holds in the queuing system, it's actually a law that comes from an examination of the mathematical models used in queuing. That's the real system, okay? And if I skip the simulation model and I look at my data and my data actually came up with this thing called Little's Law. L is equal to lambda times W. What does that say about my whole process? That's kind of like saying, I've just validated my simulation model because I use data from the real system to create the simulation model. I use data from the simulation model to create my metamodel. But all of a sudden I can prove that my metamodel has a characteristic that's supposed to be true of the real system. I mean, it's a very, very interesting finding. It's an incredible finding. It makes you feel good about the whole simulation process, not to mention about the multiplicative model that you then took logs on and applied regression to, which you kind of say, I have no idea why that works.  So what do we benefit from this? Imagine you're trying to study the real system, you're trying to study a particular system, you build a simulation model, you want to do certain things, all of those different kinds of experimental designs and the experimental design lecture, you want to do them. You might want to estimate, you might want to compare systems. You might want to optimize, you might want to say, what's the, what value of the X's do I have to have to come out with the best value of Y? So you might want to optimize. All of those things are easier to do with an equation than with a program and with a simulation program. If you could take the simulation program, generate data according to some kind of table of design point. So you need a good design for that. Generate data, use that data to build a model and equation, no matter what the form of the equation is, you could throw away the simulation. You don't have to go back to the simulation to work with it anymore. So let's, let me just go, I don't know if these are in any particular order. Yes, I have simplified my model tremendously. I can explore and interpret the model better than I can with simulation. You look at, I can look at the slope terms. I can look at things I can't look at with the simulation model. I can optimize and I don't, I'm not limited to the system configurations that I built the meta model on, although we all know the danger of extrapolating versus interpolating. I can get a better understanding of the relationship between the input and the output, which is what we really, that's all we mean by system behavior. I can generalize to other models of other systems. If I have an equation, I could say, oh, look, this was a linear model or oh, look, this was a multiplicative model. Other systems, what do they have? And other people, there has been other research about systems, not necessarily simulations, but other systems. So once you know the form of your meta model, you can relate what you've learned to the research from other systems. You can do sensitivity analysis. It's much easier to do that using an equation than using simulation. You don't have to keep doing repeated runs. Same thing with what if questions. What if I try this particular level of X and this particular level of Y? It's easier to answer inverse questions, which I mentioned before, like the optimization. If I want a particular value of Y, what X do I need? And of course, I can test many hypotheses without running costly additional runs. How do we validate a simulation meta model? Well, we kind of saw one way in a sense, if you can come up with something like Little's Law, you've basically validated, although you can't depend on having that kind of luck all the time. You want to look at internal validity and you wanna look at external validity. Is the meta model a valid representation of the simulation model? Okay, so at the very least, you wanna say, can I prove that my equation really does represent the simulation? And you do that by doing the fifth test and testing the model. Then you wanna ask about external validity. Is this meta model a valid representation of the real world system? And here's where we may want to actually collect data and test it against the real world system or in the sense of the MM1Q, we know the true averages of the real world system. And we're gonna do that in a minute. Okay, so the simulation meta model is database. One of the, I don't know if you've studied this, but one of the best ways of testing your regression model, because remember, when you do regression, you will always come up with a model. Okay, so the question is, how do you know it's the right model? One way to do that is to take a holdout sample. You have your data, don't use all the data for your regression. Use two thirds of the data, let's say. And then the last third that you held out, use it to test, see if you can predict the y's with those x's. Okay, see if you can predict what you got already. And that's what you do with the holdout sample. Now what some people do is they just split their data in half and they use each as a control for the other. They'll develop a regression equation with one and test it with the other and then switch and see what happens. We can look at the mean absolute percentage error in terms of comparing the prediction that we should have gotten compared with what we got. And this is very, very elementary. I would hope that some of you in your regression class, if not all of you, have learned techniques like this. But with the meta model, we not only have the data we use to build the model, we also might have information about the real system. And that's what we wanna do with external validation. So we not only wanna prove that the meta model is a good representation of the simulation, but we wanna show that it's a good representation of the real system so that any conclusions that we make from the meta model can be applied to the real system because that's actually what we're studying. We don't really, we're not studying the simulation model, except as it helps us understand the real system. So you could take historical data and test the meta model by using the Xs to predict the Ys. But in our case, in this particular example, we know the real system. We know everything about the MN1 system that was simulated. So here's an example of external validation when you have a theoretical system to compare it to. Here are various levels. And notice it's not only the design points that we used to build the meta model. We have an arrival rate, a service rate. We made sure that arrival rate was always less than service rate, that much we did. Here's the analytic values. Here's the, when you use the queuing formulas and you put in the particular arrival rate and service rate as lambda and mu, here's what you get for each combination of arrival rate and service rate. Here's what you get for L, for W and for utilization. And when we take the meta model, and I'll just remind you for a minute of what the meta model looks like, it's this. This is the meta model, okay? Separate equation for L, a separate equation for W, a separate equation for utilization. And so if you take the meta model and apply arrival rate and service rate, you get these values for L, for W, and for utilization at each of these combinations of X1 and X2 or of arrival rate and service rate. So now you just need a way to, we need a metric to say something like, when arrival rate is eight, service rate is nine, the value of L should be eight, I got 8.06. How good is that? So that's what we're looking for. The measure we used is average absolute error, which basically is meta model minus analytics. So it's the deviation between the output from the meta model and the analytic long-term average divided by the analytics. So it's basically what you're looking at is a proportion of the true value that it was supposed to be. If it's exactly the same, if you got exactly the analytic value, that's a hundred percent, right? So that's what this is. It's a proportion, I think I left it here as a proportion instead of a percentage. It's a proportion of the universe of one. So depending on how far away the value of the meta model was from the analytic, and over all the system configurations in the range studied, the average absolute error for L was 0.0771, for W was 0.0772, and utilization was really good. It was 0.005. And I'm missing something. There's a way to determine what's good. Obviously, these were good. I think since they were less than 0.1, it was considered good, but I'm gonna have to try to get that and put it back in the set of slides. So if you need it, you should be able to find it in a week or two. But basically this was good. Okay, what I don't have here is how I knew it was good. Look, you know, first of all, it's way, way from 100%, right? Well, actually, no, that's not good. We want 100%. Meta model minus analytic over as a proportion of analytic. Okay, I think I'm missing something. Maybe it's missing something. I gotta get back to you. I did, it's not like I didn't review this before class, but I missed this. Okay. Basically, if you learn nothing from me all semester, you'll learn it's okay to say I don't know. That's my, that's another one of my favorite lines. This is just a summary. We've got the real system that inputs, X's and outputs the true averages. You've got the simulation model that inputs X's, the control variables and outputs the responses. And then of course, you've got the simulation model which does the same thing, but it does it in a much simpler manner. Something is overwritten and so I think I saw that. Thank you. Yes. This is looking at the residuals. And it's kind of the same thing, residual over Y. But yeah, thank you. It is overwritten. I'll have to fix that. You can see this is a work in progress, but I hope at least it interested you and gave you an idea of number one, why an explicit metamodel might be a very valuable tool, but number two, how any of the statistical techniques that you're using anyway for your simulation to analyze your simulation data are really in a way applying an implicit metamodel because they all rely, they assume an underlying linear relationship. Yeah, thank you very much. Any other errors that you find? And of course there won't be any errors, but just in case, please send them to me. I'm gonna stop recording now so that if anyone is shy, they won't mind There we are. More. Professor, could we look at slide number 10 at the Wilkes-Lambert results. So the p-value is below 0.01 for the model x1 and x2. Right, the p-value will come out on the whatever statistical software you're using, but it was very, very low. And I didn't even, basically when you look at research papers, they'll usually just say something like this. It was so small that I didn't even bother, it was less than 0.01. So that means that the model has validity. It means that the model about validity, it means that the model has an effect. It's a significant effect. It has explanatory power. Right. So it's just, I don't know what the null hypothesis is here. Have you had the multivariate course? Okay. So I'm not that interested in going into a multivariate course, but you're right. It's not a bad idea. Says tests of multivariate hypotheses. I'm gonna put it together and I'll put it in this slide. Thank you. It is a good idea. The lack of fit being such a high team means the model has, as I say, as a family model. Yeah, basically you're looking for the high P in that case. Yeah, yeah. Yeah, yeah. That's the only one that we want to say there. Yeah, the coefficient on that is zero.


The top ten highest paying technology degrees of 2023, that is what this video is going to be all about. We're going to be going over the highest paying technology related majors. All right, so this is going to be a new and improved version of last year's list. One of the major changes I made to this year's list when I was evaluating it and trying to make it even better is I added the demand at the entry level, right? Because there's some careers out there that have a lot of demand, but they don't necessarily have a lot of demand at the entry level. So it can be really difficult to break into the career and get that first job. So if you appreciate this type of video, go ahead, smash that like button like Donkey Kong and let's go ahead and jump right in. All right, so first one on the list is going to be computer information systems. So this one you can expect to make about $56,000 a year starting out and $97,800 in mid-career pay. One career path you might go down is becoming a database administrator and they make about $93,000 a year. There's 132,000 jobs available and it's growing at 10%, which is much faster than average. As a computer information systems analyst, according to Glassdoor, you'd make about $105,000 a year and if you look up computer information systems on LinkedIn, you're going to see 47,000 jobs at the entry level. So this is really good. Number nine on the list is going to be information technology management. With this one you would expect to make about $59,000 a year starting out and $103,000 in mid-career pay.  There's 632,000 jobs available and it's growing at 7%, which is faster than average. And if you type in information technology management on Glassdoor, you're going to see they make on average about $124,000 a year. And if you type in information technology management on LinkedIn, at the entry level you're going to see 46,000 results. So very good. Now information technology is a very broad subject. There's so many different ways you can go by getting into IT. It's almost like saying a technology career. But with that being said, the easiest way to break into information technology is to get an entry level IT help desk job. Now this is a job that pays around $40,000, $50,000 a year and this is typically the job that most people are going to start out in. Now the cool thing about an IT help desk job is you don't necessarily have to get a degree in order to break into it. There are actually many different types of IT help desk jobs that you can break into with just a certification. And with that being said, I've done tons of interviews on this channel of people who have broken into the tech industry without getting a degree and without having any previous experience. And what's really cool is course careers. I've talked about them before on this channel teaches you exactly how to do that. So if you want to check out a free training from them on how to break into the technology industry, I will put that down in the description as well as the pinned comment below. Next one on the list, number eight is going to be information security and assurance. So with this one, you can expect to make about $60,000 a year starting out and 103,000 mid-career pay. One career path you might go down is becoming an information security analyst and according to BLS, they make $99,000 a year, there's 131,000 jobs available, and it's growing at 30%, which is much faster than average. If you look on Glassdoor, they say information security makes about $90,000 a year. And according to LinkedIn, when you type in information security, you're going to see 141,000 results at the entry level. So that is phenomenal. Number seven on the list is going to be one of my favorite degrees and that is management information systems. So at this degree, you would expect to make about $61,000 a year starting out and 107,000 in mid-career pay. One career path you might go down is becoming a computer network architect and they make 112,000 a year. There's 160,000 jobs available and it's growing at 5%, which is faster than average. If you type in management information systems analyst on Glassdoor, you're going to see they make about $83,000 a year. And if you type management information systems in on LinkedIn, you're going to see 30,000 results at the entry level. Now this one is technically a business degree with a focus on technology, but depending on the program you go to, they're going to have more or less focus on tech, but that is a great combination. I mean, business and technology is just a phenomenal combination. Absolutely love this degree. Next one on the list, number six is going to be software engineering. With this one, you'd expect to make about $71,000 a year starting out and 114,000 in mid-career pay. If you type in software engineer on Glassdoor, you're going to see $105,000 a year. And if you type in software engineering on LinkedIn at the entry level, you're going to see 433,000 results, which is absolutely phenomenal. Number five on the list is going to be a very similar degree, which is computer science. In fact, in some countries, those two degrees are the same exact thing, but in the United States, they are separate, but they do cover a lot of the same stuff. But with that being said, with this one, you'd expect to make about $70,000 a year starting out and 116,000 in mid-career pay. If you look on BLS software developers make $107,000 a year, there is a whopping 1.4 million jobs and it's growing at 22%, which is much faster than average. If you look up computer science on Glassdoor, you're going to see they make about $91,000 a year. And if you type in the keywords computer science on LinkedIn, you're going to see 191,000 results at the entry level. Next one on the list is computer engineering. And a lot of the time, I just say get an electrical engineering degree. But yeah, computer engineering, you'd expect to make around $74,000 a year starting out and 123,000 in mid-career pay. According to BLS, computer hardware engineers make $117,000 a year. There's 71,000 jobs available, but it's only growing at about 2%. But with that being said, there's a lot of other career paths you can go down with this degree. If you type in computer engineer on Glassdoor, you're going to see they make about $73,000 a year. And if you type in computer engineering on LinkedIn, you're going to see a whopping 110,000 results at the entry level. Next one on the list, number three is going to be computer systems engineering. With this one, you'd expect to make about $74,000 a year starting out at 124,000 in mid-career pay. According to Glassdoor, they make about $90,000 a year. And if you type this one in on LinkedIn, you're going to see about 56,000 results for the keywords computer systems engineer at the entry level. And on BLS, one career path you might go down is a computer and information research scientist. They make 122,000, 32,000 jobs and growing at 15%, which is much faster than average. Now, one thing I will say about this one and the next view on the list is we're getting into that territory where these are degrees in many cases where not a lot of people graduate with them. So the numbers aren't necessarily as accurate, right? Because if there's only like 300 people graduating with a degree, that's not going to be nearly as accurate as if you have a degree that 10,000 people are graduating with. So just keep that in mind. Take this list with a grain of salt. Next one on the list, number two is going to be operations and information systems management. With this degree, you'd expect to make about $63,000 a year starting out and 125,000 in mid-career pay. And if you type in information systems operations manager on Glassdoor, you're going to see $91,000 a year. And if you type in operations and information systems management on LinkedIn, you're going to see 11,000 results at the entry level. So yeah, this is another one of those combination degrees, probably not that many people graduated with it. So I'm not exactly sure how accurate the statistics are. But with that being said, this is a great skill set to have in general are a very particular set of skills. So I don't think it would be a terrible idea to get this degree. But also with that being said, it's probably a better idea to get a more popular degree that is much more recognizable. And a lot of people know about it. Now, number one on the list is going to be information and computer science. So this one is basically a combination of information technology and computer science. With this one, you would expect to make about $70,000 a year starting out and 127,000 in mid-career pay.  They make $146,000 per year. There's 461,000 jobs, and it's growing at 10%, which is much faster than average. And if you type in information science on Glassdoor, you're going to see $101,000 a year and information science on LinkedIn, you're going to see 49,000 results at the entry level. So yeah, anything to do with information or data, incredibly valuable right now, not surprised to see this as number one. But with that being said, like I said before, probably a better idea to just get one or the other. Now, the cool thing about technology is it's one of the best industries to work in. And I did an interview with Antoine from the channel Black Heights, really cool guy. You can check that interview out right here. He works in technology and he makes over $540,000 a year. And the method he recommends to get into technology is actually course careers, which again, I'll put that down in the description as well as the pinned comment below. They have a free training, so check that out as well.


Big issue coming into this year, even with a lot of new personnel and Taylor really missing the last two games. Impressive that only four sacks after three games here. Yeah, I think our guys have been doing a good job of giving me some time. I think that we put a lot of focus and attention on it throughout the off-season. But it's really a full unit thing, right? It's the blocking up front, it's myself getting the ball out and receivers winning in a quick time so that the ball can come out. So I'm really proud of our guys and, you know, seeing some of the spring and training camp work pay off. Are you in a situation a lot of times where you're trying to get the ball, you're getting the ball out quicker, has it been noticeable for you? And also, are you like doing more of living to play another day when nothing's there and just throwing the ball away? I think that it all comes into play. You know, I think that just knowing what concepts you got, you know, are you going to have an opportunity to do down the field? If you do, great, you know, take a shot at it. If not, then, you know, find a completion somewhere and if nothing's going to be available, then, you know, get rid of the ball, throw it away in a safe place. Ryan, got the strong start to the game on Sunday. What's the challenge now in carrying it through half-time and being more productive and staying on the field longer in the second half? Yeah, no doubt we've got to come out and execute better in the second half. Like you said, we did some good things in the first half and really didn't do nearly as many good things in the second half. And it wasn't a drop-off in focus or urgency. I didn't feel like we just missed some opportunities on making some plays.  You mentioned the blocking, the O-line when it comes to the Colts defense, one of the best rush defenses in the league through the first three weeks. What stands out to you on the tape and just how much of a challenge do they pose? Not only for Derek, but the offense and what you're trying to do as a whole? Yeah, no doubt. They're really solid defense. You start up front, they have a big penetrating front. They play fast, their edge rushers are good. Their backers play really fast, whether Shaq plays or not. The guys they have in there across the board play with a lot of speed. They're long, they use their length well and they play with a lot of speed. So it makes it tough for the O-line to get up on that second level. Then secondary-wise, they don't play a ton of defenses. They're pretty simple in what they do, but they're really good at what they do. That's their philosophy is they're going to be good at a few things. And they mix it up by where they drop their safeties in and all that type of stuff. But at the end of the day, their focus is playing really good at what they do. And you got to tip your hat to them. They're doing a good job of it. Ryan, how good was it to get Robert Woods going? You kind of had something go in there in that last game. Just how important is to keep that?  Yeah, no doubt. I think Robert's done a good job the past two weeks, really, of kind of building on what he did in Buffalo and coming out and having another good game here at home last week. So I definitely just want to keep it going in that direction with him. And then Hooper is going to get his opportunities. Hopefully sooner rather than later, we'll see when they end up coming up. But obviously, a talented guy and have a lot of chemistry with him. So I'm just looking forward to those spots opening up in games and being able to get him the ball. With Woods, does it shorten the learning curve significantly when it's veteran to veteran when a new guy comes in? It definitely shortens it. Some things just don't, it's a quick conversation as opposed to having to introduce something. He's already seen it before, heard it before. We're just getting on the same page. Some things I don't even have to tell him. He just already knows and does it. So yeah, having that veteran presence in that room, that veteran leadership there, then just as far as getting on the same page with me, he's done a great job so far and it's showing. How do you like the guys to come back to you and tell you they're open on a play when you may not have gone to him? Like how has that type of conversation gone for you guys so far this season? Oh, it's good. I mean, it's going to happen, right? You're not going to see every open guy every time. You know, you're kind of going through your reads. Safety's may dictate it, coverage may dictate it, or it could just be a play dictated. So you're not going to see the full field every single play and occasionally a guy's going to win on the back side or something. So just having that dialogue of, okay, how they're playing me on the back side.  I'm getting this outside leverage. I think we have something here on the inside. You know, whatever the case may be, that dialogue has been good and it's going to be critical as we move forward throughout the season to take advantage of some of those situations that arise throughout a game and we can make mid-game adjustments. Brian, after the start to the season, how nice would it be to be able to get back to 500 at the start of October? Oh, it's huge. We want to go one and O this week. That's to go each and every week, but crucial at this point in the season to go get a win. We got a division opponent on the road in a tough place to play. So have our hands full, but it's going to be important for us to go out and play well. It's part of the reason you think, Brian, there's been so much variety in the receivings and receiving yards this year. Don't necessarily have that dominant number one receiver. Are you more likely to look broadly, I guess, at your options than maybe you were? Yeah, I mean, I think I try to find a good spot to go with the ball each and every time I get to throw it. You know, whether that's a matchup dependent or play dependent, it all kind of varies depending on what we have going on, but our guys have shown across the board that each of them had different strengths and can get open for me. So have a lot of confidence, no matter what the play call is, that we'll find a good spot to go with the ball. Were getting Derek involved in the past game on Sunday and with what Hilliard's been able to do, how good is it to know that the check-down is not necessarily a give-up play? Yeah, anytime you check the ball down to Derek or Don Trell, it's not even close to a give-up play. Both those guys are explosive with the ball in their hands. Derek had some good gains underneath, as well as Don Trell, and what he's done a little bit earlier in the season. So yeah, we're going to try to push the ball downfield and take our shots and give opportunities to our receivers, but not going to force it into a window we shouldn't and drop it down when we need to. Has the learning curve kind of come along for Jake, in your opinion, and when do you think he might be, make possibly more involved in the offense? Yeah, if Jake, I think, has done a good job with what we've asked him to do, you know, we'll see, we'll see hopefully we can get more involved as the season goes on. But right now, he's jumping into everything we're asking him to do. He's obviously shown, you know, he can make some place for us on special teams and on offense. So yeah, if he can continue to progress, then we want to get him involved and help create another weapon on our offense. You had a chance to say anything to Taylor, I guess, since he got put on into reserve? Yeah, I've been communicating with Taylor, man. It's just tough to see him go down again, you know, for the second time in, I guess, three years. Just really a tough situation with how everything went with him and why he's out and all that. I'll let him get into that, but really just frustrating situation for him primarily and just missing, missing having him around, missing having him, you know, anchoring down that left side for us. What's it like when the couple gonna be a little bit quieter now the rest of the season? No doubt it'll be a little bit quieter. You know, sideline conversations from the O-line department will be a lot less, but definitely gonna miss Taylor. What's it like when a new guy comes in, whether it's Dennis Dailey or maybe LaRave and Clark now and building trust with those guys? Yeah, you have to trust him from the jump. You know, Dennis kind of got thrown in there after being here just a short period of time, but he came in, he did a good job for us and I think, you know, he's getting kind of settled in a little bit over there on the left side. We'll continue to build that trust even more and, you know, I'm confident in what he's done so far and that he can continue to, you know, do that and be better. How much of you watch it? I know a lot of it was maybe on the other side of where the hurricane's coming in, but how much of you kind of watched the hurricane coverage up for Florida this week? Yeah, watching it, you know, over the weekend, you know, just kind of seeing where that thing was gonna end up going, obviously kind of turned into a big storm there. So, you know, thoughts and prayers go with everyone there on the west coast of Florida. Really hope that everyone can remain safe and it's not too bad as far as the east coast goes and for Lauderdale, you know, it ends up just being, I think, a little bit less than tropical winds and a lot of rain. So, yeah, thankful everyone down there is safe, but pray for the people on the west coast. Hi Teresa. Mike, you had the strong start on Sunday in the first half. Now what's the challenge of trying to get that to carry through halftime and play that well or close to that well in the second half? Well, it's always a challenge, but it's something, obviously, that we have to continue to look at in all three phases.  It'll be critical, you know, against the Colts, we've been up, we've been behind these games.  We'll have to, you know, make sure that whatever, whatever we're doing early, if it's positive, we'll have to continue it. And then if not, you know, make sure that we're battling to get back in the game like we did last year. And, you know, if we do get up, it's gonna be a struggle to be able to close anybody out. What would you say are some of the keys to you guys being able to get the rushing attack going? Well, I mean, it's no different than what it is. It takes 11 guys. It takes us, you know, playing with fundamentals, playing with pad level, playing with, you know, good, great technique, breaking some tackles, you know, everybody getting to the support players and making sure we're working to the right guys. As you watch the film, do you feel like you're getting closer to one of those long runs by Derek Hemmle?  We've had some, you know, we had some runs that were really close. We've had some really good official runs that had one that got us down to the one yard line. We broke through, we've gotten on the safety a couple of times. So confident that if we can continue to do some things that we were doing, you know, the other day that'll be a challenge. This is a great defense, fast, you know, they run the football. It hasn't been easy to run the football on them. So, you know, we'll have to, it would be great this week. For defense, it is Colts coming in with third best rush defense. You mentioned they're fast up front. What are some of the challenges they bring? Well, they're athletic. I mean, they have a very consistent scheme. I think that, you know, they don't, they don't really, it's not like they're trying to trick you.  You know, Buckner and Stewart are the leaders in there that, you know, they can move. They can stay in their gap. They can play with their hands. They do a lot of different things. The edge guys are active and fast in a tough match up. And then, you know, just really inside the linebackers are fast. They tackle, you know, it's gonna be critical that, you know, we try to get on them if you don't put a hat on them. You know, they'll be there and most likely, you know, trying to hammer the football out. What makes your title kind of unique? You fit a lot of good running back. What makes a title unique? Well, speed, breakaway, speed, toughness, but also patience to dip in and kind of burst out. You know, we're all at the point of attack where we're gonna have to have a great wall. You know, can't give them a whole lot of seams and, you know, tackling him one-on-one is not where you wanna be all afternoon. See, as close a challenge as there is to like Derek Henry right now, like you get a sense of what other teams deal with against you this week? Well, I mean, I don't really like to get into comparisons. They're a different type of back, you know, they have an obviously different running style. But that's the challenge this week is what they'll try to do and how they've used him in the different, you know, schemes, whether it's under center in the gun, inside zone, outside zone, you know, gap scheme. So, you know, there's a lot of different things that they present.  What's kind of the best strategy there? Well, I think you have to stay patient.  How much is Danico's ability to slide out to the edge, helped you to offset some of the depth issues you've had with injuries at outside line? Well, his, you know, his versatility and really just his attitude and presence here since he's been with us, has been outstanding. It's, you know, a player that we've always had a lot of respect for. Love coaching him, love watching him compete and help him, you know, he's factored for us and he can move inside, you know, do a nice job on the edge. So, you know, that's a great skill set to be able to do both of it for us. What's your assessment of where Ryan is with this group of receivers at this point?  We've had some passes that we'd like to have back. You know, we'll keep improving and trying to get better. But I think that it's, you know, there were some really nice things that we did the other day. Unfortunately, we threw, you know, we threw the interception, Ryan threw the interception and, but, you know, there were some X plays in there. There were some times where we had to, you know, pick up some big third down conversions. And so, you know, we're just looking for consistency.  What are your thoughts on the way Jelani Woods emerged, especially as a red zone target? I know he was the guy that you guys had some interest in coming up. Well, I think that as you look at that last statement there, it's around, we have interest in all of them.   And again, it's just the catch radius is a lot bigger when you're looking at a guy that size. Some of the factors regarding... The tackled numbers are pretty good right now. Is that a good thing, or does he need to break up a few more passes along the way? He's gotten a lot of the, you know, I mean, you look at some of those and again, those can be misleading, I think. I hear where you're trying to get to with as far as like the, you know, catch tackle and the guy just keeps throwing.  And so, you know, every DB and everybody that we have cover needs to try to cover better. But I would say that his production in there has come from, you know, triggering in a run game or whether it's a screen or, you know, a vice tackle, they catch it in front of you. So, you know, we'll keep working with Roger and his versatility. What's your like about Schobert? I know he's only on the practice squad, but what's the challenge for a guy like that who hasn't had a full camp or weeks of practice as well? He's played a lot of football. He's played a lot of football inside, inside linebacker. He's an instinctive player and we'll see where he is at the end of the week and how he can help us just like everybody else that we asked to, you know, prepare like they're gonna be a starter this week. A few more guys, just got here. Mario Edwards and Raven Clark, what do you like about those guys and how important is them to get up to speed as quickly as possible?  That's kind of what you target at this point in time in the season, you know, with guys that have been, have played, you know, and maybe in different systems. So he's working hard with the coaches. It was in here the last couple of days since we signed him.  So, you know, we'll see if he can help us. Beyond just blocking better, has there been any other factors that have contributed to the past protection being better as Ryan getting the ball out quicker or is he just throwing it away when he knows there's obviously nothing there? Are there some of those things coming into play? We're just trying to do the best each and every week. You know, we've played some pretty good players, you know, so just doing different things and, you know, in protection and with the route that we have, knowing that you don't have all day when, you know, we've played some of the guys that we've played. Are you able to get to 500 after the start to this season? I think that's just all you're looking for is just try to get, you know, back to the surface. You know, it's obviously the next week, it's a challenge, it's on the road, you know, first division game. So, you know, it's going to be a lot of work this week. You know, we're going to have to put as much into it as we ever have. I think that the guys are excited, you know, for the challenge and making sure that, you know, we understand who these guys are. I have a lot of respect for this organization, for this team.  So, you know, that preparation has already begun.


Good morning. This is my final opportunity to speak to you about the Student Success Toolbox Project. This was a project with four institutional partners, DCU, Manoos University, DKIT and IT SELIGO. This project has created a suite of digital readiness tools that program teams or institutions can use to facilitate flexible learner transition into higher education, focusing on the early stages of the study life cycle. To list the key outcomes that this project had, we sought to examine the literature in this area. Second, we sought to investigate what existing tools were there internationally in this space. Three, we sought to then use that information to create a suite of digital readiness tools that could be shared with the sector. And four, finally, to provide guidance on the use of those tools and disseminate the information around those tools and around effective flexible learner transition as widely as possible. With regard to showing how we have achieved outcomes one and two, last May we released the Lost in Transition Report, which detailed both our analysis of the literature and our analysis of the existing tools and use internationally. And we've made that report available through multiple channels. In relation to outcome three, we now have our eight digital readiness tools, which have been produced based on what we learned from our analysis of the literature and the existing tools out there. And importantly also from feedback obtained from existing flexible learners at key stages of tool development. With regard to demonstrating achievement of our fourth project outcome, we have released a guide for supporting flexible learner transition through the project website in order to give those seeking to use the tools information on the tools themselves and key advice on effective flexible learner transition. We have disseminated information about the project through our project website, through Twitter, and through over 20 different conferences, workshops, and different events. I now want to move swiftly on to the impact made by the project starting with its reach. We have used our project website to disseminate information relating to the toolbox. The website has had good traffic since it was created with nearly 900 people spending an average of five minutes exploring the website. In terms of location, we've had people logging on from lots of different countries, the US, the UK, etc. But focusing in on Ireland since the project's creation, we've had 1,300 hits on the website with 60% of those involving people exploring multiple pages on the website. We've also used the project GitHub page to share six of the tools developed. Our Twitter account has been very useful in disseminating information in that when we were releasing information around events or releasing documentation or releasing reports, when we did that, individuals or large international organizations, when they retweeted or tweeted about those, that was having a great ripple effect for us. For example, the bold research bank tweeted about our Lost in Transition report, and they have 1,800 followers, so it was a way of putting something out there through Twitter and seeing it ripple out into the community. The project has featured in Contact North's online newsletter in their Pockets of Innovation series. This newsletter has a circulation list of over 11,000. In terms of competitive peer-reviewed awards processes, after presenting at the 2016 IMS Learning Impact Awards last May, the project received an honourable mention. From a field of 53, we have been selected as a finalist for the 2016 E-Learning Excellence Awards. We will make our final presentation for that in two weeks' time at the European Conference on E-Learning. Being a finalist means that our submitted case history for the project will be published in a book, and that will be an important dissemination for the project. Finally, the project is a recipient of a 2016 Effective Practice Award from the Online Learning Consortium, and this is a particularly prestigious award. The Online Learning Consortium is the largest US-based institution and professional leadership organisation focused on integrating online education into the mainstream of higher education. We will receive that award at the Online Learning Consortium's International Conference, which will have about 2,000 delegates in November, and that includes an invitation to present on the project to the delegates. We have disseminated the information relating to the project at as many different events as possible. For example, at these review panel presentations, we've had five poster presentations including at the National Forums, National Summit for the Enhancement of Teaching and Learning last December. We have 12 conference publications, two project workshops, and so far one awards presentations. In three of those categories, we will have more events upcoming between here at the end of the year. The second aspect of REACH that we've been asked to talk about is around dialogue and discourse. We have saw every opportunity to engage colleagues in higher education about the project and about the importance of supporting flexible learners, transitions into higher education. We've done that not just at the more active discussion-based activities like poster presentations and workshops, but at conferences and other events. Even if it had nothing to do with our project, we would end up invariably talking to people about what we were doing and why they should be hearing about it. Through the dissemination events and the dialogue we've engaged in, that's had an interesting byproduct in that we have had expressions of interest from a number of international colleagues about them being interested in utilising the tools. So just three examples of colleagues from institutions where that has happened. The University of the South Pacific, Charles Sturt University in Australia and Alberg University in Denmark, there are more but time. The other important stakeholder group with whom we engage in dialogue were flexible learners themselves. During the initial stages of tool development, student feedback influenced the process at two stages. The first was when we had our first storyboards for the tools and the second was when we had our first prototypes for the tools. We got student feedback in on the tool development at those stages. When we then moved on to our pilot evaluation stage for the tools, that involved flexible learners providing feedback on the tools, through surveys and focus groups. This feedback was then used to make final changes to the tools before they were released as OERs to the sector. There is a document relating to the student feedback from the pilot evaluation stage on the project website. With regard to the impact on teaching and learning practices, staff and the project team, what we can demonstrate are the competencies developed by the project team in the area of developing digital tools, developing digital readiness tools, as evidenced by the suite of tools that we have created. The other thing we can demonstrate is the development of competencies relating to developing a socialization strategy and practically implementing that strategy, and I'll talk about that a little bit later in the presentation. We can also show impact with regard to increased institutional collaboration, principally through the project itself, with the four institutions coming together to co-develop the tools, but also in that colleagues from DCU, ITTRELE and IT Sligo are presenting a joint paper on our transition projects at an international conference next week. In terms of the project's impact on learners and learning, while this project has been primarily focused on developing tools for the sector rather than directly focusing on learners themselves, we have some early indications of how learners will be positively impacted by use of these tools. As I already said, we obtained feedback from flexible learners at early stages of the tool design and prototype production of the tools, and that feedback was mostly positive, but where we received constructive criticism during those stages, it allowed us to make further changes to the tools before we released them. The other indication that we have that learners will be positively impacted by the effective use of these tools is data we have from the soft pilot of our Head Star Online First Steps to Flexible Study MOOC. This course is a tool in its own right, but it also brings together customized versions of six of the other tools and is built according to the guidelines that we produced in our online orientation tool. Our pilot of the MOOC ran from the 15th of August to the 19th of September on a new MOOC platform, Academy, built by Moodle HQ, but it was very much a partnership with us in this instance because this is the first MOOC to run on that new platform. We had 150 people in role on the course, and 50 of those went on to receive a certificate of completion at the end. Those selected to take part in this small pilot were made up of prospective learners who were intending to take up flexible learning courses in this academic year, and also approximately 70 learners from a German organization, Kieran. They support refugees in accessing higher education. As I said, the Head Star Online course incorporates six of the other tools in its structure, along with additional text and video content, Moodle activities, and discussion forums that were designed to try and encourage socialization between the MOOC participants. The feedback received indicates that a course like this that uses our digital readiness tools has a positive impact on prospective flexible learners when asked if they felt more prepared to become a flexible learner after taking the course 12 of 25 respondents agreed that they did and another 12 strongly agreed. The feedback also indicates that the tools do what they were designed to do. For example, the MOOC participants perceive that the time management tool helped them feel better able to manage their time. At the end of the course, eight of 25 respondents agreed that they felt better able to manage their time and another 15 strongly agreed. To return to the impact on the competencies developed by the project team and also talk about the impact on organizational practices and systems, we now have a number of ongoing implementation projects or implementations of the project. Colleagues in IT Sligo have done great things with the tools in that they have customized them and presented them to new learners in an online learner induction course in Moodle, which has been designed using the Toolbox Projects online orientation tool. IT Sligo have also placed these customized tools on their website for new learners. Maneuth University is also developing a website that will house their customized versions of the tools with their adult returning to learning students. In DCU, besides the creation of the MOOC, we have brought together a number of the tools into a Moodle course for our new off-campus students. There is a need recognized at national and international level and emphasized again in the recent action plan for Irish education that we need to bring more flexible learners into higher education. And when we do that, we don't want to lose them in the first few weeks of their first academic year. So this project has been focused on giving tools to the sector that can be used to try and chip away at those barriers to flexible learners coming in and staying in higher education and going on to be successful. And that is our contribution, that is our impact on building a culture of enhancement. Sustainability is a key focus for our project and that is why we develop the tools as OERs so that they can go on to live and breathe beyond the lifetime of the project. They've been created as OERs with a CCBY Creative Commons license such that they can be shared as widely and used by others in as many different ways as possible in order to encourage further innovation and development. Finally, to give feedback on the project process itself, if we were to give one suggestion for change, we would put forward for consideration the evaluation model where one evaluator would be assigned to a project for the lifetime of that project so that they can kind of build up a relationship with that project team and maybe take on a kind of a critical friend role as the project evolves or gets rolled out. One point for improvement would have been beneficial to have the project impact report that was provided in connection with this presentation at the very start of the process so that the design of the project and the criteria in the impact report could have been aligned in the project design. Finally, in our positive feedback, what we really want to emphasise is the fact that this 2014 call focused on this key area of transitions in higher education has allowed us to do this project, has allowed this project to exist, has allowed us to try and put useful tools out there into the sector that can be used to facilitate flexible learner transition. And we would like to sincerely thank the National Forum and the review panel for the opportunity to do this work. Thank you.