Live from New York City, it's theCUBE at Big Data NYC 2014. Brought to you by headline sponsor, Juan Disco, with support from EMC, Mark Logic and TerraData. Now, here is your host, Dave Vellante. Welcome back to Big Data NYC, everybody. I'm here with Jeff Frick. We've been going two days Thursday and Friday here at Big Data NYC in conjunction with Strata Plus and Duke World. Joseph Suresh is here. He's with Microsoft and the Machine Learning Group, which is inside of the cloud group, I understand. Yes, yes. So thanks very much for coming on theCUBE. Thank you very much. It's a pleasure to be here. So you've been at the event. Hopefully traffic wasn't too bad getting over here, but what's it like over there? What's the traffic at the Javits Center? Very, it's amazing. I believe the conference has grown by a factor of two. You know, just from February of this year, it's amazing to see the traction, Big Data and analytics is getting. Now, have you done other Hadoop worlds before? Not give a keynote, but today I did have a keynote, but you know, Microsoft has always had a presence at these conferences. Oh, sure, right. But you personally, this is your first one? No, this is my first one. So we're celebrating five years as Hadoop world. Mike Olson was here earlier on theCUBE and we were reminiscing about it was probably eight or 900 people. I think the very first one was 400. That's right. And then the second one we had theCUBE at, so it's really exploded. And a lot more enterprise people are coming over. So what was your keynote this morning? Yeah, the keynote was about a new data science economy. And the point I was trying to make, you know, while software developers have had the app economy, where software developers can go publish apps, self-service, and you can have huge wins like Instagram and WhatsApp, data scientists, you know, in spite of being some of the smartest people and what I'd like to think of as some of the smartest people around, me being a data scientist, by the way, they haven't had an outlet for their creativity. Now, data scientists don't build software apps. They build machine learning models. They build analytical systems. They build visualizations. They don't have an outlet to monetize these. And what I was talking about is about creating one such like that, right? So what's the equivalent of an app store for a data scientist? Well, I believe personally that what you need to do is package up machine learning models and visualizations in cloud-hosted APIs. And then you have an API that you can use on the cloud as a service, it's a web service. Well, then you have something that people can pay for and buy. And so what I was talking about was I took this example of a website that needs to integrate recommendations in. You know, building these recommendations is a hard analytic job. It takes a data scientist to do that. But then there is another part that's equally hard. It's to build an API that hooks into that web page and then can handle the volume of traffic and reliably serve recommendations with every click. And that's a very hard part of the job. And companies that do it today, like my former company Amazon, they have a lot of effort into building some of those. Well, with the cloud, we can simplify these things dramatically. So I took that example and I showed how on the cloud with the machine learning product we have, you can build a recommendations API, publish that into a marketplace where you can actually start charging for that by usage, by per click, and then anyone, you know, any particular, practically any website could very easily with a little bit of JavaScript integrate that API in and get recommendations live in minutes. And so now you have something that is a seed of something that can be traded in a marketplace that have intelligent services on the cloud. And so we have this great product that we launched. In summer called Microsoft Azure Machine Learning. That allows you to now publish APIs that now can also be published into a marketplace and people can now start charging for it. And that's hugely empowering to data scientists. Well, the recommendation piece is quite interesting, right? Because that, you know, reputation, recommendation, you know, our social graph is now cutting across all kinds of activities that we do. I want to, I have a question about, from a data scientist perspective, it seems like historically, industries build expertise within their domain. Retail people have retail expertise, financial services, insurance, manufacturing. We sort of stay there for 30, 40, 50 years, retire and that's all good. Data scientists seem to not want to get put into that vertical bucket. Is that a perception that's correct? Well, there are lots of data scientists in financial industry and so on that have actually grown up and have accumulated a great amount of domain knowledge. Now, of course, there are lots of people who cross industries. By the way, financial industry in many ways was the birthplace of a lot of data science, really, credit scores, fraud detection. My first job 20 years back was credit card fraud detection using neural networks. So that was my, it was a very interesting thing to do. So- But expensive. Yes, of course, yeah, yeah, yeah, absolutely. But I think data science hits its magic when you combine the magic of software and machine learning and all of these algorithms with domain expertise. But we're actually taking a track to leverage those kinds of domain expertise as well to create a long tail selection of a lot of different kinds of machine learning APIs in the marketplace. See, the thing about these marketplaces, what's great is instead of just having this one product that it tries to fits all, you get a lot of specific things. You might get a forecasting API for forecasting Wall Street time series and another one for forecasting, hey, my sales, forecasting something else. There are lots of variations and when they're customized to individual domains, they become very, very useful. Yeah, so that's, I guess, my point. I didn't put a frame recovery. It seems like data scientists wants data inputs from just outside that one slice. That's why I've brought up the example of the social graph. There's other data inputs from, maybe it's other industries, other sources, as opposed to sort of the historically having data just from that one slice. That I completely agree. And so that is a new trend to see. It's an art. Yes, it's an art. And I wonder if you could talk about that a little bit. It's an observation that we've made that is, I think. Let me just take an example of forecasting demand on an online store, for example. You have products, right? Well, I mean, previously people would just take, hey, how much that product sold in the past and then try to forecast out that demand. But today you have so many signals. You have tweets, for example. How much people are tweeting about those products? How much is purchases of those products being shared on the social graph? You have then search query signals. How much people are searching for these things? So you have so many other signals that you can now incorporate in. And a lot of this data comes from different sources in the cloud. And if you're a data scientist running, say, machine learning in the cloud, like with Azure Machine Learning, you can pull in all of those kinds of data sources, integrate that in the cloud, and build these really accurate, fantastic forecasting models. And that adds a lot of power. So it's interesting, the machine learning piece was within Azure. Yeah. Talk about that a little bit. Microsoft's culturally, Satya Nadella, from the top down is really moving to the cloud and embracing the cloud. Your background is Amazon, which is quite interesting. Talk about that culturally a little bit. Well, even culture from coming to Microsoft. You know, by the way, Satya recruited me to Microsoft and my passion has been machine learning. So I came over there to build this. In many ways, sort of I'm betting my career on building out this machine learning marketplace and making it a big success. It's actually a new Microsoft, really. I mean, Satya and others have really changed how Microsoft actually really sees itself. The challenge of mindset, we are moving much faster. We structured ourselves to actually ship in much shorter cycles. So it's changed from, for example, shipping software in the traditional way, shipping it on a disk, package software through these channels to now shipping things in the cloud. And these are live services that requires 24 by seven monitoring. And the great thing about cloud services is you can take customer feedback live as opposed to these multi-year cycles for enterprise products. You take customer feedback live and the product improves constantly, right? So building these products that constantly improve and become better for the customer, that requires a sort of a new DNA and a customer focus and a way of thinking. And that's really happening under Satya's relationship. And that's really what excites me all the time. I mean, this is an outside observer. I've seen a dramatic change. Yes, absolutely. It was almost like there was some resistance and then all of a sudden that resistance broke through like a dam breaking. And which it's interesting to note, I think today's companies are much more open to change at least recognize the need. Do you agree with that? Well, perhaps, I mean, certainly engineering companies like Microsoft, if you look at the engineers, they know technology around them is changing at such a rapid pace. At their heart, they really want to move with the technology. I mean, no technologist wants to be outdated. I mean, we're geeks, we play with stuff. We love the latest stuff that is coming out. We really want to really move fast, right? And so in some ways, perhaps the change was not necessarily unexpected. It was just a leadership setting, the orientation, right? And then the engineers gravitating to what naturally comes to them in many ways. And perhaps that's why Microsoft seems to be changing quite fast, as opposed to perhaps people with not as much of an engineering culture. But I think Microsoft is adapting and moving very fast. So it's top-down and bottom-up. Yeah, absolutely. At least certainly the engineering folks have always felt this angst about they really want to win. They really think, you know, and they should be at the cutting edge of technology. They are used to creating the path and now they feel not like others are creating the path and they really are hungry to win. And so they're eager to learn. They're eager to have the customer focus. They really want to adapt, I think, and with the new leadership, it just became really easy.  So Joseph, talk about this kind of two sides of the coin where you're a data scientist, you've got a lot of data you're disposing of. You're building algorithms for people to be able to rent and use and leverage. So you're smart. At the other end, there's never enough data scientists. Everyone complains there aren't enough data scientists. I can't get them to come work for my company. And at the same time too, like Tableau's Message, another great Seattle company, they want everyone to really use analytics in their decision-making process and they really always kind of go back to, you know, how Excel-like can we make it? Because that's everybody's a big data tool to jure. How does those things kind of come together where you leverage the power and the expertise of really smart guys like yourself and the algorithms and things that you can build. But at the same time enable everyday line people to be more data-centric in their decision-making and to apply some of this to make their lives better and their businesses better. Yeah, now great question. You know, I think, Jeff, the answer is you need finished products that everybody can use. And in one of my sessions earlier today, I made this example.  Okay, I mean, in the olden days, people would buy clothes and go to tailors. And it's sort of like the tailors are like the data scientist today, maybe.  And nobody does that anymore. Guess what? I mean, manufacturing clothes became so automated and so easy and there's such an incredible selection of clothes available in all department stores that you can actually go to department stores, select the sort of things that you like and by and large it really works well on you, right? And so you need to create something like that. The finished shirts that I can buy and what's equivalent of that for data scientists? So if I am a company that wants a recommendation API, you know, I should be able to go to a store and there should be a thousand recommendation APIs from which I can select the one that suits my need the best. So I shouldn't have to really hire data scientists or have a tailor equivalent, right? I shouldn't have to have my own tailors or my data scientists. I should be able to go to a store and actually get what I want, right? And so now the question is, how do you create a million APIs that are finished products in the cloud, right? So I have enough of a selection, enough of a diversity that vast majority of people can get what they want. I think that's the way really we need to go and instead of making tailoring more, well, you can go two ways, right? You can make tailoring simpler and you can empower them to, you know, that's an Excel like thing. That's one way to go. Or you can say, hey, I don't really need tailors that many anymore. Tailors are in short supply anyway. Why don't we just get mass manufacturing of clothes? So I can just get the selection I want, right? And what's the monetization model? You've mentioned that before. It's by usage. So if it's a recommendations API, the recommendations API is used per click and for a certain number of clicks you charge a certain dollar amount, right? So just to pay by usage. Right, and I can cap my usage. Yeah, utility model. I'm in control. Exactly, yeah. And then I have to pay only what I use. I don't have to, I mean, otherwise you have this lots of fixed investment. In this particular case, you're just subscribing to an API and you pay for what you use. Recommendation. I mean, that's a very revolutionary model. Well, it's such a great example, right?  You know, and it's just, you can even learn to decode the recommendations and you can build your own system as a user. Exactly, and yeah, by the way, I don't think there is one recommendations algorithm or one recommendations API. I actually think there should be a thousand, right? There will be a lot of diversity in the kind of recommendations people want. I mean, I say, you know, recommending wine is not the same as recommending a book, right? It's just different in character, right? And people who know wine should create a recommendations API for wine. And people who know book should create a recommendations API for books. And people who know clothes should create recommendations API for levels. But this gets to the first question that I ask in your response. So it's the domain expertise, but the fundamental code, in this case, can traverse domains and then be applied if you want to tailor the suit. Great, you know, but this base code can go across any industry, for example, or any use case. Yeah, exactly. That's very powerful, that's different. The data becoming a transport. That's right. The horizontal transport. Yeah, and so what we want to create right in the cloud is this factory for intelligent services. Think about it that way, right? It's like the clothes factory, right? And so I've created a platform, the Microsoft Azure Machine Learning Platform, and the marketplace to publish it into. So this should be the factory in which data scientists come, create a large number of intelligent services that then appear in this marketplace. And that's a store where you, as customers, ordinary people would go, get those APIs, connect it into Excel, if that's where you want to use it, connect to a website, if that's where you want to use it, and you just go. And that's how economies take shape. Well, right. And why would anybody develop that in-house? Why wouldn't they just move up the stack and focus on their business model? Right, right. I mean, exactly. So you should hire a tailor when you have a specific need, when you're gonna have a wedding, then you spend the money to get your clothes tailored. Rest of the time, you consume what's out there, and they work by and large for you. And in any way, the business is so dynamic these days that you're better off tapping into that as opposed to these slower cycles of billing everything from the ground up, at least for the vast majority of companies. Right. Joseph, another thing I want to get your kind of 101 on is, again, smart data scientists dealing with a lot of data. But then there's the human factors of, and people talk about visualization, but I mean, how do you make a billion of anything really cognizant to me to be able to make a logical decision based on such a big data set? And how are kind of the human factors being now applied to machine learning and data science to let regular people actually be able to make cognizant decisions based on a billion data points? And you see some of these visualizations, and they look pretty, but it's like, well, how's that telling you where I'm supposed to go with my next decision? And to tell you the truth, I mean, I don't think beyond a point these visualizations are all that useful. I think you do need good hypothesis testing, and you really need the ability to understand scientifically whether there is a nugget there or what you should be seeing, because data points are like stars in the sky. You can see anything you want. You can see what more visualizations you have. You can see all sorts of constellations. And in many ways, sometimes, in fact, there was a wonderful speaker today from NPR at Hadoop World who was talking about how data can be used in so many misleading ways that you can substantiate your position, anyone's position using data.  And how to lie with statistics, right? Yes, exactly. I always thought that was Disraeli, okay. No, I just repeated that Mark Twain. Yeah, I didn't realize that. Disraeli ripped off Twain, okay. Like Kennedy ripped off Cicero. That's not that. And so that's the thing about pretty pictures, right? You can make pretty pictures to substantiate any viewpoint you like sometimes. Right, right. And the other thing that I think is interesting, we had a great conversation with a professor from Cal who's a mathematician. And he talked about mathematics and statistics is very different than computer science because everything is wrapped with an error bar, right? It's a confidence level. There is no yes, no, right, wrong, binary switch, one, zero. It's all probability with a confidence level and the two of those worlds are now crashing together in the paradigm of data science and actually trying to drive decisions based on correlation, not necessarily causality. That's absolutely true and that's so absolutely true. And like, there's one example, again, in my in the past when we used to do forecasting. Let's say I forecast that your book, Jeff, is going to sell 20 units next week. It's never going to be 20 units, right? It's going to be, there's going to be error bar around it. It's going to be a variance, a distribution. And in fact, somebody if a store were to, bonds and all were to stock only 20 units, maybe 50% of the time they'll be out of stock because there's an error bar around it. What you really need sort of is that distribution of the error bar so you can stop to the level where you maybe sell out only 10% of the time, right? So that probability is actually an integral part of this whole thing and it's incredibly important to use that as well in making decisions. And we're just getting used to making decisions based on probabilities. Most people find it an uncertain. Right, just the big database decisions, right? Instead of cut decisions, that's still an evolving process, I think. That's right, yeah. Let's say there's some college students in the audience and they want to know what makes a good data scientist? Where should I start if I'm interested in data science? Yeah, I know that's a great question. I think if you are a good data scientist, you really have a curiosity for unearthing patterns in the data. You really love munging the data, but you have, you're really good at coming up with a lot of hypothesis. And you know how to test them. You have lots of ideas, but you don't know that any particular hypothesis is right. You come up with ways of testing hypothesis. And then you come up with ways of deriving patterns that in some way predict the future better. Like, what are the patterns that are truly predictive of what will happen in the future and what's noise and you can separate them and so on. So that skill is what you need to develop. But then you need a lot of tools for it. You need to know statistics.  You need to go learn Python or R. R is the language of data science today. And you need to know a bit of SQL because you typically need to go to databases and extract data. And then you need to know how to pull data from the web. So learning a collection of these skills and then being able to derive insights from the data, that's really what a data scientist does. But you know, to students, look, this is actually not any harder than AP statistics. Data science and machine learning, I mean, they have this aura of being very new. But, you know, reality is it's something that is actually a lot simpler than many AP classes that you might take. By the way, I'll tell you another story, you know, like some of the earliest things in data science, like the least squares method, was invented by Gauss at the age of 18 in 1795. And he was trying to see patterns in the stars. Tell me more about that. Yeah, he was trying to see patterns in the stars. So the big data they had at that time were how planets were moving in the sky. And so he had to fit, you know, mathematically and find out what the patterns in the sky were and how these planets were moving. So he invented what was called the least squares method at that time at the, you know, tender age of 18. And that's how, you know, in some ways data science started. How important is it to be unbiased? Well, you have to be a scientist. I mean, everybody has to be unbiased, right? But data science is a scientific method. The goal is to apply science to data.  You have to be objective. You have to be experimenting. You have to discover insights by experimentation. And yet, you know, when you have a hypothesis, you have to be equally on the side of proving it or disproving it, right? And that's what data scientists say. I worry about data science abuse, data scientist abuse. Somebody comes down and says, I want to show this. Find me the data that shows, you know, A is related to B. But it goes back to what we said. You can't find the data that supports a point of view, right? I mean, you can absolutely do that. So I don't think there's such a thing as pure unbiased. Now you can certainly test a hypothesis and then accurately decide whether that hypothesis is proven or disproven. And even again, that's a probability too, right? Point of 5.41. But you know organizations will abuse that capability and try to, you know, coast the data scientist. Even sometimes when you have two political parties, each party can use data to substantiate their point of view, however they like. And it's actually sometimes very hard to disprove, right? And so that is the world in which we live. So we shouldn't take data for granted that data is the truth. Because at the end of the day, it is the derived hypothesis from that data. And who derives it? And what their original point of view is, that's coloring the interpretation at the end of the day. So you do have to be careful. The data doesn't lie, people do. Joseph, thanks very much for coming to theCUBE.  Nice meeting you. Keep it right there, everybody. Jeff Frick and I will be right back right after this word. This is theCUBE.


What's going on, everyone? Sam here with Layton Sports Cards ripping 2023, Panini-certified football hobby 4 box break number 47. Pick your team style, names and teams, Brett W. Eagles commanders, Brett T. with the Saints, Bruce B. Raiders, Bud H. Jets, Charlie P. Seahawks, Chris S. Bears, Christian W. Cardinals, and Patriots, Corey G. with the Packers, David M. Niners, Eric J. Bucks, Frank S. Jags, Frank S. T. with the Jags, Frank S. G. with the Panthers, Heath F. Cowboys, Jeremy H. Chiefs and Vikings, Joseph M. Bills, Keith B. Ravens, Ken W. Falcons, Lyons, Kenny S. Texans, Lawrence T. Dolphins, Mark T. Giants, Matthew B. Browns, Michael V. with the Chargers, Nathan F. Steelers, Nick B. Titans, and Colts, Pascal P. Bangles, Stephen W. Rams, and Todd C. with the Broncos. So good luck here, everybody. Let's do it. Suede-ified. Thank you again for getting all the masterwork filled up. Appreciate that, guys. That was huge. That was huge. That was key. That was clutch. George Kittle, Orange, Devon Witherspoon to 149. Base, base, and base. What do you mean, Brian? For what break? Marv Levy, Devon Ochein to 325. Base, base, and base. Joe Burrow. Base. I don't think so. Jalen Hurts. Are you the next one, maybe? It might be in 48. 2199 is Austin Eckler on the Jersey card. Chargers, piece of the game. Austin Eckler to 199. Debo. We've got, nice. Derek Henry to 25, 425. Three-colored patch. Rokwon, Gold's team to 249. Nice. Pukatakua to 99. That's nasty. Congrats, Rams. That'll work. Tei Adams. Bronze, Cameron Latu to 275. Base, and base. Joe Thomas. Josh Downes. Nice one there, 2149 on the piece of the game auto. No, you're good, Brian. No, no, no. Don't worry. No worries. No worries at all. I'd rather it be right than anything. So, no, I'm glad. I'm glad that you're in the next one. CJ Stroud. Nice. Rishi Rice on the color match to 199 for the Chiefs. That's on that KC. Rishi Rice, and base. CPAT, Josh Allen to 349. Bills, and base. This whole ceremony is a little lame. This is a little much. Aaron Donald. Ryan 10, a little 50 for the Titans on the foil. A2199, Dante Hall. Zach Charbonate to 199 for the Seahawks. Mark Andrews on the base. Gold team Tyreek Hill. Tank Bigsby to 199 for the Jags. That Prescott calling cards. That's an Aaron Rodgers to 149 on these certified ballers. Jersey card. Congrats, Jets. That's one there, Aaron Rodgers. Base, and always going to be base on the back. The inserts and hits will be on the front. Chad Johnson. Pink, Nico Collins to 199. He's having a great year. Huge fantasy pickup for me. I remember taking him in like the second to last round. I'm like, all right, I guess I'll take Nico Collins. And now he's an every day, every week starter. Justin Jefferson. Duzvon to 399. I beat you last week too, by the way. Call that skill. Romeo Dobbs to 99 DK. Nice one there. Congrats on that Packers. Oh, Collins. Nico Collins. They're like 34 points or something. George Kittle to 149. That is a freshman fabric of Chase Brown. So 399 for the Bengals. Rookie at Jersey, Ottawa. Balls so hard in university. Came back. Tim Brown, 349 tomorrow, Davis. Hey, we respect each other, okay? No, we do. We split the championship when we launched here. That's true. No, we are mutual, mutual. Balls so hard in university. Because he technically beat me. I don't think I would have won. Yeah, I think if that game played out, you would have won. Big Ben. Certified ballers, King Henry to 199, Derek Henry. I mean, I'm not going to make the playoffs this year in ours. Bit of a fall from grace. Will Anderson to 99 is Christian Wilkins. I think I'll sneak in. You might, yeah. But I'm probably not going to win very hard. Travis Kelsi to 75, calling cards. Brenton Strange to 399 for the Jags. It's Joe Klecker, I remember that. I couldn't even. Mirror signatures. Joe Klecker, I remember that. So, so this. Nick Bosa. Freshman fabric, Jake Hainer. Two at 399. Tim Brown to 99, Immortals. Daven Cook to 349. CMC, Orange to 149, Jordan Love. Harry Kill, we got Nolan Smith to 199. What's up, Jonathan? Jonathan. How you doing, buddy? How you doing, man? Jaylen Hertz, Aaron Donald, and base. Joe Burrow, calling cards. Piece of the game to 50 is Stefan Diggs, 750 for the Bills. This one at Buffalo.  Looks like Trevor Lawrence is starting. So, Trevor Lawrence is starting. Yeah, Lakers took the W on that one. Not super surprising. I want to see if there's like hockey on or something, something like a college basketball. Let me get a chance. So, you get what you can wait till after the break, but. Monro, Deontay Banks to 399, 249 DK. Yeah, this is so lame. Tim Brown, Tyler Scott to 199, for the Bears. I love certified. Aaron Donald, nice. Aaron Rogers to 125. Congrats on that one there, Jets again. Manjrews, I'd say pink. Aaron Jones to 199. Tyree Kill, Michael Wilson to 325. Jack Prescott calling cards. Michael Vick to 99. There you go, Falcons. That's dope on the mirror signatures. Mike Vick. Take that all day. OG, Mike Vick, too. Chad Johnson, Freshman Fabric, is Chase Brown again to 299 for the Bengals. Doubling up on Chase Brown, RPS. Very nice, very nice. Hock, K Von Tibido, Mike Evans, Matt Judon to 349. Face and last pack of the break. We'll do a quick recap as well after this. Gold team, Nihime Hines to 275, Cooper Cup for the Rams. Manjrews, CPAT, and Ryan Tannell. All right. Saw some of the Velcro coming off of Ron's beard. That is Austin Essler, Derek Henry, Tyler Scott, Tank Bigsby, Aaron Rodgers, Aaron Rodgers again, Stefan Diggs, and Derek Henry to 25. Got autos of Jake Hayner, Chase Brown, Chase Brown again, Joe Cleco, Rishi Rice, Josh Downs, Michael Vick, and Romeo Dobbs. That'll do it for the break, everybody. Thanks again for the fill. Up next, number 48.


Our talk today is by two wonderful researchers. Dr. Michelle Milade is professor of communication studies in the Wilkinson College of Humanities and Social Sciences at Tappan University. Before she moved to Southern California, although she was previously a U.S.C. graduate, she served as the founding director of the Penn State Qualitative Research Interest Group, which is an interdisciplinary community of researchers involved in supporting qualitative inquiry. Our research addresses human communication and health, including areas such as substance use prevention, suicide, and family mental health. Our community-based participatory research has involved numerous creative projects to translate research findings into social change. For the past 20 years, she has served as the principal qualitative methodologies for a National Institute on Drug Abuse Minor Research. This work has developed one of the most successful evidence-based substance use prevention programs in the United States and reaches youth in 43 countries worldwide. She has lectured, conducted research, and served as a consultant at universities across the U.S. and internationally. Please join me to welcome Dr. Michelle Milade to our campus. She will be joined by Dr. Michael Heck, who is a distinguished professor of communication arts and sciences and crime, law, and justice at Penn State University. The National Institutes of Health has funded Dr. Heck's long-term research on drug-resistant strategies continuously since 1989. That is a remarkable feat for anyone who is ready to reinvigorate in this day. He was among the four to study the social processes of adolescent drug owners, including an examination of the role of ethnicity and co-op operation in these processes. The project has produced a number of theoretical advances, as well as the development and efficacious multicultural school-based interventions for middle school communities that, since it's adopted by TARE, you all know that TARE America believed to be the most widely disseminated middle school drug prevention program in the world. A new elementary version of the curriculum based on social emotional life is being implemented worldwide as we speak. His book, Adolescent Relationships and Drug Use, summarizes some of this work. I'm pretty sure I have bookstore currency, and the video test produced for the project had won numerous international and national awards, including a series of 2000 regional Emmy awards for student productions. Dr. Nex has received numerous awards, including the National Communication Association of Darrell Power-Phillips Award for Distinguished Applying Communication Scholarship and the Society for Who's Mentioned Researches Mentoring Award. Please turn me to Dr. Nex to how it happens. They will both talk to us today about why it's supposed to work and why it's not supposed to work. Or not. Yes, baby. Thank you so very much for that lovely introduction. I'm actually going to start with a story, may seem off-topic here, but my mom is a wonderful cook. She spent a couple of years trying to perfect the cheesecake recipe. The cheesecake recipe. And she had all the ingredients, all the measurements, the timer was supposed to be in the oven, everything. So I, when I was first learning how to cook, I'm like, okay, I can do this. For a family that's going to make her cheesecake. But I didn't have sugar. I had sweet and low. So, did it make no sweet and low? How it's concentrated? It's like a ratio of one to three. Okay, so you need one sweet and low for like three tablespoons or two tablespoons of sugar. Well, I had as much as a sweet and low. So what did I do? Three cups of sugar. I poured them all in. What do you think happened? Yeah, exactly. It was totally inedible. I thought I was being created. I thought I was being created. There was this, this cheesecake recipe that was, was developed down to a science. But then I wanted to do it. I changed an ingredient for convenience sake. And it totally didn't work. Should have worked in theory. It was a recipe. It won't do that. I thought it was sugar or something low with sugar. But it's not. Okay, so I'm going to go back here. And that's a way of introducing here the best laid plans of mice and men. We develop health interventions. We develop them with care. We spend lots of time developing them. And then they go out into the field. And somebody put sweet and low instead of sugar that we've developed there. Now somebody put chocolate sauce on the cheesecake and it could be better than what we planned. So what we're going to talk about today are ways that we adapt in the field. We develop health interventions. We put them into the field. They get adapted. Historically in the field, they say we should have fidelity, right? We should have fidelity. And do our interventions the way that they're planned without any adaptions. Adaptions, our adaptations are bad. So we're going to challenge that a little bit and say, you know, adaptations are. They're going to be. So let's live with it. Let's roll with it. There's my little cheesecake. Although prevention programs are developed carefully in implementers such as teachers. In our case, we have school-based interventions. Our trained to implement programs of fidelity. Real world events sometimes conspire to influence program delivery and ways program developers do not imagine. Again, we cannot anticipate all the way that people implementing, whether they're medical assistants, medical educators, teachers, or in the dare, in the case of dare, officers going to implement our programs. So what we're going to talk about today is creating, adapting, and disseminating effective health messages in particular interventions. We're going to talk about developing messages, adapting evidence-based messages, intentionally and studying unintentional adaptations, and then a little bit about dissemination here. So, you know, you ask yourself, do we still have to worry about drug use? The war on drugs in my era, we don't hear that rhetoric so much about the war on drugs anymore, but it is still clearly a problem. Again, reducing substance use is one of the objectives of Health and People 2020. It's still an issue. And how many of you know someone who has had a problem with drug addiction or substance use problems here? It's still, we're talking across cultures, urban, rural, we're going to talk a little bit about urban and rural kind of things, across ethnic groupings. So it's still a problem. We still need to build health interventions for this. And I'm going to give it over to Michael now to talk a little bit about developing our program. We're going to try to confuse you by switching back and forth. The program we created was called Keeping Real. It's a school-based multicultural intervention that grew out of Dr. Miller Day's master's work, actually. So those of you who are students here, long-term, big, great things can grow out of the work that you do right now. He was my advisor. He's not old. So for over 25 years, we've been trying to do two things. One is understand the social processes by which drugs are offered. Some of you know the, just say no movement that emerged in the 1980s. Nancy Reagan was actually pushing this idea. It was based on real science which was that we needed to understand what was going on when kids were offered drugs. And what did they say? What went on? But it turned out that not many people were studying what went on in this situation. And that was the basis for the intervention that we created called Keeping It Real, which proved effective in reducing alcohol tobacco marijuana use in a 14-month period. So what we thought we were doing was developing an ethic for creating interventions. So we thought the methods that we used, we went out and studied what happened and then took those narratives, created performances, and created a curriculum out of those performances were something that would be replicated as an ethic. Well, we did this and suddenly the National Registry of Evidence-Based Programs and Practices labeled us an evidence-based program and people wanted to buy this program. You know, we're professors. We didn't even have money to make copies of the videotapes to sell them, to just mail them out. So we suddenly became a commodity and people did not want to go through the effort of creating their own program using a method. They just wanted to use the program we created because it was the only multicultural evidence-based program. In particular, it was the only one that included information and cultural references to Latinos. And so it became widely used and it was disseminated in a number of places. So the dissemination process, as Dr. Melendezia said, traditionally tries to seek the depth. So you create this program and then you try to replicate exactly in various different places. And any change from what you design is seen as an error or bad or something to be eliminated. And some of the programs have been more successful. Gary Holtz has a nursing program where he trains home nurses and he's been pretty good at getting fidelity to that program. Most of the school-based programs may be not so much so. What actually happens most of the time, a doctorate evidence-based practice, if they do, they use some of it, they mix them together, they add their own things. I just met with our local school district in Skate College and they said that they don't really want to use my program. What they wanted to do is work with them and integrate three programs they have into something just for them. And I said, well you lose the evidence-based claim from that and they said they didn't care. They just wanted to do what they wanted to do. So rarely did programs actually get used the way they were designed. So what we've argued is that because the actual practice is described as this adaptation, that in reality what we need is a practice-based science of prevention. We need to base what we do and what we think about when we design prevention programs based on how they're going to be used. When we come up with a different definition there which is fairly controversial in the prevention field and at this point some people throw things at us and I try to talk. Some people like it, some people don't. Usually people are out in the field. How many of you have been in the field trying to do health work or some work with kids or something like that?  So based on that designers should reinvent new populations implement those will adapt and participants are not passive recipients. Now we're not going to spend a lot of time on the last one we could do during the Q&A you can bring this up.  That they have cognitive processes and interpretations and social processes and that's probably all we'll say at this point unless you have some questions. So overall model, we've come up with to talk about the process of adaptation. Going in very specific directions and looking at the three forms we're going to emphasize the top two section segments. Back to that brilliant idea. Okay, designer adaptation. So the thing, again the hope is about fidelity keep it the way it is don't change it. Really negates the need to adapt to different populations and to different audiences. You need to work at the field of communication. You need to do audience analysis. You need to know who's going to get the intervention and the messages that they will be getting. So the principle of cultural grounding for health message design starts with culture, obviously. It's identifying from the ground up so instead of from the top down so this is what we know what works and this is the information we know that they need let's create this intervention, send it out it's instead going from the bottom up what is this culture? What are the values? What are the norms? What are the standard ways of interacting around whatever this health topic it might be? So starting with the culture and addressing its complexity multiple identities Again we have the strongest influence messages influence us very strongly when we see ourselves in our group memberships represented and reflected and acknowledged not ignored. Our very first curriculum that we're going to talk about in a minute is multicultural. The most effective one was multicultural. We did test again specifically at a Latino version, a black and white version and a multicultural version and which one do you think was the most affected across black, white and Latino populations in that in the particular area we were testing it is was multicultural because everyone saw themselves reflected in those messages and in those images active participation of cultural group members in message construction this is, I'm going to pause because this is really important instead of all of us academics up here just saying this is what the message should be really talking to the people who are going to be our recipients who are going to be the people who are targeting the target population members what are the messages how do we get them to help us construct messages so that the voice is represented as well in the intervention. So from us our programs are culturally grounded prevention messages and we point the from kids through kids to kids creating concentric health messages because we start by talking to the kids whatever messages where they're coming from and if I talk with rural kids their experiences of drug offers in particular our program is not about don't do drugs our program, our intervention is not about communication confidence it is about being socially competent if we choose to resist substances and it's about decision making and making healthy choices but getting youth narratives and rural cultures may be different than you say Orange County versus San Francisco youth are involved in the narrative message development and youth are involved in actually pilot testing intervention itself so why narrative, why kids stories why do we want kids to tell us their stories narrative evidence so again we have statistical evidence say 80% of kids will X well kids don't often see themselves in 8% what that 8% is they often see themselves in the stories of others so when I got offered drugs or when I was at this party this happened, I didn't know what to do and so X narratives give voice to student experience or youth experience it paints a picture to illustrate some of the statistical evidence that we might provide it's more engaging sometimes than statistical evidence it often reaches low knowledge audiences much more than statistical evidence might and often times it's just telling a story that renders complex information more understandable so we want to create new narratives for kids who really haven't really thought that so we have 5th grade 7th grade interventions we want to get a kid that passed before that really started using so large levels of use so we want to create new narratives of resistance new narratives of new norms, new narratives of how they can interact when they get offers of substances, alcohol, tobacco other drugs we want to reinforce healthy narratives if they already have certain narratives that are replete in this particular cultural group we want to reinforce those in the intervention that we have and in any unhealthy narrative what we deem as unhealthy in terms of substance use we want to be able to challenge and provide alternative narratives for them hopefully in that situation so the narrative engagement theory that we have several publications that deal with this so we have narrative messages that include characters, setting, actions problems and solutions we have a video we have a 10 lesson curriculum and there are videos in 5 of them but there's a story line across all 10 lessons these help build novel mental models alternative ways of thinking and understanding and not just don't do drugs but alternative there's a grain there alternative ways of thinking about substances and substance use giving them behavioral models of cool kids who don't engage in alcohol and drug use and improving narrative knowledge which isn't just knowledge of facts and understandings and thinking cognitive narrative knowledge is experiential I totally understand there I've been in that situation and I didn't know what to do or maybe now I do know what to do here's some alternatives for me leaning to changes in skills shaking norms, attitudes and intents more cognitive and then social proliferation we want kids to talk about the videos we want kids to talk about the messages we want them to talk about their other kids the other their peers the other members so we want this proliferation of the message to happen across the not just in the classroom but across the school and maybe into the household and we predict that this will impact health behaviors how do we do this so our method that you talked about that we thought we were developing this method is we start by interviewing to a lot of interviews with our target population getting stories getting stories of when a drug offers but it's also stories of when you used and stories of when families and friends have been impacted so what are the messages here that you've gotten from the situations that you've been in doing focus groups not only with kids but with people who work with this so focus groups with say teachers counselors individuals who work with the target population I was a teen advisory group so at every step of the way they didn't give us feedback saying yeah that's really that's really corny having them develop our messages like the logos and having them really part of message development video development from kids through kids to kids so from kids the interviews the narratives we always have kids who've never had younger than high school aged kids creating videos based on the narratives that we collected from the younger kids collect the narrative develop these video interventions with guidance by professionals the high school kids creating these videos to go into the curriculum and then thank those two kids so from kids through kids developing those messages those media messages to kids the audience and then we adapt the curriculum we talk about it a little bit we adapt it in many ways for different audiences and then we go right back to the beginning what you know what's working what's not working and keeping on adapting in that way so keeping it real is the name of our curriculum keeping it real AIR okay Nancy Reagan just saying no drugs from a communication perspective we've learned actually cross cultures cross regions there are pretty much four ways resistant strategies that seem to be effective refuse, explain, avoid and leave that was our original keeping it real logo that the kids designed and that was back in the 80s so refuse, explain, avoid and leave and we keep expecting new kinds of resistance strategies to emerge with all the people that we've talked to but most resistance strategies can fall into those categories keeping it real R-E-A-L stands for refuse, explain, avoid and leave and Mike is going to talk more about the curriculum okay so as Dr. Miladay mentioned it's a narrative intervention so the idea is that we have these stories that we want to enforce healthy models or reshape the stories that they have the approach is not to be moralistic and we train the teachers and the other implementers not to do that not to use fear messages fear messages are pretty effective on adolescent kids they don't work very well in those situations and so basically our intervention is involved with the stories that kids tell fed back to them and discuss with the classroom and what we're trying to do is to talk to them about the choices and the options and the way they make decisions about risky situations in their lives and give them a new way to think about those things you can see the five of them are video based we'll play some of the videos first video sets the stage here's an introduction attempts to get the kids I'll stop there and then one video for each of the usual strategies this is very much about how to be along with their friends how to talk to their friends how to form healthy relationships these are the narrative elements that we have in the curriculum the videos, discussions, role plays all created out of the kids own stories that Dr. Nolday referred to create all of these different narrative elements so this is the first video this is the interactive video for the multicultural intervention and when we show you the rural video let's see if you can pick out some contrast think about what you see as a similarity difference so this is the video that there it is that the South Mountain High School was the former arts high school graded for the schools in South Phoenix and in Mexico difficulties those of you here early weren't loud enough before right any questions while we're playing we'll try again if you want to work we write the script we go out and shoot it in this program we get to work every position learn everything and choose what we want to get into grade schools and we ask them what's the proper grade what then who would you expect to make a story out of it we script write it during the summer the two weeks from there when we came two minutes I said everybody can relate so I said I got here I know these people I can put myself in their place and this is how I go I did a situation on that so it's more of this to surrounding the people that are around what we're doing I didn't expect to use videos that I haven't seen in other videos is we're using realistic looking fake substances like vitamin B for crack or ready help and most of the things they don't really show okay so this was intended to be a multi-cultural urban videos or anything that would lend you to believe that we accomplished or didn't accomplish our wills the urban data the multi-cultural so there is American culture when we are talking international but the students that I saw in that video I wouldn't consider the diversity that's represented in the American population it looks like a small excitement or multi-cultural a person that was developed through the schools in south Phoenix 65% of the kids in those schools are Latino mostly Mexican art now 20% of the time we were doing this we were white and about 10% were African American so it reflects the cultures largely in those three groups we need to get more of the videos because as the videos develop they are based on narratives of members of each of the groups narratives from all over the groups I have to tell you that this is kind of a grand thing we started doing this as ethnicity and we got labeled the ethnic group in NIH the differences by gender were much bigger than they were by race youth culture overrides a lot of the ethnic differences in these schools because they were behind the integration that are true and that's one of the reasons that we were surprised for example when schools in Georgia and Florida and New York started to adopt these videos because they were both Latinos and it just seemed like it was not totally appropriate but then they would say to us we don't have anything that's any element of Latino culture so please let us use it what did the video attempt to do? anything you got out of that? what was the purpose of the video? why did the kids talk to the video? why would the performers talk to it? yeah kind of what you were referring to getting credibility and authenticity I was talking to resist the message given it's something you relate to we did, we did the whole thing ourselves right and then they're involved and how they were involved eventually they talk about why they were involved in it so there's that whole big process beginning there's the overview of the backwards narrative it starts off with that shot of the freeway you really won't see that in Central Pencil can I say it? Central Pencil Tucky Philadelphia Pittsburgh, Kentucky in between so anyway that's the opening video this is a very great story when we worked in South Angus huge neighborhood in South Angus it's just really really big that was the best people in there when we got there we discovered there were no shopping centers and no movie theaters in Carnet all of the kids from this neighborhood 45 schools are involved in our study on the same bus and went to Alituki which is a white middle class neighborhood just outside of South Phoenix to the same shopping center to go hang out and these kids that we see became huge stars middle school kids would be running up to them and say oh I saw you in the videos in our school you're really good actor you're going to be a star the woman that you saw talk about at Harvard by the way pretty cool and actually one of the people in our cast high school kid was a very serious alcohol and the little kids were running up to him and he said the sisters and the brothers need me and he went cultured we were trapped for four years we were sober for four years I believe we saved that kid's life totally unmeasured, totally unreplicated probably the biggest success they've ever had in 27 years of prevention research but yeah and that's one of the things that we lose when the program goes national because there's not that feeling that I know that kid he's like my friend's father brother and cousin and things like that which you get in the world of community so this study as I said there were 45 schools there were four different conditions the videos, the Latino condition were made up only of the stories that we got from Latino kids that were actors from different racial groups but mostly the stories were all Latino I could talk about why it became white black later multicultural mix up we gave pre-test, pre-post-test the last post-test 14 months after the intervention ended in which we find using growth model, general estimating equations we find statistically significant effects on tobacco, alcohol and marijuana use this is kind of graphically related what's interesting in this that you don't always find in this population is the fact that marijuana use is higher than tobacco use but that actually reflects University of Phoenix and Latino culture and their choices so did you explain what the control was the control group did not they continued with what they wanted and one of the problems with school based prevention is you can't tell a school well they don't use anything now so some of these were using evidence based programs of time kind of waters down the effect but yeah they continued to use whatever they were using up to that point so that they in fact many of the schools to this day are still using it but we can talk about that too so currently finishing up a project in rural Pennsylvania where we adapted to keeping real, re-grounded keeping real to a rural community in the research that attempted doing that by open source the key factors, the biggest difference in REAL is that for the explained factor rural kids say this is not who I am and they all know the only other adaptation beyond that were among the lives of a group at the University of Central Florida that adapted to teaching Latino sexual prevention resistance and the other strategy that came out of that discussion was I would punch you in the face so I don't know if they actually do that but they said new materials, new role plays the eighth grade for booster lessons where they create their own messengers and this is the the new logo that was created by the kids for their own and here is the video let's go to the tape hurry everybody, quiet on the stage get things started, okay? and action we're going to start with the blind shot and we're going to repeat it I want like, you know, with the kids to see what I schools really like these children are all made by kids they're written, produced filmed, edited I like how they're having fun before the grads come our goal is to make these scenes as much like real life as possible and the movies use techniques from keeping it real how the kids see that, drugs and alcohol do not benefit you anyway does this look different from the other one? okay, the rural kids said please just don't show cows they're a major reverse or Amish people now, no cows, no Amish people don't portray us that way but you've got the same idea with the kids trying to say oh, we name this, it's us it's from kids through kids to kids to have some identification with it and then said they're really immediately on rural environments what looks like them there were no other programs they created for rural kids and all dated on the use of rural kids is the first program that will have to fill that purpose it's surprising that 20% of the country put into the kids' intervention in rural environments they're really quickly so they came to us and said how many of you have been through there? how many of you like there? you like the new there so I've been telling them they're starting to let it and they always created their own interventions and after a number of years and a number of programs that shelly said did not show the effects they would like they decided to adopt an evidence-based program they looked at a bunch of them they decided we'd get them the best and hence they didn't charge them very much and so in the middle school version they were fighting with them 250,000 kids, 23 countries then they decided that they should redo their elementary program where they get terrible results in Philadelphia and that is just going to fill out or go to about 1.25 million kids and 44 countries around the world making the largest prevention program in the world it's thought they could have something more than that again but this is the cover let's get through this the brief intro it's a national deal so the challenge is it's national how do you culturally ground it so that's a question to consider these are the representatives there are rural, urban, and sub-urban versions of these videos but they don't have the same intro which is national now the fifth grade curriculum we chose a little different because fifth grade kids there are a few kids are actually using substances at this point so we're getting them at the decision-making stage the time when they are really pulling out the way of becoming independent thinkers we started out with live-action with these kids but we are using peers so we're using these 10 to 12 year old kids where a psychoanalyst is one of the talented kids trying to get real full performances out of them is a challenge so what we chose to do again, for kids is that we we are using live-action and animation so the intro of the videos for the fifth graders for Der is live-actions, live-action characters but then they work and transition into animated characters by the end of the very first lesson and then there's live-action and animated characters throughout the entire curriculum but I'll show you just a little bit of a clip from one of these and then I tried it I'm really glad I did I really love it I was like, what? I just looked at it and said, no way I'm just awake right now and everything I'm not going to cover for you What do you think a lot of your mom does find out? Oh well, I'm also my karate class when friends talk to me to take me a chance and try karate, nice to know for a few months and then I tried it I'm really glad I did I really love it If we go into positive risks not just drugs or evil and bad what are some positive risks that we take in our lives and we're trying karate we like it if you're doing it but some of how we're customizing the curriculum to fit different audiences Now let's move on because we only have a few minutes here and I'm going to like about three or four minutes because I want to take questions that's designer attitude that's us as designers adapting for different audiences because it's four kids three kids, two kids trying to adapt but we hand it off, right? we create these, we hand them off to teachers to implement or to officers to implement we're really interested in how are they changing the recipe, right? how are they altering the recipe what's happening in the classroom so school adoption of evidence-based programs are very popular now because for example, SAMHSA has a registry of effective evidence-based programs so if I'm a school principal I can just go to this list like data did and choose what are the effective evidence-based effective programs out there for my topic here obesity prevention, drug prevention what have you in terms of that so they're becoming much more prevalent so we did a study here we did 31 teachers in 25 schools 70 students that we studied we wanted to know the degree of implementer change, types of changes reasons for changes we got teacher self-reports as well as we got observer ratings we randomly selected 276 of the 730 videos the teachers put we had video tapes at the back of the classroom and the entire lesson was video tapes all these lessons shipped over to us and they're all digital so we have all this digital data if anybody's interested in doing video tape analysis come talk to me but we randomly selected 276 videos to look at the adaptations the teachers were making teachers reported that 32% of the time they delivered in fidelity they did it as we asked them to do it in the curriculum however the observers when they rated it 2.6% of the lessons were developed with fidelity I'm not going to go over all these but suffices to say partially a many lesson component was most popular because it was coded as that if they left out something but we don't know from just this if it was important something what I think is consequential is completely omitting there was clearly a component of that lesson that we put in there for a reason so completely eliminating it adding their own content their own two cents and revising content doesn't mean that what they added was inherently bad so no it wasn't controlled and the message wasn't in line perhaps with what we had developed revising the content changing saying something that was supposed to be done as an individual into an entire class so what if it's about privacy what if it's asking kids to disclose some information you make it a full class activity or if it's something that we wanted to establish norms and we wanted it to be a whole class activity instead you do it as an individual or a dyad or something so again these kinds of changes there's all sorts of give the example of the one guy who so we train them among other things we teach them set up a camera every lesson ends with roll points he walks in and he goes like this he opens the book and goes okay we need to do some roll points today break through the end of the thing and he's taking it himself he's taking all the pain to do this lesson any of you were teachers we do that we adapt the curriculum the information to suit our needs and that's exactly what happens is teachers give reasons for doing this mostly practical considerations that didn't have enough time I had other things going on we had an assembly whatever so I needed to truncate it to the most fun stuff and I'm adapting it to adapt to our rural students or because half of these students we had the classic curriculum half had the rural and then there were controls so the ones who got the more urban curriculum would say we're adapting it to make it more rural or make it more appropriate for our 7th grade students or to my actually my own teaching style we found that teaching style is huge so we're trying to really identify what are the styles that teachers are using to implement curriculum maybe customize messages specifically for them or lessons that cater to those particular styles so implementation quality again for us we're looking at the delivery quality the kinds of styles that the teacher had the fidelity of the implementation and outcomes it did have a significant effect so implementation quality quality quality so we don't know right we want to measure outcomes we want to measure drug use if we don't find any effect is it because we have a bad curriculum and we didn't adapt it well on our end or is it because it's not being implemented in the field in the way that it should be so implementation quality and adherence to it did have an effect on our particular outcomes in the onOS test delivery quality was significantly related to norms and substance use but not related to efficacy child's efficacy adherence was significantly related to norms marginally related to substance use and not related to efficacy do you want to step in there with any clarification well I just think it's interesting that if you deliver it well you're animated in the way you teach it and the students are engaged in it's not important it's more important if you actually teach which is inconsistent with that view of adaptation and pure fidelity can you explain maybe the efficacy part I mean it's not clear to me why you wouldn't correlate with efficacy if it's not delivered appropriately you go back in the last two bullets yeah efficacy means itself efficacy is their belief that they can perform the behaviors and so the I think one of the problems we've always had with these particular set of RAAM strategies is because they're so indigenous to them students recognize them you give them a self court measure even if you haven't got a court certificate they say they do them all and I think part of the problem is that when you're asking about that they say they can do it because it's the way they naturally do things so I think there's an epilogical problem in traditional design where you give people pre-test and post-test I would say a bunch of things that would you know for the future that we want to do and things that are curious, we only have a few minutes so I would like to build questions questions yes so I guess most of you are in the test of NILA it's kind of interesting to look back um this is I think it's a long time ago so when you're looking at these populations it obviously makes sense to address different demographics you know in different regions but within a high school cohort even between saying these demographics there are even different subgroups some people are definitely more susceptible than others and so I can imagine you know there's people who are kind of on the fringe who maybe associate with groups that do drugs and groups that don't do drugs there are those who don't associate and there are those who are simplifying it, there are those who are very strong or engaging in it and so I can imagine that in each of these three groups they're going to respond very differently to these videos and some may just look at it and be like what's my name and some may be like I don't know I'm kind of hesitant towards it and others may be like wow this is great so you know within this are you targeting a certain one of those groups or are you in fact somehow trying to put in a specific facet that that will try to attract each of those yeah the targeting this is a prevention, intervention we're not dealing at all with really high risk groups or treatment in terms of that so it's not that we're eliminating that but we don't really address that it's much more prevention inhibiting initiation as well as decreasing use once it does happen in terms of really high risk so we don't really fully address that in this program yeah I think it's really a good question the answer is part of the thing we can go on for an hour about this question but remember that there are multiple narratives in every lesson and our goal is to select narratives that cut across a group of people to basically re-brand or re-story what means to be a drug user or a non-user and basically create this image of non-use that is popular and exciting and mature because the indigenous narrative about drug users are that they are mature adventurous, popular it's a very positive image of people who use drugs in middle school and early high school and frankly when we measure the personalities of those kids it's inaccurate portrayal of them secondly one of the things that we are looking at down the road that comes into that participant thing is we have a number of people on our team and what we hope to be able to do is to track the message into networks narrative networks and track where it's infused and not there's a woman named Linda Collins at our university methodology center who's talking about a chapter in convention and the idea being that once you get the message out you see how it's working and then stage two is to adopt to its values so if you look at who the message reached and who it didn't reach you can send the second message out and see who it's not reaching decide what message it could be and how to get it into the audience that's the future your work the dare program see dare started as a program and then came to the realization that they were a dissemination vehicle that you have these officers who by the way deliver curriculum with greater fidelity to the kids in class so they had this training marvelous training and dissemination vehicle but they didn't have the right curriculum so they adopted ours but we have not been in the field to evaluate it taught in that framework yet looking for a donor seventh grade's been in for two years and the elementary discipline in this year so they're using keeping real now the officers got retrained to use keeping real so we did that so we had our curriculum and then we adapted it specifically for officers to go into the classroom and I'll tell you this talk about constituencies we have officers, we have sheriff's departments we have education people, we have counseling people you have the kids so a lot of people left the table redesigning now and officers are not teachers so the expressions are very explicit you know all we know is that there has been very successful in marketing our program and the number of schools and the number of countries has increased we don't know yet starting next year we're going to be evaluating we're doing a pilot study right now yeah we're doing a pilot review well first of all this was an incredible reputation and a really excellent example of how development interaction and then suddenly in the past it was just very exciting to see I have a few questions the first is the intervention rate on that so there are a different components do you ever break it down to what you can see exactly like behind the block box which piece of the activity will be in and which part and then the second question is obviously this is expensive I mean your videos and production it's just incredible those of us who are beginning our careers and how do you move from sort of your basic program to something that's finally wonderful I think for you I have a great mentor and I know people want to get going so quickly and I can talk to people at lunch about this but yes we have plans actually what's the secret sauce what are the four dimensions of the different curricula so then if I have 20 minutes which one is the white kid and we are studying that behind the black box what exactly you know how does it affect how do you do the two second explanation how do you do that the same person I mentioned before in one of the columns there's an article about something called the most system it goes back to now some variants factorial designs you just give some of the program to some of the people some of the program to other people and you do it that way I have to tell you that schools tell us but the reality is there are kids in every school that can make these videos and maybe better than what we have here I actually don't think it costs very much for every school in the country to make their own videos their own versions of the names because you know there's little cameras right here they're fine they cost $100 they're $150 and so I think you could disseminate I mean you know what I don't think it would be so terrible if they used iPhones and created YouTube videos and that version the original video the original keeping a real video costs $48,000 so the Metro version is statistically at first one of those components the things that were dropped the things were kept in kind of cursing those out and identifying those and then going out and testing it using that strategy we do know that if they see just the video well it's not a real experimental design if they see just the five videos that you're stabbing then they're done it's actually four or more thank you thank you