Okay, I'm going to kind of go fairly quickly because I have a little less time than I thought I did, but I'll be putting everything up online and all the examples and everything so you can play with them later. So I'm going to talk about something kind of goofy that's not my day job, so keep that in mind. I should just warn you, this is actually rated TV, M-A-L-S-V. And it also has spoilers, but everything is text, so don't worry about the spoilers too much. I'm going to try to be very, very careful about the sex. The reason is because I'm using 50 shades of gray. So these are the book stars of today. We're going to do a couple of Dan Brown books and only a little bit of Twilight and the 50 shades books. I'll let you guess which out of these books on here I read and liked. All right, why am I doing this? I actually sort of grandiosly think it's important to study what's popular because it tells us something about people. And unfortunately, this particular topic genre bestsellers is not something I get paid to look at, but I read trashy books and I enjoy it and I'm unapologetic about that. So yes, so let's take a little bit of time to look at them. And I also wanted to do some fun statistical tricks with text data that I haven't gotten to do and build some visualization types that I haven't gotten to build. So that's why we're doing this. So first, what inspired me to look at 50 shades of gray was this info pornographic from, believe it or not, the economist. This graphic design firm called Delayed Gratification did this about six months ago. And they had some poor person sit down and chart out where the sex scenes were in all of the books and their level of kinkiness, sort of apparatus is involved in them and all of it. I mean, and mostly, you know, what's funny is that I was kind of attracted to this, like the very basic displays here and paying very much attention to this. And also I had not read the books. So I was curious that there was a little more sex in this and it decreased. And I think it I think it got kinkier, although I didn't read all this in detail. It doesn't matter. But my main question was they paid some poor person to do this by hand, like flip through the books and score where the sex was. And I was like, can't you just do this automatically? Because that's what all of us data scientists really thinks. Like, let's just do this stuff automatically. So I immediately thought of various text classification methods in machine learning. Commonly text classification is done as sort of a bag of words. It ignores structure in a document and it just looks at the words that you care about and counts up frequencies and uses those as essentially feature vectors in a classifier.  There's only certain words involved in sex scenes and behaviors. It ought to be perfect for a classifier, a simple classifier. So we'll show you whether I was right. So in supervised learning of the type that I was aiming at, you take a data set and you label it as an example of a positive or negative for the thing you're looking for. So in this case, is it a sex scene or not? And then you feed that data into some algorithm, which is often described as a black box, especially why people who don't know anything about data science. And the algorithm learns from what you gave it what the properties are of a thing that is, in this case, a sex scene or isn't a sex scene. And to do this properly, you split up your input data that you've labeled into a training set and a test set and you run on the training set and then you evaluate on the test sets and you know how you did at the end of it. Like you can just count up how many did it get right. And then you tweak parameters in the black box until you get like a score that's good. And then ultimately, you use that as a predictor in the wild. So like spam classifiers are the classic example of this. That's like data science 101 as you build a spam classifier. So for the 50 shades problem, I bought 50 shades on Amazon, which no, I did not own it. I unlocked the text in Calibre, which is like open source EPUB conversion application, which is awesome actually saved it as a text file. I cut it up into 500 word chunks using Python. My machine of language learning of choice or language of choice.  So I thought, well, I'll have three because there might be, I might want to have sort of a fuzzy classification at some point.  And you know, doing this was actually horrible. I realized that I did not like the book. I didn't really enjoy going through and manually and doing the scoring. And I already knew that I needed to do probably two of her books so that I could really test on the second book. So I went on Twitter and I complained about it and people are like outsource this. So because you guys are my friends. Anyway, this is an example of the text of 50 shades of gray. I'll just, if you're close enough, you can read some of it. But it's basically, it's not exactly my style. I read trash, too. So yeah, so I outsourced it to Mechanical Turk. This was fun. This was actually the first time I'd played with Mechanical Turk for, even though it's a classic machine learning like platform where you get people to classify things and then you build your algorithms. I hadn't actually done it on anything. So this was really fun. I'm showing you, this is not actually 50 shades text. But it's essentially the size of one of the text chunks I gave them. And then they got a category to pick. Those are my three categories. And then I had examples like under those little blue eye things that told them the kinds of things I thought were a sex scene or not. So, but like you should know, I mean they're getting a small 50 or 500 word chunk. And it's funny, even though I thought this would be a really easy problem, obviously it's a little problematic to say is this part of a sex scene or not when it's 500 words. And so I was sort of torturing them because it was already hard for me. I mean if you think about it like just visually, like some people find this sexy and some people don't. Now this, which is actually what 50 shades of gray is about, I don't find sexy and that's by the way a kit available on eBay in the UK if you're interested. So like this was one of the things about classifying what sexy is. It's highly dependent on yourself essentially. Now suppose you're reading like chunks and there's a shower scene like this. And in the middle of it, you know, actually she looks a lot more interested in that awesome shower than she does in that guy.  Anyway, so it turns out in the book that there are a lot of these scenes where they start sort of like making out or whatever and then they stop and they do something else. So that's hard for a person classifying it to know. Like is that actually part of a sex scene or not? Like it gets problematic. And in particular, they spend a lot of time in the first book arguing about a contract as you might imagine. It's in fact a sexually explicit contract about things they will or won't do together that she's supposed to sign.  It's actually like legalese text for pages. And there's email messages about this contract.  All right, so this is what you get back from a mechanical turf crowdsourcing job.  These are on the left hand side is my chunks and they've been randomized. People got them in random orders. I paid for two Raiders so that I could have two opinions on everything, which was the minimum, I mean, I was paying for this myself. You get, you know, the original data and then you get what answer one was, what answer two was, and a little bit of stats with like a code for who the Raiders were. So I scored those numerically and took a look at just briefly in Excel, Raider one versus Raider two. So it seems that Raider one thought the beginning of the book was a lot steamier than Raider two and the big spikes are where they agreed or didn't agree on what was actually a sex scene. Okay, but so since I grafted in this nice, stupid, simple bar plot, we can compare it to the pornographic. There's a surprisingly good match. The person who made the economist's infographic and went through this process matched pretty closely what my Raiders said, even though that person was probably reading it in order and mine were looking at a 500 word chunk out of context. So this was for me more evidence that it might be possible to actually do this automatically without context, right? And on the second book, which I did as well, so I'd have more data, it's, sorry, I didn't fix up there. This is part of the pornographic, it lines up. So here's, essentially I got only 68% accuracy using naive bays in Python's natural language toolkit, which is not so great. There are plenty of examples where you can do better with naive bays. The thing that I want to show you is it does output the words that it thinks are the good indicators. So there's a list, eases, moan, raw, beg, packet. They actually use a surprising number of condoms in these books.  So I switched to Python's scikit-learn toolkit, which is much more developed and has a lot of people working on it who are really interested in text data. So they have this one chart, which the details of it don't matter, but it's basically all the sort of classifiers you can use easily on text, it takes sparse data where you have only a few observations of each feature, that kind of thing. And they show you in this little chart how long it takes to train it, how long it takes to run, et cetera. So I picked one that just somewhat randomly, although it has good reviews, which is the stochastic gradient descent classifier, which is in the middle of this chart you can't read. And you should also notice there's a passive-aggressive classifier, which I think is the best name ever for a classifier. I want to try that one next. So in Python, in sklearn, they've made it super easy to do machine learning routines and to do lots of iterations, do a feature search, and get a ton of data very, very quickly and compare things like this. This is the only dense code slide I have. But basically, they've created this pipeline concept. And I don't know why it's cut off. In any case, all the code will be up there. But it's only a few lines to set up a pipeline and then feed it your data in the target and then get the results of how you did on a test set. It's like that few lines. And I got to 88% pretty quickly just doing that one switch. I didn't even tweak any parameters. I took out of the box parameters. So that was pretty good. Now, at this point, though, I was like, well, I think I want to go back to the data and figure out, where did it make mistakes? And so that's when the tool buildings really started. So I made a tool to look at the false positives and the false negatives against the truth to be able to see. And this is where the D3 skills really come in. It's super dumb, because it was just for me to wait. So hang on. This is where I'm afraid I'm going to really roll over something embarrassing. You see this badly formatted text down there? Because essentially, I was starting to do tokenizing on it. And I'm sort of showing you bad looking text. Future text from Dan Brown looks a little better. But here up on top, I've got the target class. All the purples are where it agreed with the judgment of the raters. And then the blues are where we missed. It's a false negative, and this is a false positive. A lot of times, the mistakes are at the edges of the actual sex scenes, which figures, right? You're out of context.   And when I do roll over some of the ones that are false positives, there are things like they're getting undressed or something. So it's pretty understandable to me why it made the mistakes it made. I wasn't that upset with these results.  This was just for me to visualize the results, because I couldn't figure out. And I needed to be able to see the text next to the actual score that it got. OK, that was tool one. All right, and then I have a surprising PS that was just out of curiosity. I took this a step further to see if I could do better than 88. And I paid for a bunch of fan fiction to be coded. Now, I didn't want to spend arbitrary amounts of money on steamy romances that I wasn't going to read. So I went to friends of mine who read sexy erotic fan fiction, and I said, give me things that are long, that have a lot of sex scenes. I don't care beyond that. And got a list of six things free, threw them up on Mechanical Turk, and got ratings, fed that in as more training data. And you should remember, 50 Shades started as Twilight fan fiction.  The genres are actually pretty similar. Fan fiction didn't feature sex scenes, and 50 Shades come from essentially the same background. I got an astounding 97% accuracy after that. If only someone had paid me to do this. It's like, amazing. All right, so anyway, so I was right. It was easy to do the sex scene detector, essentially. 50 Shades and the sequels just happened to have this extra kink component and discussion of contracts that I didn't expect when I started this. All right, now in my proposal, I also said I would talk about story arcs. So let's talk a little about that as the next sort of section. So initially I was thinking about, if you're writing a screenplay say, you get this advice to have this three act structure and you have like, you can see on the lower right, sort of a graph of like rising tension with various crises that up the tension. Then you have some relief and you go up and then, and then at the end there's this like climax and a denouement and things cool down again. And I thought, well, you know, I bet I can detect that too. So the rest of this talk is me trying to detect that. And since it was just for his talk and not for a paying client, I'll just show you the process I went through with all the visualizations and techniques and let you decide if I got somewhere at the end. But I should tell you first. So after doing this, while I was doing a search for story arcs, I very quickly hit this awesome article, you know, the Hulk movie critic. He is awesome. Anyway, he has this rant about the myth of the three act structure and then what garbage it is. And he goes on and on and on. And this is absolutely my favorite paragraph of this.  Which it's true. Like you could divide a line anywhere about what the middle beginning and end is. It's hilarious. Yes. Okay, but back to story arcs. So at that point, I'm like, okay. So even though I think Dan Brown's really effectively trying to write for the movies in book form and I think that's arguable. I went back to like basics. So this is Vonnegut's story arcs. And now his story arcs when he's talking about fiction are essentially a sentiment thing, not about tension, but about like the Y axis is misery and ecstasy. And so these are like some arcs that he thinks exist out there. And you especially want to notice this one, which is me writing this talk essentially. I mean, and the thing to point out here is it's not just a sort of rising peak and then a fall. It's more complicated for him about what's interesting to graph graphically. And he's talking about structural differences in stories. So I think that's the gist to get away from this. And then this is his graph of real life, which I saw this after looking at the other stuff. And I go, my God. So here's a picture of a cute little bird taking a page from Dominicus, which should perk you up right here. All right, so yeah. So this is what we're really after and we're gonna see if Dan Brown has it essentially. This is the generic version. So I did the same thing because I had had so much luck with Mechanical Turk. I took Dan Brown books, two of them, and I chopped them up into 500 word chunks and I sent them to Mechanical Turk. And I just, I mean, it's a factoid, but I got ratings for 50 shades of gray in two to four hours. Took half a day to a day to get Dan Brown read by Mechanical Turk. Make of it what you will. And here what I was doing was I was asking them to use sort of exciting scenes, like exciting action scenes and revelation scenes, that kind of thing as a proxy for the action in those arcs. I thought, well, there's probably something different about those scenes. Let's try to build the action scene detector. And action scenes are actually a little difficult to this obviously action in Indiana Jones, right? Now, this is an interesting one that most of us remember from Jurassic Park. It's in the middle of a very tense action scene, but it's technically a close up of a rear view mirror. So imagine you're a reader 500 words out of context, if it's a description of the mirror, et cetera. That's not super exciting. It's just a we know in the context. And the other thing about this shot that's interesting is it's a joke.  So this is not exactly the equivalent of talking about a contract in the middle of a shower scene, but it's sort of in the middle of an action scene. It changes the tension of that scene. So yeah, so this is Dan Brown and what the Da Vinci code looks like. I worked a little bit to find like a classic action scene, but I'll just let you like skim the style that we're talking about. Yeah, okay, there are giggles, right? As my next art project, I'm going to do some kind of visualization of the last sentence of every chapter of Dan Brown books because they almost all are three words and end with an exclamation point. It's just amazing. And there are a lot of chapters. He has 108, 130 chapters. Yeah. Okay, so text worked really well on the other problem, although I was really interested in structure. Text still really worked quite well. So I thought I would do the same kind of class where I took the stochastic gradient descent class where I tried to figure out exciting scenes and it didn't get anywhere. So I switched briefly to topic analysis. I'm not gonna spend a ton of time on this except to show you that this, because this is a complicated thing to understand the results of when you run it, it's an exploratory process that comes up with groups of words that effectively are trying to describe a document that, let me show you this picture. David Blay, when he made this popular a few years ago, produced this, this graphic doesn't matter that it's cut off. Essentially, you have a document over there. You have associated with it probabilities of content associated with latent topics. So, you know, the different colors correspond to different topics here and it looks like the yellow topic, which has those, that many yellow words is winning in terms of assignment for that document. So it's a probabilistic method. It produces a large amount of data if you wanna look at it and it's pretty complicated to understand the output. So digital humanities, people who are wrestling with there are all trying to do visualizations and it's something we could help with. This is Elijah Meeks doing a visualization. Networks are the obvious way to represent this but it ends up being these kind of like, there's a topic in their documents and they're associated and there's a lot of so what in this, in the outputs of this algorithm. I thought maybe topics would help me describe things like I'd see action-y words in some topics and like boring words in another topic but I'm not sure that I got a whole lot out of it but I went pretty far down the trying to visualize it route. The obvious first step was this arc diagram which you know, we've had many people already at this conference show dismiss, try in a Mike Bostock thing and then reject in favor of something else but see I didn't wanna do a straight network diagram because I have ordered data these are chapters going from one to whatever and I wanted to see relationships across chapters so since it's network data essentially I decided okay I'll just do it as an arc diagram and see what the structure of the topic assignments is and down at the bottom in this ugly kind of bar I have how exciting that chapter was according to a writer on Mechanical Turk. Okay, I looked at the set I don't make anything out of this there's some structure going on I don't know what it is. So I outsourced chapter descriptions to a friend who was looking for work had her write 108 chapter descriptions of the DaVinci code which she hated by the way she hated that book. She earned her money and then I stuck them under to see if there was any relationship between the chapter description she wrote and the topic and at this point it's like it's getting hard to read so I'm like hey I'm in SVG, I'm in D3 I'll just rotate that sucker with one line of code I mean I fiddled a lot with where to hinge it and do it and setting it and Doug's like ha ha ha SVG. So it becomes a lot more readable and definitely more interesting to play with and so then I added some stuff to help myself understand it like fade out. So if I'm on a topic there those chapters were all assigned to that topic and then over there is like a tool to saying what's the words in that topic were so I'll briefly show it to you I'm not gonna be able to do all of these but all right so essentially it works the way you'd expect it to work and I mean the main thing is that here we have topic chapter zero, topic 19 with these words the loop how the beginning of the DaVinci code as if you've seen the movie I cheated and I saw the movie it has stuff about the loop but this one at the top is super exciting and this one isn't so in a quick browse of this data I could tell that there wasn't a tremendous relationship between topic and excitement of the chapter you can see right there. I even added a switch to color everything by excitement on the text as well to help it be more salient for me and essentially there doesn't look like a close relationship but it's fun to play with so whatever this was officially the Crayola color scheme I'm not a visual designer I literally went for the biggest list of colors that are very bright and divergent and that was the one I used so what do I do next the obvious thing with what looks like network data please advance I made a network diagram so I borrowed my friend Jim Vlandingham's code and did a network diagram of the words in the topics and their relationship to chapters and excitement and I'm worried about time so I'm not actually gonna play with this but I'll put it up online and I made a filter on the top for exciting chapters and dull chapters and looked for any relationship among the words in the topics and I didn't find very much so this is a case where I had to use visualization tools to actually understand the outputs from LDA and then I realized that it was a dead end so bummer I probably needed another cute owl here but here's an accident in what I did I didn't prune word to topic relationships and I ended up with when I fade out over these nodes these interesting constellations that turned out to be a cool trick in the unlabeled state when I'm rolled over one of these you see these little constellations that's words that are shared across multiple topics it's kind of cool actually, it's a nice trick all right so onto the totally geeky but slightly more profitable path for another talk earlier this year I was using Nodebox which is a Python graphic programming like toolbox that's a lot like processing I was doing many sort of stuff with books and I plotted dialogue to exposition in these books just as this simple bar and I also at the time did Angels and Demons because it was on sale on the Kindle daily deals so and what I found please I had a lot more sort of non dialogue bits in the red bits are dialogue and when I put the slide up I was like is this lots of running around and stuff is that why there's no talking and I think there was something in that anyway so I went back to Python I cut all the books up by chapter got part of speech tags got punctuation counts got word counts counted everything I could count for each chapter imported the scores from Turquers and incorporated it and then did some graphing this is radar disagreements about whether something's exciting or not there were tremendous disagreements about excitingness in Dan Brown so that could be a source of problems now this is me plotting in Python just all the nouns by chapter look how much variation there is in that that in itself doesn't tell us anything interesting and then I decide okay I just need to smooth out this data so with the pandas data frame library I did a bunch of quick smoothing tricks with rolling means until I got something that started to show there's some structure over time this is nouns and I'm not sure this essentially probably correlates with something else I'll show you the rest of the analysis and you can decide what I should do next so I threw it all on axes including down at the bottom is the thin black line that's the excitement score in which you can't compare to anything so this is totally uninterpretable so I standardized to get them all on the same scale one line of code in Python re-plotted I'm working in an IPython notebook re-plotted and then at this point I was like well this is the point in the investigative process where you want something interactive again but I had just spent all this time building like cool network diagrams even though I was using some of Jim's code modified heavily and like I was like either there's gotta be an easier way for me to get a quick interactive out of this and that week my pal in Portland Rob Story posted Bearcart which is a little tool that lies on top of rickshaw and you can call it from Python to create a little D3 line chart in a few lines of code from so this is like how tools for data mining are going like we're heading towards these push button solutions to get something interactive that's built on top of D3 so that you don't have to do all the programming that I did I he had to fix a bunch of bugs that I was his first beta tester but essentially you get out of it this this is Twilight actually and all of the things I was counting are along the right with check boxes I can turn them on and off and see what seems to be correlated with what and obviously I could do a giant correlation matrix or any other stats as well but I wanted to just play with this and see the structure visually so I did find a few oddities nouns and verbs seem to go in this cyclic thing through the course of Dan Brown and in the Twilight book there's sort of an inverse relationship in terms of which one is the higher proportion per chapter and it's very cyclic I need to investigate a little further what's going on there these are the excitement action arcs based on what the Turquers said was exciting in these books so Angels and Demons follows a fairly classic rise towards the peak action scene and then fall Da Vinci code they thought the beginning section was more interesting and I actually think having rewatched the movie recently that that's because there are all these revelations all the stuff about Mary Magdalene sitting to the left of Jesus in the picture and all this stuff like all of that stuff is pretty exciting there's a lot of running around early on so I think it's actually pretty accurate now it did turn out that the amount of talking was inversely proportional to the sort of action scenes in a bunch of places and only one book was statistically significant in a logistic regression but I need to do a couple of other explorations so I think I was right in my initial stupid little bar chart where I said lots of running around and stuff where people weren't talking so the dumb little picture wasn't totally wrong there you go and Twilight has a bunch of talky bits and if you invert that you get maybe what's the action I need to go check what's happening in chapter six and I don't know, I don't remember so I'm totally out of time so I'm not gonna bother showing you this is based on the same pattern but my next stage was oh god I do need another tool because I don't know what's happening at these peaks where the excitement is so I loaded in the chapter by chapter for each of those little blocks and then I can color the bar underneath the red bars with the metric that I'm interested in and then roll over the block to see what's happening in the chapter of those books so it's a visual way of browsing the text to figure out how the metric corresponds to what's in the actual text so basically this for me was a fun investigation of some essentially pot boilers but ultimately it was also a test of how awesome D3 was for doing very quick visualizations of data that was really too complicated to understand especially the machine learning output was too complex for me to actually see or figure out even how to visualize without something interactive and so we as data vis people and as data scientists should be thinking hard about visualization tools to help people with these algorithms I think that that's a big frontier for us and the other thing I learned in the course of doing this of course was that you could put an insider novel easily in a browser and have a little interface around it looks like you're working all right anyway thanks a lot guys


I was raised here in Pasco, Washington. As I was growing up here, I grew up in a split home. So my mom and my dad are always arguing and fighting. And there was really no attention on me and my siblings. So in the middle of all this, I kind of just started hanging out with the wrong crowd. And I was able to get out of the house with nobody really paying attention to me. So I ended up becoming involved in this gang. In this lifestyle with the drugs and kind of like trying to fit in and be a part of the scene. Because I really don't have a father figure, you know, like my dad's always gone. So these gang members that I'm being surrounded by, they become my father figure. And I see that as there are people that get respected and people that get looked up to in the school. So that's who I want to be. And so I end up becoming involved with these gangs. And at the age of 15, we get arrested and stuff. I get arrested for possession of a firearm charge. And I end up going to do some time for a minor in possession of a firearm charge. And I end up doing some time for that.  And so it just kind of makes me get more involved in the gang because I don't have to go to school now. So I'm kind of more in the mix of all the gang stuff. And at the age of 20, I'm about to turn 21, I get arrested for a felony rioting charge and a felony possession of a firearm. And I end up having to go do a prison sentence. So a friend of mine, two friends of mine got shot. And now they're saying that now I'm starting kind of catching up on everything that's been going on. And another friend of mine got shot and got shot with a finger in his head and he survived and he was good. But, and as we're driving down Coors Street, the rival gang pulls us up right next to us. And at first I don't notice it, but I hear like someone yelling and stuff. And so I rolled down the window and I noticed that it's these guys, you know? And I have my kids in the back in the car seats and stuff. And so they're over here and one of the guys waves a gun. And I mean, I had a gun as well in the car at the time, but I didn't want to shoot or do anything because my kids were in the car. And so I told my friend, I was like, hey, just take me home. And I kind of sit there with the kids and I'm kind of panicked a little bit too because the situation could have gotten really bad. And I knew if they would have shot the car, they would have shot my kids or just the car alone. So this girl tells me that I should go to church and stuff like that. And so I go to church and I'm at the church and the message that they were preaching kind of was speaking to me like to what was going on in my situation there in that time. I wasn't a person that had a lot of patience so I want stuff to happen immediately and stuff's not happening for me. It's already been like a couple of weeks and Pastor Tony had gave me this vibe when I'm reading scriptures and all this stuff. And I was like, man, like I've been praying and nothing's really happening. I'm already looking for other alternatives to instead of just waiting, forgot to come through to like start selling drugs or something so that I could get myself back on my feet. That week I had gotten with my brother and gone to a dispensary because when I went into prison there was no dispensaries and I got out and weed was legal. So I had gone with my brother and we had been smoking dives. And now I'm here like a couple of weeks later and I know it takes more than 30 days to get clean. And my probation officer calls me and he tells me like, A, you need to come in. You were supposed to be here yesterday. And he pulls out a U.A. Cup and he tells me he goes, can you give me away? And so I was, I kind of stared at it for a little bit and I was like, oh yeah, yeah. And he tells me like, are you clean? And I don't know if he could tell that I was like nervous and I told him, I was like, oh yeah, of course. Yeah, I'm like, I'm clean. But I remember I was saying this little short prayer and I was like, God, I was like, if you're real I need you to show up like right now I need you to help me. So when he gives me the cup, I look at the cup and I'm clean like, like THC is all normal. Everything is normal. And I'm kind of like, at all a little bit I'm kind of like, oh my gosh, like I'm clean, you know? As we're driving back home and stuff my sister was a Christian and she was also trying to get me to church. And I just never really went with her. So she's listening to this radio station called Caleb. And as we're driving, there's this guy and he comes out and he's on the radio and his name's Luis Pallon. And he's talking about how in the Hebrew language there's no word for coincidence. And he was saying that that is happening is because God's ordained it that way. And that the Hebrews believed in that. And that, so the sooner that we stopped saying that it's not a coincidence or that it's luck but we started acknowledging that it's God that the more we'll see the blessings poured out upon us. And it just like, to me it just hit me and it hit me hard like God was like telling me like, no, he's like, it's not a coincidence.  And so I'm looking out the window and I'm just kind of like lost. And so as we drive home, I turn around and look at my sister and I'm kind of emotional. Now I'm just seeing God move and like, I could just see him that he's all over me. He's kind of like pushing me to do like go to church. And he's like, I feel like he's just answering me like everything that I'm asking is just happening fast. And I get to church and the service that they're having is just, it just hits me again and God's all over me. And I just feel like emotional and they call people up and they're like, if you've never given your life to God or to Christ to come to the altar, I'm just like emotionally broken and I start crying at the altar. So I cut out like everything I cut out on other relationships, other friends and stuff like that.  I cut out doing tattoos because I was doing tattoos as well. And I know that that was something that God wanted me to cut out. So that was something that was hard for me because I was making good money and it was something I didn't want to let go because as a kid I was always like drawing and doing art. So I ended up cutting that out too. And I already knew that it wasn't what I wanted, that it's what God wants. And that's what brings me joy is doing what God wants. So that's what I decided to do. And that's what I'm gonna continue to decide to do as I continue forward.


Thank you very much Chairman. Good afternoon. Excellencies, dear delegates, colleagues and friends. You might not have recognized that Tune is perhaps not quite as popular as Chase Herbs, but it's very well chosen, I have to say. I don't know who chose it, but it's a very nice song. It's a Welsh song, as I'm from Wales, as you may know. So that was very heart-rending. Thank you very much for that. Chase Herbs said he'd been an ITU 35 or 36 years, I think, but I can beat him. In fact, it's 40 years, this year since I first participated in ITU. And I must say it's been a truly wonderful experience working in ITU, firstly as a delegate and then as Director of TSB, and finally as Deputy Secretary-General. So I'd like to express some hopes for ITU's future, which as we know is somewhat challenging. I believe that ITU's unique tradition of reaching decisions by consensus based on the willingness of everyone to compromise, together with its reliance on regional preparations and regional common proposals, will see ITU through these rather turbulent times. With so many organizations, including UN bodies, now using digital technology to achieve their aims, there's an increasing risk of duplication, which makes my three favorite words ever more important, and that is collaboration, cooperation, and coordination. I hope ITU will continue to focus on its core competencies and provide duplication of effort and concentrate on pooling its limited resources. ITU has always kept in the forefront of technological advancement, and this will be very important to further improve the working methods, such as paperless working, teleworking, and flexible working, remote participation. These are all integral to the modern ITU culture, which will be in the new headquarters building. That's one regret, of course, I have, is that I won't see the new building project through to completion. The new building will be a headquarters worthy of ITU staff and delegates well into the 21st century. So let me again give my thanks to all those administrations that have sponsored and supported the project. Since joining ITU, I've always promoted work on climate change, accessibility, gender, and bridging the standards gap. So it's very nice to see how these have progressed in those years. And I always recall when I published the first ITU report on ICTs and climate change. In 2007, it attracted so much skepticism. I was always being asked, what has ITU got to do with climate change? What have ICTs got to do with climate change? So it's nice that that question is not raised anymore. It would take too long for me to name all the incredible people I've met and had the pleasure of working with, both inside and outside ITU. But there are three persons I have to mention. Hulin Zhao, of course, who I've had the pleasure of working closely with for very nearly 16 years, and who has become a close friend. We always seem to have the same view on everything. Sophie Beraz, who has been an outstanding assistant to me throughout all the years, I would have been totally lost without her. And my dear and long-suffering wife, Catherine, who has always been my support throughout. Many of you know her because she's been at my side at every Planet Potentially conference from Minneapolis to Dubai. So she sends her best wishes to you all. But of course, I have so many friends and colleagues from so many different countries, so many administrations within ITU, Secretariat. Let me just say thank you to all of you from my heart for your personal support to me, for your friendship and your collaboration over the years. I also, of course, am very honoured and obligated to the UK Government for its strong and vital support to me in all my elections. And I particularly like to thank all the hard-working members of all the campaign teams that have worked hard for my election. So I look forward to a new era for ITU, a new management team, a new council, including, I'm pleased to say, the UK, an invigorated ITU that will face the challenges ahead. Let me wish you all every success for the future. Thank you and good luck.


Teaching ESL and knowing your audience, young learners versus adults. Whether speaking at a public event where thousands have gathered, or talking politics around a table with friends, this simple piece of advice is well known to anyone who has ever done any public speaking. But, it is also critical advice for the aspiring teacher, whether those teaching English as a second language, or those teaching chemistry in high school. The more well acquainted a teacher is with his or her classroom, the more successful he or she will be in engaging their minds in the topic at hand. So, of course, this is a vital component to any teacher who is seeking to help students wanting to learn English. It is important that the teacher asks such questions as, what are the students goals in taking this course? What do these students hope to be able to communicate when this course is through? Who has encouraged the student to take this course? But, a large part of how these questions are answered is determined on the age of the class. Herein lies a distinguishing characteristic between teaching young learners versus adults. Just as it is important to know your audience in any type of public speaking, it is crucial that English teachers know the age range of their students as this drastically influences the topics that will be covered, how the information is presented, and how to keep the class engaged. Motivations of Young Learners With young learners, they are often taking English courses because of some sort of encouragement from their parents or families. It also means that their English level will likely be less advanced, simply because they are not told enough to have much exposure to English. Most of their knowledge of the English language will come from TV and music and they may not have a high literacy or retention rate. So, generally speaking, the topics that will need to be covered with them will be basic topics, such as introducing themselves, describing objects, talking about their families or hobbies, etc. The goal of teaching them these topics is to help them gain confidence in expressing these basic conversations. They likely won't be going for a job interview anytime soon, or applying to a university, but what they will learn in their youth will be the foundation they build upon as they expand their vocabulary for those important phases of life in the future. It's important that they are learning good pronunciation in basic conversations that they can build upon, helping them to speak with confidence in basic conversations will serve them well for their future English studies. Information Presentation Another unique characteristic of young learners is that the information needs to be presented in a way that engages all their senses. Songs, rhymes, hand motions and lots.of movement is helpful and encouraged when teaching young learners. They will retain this information much more readily if it's not just dumped upon them, but rather if they are engaged in what they are learning. Teaching young learners is the time to let your creativity as a teacher out. Think of ways to let them talk about the topics, sing about it, learn hand motions, and even touch, taste and smell the topic, where appropriate. If they are learning colors, let them do an art project that involves them following instructions of which colors to use. Let them use finger paint and touch those colors as they are given instructions. If they are learning verbs, let them jump and clap and dance as much as possible. Language Retention This leads into the final distinguishing factor with young learners. The more involved they feel with the lesson, the more engaged they will be. Adults learning English may want to cram vocabulary and learn tough business words. But, young learners will retain much more if they have learned the vocabulary experientially. Teenagers will remember new words if they are from a favorite pop song. Elementary age children will continue using their new vocabulary if it's put into a game form that they can continue playing with their friends. The more interactive, the better when it comes to young learners. So, those hoping to teach young learners must know their audience and know that teaching young learners is very different than teaching adults. The more the teacher enjoys her class and lets creativity be unleashed, the more potential there is for the students to excel and build a healthy foundation for learning English for years to come. Take a TEFL course and start learning to be a better teacher. Speak with an ITTT advisor today to put together your personal plan for teaching English abroad. Send us an email or call us toll free at 1-800-490-0531 to speak with an ITTT advisor today.


So on that creative theme, I'll hand over to you to hand over to our last keynote of the day. So we'll end the day with a special presentation by Diane R.C. For me, it's a special presentation because Diane is an artist. And I think this is Diana, if I apologize, extremely important. Art is something that's a wonderful commentary and a voice that's slightly different from the voices I typically hear at conferences, you know, presentations, panels, research reports. And sometimes this voice is very missing and it's also a voice that can be extremely strong and provide a very good or alternative view. So I think we will, with Josie, very inclined to have this kind of talk that maybe some might feel kind of stretches what is a typical presentation for us. This is exactly what we'd like to hear. Diana, Diana, I apologize. In Poland, this is a Polish name. Diana. I just say Diana. Okay. Diana. I'll get there. It's an artist who does socially engaged work. I won't go into details. You'll probably talk about your work. But Schubert said that space where art connects basically with social issues and social justice in one of her projects even explicitly connects activists with artists. I think there's no one better to tell us what's art's perspective on open education, open educational resources and education in general. Diana, please welcome. I have the hand up. Do you need to switch something for you? Yeah, I got this. You got it. Okay. And the clicker? We practiced. The bam. Okay, so my name is Diana Arcy. I am an artist living in Berlin. I am what in the art world we call a social practice artist. But I call myself an activist artist. I'm interested in the intersection of working with audiences to engage them in social, political and cultural ideas and finding ways to connect with them in order to bring forth the issues that are the most important to all of us. And specifically I'm more interested in actually reaching people who do not travel in art spaces, people who do not travel in academia, people who are disenfranchised, people who hate politics. This is basically the kind of people that I'm looking to work with. So I do this. I'm also out of starting doing this work. I've been doing this work since the late, what, 90s? 99 I started. Yeah, so in the process of doing this work I realized that I also had a knack for helping other artists engage with these kinds of topics. And so I started an organization. It's called Artists Without a Cause. We are artists with a cause. We just don't have the organization backing us. So the idea is to connect artists like me and a lot of the artists like the ones I'm going to be presenting today with organizations who are working on different issues in order to amplify your message and to reach out to people who you want to talk to who you maybe normally wouldn't encounter. I'd like to thank Josie for inviting me. I met her at Open Knowledge Festival in Berlin in 2014 where for the first time ever I was able to bring a bunch of artists to show people working in open data how we can story tell with their data. I'm not going to present too many examples of that because I'm sort of, there's some examples of it that we will see. So who knows what this is? Yeah, Wall Street. So we know the bull, right? And you've heard about the fearless girl, correct? Okay, so some interesting points. Actually I wanted to start my timer, sorry. So Arturo de Modica was the guy who created the Charging Bull. The Charging Bull was created in 1989 and it was placed without a permit in this square. Okay? The original point of the Charging Bull was to celebrate the can-do American spirit and especially the spirit of New York with people from all over the world who manage under horrible circumstances to make something happen. His idea to place it here was actually stemmed out of the financial crisis that happened in 86, 87. So he thought placing it here would be a great place to put it. But it had nothing to actually directly do with Wall Street. Very, very interesting. So the location of this bull turned it into something else. This fearless girl, who's here's a close-up of her. The fearless girl, she mysteriously appeared on International Women's Day. She did, she was permitted. She did have a permit. And she happened to be created by State Street Global Advisors which is a hedge fund group. Yeah. So the artist who created this, her name is Kristen Visbal and they worked in collaboration with a big advertising firm, McCann and placed her in the square. So this is what it looks like. This is her looking, staring down the bull. It is very powerful, but it's very powerful for what I believe are the wrong reasons. The idea of this from the advertising firm was this supposed to represent more women in the workplace, more women in CEO positions, in Wall Street firms, and so on and so on. I think it's kind of funny that they chose to use a little girl instead of a woman. A little bit problematic. But then the thing is too is if you think about it, how much more powerful would this have been if it was placed by Planned Parenthood? How much more powerful would it have been if it was a different organization who chose to put it there? What would the message have been if someone else chose to do it? And the bull lost its message in the placement of being placed on Wall Street. So there's two key points that I've learned in my work as an artist and working with social and political artwork is that location is everything. Location is everything. And that it's even more important to also change the format because this is a very passive format. This is something that people can come and take a photo with, but the experience of this, it's fleeting. You take a photo and you're done. You weren't actually engaged in any production. You weren't engaged in... There was no active engagement within the work. So of course, location is everything. There are several things I notice when working on finding spaces. So these two aspects are not... There's not one that's more important than the other. And I'm going to try to talk about one and then the other, but one is not more important than the other. When people think about artists and working with art and working with creatives, whether that's traditional artists or people who do more experimental work, they always think about a very traditional art space. And what I like to say is that if you're working in traditional art space, you really need to think about who the partners are because if you don't have actual partnerships with local communities and you cannot break down the barriers of the art space. Art spaces are places that not a lot of people actually go to. They're very homogenous in the people who go into them. So this is a picture of the Oakland Museum of California. They're in a museum of the actual state and they do a really interesting program called Friday Nights at OMCA. And it's using the model of the block party. So they're in Oakland and they're in a very black neighborhood in Oakland and they have used... They've worked with a partner that works with local food trucks, which most of the people who operate food truck businesses are also not people who tend to be in the museum. They work with them directly and they throw these block parties every Friday. They're free, they're open to the public, there's plenty of place to picnic. It's a bit of dark photo, but there's all these people, they're all dancing. And so using another tradition within the space has made the space more accessible to other kinds of people. This is a performance that happened in the space and actually it's a performance in which they work with the people in the public to teach them how to also perform. And then in addition to this, you have local people in the community who are selling food, selling products, there's music. It changes, it makes a space a different space. It has now turned into a space that is now active for other kinds of people. This is for me the best example if you plan on working with an art space. If you are interested in actually working with artists to get your message across and you want to work with the gallery, if they're not doing something like this, then you should just go back to the drawing board, is my suggestion. This is a short clip. We will go back. Okay, so this is a short clip. This is on 42nd Street in New York City and this is around the corner for where Enron happened. This is a piece that I created in 2004 and what I'm trying to also say with this piece is that if you have, depending on what your message is, regardless of what your message is, you should try to go to where the people are. So I placed myself right on 42nd Street between Grand Central Station and the U.N. Building and everybody who worked for any major corporation had to walk by for seven days and watch me do this. It took seven days to fill this room with paper and every day the same people kept walking by and every day people would come, they would come up to the window, they'd knock on the window, they would talk to us. One guy actually offered us his paperwork that he was shredding in his office. I mean this is I think the sixth day. This is around when we decided to, this is when we were slowing down. But yes, so you go to where the people are because it's a much more powerful message if the right people see it. Examples of these kinds of spaces go, you can think about working in schools or universities, that's an easy one, but pubs, the cultural centers, women's centers, restaurants are a wonderful place. Public space, depending on where you are, is a fantastic place to catch people. Parks are great squares. And also if you want to get very, very interesting, if you want to go into actual people's homes. If you work directly with people, a lot of different things sort of happen. So when you get to this point, one of the things you have to think about are the demographics of the kind of people that you want to reach and what is the message you want to bring across to them. So this is another piece, sorry. This is Project Row Houses. This is in the third ward of Houston. It's a black neighborhood. It's been going through an economic downturn since forever. This project was actually founded in 1993. And there was a row of 27 row houses from the 1930s that were dilapidated and falling apart. And there was no community center and there was no real interaction with most of the people in the communities outside of the typical bad stories you hear about gangs and stuff like that. There were no community centers. So Rick Lowe got together, he's an artist, and he got together with a large group of community organizers and he got a small grant. He got a grant for $25,000 from the National Endowment of the Arts and that money was used to refurb these 27 houses. And he used it to refurb these 27 houses. He engaged the community in building this. So this was not built by him and it was not built by his partners. It was built by everyone who lived there. So as you can see, there's a number here of houses, 2505, 2517, are art projects houses. The rest of the projects, the rest of the houses were used as temporary housing for battered women. Artists are invited to come to this space and to make work. People can interact with it firsthand and the artists are encouraged to, if you get to work there, you're encouraged to work with the local community in building your products. So they work with parents, churches, youth programs, and senior groups primarily. And then what's often happened with this project is that people who participate in this, young people who participate in this, they come back when they get older and they work here and they do different projects here and they come back also as artists. So he got a MacArthur Genius Grant in 2014 for this work. And they've been thinking about a while for how they could possibly replicate this in different places. One moment. Yeah, so it's an art project, but it's something that revitalized an entire community. It didn't cause gentrification. The people who participate in the people who work here and the people who live here are people from the community. And now the community has a space in which they can do all the things that they want to do and have a place to also reflect about how amazing their community actually is. This is an artwork. This is where people like me, this is what we're thinking about. So we're going to come to one of my projects. This is a very interesting project. This is White Guilt Cleanup. So when we're thinking about location, one of the very, you know, what about the internet, right? The internet is a very important location. And in my experience as an activist, I have realized that many other people of color were dealing with a lot of trolling, were dealing with a lot of explaining, you know, getting exhausted for explaining to white people what white privilege means. And so I decided to provide a service, not only to people suffering from white guilt, but from victims of white guilt. And like with this project, you know, I asked these questions, I asked the question of who is my audience and who do I want to reach. And I've been able with this project to reach both of these people. It exists primarily only online and through consultation, one-on-one consultation services. So this was an ad we placed in a music magazine. And through this ad and through Facebook and through social media, we have done one consultation for a major fashion magazine in Berlin. We are also doing a consultation for two magazine festivals where they want to incorporate, they want to be more inclusive of people of color. So this is the offset credit. We were thinking, in this project, I was thinking about different models of how to sort of finance this project. And one of the things is like, well what could, what would happen if you could sort of get rid of your white guilt for something? You know, if you would be relieved of this for this moment, could it make a better ally? And how could we make a better ally and a better accomplice to people of color through this process? So I have sold one of these. I have sold one offset credit for 200 euro. I have given a consultation for 25 euro for one, in which in the end we decided not to give them the credit. You can't just, we can't forgive everything. Thank you. And it's this interactive, it's the first time I've actually ventured into sort of object art. So it is this object, but the object comes with a process. You can't just get one, you have to have a consultation. And through the consultation is how we then can work with the individual on recognizing what the issues are with what they have done or what they're dealing with and find better ways and better solutions to move forward. So we didn't actually give the magazine an offset credit, but they actually deserve one. This magazine in Berlin, they're called Indie Magazine. They're pretty famous in the German speaking world. They invited a white woman to give black hairstyles to people who are coming to this workshop. And it turned out that she was also from Holland and she did blackface. So of course the people of color community in Berlin got very angry. We were alerted to this by other people of color. They called us in. There was a message sent to us. I said, white girl, clean up, bring your broom. And in this process of everything I do, we try to do it in a very positive manner. So we're never yelling at people. We're never getting angry at people. We're just like, hey, have you heard of cultural appropriation? Here's something you can watch. You know, here's a great article about this topic. On our Facebook group, we post pro tip do's and don'ts. We talk about current events. And it's always in the context of how to be a better ally. So in the end, the magazine, they ended up hiring us and we put a panel together for them on cultural appropriation and fashion. And we gave them a list of demands and including more diversity within the actual pages of their magazine. Not only models, but photographers, designers in their cultural section, more writers of color. And from the year that we first met with them until we did the panel, they actually did all of these things. So even I was surprised. I didn't think that this would work. I thought I would get somewhere, but I didn't think I'd get this far. So it's kind of amazing what art can do if you let it. Yeah, so this is the Democratic National Convention in Boston in 2004. And the reason why I'm showing this is to also, when you think about working with art or working with artists and thinking about not using traditional spaces, you can sort of circumvent these physical barriers because you're not a protest, because you're not a conference, because you're not any of these things. So this is the free speech zone. This is where they rallied all the people who they didn't want at the convention. And they kept them caged in there. And they couldn't go, they couldn't travel freely in Boston. 2008 when I was there with Polita Oki, you could not travel, in 2008 the Republican National Convention was in St. Paul, Minnesota, which is right next to Minneapolis. You have to cross one bridge to get there.  I could cross this bridge. I went, I came up to a barrier, there were all these cops and they said, what are you doing? And I was like, I have a karaoke show. And I was allowed to cross the barrier and I was allowed to interact with people who were actual attendees of the convention. So this is also sort of the amazing part, how art can also function as this amazing Trojan horse.  They're just doing this. And so it's very easy to let us out, to let us through these barriers. This is an image of Polita Oki in Jerusalem. Now, when you're working within public space, you have to also understand the context of the public space that you're working in. So this image, this is a bar with an outdoor patio. And the second image, this one here is a public square. What you cannot see in this picture is that this entire public square is surrounded by police protection and tons of barriers. This show I actually did not want to do originally, but then was kind of pushed in the organization to do this show. And the show that I originally, the only kind of shows I wanted to do was this because we could circumvent police protection and barriers by being in a private bar. Even though the bar was in public space and the bar was, there were no barriers for anyone who was walking on the street to come and enjoy what we were doing. So it's not as easy as like A or B in this sense. But this is a thing though that like, you can still, you can do this with a karaoke show, but you can't do this with a protest. You can do this with an artwork, but you can't do it with a protest. So the key points here to remember are that if you're going to work in traditional spaces, make sure you have local corporations to bring the community to you. Better yet, don't work in traditional spaces and go to the places where the people actually are. Know your audience or know at least who you think you want your audience to be. And also always remember that messages that could be perceived as political or protest or like anything in this sort of realm can be circumvented by art through this Trojan horse mechanism. Now I'm going to talk about changing the format. And I mean, everyone's noticed that here in these rooms, like on the outside, there's an ex... I don't know, has everyone noticed that there's an exhibition out here? Yeah, so there's an exhibition out here, but can anyone tell me like a particular piece that they really loved that they saw?  Okay, so there's a few people, but this is like what? Four, five, six? Okay, there's a little bit more, but there's not that many. How many people just walked past it? Yeah. Traditional formats are absolutely wonderful and they're fantastic and they do great work, but they're not the only way of working. And one of the things that we run into at Artists Without a Cause at AWAC is that so many organizations are like, oh, let's do an exhibition or let's do this. And the issue with this is that it is passive. And I don't have to explain to you guys experiential learning, you all know this. This is not something we need to go over, but it's important that when you're thinking within an art context and you're thinking about sort of transmitting messages that if you can fully engage the people into the process of creating and understanding the knowledge that it will stick and that it will also function as word of mouth that it will get around more. So don't just make something to look at and don't just bring in people to make something to look at. What you can do, you can bring in people to make something pretty fantastic like this. This is Conflict Kitchen. They're in Pittsburgh. They started in 2010. And it's a takeout window restaurant. And it only serves food from places that the United States is in conflict with. And they use every aspect of everything you can imagine that comes out of this place. So all the people who cook there come from these places. All the food comes from these places. The wrappers are full of information about what the conflict is and what is going on. Everyone who takes orders is also trained to sort of talk to people. And it's a work that like... So basically what they're trying to do is they're using sort of the socialness of food as an opener in order to engage people with the topics of these places and why we're in conflict with them. And they do, in addition to just serving food, they also have a lot of side programs. So they work with local organizations. They work with community groups, cultural groups. And they come in there and they hold events together. They also hosted a dinner, a Skype dinner with a bunch of people in Tehran as one of their projects. This artwork was created by John Rubin and Don Willeski. And it started in 2010. They actually opened a second location for a few years. And then now they're back at the single location. And their main goal is to sort of to counter this like clear government rhetoric about, you know, these people are batting, to give an alternative to the media headlines of these places. But they don't ever tell anyone what to think. There are many incidents there where people come and they're very angry, but the people who work there are trained in communicating with people.  And by not telling people what to think, they create this open space where people can really learn and people can start to think about these ideas. And if you enable these choices, then the audience can maintain their autonomy and it increases the likelihood of their willingness to listen and engage with other ideas that might not necessarily be their own. This is Polita Oke. Polita Oke, my name is Polita Oke. My name is Polita Oke. My name is Polita Oke. My name is Polita Oke. My name is Polita Oke. My name is Polita Oke.  George W. Bush. I'm actually interpreting a dance, a petite contract. If you don't want to say something, there is a limit to what you can do. You really don't understand and you don't understand a normal person like Donald Trump. He decided to do it in all the accents that he could come up with where the wrangling was on the rise. The deep cuts and the government spanning and returning power to the people. That, that we are. So that is Polita Oke. Here is a picture of Polita Oke. And these are some of the interesting cavits of this work. It does not exist without an audience. The work is not finished. The work is not the karaoke videos. It is not the moderation. It is the audience. Without the audience participation, there is no work. And within that, they become direct collaborators in the creation of the work. And they become owners, co-owners of the work. We, so the project itself, we pull together as many speeches as we can find across as many different political ideologies as we can find and we make all of them available. The audience comes, they pick the speeches that they want to give and then they are free to give them however they want or free not to give them however long the speech is. That is how long they get the microphone for. And they can come up multiple times. It is a pretty easy concept. One of the important things about it though is that the project itself can actually turn very easily into propaganda. So it is about maintaining through the moderation of the project and with the political discussions that come out because of the project, maintaining a sort of sense of neutrality so that the audience members can determine their political ideologies freely. But the project itself has no political ideology. And this is how we keep it from becoming propaganda. So this is in a public square in Minnesota. And this is a very young boy who decided to participate in the project. This is another image of another show. It is very dark. You cannot see it. This is, okay, here you can see it. It is like full of people. It is a bar. This is in Dortmund. And here. And so, yeah, so this is Polito Oki. So what I would like to end with are some suggestions about how to work with artists both in the short term and the long term. Both in the short term and the long term. So one of the best ways to do it is to combine forces to make it possible to spread your message. Your work and the tools that you are creating with the people that you haven't been able to access or reach before. So if you work together with this artist, you can make these determinations of who these people are or what kind of groups you want to target. And they can sort of think of creatively what is maybe a way to do this. One thing that I did with Polito Oki was that I worked with an organization named Declare Yourself. They no longer exist, but they were a voter registration organization. They had advertisements with Christine Aguilera and all these big fancy people through these big parties registered nobody to vote. Everybody was already registered to vote. People who hated politics came to Polito Oki. And they registered, you know, in a show of 40 people, they made 20 voter registrations. And so it was a very fruitful collaboration. So this is one of the things that you should do. I think one of the other points is that it's also using artwork as a way to pick up people after the momentum is already built. So in making this kind of work, in making White Goat Cleanup, in making, you know, in Conflict Kitchen, Conflict Kitchen, they work directly with local community organizations so people know where they're supposed to go. Other artists who are making this kind of work, if you don't have this connection already, this is a great way to introduce whatever the work is that you are working on in order to pick up people who have already expressed interest. And then another great thing is that any data or information that's produced as a result of the work is something that can be released in open platforms. So I have a very empty... I have a very empty platform, like a very empty site on Say It, which is an open source transcript program that was started by my society. And the idea, the hope is, is that I can eventually release every single transcript that I create for Polito Oki onto this platform. That means educators have access to it, journalists have access to it. And that means no longer having to dig through PDFs on government websites. So for me, that's something very, very exciting. Now you have to think about where would you find these artists? So you can come to someone like me, of course. This is something that we do at AWAC. We research and gather this information on these different kinds of people. You can reach out to different universities or art groups. If you would do something like an open call, it's something that I don't generally suggest, because especially in these days and time and how precarious the art world is for anyone who's an artist, anything that's attached with any kind of funding will lead to thousands of applications that you won't even really know what to do with, unless you write these things in a very sort of specific manner, that it's very, very clear that someone already needs to be engaged in the topic. That's very, very important. Almost through, if you're skeptical about these ideas, because this is not really the easiest thing to sort of just accept as something to try out, or if you're only thinking about it in a more traditional way, one of the things you can do is that organizations like my own, and then there's a wonderful, fantastic organization called Center for Artistic Activism. They're based in New York, but they travel worldwide. They do creative activism workshops where they really explain how to think about these processes from an artistic perspective. All these workshops are framed in a way where you do produce like a work in the end, but it really gives you sort of insight as to how these processes could actually improve different kinds of campaigns or different kinds of topics that you want to make more widely available to other people. And then this is a great opportunity for both organizations and different people working on different topics and artists, because this is a way to also pull resources and share information, share research. If you support artistic partners with these sort of things, you can lead to better targeted projects, and you can make this part of your actual strategy. One of the things to also think about is that art is also something that can be used as a multiplier, thank you. Polita Oki, for example, we're in the stages right now of turning it into a toolkit. We want to make this available to other groups. We want to make this available to other people in other countries to make it in other languages and make it more available to other people to use, especially if they're working on topics like voter participation or elections. For these kinds of things, too, though, you also have to remember that like open access doesn't, fully open access doesn't always work. Projects can easily be turned into propaganda if you're not careful in who are the stakeholders in it. So, something to think about. This is a, this is a resources and reference list. This is the AWAC website. Actopedia is a wonderful project that was put together by the Center for Artistic Activism, and I believe the Yes Men were involved. This is where you can find just like different kinds of activist art projects that different kinds of people have been doing. Open engagement is a conference that happens every year. It is a gathering of artists who work in social practice, who work with politics, social issues, environmental issues. That's a great place to find people. And then again, artistic activism. They do tons of research on artists who are making this kind of work. And like also in the same, in the same side are looking at ways for how to better incorporate this into, into larger practices. My last point is I wanted to say to, in the long term, this is Chris Lehan. He's a political strategist. He used to work for Al Gore. Now he works for the state of California. And he basically makes art. He, they were trying to pass a law in Sacramento that his employers decided they didn't like. And it was about trucking, I believe. And he surrounded the, like the state house with truckers who were like honking their horns so loud that they couldn't hold their session. It's a performance art piece. Right? And think about it. I mean, this guy, he's not really, he's not really the funnest looking guy, right? You know, if you would think as an artist, as a strategist, you know, I mean, he's tough. So in the long term, I'd like everyone to think about maybe thinking about the artist as a strategist and like how you can bring them into your organizations in a more longer term context, maybe through fellowships or grants and like having them within your organization because it'll just lead to better projects on a whole. So don't go it alone. And to give us a little break from Chris Lehan, this is my bunny. His name is Peter. And yeah, that's basically it. That's all I had to say. I made it to 31 minutes. Do we have time for questions? Some questions? All right, back there. Do we need to bring our mic? I can wait just wait a second. Thank you for that amazing presentation. I think we've been having quite a lot of discussions today about how we articulate what we want to do with open education and how we move our language, our work forward. And I was just wondering if you'd done any work with education, you know, in a university or in a school or maybe collectively, I think this is maybe a question to everyone, if actually maybe we need to collectively start thinking a bit more about how we harness art in terms of how we are telling our narratives around open education. But I was just wondering if you had any examples to share about open education. I've done some work mostly in primary and secondary education. I've done some university work, but I find that the least interesting. Mostly in the arts. The Germany has a very studio based practice and I like to read a lot more. So it doesn't really go over well in Germany if you tell a bunch of students to read 200 pages of something if they're in art school. So I've done some work in arts education, but it's arts education. So it was mostly based in after school programs in poor economic neighborhoods and finding ways to sort of engage people. Yeah, that's been most of my experience. I haven't directly worked with anyone, like any of you actually. This is my first time. Is that an offer? Yeah, it's an offer. I think I want to take that offer and start to think about just thinking about all the ways in which a lot of our activism isn't taking place where people are. I'm just, my mind is blown by all the different examples you gave and I think I'm going to keep thinking about this for years. But I will definitely stay in touch with you to try to make things happen more in my own context and try to imagine how it would work for the open education community and what those kind of spaces look like online also and not just face-to-face, but also face-to-face. A lot of what we do in the open is online, but it doesn't affect the people in our own face-to-face lives and I'm just trying to think about that. So thank you.  For me, the dream would be is that being an artist, working with a group of people, working in an organization, really seeing and getting to interact with what they're doing, because I think that's how you best understand what they need to say.  It's a big ask to let someone from the outside to come completely in, but I think in doing that, you enable the artist to be able to, you know, if I come in there or someone I know comes in there, I think we would be able to tell you what you're trying to say. So I'll jump in, and then I'll hand it over to somebody who maybe has a question, so I'm going to do that thing that academics do, where we don't really have a question. We just kind of talk about, you know, how we feel about things, but yeah, I think that you're absolutely right. And I wanted to say that I feel like this community has really got a lot of artists in it, right? This community particularly, that's one of the reasons why I avoid some of the more corporate conferences about buying a product than it is about creating something. So I think that, you know, we're kind of ripe for that kind of conversation. I think it was a question over here. Hi, thank you again, like everyone else is saying. So I recently saw a documentary called Hypernormalization by Adam Curtis, and there was a segment where they were talking about this advisor to Vladimir Putin, I think his name was Vladimir Serkov, and apparently he used a lot of our techniques to just kind of mess with the political environment, create confusion, and actually further authoritarianism. I haven't quite figured out, I haven't processed your talk yet, but I think something about what I've found really inspiring, I think others did too, is the way you're not just provoking and just kind of stirring shit up, that you're actually seeming to kind of move the bar ahead a little bit, with a bit of a tangible outcome. So I don't know if that's a question, but do you think about those sorts of things? I do, I mean like the CIA had the most amazing art program during the Cold War. Like, they were using art. They were using art to push their message. And a lot, if you look, I mean for me, Chris LeHann is like, you know, I look at Chris LeHann and I'm just like, and I look at the different kinds of like campaigns he's put together, and I'm like, wow, this is a very fantastic performance art piece. This is a very interactive participatory work with the local community. Like, he's doing these things that are completely informed by art. And then I look at other groups, I look at the groups that I love and the groups that I think are doing great work, and they have these great messages, but they're not thinking in these sort of radical ways to reach out to people. Art can be a crazy kind of disruptor. And so for me, it's not only about disrupting the space, but it's about just like, it is about pushing the bar. It's about pushing it a little bit forward so that more people can become involved, because when you get a critical mass and things can move. So what can we do in order to build on the work that everyone has already been doing, and get more people involved in the work? Hi. It was an inspiring talk. Thank you very much. I'm kind of tying it back to things that I've heard throughout the day. And I was at a session when Rob Afaro spoke, where Annalisa Manca spoke. Rob Afaro was talking about theory and getting ourselves to think about theory in relation to technology and the underpinning philosophy in relation to things. And Annalisa was talking about critical pedagogy. And as having a conversation over lunch about somebody, what struck me was it came up as, like how do we influence? How do we get that influencing position? But I think you've just provided me an answer anyway with that, that it possibly could be through art. And I'm thinking for any of those people that I've just mentioned, like Rob or Annalisa, do you think this could be a way of broadening the message of their research and of what they think about this? I don't have any... No, absolutely. I think this is the way in which I am a lot of other artists who are I guess, political social practice artists. This is how we are working. It's very research-based, so it's taking all this information and looking at these structures and seeing what we can do in order to reach other people. And so that's absolutely possible. Thank you for that incredible and really inspiring speech. And I don't have a question either, I wanted to share something, sorry. But it's because somebody asked the question about your practice and how that intersects with education. I just wanted to share a little example of something that happened here in the UK, because obviously a lot of your examples are from Europe and the States, naturally. So in my previous life I worked in the arts. Josie said I wasn't allowed to talk about this, but I'm talking about it now. I basically did a project called Represent and the culmination of it happened as part of Parliament's London 2012 programme exhibition, and it happened in Westminster Hall, but basically we got a group of spoken word artists together and they worked with a group of young, aspiring sort of social artists, activists. It's a group called TripWires. It's run by Index and Centorship. They worked together over two days to design a workshop for disenfranchised young people. So 16 and 17 year olds who don't have a voice and have a vote in Parliament to say what does representation mean to you? And then this amazing performance poet called Inua Ellams which is our source poem that we also asked the general public to contribute to. We had some billboards outside Parliament and he created this poem which was made into a film that was shown in Westminster Hall. So those frankly disenfranchised and pissed off voices were really, really heard. And it's the thing about location and having that playing in Westminster Hall where MPs and peers are walking through every day rather than having it playing in an art gallery or in an art space or in the schools just made it so much more effective. So I just wanted to highlight that some of this practice was, I mean that wasn't with higher education, it was with further education but some of this practice is happening in the UK as well. But thank you, it was so inspiring and slightly nerve-wracking now. Any last questions? Okay, we're going to draw to a close today then and let people have enough time to go back and refresh themselves before tying themselves out again a little bit later on. I think we need to give Diana another round of applause because she's so happy to have you here. Thank you. Thank you. There's Polita Oki pins on the front like thing. I want to say in German, season on the desk. And there's also pins with AWAC on there. Thank you so much. Thank you everybody and see you later on.