Welcome back. You're still on to the Breakfast on Plus TV Africa, and now to a second discussion. Some troops in Russia's military districts adjacent to Ukraine returned to their bases after completing drills. The Russia Defense Ministry was quoted by the Interfax News Agency as saying a move that could de-escalate a friction between Moscow and the West. This comes following talks between German Chancellor Olaf Scholz and Russian President Vladimir Putin in Moscow. Speaking on Tuesday during a news conference, Putin said he is, quote, ready to work further with the West on security issues to de-escalate tensions over Ukraine, but emphasized the need for the West to heed Russia's main demands. But later on Tuesday, U.S. President Joe Biden cast out on Moscow's claims of a partial withdrawal, saying that 150,000 Russian troops near Ukraine remain in what he called a threatening position, warning that an invasion, quote, remains distinctly possible, end of quote. In a televised address from the White House on Tuesday afternoon, Biden combined a repeated offer of security talks with a warning of severe repercussions if Russia carries out its threat of an attack that the U.S. intelligence has reportedly assessed could take place as early as Wednesday. Meanwhile, Ukrainian President Volodymyr Zelensky said earlier that Wednesday would be a day of unity for the country, urging people to fly flags and sing the national anthem at 10 a.m. Russia has always denied planning to invade Ukraine, saying it can exercise troops on its own territory, as it says sees fit. It has been pressing for a set of security guarantees from the West, including a guarantee that Ukraine will never join the North Atlantic Treaty Organization NATO. Now, what do these developments mean for the situation between Russia and Ukraine? Joining us to analyze this is Dr. Katch Ononijou. He is an international affairs analyst at Dr. Ononijou. Thanks for joining us. Nice to have you. Thank you very much. How were you? The United States intelligence has put it out, and President Joe Biden has televised a speech addressing the world, and they believe that Russia will either invade Ukraine today or will throw missiles into Ukraine today. You heard from the intro there that Ukraine, rather Russia, has pulled out some troops from that, what it calls a mere training exercise after Putin met with the chairman and chancellor. Do you believe that the U.S. intelligence report is true and should be held on to? Well, there are quite a lot of things we know. For a very long time right now, there are crises in Ukraine, an internal civil war in Donetsk and Libya. So you need to look at that, and those are separatist people fighting to link themselves to Russia. It's been going on for very, very long time. The question is, what did Europe do all along while this war raged on? Okay? And what is the intention of the West on a long train? Don't forget the Cuban Missile Crisis. America couldn't accept that Russia should have an ally on its border, and on the issue of Russian demand for guarantees and regards to Ukraine, of course, yes, we know internationally you cannot tell a country not to belong to a committee of friends it feels very comfortable with. But I am aware also that President Biden has stated openly to the Russians that the NATO is not their enemy, and they do not have an intention of putting arsenal and elements on the Russian border. They don't have any intention of bringing Ukraine into NATO very quickly. So what is the problem? This issue is not going to go away. There are guarantees that Russia seeks. Those guarantees look very, very difficult. Yes, they are not very possible for us to respect. If Russia does, as everybody fears, in favor of Ukraine, what will happen? Well, I don't know. It could change human history because of the war, but I also believe nothing also may happen. One, they've taken Crimea. What has happened? Nothing. And they've masked their troops on the borders between Ukraine and Belarus, and also on the Russian border with Ukraine. And that's gotten everybody very, very angry and jittery. So I believe we must not break the long-term engagement with Russia. And I believe that engagement has not been given the kind of relationship and strength and commitment it deserves. Why do I say this? The ongoing crisis in Russia did not start today. The crisis inside Ukraine did not start today. The civil war in Ljubljana and Donetsk did not start today. What did Europe do? Why did Europe not all along find a way to try to ameliorate and bring peace, to do what they are doing right now, yesterday, in order to bring peace in Donetsk and in Ljubljana? So it's only now that we are all beginning to scream, as if this crisis just started yesterday. It didn't start yesterday. It's been on for a very long time. What did the world do all along? The world didn't do anything, and now we are screaming about what Russia wants to do. I think the Ukraine issue, for as far as I'm concerned, is a convent. How will it be resolved? I do not know. But I also believe, I don't see a major crisis coming out of Ukraine. I believe the West will advise Ukraine to go on gently. I believe the West will find a way, because now you have Russian separatists, Russian-speaking people in Ukraine fighting an internal civil war for a very long time. What have we done? We need to look at that. Solve the crisis in Donetsk and Ljubljana if you want to solve the Ukraine crisis. So I believe. Well, Dr. Katya Nunoju, the United States has been accused of trying and backing the Ukraine in this crisis that's ongoing. And some people are saying that the United States has a personal interest to achieve. You can actually see track records in the Middle East. You see that in Yemen, Africa, according to the School of Thoughts. Let's say that the U.S. is strongly behind, and because they have an interest in Russia. Well, not just having an interest in Russia, the U.S. is not very comfortable, which Russia has moved, even with Yemen. Look at the way everybody wants to link the Nordstrom II to the crisis. The Nordstrom II goes through, and Russian gas gets deep into Europe. You can't know to recall that. That means Russia will become permanently embedded into the energy needs of Western Europe. I don't see you removing that. So on the long term, everybody should be careful how they play with Russia in the regards to Ukraine. Now, there is nothing too much if the Russians beg you, don't bring NATO to our borders. And I believe if Russians believe that the West, the United States, seem to want to use the hyphen of the, there will be an invasion, there will be an invasion issue as a strategy to bring its weapons and troops into Ukraine. They could precipitate a Russian action, could not be good for the international order. I believe that is a game, and that's why I have seen and I feel encouraged that President Biden is making public commitments through his speeches that the United States and NATO do not have any intention of bringing troops there or bringing armaments there on the Russian border. That means in Ukraine, I do not know what they are going to do because right now, as we say of the Russian beer, it actually is powerfully bringing down the neck of Ukraine. You can see what's going on. So on both sides, yesterday, Russia said they have removed some of their troops that went there for the military trail. And I believe that to be a payback to the efforts of President Macron, and I thank President Macron for that. But on the long term, this crisis is just started. So what we're beginning to argue about is we seem to be having a network issue with Mr. Khachar Raju. Dr. Raju, can you hear us, please? The Federation. And a lot of those elements are there. Yes, I can hear you. Okay. For those who are wondering, trying to understand the situation, can you tell our viewers why Russia has a problem with Ukraine joining the North Atlantic Treaty Organization? Why it feels threatened by that possibility? Well, one, the way the Soviet Union was arranged, each nation had capabilities increased and strengthened in places where they think they have an advantage. In the whole Soviet USSR, apart from Mother Russia, Ukraine was the country where a lot of the military equipment were being done, as you've seen, like it happened in Crimea. That was also part of Ukraine. Russia has taken that back. So there were quite a lot of things that Russia believes that it needs Ukraine for, and doesn't want Ukraine in the hands of the, what do you call it? The West. The natural alliance. Okay. And that's why, I believe, what may allow Ukraine to remain as an independent country. Don't drag Ukraine into nature. If you try that, you will have a Russian reaction. That is what I believe all of us should work towards. Don't try to get Ukraine into nature. Don't also try to get Russia retentive by seeing equipment, elements and truth from other countries put into Ukraine. Whatever it is you want to do to help Ukraine, let it be true diplomatic relationship and negotiations with Russia. If not, Russia may be forced, at least they have shown that they have no problem in getting Ukraine from Belarus or in getting Ukraine from the Russian border. But, Mr. Captain. We don't play jokes. Yeah, Dr. Naju, this is interesting, you know, the fact that you've said, you know, Ukraine is sandwiched in between Belarus and Russia and the dynamics of the history of the USSR with the Ukrainian, you know, region involved in the USSR and all that. So let's look at the mass of troops on the border between Russia and Ukraine. Some school of thought will say this is just a strategic deterrence, you know, going back to what happened between Cuba and the United States of John F. Kennedy, where you had those missiles pointed facing the United States of America from Fidel Castro's camp ready to fire. But that was all in the bid to ensure that some steps were taken or actions were prevented. And so in terms of this idea of strategic deterrence, do you think that, yes, that school of thought is true and Russia really does not intend to throw missiles into Ukraine or invade Ukraine? Russia will forever be interested in Ukraine. So we have to find a way of having Ukraine or being a member of the international community and yet not angering Russia or making Russia feel very uncomfortable about the way Ukraine has to be a special case. Ukraine cannot be a member of the country. Ukraine cannot be like other countries who want to join Russia. But you think they are doing this to just make the change the minds of the NATO allies and to encourage them to step back a bit and say, hey, we are here, but they don't really mean to start going to Ukraine. Well, they will ultimately step back. Russia has shown them reasons why they should step back. Nobody can try that. Don't forget, you're talking about the area that has the suppressible port, the area with Crimea, the area with nuclear submarines and nuclear warheads. You don't want to go there and go to a fight. You don't want to do that for the second world, because if anything goes wrong in Ukraine, it will change the world history. OK, just as we close the conversation down. Now, with all of this, we know that the world is a global village. COVID-19 has actually showed that in different pandemics that we have experienced. Will this actually impact? I mean, with the ongoing crisis right now, the Russia-Ukraine crisis, is there any impact on the globe and on Africa? And Paraventure, you know, Russia decides to invade Ukraine or the other way. What will be the impacts on the entire globe and Africa and Nigeria? I mean, we've been making light jokes behind the scene, talking about missile getting to some parts of the world. Well, if there is a global configuration, of course, everybody gets affected, everybody gets involved, but we don't want that. I believe the immediate impact of the crisis will be the realignment of forces and alliances. Right now, we're already hearing about Russia and China being very sweet with themselves. I don't think the United States like that. So you don't want to play up those things that will suddenly make those who are with Russia to stand up and take a chance against you. So we need to go very gentle, manage this, de-escalate it. De-escalate it. We do not need any crisis in the Ukraine. And don't forget, if you look at the map, which you are showing, you will see that Ukraine is like a very small province right up there beside Russia. In the West, I think they will go there and then have the base. The U.S. must consider how it felt when the Russians came to Cuba and tried to put a base there. The U.S. didn't allow it. So whatever it is that the U.S. didn't want Russia to do in Cuba, the U.S. should, in turn, consider it should not be between the same thing in the Ukraine. If it knows it, you're going to have crisis. Interesting, very interesting analysis from Yucatian, where we'll get to see what happens. Thank you very much. I think Messi's last question was born out of fear. Maybe she feels some some missiles will land in Nigeria. But it reminds us of, you know, the the Operation Desert Storm. I think, no, not Desert Storm, but the first time war between America and the first Gulf War, you know, when George Bush Senior was in charge. And then I remember as a youngster in Nigeria, we're thinking that Saddam Hussein was going to throw some weapons of mass destruction in Nigeria.  So I know where Messi is coming from. But thank you very much for your time, Mr. Dr. Khachan Rajeev. Thank you for having me. Thank you very much. We appreciate your time. All right, Messi, you can you can rest easy. We are we're safe. Well, you never can tell what happens. This weapons are very sophisticated. But we pray that, you know, crisis don't happen. And apart from that, we're hoping that all of the ready events are about to fuel the situation. Let's do it. No, no, because it would affect everybody. However you say it. Well, that's the size of the conversation. If you missed about on any part of it, it's all right to follow us on Facebook, Twitter and on Instagram, but plus TV Africa and lifestyle on YouTube to not forget to subscribe. I am Messi Boko. Have a fantastic Wednesday. And I'm Kofi Bartels. We return tomorrow. Good morning.


the King of the Chronicles? What are you doing with your life? And I'm talking to our people to do my work. Manderlay is iconic. And I hated that book so much. Vibes. Hey guys, it's Leigh Ann and I'm here today with no nuance, just vibes. So I know this title and this thumbnail are a bit cheeky. I just want to be clear up front that I don't mean to suggest that any of the books that I'm going to talk about today have no nuance, have no complexity, are not intellectual in any way. But basically, I'm going to be talking about books that I think really just like deliver on vibes, which isn't to say that they don't deliver on other things. I mean, setting is important, but I don't to me vibes is different from a setting because setting is, I don't know if there's a way to explain, so setting is like the bricks and mortar and vibes is just like the the mood, the tone, the the lighting, if that makes sense. So yeah, I think I think that's clear. I don't think it's a very high concept video idea. So yeah, I'm just gonna have a list of books that I think have incredible vibes. First up is Rebecca by Daphne de Morier, which is like one of the number one reasons I wanted that mega disclaimer of like that does I'm not saying they don't have any nuance, even though that's literally the video title no nuance. But so Rebecca, if you've never read it or never heard of it, Rebecca by Daphne de Morier is it's a classic, more of a modern classic, she wrote it in the 1920s I want to say. And if it is a character driven story, and so with a character driven story, you might think it's not going to have incredible vibes, but it absolutely does. This follows the story of your unnamed narrator, who is the young new bride of the master of this house called Manderlite, or the owner of this house called Manderlite. And it's it's a creeping slow moving kind of psychological, I guess thriller, I mean thriller I think oversells it a bit like if you're expecting a thriller that like what we're used to like Gone Girl, it's not like that. But it is this like psychological thriller is the best one I can think of. It's basically like you're following this young woman as she's unraveling the mystery of Manderlite and about her husband, and about the his first wife, and what happened to her in Manderlite. And Manderlite, the house, is basically a character in the story, like beyond just having a prominent setting, beyond just having vibes, and the location be something that is noteworthy, it is like literally a character in the story. I feel like no list of books that's talking about like setting being important, which is kind of what vibes is, would be complete without Rebecca. Manderlite is iconic. So I mean, I highly recommend this book. I think it's one of those classics that like it doesn't read like a classic.  It is slow and it is wordy, but I think it's more highly digestible for a modern audience than a lot of other classics. Like it's enjoyable to read. Not that classics aren't enjoyable to read, but a lot of classics can be like a bit more effort to read. I don't think that Rebecca takes a whole lot of effort to read and enjoy. So I recommend it in general, but also Manderlite, so many vibes. Next up I have The Bear and the Nightingale or The Entire Winter Night Trilogy by Catherine Arden. I picked the least Vibe cover to hold up mainly because it was nearer to hand and also because I never ever show this cover off and I wanted the excuse to, but this is bothering me. The more vibes you cover because I mean, yeah, this one doesn't doesn't doesn't demonstrate vibes so much. There's a child screaming outside and I need that to stop. So yeah, Bear and the Nightingale, I'll put this one down. Bear and the Nightingale delivers these vibes, this town. If you've never heard of it, never heard me talk about it, never read it. The Winter Night Trilogy is sort of Russian folklore just in general being retold. So it isn't any one particular Russian fairy tale that's being retold, but it is a sort of like it's kind of like historical fiction because it is taking place in Russia, not a fantasy world inspired by Russia, but it's incorporating folklore as like active elements. Characters from folklore and like concepts from folklore, from Russian folklore and the main character. It's kind of like the battle of like pagan folklore versus Christianity and like the main character being very much in tune with sort of like the little gods of nature and that and budding heads with the Christianization of Russia. She's fine with people being Christian, but the Christians are not fine with her leaving bread out for like the little like house elf essentially. Dumb a boy, but all these like little Russian spirits anyway. So it's an excellent trilogy about like the vibes. It is so much like Russian nature, Russian winter, Russian food even. It is it just absolutely you just like sink into this fantastical magical Russia. And then in the later books like it isn't just the first book takes place more in a rural area and then later you're in Moscow and that's once again the Moscow vibes are off the charts. This series definitely feels like vivid. Like it feels like you are being transported to a magical Russia. A dark Russia, an unsettling Russia, a sometimes scary Russia, but a magical lush Russia nonetheless. Next up I have a book that I I mean I must have talked about it in the wrap up when I read it but I don't think I've really ever talked about it that much and that is House of Sulton Sorrow by Erin A Craig. This is one of the few more recent YA releases that I really really enjoyed and I remember specifically like when I picked this up because I had just I had gotten this book and another book around like Halloween time and this had just come out. That's why I got it not because I thought it'd be a good Halloween time read. I think it came in a book box but I got like this other YA book that I thought was more likely to be my jam. It had like a skull on it and it was about these like Welsh skull monsters and I hated that book so much and that one was I was expecting to get creepy vibes from and whatever and I was like that was terrible so I picked this up and I remember thinking like this is instantly way more like it's not at all what that other book said was going to be so I wasn't expecting the other book to have a seaside theme but like that creepiness that I was expecting from the other book this immediately delivers. So House of Salt and Sorrow. This book is a retelling of a fairy tale that is not very frequently retold that being 12 Dancing Princesses and this takes place um this is more I feel like people don't or at least I don't often associate like creepy ghost stories and stuff. I usually just think of oh like decrepit castle or old Victorian manner or the Moors but I mean quite frankly like seaside things can be quite creepy like an old abandoned lighthouse um and that kind of thing. So this is definitely like seaside creepy and the vibes are off the charts like I feel like when I was reading this like I just the sense of like the dampness, the salt, the I can like hear ocean waves crashing, the sense of dread building because of this sort of macabre ghostly mystery that this book is all about. It is just extremely atmospheric and I had a great time. It's not like the best book I've ever read but I had an absolute like wonderful time just reveling in the creepy atmosphere that this book was just absolutely delivering. So if you want seaside creepiness this book will deliver. Next up I have the book series that actually inspired this whole video idea. So that is The Raven Cycle by Maggie Steve Otter. The Raven Cycle is it's just maximum vibes. I mean I know this video is about books that have vibes so I know I'm going to say vibes about every single book and that must be getting slightly annoying. I can't help but that's what this video is about. But The Raven Cycle like when I think of vibes I just think of The Raven Cycle slash when I'm reading The Raven Cycle all I think to myself is just vibes and I can't put it any other way because yeah like it's a cool setting. This takes place in Virginia but in like very rural Virginia and there's a lot of like there's like a magical forest and there's a lot of, I mean mainly it's the magical forest. I mean everyone else is like living in a house or dorms or whatever but that's the thing about Maggie Steve Otter's writing that like so then the female main character, Blue Sergeant, she lives with her mother and her two aunts in this house because they're all psychic so it's this like messy cluster of chaos because she lives with psychics. So the vibes of Fox Way, a 300 Fox Way where Blue Sergeant lives, it's kind of a little bit like the borough like where the Weasleys live except I mean it's totally different than the borough but in terms of like this place that is kind of otherworldly a little bit because you have the psychics and it's very chaotic and messy and it is this kind of like you have to love the mess because we're all living here and it's kind of chaos and we're not super rich so everybody has to share and it's a little crazy so there's a lot of vibes in 300 Fox Way. Cabe's Water, the magical forest just off the charts magical vibes. I feel like I can smell the greenery I can hear the water rushing like it's just vibes and then the characters themselves, Maggie Steve Otter's writing, it's less than what I would call characterization and more vibes. Each character to me is just they each have their own specific vibe and that is the character and I don't know I think that might be why some people don't like Maggie Steve Otter's writing I love it but that's why I feel like the vibes are off the charts because it isn't just like the setting has mega vibes the characters have vibes their clothes have vibes every place they go into has vibes like the cars they drive have vibes the magic has vibes just it is just the vibeest of vibe series so if you're not down with that she's not the writer for you but that is what I love about it and it is it is just so atmospheric at all times. Next up I have Cersei by Madeline Miller oh my god all the books I'm holding today are so shiny um I have a less shiny edition but I thought I'd show off the pretty shiny one. Cersei if you don't know is a retelling of the story of Cersei from Greek mythology and Madeline Miller's writing I mean I might equally have held up Song of Achilles but I do feel like Song of Achilles also by Madeline Miller also a Greek myth retelling. Well that does deliver a lot of atmosphere and vibes I think Song of Achilles is more action packed if that makes sense whereas Cersei is so much about sort of like the island that she lives and what that's like and her own kind of vibe and it's just more kind of slow and contemplative than Song of Achilles and for that reason it feels more vibey to me than Song of Achilles because Song of Achilles again it absolutely has great atmosphere I don't want to say that it doesn't but this book is spending more time just kind of being in a place without plot moving forward if that makes sense and I feel like that makes the book sound really boring and I'm certain there are people that do find the book boring but I don't think it's boring I think it's just more contemplative and for that reason it gives you more of an opportunity to kind of just savor the place that it's in and kind of sink into all of that. I feel like I'm really not articulate today I'm so sorry. All I can say is vibes. Cersei is an excellent book just in general I highly recommend it but also vibes. Next up I have Strange the Dreamer by Lainey Taylor another extremely shiny book. Lainey Taylor's writing also so similarly to Madeleine Miller's two books like I could have put the Daughter of Smoke and Bone series on this list it is also jam packed full of vibes but I feel once again like between Song of Achilles and Cersei I feel like Strange the Dreamer, the duology, is slower and it is more contemplative and it does just spend more time reveling in the place that it's taking place whereas Daughter of Smoke and Bone it absolutely has incredible settings but it is more like plot driven action packed like let's keep it moving so even though it's like wow you're not really stopping to take in the wow whereas Strange the Dreamer actually similarly to Rebecca where I said like Manderlay is like a character in the story Strange the Dreamer likewise place almost becomes a character in the story because the very beginning of the book I think all of the the like promotions for the book also like brought this in into it about the place called Weep I mean like the bottom of the book jacket says welcome to Weep it's like most book jackets I don't feel like make a point of welcoming you to the place that they are taking place without spoilers and I feel like this is one of the reasons why people find it difficult to explain what Strange the Dreamer is about because you don't know very much going into it and you can't know very much going into it because knowing almost anything about it spoils it but the place called Weep is a place that kind of like disappeared it's this lost city and the main character Laszlo Strange is obsessed with finding this lost city finding any scrap of information anything he can find that is just even proof that Weep existed so again I can't really say anything more than that but like Weep the city of Weep is like a character in the story for that reason and so there's a lot of focus put on everything to do with that place and everything to do with people that would have come from it or how the fact of it being there and then being gone and how that makes one feel and I mean outside of Weep I mean the library that Laszlo Strange works in where he's digging up information about Weep also is very atmospheric as a library not as like whimsically manually atmospheric it's a library but like Laney Taylor's writing style oh it's very different from Maggie Steve Otter's writing style I don't want to say that they're the same but similarly to Maggie Steve Otter this like this approach to giving everything a lot of vibes so there is never any mundane part of anything that Laney Taylor writes every single part of it is like technicolor so Laszlo in the library researching is is lush and imaginative and beautiful and whimsical even the most sort of like boring straightforward part but yeah Strange the Dreamer is just a feast for I want to say a feast for the eyes which doesn't make a whole lot of sense because it's a book so you can't it's not in front of you you have to imagine it but like it is like a feast for your mind's eye is the best way I could explain it so I highly recommend again just in general but also many vibes next up I have obligatory The Name of the Wind by Patrick Rothfuss if you don't know anything about me or my channel then you may not know that I'm a big big big fan of The King Killer Chronicle by Patrick Rothfuss this is a little different from the other books on my list because I do feel like it is vibes aren't like the most important thing about it but I do when I think about Name of the Wind when I think about reading The King Killer Chronicle a lot of what stands out to me a lot of the enjoyment that I derive from it is the setting is this cozy vibe lots of taverns and cider and ale and musicians playing in a tavern where you can just sort of like smell the wood paneling and it very much places you in every setting that Kvothe is in like be it the tavern be it the school be it wherever he is you have a very distinct sense of how this place looks how it smells how it feels you feel like you're walking through this world with foe so it isn't like the magical whimsy and wonder of Strange the Dreamer for example but it is nevertheless very much like sinking you into the tone and feel of this world albeit a more grounded world than Strange the Dreamer but yeah I just I couldn't talk about vibes without the Name of the Wind which again just in general highly recommend if you haven't read The King Killer Chronicle what are you doing with your life yeah read The King Killer Chronicle but also it makes a great autumn read because of the autumn vibes and last on my list is a book that you may not have expected from me and that is and then there were none by Agatha Christie this is a who done it this is probably if not her best known it's I think one of them that her most well reviewed or more universally enjoyed but I enjoyed it a lot more than Murder on the Orange Express which was a big let down for me and then there were none is like the quintessential closed circle mystery it is isolated closed circle mystery like when I feel like every single book that has an isolated isolated closed circle mystery you know how like every epic fantasy is like the next game of thrones every isolated closed circle mystery is the next and then there were none so if you haven't read it or heard of it it is it's not like one of her series with like the famous detective like Poirot or Marple or anything like that this is just a standalone and there's this island where all these people are invited to come to this island and none of them know each other none of them know why they've been invited and they all start dying one by one and everyone on the island is trying to figure out who is doing this and why and it is just this this tiny little remote island where you can't make contact with land because there's also pre cell phone days and it is it's it's so creepy and well done and that sense of panic and isolation and the loneliness of being on this island like even if there wasn't a murderer it sounds like a pretty isolated lonely depressing place to be if i'm quite honest but then there's some there's clearly murders happening and you're next and it is just so well done and the entire time I feel like you you feel like you are on this remote deserted island um and that like ticking clock of like who's next who's next who's next is it's just so well executed this is so so good classic for a reason highly recommend again probably a good halloween time read because it's quite creepy also many vibes so yeah those are all the books on my list of books that deliver on vibes let me know in the comments down below if there's any books that you would have added to this list if there's any books that you would take off of this list there's any books that you would have expected like knowing me for me to have put on this list uh whatever you want to let me know i post videos on saturdays other random times as well but i think saturdays so like and subscribe join my patreon if you feel so inclined and i'll see you when i see you bye


Hey guys. So this is a 2 o'clock weaponizing data science and social engineering. These are the guys and we're going to kick it off. All right, so DEF CON goons are no longer allowed to drink at red shirts nor are they allowed to do shot the noob. I'm going to keep this short. It is Phillips first time speaking at DEF CON. John spoke last year but wasn't able to get a shot. So let's do a shot with him and have a good time.  Hey guys, my name is John Seymour. So welcome to our talk on weaponizing data science for social engineering. Wow. Dude that was strong. Weaponizing data science for social engineering, automated end to end spear fishing on Twitter. So we think this talk is actually a pretty good fit for this conference, right? Every year Black Hat does this attendee survey and every year social media, you know, fishing, spear fishing, social engineering is near the top of their list of concerns. We wanted to try our hand and see how effective using AI to actually automate spear fishing would be. And so things like social engineering toolkit actually automates the back end of, you know, social engineering, right? So creating a malicious payload, things like that. We're actually interested in more of the front end sort of stuff. So actually generating links that users will click. Traditionally, there are two different types of approaches to this. There's fishing which is very low effort, you know, shotgunning tons and tons of messages, but it also has very, very low success between like 5 and 14%. There's also spear fishing which is highly manual. It takes like tens of minutes to actually research a target and create a message that's, you know, handcrafted to that actual person. But it also has very high success. The social media pen testing tool that we released today actually combines the automation of fishing campaigns with the effectiveness of spear fishing campaigns. And with that said, I'm John Seymour. My hacker handle is Delta Zero. I'm a data scientist at Zero Fox by day and by night. I'm a PhD student at the University of Maryland Baltimore County. And in my free time, I like to research malware data sets. All right. And my name is Phillip Tully. I'm a senior data scientist at Zero Fox. And in a past life, I was a PhD student at the University of Edinburgh and the Royal Institute of Technology in Stockholm. So in that past life, I studied recurrent neural networks, artificial intelligence, but in a much more kind of biologically oriented way, I was trying to figure out how you could combine neurons together and connect them up with synapses and simulate networks of neurons to try to get some storage and recall of memories. But nowadays, instead of combining different patterns of spikes to create some biological representation of a memory, combining text to try to, using AI, so using similar techniques to try to generate text. This is not necessarily anything new. The field is known as natural language processing. It's been around for a really long time. One of the kind of fundamental examples happened over 50 years ago with the Eliza Chatbot. So this was designed by a psychotherapist named Joseph Weisenbaum at MIT. And he used it in a very clinical setting. So he wanted to try to half his patients who were either on their death bed or close to death, be able to interact in some way with the computer. So it was very kind of naive, very ad hoc. It was based on parsing and keyword replacement. It would simply do something like if the input to the program was my head hurts, it would output something in response like, why do you say your head hurts or how bad does your head hurt? So something like this. And these kind of very early examples were inspiring for people because they passed some very simple versions of the Turing test, right? So using these kind of questions in this very ad hoc feedback, it was able to not really or fool people into believing that they might be talking to a human rather than a machine. Fast forward 50 years and we have Microsoft AI which came out with a neural network that was based, or it was called Tay and you. And so if you've seen this in the news recently, it was kind of a dynamically learning bot that was released on Twitter. And it was a really cool idea. So each time a user, a Twitter user tweeted at it, it would kind of learn from that tweet and then reply to it. It was a chat bot. And you see this a lot popping up now on Facebook and other kind of social media services for more of like a marketing twist. But what they didn't foresee was the fact that Twitter tends to be cesspool sometimes and tends to be filled with porn and sexually explicit content and overall kind of bad stuff.  So indeed we view infosec and machine learning as kind of prioritizing the defensive orientation, right? So you set up perimeter or you try to detect incoming threats or you try to remediate it once it's already happened. The adversary has to do something in order for you to react to it and defend your network or whatever it may be. So you have some examples here. These are historical black hat talks over the last 10 or 15 years. You have some machine learning talks one or two per year usually and they cover anything from spam filtering to botnet identification to network defense to intrusion detection. But what we wanted to propose here was rather that you could use artificial intelligence techniques and machine learning not only on defense but you can use data to drive an offensive capability. We call our tool snapper. It's the social network automated phishing and reconnaissance tool and it's split up into two separate phases. The first phase takes us input a set of users who you want to target. And it takes a set of users and extracts a subset of them that it deems as high value targets so it prioritizes them. We'll get into more about this later. And then the second phase of the tool takes those users and crafts a tweet directed at them based on the content that they have on their historical Twitter timeline. And the end result of this is a tweet with an app mention and the crafted machine generated text and then a shortened link which we measure success using click through rates. So with that if anyone wants to partake in the demo we're going to do later on in the talk please tweet at the hashtag snapper and that's hashtag SNAP underscore R. We're not going to target you with any kind of malicious payload. It'll be a shortened link that just redirects to Google.com or something like that. But if you want to have your timeline read dynamically and then have a tweet spit back out at you please do that in the next 20 or 25 minutes. So the talk will go I'll end it off to John to talk about machine learning on offense and then we'll go into the two parts of the tool target discovery and spear phishing and talk more in detail about how to generate the message content that's kind of the core of the tool. And then we'll talk about how we evaluate the tool and how that evaluation compares to other techniques that have been found in literature. All right cool. So the first question is like why is social media such a great place for spear phishing people right? Why Twitter in particular? There's a lot of answers to this and we put a few on the slide. First being a lot of these social networks have very bot friendly APIs right? Whenever you post something on Twitter then people can go and scrape your timeline, your activity records, things like that very easily because there are Python APIs for all the social networks just straight up available. Another thing is there's a very colloquial syntax on Twitter and social networks. For example when Nikita actually posted this tweet I really quick snapped her and said hey can we use this for our talk. 20 years ago you wouldn't have any idea what this meant. So the idea here is like basically machine learning tools especially generative models tend to be pretty bad if you've ever seen subreddit simulator and things like that. But the fact is the bar on Twitter is so low to have a good tweet that people will be interested in. Even generative models can do pretty freaking well. Some other things are like due to character limits. There are a lot of shortened links on Twitter I don't know if you've ever used it. So basically if you're trying to obfuscate a payload or something like that people don't actually think twice about clicking links on Twitter that are shortened because everything is actually shortened there. Then there's also the fact that people sort of seem to understand email or at least some people do at this point. Like Nigerian print scams, things like that. A lot of people actually can tell you hey you get an email check the link before you click. On Twitter and social media, social networks, people don't actually think about what they click on. You don't have that sort of years of awareness built up yet and that's one of the things we're trying to actually bring about with this talk. And then finally people actually want to share content on these social media networks. For example Reddit, you want to get up votes. Twitter you want people to share and like your content. So there's sort of this idea of like incentivizing data disclosure.  So we wanted to give a quick shout out. Actually at Shmukan there was a really really cool talk about you know fishing the fishers using Markov chains. And that was actually a huge inspiration for this talk so we just wanted to give a quick shout out. But getting right into the tool itself basically there are some things built into the tool directly and there are some things that we also add on top of the tool right. So things that the tool does directly are it pre-pens tweets with an app mention and on Twitter this actually changes what the tweets are categorized in their process right. Tweets that start with an app mention are called replies and only people who follow both the person tweeting and the target can actually see those tweets. So if our bot doesn't have any followers that means the only person who can see the tweet is the target itself which actually is very useful in determining whether or not an individual you know target has clicked. Another thing that's actually built into the tool is it shortens the payload uniquely per user and we'll get into that in a bit. So that way we can actually go through and each of our shortened links that we generate we can check whether or not that particular link was clicked and map that back to the user who clicked it. Also we triage users with respect to value and engagement so we have a machine learning model that we'll talk about in a bit that actually goes first before it actually fishes the person checks to see whether or not they're a valuable target whether they interact a lot with the platform for example. One reason this is useful is for example a lot of people have what's known as egg profiles or profiles where they haven't changed the default settings. These people tend not to post a lot they don't they're not very engaged and we don't want to waste API requests or you know waste like possible awareness of the bot right by trying to fish these people. So we just go ahead and actually triage these users out so that we don't have to worry about them. And then finally the tool itself obeys rate limits. This is because we sort of wanted to release it as an internal pentesting tool. Obviously you know people can get around that but we hope you guys don't. That's all I'll say about that. Some things that aren't actually built into the tool that are very very useful. First off Twitter is actually pretty good if you post every single post of yours has a link in it. They're good at finding that and shutting you down. So one of the things we recommend is post a couple you know non fishing posts in there or get ready to make a lot of accounts. And then another thing is if you yourself the bot have an egg profile you know nobody's going to actually click on your links because obviously they like to see believable profiles before they click links. So a very high level of design flow of the tool. First we have a list of Twitter users that we pass into the tool. It goes through each user and asks whether they're valid you know whether they're a high value high engagement user or not. And if they are it scrapes their timeline to a specified depth. So for example 200 or 400 tweets that they've sent and uses that to either seed a Markov model or a neural network model. And that generates the actual text of the post. After it's generated the text then it you can either have it schedule the tweet for a later time when they're most engaged and it actually calculates all that for you or you can post the tweet immediately and have the the tool suite to obey rate limits. And that's actually useful if you're doing an on-stage demo. But yeah cool so let's get into the tool. I'll talk about the first face here automated target discovery. So this is what Twitter looks like if anyone's been living under a rock for the last 10 years. Twitter is full of interesting information and personal information like John said. You have this incentivization structure for disclosing personal data. And by that I mean it's not necessarily just the content of the posts. So the last tweets that were made you also have super valuable information present in the description. People on Twitter tend to like to post about what their job title is and what their interests are generally. You get different kind of data not just text you have integers like how many followers and how many followers you have how many people are following you. How many lists you belong to. You have a lot of kind of Boolean fields like have you changed your background profile image have you changed any of your other default settings from the original instant instantiation of your registration. It's filled with different dates like you're created at date and URLs within the text that you post. So this is what the the raw API call looks like from Twitter when you when you grab when you grab it. So I'll use the example for this section of Eric Schmidt the former CEO of Google. So we we implement a clustering algorithm so it's based on machine learning and we go out and we grab a bunch of Twitter users and we extract features from these from these API calls across these different users and here I list a few of the most most interesting and most relevant features that we grab. So like I said in the description if you have words that tend to correspond to a job title like CEO CSO CISO even like recruiter or you know engineer or something like this. This is probably going to end up being someone who you might want to target right. They might have access to some sensitive information company information or whatever if you belong to some other organization. Also your level of engagement so how many people are following and following you and how many people you're following. You can imagine you don't want to you don't want to target somebody who's not very active on the platform. You want to make sure that someone who is actively engaged and is likely to click on links and is getting updates on their phone. The account age is a good piece of information to the created at date of the Twitter profile. You don't want to really target somebody who's just made the account and is just trying to get started up with the platform. Same thing for hashtag my first tweet. And then also a good indicator is the default settings. So people who tend to engage a lot in the platform will kind of make it fancy. They'll change all the default settings and they'll make it more matching to what their interests are and what they like. So in a nutshell this is how it works. If we take the clustering algorithm and we start out with our target Eric Schmidt, you can imagine now that each Twitter user is represented on this 2D plot as a single point. Again it's projecting it into two dimensions. Originally it was a very very high feature high dimensional feature space with all those different settings like the description, number of followers, etc. Projected into 2D and Eric Schmidt falls on this 2D plot somewhere there. Great what do we do with that? We pass it through the clustering algorithm that we have and I'll talk in the next slide about how we choose that. But once you do something like that then you actually get to extract a subset of these users that you might deem as a relevant target or a high value target. So up in the left hand corner the plot of red points there might be the group of people that you deem as high value targets and the users who belong in the blue and the green points you want to throw them aside, deprioritize them. So in the machine learning world there are many different clustering algorithms you could choose from and each of those algorithms have a certain set of hyperparameters that you could tune to kind of optimize your technique and optimize your clusters. How do we choose this?  So we have k-means and a parameter for k-means clustering algorithm is the number of clusters that you choose from a priori for example and you take those and you fit the models for each of these different set of algorithms and their set of hyper parameters and you choose the one that maximizes the silhouette score. So the silhouette score is bounded between negative one and one and anywhere a positive number the more positive the better and anywhere from kind of point five to point seven up is is considered some kind of reasonable structure. The silhouette score kind of measures how similar a data point is to its own cluster so the cohesion within that cluster to how it compares with data points outside that cluster the separation of those data points. So on this plot each individual data points of each individual Twitter user is represented kind of as a horizontal bar and the hyper parameters are on the y-axis so if you look at the first the top plot there you have two different sets of hyper parameters for k-means one might have two clusters one might have three clusters. So you calculate the silhouette score for each individual data point and you calculate the average of that which is shown here by that red dotted line and basically you want to choose the algorithm that pushes that red dotted line all as far right as you possibly can get it to. All right cool so before we actually get into the cool machine learning models and stuff for generating text we're going to tease you guys a bit with some of the boilerplate that goes around the tweets. So one of the first things that we actually ran into was we wanted to choose a URL shortener right and we want a URL shortener with a lot of different qualities one of them being you know actually can shorten malicious links and so the first thing is we went out we found a malicious link we verified using virus total that it is indeed malicious and we actually went to it too in a sandbox and all of that and we tried it through a lot of different link shorteners and apparently goo.gl lets us shorten it right and so actually several others also let us shorten it but goo.gl gives us a lot of cool other things first off it gives us sort of like a timeline of when people click and apparently this link has already been shortened before and people have clicked it but that's you know a tale for another time. goo.gl also gives us a lot of cool analytics like who referred the link for example t.co what browser did the target use what country were they based in or at least you know that their like actual machine say they were and what platform they use so windows chrome you know those sorts of things android and all of that so yeah so goo.gl actually looks pretty legitimate I ran it by a few guys I know and they were like hey yeah like it comes from Google it's got to be safe right and no it can link to malicious sites so we verified that it also gives us really cool analytics which is very useful if you're you know trying to spearfish internally right you want to know which users clicked but some other cool things that it gives us is you're able to actually create shortened links on the fly using their apis so you can actually say hey here's this you know general payload www.google.com let's shorten it uniquely for each individual user and see you know which individual users actually click on the link and then you can also obtain all of these analytics programmatically so there's really like no manual you know a process that you need at all in this entire process and we'll we'll go ahead and give the note that we never actually posted any malicious links to any targets we just verified that you can actually shorten malicious links in here so please don't get mad at us about that and then finally another thing that the tool does in the box is it does some basic recon and profiling so two things that it does is it figures out what time the user is likely to engage the platform and it looks at what topics that they're interested in and tries to create a tweet based on one of those topics so for actually figuring out the scheduling the post of what time the user is active we just use a simple histogram for tweet times what which hours that that user tweets and over on the left you'll actually see my own tweet history timings so you can actually see that I'm most active at 11 p.m. at night take that what you will but it's it's actually very easy to find this data right and for topics we actually started like when we first started this project we were thinking really really complicated like you know super LDA all the things and whatnot but we found actually pretty early on was just a simple bag of words and counting frequency does really well for finding topics as long as you remove all the stop words so with these two things we can actually see the models in suite you know the tool to tweet at a time that the user is likely to respond and also tweet on something that they're likely to be engaged with great so so at this point now we've taken a bunch of input users and extracted a subset of them that we want to target and we calculated what they like to talk about the topic and we've also determined at which time they're most active with with Twitter or with the Twitter platform so now how do we go about getting getting them a tweet that they might be more likely to click on than your normal any random question so we do we do this in two separate ways and the first way is we leverage markoff models so markoff models they're popular for text generation like John said the subreddit simulator or in the info sect talk title bot but how it works is using Twitter API you can go and grab the last X posts on someone's timeline right two hundred five hundred thousand however many you want to grab and we call this the corpus so you take your corpus and you want to learn pairwise frequencies of of likeliness between these words right so for example you might you might have the word I that occurs a lot within this corpus sometimes it might be followed by the word don't other times it might be followed by the word like so based on the relative of co-occurrence of these words in your corpus you can then generate a model that probabilistically determines how likely it is to create kind of this string of sentences I like or I don't and you can continue this for the length of the entire tweet so it's based on purely transition probabilities from one word to the next on the other hand we train our current neural network and this is called LSTM and LSTM is an acronym for long short-term memory and so this is a bit more cumbersome it's less flexible than the markoff model we took five and a half days to to train this neural net we had to do it on an EC2 instance using a GPU cluster and the training set was comprised of approximately two million tweets we didn't go out and just grab your run in the mill any two million tweets because like I said Twitter Twitter is a variable cesspool so we had to go and find kind of legitimate looking tweets to do that Twitter has an account called at at verified and that account in turn follows all the verified accounts on Twitter all the ones with that blue check mark next to it and so our idea is that this the people that are that are verified accounts are probably more legitimate they're probably posting about some kind of relevant information and so we train it on this huge corpus of tweets the network properties we use three layers of this neural network and approximately 500 layers per unit units per layer sorry and the idea here is that neural networks are or at least this neural network in particular is is much better at learning long-term dependencies between words in a sentence so LSTM's are often deployed when people want to learn sequences of data and in this context you can imagine a tweet or a sentence being a sequence of words right so as the in in contrast to the Markov model which just cares about the pairwise frequency the word that follows this word there are current network neural network on the other hand considers longer term dependencies because what I talk about in the beginning of my sentence might also relate to something that comes later on this is common in all all languages in English and most common in German actually you have these long term dependencies you might not know what the context of the sentence is until someone finally finishes the word at the end of it so what were the differences between these two approaches the LSTM as I mentioned took a few days to train so it's a bit less flexible whereas the Markov train Markov chain you can deploy it and it can learn within a matter of milliseconds and that kind of scales depending on how many tweets you choose to train it on the accuracy for both surprisingly was super high so even though the LSTM is a bit more generic and by that I mean it learns like a kind of a deeper representation of what it means to be a Twitter post and I I caution myself not to call it English because as John said this isn't English this is kind of Twitter ease it's filled with hashtags and and different kind of syntactical oddities and abbreviations so the availability of both of these tools is public you can go out you can download an LSTM model using different Python libraries or otherwise Markov chain as well and the size of these LSTM is much much larger the story on disk compared to the Markov chain but like I said the Markov chain tends to overfit on each specific user the idea being let's say you're posting today or in the next week about the Olympics or something like that maybe two months from now if I go back and I read your historical timeline posts and I I tweet back at you with something about the Olympics it might raise your eyebrows because the Olympics have been over for a while and you don't really care about that anymore the cool thing about Markov models though is that you don't need to retrain it every time like I said it's very flexible you can deploy it very fast what this means is that it generalizes out of the box to different languages it's language agnostic so if you're posting on Twitter and you're posting in Spanish or even Russian or Chinese entirely different character sets because it's based on these powers probabilities it's going to dynamically learn you know what word likes to be followed by the next and you're able to post a tweet back at somebody based on the language there they're typing in so here's an example that's in Spanish and if anyone is from a foreign country here with a lot of foreign language tweets and wants to volunteer for the demo again please tweet at that hashtag snapper so we don't like to think of this necessarily also as a Twitter vulnerability so to speak this can be applied to other social networks as well they all have pretty accessible API's but the idea here is that kind of like with the rate with the rise of AI the rise of machine learning and the democratization of this as it becomes more and more possible to do this without a PhD for example and the technology grows and grows and becomes more available this is going to be become more and more of a problem right so the weak point here is a human this is a classic social engineering cool yes so before we get into the evaluation results and demo I just want to say on the tool is public so for example there's a version on your conference CDs and there will also be a github link that will tweet out as soon as we get back home to Baltimore but we first we first trained our first couple of models and started wild testing it and we were surprised it did really really well I don't know if you can actually see some of the pictures but for example we got a guy in the top right the first post is what our buck posted and the second is like the guy responding saying hey thanks but the link's broken right we actually saw this quite a bit and on the bottom you can see some of the example tweets from the first models that we made so we we use these first couple models and we did some pilot experiments we grabbed 90 users from hashtag cat because cats are awesome and we went ahead and tried to spearfish all of these users again with benign links and we were actually surprised at how well the model did right out the box after two hours 17% of those users had clicked through and after two days we had you know between a 30 and 65% 66% sorry click-through rate and so why that range is so huge actually is because there are a lot of bots crawling Twitter clicking on links so we actually don't know exactly how many actual humans click through if we use the actual strictest definition of what a human might be so making sure that for example the referrers t.co and the location matches up with the location listed on their profile and those sorts of things that's where we get that 30% number if we if we use a little bit more relaxed criteria for judging whether it's a human or a bot we actually can get up to like the number of people that we think click might be up to 66% and so actually a funny story with these initial models also we saw how well they created and an information security professional who will remain unnamed tweeted us saying hey proof of concept or get the fuck out of here so we went ahead and used him as a guinea pig and it did actually he did click the link so we will say that cool so so then we iterated on the model some and we decided we wanted to test this against a human right see how well the human could spearfish or fish people versus how well that the tool could and so we had two hours we scheduled on our calendar and the person was able in these two hours to target 129 people and he did so mostly by just copying and pasted you know pre-made messages to these different hashtags that we talked about previously I think they were Pokemon go info sec and something about the DNC and so we he was able to tweet at 129 people in these two hours which comes out to be 1.075 tweets per minute and he got a total of 49 click-throughs we used one instance of our tool so one instance of snapper running and in those same two hours snapper tweeted at 819 people which comes out to 6.85 tweets per minute and 275 of those people had clicked through and we sort of want to emphasize that this is actually arbitrarily scalable with the number of machines you have the major rate the major limiting factors are actually rate limiting and the posting mechanism so sort of a TLDR this tool that we've made there are two traditional ways of you know creating tweets or messages that people will click on the first is you know fishing which is mostly automated already and has a very very low click-through rate between five and 14 percent there's also this other method called spear fishing which takes tens of minutes to do it's highly manual you have to actually go out research your target find out what they enjoy doing what time they're interested in posting at things like that you get the best spear fishing campaigns actually get up to a 45% accuracy from what we've seen and we actually kind of split the difference we actually combine the automated characteristics of actually fishing but we still get pretty close to what the actual effectiveness of spear fishing and with that demo God's willing we'll do a live demo of this cool right so I just want to see so about a hundred fifty one of you have actually tweeted so this is the actual command to run the tool and we're gonna go ahead and run it hopefully cool I'm actually the first person on the list because I actually you know wanted to make sure that something worked right so what it's doing is actually it pulled down the user's timeline and generated a tweet for that person and cool actually okay so here's it's starting to come out so here's that actual post that it generated and it posted you know at my hashtag the text that it grabbed from my profile in the short and late and so you can see that that actually works and we're not just saying things so notice that on my actual you know timeline you can't actually see that post right and this is because it's actually called a reply but hopefully yep so here's where it actually shows up it shows up in your notifications not your actual tweet history and so you're the only one who can actually see that and so as you can tell yeah I just got spearfished if I click this link so it's actually running through all of you guys now who tweeted at the link and generating text for you and posting them so we'll leave that running as long as possible but it probably won't get through all of you guys while we wrap up the talk cool thank you demo guys right then just a few words to wrap up why did we do this we want to generally just raise awareness and educate people about the the susceptibility and the danger of social media security like john said people usually think about email very cautiously you would never open a link in an email from someone you've never interacted with before we want to have that same culture be instantiated on Twitter now and on another kind of social networks another way that you could use this tool is to if you belong to a company or in some other kind of organization you want to do some internal pentesting to see how susceptible your employees might be to some kind of attack like this this could generate good statistics for you and help you refine your kind of educational awareness programs you can also use this for general social engagement and staff recruiting reading stuff of people's timelines and then crafting a tweet geared at them might be a good way to recruit people or even for advertising the click the rates here we have our our pretty huge compared to your general generic advertising campaigns so like I said ML is becoming more and more automated data science is growing a lot more companies are hiring data scientists and the tools in the toolbox are becoming a lot more democratized you can you can easily go out there's free software you can use to train these models including the one that will release today so the enemy will have this or the adversary will be able to use this leverage this kind of technology sooner rather than later one way you can try to prevent these kind of attacks is to enable protected account on your Twitter on your Twitter user so if you protect your account we can't go out with the through the public APIs and grab your data there might also be ways to detect this stuff using as I said in the beginning to talk automated methods like machine learning classifiers or or whatever have you and also if you're ever unsure always always report a user or report a poster if you see a tweet like this maybe Twitter is pretty good at actually responding to these reports and we use google.com as our shortened link that you redirect to so feel safe to click it because if we if we did something more funny like redirect to our black hat talk people might get pissed and try to report us we don't want our bot to get our bot to get banned and so in conclusion ML can only be used in a defensive way but you can use it to automate an attack Twitter is especially nice for this kind of thing because the people don't really care if the message is in perfect English it's slang laden it's abbreviation laden and these things actually help the accuracy of our tool and finally data is out there it's publicly available and it can be leveraged against someone to social engineer them and with that we'll take some questions so just step up to the microphone if you have a question so do you I can hear it so have you tried implementing anything like change point detection for because I know that some research has been done in using Twitter for like threat analysis as well as like trying to pinpoint users who say we're for like isle or isis and have you done any research using like Markov chains or prior distribution detection systems alright so we haven't done any research for the purpose of this talk into that but it's definitely a cool thing that we'd like to look into so if you want to talk to us a bit more after the talk about it we can get some you know information and trade some ideas great presentation quick question pertaining to the environment of a mobile platform as a supplies because I know you guys touched on mobile you mentioned phone or smart phone say could you kind of just give me any additional thoughts on that area I'm sure so we haven't actually measured like the differences between how many click on mobile versus how many click you know from a PC or something like that but it's something that we can definitely do so if you're interested in it you know tweet at us and we can crunch the numbers for you you're mentioning that your neural network version of the text prediction performed better than the Markov model in terms of like temporal accuracy what about the neural network cause that over the Markov model and what would prevent that from talking about the Olympics a month from now I'm admittedly a new bed neural networks yeah sure you know I definitely recommend looking at some documentation about LSTMs neural networks in principle can kind of replicate any any kind of arbitrary function this is a special kind of neural network that has different gates in between each each layer of the LSTM and these gates kind of turn on and off dynamically and so it allows you to remember words at like a certain depth back in time and it learns these connections on the fly and it's able to turn it off and on and because of that you're able to like learn longer to contextual information in these words great president I just have a question I wanted to see what kind of considerations you had for trying to prevent bias in your training set and what were some like time biases or even just using the approved Twitter handles might introduce some bias in terms of the data you're looking at could discuss on that yeah that's that's definitely some valid criticism so you want to avoid you know comment pit falls like overfitting to specific users especially in the in the clustering thing yeah we we didn't do any kind of formal evaluation of the LSTM we have a loss that we tried to minimize over time but in terms of the markoff model we just kind of tuned it until it looked good enough and it worked in in terms of like you know we we had several different tests in the wild and as soon as we started getting pretty high click the rates we got pretty confident that it was working so fascinating work with some pretty groundbreaking implications I mean given the fact that your intent is to fake people out to believe that these are real have you sort of passed a Twitter touring test if you will yeah it's a really good question so that the Turing test now is it's really interesting I think there's even conferences dedicated to having machines try to bypass or try to pass the Turing test and so there was kind of the much simpler version that was introduced much 50 years ago or 40 years ago or however long ago it may be and nowadays you actually have to check a lot more boxes in order to get past it yeah I mean given our click through rates it seems like Twitter is is super super easy to do this kind of thing on I mean I would argue that each kind of positive result here in our statistics is more or less passing of the Turing test right in the Twitter Turing test there as it as it were yeah for training the transitional probabilities on the markoff model did you only use by grams or did you consider using a bigger window right only only by grams only by grams thanks all right thanks again thank you