Good morning. Good evening. Good afternoon. Good, good time. Something in the world. Thanks for joining us again for another interesting gaming related to OpenShift live stream today. We are talking with several folks from IBM about Watson assistant and other Watson AI technologies and how they can be used for gaming. Specifically, we're going to talk about non player characters, which everybody has kind of a love hate relationship with sorry, it's getting real bright in here. And so what we're going to try and do through some help from folks at IBM as we're going to try and make some NPCs a little bit more interesting make them a little bit more interactive, make them not seem like they're so flat. So what I'm going to do is allow them to introduce themselves we're going to start with Alex and then I'll go sort of wise around. Yeah, for sure. Thank you. I'm Alexander Stanford I usually go by Alex. I'm in data and AI at IBM and on the side to do a lot of programming and Python. And I'm just here to back up the eSports, the eSports initiative that, you know, we have going on. Thanks so much, Alex Christian. Hello, good morning, everybody. Big thank you to Eric for bringing us out here. Again, my name is Christian Vasquez. I am a technical specialist here within the integration suite on on the side of this IBM house that we're fulfilling huge gamer always been a very passionate gamer and I'm really excited to see some magic happen and know I don't need the magic trading cards. I'm talking about getting our hands dirty and show hey that's gaming to don't don't be as gaming. You're right. This gaming. All right, Duncan. Hi everyone I'm Duncan Scott Martenson I work in data and AI as well as IBM as a client technical specialist. My day job I service financial services industry, helping them manage their data in a quick and easy way but my hobbies include like gaming and Christian hating on magic the gathering I, you know, I, I trade cards so there's that, but I really, I really got interested in this concept of when I got introduced to Watson assistant. How we could use that to create better video game experiences and so I'm really excited to share that with you guys today. Awesome. Thanks Duncan, and then Derek with your doggo. Hello, I'm Derek Reese. That's my doggo Zoe. I'm a principal engineer at Red Hat and I work closely with the gaming group. Well, so one thing that we wanted to do is I know probably most of the people who have bothered to tune into this or who will be watching it later are likely to be gamers or at least to know something about gaming but we wanted to sort of start I think, and I might be wrong so correct me IBM folks wanted to start with a history sort of kind of non player characters and kind of what what they are and and how they fit into the big picture. So with that I'll turn it over to the IBM folks to sort of give that spiel. Sure. So let's talk about the history of non player characters. You might be tempted to think that they originated in video games but they actually have a much deeper history than that, going back to the 70s with tabletop role playing games and dragons. I'm a huge nerd. So I play that and that's really where this idea came from. Is that you, for those who are not like too familiar with how Dungeons and Dragons works. A lot of people I'm sure are just finding out for the first time that it's in fact not a video game is that a group of friends generally sits around the table and collectively uses their imagination to go on adventures with with each other. So Dungeon acts as a Game Master or a Dungeon Master which the video game analog is there the computer. So the rest of the people at the table are players, and they will tell the computer or the Game Master, what they want to do in the world which can include like I want to swing my sword at this goblin. I want to say this to this character. And so the Game Master has to play all of the characters. What that means is you can actually talk to a character through your Game Master basically. Now, that's really cool. But when that got translated to video games, a lot of things in video games have roots in tabletop role playing games. And the non-player character got taken from Dungeons and Dragons and the computer now simulates those characters. But computers are generally will at least back in like up until now generally weren't great at holding conversation. So how do we get around that? Well, it started back in, in the beginning we had text based video games.  Classic Zork. Yeah, Zork, absolutely. Yeah, yeah, yeah. Then then we get to, we get to more modern video games and players want to talk to the characters. And so we get dialogue systems and those dialogue systems generally work like this. You go up to a character, you press a button, the character turns around and looks at you and says some intro line like what can I do to help or how can I help you.  And it's a really fast food menu of human interaction. Absolutely. Yeah, it's an elegant system. It works. It's easy to implement. But here's where I personally, not take issue but find weakness in it. It feels like if you've ever called like a customer service hotline. Yeah, I think it was like that, like press one for for account information press one for payment to make payment press two.  It feels weird to me I don't know what do you guys share that opinion. Yeah, it's super it's super shallow and the replay value is pretty low. And while it's simple to program. It's also annoying to program. I've never programmed it but I've made some terrible you know Dr space like games in the past and you know, it's like, basically building a tree of if then statements and they only can do what they can do and there's no room for sort of ad lib and you know you get like very flat and then non interesting repeatable experiences. You know something like a Skyrim or you know a non online RPG. If we ignore downloadable content. It's like you play it once, and then you play it again, and that NPC in that town at that point is going to have the same six, you know, for romantic kind of dialogue options.  Everybody's experience is exactly the same no matter what, or we have to implement a lot of bizarre coding depth, so that my choices but even then, if I start out as a jerk what at the beginning of Fallout three that player that I encounter halfway through the game is going to respond to jerk what me in the same way, every time I start out as jerk what kind of thing. So, Derek, what do you, what do you think Well, I think it's also pretty frustrating like you kind of noted from the developer experience you're essentially building a dialogue tree, you know, and then you're trying to figure out all right. What are the most probable outcomes, what are the most common ways that a player is going to interact with my game and want to feel fulfilled, and you're just only targeting really a subset of players. There's places where like, you have players who might be neurologically diverse or might make like really interesting choices that as a writer you find fascinating and want to be able to respond to, but you're stuck with your, you know, your 15 choices that you're going to tag your dialogue items to and, and cutting everything down right and that's frustrating you want to engage with your players more. Yeah, you just described essentially the Leroy Jenkins phenomenon and those who don't know look up Leroy Jenkins on YouTube it's a video about World of Warcraft you know, everybody else has this plan and then this one character just goes off on a tangent and does something completely random. And because it's all humans essentially playing like they all react badly but like you can't do that with a dialogue tree like I can't walk up to the king and spit on his face or swear at him or whatever because that's not one of the four choices that I get. So, I guess another personal gripe is like, I don't feel like I really have agency. It's like, here's the possible choices that you're allowed to do so you're kind of in this open world but still in a cage. Absolutely. I'll like me personally the very first time that I thought dialogue options were, I guess, almost as good as they as I had seen them was in Fallout four. But even then it was still some significant limitations because you would select like a way in which you could respond whether it would be angrily or like in a happy manner and a respectful manner. And I remember I would, I would select one that was being happy and he would still say some, you know, some kind of sarcastic smart smart guy response and the case and point is that although we are not trying to, you know, speak ill of NPCs up until this point I still think there's been some really great NPCs and video, you know, across the history is that a lot of people love, but it's just limited right and it's kind of shocking that over the last 20 years, there really hasn't been anything, you know, no earth shattering. Yeah, yeah, for sure. And so kind of I think what you're getting at is like, wouldn't it be cool if there was a way to make this better. And hey guess what, there actually probably is a way to make this better and that's that's what we're going to talk about today. So transitioning. What what's next we want to introduce the Watson assistant or what do you all have in mind. Like earlier I alluded to the, you know, you're talking to a customer service agent within your video games whenever you're doing dialogue. And I did that strategically and here's why. So we have, we at audit IBM have a solution that we've used to improve the customer service agent experience, the automated, you know, phone teller experience to make that conversational and that's called Watson assistant. And so what that does is allows the conversation with like a robo caller to be, it takes it from for for account information press one to make a payment press to you call a customer service hotline and the virtual agent picks up the phone and says, Hi, how can I help you.  It's still that system of these are the things that a person is going to generally want to do here, but it lets you come up with you let's you get there faster with your voice. And so, and it gives you that sense of agency right it's not like waiting through a menu being read to me like a two year old so that I can press for because that's my problem it's no like, I'm going to verbally describe what my problem is and then it's going to react to what I said, not to the fact that I pressed for Exactly. And a common sort of objection to this is, okay, well I've seen this before on a lot of people have seen this before generally on websites like the little chat window will pop up. And it's like how can I help you and then there's buttons underneath to say like what to click that's the same thing. Yeah, as as the choice. Yeah, it's that's the press one press to the differentiating thing about Watson assistant is that it uses natural language understanding to to be able to get that to give you that agency. And so you are truly it's like talking to a person. And I'll go ahead and I'll pull up. Yes, little screen share. Yeah, I'll pull up a quick little demonstration I suppose of of this working. So let me share my screen for a sec here. You've got to give me permission to do so. Oh, bummer. You would think that eventually one day I would learn how to like do this. Okay, you should be able to do it now. Okay, cool. Can everybody see this okay. Not yet. Not yet. How about hitting the share button. There we go. Yes, it did a thing. Okay, so I have a little preview of this is a very lightweight version of a what I'd like to call a cognitive inkeeper. So you make your font size bigger. Oh, yeah, sorry, I have a really big monitor that I'm using. Okay. Not quite 4k but how's that 13k. Yeah, that's great. Okay. Cool. So this is a cognitive inkeeper. Basically, there is a there's a pretty stable role of the inkeeper in either that comes from tabletop RPGs and has bubbled its way up to video games where you find the common practice of a video game player in an RPG. He walks into town, goes to the in and then finds the inkeeper and talks to them and that's generally the source of information around the town whether that be directions to stores or where can I find this person or what's generally going on in town they're generally like a core quest giver. Yeah, but like the funny reality of this game trope is that it's not a trope at all like this is actual reality. Yeah, like, especially, you know, before Internet in the old days right like think about the TV show Cheers, you know, you go to the bar and pony up to the bar and the bartender is basically the town therapist, and knows everybody's dirt and knows everything about everyone and has all the drama and all the gossip. And so the fact that it somehow made its way into RPGs is like, shouldn't be surprising. But I don't think anybody ever sits down and thinks like how realistic that actually is. It's mostly the implementation that's terrible because again you have these dialogue trees but yeah it's totally like, you know, not unexpected for the bartender to know everything about everyone. Absolutely. And so here I have the the preview of the window right now it's text based but with some technologies that we'll talk about later we can turn this into a voice interaction. Yep, using text or speech to text. So I have, hey there traveler what can I do you for the classic in keeper line. And so I can ask it something like, like, what's going on in town.  Whoa. Daryl says he even saw tracks might want to talk to them he's sitting over there in the corner. Like real quick quest hook. And then what else can I help you with. Okay, well, I want to buy a sword. I suppose comes up. If you're looking to buy weapons and armor talk to Fergus over in the smithy. I'll point you to your map. You know, simple stuff. I can also do some interactive sort of like talking to a person type interaction so I'd like to rent a room. That's the thing that you do it ends right you get a room for the night. I've got a room upstairs. You can stay the night there cost to gold points you want it. Now I'm good to gold's classy room. Yeah, I have a pretty high standard of living in this RPG. So right I can respond to it it understands what I'm responding to using some fun little programming this and all this this entire system that I've made here it's very basic. We could expand this to be so much more complex and deep. Making it keep track of the context of the conversation so I could do stuff like, Hey, where can I buy a sword or you buy it over the blacksmith. What about potions of healing. It wouldn't it wouldn't be confused by that with, you know, being able to track context of situation be okay well potions of healing I know that you were just asking about directions. So let's go over. I'm going to point you to the, you know, which hot or something. So yeah, this this all took me like an hour to make honestly. So it to make how did you make it. That's a great question. I took a word right out of my mouth.  Yeah, and so. Yeah, sorry, I was, I was going to say something now but it actually makes sense to say it later so we'll get to it when you show how you made it. So here's how I made it right. I'm brought to Watson assistant on IBM cloud. This doesn't look anything like programming. That's that's the thing I didn't write a single line of code. So it's really easy. Good job of setting you up here. You set me up like a like the frontline of a volleyball team or like it or like a terrible in keeper. Sure. So here I have all the assistance that I've written. This can get a little complex. Hey, sorry, it's still, we're still just seeing the in keeper box. Oh really. Let me let me reshare this. See your mouse moving around. I'm still seeing just assistance and in keeper, but nothing else. Yeah, that's, that's great. That's what. Oh, okay. I thought you had clicked into the in keeper assistant. No, no, no. This concept right here is going to get a little complex because I have to differentiate some things. And it's just like an instance of one of these things talking think about it like an assistant is like one customer service agent. And it's one NPC. Yeah, one NPC. I should stick with that analogy. And the skills here is what they're equipped to talk about. So if I click into this in keeper, which notice I definitely misspelled that that's in keeper. I can give it multiple skills. So in this case I have in keeper reference. I could give it another skill here. What that means is, if I wanted to make an in keeper in one town, like let's use World of Warcraft towns, I make an in keeper and storm wind, and I have an in keeper in I don't know, I've never been able to say that. I can keep the in keeper functionality like everything an in keeper talks about like renting a room, what are the rumors in town that might that's different but rent a room buy food, that sort of thing I can get quality and reuse it across all of my different servers. And then if I want to supplement with town information. Then I make a skill for that. And then I just plug that skill into my different NPCs and I can reuse that stuff so then maybe a town's person will also have all of the, you know, storm wind information skill.  And then for the specific person you really need to talk to, they have like a unique skill or unique dialogue dream.  I'm sure it's Richard Cain course from Diablo. Yeah, I have that as a skill so this is specific information that that character would know. Let's stay a while and listen. This, this might look familiar. This is a dialogue tree. But I know we just spent like 10 minutes on dialogue trees. It's different because one, this took not much time to program. It's not a bunch of if else statements. I didn't program at all. It's not programming. And it's just, it's more of just a way of organizing this because it's not, I know this looks like this is a sequence. This is really just an order of precedence. I can hit any of these at any order. It just puts, you know, which one we'll take precedence of the other. And that'll make more sense in a second when we start getting into this. Yeah, absolutely. So we start with a welcome node. These are all nodes in the dialogue. What these nodes do are allow you to set the conditions for what should be said at a certain time. So in this case, the assistant wealth recognizes welcome, which is a system variable that basically just says the conversation is started. Then I'm going to say, Hey, the traveler, what can I do you for? I can also add a bunch of variations to this. Like, how can I help you? And then another one. Oh, English. Yep. Old English. And then set it to random. And now this thing will say, we'll randomly choose one of these responses every time I open up the chat window or talk to it. Then I have other things down here. These are, these get more complicated. Let's say I want my inkeeper to be able to handle the functionality of characters buying food from it.  This is how this is tripped is if the assistant recognizes this hashtag order food, hashtag means that this is an intent. And I'll go into intent in just a second here. If the assistant recognizes the intention order food that I'm going to respond with, here's what I have on the menu, and then I'll prompt a menu to pop up. So what's an intent? An intent is basically Sounds like what I intend to do. Yeah, it's what a person, what a person interacting with this agent or NPC would intend to do. So in the case of order food, this is basically I'm prepared to handle a person ordering food. How would a user designate that they are trying to order food? Well, they would say something along the lines of, can I get a drink? Can I have some food? Do you have any beer? I want something to eat. All of these are user examples that you just type in. That's the only way, or that's the way you train these, these assistants is you can just type really quickly. Like what do you have on tap? What's another way to order food in a video game? Right? I'll go with I'm hungry. Sorry. Yeah, I wasn't sure if you were looking for like a dialogue input or like a, like a actual, like a design element or whatever. Yeah, no worries. So I've added that to my list of user examples. And now it's just helped train the model even further. The cool thing about this is because it's artificial intelligence, every time I talk to this thing or every time a character talks to a player talks to this thing and it correctly identifies the intent that can be added into the user examples. And the the NPC will get better and better at recognizing these intents over time. Oh, that's cool. How, how does it, this may be getting us off track, but how does it know that the intent has been correctly identified? So that would be that I'm not too sure about, but I think I would have to be some sort of feedback system. So I know, I know, for example, when I try it out here, like if I say something completely random, like, I don't know, give me something that you think I haven't prepared for. Ken, do you have any beverages? Oh, I meant, well, I meant just an intent in general, but I, Oh, sorry, an intent in general, like, do you sell lingerie? Like I'm super tired. Okay. Yeah. Let's try that. Yeah. So, like, do you sell lingerie or relevant? Like it gives me this drop down here for when I'm training it. And I can, Oh, this is, this is, this is the training interface. Got it. Okay, training interface here. Okay. So it should have understood that this is something else. And when you, right, it shouldn't have understood that at all. Sorry. That's what, yeah, that's, that's what I meant, but I can set it to say that actually was somebody asking about directions to a store. Right. And then it would go, okay, I've added that to my, you know, thing about, I've added that to my user examples for how people ask to for directions. So what about the beverages example? Do you sell beverages or have So here's the thing, I don't have that in my user examples. And yet it's still able to get that something about the fact that beverages is a synonym with all of these other, you know, subject to beer. Do you have like, okay, I'm trying to order something and you're talking about food. And that's very similar to what I've also, yeah, that's what I was hoping would happen. Right. So it, it, it understands enough about the English language, like you said, to know beverages, drinks, food, beer, these are all items that are related. And so it, it made a good guess that, Hey, since all of those things are in this food intent, this interaction is probably about food. Absolutely. Yep. Okay. So we can do some more. I feel like this is arguably extended to a lot more, you know, like, What do you mean? I'm sorry, Duncan. What do you, what do you mean? Well, earlier, and I, well, I was actually earlier before I ended up disconnecting. Sorry about that. Okay. Lost lost my wifi. Welcome back. One of the things that I was going to bring up that I was like, listening to you as you were speaking was about the, about how you can have certain skills per assistance.  And it got me thinking about, you know, how sometimes in a lot of video games, there's art, there's arguably that NPC that's like your best friend on the journey, or your compatriot along the way, or somebody that you could tag team with whatever. And it's just got me thinking about how, how many different skills you could have with that person and how maybe even over the course of the game, more skills could be implemented or unlocked, if you will, for that person. So you might have. Oh, that's cool. Right? Well, you might have a skill that starts off with. Sorry, go ahead. So like, so like the friend collecting like information about you as well. Is that what you mean? Well, that's, that's one avenue, right? So like, when I start to get, when I start the game, you know, me and this character, we just met or whatever, right? And then maybe as like, the character notices that my style of, you know, taking on my enemies is usually very, I have an example of what you're talking about sort of. Oh, yeah, cool. Well, so, but it, go ahead. I was just saying it just, it's got watching this has just got me thinking about how extensive this implementation of skills of intense of, you know, like I've really just a lot of the things that we have in our portfolio here in terms of Watson assistant, just like how intimate even it could get with with with MPCs. Yeah. Yeah. If you remember, or if any of you have played Dragon Age Origins. So one of the things in that game was during the campfire scenes, where you would interact with the other players in your party, depending on how you interacted with them, you could essentially change the nature of the relationship that you had with that player in your party. They were, they were players you could control in combat, but they were still technically NPCs for many other purposes. And so what you're describing Christian is kind of something like that where like over the course of the game, as you interact with the with the NPCs, if they're in your party, or if they're just frequent, you know, people that you come back to all the time, because they're based in that town or whatever. Yeah, you could change the intents or unlock different intents or, you know, maybe the thing learns over time or whoever. Yeah. That's super cool thought. And so one thing that I wanted to say now, as we look at this is, because this isn't coding, if you think about the best games that are out there, right, like one of the games that's really hailed right now, which is kind of an RPG is, was it the last of us or last of us to or whatever, what was that one? Yeah. And so that game is basically a story. Like, yes, it's a game, but it's essentially an interactive movie with an amazing story from everything I've heard. I haven't played it. I've no firsthand experience.  Arguably like thinking big picture, right? It's the story writers. It's the it's the people that that wrote the story. Well, with this tool and and a simple web interface, a story writer with a little bit of training can be the one who writes all this stuff. They don't have to code anything. So now completely independently of the game to some degree of the underlying coding, a writer can basically craft this amazingly intricate set of intents and skills and knowledge across lots of players without ever having to touch code and without a developer ever having to care that they did this. And to me, that's super powerful. This is this is like one of the things in the red hat portfolio is rules engine, right? It's the and business process management and other tools like that in a weird way. This is kind of like a business process management problem. And and the the selling point, if you will, of a lot of those tools has always been your business analyst can can write the rules. They don't have to know how to code, right? It's like it's as easy as putting stuff in Excel. Well, here's an example of where a writer can essentially craft an exciting and interactive story experience that goes into a game that doesn't require any code. To me, that's super cool. And it can change constantly, right? So in especially for like a MMO, you could go in, you know, on a weekly, daily, hourly cadence and be changing the interactions you get from this character. You could be changing the things they know. It's super easy to roll out dialogues for new quests and all kinds of things like this is super cool. And we could take that one step further, like how Christian was talking about as you know, as you interact with these characters, you unlock new intents and stuff, you could you could you could take that a step further with sentiment analysis. So we can even unlock new ways that you interact with the characters based on how you've interacted with them in the past.    But again, without having to like, code lots of fancy stuff in the game code to make that happen, it's already built into the assistant and the dialogue. And at the control of the writer directly, not needing some coder to sort of really get in there and do it. Alex, if you had a VR on top of this, this will actually get like scary immersive. Yeah. So so Duncan mentioned earlier, text to speech speech to text. And so I hope we maybe get to that a little bit. But one of the things that I was thinking of is, you know, you have this, we're in an era of 3d modeling, where my understanding is if you build faces, right, the face moves with the dialogue, and it doesn't have to be coded to do so. So now all of a sudden, you have an NPC that's capable of responding in 100 ways. And you have the text to speech engine that's going to render it. So now you pay, actually, let's skip to Watson speech real quick. If that was what you were planning, I don't have any like asset to show that off. That was more just a conceptual thing. But well, I mean, you could just go to it. Yeah. Or yeah, because I think you can even do it from the web interface. Maybe I don't recall. But anyway, so the point is there's there's this Watson text to speech engine. And they've got some out of the box speeches, sorry, voices. And if you think about it, like what's not that I want to put people out of work, but what's a big thing in gaming, it's, you know, paying voice actors, right? Like, and they've got to come in and they got to record every single line of dialogue that's written down. And so imagine if you had to do that for something immersive, where they could respond in an interesting in different ways, like, that's really hard. With the Watson text to speech, you essentially pay the voice actor to come in and record like, what was the movie? I can't remember if it wasn't a Terminator movie, where like, they were trying to break into a bank. And they needed to get the guy to say a bunch of specific words, because that was like the passphrase, to get into the vault or something like that. And they were trying to get him to say passport. And she's on a date with them. If anybody remembers this movie, let me know. Oh my gosh, I haven't been any attention to the restream track. It's not demolition, man, which is the only thing on my mind right now. But yeah, sorry, I totally haven't been paying attention to the stream chat. So I'm going to fire that up now. And hopefully somebody knows what movie I'm talking about. But anyway, what I'm getting at is you pay a voice actor to come in and spend a few hours to train the voice model for Watson text to speech. And then now, when the chat assistant generates a dialogue string that gets fed through text to speech.  And it's and it's accurately sounding like that.  No, go for it. Well, one thing that I that I love that you brought up is again, how we're able to utilize and basically like again, we're over here brainstorming and thinking about all the crazy cool stuff that we could do. But the reality is is that both IBM and Red Hat are essentially removing the technical gap that used to exist for a long, long time. More than more than that, the very hurdles. Yeah, the hurdles, the barrier, I absolutely I couldn't agree more. I think I think that is a massive superpower of utilizing these technologies because you put really, that's what it is, right? You're putting it in the hands of, you know, insanely creative individuals that can make these breathtaking stories and and really focus on these like incredible interactions that you can have in the video game. I'm from the school of thought that video games are art. And I don't think a technical barrier should limit people from making really cool art. So it's not even just that, you know, it's game developers are smart bunch, right? Like, they have no problem going in and learning how to write, you know, neural net code and work through training models and that sort of thing. But it's an infrastructure problem, right? If you're trying to ship a game, you don't have the time to spend four months spinning up your own cloud and setting up all these models and getting all that data in there and building the interface for it, right? So you're not going to do it because it's more effective for you to go program the loot system or more, you know, effective for you to balance out the gunplay or the timing on things or something like that to ship the actual game. But if you have this sort of infrastructure already built for you, and this is a service, you can query it and hit it in real time, all of a sudden you're focused on programming gameplay code and all that infrastructure is taking care of for you. And you get these amazing features in this real depth that you can bring in and make that world come alive and be reactive to a player. It's the it's a classic it value proposition that red hat and IBM have been selling to traditional enterprise software companies for forever, right? Like, hey customer, you are spending a tremendous amount of your operational money, like building and managing things that you could buy way cheaper from us. Like, not that we want to put your people out of work, but why aren't they doing more valuable things? Like, you know, essentially, like, unclogging toilets is not the value that your development staff, like, you know, is really best equipped to do. They should be doing cool things and solving cool problems. Like pay us to fix the toilets. We're plumbers. Yeah, it's cool. We're okay with that. Yeah. And so to bring this like, yeah, bring this full circle too, though, like some of the additional work that I know that IBM has put in on text to speech, right and integrating with assistant is preserving the performance of a voice actor, right? When you do custom voice training, this isn't just your standard neural network, it's just going to pick up like syllables and start stitching them together and that sort of thing. There's actually like detection built into this system, right? And I don't I don't know much more about it than what I've leaned from playing with it and reading all the docs that are available online. But from what I can tell, right, part of that value proposition is this isn't your standard thing you get on GitHub and then spin up on a cloud somewhere, right? This is a custom suite of software that is very much intended to represent any sort of performance that an actor or a customer service, you know, actor or whoever would be able to bring to your brand, right?  That performance is preserved, the inflections, the pauses that that kind of gate of someone's voice is all there when you're able to build out that Watson text-to-speech voice model. Yeah. I was actually able during that conversation to find an example of the custom trained adaptive text-to-speech. I'll go ahead and pull that up really quick to show the audience. Can you guys see a screen here? I do see it. I don't know that we'll hear it. I'll play one of the things really quick. There's a very common type of phone. Do you hear that? One showing namely red and yellow with little or no green or blue. So that's the training data and then we can look at, we can put that into Watson text-to-speech and adapt it to the voice. The thing you just played was the actual person talking. That was the actual person and here's the model spitting that out as text-to-speech. Could you provide us with new information? So it sounds pretty similar to the guy. Yeah. He's been useful advice and has his influence. Yeah. It's got a slight like a slight British accent almost. Yeah. And then we can we take it down to here. The rainbow is caused by reflection of the sun's rays by the rain. So we have a woman with a, I think that's an American accent. It's a particular accent of some kind. I can't place it, but yeah. And then if we hear the trained. Could you provide us with new information? Like that could fool me maybe 70% of the time. Did they have any useful advice? So if we wanted to go with the completely procedurally generated sort of voice or like speaking, we can do that with still having voice actors or you could use the web hooks and the programming aspect of Watson assistant to trigger voice lines. Yeah. Well, you'd still use the voice actors. You'd just do exactly what we did. You'd bring them in. You'd have them read a few hours worth of dialogue. So they still have jobs and they're just doing stuff. But the actual in game interactions are procedurally generated text to speech from the stuff that comes from the chat assistant in the voice of the voice actor. And so now, you know, again, I get not about, you know, taking jobs away, but technology like, Hey, we've got new DLC, but we want to use the same. Let's think about a really morbid example, right? The voice actor has passed away. We want to do a new DLC with this character because they're such an important, amazing character in the game. Like, you know, like, if they wanted to do more Hogwarts, more Harry Potter movies, like they can't because the actor who plays Snape is gone.  This is why the Simpsons has run for a bajillion years, you know. So anyway, to be clear, we're not commenting on like, you know, the morality of ethics of that, right, that are involved in this. It's just all theoretical. Yeah, this is just pure, like business continuity, you know, like stuff like, yes, there's there's always a social cost to everything, whether that's positive or negative. But yeah, I'm just thinking with my business had on like, oh man, like, it'd be super cool to come back six months later, and just like add a new DLC with the model and like, here's the voice and here's the, you know, we changed the Texas speech. And now all of a sudden, the same character in the same room in the same in can talk about something completely different. Yeah. It definitely adds to scaling again. No, it's super cool. So what? Did you have anything else you that you want to show? I can't remember what was on the agenda. Yeah, I want to show one more thing within our one more technique that we could use to build out this this inkeeper to be pretty actually informative, I should say. So can we see the the inkeeper again affirmative directions? Sure. Yeah, we talked about intense. So this is what a what a player might want to do. There's another thing that wants an assistant to recognize and that's entities and what an entity is is what an assistant or what a player might be talking about. So this is basically a noun within your game or within your domain for talking business. And so I can take these I don't know. Now these are these objects and categorize them into entity types. So for example, I can take all equipment. So this is I'm defining equipment as something that a player uses. This is like a sword. Oh, super cool. Yeah. Yeah. So any anything in that list of armor is is generically armor. So it's any synonym with armor. So I put a bunch of like, yeah, kinds of armor to be a synonym with armor. And armor is a type of equipment.  So now if I go where, where can I buy racers? If you're looking to buy weapons or armor, talk to Fergus over at the Smithy. So I go to my dialogue, how does that work? I have my directions folder and then shop directions and then this Smithy thing. If the assistant recognizes that the player is talking about the logic on this right now is a little shaky. That's my fault, not the platforms. It's simplistic because it's, you know, we just threw it together. Yeah. If it recognizes that I'm talking about, well, first of all, let's, let's take a step back. If I recognize that I'm talking about the intent store directions, and I recognize that I either see mentioned to equipment or talking about the shopkeeper, the blacksmith specifically, then I talk about weapons and armor, Fergus, you know, that sort of thing. So I can also have a catalog of all of my shopkeepers kinds. So if I say, where can I find a witch? Maybe that's what we call healers. Oh, I didn't get that one right. Let's try, let's just, I know this one works. Where's the town blacksmith? If you're looking to buy weapons or armor, talk to Fergus over at Smithy, that sort of thing. So I've recognized I'm looking for store directions and I've seen blacksmith. So now what, what can we do and how can we expand that? Well, we take all of our items within the game and classify them, not even just items, we take all our items, we take all our characters, we take all our monster types, we classify anything that people might want to talk about that's around. And now I can, and now my characters can recognize that and speak on that. So maybe you have them giving you information, or maybe if you walk into a town and say a certain character's name, everybody goes quiet and like the innkeeper says, we don't say that name around here or something like that sort of. He who shall not be named. Exactly, like that sort of interaction, we get that with entities. That was no, that's super cool. And this also gets you out of the the non non non interactive non player character, right? So you have that that trope of like, you walk through the town and like everybody wearing a gray hat when you talk to them is going to give you this one liner with no interaction because they're not a real NPC, they're just like a decoration that happens to have one sentence that they could say. You can completely get out of that and now your generic NPCs can all at least be somewhat interactive. And if you really wanted to, you could like sprinkle some knowledge across a bigger portion of them in some meaningful way, but then there still are those like key people that actually have the real knowledge. So that's that's actually another super cool kind of trope here.  And then just like that's all they got. Yeah, I would love to be able to walk up to an NPC and be like, Hey man, where's the blacksmith? And he'd be like, Oh, it's that way. Right. Yeah. And the only reason that the game developers didn't code that into every single NPC is because it would have been ludicrously expensive from a code complexity perspective. Whereas with this, it's just like you attach this interface, like at the low level in the game code, there's just like the Watson interface, if you will. And there's already like, you know, SDKs for lots of languages. We're going to show the Node.js one here in a minute. But yeah, basically, you just attach NPCs to the Watson interface with the intents and the dialogues that they should have. And presto like now every single thing, I mean, you could make the plants talk like anything. And with the with the custom Watson text to speech, we could we don't even have to hire a bunch of voice actors to to voice act all those lines for every different for every different just bulk NPC walking around town. That could so maybe we have like star performances on our major NPCs, like our named characters. But then everyone else could be the custom text to speech. So then you can actually have these dynamic conversations with bulk characters, I'm calling them. Yeah, so so so for for the for the Deckard Keynes, that person is still voicing all of it. And if anything, they have to do more because you have such a wide degree of interactivity and variability that they have a lot more to record from the set of possible responses.  And so, you know, yeah, in a particular town, lots of people might sound the same, but you could still have six 1015 unique voices that are procedurally generated. And you could you can add randomness to the parameters of the custom voice to change it a little bit. Yes, there's timings. It can be altered and pitch and all this other stuff. Yeah, I mean, the possibilities are super wild. This is like so cool. Connect that back to D&D too. It's the same thing dungeon masters would do, right? If it's a, you know, run of the mill NPC, you're simply like, Oh, yeah, the NPC tells you that you can go that way, right? You're not going to come up with like a voice and a character in a backstory for like, you know, but a good DM has like a like a voice that they use when they're talking as this. Great, exactly.  That is, you know, someone that may have a hook for them or some interesting quest or something.  They're like, there's some interesting character. I'm sorry. I just had the dumbest idea, which is not like I want to hear it now. Forget video games, right? Let's go back to paper and dice RPGs. Like you can use Watson text to speech to build. Yeah, no, like, and like who doesn't, you know, and especially now in the era of COVID, when you've got people using sites like roll 20 and all these other, you know, like platforms that help make paper and dice virtually easier.  Again, I don't have to be a developer.  You know, so it's like the possibilities are really just super wild. This is so cool. I should point out that I'm using the light version of Watson assistant to make this demo. So to make what I've made today is free. So if anybody wanted to go out and play with that and make their own, you know, DM to play with, they could do that pretty easily on the IBM cloud. Also, I do, I do DM Dungeons and Dragons, and I definitely have like three voices that I do, and I like mix them up like a little bit character to character, but everyone's either Deep South, Cockney, British or Royal Snot.    All right. Any, any other stuff that you wanted to share before we flip? Yeah. Yeah. One last thing and then I want to talk about, I want to talk about the analytics portion of Watson assistant. So this is where this is where the scalability comes into play. So now we can keep, we have a built in dashboard to keep track of I mean, mine doesn't have much information because I've talked to this thing all of 12 times. Yeah. But we have a dashboard to keep track of what's going through our different agents and we can use that not only in beta testing, but in, you know, developing DLC down the line to figure out what are my players saying to what characters at what intents are they trying to do that I don't have programmed in right now so that I can add those in later and make this this is in in super pure coder terms. This would be like cash misses or, you know, like how many 404 errors am I getting on requests to services or whatever? Like this is this is sort of a humanization of that, right? Like how many times did the player try to interact with this thing where we missed the mark on that interaction? Yeah. And I can keep track of those conversations here. And you know, I have all of my all of my history with this thing and figure out like, what is it? What is it recognizing? What is it? What are people saying? You know, ways to keep track of how this thing is working and, and, you know, upgrade it over time. Yeah. Well, so again, that's that's sort of real world example, right? So, you know, World of Warcraft comes out with a new, you know, whatever, and there's a quest in it. And like, people are just lost. Like it doesn't make sense. Like something is lost in translation, right? Like how long would it take them to fix that? Whereas with something like this, if it's just dialogue direction type problems, a writer can fix it in five minutes. And so it's like all of a sudden there's a tweet storm about like, you know, this awful whatever, like, oh, hey, I just fixed it. It's already fixed like done, like there's no code to roll out. We don't have to release a new game client. There's no server update. It's just like magic. Yep. So that's that's what I have for in terms of me showing stuff off. But I thought the I thought the analytics portion of what's the system is really important. No, that's this is this is super duper cool. All right. Go ahead. And if you stop your screen share, I will start mine. Actually, I think I can just force take it over desktop to use the force there. Use the what? Use the force. Yeah, use the force. Okay, so I actually I had spun up a text to speech. We'll maybe try to use that. But this is I built my own Watson assistant, which is way more simplistic than Duncan's. And it's way more disgruntled sounding as well. Good. So that's going to take some time like our NPCs cranky here over an open ship. Oh heck yeah, man. Now we are we are all cranky old men. All right. So that's doing it staying my internet's being slow. So what I built here, move this up here and make the font bigger. Oops, that is definitely the wrong one. Now I'm gonna have to change my Watson API key. That's all right. It's free. I don't think anybody wants my cranky assistant anyway. Close that. Okay, so this is a very basic node JS application that uses the express framework. So it's a super simple response request framework, right? But the cool thing is that there is a node package for IBM assistant and IBM Watson authentication. And I think I have the documentation open somewhere. Hold on. Yeah, this is it. And it's super well documented. That's the Ruby one. Well, I guess I can just click the button. Node. It's super well documented. So this is the assistant documentation. It tells you how to create a session and authenticate. It tells you how to interact and send a message and what the responses are going to look like. Super easy, right? So I basically just sort of copy pasted my way to success here. And what we have is this very simple one. So it takes the ID of your assistant. It takes your API key. It takes the endpoint URL of your specific assistant creates a session. And then basically all it does right now is it sends a blank message to the assistant as the first thing when you come to this homepage on slash. And effectively what this does is it starts the conversation. So as a first message, if the assistant receives a blank, it's like, aha, that's the beginning of my conversation. I will respond with my introductory quip. And so here's just other kind of boilerplate Yuckey Node.js stuff. My, oh, don't care about open editors. My index.js file basically says, oh, I changed that. I'm not even using that. Yeah. So I'm rendering the index, which is actually an index view. And Node.js uses sort of a lightweight HTML-ish thing called pug or it's one of the options. This extends the layout and then it just prints a title.  And the title in this case is just express because I didn't do anything fancy. And then the layout is just boilerplate HTML stuff, right? Like, hey, include the style sheet and then spit out the content. Hooray. I have a terminal. Let me make sure that my keys aren't showing. Cool. Yeah. All right. I have a terminal here where I'm going to run this application locally. Start my express server. Scroll down a little bit. And come here and go to localhosts8080. No, 3000. Yeah. Look at that. Welcome to express. Okay. My application is working. Under the covers, what it's doing is it makes a call to the Watson assistant with the start the conversation.  Here we go again. Okay, great. So I need to get this now into my app. One thing to note, you know, what Duncan showed earlier, like, there's a way to just get this as a, like, just insert this sort of chat assistant into your game, right? But, like, in theory, my game is written in some other language. It's not HTML, right? We're not using a browser. Well, some browser-based games. That would be cool. But anyway, the point is, like, you would do some wiring of the response that comes back to your application, if you will.  Let's see here. Oh, I just shared my keys again. That's bad. I'll just delete that assistant after this. Okay. So we get this back, and it is a JSON, which is res.result, and in res.result was output generic. It's an array, so it's going to be zero text. And I thought I had done something with that previously. So if we do this, so this is going to be res.result, output generic zero text, I think. We'll find out the hard way. And then if we run into what happens. Yeah. T-I-A-S. Try it and see. Send it, as I'd like to say. Yeah, exactly. Inkeeper. And then a new paragraph with, oh, variable. Okay. So this is going to be text. We'll see what happens. Refresh. No. Fail. What did I do wrong? So we sent it, but we got a blank. Text. Is there an error? There's no error. This is the problem with trying to learn language live. Final text. Oh, no. Go ahead, sir. No, it's okay. Well, can I debug this? Well, let's do this. When in doubt, more logging. Oh, I bet I know what the problem is. Let's see. Refresh.  I don't have reload on. Automatic. Actually, I think I wrote a start script. See, this is the fun part of doing it live.  Command not found on line three. I mean, if you just need to really restart it, restart it. No, I know. A live demo. That's awful. You don't need to have a perfect automation. All right. So I reloaded it and now refresh. That's good. That's a good problem to have. That means I have to still do something else. So let's do. So result is undefined at that point. Is it a scoping problem? Res dot result is undefined on line three. Yeah, I think it's actually going to be something like. No, it's just going to turn into a string, which is an array in JavaScript. Oh, this is a string? Yeah. That turns the JSON object into a string. We could just make Alex do it in Python. I'll fight with this for two more minutes and then I'll switch to the Ruby one that I had that was already fully working. Maybe check your console. What does it say? Well, the weird thing about console log in node is that the console is the terminal. There you have the terminal. Yeah. So I thought I added the console. You have the initial one right on line 52, but the one on line 62 isn't firing, which is why I was asking if it's a scoping problem. Like, does Res dot result not exist on line 62? Because it's inside the scope of the above function or whatever. I see what you're saying now. Yeah, so you define it on line 40 and that is what inside the scope of that set of functions, right? So you need to have like a global variable, or at least a variable that's in that scope. Or I just need to actually move the render into there because everything else is error catching. So if I just do this, then it actually probably will work. So this would be there. You could do Res dot result that output, whatever. I think this is it. There's some JavaScripty way to make that less dumb, but I think this will work. Oh, but it's doing a thing. What are you doing? Can I read property zero of undefined? Progress. Yeah. And it's not global, it's generic, right? Output dot generic dot zero dot text. Sorry, say again. Go back to your code. So if you look at line 53, you put output dot global, it's output dot generic. And then zero dot text. Dot zero or bracket zero. Brace bracket. It doesn't matter. It's JavaScript. What you have is fine. Try one more time. Render is not a bummer. Yeah. Okay. We're going to need to get tricky here. So we thought we were being smart, but we're not. So what I need here is... Yeah, that looks good. And then just move Res dot render back to where it was. Yeah. Perfect. Going in circles, but that's okay. I should probably run that terminal inside my coder, but whatever. Okay. Marginal progress to nowhere land. So let's see. Text is there. Is this a scoping issue still? I mean, you can just do it at a console dot log, right? Text. It's not outputting anything, which is weird. Let's do this. A couple more hacks here. And then I'll switch to the Ruby. So generic is an object. Generic zero is an object, right? Yeah. So we actually want... Well, so this is a pure string? Go ahead and change, like, instead of Res dot result, do Res dot result dot output dot generic. Like the same thing that you have below that, right? Like text, or just do the console dot log in there, right? To eliminate any scoping issues. Oh, I see what you're saying. I thought I was interested in that. Yeah. You did it outside of that scope, right? Okay. So if you run that and you get the text and you know it's not the way that you're accessing it. So it's still some sort of weird scoping issue where... Yes. So where are you defining your let? It's at the top. At the beginning of where you enter this... I mean, when in doubt, I could use var, which is ugly, but might work better. Yeah. And it's actually, it's not... I don't think it's a scoping issue as much as a timing issue, right? Because at the time that you're running that code, text is undefined, but you're using assistant dot message, right? Not until the response happens that it's being re-rendered. Yeah. So you're rendering it once where text is empty and you're not re-rendering it after, you know, your text has been filled by Watson. So this is a timing thing. But we have Res here. So shouldn't... Yes. Which is still happening before the response from Watson. No, no, no. But I'm saying... Oh, I got you. I got you. So how does Express track? Like, what does it do with a value store? Like, I don't write with Express. I write with Koa. So if Express is tracking a value store, right, and that variable changes, how does it detect that notary render? Does it not do client-side rendering? Is it all server-side? I think it's all server-side. So then you probably need like a then statement, right? Like then render after your responses. So wherever that ends up getting defined. Sorry, where are we putting it? You could do that in a callback, right? Like this... There's a little bit more coding that needs to happen here. Okay, we're switching to Ruby. Yeah. That's what I would say. Okay. No, no problem. Maybe what I'm saying, right? Like your server-side application is building out the display before you actually even query Watson, right? Yeah. Now my keys and URLs are totally in the public. So what do we have here? We have the service that gets slash. This is going to be a Ruby application. So we've got a service that grabs slash and it responds with the start message because the input is nothing. And then it generates the index. Here we've got some simple stuff. And then there's JavaScript for a form that says when you type in the form and hit enter, we're going to send the message to slash chat message in ourselves. So we're going to post data to the application itself. Slash chat message says, oh, here's a message hash. The input text is the thing that came in. Go ask Watson about that message. And then the response is whatever Watson gave us back. And then in the JavaScript, we just update the HTML with the data that we got back, which is just going to be the message. So we're just like taking a div and just changing its content all the time.  That's what I thought. How was I actually coding this previously? Hey, that guy that did all that code yesterday, that wasn't me. I was old me. We don't talk. Yeah, right. That's the Watson node one developers are secretly ninjas. Engine X, how did I do this?  Do I have a gem file? I don't have a gem file. Let's take two more seconds. So gem install Sinatra installing documentation. Nobody wants that. This is where I'm a lot less. I don't Ruby. Oh, no, it's fine. This all worked the other day. I think I was doing it in a container instead of directly on a laptop, but I didn't save the container. And so it like all the installation of stuff is gone. So now I just I'm installing it directly on my laptop. See, if we had a fun, if we had a fun, cool cloud editor, like code ready workspaces, like I wouldn't have to mess around with any of this stuff because it all be over there somewhere. Weird. I wonder who has that. I know some weird red hat people. Oh, I'm going to need to make a gem file for this and bundler to make it work and open shift. Darn it. We're going to do no. Oh, really? There's a way to not install documentation. See? Oh, dash capital N. So we don't build docs, but we still have to build code. Bummer. That happens. You can add a dash dash no dash document to your gem rc file in your home folder and it'll skip documentation for everything unless you tell it to. Yeah, that's a good point. Could take a while, but it better not. All right, let's try this again. So the real question is for the rest of us on the call that I don't spend all day in code. How's the field? I used to do web development. I am rusty. So looking at this. This is a PTSD territory for you then. Yeah. Flashback. It's very validating. It's validating my career choice. 100%. The extent of my experience was all in mobile application development. So strictly like Java, Swift, and a little bit of Kotlin, but... Oh, it's just that's why. I was about to go chase. I was about to go chase lots of awful things. Hang on one more second. So I need Sinatra. Which is there's one gem I'm missing, which is Contrib, which requires install. Where's the installation of this? Not your Contrib. Oh, there it is.  So here we go. Hey, look, it's our innkeeper. He looks disgruntled. He is super disgruntled.  Here we go again. How are you? Oh, can we quit the chipper pleasantries and get to the part where you interrogate me about dragons? Okay, tell me about dragons. Ah, the dragon king. Look, you seem to be a pretty miserable slack of drag. So I'll save you the trouble. Instead of going after the dragon king, go to the back of the mountain, enter the cave, you'll find a door, smash it.  Trust me. Wow. Seems very trustworthy. Yeah. So that's my game here where I have my innkeeper encounter. Here's the, I'll post the photo that I used from Flickr, which was Creative Commons license. So I was allowed to do so. But yeah, so this is basically just a simple Ajaxi request response kind of thing. And if we think back a million years ago, right? Like if any of you remember Dr. Spezzo, you know, Dr. Spezzo was this artificial intelligence chat game from the early 80s where you would type stuff and it pretended to be a therapist. You know, that's interesting. Like, does your mother know you think that? But like, that's all it would ever say almost no matter what you said.   And so, but it was a game and people actually bought it and played it. And it was, it was fun air quotes, right? So from a games and interactive entertainment perspective, like, sure, I had to know a little bit of Ruby or whatever to, to make this, but like, it would not take a lot of time to learn a programming language to essentially build this video game that I have right here, which I mean, it is a kind of a game. Like, I am going to interrogate this character about stuff. And you know, there could be hyperlinks over here, you know, think back to your like early 90s web design, right? Like, you know, gray backgrounds and blue links, like I could have some links over here that's like, visit the blacksmith, visit the whatever, visit the whatever. And so Duncan, actually, if you in the conference chat, if you send me your URL, your API key, and your, what's the other thing that I need to hang on? I need your assistant ID, your API key, and your service URL. And if you send me those in the back end chat, I can swap out my assistant for yours. And this game will suddenly be using your IVR. Yeah, let me get that for you. Sure. I mean, it probably won't be as sassy as an innkeeper, but definitely won't be as sassy. I thought about putting in some sass, but I and then we don't have the, we could try to work on OpenShift. Go ahead. Yeah. Well, we don't have the text to speech API access, right? Because you could even just fetch like the output wave file. And well, I do, I do. And that's the thing. So you can do that with curl. So let me, let me pull that up because I did that. I actually did that very thing. And I actually tried to figure out like, how can, how can I fetch the wave in a way that I can actually like do, do something with it. So here's this, here's the getting started tutorial. That's totally not the thing that I wanted to do. I need the, this is what I need. Okay, so I want curl. This is the text to speech. Oh, okay, here's his API key. I will do my best to not expose this on the stream. All right, API key, service ID. I'm not sure I have the service ID, right? Because there's like a dash after the service ID. We'll see if, sorry, is that the, where's the assistant ID? Oh, okay, let me find that for you. I think I'm just in the, I was just off the IBM cloud page now if I'm going here. Yeah, the assistant ID is like, actually, I'll show you where you would find it. It's in the assistant settings. Okay. And so everybody can see my assistant ID now. Again, I don't really care.  It only talks about dragons. Incoming. All right, triple click, which is going to give me a carriage return, which is always fun. All right, so if I restart this application and I go back to, nope, that's the express one. This is, there we go. Hi, how can I help you? I would like a beverage. Here's what I have on the menu, a show food menu. Where can I buy a helmet? I didn't understand. So that's another thing that will be really important in developing these MBCs is really good. I don't understand what you're saying. Yeah, yeah. What are you talking about? Crazy person needs to be very convincing and not just like the IVR that said it wants you to appeal. If you're looking to buy a helmet, I'll point it to you on the map. But just like as an example, excuse me, I changed three lines of text and my game is now completely different because I just swapped out an entire assistant. Now, you probably wouldn't have to do that in reality, but that was a pretty easy way to do it. Let me see if I can get the text to speech thing going because that would be pretty cool. Speaking of in reality, Watson Assistant has actually been used in a video game and it's called, I think Ubisoft made this game, Star Trek Bridge Command, the Bridge Crew. And basically the way they use Watson Assistant in the game is you can issue voice commands. You're the captain of the USS Enterprise from Star Trek and you can issue commands to your crew. I'm not a Trekkie so I don't know like a common thing that you would that, you know, Picard would say to... Shields up engage or the... Shields up. Shields the maximum or something like that. Yeah, you can say that into your microphone and then the crew will, Watson Assistant will basically figure out what you're trying to say and then execute the crew action for that. So that's an example of this being used in a video game. It's just not dying. It's more of a command system. And that's the speech to text interface. Yeah, it's the backwards of the interface. Sorry, I'm looking down because I got another monitor down here. But really, there's nothing stopping anyone from getting that full system, right? Like, users says something to their microphone. Microphone parses that into text. That text goes to your AI Assistant. That AI Assistant responds in text. It turns back into speech. And all of a sudden you're having a... I'll be at slower conversation like you could with Alexa or Google Assistant or whatever, but specifically for your game without having to go build, you know, some sort of horribly integrated Assistant thing for, you know, that massive ecosystem. Yeah, currently, this would have to function using APIs. I don't think it'd be able to run... You don't have an on-prem version? Yeah, I don't think we'd be able to ship Watson Assistant to a player's PC and have it run well. So this would have to be an always online sort of game. Yeah, but it's not like those don't exist. Yeah, it's not like those don't exist. There's just... I just wanted to point out that it isn't a limitation and there is some dialogue around that. So here's a curl command that was run locally against the Watson voice Assistant. And you can see it used URL formatted by Hello World. If I do a LSLTR in this folder, I see a helloworld.wave. If I bring up VLC, I have no idea if this is going to be audible, media, open file, slash 10th. Oh, which it doesn't have access to or does it? I have to make this super loud and it's not coming out of anything. I think I did it here. Hmm, why is that not working? Let's do this. I have weird problems with my laptop audio, which is also probably part of the issue here. All right, that should work. It's only 41 bytes, which seems problematic. Small for a WAV file, yeah. Well, yeah, no, here's what's in that file, resource not found. I had this working. Many things that I had working two days ago. V1synthesize, accept audio. It's the Watson Assistant. Where's the... Oh, I think I used the wrong URL, that's why. That'll do it. 34 is different. Forbidden, okay. Did I use the wrong API key as well? Again there. Yeah, slowly by slowly. Export. What is it? API key equals control L, move this up. All right, one more time. Hey, that's a bigger file. 187. Bad request. All URL arguments must not be empty. Which one is empty? What does that example have? Because you only have what? Accept and text and voice? Yeah, which is like literally what it... That's all there was. I did copy of this. Accept, text, voice. Yeah, that looks like all of them. Maybe it's just a docs issue, like it shouldn't be and ampersand, it should just be a straight up ampersand. Like, get rid of the and amp and just get rid of the A and B. Oh yeah, yeah, no, I think you're right. That's what I'm calling. That was a smart catch. I've worked on docsites before. Somebody URL encoded some text and they shouldn't know. Oh, that's a much bigger file. There we go. That's good. That's what we wanted. All right, media, open file, downloads, hello world. Can you hear that at all? Yeah, I can hear that. Okay, so now what we need to do is we need to put the text right into there, right? Yeah, talk to Fergus over at the Smithy. So I need to URL encode that though, right with the with the percent 20s. Yeah, whatever. Ask the internet to encode it for me. So now we come back here and I'll leave it as the hello world file, which is fine. I will change a bummer. There we go. Hello world and voice. So this is where we go text, paste. It's a much bigger file. 260k. We'll open it again. Hello world. I don't know if you can hear that at all. I can barely hear it, but talk to Fergus over at the Smithy. I'll point to it on your map. There you go. And so that's the that's the female voice and then there's the default voice. So if I get rid of this one, we get a new one. And then if I open the file again, open file, hello world. Yeah, that's the Watson voice. Yeah, so that's the generic dude. And so, you know, any of us could go in and train a voice thing and have our own model. And if you look at this, basically what we have is a curl command, but just like we had the node and the Ruby stuff in their docs, they have, you know, they have Java examples for anybody that would use that terrible language. They have node examples, synthesize audio, which looks pretty trivial, because it's just synthesized when you pass in the parameters. Ruby, which is probably going to be the same, like what's the text, what's the whatever. So you could very easily in the application. Whereas let me go back to my node one. Oh, that closed that file. Open recent. Eric, if you see there, one of the languages included is also unity. So you can put this into a unity game. There you go. Which is just C sharp. Yeah, nice. Yeah. And I mean, this, there shouldn't be a stretch, right? If you're working with a curl request to put it into JavaScript. So in your little Ruby app, you could probably write like 15 lines of JavaScript that just says, you know, Jason, to stringify your item, right? Yeah, totally. You encode it and everything. Well, and it would do it. In the Ruby app, it would do it on the back end. So you wouldn't even necessarily need to do it all on the browser because you make the call from the browser into the application itself. That makes the call to Watson to get the text. It would take the response from the text, send it back to text to speech to get the wave. And then it would respond to JavaScript with the wave data. And then you would just play that using the browser interface. Exactly. That's just certain on this, but I think you can get the the text to speech models to be run locally. So then instead of having to do like call the Watson assistant, take the text call to text to speech, it would probably be implementation wise better to do call the Watson assistant because I think that's necessary, bring back the text and then pipe it locally into text, text to speech model that you downloaded onto into the game files and then run it through that. Super cool. Super cool. So yeah, we're talking to our endkeeper now. Yeah. Didn't have any, didn't have a whole lot else. I mean, deploying this on OpenShift is it's almost like not even worth. Here's the redhead arcade. I'm not actually going to deploy it into this project, but I'll just log in to show. I think this is right. I probably have to log in several sign on. No, okay. I do have, I don't have the Ruby one, but I do have the JavaScript one redhead game. So here's the node JS application, which doesn't really do much other than talk to the thing, but to deploy this Watson integrated application on OpenShift, it's quite literally click add, click get repo, paste in your get repo. There's an issue because we are using the main and not the master and this OpenShift doesn't have the update that fixes that, so I have to actually remind it that this is node code. Choose node 12. That's fine. You know, deployment, whatever, hit create, and then it's basically going to build this and deploy it and we're done. We have to give it some environment variables because I have to give it my key and my, let's see where's the environment variables are in here somewhere. I think after we hit the deploy button, they would come up, but I mean, that's it. It takes like two minutes to get this deployed into the real world and our end keepers up and running. I don't have anything else. Anybody else thoughts, comments, concerns? Looks like we lost Christian again, unfortunately. I think Christian's still here. I'm right here. Oh, my bad. I'm seeing myself and not you for some weird reason, mostly because of the chat application. There you are. Hi. Hi. Anybody have any parting thoughts? Well, let's talk about kind of what we like, let's summarize what we did today. So we went in, we're able to take a, without coding anything, we're able to just plain text, train a model to be able to understand how to answer different intents and entities to be able to empower our NPCs within the game to be able to talk to players and also have information about their, you know, the domain that they live in. Then we're able to take that into a simple, you know, Ruby on Rails application or Ruby application, I'm not sure which one it was, put that into a simple application connected to some text to speech and boom, we've got a document in Keeper. And that was really easy to do. We got a simple one up and running. As a developer, you could really take, you know, take the ball and run with it from how deep you want to get with this dialogue. Like I alluded to managing context variables to be able to track different things like, you know, a character sentiment towards a player or what a character is talking about. And with that, you could really create some really immersive video game experiences. Like Alex talked about, if we plug this in the virtual reality as well, man, that would be a game that I would, you know, I'd sink ours into that. Yeah, totally. And that was my motivation to bring this onto like a stream is like, I want someone to make this so bad. Yeah, yeah, please, please do this. Here's the idea. Go run with it. Skyrim modders out there. And I mean, there's already some Skyrim mods that do like three quarters of this. So just toss them, you know, an IBM cloud account, right? And we'll wrap up the rest of it. But I think for me, like that's, that's the most important thing is how long did it take us to get here, right? Like including prep time, which was only like maybe two hours between everybody involved, right? Not even. I spent more time fighting with languages and, and, and getting like typical programming language getting started stuff like, Oh, I have a brand new laptop. I've never installed Ruby before. Oh, God, here's here. Here goes an hour. You know, just losing time or Python, you pick your language right that getting started experience with like, how do I work with this language is awful. Hence code ready workspaces.  Yeah. Like that was my, of your developer at like a AAA studio, right? And you know, your leads come in and they're like, yeah, we really wanted to take a look at like building out these custom NPCs that can kind of be fully responsive and have all these extra details.  And it's not a quick and easy process at all, right? National language generation is pain. At the minimum, it's, you know, weeks worth of work of setup and experimentation and finding the right libraries and all this stuff versus this is literally just a couple of hours, right? In an IBM cloud account. And that's really like the most important thing that we've kind of covered the day is like, you can do this, you can literally go demo to your lead like, hey, this is how we can build our NPCs in the future, right? With just a couple of hours of work, right? And that's a proof of concept, you know, and this stuff is already production ready, right? Like these are used for telecoms. This is these are used for different customers that IBM has, you know, already out there. So there's, you know, this isn't something that's just experimental and like, you know, who knows if this will work. This is like, go build a proof of concept on this and you know, show that it's actually a real world possibility. It's like banks use this. So it's, it's ready for this. So yes. There's been a breath of life now that's been brought into the, well, oh, hopefully, right? Right into the video games industry. And I think we did a really great job demonstrating that. I'm excited to see if folks will, you know, hopefully go ahead and grab it and go running with it. I couldn't agree more, you know, so.   That's super simple. Like text-based games used to be amazingly super fun. And I think that if you had one that was more immersive and interesting, you know, you could sell that for two bucks. Like I might buy a cool text-based game for two bucks. It was using like AI and was actually interactive or whatever. And then from there, you make it more interesting. If you remember sort of the history of a lot of these RPGs, it was like purely text-based. Then it was sort of a little bit of a user interface where there's a static picture of the room that you're in. And that gives you some cues as to like what to do in there. And then we got to the place where there was maybe either some audio or like you directly controlled the character as they moved around. And so like just because you're not a full-on crazy whatever program, it doesn't mean you can't make something interesting. And really, I think in today's market, like even the definition of game is changing. So I was listening to a podcast the other day. It's called Developer Notes. And at least I think it's called Developer Notes. I'll pull it up. But they were talking about this game where it was basically you're exploring designer notes. You're exploring a girl's house and like uncovering like audio diaries and items or whatever. Yeah, gone home, right. Like gone home. Is that a game? Is it an interactive toy? Is it a story? Like it's all of those things, right? And so I think sort of what it means to actually be, you know, we have to stop using just the term game like interactive entertainment. Like you could make cool interactive entertainment with Watson and Red Hat and OpenShift and IBM. An idea that I have for it is I thought of the investigation scenes in the Witcher series. Or Police Quest. Yeah, sure. But like you could maybe implementing this as dialogue, like that might seem as a very daunting task because the world of things that someone might say to someone is huge. So if you wanted to just experiment with this and kind of prove and concept it out, you could bring it down to like an investigation scene. Because I remember in the Witcher, the investigation scenes are like, look at the feet or like look at the slash marks on the body. And it kind of walks you through it. And it's like, like, yeah, you're investigating, but you're not really investigating.  Well, here you'd be presented with like an image of a scene. And then the player would maybe, you know, talk out like, I want to look at this, and then it would feed you information like that. And so then you're narrowing the scope of the of the domain that you're going to be talking about. And you don't have to handle as much like randomness from a player talking. So that would be another way to just yeah, group of concepts. And that would be fun. I would play that. Yeah, yeah, for sure. All right, anyone. The only other thing I wanted to ask just that thought it would be cool. Do we have memorable NPCs that are favorites to you guys? If you think maybe the audience would really like to know about? I can start. Yeah, go ahead, Christian. If you guys have ever played Red Dead Redemption 2, it holds my favorite protagonist ever of all time, which is Arthur Morgan. But in the same game, Dutch is compatriot is lead. I'm not going to spoil the game for anybody who hasn't played it. I thought it was just fantastic voice acting, a fantastic dynamic that existed between Arthur Morgan and Dutch. And honestly, you know, just one incredible story. Absolutely worth playing. I played it twice over. We have like 170, 80 hours plus in one game. So anybody else? I think a notable example is the antagonist from Borderlands to Hans and Jack. Yes. That's what that goes down. He's already claimed as one of the one of the best video game antagonists in history. And the relationship you have with him where he just calls you randomly, the entire game, he's like the final boss of the game kind of right. But from the beginning of the game, he's like texting you and insulting you. Yeah, it's not very interactive, but it's super memorable. It's funny. I'm playing through Borderlands to right now. And so like one of the yeah, go ahead. No, go ahead. I want to hear it.  Like have the app on your phone, even take that a step further, right? And while you're playing, you get text on your phone from and you got to respond and it shows up in game, right? Right. But we just hit the actually wouldn't be hard to do an anniversary of the game Firewatch. And those that haven't played, this is a game where the other person that you interact with the NPC that you interact with is only via radio. You never see them. You never actually like, oh, this is like the radio for a stranger game, right? Yeah, yeah, exactly where you're literally on Firewatch. But you know, that's that's a perfect candidate for a game where like it's kind of an unconventional method of interaction. But imagine if it was literally you and the radio just talking and this person talking back to you guided by a writer who's built that personality and that story, you know, into that interface, right? It's no longer selecting from a list of five options like the dialogue tree we talked about, right? It's you on the radio going through this story experiencing this world. Well, and you and you get to use static and bad communication and bad connection as the design trope for like, I didn't understand what you said. Yeah, wait, can you say that again? You're breaking up. Yeah, if they get if they start going towards an area that you just like not prepared for any intense for just yeah the radio. Alex, you've been super quiet and I'm really interested to hear what you think about all of this. Not to put you on the spot or anything, but I'm putting you on the spot. I have to step away for a second. Hang on. Keep going, everyone. Yeah, I was really just listening. So as far as in PCs, I would say we actually talked about this earlier. We were talking about on cyberpunk. Have you? Did you guys play cyberpunk? Still working on it. Oh, again, for sure. Well, Jackie and that one is probably one of my all time favorites. I don't play a lot, a super lot of RPGs. So I really just know, you know, I usually look at it as they're just there to advance the story. So that's why I really love this conversation. It's usually just a means to advance the story really rather than just being in depth with them and actually having them being your friend and things, you know, actually having a conversation with them. So yeah, I see a lot of value being added there. Yeah, be able to bring up some of that personality and interactivity, right? Like Jackie's, you know, big thing is like his physical presence, right? But that's almost it. Like you get your dialogue choices and imagine if you could, you know, if you were curious, right? I want to know more about you and I want to get more of that story, not just what's being handed to you and the writers, but allowing you the agency to discover all of that background yourself as a player. Yeah, and it really makes the game memorable from that standpoint, because like when Doug was saying in the earlier conversation, it's really just at a certain point, you just like, okay, forget the story.  So but it brings people in and gets them way more engaged.  And I was like, Hey, what if, you know, you did have a friend going with you and they actually learned about you know, and that's why I was like, oh, it might get scary, scary immersive if they actually learned about you. And after a certain point, you like, like, are you a real person? Like we're in VR, like Yeah. And like what Alex said about NPCs being kind of a means to an end in video games, feeling like that right now, like, that's exactly why I wanted to have this conversation is, is you get to a point in a video game, like where you start to, you start to see behind like the curtain of how these NPCs work. And the second I find myself spamming my way through a dialogue system is when I know I'm like, oh man. So I just want that to not be a thing anymore. So one of you in the audience make this point. As someone who replays a lot of games, I could tell you that like, you know, listening to NPCs talk the first time, I'm usually like, yeah, you know, I want to know, I want to, I want to hit all the dialogue options. But, you know, second time playing through just get me out of here. So this was this is a really incredible session. I had a really great time listening in and also conversing. If there are any, oh, actually wait, no, I'm sorry, Derek, I don't think we got to hear Derek or Derek, I don't think we got to hear from you guys any memorable NPCs that you guys love. Can you go ahead? Yeah, my video appears to have died and my web conference tool is totally frozen, but it looks like you can still hear me. Man, I don't play a ton of role playing type games with NPCs, but I would say, you know, in general, like most recently, I've been playing through Borderlands too. And so, you know, all those, for the most part, those NPCs are pretty good. But again, like the ones that aren't are you have that classical problem of everybody you encounter that's not a real NPC is one of those like, huh, you know, like they just have no function in the game. So that would be pretty cool. Interestingly, I do a lot more like sim racing type stuff. And one of them has an NPC, which is your crew chief.  It's okay. But, you know, the dialogue is pretty limited right now. And there is a like a mod for one of the games called crew chief differently, where you can talk to it, speak to text, and it understands like, you know, that you want to pit or, you know, what was my last lap time again. And so, you know, I see applications for that to make it a lot more, you know, interactive and immersive. Because like even after the race, like in the garage talking to the engineer, like, oh, the car was like doing this, and the engineer can make suggestion. Oh, well, if it's doing that, we should change this on the car. Project cars has a little bit of that.  So I think it'd be cool to for that to be more interactive.  I think that's like technically what they are on the passenger side.  But I hate this drift, super crazy. Yeah, for well, for rally games, you know, that's a thing, although generally the navigator has like a DSL basically, like a very specific domain language that they use to give the driver instructions about what's coming up. So it's like, well, it'd be cool for that to be somewhat AI inspired in that particular domain. It's actually like, you actually do want the super like no variation kind of like you're always going to get the same instruction the same way kind of thing. But yeah, I mean, there's there's tons of applications in all kinds of things like in cyberpunk, right?  So what if you know, you're in the car and you're like chatting with the person who's in the car, or you can call somebody on the phone in that game and be like, yo, like, I'm going to be at this place, meet me there. And NPCs can actually be more interactive and respond to what you're telling them you're doing, right? Like, well, we want to increase the immersion. Most people play with a headset anyway. So how hard would it be to just have, you know, open mic and just figure things out, right? Yeah. And will you say that? That makes me wonder like, how compelling, how compelling is the interactive like voice to chat thing? Because a lot of a lot of what makes cyberpunk such a good game as far as the NPC, because I know at the end of the day, the system is just, you know, press one or the other. But a lot of what makes Jackie my favorite character is actually like the voice acting and how like real he feels.  But a simplistic person. Yeah. I think really, it's the technologies there. It's the investment in the time and effort to build out those dialogue trees in that depth of, and that depth of stuff, if you will. You know, as we saw with what Duncan put together, like it's not picking on you like it's shallow only in the sense that like there's only one response to most intense and questions or whatever. But you know, somebody who actually a good actor or a good writer can put themselves in the mind of the character and construct the way this person talks, right? Like some of the best acting that's out there ends up being the things that get totally ad-lib, like in the heat of the moment, because that person in like is the character. And so from the creator of the dialogue perspective to make those characters feel realer, it's just sitting sitting in front of the interface for longer, thinking about quippy responses and other intents and and sort of exploring back information that can be added in a way that's meaningful and interactive and all this other stuff. So I think from a storytelling perspective, like the limit is the storyteller, not the technology right now.  She's absolutely incredible, played by the wonderful Merle Dandridge, and you kind of become best friends with her over the course of that story. She's just an amazingly powerful person, and so much of that is the physical acting in the game, but also the voice acting and the way that she responds a lot to what you do in the game. But when it comes to like, yeah, delivering that performance, and, you know, what about a character that's not a human, right? And that made me think of BT 7274 from Titanfall 2. This is a character where it there's a love story between the main character, Jack Cooper, and this robot. And it's it's literally robot. It's voiced by Glenn Steinbaum, but it is a robot character. The intonation is all very flat. There's not a lot of, you know, variance to it. And his performance is something that might actually be almost like amplified by having this like Watson powered system behind it, right? Where you have, you know, Matt Mercer and Glenn Steinbaum, and they have this love story, but one of them is a robot, 100% the performance is a robot that'll leverage the robot. All of the post effects, it's a robot, right? So this is this is like a scenario where you really could fall in love with a character and have this intense, you know, video game relationship with a character delivered by a computer. It's totally possible. Speaking of another computerized, like, NPC in a game, one of the most notable antagonists of all time is GLaDOS from Portal. Voice by Ellen McLean, yeah. Valve. That was good. Valve. Please. Yeah. Yeah. Portal 3. Here we go. Portal 3. GLaDOS that I can talk to. Yeah. Just saying, just an idea. Just saying. I'd buy it. I'd buy it, everyone. Well, we are, we are at the top of the hour. It has been a solid two hours. This has been super fun, super interesting, super awesome. I hope our viewers had as much fun as we did, because we definitely all had fun. Thank you, Alex. Thank you, Duncan. Thank you, Christian. Thank you, Derek. Thank you, myself, for participating today. Maybe we'll do more of these in the future with other sort of Watson and interesting IBM cloud services. But thanks much. Cheers. Bye. Bye, everyone. Thanks, everyone.