Welcome back everybody. We're going to get going. Next up we have Mike Reese who is one of the locals here and the organizer of the Utah Valley Meetup because after Brandon bailed and went to Austin, I didn't want to do it again, so we made Michael do it. But Michael is a good friend of mine. He lives in my neighborhood. We go to church together. We're home teaching canyons. It's pretty awesome. It's pretty epic. So I'm super, no nepotism involved at all on Michael speaking. But actually it was kind of a, Michael was our last, our ultimate. And then it was quickly, oh, can you do the full 45 minute give a whole session? And he's like, sure, I have no stress at work. It's totally fine. So, Mike, I'm really looking forward to this. This is going to be fun. It's all about robots and junk like that. So, all right. All right. It's always good to be known as the bee string. Try and carry that tradition on. So as Mike talked about, I today really just want to talk about robots and fun stuff. I really hope that no one gets anything useful out of this presentation at all. You shouldn't be able to tell your boss about anything you learn during the next 45 minutes. And I'll make that as true as I can. So, yeah, stuff like this. There's going to be lots of this. Quick introduction. Like Mike said, I'm Mike Reese. I go by MMM Reese pretty much everywhere that I happen to be on the internet. I work at MX. But my real calling in life is making babies that have ridiculously good hair. I don't have any stickers, but if you want to come swap some pictures of babies, just be prepared to be embarrassed because my babies are really good looking. So, this is Winston. It takes after me. Thank goodness one of them did. So, I'm really happy to be here at the last Mountain West RubyConf. I've gotten to speak here once before. And it is a bittersweet thing as it is for many of us here for me. I was remembering the other day, the first time I attended a URug meetup. I was still a student at BYU at the time. And I remember like the week before, I had just gotten Ubuntu running on my ThinkPad laptop. And I thought that I would be one of the cool kids when I showed up at a URug event with the Ubuntu laptop. And instead, I got there and everyone had a Mac. And I felt so insecure about my hardware. And I didn't know it, but I was just a hipster ahead of my time. Now it's like all about overthrowing our corporate overlords and using Ubuntu would be cool again now. But I was too ahead, too far. And specifically within this URug universe, I really want to call out Mike. As he said, we have had plenty of time together. I've never gotten to cry into his shoulder like Brandon mentioned, but I'm actively looking for a reason to start crying if anyone wants to suggest one. And some of you may not know, but Mike is a secret millennial.  Or perhaps Mike would prefer the metaphor of a group of ponies hiding underneath the trench coat. If you don't know, this is actually why Mike really loves to troll everyone so much, because he's actually just a kid. And so he likes to make fun of all the rest of us adults for the adult things that we do. So that's pretty much all you need to know about him. And also I wanted to call it really quick. I don't know if anyone else noticed this, but Tenderlove at this conference has been looking really buttoned up. He's looking almost enterprising, dare I say. Like wearing a jacket and a button up shirt. I was thinking that maybe we'd get the weird hat Tenderlove. Or perhaps the wizard in a bathrobe Tenderlove. Or maybe colonial Tenderlove. Or I was really, really hoping to get the hands of Unabomber Tenderlove. And none of those things really panned out. And so it made me think he sort of looks kind of like a professor. And so I'm going to propose that we call him for the remainder of this last Mount West RubyConf. He should be known as Professorlove. And that was my best attempt at an emoji representation of that name. All right. So as I talked about the beginning, when I was thinking about this talk, and when Mike was like, hey, can you do a 20 minute talk? Hey, just kidding. Can you do a 30 minute talk? Hey, just kidding. Can you do a 45 minute talk? I was thinking about the useful and the fun aspects of what I wanted to talk about.  And this mattered to me because similar to one of our earlier talks from Jameson, I've gone through some ups and downs in my career of times when I get home at night and I play hard with my kids. But as soon as they're asleep, I'm just like, boom, got to get on that GitHub, got to get some PRs going, pad those stats, and I'll just be really into it. And then a month will go by and really all I want to do is like watch reruns of psych. And I've been on that roller coaster. And recently, one of the things that has for me been a really good way to reconnect with the fun of programming is robots. A lot of people I know, they got into programming because they wanted to make video games. And that's a really fun experiment. But for me, I've always wanted to make robots. That's always what it was about for me. I wanted to make robots that did cool things like bust out some sweet dance moves. I wanted to make robots that would fight each other and stand up really awkwardly. I wanted to make robots that could make some sweet jams. Actually, I have to just comment like this robot in the top right here, what? I don't understand his job in this robot band because he's not playing any instruments. I thought maybe he's a backup dancer, but it doesn't move. It just has giant saw blades on its head. Maybe that's all you need to do to be in a robot band. It seems like this might be the nickel back of robot bands. It'd also be cool if I could make a robot that would just raise its arms but somehow look really scary and intimidating in the process of doing that. And finally, if one day I could make a chef bot and it would make some sweet farm to table health Oreos very finely diced and really be dedicated to that job, it would just bring a lot of happiness into my life. So I'm not there yet. Chef Oreo is not ready quite yet for demo day. But also, as I was preparing this talk and actually just earlier today, I heard from one of the other attendees about a place that's here nearby an area. There's a little museum just down the street called God Hates Robots. A little intimidating. Maybe I should have thought of this before I decided to give my talk on robots. For anyone who's out of town though, actually this other place right here, Ten Angel, super good food, pro tip, go eat dinner there tonight and save room for panna cotta at the end. But yeah, I didn't realize this was like a thing. I really didn't know that. It really throws a wrench into my understanding of things. And I'm feeling pretty intimidated. So hopefully not very many of you have similar feelings about robots in this crowd. So the robot that I set out to build, the main goal that I had for my robot was I wanted it to be a robot that was built to interact with people. So a lot of the robots that I had experimented with in the past, they tried to accomplish some sort of practical job like vacuuming a floor or carrying something from one place to another. And they sort of tried to do that with the minimal, minimal amount of interference from humans as possible. And this time that's not the robot I wanted. I wanted a robot sort of that would feel like R2-D2. It would sort of like when it ran into a good guy, it would sort of make happy beeps at them and maybe throw a lightsaber out of its head. And if it met a bad person, it would somehow know that and kind of like back up and make like little nervous beeps or something like that. This is also not totally in the realm of feasibility along with Oreo Bot yet. But this was what I was setting out to do. So I began by asking myself the question, as a Rubyist, what are the skills that I can fall back on? What are the skills that we as a community have acquired over our history? So right off the bat, Rubyists are really good at taking things and gluing them together. We may not understand how all the pieces actually work. We may not be worrying about the failure scenarios that they introduce. We just glue them together and make mash up websites. And this is something we're really good at. So that seemed like a good place to start. And so for me, what this meant is I didn't want to get bogged down in the details of like, how many things am I going to have to order off of SparkFun and Adafruit before I actually have something that moves? I didn't want to spend a whole lot of time buying resistors and capacitors and doing soldering. So I found this amazing platform, and I highly recommend it. If you are someone who wants to play with robots, the company that makes Roombas, they sell this project called iRobot Create. They're on the second version of it. As you can see, it comes with batteries included, motors already set up. It has infrared sensors, bump sensors, cliff sensors, wheel encoders, buttons. It even can tell you the charging state of the battery and how full it is. This is a whole bunch of stuff that if you tried to build from scratch, it just takes several months of learning. And that wasn't the part of the project I wanted to work on. I wanted to work on its behavior. So this was a good place to get started. And also, I didn't really want to go learn more C. C is a great language, and I'm sure it would do a magnificent job of running this robot. But it would just be a lot of new tools and a lot of new gotchas. So a great thing about projects like Raspberry Pi is that it has an SSH daemon that runs when you first boot it up. I know how to do that. I know how to SSH to things. I know how Linux processes work and how they can communicate. This just does all of those things. It's sort of like having your own mini cloud on a card. So that is loads and loads of fun and saves an enormous amount of time. And then I thought some more about what it is that Rubyists are really good at. This is probably a good time to note that I decided to open a GIF consultancy. If you need the next great GIF for your presentation, I'm available for hire immediately. And so since we're very good at grabbing snippets of code off a stack overflow and mindlessly copying and pasting them into our projects, seemed like a good place to go as well. So I went to RubyGems and one of the really hard things about a robot like this is having the concept of people. Like, how does a robot know if there's a person around? That's a hard problem to solve. There's a great set of libraries, collectively known as OpenCV, that do a whole bunch of computer vision sorts of things. But it's written in C for performance reasons and for historical reasons. And I don't want to go and learn all of computer vision theory. And so I don't have to. I can use spyglass. Thank goodness for Andre Maderos and the amazing Ruby community that we have. As an example here, this is an example pulled from their examples directory on the GitHub page for this project. And you can see that basically we create a new video capture and then we use the reverse shovel operator. I don't know if there's like a Devron instead of a Chevron name for this. But basically pulling frames out of the video capture device into a variable. And then you can pass them into a cascade classifier. Don't know what that thing is. Don't understand how it works. I'm a Rubyist. And so I just steal this code. And it does work. And it does things. And that's good. So another thing that Rubyists are good at. Apparently sitting awkwardly on a cloud. So I really don't understand what this little blue person is doing. But it seems somehow inappropriate. And so anyway I thought, okay, this will help me to look for faces. But I have no way of knowing which faces are the Jedi and which faces are the Empire. And I needed some way to be able to differentiate the faces that I was seeing. And luckily Google Cloud Vision API got announced during the prep for this talk. And thank goodness that it did. That's that's why I was able to extend it. Good job. Thank you very much Mike Moore. I will steal more of your code shamelessly on stage. So this is a really fun API to play around with. If you haven't played around with it yet, it'll do OCR to pull text out of images for you. It'll also look for taggable entities. So it'll notice sailboats and dogs and kittens. Probably dinosaurs but I haven't tested it yet. Aja, she probably already knows. That does dinosaurs according to Aja that's good. And it also does facial analysis. So it does like a sentiment analysis of the faces it sees. So I still can't tell the difference between a stormtrooper who's really happy that they're about to destroy my robot and a Jedi who's really happy because they understand the force. But as long as you're happy, I know generally some way that I can respond to that. So next thing that Rubyists are very good at. Another skill that I can leverage is having public opinions. So just as a hypothetical example, some people in this room, we might have opinions about the difference between mini-test and R-spec. Hypothetically speaking. And some of us might also have opinions about things like RVM versus RBM or Vim versus Emacs versus other editors. This seems to be something that we're very good at as a community. So I thought, you know, beeping and sort of moving around, those are really great intuitive ways to have an interaction with the robot. But it would be even better if my robot was more like a Rubyist. So it is. It has a Twitter account. And it will tweet its somewhat trolling opinions of the people that it sees. And just a minor note here. I do have a good friend named Chris in the audience who I'm pretty sure if I ever posted a picture of him on any sort of social media, it would have to be signed with a GPG key before. And so it will only tweet pictures of faces that are very close to the camera. So if it scans over the room it's not going to be like magically getting all your faces. The webcam that it uses is terrible and can't actually see almost anything beyond a few feet. But if you get real close like Tyler did, then you get a pretty sweet picture with some sort of opinion from a robot posted about you publicly. And that makes you even more of a Rubyist. So the current version of FriendlyBot looks something like this.  Because it is. So the basics here you can see the Raspberry Pi is riding on back. It's sitting on top of an external battery. It's totally possible to wire into the battery of the Pi. But then you have to do all sorts of level shifting with your power supply. And you have to worry about power surges. And I don't want to worry about all that stuff so I just buy another battery instead. It has a really bad webcam like I mentioned on the front of it. And at this point it's probably worth just pausing for a second before my inevitable failure of a demo. And pointing out that I just talked about like six small things and none of them were super complex. And none of them were super in depth. And all of them were a complete blast. The first time that I actually sent a binary command to this Roomba was at about two in the morning when all of my kids were asleep. And I didn't know what the unit of measure was for the speed parameter. And so I accidentally told it to drive half a meter per second. And it goes tearing across the kitchen floor and rips itself out of the plugs that I had wired into the back of it. So I can't send it the stop command and it's running into things and I'm positive that it's going to wake up everyone in my whole family. And when I finally wrestled it to the ground and unscrewed its battery to turn it off, I had this feeling of pure joy. A little bit different than debugging a production system. But a similar feeling nonetheless. So since this presentation at this point really can't get any worse, I figure it's time for the demo. Let me just do a quick recap of what needs to work in order for this demo to go correctly. A Raspberry Pi has to boot without its SD card becoming randomly weirdo and not booting anymore. So like 10% chance that that goes right. It needs to be able to talk over a serial interface at 115,000 BOD to a robot which is hopefully charged and it needs to be able to connect to the Wi-Fi. Thanks guys. And luckily we're using Hotspot. And then if the Cloud Vision API is also up and relatively responsive then it should be able to move and take pictures and if the lighting is okay then it will even get a decent picture and it will then tweet about it. So yes many many things can go wrong and it will probably go something like this. All right so here is its Twitter account. I highly encourage all of you to go follow this Twitter account. Hopefully it will be tweeting here in just a minute. Yes grounding grounding grounding. And dear robot please work for the love. Like that. All right I'm going to have it come this way so the light's on my face because the low light is the hardest thing for the camera. MWRC bot. Thank you that extra light will actually help a lot. So it takes about a minute here for it to get all started up. It has to start a Ruby VM, an Elixir virtual machine and a few other things and boot up the camera and there it goes and so if I get close it might play a sound. Try again.  You can do it. Did the sound. Random shouting this is not a republican debate. What? Someone please help me find my robot later. All right all right Mike Moore is going to go rescue friendly bot. All right that went way less terrible than I expected. And as a result I have a little bit of time that I can talk through the code of how this thing actually works. I promise not to go into too much detail but we'll just kind of roll through this stuff. One more time. There you go. All right so the first thing like I mentioned that I had to do to get this thing to start moving around was send binary commands. There's a really great PDF that the iRobot folks have put out that shows the exact shape of all these commands. It uses standard encodings like like two byte signed integers and things like that. And it turns out that Elixir and Erlang have a really great syntax for both parsing and generating that kind of data. And so I decided to again it's fun so I just wrote in another language and Rumebex was born and you can use that if you want to. Here's an example of what the code looks like so this is parsing I get back a single byte that represents the state of all the buttons on the top of the Rumba. There happen to be eight of them and so I can just say hey bring in this one byte and pattern match it and turn byte number zero into clock and byte number one into schedule and just by setting the sizes it'll figure out how that lines up and it'll end up giving me back the state of all my buttons as zeros and ones. So that was a lot of fun. Then I needed something that was kind of stateful that could up of the Rumba. There happened to be eight of a robot and remember am I supposed to be driving right now or am I supposed to be stopping so I can hopefully get a half decent focused picture and I named it DJ of course after our dear departed friend DJ Rumba and so DJ contains the majority of the code in this project. An example of what DJ does is it receives updates about the sensors it's checking the sensors 30 times a second and as it's getting in the sensors it just does pattern matching to look for common scenarios of this this bump sensor being pressed or released and things like that and then changes its driving command and in that way it'll just kind of shuffle itself slowly around it's somewhat painfully slow but hopefully that makes it easy for it to find faces in the crowd after my talk. Then what it needs to do is I also wanted to do as I mentioned all the open CV and the computer vision stuff was going to be done in Ruby so I did all of that using Ruby but I needed to talk between the two because when the Ruby program sees a face it needs to say whoa whoa stop for a second because this webcam is terrible and I need to be able to like get a half decent frame of someone's face so I can try to analyze it and so I did this with just UDP and so here I'm just listening for some little UDP messages like stop go seeing and cancel and then I needed the code that would look for stuff and decide when to move so here's here's a great piece of code that you can share with your team global variables all over the place and also in true Ruby is fashion you know I figured now's a good time to declare that local variables are dead it's not something we should be doing anymore long-lived globals if no one ever needs to read or understand your code global variables are a lot of fun so here's the main loop of all the Ruby code does over and over it just basically tells the wanderer to go that's publishing the little UDP packet then it looks for a face this is a blocking call that's just sitting and sampling frames it's able to process about 10 frames a second only using about half of one of the cores on the Raspberry Pi which is better than I expected then if it finds a face or thinks it sees a good face in the image it'll stop and then it will try to pick a good frame and this is what this one is just doing a more selective looking for somewhat more focused and looking for a little bit of a bigger face in the frame and if it finds one that matches those criteria then it sings a little song and kicks off some asynchronous processing to do the analysis because I don't want to wait for the internet to be available before I continue moving on and looking for new people if it doesn't find something then it plays a sad song some sad beats you guys will probably hear those later in the lobby and an example once again this is pretty much copying and pasted as I mentioned from the spyglass project and thank goodness for the good work of Andrei Medeiros and then the analysis basically looks like I pass it to this analyzer object the analyzer sends it to the Google cloud and gets back a result it also checks for some additional blurriness because my local check for blurriness is not very good and if that all passes then it decides to actually send it to Twitter and it picks a pseudo random sort of troll to send along with the image so we talked about and I've spent most of this presentation sort of trolling our community the things that we're good at are stealing people's code using things without understanding them etc etc but there are a few things that I actually think our community is totally outstanding at and a few examples of those are things like the friday hug celebrating our heroes why the lucky stiff has already been mentioned here but these are people who emphasized whimsy and playfulness and joy rather than things like technical excellence and being smarter than other people in the room things like ruby friends an entire project and website whose only goal is to try to make groups of rubious a little less socially awkward so we can find a way to take pictures with one another and hopefully friendly bot can be another another project in that vein that helps rubious have a reason to talk to one another at a conference and finally this idea of minus one tend to love did not give us enough puns so this is a mini swan but that's not what minus one about is about its mats is nice and so we are nice and this is one of the things that drew me to the ruby community very early on this is a community of people who believes in being welcoming to people who have been neckbearding for 30 years and people who are in the middle of a boot camp and just learning to program it's a community of people who believe that it's important to be nice and to be kind and to me that that is why i'm here that's why i continue to be a part of this community um i think mountain west has done an amazing job of embodying those values over the last 10 years and certainly my own experience um you know i went from being the person who was so intimidated to be bringing a pc to a mac party at my first u-rug to being someone who was uh essentially coerced to giving a talk at a u-rug and uh and then coerced some more times and then eventually coerced into giving a talk at mountain west ruby conf and all along the way i've been surrounded by good friends and good people who were kind and forgiving of me and of the times that i made mistakes and i hope that whatever is in our future as a community uh whatever meetups and conferences we do this next year in the years after i hope that that remains at the core of what this community is and what we choose to do with our time in effort um and that's all i have thank you very much i shouldn't repeat it because it's a troll but i should because it needs to be repeated for future humanity the question was did i get a lot of design input on the robot or was it designed in a vacuum um thank you for bringing it in stronger and all right any other questions yeah um so the question was did i look at the things like the r2 framework so um there's a really good robot like robot related framework called r2 for ruby um it's made by the same people who make cylon j s and go for bot i believe is the one and go um it's a really cool framework um i did look at it but i kind of um despite the fact that i poked a lot of fun at not wanting to get into the resistors and capacitors in that part of the whole problem i did like the idea of getting into like what are these bits and bytes look like as they fly over the wire and so i was kind of excited about writing that part of the code and since it it fit my fancy i just figured i'd write it myself oh thank you oh there should have been another one earlier too here we go here we go look at its legs it's still trying all right fall all right so the question is there is limited times we have families we have other things how do i try to time box or find time for doing these kinds of projects i'd say that i'm in a big way a victim of my own interests i'm somewhat like ajah's talk about feeling like a ferret um this just happened to hit me at a time when i looked around and i had always wanted to build a robot but i in previous times when i had looked around i just the the raw materials that i had to start from were at two layers of distraction lower and i when i looked at it i thought oh that's that's a thousand-hour project and there's no way i'm going to get through all of it this was a time when i looked at it and i thought i can start from this Roomba thing i can start from raspberry pi i think all said and done it's this is probably about 90 to 100 hours of like coding time and debugging and playing around with it it's still a significant effort the only real way that i organized that is once a week me and my wife plan out our week and she tells me what nights she's going to go like to dinner with a friend or go to a pottery class and i tell her which nights are programming nights uh and that just sets the expectation that like once the kids are in bed i'm probably going to be a terrible husband until the morning so uh we just we just plan it out and that way we we know what the expectations are that's that's all i got oh here we go this one's really good that you got to wait for this one there's a little bit of a buildup clear shot samba samba samba and then so good the little the hip shake is what gets me so the question was about the image quality it's just because i'm a total cheapskate so i i bought like an $18 webcam off of amazon if it was using something even halfway decent i wouldn't have nearly as much problem with white balance and being able to focus quickly and things like that no yeah no it wasn't an issue at the operating system level or the raspberry pie level it was just with the hardware of the camera that i was using yeah so actually works a lot better if you if you stick the laptop on top of it it just looks a lot less friendly oh gif gif thank you oh man this one's good simple but it's good i don't i wish we could see the buildup to this because i don't know what this challenge is but clearly it failed brandon did you have something um i i'm we take strong bet that in this wi-fi it would be limited by your bandwidth uh when i tested it from a good internet location it was still taking around like a second and a half um you're you are still sending an entire image which is a lot bigger than like a normal json payload um and the analysis it's doing is much more complicated than just checking equality or things like that so it's still it's i mean it doesn't take a long time but i knew that i was going to be in a somewhat questionable wi-fi environment when i demoed so i planned on the fact that sometimes those asynchronous jobs they just fail and so that's fine just let them fail and leave an image sitting around on the raspberry pi um so the question is that i look into running the ruby code sort of bare metal without an operating system so that'd be something like a microkernel or something like that is what you're imagining um i didn't really look into that like i said i was uh this was definitely just a case of i know i'm not going to get into all the details so i'll just pick and choose which details i find interesting uh so i i just didn't really play around with any sort of tuning or or running micro kernels no oh do i have another gif you guys are asking a lot of questions oh man i'm out hold on we'll just go back to this one this one is so good all right enjoy i'll i'll just keep answering questions but you guys can just you guys can just enjoy that um so what are my plans for the future with friendly bot um a few things i definitely am gonna let it have some time just roaming around my house and scaring the heck out of my little baby like little baby winston you saw at the beginning uh he's not a big fan of friendly bot um but this this is how he gets to know like you know until they customize it's just they're gonna feel that way so i'm probably gonna scare my children a little bit uh i would i would actually really like to um add on the ability to recognize faces when it sees the same face many many times um and i would also really like to get to the point where i have some sort of the locomotion that i can get up and down stairs um and i thought it'd be cool if uh it's sort of logged activity of like who spends the most amount of time in various parts of the house i don't know why that metric appeals to me um i'm a random human being but for some reason that metric does like i'm curious do we even use that one bedroom in the corner of the upstairs and uh if it noticed when it saw faces heading in and out of that area then maybe i would have a metric for it instead of just telling my wife no we shouldn't buy a chair for that bedroom my favorite show featuring a giant robot um i don't know i feel like i'm really failing you by not having an answer to that question but i don't really watch any shows with giant robots i don't know ask mike more i know he watches really terrible movies like transformers and pacific rims so i'm sure he'll have an opinion does anyone else want to ask a question that i can make fun of mike more as part of my answer all right well um friendly bot will be hanging out over here by the table in this little lounge area like i said it's not going to like creep on people or try to like stalk you it's just going to be randomly roaming around and if you happen to crouch down by it it'll take a picture of you and tweet about it and then you can retweet it to all of your friends uh or maybe get another rubius to take a picture with you you guys get like you know maybe we can even we can get some synergy with erin paterson's selfie stick technology and put it on top of the room but possibilities are endless but it will be roaming around out there and if you're interested in robots come and talk to me thanks everyone


Welcome to the World Economic Forum's AI Governance Summit here in San Francisco, and also to all of those joining via live stream. If you are following along and want to share on social media, please use the hashtag AI23. I'm Ina Fried, Chief Technology Correspondent at Axios. It's my pleasure to welcome everyone for what I know is going to be a great discussion. As everyone is keenly aware, it's been about a year since chatGPT changed all of our lives. But I think we're all still trying to figure out what it means, what it's going to mean, and I'm thrilled to be joined by an eclectic panel, as was discussed backstage. But I think people that will give you a few very different glimpses. I think, again, we're all trying to figure out what does it mean for me? What does it mean for my job? What does it mean for my community? What does it mean for my culture? And I think we have a lot to dig into. Before we begin the discussion part, I'd like to invite Jeremy Juergens, the managing director and head of the Forum's Center for the Fourth Industrial Revolution, to deliver a few remarks, and then we'll kick off the discussion. Mr. Juergens. Thank you, Ida. So thank you, everybody, for joining us here. We have over 250 people assembled for the AI Governance Summit. This is the first time that the World Economic Forum actually convenes a group and a summit around a specific initiative here. And in this case, it's our AI Governance Alliance. We established this earlier this year following a convening of over 100 top leading AI experts in April of this year. And out of those discussions, we understood how quickly the technology was moving, the dramatic effect that it would have on the economy, on society, on livelihoods, and how even among the world's top experts, how little it was understood both how we might actually guide or even control and put in place the necessary safety mechanisms, as well as how we would govern and take into account that we balance the risks along with the opportunities and what it would take to fully harness and unlock the innovation benefits that might emerge there. On the basis of those discussions, we published the Presidio Principles. This, again, attracted attention from a number of not only business leaders but also policymakers, civil society groups, and academics. And with that, we established the AI Governance Alliance. So today is another step in this journey of helping us understand and go in depth on each of the different domains we had here. So a lot of discussions looking at safety, what is the role? What are the technical pieces that we need to put in place there? Considering governance, we know that we've had discussions both with this announcement out of the US recently, out of Bletchley Park in the UK, the Hiroshima process, and other groups. So a lot of different jurisdictions are navigating this governance framework there. And importantly, also, how do we not lose sight of the opportunities that it does present? And not only in developed countries, the US is in the lead. China has a very clear role. But also, how do we make sure that those benefits extend to developing countries and even other developed countries that may today not actually have the capacity to put their own models in place to do the necessary training? So all of these topics we'll be covering over the next two days. And with that, I'd like to thank all of you for joining us in these discussions, for the contributions that you'll have both here today, as well as in more depth, the technical discussions tomorrow. And I look forward to navigating this new era together. Thank you. Well, thanks. I mean, I think you hit on probably the three most important conversations that I think we're all trying to have simultaneously. How do we harness this incredibly powerful, incredibly fast-moving technology? How do we recognize its limitations, its risks, its biases? And then, how do we reshape ourselves as humans? It's sort of ironic that the technology that we're able to most easily communicate with, the technology that finally speaks our language, may require more change than learning any programming language we ever had to learn. I do want to tap into the expertise of each of the panelists here. Maybe we'll start with Minister Ghan. Minister Ghan, talk about how you're thinking of it from a public sector perspective. What does it mean? Singapore's obviously been on the leading edge in a lot of ways of technology and business. How are you looking at what we've all witnessed over the last year? Thank you very much. First, let me thank all of you for inviting me to this forum. Singapore has a very unique situation, because partly because we have a very scarce human capital. And therefore, AI actually promises to allow us to augment our human capital. And it also promises to solve many, seems to be able to solve many of our challenges of our time. But AI at the same time also brings about new risks that we will have to manage. Its capability of generating content at speed and at scale will create new risk elements, including, for example, the possibly the higher risk of misinformation or disinformation with fake news. And it can also be used by elements that have ill intent. It is also an area that we will need to think about how we can encourage the appropriate use of this technology. But if we use it properly, it has the ability to uplift our economies, our societies, and also empower our workforce to do even better. For Singapore, I think our experience is that we want to embrace AI. And Singapore is one of the first nation to have rollout and national AI strategy. And this was done in 2019, and we continue to invest in building our AI innovation capacity. But with the advent of generative AI, it's even a more exciting era that we can think of doing things that are seemingly impossible in the past. Now we are enabled by generative AI. But it also requires us to pay specific attention in the governance of AI development. Let me share Singapore's experience in three particular areas. First, I think to fully embrace the potential of generative AI, we will need to focus on preparing our businesses and our people. This is very important because it will then minimize the risk of widening a digital divide. So we have to help our businesses transform. Equip them with digital tools and at the same time share with them, provide them with the necessary support to harness the potential of generative AI. By the same time, we also need to make sure that our workers are able to harness the skills they are needed to be able to do well in taking advantage of generative AI. So based on the recent reports on the future of work, Singapore workers is found to be one of the fastest in the world to adopt AI and at the same time to be able to pick up the skills they are necessary to do well in the implementation of AI. So to prepare the businesses and people is very critical for us to be able to harness AI. Secondly, because of the speed at which AI is developing and evolving, we will also need to adopt a nimble and flexible and practical approach in the governing of AI development. One size fit all the rules and regulations is going to be very difficult because by the time we roll out the rules and regulations, technology has moved on. So it's important for us to take a different approach, have a nimble and practical way. And that is why Singapore has also rolled out an initiative called AI Verify. This is a testing tool to allow us, the developers and the owners of AI system to test, to demonstrate its governance on their AI systems. So these are ways that we need to continue to evolve and to adapt to the new environment. So thirdly, it is also important for us to come together collectively through a multilateral collaborative platform. Platforms like this allow us to share our views, our experiences so that we can collectively come together and address the challenges of AI and tap on the opportunities of AI. At the end of the day, I think it's important for us to at the same time, while we recognize the potential and benefits of a generative AI, we must also recognize the risks involved and it will be important for us to come together as a global community to address this issue squarely and to collectively steer the development of the technology so that we are able to advance the technology for the public good and do so in a safe and secure way. Thank you, Minister Gan. I hope you'll have more opportunities to share where Singapore sees itself and also some of the projects you're working on. Dr. Kler, I was hoping that since you have this vast knowledge of the technology space and have been working in AI since before all of us realized it was a thing last year, one of the things that struck me is AI is not new. The idea of it, many of the places we wanna go were laid out by researchers, were laid out by science fiction for decades, but the technological way we've gotten where we are today has surprised a lot of people, including a lot of people in the field. What does it mean that it's generative AI that has suddenly come to the forefront versus some of these other technologies? A lot of other AI, machine learning, different things we're trying to analyze, we're trying to predict. This technology, you know, sort of, just, I mean, a lot of people have called it autocomplete on steroids. It's amazing the results we get from this technology, but I assume by its very nature, there's some shortcomings to what it can do given the way it operates. Help explain really quickly how we got here, what does it mean that this is the technology and how do we go forward with it? Yeah, so first of all, thank you for inviting me here. It's a pleasure to be here. Yeah, I'm what you might call an OG AI person. I've been in the space since the early 90s, long before the space was, as the space was coming into being, and I remember chairing an AI conference that was the biggest in the field and it hit 1,000 the year I was the program chair, and we thought that was a big event, so it just shows you how quickly things have evolved. What seems clear when you look at the arc of the technology is that we're living in the midst of what is an exponential transformation, and that's what made it hard, I think, to perceive and to predict the future, because when you think about how an exponential curve go, initially, the time that it takes for a doubling, if you will, of capabilities, whatever those capabilities might be, takes a long time, and so you think you're kind of on an ambling, slow, kind of linear curve, and then it suddenly starts to accelerate and you say like, whoa, wait a minute, what happened? And I think this is where we are now is at this inflection point where we suddenly realize that we have been on this exponential curve all along, it's just that we hadn't noticed. And I think that exponential curve has been enabled by the converges of three tidal waves that have all come together at the same time, one which I think is actually the most important, and I'll come back to that in terms of where things are going, is the availability of data. When I started in the field long time ago, a large data set was a couple of hundred samples, and you felt fortunate to have that. And now, we've been training literally on the web in terms of the text and images and increasingly speech and video and so on, and it's that amount of data that allows the kind of models that we're seeing to be trained. Without that, all of the methodological innovation would not actually amount to anything. I know because we were there at the time. The second, of course, is enabled by the amount of data. Now, more sophisticated, more complicated models can actually be trained, and they become increasingly more powerful. The more data you feed them because of a combination of how the technology is adapted to dealing with the richness of the data. And the third, of course, is compute on tap, which we also didn't used to have way back when. And so, those three have come together to enable the sort of incredible acceleration in the progress that we're seeing. Now, as to why generative AI has been the thing that has taken over, I think there is actually two elements to that answer. The first is just that it's so understandable, so relatable to people.  And I think that's part of what we're seeing, but I think it's only part. The other part is, I think, the somewhat surprising observation, and I think it was a surprise to pretty much anyone, is the extent to which this autocomplete on-steroid task and sort of creating kind of like, almost like a balloon of model around it that's forced to sort of create in a very realistic way what the next word is going to be, how much representation that has forced the model to learn, how much world representation, how much reasoning is required to do that task really, really well. And so, I think that is an important lesson learned, and I think has certainly transcended into how we think about how to solve other problems in terms of the different pieces that need to come together, which is defining a really high value proxy task, like autocomplete combined with enough data to actually make those models learn something that is a meaningful model in a particular domain. And for those of us like myself who work in data-poor domains where you really need to be thinking all the time about where do you get more data or generate, we print data in what I do, how do you generate enough to leverage that sort of incredible sort of combination of those methods? And I think, Dr. Clark, you hit on one of the things that I think is really a key part of this, which is these generative models, what they really are is taking very complicated or what they accomplish is taking very complicated tasks that used to require a specialized form of communication, whatever that was. So for some people that was Photoshop and editing photos, you had to know how to use a very specific tool. In this case now, you can just describe what it is you want. In that similar open AI had its event last week, they announced the ability to build custom GPTs, and I go to developer conferences all the time. It's rare that I go home from them and build anything because in the past, they required knowledge I didn't have. I went home from that developer conference and was able to build a couple really useful tools. And what it reminded me is chat GPT in a sense is the parlor trick. It's the introduction, it's the tutorial, but the power of these models isn't just reasoning against the whole world. It can be incredibly powerful when you're using that reasoning against a specific set of data. For me, it was planning my coverage for next year's Olympics, but for companies, and I wanted to get to use Sebastian because Salesforce has been partly at the forefront of how do businesses make use of this? Businesses obviously bring a different set of concerns. It's one thing if I go home and use my data, I'm the IT manager, I'm the CEO, I'm all of those things, so I get to decide. It's really tricky when we start getting into businesses using their data. Talk about, you know, Salesforce has spent the year rolling out a bunch of tools, a lot of them in sort of very limited feedback, limited programs so you can get a lot of feedback. You know, I've been to all the events where you've announced all the tools. In a lot of the cases, there seems to be a big emphasis on letting these systems generate a lot of information, but not letting them take actions, leaving that to the human. My suspicion, and you're the chief legal officer, so you can probably tell me if I'm right, there's two reasons for that. One, it's a best practice. We don't know these systems that well. It's a best practice, but also, it shifts the liability to the person doing it and not the company providing the software. Talk about sort of where we are. What have you been able to do in this first year? What are some of the concerns you're still grabbing with? From a company that's been at the forefront, where are things right now in terms of businesses being able to make use of these tools on their data? No, sure, thank you and delighted to be here and delighted to be here with all of you. I suppose I'd come at that kind of with a couple of ways. First, and to echo your comments, AI fundamentally is not new, right? And in a way, though, this exponential transformation could not have come at a more important time, right? We think that businesses need AI now more than ever. Governments, constituents, stakeholders need AI more than ever, and the broader society, we think, need it. Now, what do we need? Well, we need the benefits, right? And then it's how do you kind of achieve those benefits taking into account the relevant risks? Of course, look at Salesforce, and we had decided to make a very significant additional investment around AI research, probably around 2014. And then sort of working with our customers really were viewed as we're pioneering the AI era around predictive AI, and then we launched Einstein 2016, and trillions of predictions, right? What's the next best action XYZ? The funny point you're making around, we haven't yet fully deployed what I think you're referring to autonomous, right? Activity autonomous agents. We do see ways with predictive era, generative era, autonomous era, and then we can talk through what we think of the other sort of future areas. I would note, when we were sort of laying out what we called sort of our Einstein GPT, one concept was around, could we let people build skills? What are the skills, the actions, the capabilities, that a customer, a developer would want to create in using sort of various software, right? Or sort of different technologies, more to come on that front. I think what I would also say if we step back, we think that, look, at Salesforce, and I think many kind of companies and institutions and certainly governments, trust has to come first. For us, our core values, we lead with trust, trust, customer success, innovation, equality, and sustainability. I think the question, we think asking the right questions now will enable us to create the future that we want to have rather than the future that we may end up with, is how do we ensure that a trust first motion, that these concepts of responsibility, of ethics, look, of legality, how do you build them into the use, development, deployment of this, you know, we think is sort of exponential opportunities here in terms of technology, build it in early. We do believe that the, this AI revolution, is it worth, it's a trust revolution, or it needs to be, right? A trust revolution. When we had designed what we call the trust layer, you know, again, listening to our customers, listening to, you know, we thought we're having early views, actually also of regulators and the like. The idea sort of with the trust layer is how do we make this safe and effective, taking into account, you mentioned data. Well, when we think about data, quality of data, integrity of data, security of data, and when we think about risks, mitigating risks, monitoring risks, but also fundamentally understanding, you know, the risks. I think there have been two items that I've been surprised by as we see how this sort of new sort of phase of generative AI has developed. The first, how quickly we all appreciated and embraced that we need a multi-stakeholder lens on this new technology, this opportunity. And I felt it was very quick that we know as we think about how do you partner, right? Sort of academia, right? Civil society, private sector, public sector, human beings, right? Employees, workforce. This idea of needing a multi-stakeholder approach, I was surprised how quickly, again, you know, at Salesforce, it's something we've always thought of the broader Salesforce ecosystem and that business can be the most profound sort of system, you know, and kind of approach to drive positive change. The other element is that when it comes to governments and regulation, because I do believe, by the way, one of the great opportunities for governments will be how do you improve the constituent experience? So certainly AI is going to transform how businesses interact with their customers and in time, how businesses interact with other businesses. But what about, you know, for all of us just as human beings, right? In our communities, you know, and the like, how could you improve the constituent experience using AI and technology? I think that's an incredible opportunity, you know, for the private, for the public sector. And so the other surprise, again, so multi-stakeholder lenses and that that's important and you had to bring in diverse and different sets of voices, but also that we need collaboration between the private sector and the public sector so that we can move really fast but thoughtfully. You know, I do think, you know, kind of when you think of kind of the legal landscape, you know, more broadly and kind of what's the ideal role of law, you know, and the like, look, we need to figure out how do you use AI to accelerate the velocity of wise decision making, not speed for speed sake, right? I think the other concept I'm just gonna sort of introduce here, we're all really excited about talking about what's the kind of, you know, the moonshot, the opportunities that can arise, right? If we sort of get AI right. How is it going to just incredible, right? Diagnoses, right? So for various health conditions, how can it, you know, enable, right? Educational sort of outcomes, you kind of go down the list. They're really exciting ideas for myself and actually what we're also looking at very much at Salesforce, I wanna talk about something that's a little bit more boring, but I think much more important when you think of scale. I wanna talk about the problems of mistakes in society. I wanna talk the problems of how you think about inequalities where people who don't have access to the best, the best of anything it might be. The best healthcare, right? The best technology, the best software. What if we can all focus, and you could look at it with the SDGs, right? Each of those different sort of sets of the SDGs. We believe everyone should, they should be available to everyone, right? What if we use AI to raise the floor? Not just go for the moonshot, of course. Everyone's gonna invest and focus on that. What if we think about raising the floor, reducing the impact and the likelihood of mistakes occurring that have horrible impacts on people that never really get sort of written about or focused about. And just thinking about this idea of raising the floor in every potential venue and in every potential region, I think it's a tremendous opportunity. And just the other thing I would ask, on this issue of accuracy and reliability and ethics and responsibility, one question I have had where we're all very concerned about hallucinations in these models. The area that I have just not heard enough about and I just wonder is where, to what extent are these hallucinations a feature rather than a bug? And what I mean by that is, is there an element to which where we're excited about gender variety, being able to create new content, which then creates a whole host of new ethical issues and the like. We must have accuracy or at least transparency about where something is just flat out wrong. But is the solutionation element part of how the potential sets of technology are able to create and generate sort of new ideas? And how do we deal with that? And that's what at Salesforce, when we build on the trust layer and other items we're trying to grapple with some of those opportunities and challenges too. And I think you raise a lot of the issues that we wanna dive into and we are in a sec, but I think before we get into the risks and the benefits, the other element that I think is important to address and Dr. Kohler sort of talked about this in referring to it as an exponential amount of change is just the speed. And I think we have to take a moment to talk about the speed with which all this is progressing because I would posit that stuff is actually changing faster than many humans are able to adjust let alone institutions. How do we deal with a technology that is moving this fast? I mean, it's AI and generative AI is often compared to other big shifts in technology. We had the computer revolution, we have the internet, we had smartphones, all these were major shifts and changes, but at least in covering each of those technology waves, and I've been fortunate enough to get to cover each of them, the pace of change wasn't actually that great. They were huge shifts, but it took us quite a while to make those shifts and the technology underlying those shifts wasn't actually changing that fast in part because it was hardware. I have been amazed and continue to be amazed over the past year just how fast this is moving. I will talk to a company about where they're going and three months later, the amount of progress you'll see is dramatic. One example I saw at a recent conference was showing a slide of the same prompt in mid-journey a year ago and it looked like something that my parents are Shigal that they might have that was very interesting and artistic, but nothing that any of us would call realistic and then the same prompt today generates something photorealistic, similarly with other types of generative AI. My question, and maybe we'll start with you Jeremy, since you're sort of managing this for the forum, but I'm kind of interested how everyone is dealing with this is are we capable of keeping pace with this rate of change? Because even if we want to get to all the things that Sebastian's talking about, we have to not, you talk about mistakes, we have to not make mistakes as humans in what we allow and don't allow. So I'm curious how all of you are thinking, if anyone disagrees too, that the pace of change isn't that fast, I'd love to hear it, but how do we deal with technology that's changing this fast? Yeah, so I think that's a great question and something we've been focused on quite a bit over the last year since, I think I was playing on mid-journey with my kids and it was before it was kind of mainstream with the chat GPT release. I was like, oh yeah, this is pretty cool, amazing. Don't need the Photoshop there. With the release of chat GPT last fall, all of a sudden it's kind of people woke up and it wasn't that AI was new, right? The AI had been ongoing and it reminded me a bit of how companies approach the digital transformation in their companies. There's a period where every company was trying to go through digital transformation and at some point it became mainstream, they took it for granted, they hired a chief digital officer and then they moved on to other things, ESG, sustainability. In the last year, they've all been focused on AI and so there is this kind of high-level element there but to come back, how do you address the speed and I think not only within organizations but across society as well because I think there's a lot of fear that comes with this. We hear about the fear of killer robots and extreme disasters and so on. Quite exaggerated. I'm actually much more concerned around just exacerbating the imbalances that are already in the system today. Around digital safety, around inclusion, around very simple and basic things that we don't even have to worry about further out down the road there. And to address these, all the most important element that we focus on is inclusion. We talked about multi-stakeholder here and I can give an example of work that we've done in India. We've been looking at how you can use AI for agricultural innovation for smallholder farmers in India. Over half the population works in agriculture in India so you're thinking roughly double the population of the US. 85% of those farmers are managing a little bit less than two hectares so relatively small plot. And a lot of them have limited access, a lot of them don't have phones. So it's like, okay, how do you then leverage the technology for the benefit? You know, one of the first places we started was actually bringing the farmers into the discussion. Through farmers cooperatives, we brought in startups, we brought in large Indian companies that are at the forefront of agricultural production. We also brought in foreign multinationals, we brought in governments, we brought in agronomists, et cetera. Now that discussion took time, right? We spent over a year on that. But in the process, and Sebastian, you mentioned the word trust, the trust actually came from including a much larger ecosystem in the dialogue. So it wasn't just some company coming out and saying, okay, we've got a wonderful solution for you, it'll solve everything. We actually involved the farmers, we involved throughout the value chain all the different elements there. First, yeah. I wanna press you a little on this because on the one hand, the opportunity is incredible. Like I remember again, in the cell phone revolution, just giving subsistence farmers, typically they had to, these decisions of what crop to plant, of when to plant, of which market to go are basically the difference between feeding your family and not. And it made a huge difference when you could send a subsistence farmer in India or in Africa, which market?  If you know which market's gonna offer a higher price, again, that's the difference between feeding your family. So having more data and being able to deliver it in a conversational way where the farmer doesn't have to do much more than ask a question is amazing. But I also wanna challenge you, you were meeting for more than a year. I suspect that technology shifted a ton. So how do you have those long-term conversations? How do you build that trust and not have the result be, here's the best thing we could have done a year ago that has less relevance than you would hope today? Yeah, so this I maybe would disagree a little bit on the speed. The speed in the lab is happening quite quickly. The speed in deployment does take more time. We exist in very complex political systems, complex economic systems. And if you just come back to this question of healthcare, for example, yes, you could get a diagnosis in chat GPT on something. And the question is how many of you want your diagnosis or read from a combination of Reddit data, Twitter data, ex data, so on, versus verified validated trusted sources. So we do have agency in this. We have agency in determining when we decide to use the technology, conscious decisions about when we don't and we delegate that to someone else. And so I think it's important that as we navigate something that's happening very quickly, we move slowly enough that we're actually taking conscious things and not just allowing ourselves to be pushed along through that rush. And we'll actually get more benefits of that. And if I come back to the Indian farmers, what we saw is we ran the first pilot with 7,000 chili farmers in the state of Kerala, Ghana. We saw improvement in yields, time to market, reduction in utilization of fertilizers, which have an expense, especially in the face of the recent energy crisis and increased profitability for each farmer. We're now looking to extend that to other states. And we didn't say, okay, let's now just roll it out to the whole country, work with the agricultural ministry, 600,000 farm cooperatives. We're taking a step-by-step approach. And because the underlying framework is developed in a system that recognizes that the technology will continue to evolve, we'll be able to bring in new developments as they occur. And this is also where I think the start was playing an important role in those discussions because they're often moving much more quickly than the large companies. They don't necessarily have the capacity to immediately scale up their benefits there. So I'm actually optimistic that if we consciously include a larger ecosystem in there, that we work deliberately, that we recognize the agency for the different individuals, that we can harness the benefits even as we mitigate the risks along the way there. Yeah, please. It's a very different perspective. I think technology has great promises. It can solve quite a lot of problems, but in itself it's not a solution. I think it's just an enabler. So for example, AI can provide answers to how to improve the crop yield, what kind of fertilizer is the best for this type of crops, and which time of the year is the best time to plant the seeds and to harvest. So these are technology, and AI can help in finding solutions and answers to that. But to ensure that the farmers have a better quality of life involves many other factors, including socioeconomic issues, geopolitical issues. And you need to also make sure that despite having the best yield in the crops, the logistics and the supply chain has to be in place. So you need to work at different aspects of the problem to be able to solve the issue, to deliver good outcome for the individuals on the ground. So I think AI is a good tool, but it needs to be used in combination with all the other measures in place to be able to deliver outcomes. And you also brought up a point, Jeremy, that I wanted to bring up with the rest of the group, which is, no, I certainly don't want to rely on major medical decisions by typing into chat GPT, but I think that's one of the fundamental misunderstandings a lot of people have, is that what generative AI means is I go type into chat GPT and expect a miracle response, whereas I don't necessarily want that, but I do want the doctor that I go see who went to medical school 10 years ago to have access to a tool that's probably trained on the really good data, not Reddit, not everything that's been said on the internet, but the medical things, and by the way, spoiler alert, if you read my newsletter tomorrow, this will make more sense, but how powerful is the combination, and maybe start with you, Dr. Kohler, since you're in all these fields, both AI and healthcare, how powerful is the combination of this interface, these models with very specialized data, particularly in healthcare? So I think that's a great question, and I think actually that interplay between a person and the computer is something that I think we should be leaning more into, and there was a recent study by Eric Brilliusson that basically showed that you could deploy the technology entirely on its own, you could deploy the technology in partnership with people, both of those are legitimate productivity gains, but when you have the partnership between a computer and a person, you actually have a considerably greater efficiency gain, and I think also are able to avoid some of those issues of loss of trust and hallucinations and some of the other risks that if you just kind of launch of the technology on its own, it just doesn't make, you run into all those risks, and in the field that I'm in, you oftentimes hear people like, oh, we have the first entirely end-to-end AI system for X, Y, and Z, and my question is, A, do you really, what does it even mean, and is that a good thing, even if you did have it? So that's a place where I would, I think, ask questions. Now, to the other aspect of what you asked, I think there is clearly a need to create specialized agents that are trained on the kind of high-quality data, not the Reddit, it's not the X, not whatever, where you need to kind of curate the data, make sure that it's actually representing truth. In some cases, you need to generate data, you need to invest in harmonizing, cleaning your data. I mean, let's just say that garbage in, garbage out was always a thing in machine learning, but I think as you make the AI more powerful, it gets better and better at anchoring on falsehoods and inaccuracies in your data and amplifying them. And so I think making sure that as we create what I would call an app ecosystem on top of whatever generative AI models are out there, we empower those apps with the right kind of high-quality, curated data. And you ask me, I would say the value is going to be split quite nicely between the baseline foundation models that create the core representations, whether it's text or images or whatever, and these verticals that were trained in a very thoughtful way with the right feedback and the right input from humans on high-quality data in a domain-specific way. So you bring up, I would say it's the power of the technology and the limitation, which is it's going to make incredible insights, but based on the data it has. And a lot of these systems have been trained on the whole internet. And I think when we talk about the risks, I think a lot of the risks are tied to the fact that the internet was created, the data was created by humans, it's processed by machines, but it has all of our biases, it has all of our proclivities to division and all the problems that plague our modern world are well-represented on the internet. How important is it that we now, while we're just in the infancy of this, are recognizing those limitations? Where does bias rank? Maybe, Sebastian, you're dealing with all these risks. Obviously, you've got to prioritize them. Where does bias fit in? Where does income inequality fit in to sort of, versus, again, as Jeremy was saying, a lot of talk about the robots taking over. I usually try and suggest people separate those conversations. I think they're both worthy conversations to have, but you can't really talk about these near-term concerns, misinformation, bias, income inequality when you're saying, oh, but the robots are gonna kill us. So let's have special time devoted to our fears around the robots killing us. I do think the more we talk about that and the more we recognize it as a possibility, the less likely it's gonna be. But I also think we have these other problems that are here, they're present, misinformation's probably at the top of my list. But how do you move forward, and then I want to come to Minister Ghan on the same question, how do you recognize these risks? Work to address them and still move forward. You're not gonna get rid of bias, you're not gonna get rid of income inequality. How do we move forward? You talked about lifting the floor, that's certainly one way. I think it's so important that you raise this, because I think this is exactly the kind of conversation we need to spend more time on and resources on, right? And focus on these solutions. What you're highlighting is, we think there's sort of this three prong lens, you can take trust, responsibility, and impact. You asked me, how important, where do these issues of bias, where does the issues of toxicity, where do these other issues of inequalities fit? They need to rank really high. So what do you do with it? Well, so to look at Salesforce, the way we looked at it is we said, we need to make sure we're always really shifting left on this trust type of conversation, which means as we think of developing our product solutions, partnering with our customers, our customers, whether our customers are private sector governments, nonprofits, foundations, what have you. Civil society building in one second. Explain shift left because it means one thing to developers. Salesforce is known for shifting left, and that means something totally different. Talk about what you mean in the development side. So what I mean, Simon, for the more sort of approach, and you'll see there may be some more ties to it, is here's what we can't have happened, or what in Salesforce we cannot let happen, and hopefully everyone here says let's not let it happen. That trust is an afterthought, bias toxicity issues are an afterthought. That ethics, responsible innovation, and grappling with ethical and humane use of these issues is an afterthought that you figure out later after products have been deployed, develop sort of its scale. Shift left means bring these topics around bias, how do you deal with bias, toxicity, misuse. The problem of, well humans misusing sort of the technology, shipped it really early into your product development life cycle in your design, whether it's again at your organization, whether it's your company, whether it's your government, whether it's in your local sort of municipality, what not. Having cross-functional, say councils, as you're thinking about developing technologies or developing solutions, or determining what are the impacts we're seeking to have. Very, very early, when you're creating your product roadmaps, right, or your impact roadmaps, or your inequality sort of solution, or mass, right at the beginning, and then throughout the early stage, not at the very end, like, oh by the way, shouldn't we have started to look at these risks and then play catch up on it? Very, so what does that mean sort of for us? So we said, okay, let's build toxicity filters, right? Let's grapple with sort of these bias related topics. Let's think about, how do you, when you talk about bridging the old digital divide, sort of the AI divide, how do we vary early on also focus on enablement, upskilling, re-skilling. We've been partnering with sort of, many folks across the globe around how do you use our trailblazer platform, which is just, it's free content that we've just developed about how do you train and enable people on all sorts of items, right? And making sure that whether it's at our schools, or it's our workforces, again, upskilling and re-skilling, people understand what the opportunity is and they can deploy it really fast. We do think there are technological solutions to some of these issues, but it also requires companies to decide, let's build in, right, trust by design, compliance by design, responsibility by design. Again, rather than, oh, someone else will figure it out. One quick point I wanted to also raise, you talked about the farmers. I think this raises an even broader and interesting sort of question. Where are the domains or the context, you know, and this relates to what you were highlighting. What are the domains and context where in order to achieve the benefits in a way that is thoughtful about risk, but achieve the benefits, you know, for all or in an inclusive way, do we need to have adoption and embrace of the solution? And if you're in a domain or a context or a use case where you're gonna achieve the really important benefits you must have quality, embrace, and adoption, then you must do what Jeremy just outlined, right? Because it's not just someone from up on high that says, okay, now it will occur. You have to have the human beings, the organizations, those that may question, that may be skeptical, that may doubt, may lack trust. And that's where you have to work, I think, really early and have eternal vigilance around making these solutions, this technology, and the like worthy of the people's trust, worthy of enterprises trust, worthy of the public sector's trust. And it's really hard, because it requires you to grapple with very serious trade-offs. Minister Ghan, maybe that's a good time to come back to you in terms of, you are looking at this for, like, AIs here, how does my country take advantage of it? How do I use it to improve the well-being of my citizens, the economic competitiveness? How do I make sure I'm not introducing a new dependency? I think we've all learned over the last five or six years that who you're getting your technology from matters a lot more than we thought when we thought it was one global economy and it didn't matter geopolitically. I think, you know, things have shifted quite a bit. How are you thinking about, you know, where are the greatest opportunities and where do I need to be careful? First, let me also address the issue of the risk. I think with any new technology, there's always risk, and particularly, technologies like AI or generative AI, there are risks that we are not familiar with, and therefore there's always this fear that we will not be able to manage the risk. But I think what we should do is to do our best to identify the risks and to tackle it, to mitigate it. Because even human decisions, the process also is risky. We also have human biases, we also have our own experience, and even doctors make mistakes. As a former health minister, I must say that AI actually has great promises in healthcare, for example, in drug discovery. It will help to speed up the process of drug discovery much faster. At least Dr. Kohler is working specifically in that area. At a much lower cost and have fewer mistakes, fewer errors, I think this is a very important, and another area which Singapore is also looking at is how to make use of AI to encourage or to promote public health, and the things that you should do, the exercises that you should be undertaking, the type of food that you ought to avoid, and we can make use of AI to customize for the needs of individuals. At the same time, it is also a possibility for us to begin to embark on a precision medicine, to provide the customized medicines depending on the makeup of individuals. So gone may be the days where we have a generic medicine that is a bit of a trial and error. Sometimes this works for you, sometimes it doesn't, whereas with precision medicine, it has a potential of a more targeted way of therapy. So I think these are things that we are looking at, and Singapore is also looking at how we can apply AI and generative AI in the manufacturing context. How can I use artificial intelligence to optimize my manufacturing process, to optimize the management of an entire supply chain, particularly in today's world when we are beginning to operate globally? I think it's very often it's beyond human capability to be able to manage global operations, and with the assistance of AI, it will allow us to do so in a more efficient way and in a more optimized way. So I think there are many opportunities for us to develop AI capability and application, use cases. By the same time, we need to be conscious of the risks that it involves and have a robust governance system, and the system must evolve as the technology evolves. It cannot stay static. And therefore, Singapore has adopted a sandbox concept where we allow the technology to evolve, have a very light touch regulation. By the same time, we keep a very close watch on the development of the technology, and we need to make adjustments to our rules and regulations as we go along. So this way, we allow the development and evolution of the technology, but at the same time, making sure that this application is safe as much as possible. And if we discover a risk and we must be quite prepared to move and to adjust our rules and regulation, our governance system as quickly as possible. So we do need to have a flexible, practical, and nimble governance system on the use of AI and the development of AI. Can I just add on top of that, and I love the examples that you gave Minister Yan about the use of the technology in drug discovery and in healthcare, and I want to use that to highlight a point that I think often gets lost, because the consumer use cases that we all find so immediate and relatable kind of have taken over people's imaginations of this is what the technology can do. And I think the examples that you highlighted, which are the examples that also speak to me, are use cases where the computer actually does things that a human simply cannot do. In terms of assimilating, tremendously large amount of complex information and finding the patterns that allow you to whether it's identifying a new therapeutic intervention that a human will never identify, or whether it's being able to finally dissect the complexity of human biology to the point that we can actually identify which patient is going to benefit from which medication versus not, which requires assimilating so much information about human biology, human anatomy, omics levels, genetics, imaging information, and really create a diagnostic and a therapeutic path that a human clinician will never be able to get to. And I think those use cases often get lost by the wayside because people are like all about, whoa, diffusion and we can create beautiful images and we can no longer need Adobe Photoshop, which is great, but there's use cases that are just beyond the realm of human capabilities. And I think those are actually going to be the ones that in many ways are going to be the most impactful of all. I agree, and I also think that's where our challenges get pushed even further, is how do we as humans manage more and more work that's being done above our capability? Again, there's all this talk about AGI. I don't think any of these technologies are going to replace humans, but to your point, I do think already and increasingly they will be able to do things at a pace and scale that humans can't do, but also factoring in way more things than a human brain can process. I was struck, I had the opportunity a few weeks ago to drive an autonomous vehicle or ride an autonomous vehicle with Toyota, but I also had the opportunity to drive one where it was the human and the car, we were both playing a role, but I wasn't playing the role that we're used to. So we're used to direct manipulation. I turn the steering wheel and all the wheels go that way in the most advanced system, maybe only the rear wheels do. Computers capable of saying, well, the human driver wants to go left, I know the conditions, I should actually turn the left front wheel this way, I should, you know, anyway, my point being that there's actually a lot more factors when a computer's in charge that you can take into account that we've simplified because humans can't do it. Sebastian, how are you thinking about how do we manage responsibility when the work being done is, I don't wanna say above our pay grade, but more factors than we can handle. And in moving into a world, which correct me if I'm wrong, if anyone disagrees, but I think we are gonna move into a world where we are letting computers take action. I think that's the most exciting part, it's the scariest part, like right now, we're using it to inform human decision-making. It's decision support, but it's gonna be taking action. How do we prepare for that world? Yes. No, I'm smiling just because this, you know, at Salesforce we've had for a long time, a dedicated, I think this is public information, Salesforce Futures. And it's a dedicated Salesforce Futures capability that we really infuse into all of our kind of discussions around what are all the potential future scenarios that could occur, what do we think is more or less plausible and what are the ones that we wanna help, right? So to support or just risk to mitigate. You know, the lens of trust, responsibility and impact. So I think there's three kind of interesting areas that we're focused on, but maybe kind of other organizations and sort of society, you know, could be focused on. So there's an old line, you know, in the business world, but I think it applies to all organizations, culture each strategy for breakfast. So if we think about the pace of change and velocity of change and just speed, speed, speed, have you built an organization that is able to respond, has built the capability of innovation and a culture that can be fast, that can take in inputs, that can adjust, that can pivot, right? Can take in new information and then make adjustments rather than have a culture that is totally dependent on, right, the past. I think number two, you know, on these sort of the issues that you raised, how do you think about human capital management, right, and corporate strategy or government strategy, right, or non-profit strategy, or when you're managing a mix of human beings and what we call, you know, autonomous GPT agents. You know, when we launch our Einstein GPTs and those sort of different skills and the like, you know, I'll give you sort of a brief sort of a little vision, you know, and but again, this issue of how do you manage like this broader kind of workforce that hopefully again, you've got humans who are amplified and augmented, right, sort of with, you know, Einstein co-pilot, you know, whomever it or what it is, but partnering with AI to do their jobs better, to have their impacts be more meaningful, and then also dealing with folks that, right, aren't human, right, autonomous agents. But here's sort of the interesting dynamic. So right now we think about, you know, how do people interact with each other?  Again, our view, our strong principle and acceptable use policies and the like is disclosure, be transparent, we should all know, right, when we're interacting with that, but there will come a time where it's not a human interacting with an AI. It's autonomous agent interacting with autonomous agent, and then continuing to interact with autonomous agents, and it's all going back and forth, whether it's in any sort of potential domain, certainly in business, right, whether it's in sales, whether it's in R&D, whether it's in marketing, right, whether it's in procurement, whether it's in all this, we're already by the way seeing, you know, the first legal contract, it was in the NDA space, you know, was negotiated solely by, right, agents on each side, overseen of course, right, by lawyers who hopefully were paying very close attention. But this whole issue of agents interacting with agents, and then the concept, you know, we talk about human in the loop a lot. You know, I think you raised an important point, look at the current general sort of state of the world around AI and these technologies. It's actually not just human in the loop. Humans are making the decisions with the benefit of the inputs, AI inputs, the analytics and whatnot, but humans are making the decisions. This is very important by the way. You could say maybe this is part of trust responsibility impact, how do you ensure that it continues to be the case that we have, right, sort of human oversight? But will there come a time where a, hopefully again, a trust-first, trust-infused, responsibility-first, ethics-first, responsible innovation type of decision-making apparatus is actually AI led? What do we have in a world where AI is making capital allocation decisions? If AI is making, and here, you know, I think this is, you asked about what's the significance of change? And if things happening really fast, what happens is the law, right, the reality on the ground, the technology outpaces the law, it outpaces the public policy. And in order to achieve the future that we want to have, rather than the one that we may get stuck with, we have to have strong law and sound public policy, but also one where these elements are gonna continue to drive forward, I think, by the private sector. And to some degree, and perhaps to an increasing degree, you know, depending on how items occur, by the public sector, you have to embed in that voluntary set of actions, and hopefully cross multi-stakeholder actions, this sense of responsible guidelines, right, acceptable use and trust, so that even as you protect and safeguard and hope maybe accelerate innovation, you're accelerating responsible innovation and people are really grappling with these issues really early. Jeremy wants to get the last word, so I wanna give you real quickly. I just wanna come in, because some of these questions aren't necessarily due, right? So if you look at financial services, you know, a lot of you have already used a credit card or a payment app today, there's AI behind that. If you think of high-frequency trading, we actually have safety mechanisms in place, so when the agents are talking and basically trading against each other, say, oh wow, this is actually getting a little bit out of hand, I think what we can do is actually learn from some of the heavily regulated sectors, like healthcare, like financial services, and say, now knowing what we know, we'll actually start needing to apply this in more areas than we have previously, right? So it'll actually, because AI is going in and being embedded in a lot more domains, we'll need safety switches, we'll need to think about what happens when agents, talking to agents, get out of control, when do we want someone to say, oh, hold on, put a pause on that, do you wanna do my whole bank account or just the trade for my Netflix subscription, right?  Recommend everybody look in our report from last September, we don't see jobs being displaced so much as specific skills, so we recommend all of you to look inside your organizations and think what are the specific skills or roles that are most ready for automation, and also have that discussion with your team members, with your customers, with your organization, and this will actually help us navigate learning on knowledge in both successes and mistakes from the past and actually helping prepare for that better future there. Well, we are gonna have to leave it there. Thank you so much to all our panelists for the insights. I know we're gonna have a fascinating couple days of the conference. If you enjoy discussions like this, quick plug, I do a daily free newsletter for Axios. You can go to aiplus.axios.com. I'm sure we're gonna be talking more about this, not just here, but again in Davos in January where last year chat GPT was the talk of everything. I think this year, how we deal with a world that's been changed by chat GPT will be at the top of the discussion. I would ask everyone in the room to just stay in your seats. Our next panel is gonna be on the new age of Gen AI governance. It's gonna begin in a couple minutes after we quickly refresh the stage, and then after that session we'll have our morning break around 10.45. Thank you.


Good afternoon, everyone. You heard President Biden speak very powerfully and very clearly to the pure evil that we've seen afflicted on Israel over the last few days. And he was very clear that we stand with Israel. And that's a message that he's asked me to carry directly to our Israeli partners when I travel to the region a little bit later this week. In the meantime, we're also very engaged with partners throughout the region, as well as, of course, looking out for American citizens, families who have had loved ones killed or have loved ones missing. I have a lot more to say about that tomorrow and in the days ahead. The Yemeni-led political process under the auspices of the United Nations. Thank you very much. Thanks, everyone.