It looks like we're live. All right, what we're going to do is screen share, jump right into this. We need to share the desktop, here we go. Okay, we have a lot of ground to cover in a very short period of time. Hopefully you guys are ready for a wild and fast ride. We're going to be talking about reactive programming and we're going to show you a lot of really cool demonstrations of a technology called VertX. And actually I have a ton running here on my desktop already. We're going to get a chance to play some games together, look at some interactive content. And do note that the chat is over here, but I won't be able to see it too easily. Please do ask questions via the chat though, and I'll get to it when I can. We'll break out into demo mode. Do look at the slides. The URL for the slide deck is actually right here. At Bitly, going reactive Java, that'll get you right into the live deck and then you can actually follow along from a slide standpoint and feel free. It's a Google Doc to fork it, take it on your own and run away with it. My Twitter handle is there as well as my email address. Feel free to email me, hit me on Twitter. Anyway, you would want to communicate with me. Okay, let's dive right in. The first thing to realize is that this is a brand new show that we're now launching as of today. It is known as Day of Nation Live as a tech talk. We're going to basically spend about 30 minutes together on a reoccurring basis, the first and third Thursday at noon. So the first and third Thursday every month, we're going to try to bring something new and cool for you guys. So this is episode number one and we're going to show you a bunch of really cool reactive programming concepts and reactive programming demos to get it so you guys get a feel for where we're going from a vertex standpoint. But in the second episode, we're going to dive right into Kubernetes and spend a lot of time with what it means to build Docker based containers to run on the Kubernetes backbone. That'll be super awesome. And Rothfield does a great job offering a workshop around that concept. The third episode, we're going to actually show you something pretty futuristic around the concept of a project called Istio and Envoy where you actually build side cars to add to your pods and Kubernetes and you can build on microservices mesh and Christian Post has been doing a lot of R&D work on that for us. And then we're going to show you some big data stuff. So you can see Apache Spark within Finispan. That'll be super awesome with Galder. And then we're going to do another deep dive into products and reactive overall with Clamont who actually wrote our first book on this topic. So there's a lot more coming and we're going to be scheduling numerous rock stars from presentation standpoint and numerous topics as we get going here. Now there's a book available on the topic that we're going to be focused on today. So if you go to developers.red.com or if you join the slide deck, you'll find this is a link or this is a link and you can download this book and actually walk through some of this content more slowly. So feel free to download that book and get started but go ahead and pay attention right now. Don't start your reading just yet. Now, I often like to start my presentations with the concept of the evolution meaning that we're always evolving, we're always moving forward as developers because in the 70s as a developer, all we had to learn was COBOL or JCL or WFL. That's all we really had access to or all we had to know from a developer standpoint. We worked on a mainframe whether it be UNISIS or COBOL and in my case, I was more UNISIS. So I had, I used WFL, right? I was not so cool. I didn't couldn't do JCL back then. Workflow language is what WFL stood for. But in the 80s, we picked up things like C, C++ those become much more dominant in the overall business application space. 4GLs also started coming on the scene whether it be progress or focus or Ingress all back then. And I don't know if you guys remembered Netware but I was actually a Netware administrator. I know that's kind of old school for some of you guys but yeah, that's where I was. And of course, relational database and SQL was kind of born in that era. In the 90s, we started picking up things around the web. HTTP, HTML, you can learn HTML in three days by reading Laura Lame's book. I did that back in the mid 90s and we learned the CGI, you know, Common Gateway Interface and get and post. We learned about cookies. And of course, cookies were terrifying back in the mid 90s. I don't know if you guys remember that but I used to have to explain to users that a cookie was not going to steal their identity. A cookie was just like an ear tag on Anlope. And if they were watching Mutual Boomahal's Wild Kingdom, they would understand what that meant. But then I learned most people didn't actually watch Mutual Boomahal's Wild Kingdom in the 80s like I did. So just keep that in mind. Cookies can be a little terrifying for some people. And as you go into the OOs, right? You would see things like MVC, right? The Struts, DI, Spring, Hibernate, all those cool things came on board. We did a lot of XML. We did a lot of SOA and web services. But really, in the modern era, we're now having to learn things like Angular, React. Maybe we have to do mobile, iOS and Android. We might learn Maven and Gradle. Now we're learning Get. Instead of Subversion is a good example. But my point with this whole presentation, this whole part of the presentation, is that Reactive is just yet one more skill that you're starting to pick up on now. So just one more skill, one more tool in the toolbox. And of course, our ultimate mission is to all be like the GitHub rainbow unicorn here. I love that guy, but the mythical beast. Let's keep going. So let's define Reactive. There's two forms of Reactive out there in the world. Reactive programming and Reactive systems. That's how people think of it. Or at least they should be thinking of it. There's Reactive programming, meaning how do you write syntax that actually demonstrates Reactive properties? And then of course, how do you build a Reactive system? So let's kind of break those down. The good news is you can just relax the use to go in either category. And then it all should be noted that a Reactive system does not necessarily have to be based on Reactive programming. Though obviously that does help you. Reactive programming is programming with asynchronous data streams. So I really like this definition. And it's out there. Here's the URL for it. Because that is really how you should be thinking of it. In a world where everything is async and ongoing and it's never ending, right? The data stream is just constantly coming at you. And one of the demos I'll often do is actually an IoT demo where I show sensor streaming because your temperature sensors and other sensors out there in a while, they have no concept of you're overwhelmed. They just keep sending out their data. And it's kind of like you tuning into a radio station. You basically tune into the radio station and you get the stream. If you tune out of the radio station or off the radio station, you don't get the stream. The radio station does not care that you're paying attention or not paying attention. They're just gonna keep sending the data your way. And it's up to you to manage that asynchronously and buffer it accordingly. So Reactive programming and asynchronous data streams is super important. Now this is where we break into reactive systems. So not necessarily programming. So in a reactive systems world, this was defined by the reactive manifesto. You can read about it on the internet there with the URL. And the key component of that is it's message driven. Some people would say event driven. The idea being that it's going to be asynchronously invoked. And you can think of these messages flowing independent packets as flowing through those streams. So by default, here comes my stream. Here comes a message. Is it right here? Okay. I have a lot of things running. And I'm actually running this in the public cloud. So you guys are free to join me. And if you actually look at this URL right here, it's gonna be live here momentarily. So the first thing I'm gonna do is I'm gonna run a standard Spring Boot application. We'll show you the code for this in a second. But let's just run a simple Spring Boot application. I'm launching it up again in a public cloud instance with the main name as web.bird.red. But this is running on 8080. Just keep that in mind. We'll switch that in a second. But there's my Spring Boot application, okay? It's real super simple. It's kind of like Hello World, but we call it Goodbye because we're gonna actually injure it a little bit. So by default, if you guys are clicking away on this also, you'll see that it's just responding nicely to the user. But what happens when we pound on it? So this little guy down here generates 51 requests to this bigger application. And I did change the thread pool on Spring Boot to be 25. So in other words, there's more requests, more users, than there are actually threads to respond to that user. And you can see right now, it just locks up. It's no longer responsive until of course the thread becomes available. And then if I hit refresh here, you can see it's now locked up again. So the whole concept that we've had in JavaE World, right, whether it be Tomcat or traditional JavaE is that by default, it assumes a thread per request. And that's just the model it's in. But if you use all the threads, then you can't handle any more requests. So now you can see that our threads are completed and we're back to working again. And I don't want you to think that I'm actually picking on poor Spring and Tomcat here. Here's our application, right?  In this case, I'll run Wildfly Swarm. And so if you're refreshing with me now, you'll notice that I took the server down and it's now coming back up here. But here we go. So there we go, right? We got the Wildfly Swarm up again. Oh, I got messages flowing in here. Says I'm not sharing. Let's see if I am sharing. All right, so we'll reshare. Dun, dun, dun, dun, dun, dun, dun, dun, dun, dun, dun, dun, dun. Okay, retry. It says, screen sharing, filled to start. Okay, mm-hmm, mm-hmm, all right. Well, I'm not sure how to get the screen sharing to restart, it says it will not start. Let me see if I am, I'm gonna have to refresh this browser window here on Google Hangouts and we'll see what happens, okay. Back to sharing. Okay, we should be sharing again. There we go. All right, if you guys need me, just ping me. But there we go, let me see, bring up these little windows again, back to it. All right, where did my little windows go? Okay, so we just brought up Wildfly, as you can see here. All right, so the same thing as before as the Spring Boot application. But again, the same kind of concept. If I hit it with overwhelming load, we're going to be in a situation where it basically pauses or hangs based on the fact there's 25 threads to process user request, but there's 51 requests coming in simultaneously. And so whenever, again, you have a thread per request model, like we do in traditional Java applications, you will eventually run out of the gas and then your users will wait. They can cancel that transaction by hitting X here and refreshing, all that does is send you another transaction that you cannot handle and just, of course, there's no one listening anymore to respond to that. All right, so now we're back to good again. So that is it from a Wildfly swarm standpoint in Spring Boot, but let me show you VertX, okay? Let's go ahead and get started. Let's get the VertX system started. So VertX, of course, is an asynchronous, pure reactive system, and I'll show you the code behind us in a second. Let's go ahead and get it started. Okay, you can see it starts super fast and I can refresh, refresh. And again, you guys can use that public URL there also. But if I go, let me run the command line again. Again, we're gonna send in a bunch of requests and you can see the requests are going in, right? So they're all in there pounding away at the system and then, but I can still get my responses. So the user is not hung up. He or she is still gonna get responded even though that the system is totally overwhelmed with load right now. So even though all 25 threads are being used, right? The foreground thread, the thing responding to users is always available and that's known as the event loop. We're gonna see more of that in a second. Okay, all right, so just keep that in mind that that's the concept of a reactive system. Let's jump back into here. All right, and we'll go here. Okay, so reactive systems. Now, let's talk quickly about microservices. It looks like a lot of people have joined my slide deck. That's good. Now, in a microservices world, you have to think about this entire evolutionary process, microservices being at the end point of this evolutionary process of DevOps, self-source on demand, elastic infrastructure. These are all critical components. So think about these things as we get there. We're gonna dive into more of these topics as we go throughout this definition live series. But today I'm gonna really focus more on the microservices side of it as if you're already at the end game and how do you use a reactive system like VertX for that. Now, I'm not the only one who talks about this evolutionary cycle or how mature you must be in order to take advantage of these capabilities. This is Mark Ballard's version of it. And you must be this tall, right? I love that concept that you must be this tall to ride this ride. If you've ever been on a roller coaster or to a park with roller coasters, there's often a little sign that you've got to be this tall to ride this ride and I'm not particularly tall. So as a little kid, I was the one who did not get on the ride in a few cases. So a maturity model to be ready for microservices. These are the key principles and characteristics. And again, we'll drill down more on these in future sessions. But just know that there's a lot going on in the microservices architecture that we're not gonna spend too much time on today. We're just gonna show you how to build some systems, write it quickly, write some applications quickly. Key concerns. One reason to think of these key concerns or key capabilities is that we tend to run them on Kubernetes because with Kubernetes, we've got load balancing, scaling, elasticity, we get source discovery, you get logging, monitoring, you get a lot of those things for free. And that's why we focus on it from that perspective. And I'll show you a quick example of that, a quick demo of that when we get going. Okay. There's a lot of recorded demos along these lines. And again, we're not gonna spend time on that today, but just keep that in mind that you are also available to you in the slide deck. Now, let's talk quickly about VertX and I will show you some quick demos. VertX is specifically a reactive toolkit. Okay. It's a jar file. You load in your POM XML and you have reactive capabilities you can add it to your application today. It has all these capabilities in the core. So HTTP one and two by default, you can listen for HTTP one or two, UDP, TCP. It also has this concept of the event bus. We're gonna really focus on the core pieces today, but we'll also touch on the router as well just briefly in the demonstration. But you can see it does a ton of other things also. So it can be a full stack CRUD application if you wanted to be get, put, post and delete traditional CRUD, but it doesn't require you to think that way in this new world. So hopefully it'll make sense when we see the demonstration. You're gonna hear this concept of the vertical. Think of it as the programming unit as my class file if you're in Java, but if you're in Ruby, it's just a Ruby program or a Ruby program depending on what language you're using because VertX by default is polyglot. But just keep that in mind. We're gonna be focusing on this concept of the vertical. There's also the event bus that'll be super critical. The event bus of course is distributed across JVMs by default, which allows you to have this nice in-memory programming model that allows you to communicate across all the different components. We'll see that in the demonstration also. The event loop, which is the core architecture of VertX allows you to simply receive events, find dispatchers or find handlers and dispatch to them. So think of that as a central cue where the events are coming in, whether it be responses from other things you've done or just events coming from your user like get, put, post or delete. That's a good example. And if you have multiple cores, which I do on my demo server, you actually have an event loop per core because in a Java LAN, you actually have that opportunity to be multi-threaded. But by default, from a programming standpoint, you don't have to think about that. We take care of that for you by using the event loop, okay? Let's go ahead and show you some quick examples. Let's go ahead and get into the super fast getting started here. And what I'm gonna do is I'm gonna actually just copy and paste directly from these slides, just like you can go into my, let's see here. I have this directory out here called stuff. And I'm just gonna basically copy and paste, okay? So I'm going to paste that line in, which basically says what do you want for a group ID? And I'm gonna say that, and I want artifact ID, 1.0 snapshot, that looks good to me, okay? I can do a Maven clean compile, like a type today package. And there we go. All right, looks like it's good. So basically it generated a Maven project for me. And then you can kind of see it right here. And I can do something super cool like this. Maven products run. So it actually does dynamic or hot deployment. So let's do switch out of here. And let's see, do I have Maven running? Oh, get this set up correctly. Maven, all right, fantastic. We do have a Maven, very, very good. Okay, watch what happens here. We're gonna go to source, main, Java, stuff. There it is. All right, let's just interact with this. So by default, you get this basic vertical, right? You can notice that it's a start method, but we're gonna add some stuff to it. We're gonna make this a little bit more interesting. Well, let me spell things correctly. B-X-T-E-E-W-Eb and router. And let's come over here now and let's add a little logic to it. So the start method is always called by default. This is how you get things going. And I'm gonna say router. I'm gonna declare the router. And so the router is a specialized piece of middleware as part of Vertex. And what that allows them to do is specify URLs in a clever way. So I just hit save there and you can see it's automatically redeploying it. Now we're not done anything yet. I just hit save to see what it looks like and see that it compiles. Okay, let's add router.get. And we're gonna listen for the root of that. So basically any traffic that comes to the root will run this handler. So request and we'll put our lambda in. And based on the index, right, request, we're gonna actually display the host name. So let's do this. And system.getenv.get or default, okay. And we want the host name of the machine we're on. You'll see why that's important a little bit later. Unknown, okay. And then we're going to respond to the request. So request, response, end. And so, hello. So I'm really just coding this up in VI for real here. Not that I'm a VI expert and then those of you who are actually out there going, hey, I love Emacs, I apologize. I don't do Emacs. That's okay though. Host name, okay. And if I did all that correctly, let's close this up now. Bum, bum, bum, okay. And let's write it out. And we'll go and see if we, so it's gonna dynamically reload. Okay, fantastic, it reloaded our thing there. But it's still not actually listening yet, okay. So let's actually show you what it means to listen. So we're going to basically vertex, or X create HTTP server. And we're going to bin request handler. And then we're gonna basically put the router in here. All right, so this is a method reference. So basically it's gonna map the router and then listen on 8080. Now if we did our job right here and I didn't get anything syntactically incorrect, it'll reload and we'll get a nice clean redeploy there. And if I go back to my user interface here, okay. So there we are. So I'm at the browser and that's hello world. And you can see it happening right there and that is this host name. And actually let's go modify this a little bit. Let's take out hello. And I'll come back with my favorite. I'm in Seattle today, but we'll go ahead and put a lo-hi and I'm pretty close to home here. So let's put a lo-hi in, refresh and there's a lo-hi. Okay, so that's our application. Something as simple as that. Now, here's what's kind of cool. I can actually take this down now and I'm gonna go back to the slide deck and I'm gonna copy another command out of the slide deck because one of the things that you can do is now that you filter application, is you can basically map it with fabric eight to make them plug in over to a Kubernetes instance.  And then if I come over here and say, cube CTO, get namespaces. All right, let's see OC new project stuff. So this is actually adding a Kubernetes namespace and I can say, made in fabric eight deploy. Okay, so that basically will bundle up everything I need, including creating the Docker image, creating all the YAML files you need to actually deploy to a Kubernetes instance and actually deploy it out there. If I go back to my browser, you can kind of see their stuff and the deployment is happening. Okay, so you can kind of see right there, the deployment is happening. And as it goes through that Docker build, it'll actually create the image and push it out there. And so there's the process it's going through right now. We don't have to stop and wait on it, but just keep that in mind that this is the process you could go through to quickly deploy a new endpoint into a new microservice directly into a Kubernetes cluster. And again, we're gonna see more on Kubernetes in our future session. But I wanna show you a couple other demonstrations here. Okay, let's actually jump over here. Yeah, let's get out of here and go back here. And what else do I have on the server? I got a lot of cool stuff. So we did that one, we did that one. Let's show you the event bus. I wanna show you the event bus just so you get, and just so you see the code. The event bus has this concept of the web server and the bridge. So you can actually use your bridge from the web server out to the browser. So what this allows us to do is bright directional communication from the browser to the server. It also connects all the JVM instances on the server together. So if you actually look at the publisher for Ruby or consumer for Ruby, you can kind of see it's publish or it actually creates a handler right here, consumer. If I publish with Java, same kind of concept here. And I can consume with Java, right? It's a very simple syntax. And we're not really focused on syntax day. I just wanna show you the key capability. So let's go over here and let's bring up the web server. And I'm gonna go back to my browser and let's see. Okay, it's coming up now. Oh, there we go. There we go. So your web server once gets connected, it's gonna basically start shooting messages at the browser.  I'll notice also I switched from 8080 to just plain 80 to make it a little bit easier if you're typing on your phone. But let's do this now. Let's see. We can actually add another consumer or let's add a Ruby consumer. And so what I'm doing is I'm connecting up things on the server side. I'm gonna add a Ruby consumer and I can also now do a, let's do a Java publisher, okay? So these guys are now coming online on the server side. Gonna connect and right. So connect, connect, okay. And I can see some people join me. That's what the socket connections mean. So I know some people join me. So there's my Ruby consumer and there's my Ruby consumer. And now my Java system is connected. So the Java actually sends out a message that turns the background color yellow. I just thought that would be funny. So you can see Java messages and a web server message showing up there. And if I wanted to, I could just go ahead and kill this and let's do this. Let's actually start pushing out Ruby messages into the mix. So again, all this code is super simple but allows you to do server side push in such a super simple way, super easy. So again, if you're running this on your phone or your desktop, you'll be seeing those messages fly along there. And again, my whole point with this is to show you that what's possible with this kind of technology. And again, it doesn't take much code. If I flip back over to the code, kind of see like my Ruby publisher, where's my Ruby publisher? Simple example, Ruby Now, the date timestamp, and of course the specific feed. That's the channel through all that we've exposed. If you look at my web server here, it actually says that specific channel is what's outbound. So just keep that in mind. You have to have this outbound committed in order for to make that work.  So a lot of activity I know happening there but that is just a nice little example of what it takes to actually do something interesting.   So now we're just bringing these things down. You can see I have several things running still and let's go ahead and bring down that web server. Just since I killed Java and Ruby, we're just at a regular web server now. Okay, yep, yep. Let's see here. I wanna see if I have Java processes running out here. Let's see, I do have some still running out there.  And boom, boom, boom. All right, still one more. I just wanna make sure we clean up nicely. Okay, maybe everything's down now. All right, it looks like everything's down. Fantastic. So let's show you this. That's a cool example, but I got even cooler one for you. So let's do sudo. By the way, I can do effect jar deployment as well. So you can see that's what we're doing right here. sudo dash jar. The reason I'm using sudo is because it's also running a port 8080.   My phone, my phone, my phone. Let's see here. All right. Okay. Let's get on the right URL. Ta-ta-ta-ta-ta. Web.bird.red again on port 80. And now what's what happened? So I paint a picture and it shows up here in my dashboard. So if you just go to web.bird.red now, this is an example where we use the event bus in a bi-directional way. Basically, we're connecting from the client side, mobile browser, painting pictures. And then I can basically now, as each stroke finishes on my phone. So if you watch here, I can use it different. As I finish a stroke, it'll show up directly on the dashboard here. So this concept is actually very powerful if you think about it, because now I can build a full reactive application. In this case, it has very little code behind it. And actually the server side of this code is almost nothing. It allows me to basically take an HTML5 canvas from the client side and move it out to the server side and then back out to the dashboard. So I made a little base here. There we go. And it's in yellow. So I guess that doesn't show up very well, but I give them some red hair. That's pretty nice. And you can see other folks are playing along with me and getting connected also. So this concept of the event bus is super powerful and super critical. The idea that you can have this real-time reactive application interacting with maybe a mobile client like see her on my phone or even a desktop client is super easy. And actually, let me flash over to the, you guys can feel free to draw your pictures, but let me flash over here to the code real quick so you can get a chance to see what I'm talking about. The server side of this is super simple. And all this code, by the way, is all open source. In this case, we are outbound and inbound permitted because the same feed, the same bus, if you will, is exposed from taking canvases from the client side and pushing them out to the dashboard that you see there. And you can see it's just listing on port 80. So the server side is just this. That's all it is. The real meat of this effort is actually in the actual client itself with my really ugly JavaScript right here. So really hideous stuff, including the buttons and how you select colors and how you draw pictures. And then really the magic is on this in stroke, right? Send the canvas and basically send canvas, grabs the serialized canvas, makes a JSON string out of it and publishes it on the event bus. So everything you saw earlier, in this case, just client side JavaScript back to the browser standpoint. So you can see it kind of goes pretty crazy, pretty fast. Okay, well, let's see, let's find the right window again. Here we go. Nope, we got a lot of pictures drawing out there. Some people already cleared those. Fantastic stuff and you guys are keeping it clean. I like that, I appreciate that. In some audiences, you get some really bizarre looking images. I just don't, no, we don't wanna call attention to that. Let me show you one more demo that why you guys are on your phone. Okay, and you guys are gonna hang out here with me. Let's see here if we can get this party started. Okay, excellent. So I'm gonna show you one more demo. This is on a different one. This is game.bird.red. Okay, let me bring that up here on my phone. There we go, get connected. So game.bird.red. And what it is, it allows me to pop balloons like fruit ninja style. You kind of see as I pop balloons, I get different accolades there. And so you guys can go to game.bird.red, okay? And then pop balloons, so game.bird.red. But I'm gonna actually bring up some dashboards real quick. Let's make this a little bit bigger. Game.bird.red, all right? So here's a scoreboard. Okay, we've got six people already in playing right now. Feel free to join, cause it's actually a fun game. And let's go to the leaderboard, okay? And so again, there's those six people that have connected, seven are now connected. And you can sit nine, so we're getting people connected now. So feel free to connect up. And what's happening is as you connect, of course, you can see our pop count, our player count goes up in real time. Someone lost their web socket connection. Now two more are joined. You can see our leaderboard for our top 10 scoring folks here. You can see this one's called puppy puzzle puppy over here, so let's score over here. And then of course my mobile app over here. Now, this is a super fun one because of course it's a nice interactive game, but you should understand something about the architecture here. So I need your attention for a second, so let's pause you. Okay, now here's the idea. Let me show you this in the slide deck. Now let's go back over here. Here's why this game's important. We have the Bloom popping game on the client side. It's interacting with a Vertex server via a web socket. This is a single JVM running all you guys pops. We can run hundreds and hundreds of users and currently pounding away on the system. Each pop of the balloon is actually a transaction through a back end microservice for calculating achievements or basically looking at what the achievement options are. And then it actually goes to a traditional JVM application for an intraditional JBOS enterprise application platform and running the Drool's project or the BRMS technology to calculate what an achievement means. Like if you get three in a row, 10 in a row, hit a certain level, right? Those achievements are super critical. So this architecture is all part of the back end. So I recognize I paused you guys. You're probably unhappy with me. Let's go and get started again. Okay, I'll let you guys keep playing. Watch what happens though. I can actually make real time changes to the application. Like make those balloons much easier to hit. Make the red balloons count as 10. Maybe change the background color and then send you an update. So now you have big fat balloons and it's much easier to score points. Where are my dashboards go? Let's see if you guys are really pounding away at it now. All right, fantastic. Okay, so it does look like people are really scoring points like crazy now. You can see the leaderboards changing rapidly. But each pop you guys make is going through that entire back end architecture. And as our achievements are scored, you can see them showing up here in our dashboard here in real time.  And actually there's one more trick in this application I'll show you here. And that's because the team thought it would be funny to add the golden snitch. And so you'll see that show up in the screen. Oh, looks like some people already hit it. There it is. They decided to put my face on a balloon. So as soon as you hit that and get that achievement you score a lot of points and then you can really hit it fast. But look how many transactions already 2000 plus transactions into the system just with 15 people playing with this simultaneously right now. So just keep in mind you can build a really interesting application really fast by simply building a nice product system. And I know I'm running out of time. So game over. Okay. Well, I know I apologize.  Maybe in a future date we'll play some more at some point but you can see there's my puzzle puppy. And then you can kind of see what my screen looks like here on my mobile screen too. Okay. Let's kind of jump back in the slide deck because I promise to you guys we'll hang this, we'll run this about 30 minutes. The repos are all here. And so if you want those balloon repos there's also the demos are here. Keep in mind that the entire slide deck is available to you at goingreactivejava. And then of course monitor debonation live because in the future we're gonna bring it all kinds of cool stuff related to this and more with Kubernetes or Istio and Envoy, deep diving into Docker and Kubernetes as an example on how to build job, bring your Java workloads to it. Of course, big data and other topics. So if it's a cool happening topic and you're interested in it, please let me know. Feel free to hit me on Twitter or via email and we'll continue to bring you great content the first and third Thursday of the month. And again, thank you so much for coming out today.


Hey guys, Coach Mack here with this week's Beneath the Surface powered by Microsoft Surface beginning now. All right, today Beneath the Surface, we're going to look at an excellent screen game that's been developing throughout the season with the Tennessee Titans. First screen we'll have, this is second 13 in the first quarter against the Raiders. We're an 11 personnel, 2x2, this is against a five man front. Drop down safety comes down, so now you've got a three deep look. Number 22, Derek Henry comes across the formation, Ben Jones, Nate Davis. Let's them take some great, great blocks downfield. Number 22 juggles the catch just a little bit, but he's got an excellent convoy, 23 yard game. This is executed very, very well. The blocks are in perfect position and everybody is involved in this big game. Next play we're going to look at, this is against Indianapolis Colts. Now this is first and 10 in the first quarter. Now the Titans are in a trips bunch right, all right, with the numbers minus three split on the left side. This is an eight man front. The running back is now out of play action strong. Henry looks open after the fake, nice, nice bump by Jones and Brewer. Timing is perfect. The two defensive ends are up the field. Swing is a nice front side protection. Ben Jones, Davis or downfield, Nick Westbrook, Akina again showing up downfield blocking. Great job of everything working together for the run. Another 22 yard game for the Tennessee Titans in this play. Next screen we're going to look at is against the commanders. We are now in 21 personnel, two running backs, one tight end, two wide receivers. I write formation, this is a five man front from the commanders with the two high safety. This screen is going to come off of a split zone throwback look to the open side. Brewer with a beautiful block to get it started. Perfect timing. Ben Jones again, out front, Carter, Radence, also downfield, plus 24. This play right here will set up the next excellent screen we're going to look at. You can see now that the Tennessee Titans offense is building a repertoire of screens that Daisy chain one off of the other. Our last look. Now this is first intent, the plus 13 yard line, this is 21 personnel. It's two running backs in the game, they're in red or split back formation. Henry motions to the tight end side, draws the defense with him because Derrick Henry is a big focal point of this offense. Defense is drawn to him. Tannehill pumps to Henry, timing is excellent. This is a throwback to Hilliard on the open side, Ben Jones and Brewer again. Perfect throws downfield and the thing that they do so well, they combo up first of all, off of the tackle, then they get downfield, left tackle Dennis Daly does a nice job of drawing Montez sweat up the field. This is a very clear lane for Hilliard. This is a touchdown. This play came directly off of the setup of the previous play that we just saw. Tennessee Titans are doing an excellent job in their screen game so far this season. Look for them to build upon these concepts as we continue in the 2022 season.


Going on everyone Sam here with late sports cards doing a three box baseball personal for Tom C We got two 23 tops finest and a 22 bowman's best. So start with your finest. I'll see something big man See you moments factor moments out of Pulled the Riley of that already Headliner is Lattie G Corbin Carroll rookie and Alvarez rookie. Oh, there's Freddie Freeman refractor Non-extensive Rookie is Yoshida Masataka Yoshida Riley green rookie and Oscar Gonzalez When orange orange rookie Michael Harris 11 to 25 nice one there Tom congrats on that buddy rookie of the year 11 to 25 D. All Hall can tell Marta Christian Yelich refractor Mark Appel rookie refractor auto for the Phillies and Joy Manassas Spencer steer. There's a Griffey headliners Goldie and Josh young Miguel Vargas. There's Ronald Cunha 300 Harper Juan Soto There's Joey Manassas refractor rookie Nick Prado Carlos Correa headliner is Mookie Brian Bayo and Michael Harris Graffiti Riley green to 250 blue teal blue vapor Mitchell Shea Langlears Volpe rookie Josh young nice one there. We got a moment of Ryan Howard November 20th 2006 That was the area one a year he won his MVP following his rookie of the year nice one there for the Phillies Jayrod world's finest Miguel Cabrera Crawford and Nolan Gorman second box of finest Cola Steve Winfield that's actually kind of nasty Luis Castillo to 150 nice one there for the Mariners on the auto Gonzalez on her green Logan Webb Freddie Freeman Riley green rookie Spensastia The 125 simian Woods Richardson for the twins green mini diamonds Joey Manassas and Christian Yelich True blue Ken Walde shock rookie to 150 for the A's Goldie and Josh young rookie Lede Zach Wheeler refractor for the Phillies Brice Hapa and Juan Soto You go there and Robbie Devers headliners Nick Prado and Carlos Correa Second mini box See something big man. It's like a gold Adley. Gold Adley auto. That's my prediction here Sean Corbin Carroll refractor rookie Carlos Correa, let's go On the super short print But it's called let's go. Yeah, let's go Like that too nice one there that is called. Let's go super short print. Let's go In the middle of the break get in here Maddie Libertura Alvarez refractor rookie That would have happened that Fordo it's a 175 Oswald Parazza and Brandon Crawford Volpe rookie Chris Bryant headliners Tati's Nolan Gorman and J rod rep baby rookie refractor Michael Stefanik nice one there on the rookie auto for the Angels Garrett Mitchell and Shay Lingliers Freddie Freeman rival Reese Gunnar Henderson rookie Brian Bayo and Michael Harris All right under your bowman's best 22 bowman's best now say ladies 22 be best be best baseball Winske Pache there's a Josh Lowe atomic Kumar rocker refractor It was wood Shohei masterpieces Colton Couser and Albert Pujols Jay rod rookie Ladi close amount gummery refractor auto nice one there for the White Sox Green wander Franco to 99 47 of 99 Raise Brandon Marsh refract rookie Pena rookie base elements of excellence water Jordan Lawler Say as a Zuki refractor Mount Castle J. C. Young Javi Baez Chateau John Kenzie Noel base prospect auto for the Guardians Ricardo very UFO and salve press idiots You are really like dude close traumatized by that one Here salve press Miranda Kenzie Gore nice gold Jordan wicks six of fifty prospect auto for the Cubs nice one there congrats Tom UFO Pineda and Oscar Coloss Adela Cruz perspective royalty Josh Lowe base auto Alan Serda for the Reds there you go Tom a Chris Bryant teal lava to 199 a logic green refractor Many diamonds Vidal Bruhan to 299 water O'Neill cruise rookie and yes, sir. Yes, sir Mercedes Marcelo Meyer and Jace young that'll do it for your break there Tom Thanks again for your support man. We'll get him right out to you buddy


So this is a tag team talk between myself and leaping Wang and Victoria Lim in my group who is I'm gonna be talking some about some some work we've doing been doing in this area as well So what part of this is is quite simple. So for valence parameters This is the other part of the the bonded terms aside from torsions a lot of it just is gonna rely on force balance But so here's here's the big picture again so this in some sense becomes basically part of what we're talking about is basically part of the fitting process inside the the parameter optimizer specifically the part for Valence terms and relating to chemical perception as well so Right now where we stand in Smirnoff 99 frost which is our starting point small molecule force field We have 87 bond stretch parameters that covers basically all the chemistry we cover and just 38 angle bend parameters, which is quite small and probably that's a little too small in some cases So I think for for rings in many cases We've relied on the geometry of the ring to set or the geometric constraints on the ring to set its shape And so we have perhaps fewer angle parameters than we might end up needing But also it deals with a lot of the the duplicate parameters that force fields Otherwise have just by using the better form of chemical perception So we don't have thousands and thousands of duplicate lines So we think this means it's a really good starting point for for fitting for refitting these these parameters and in some cases Perhaps adding a few more So we aren't going to any more see these hundreds and hundreds of lines of exactly the same thing in slightly different combinations of add-on types, so We expect that a pretty simple approach is going to give us significant improvement What that is behind the graphic, but So we're going to start off by basically refitting all of the valence parameters using the current chemical perception So without changing the typing we're just going to be refitting these using force balance Well, also if we identify cases where We have major problems like we need an additional parameter for a ring system or whatever then we can modify the chemical perception by hand That could be the case for some ring internal angles where we need to add some other ones So be running lots of quantum mechanics calculations geometry optimizations. It's on for a good size molecule set to refit these and that's sort of the first phase that will come before going on to Adjusting chemical perception one so to basically there's three generations of valence parameters that we imagine So the first one will be getting the infrastructure working So we'll just be refitting selected valence parameters those that seem especially problematic or high priority without changing the chemical perception And so Vicki in a minute is going to talk some about how we're identifying some Cases that are either particularly problematic or particularly informative for refitting To prioritize for that first generation and and as you get your hands on this and start trying things out if you identify Cases you think are particularly problematic or particularly Informative we like your input Then the second phase is we'll have all that infrastructure working for fitting these and then we'll be refitting all of the valence parameters With fixed chemical perception though possible possibly some some high priority fixes to the chemical perception that would come in by hand And then the third generation would be refitting all the valence parameters with The Kemper based chemical perception so automatically determining which types we should have And so we think probably something like less a hundred or less bond and angled parameters We'll take us through the first two generations of us and and generation one is mostly what I'm gonna be talking about right now because that's most of the infrastructure work Generation two and three is basically just more of the same So a lot of the infrastructure the construction construction picture has to do with the generation one So we're gonna go over a bit of the detail plans here that just sort of roughly fall into this Generation one is refitting selected valence parameters. So we're gonna select the molecule library with particularly informative molecules Then have a to do conformer generation There's some working you to do on QC archive to get it to interface with this And on geometric and force balance and leaping will talk about that We also want to project vibrational frequencies onto internal coordinates to help with some of this and leaping is going to talk about that Then we'll refit the parameters and in the meantime, we're also building infrastructure and concepts for fitting improper We're gonna make a broader use of Proper torsions than most horses. We've been using have in the past and We'll talk about that for a minute So then generation two we're just gonna use that to instead of do selected valence parameters We need to do all of the valence parameters Also start using improper is a lot more and then generation three will be refit them with Chemical perception that is data-driven The key you want to jump in here Or the podium mic, I think works better As part of our G1 goals, we want to develop an initial molecule set in order to fit these force fields so you can imagine that if you put in a force field with only Carbons, hydrogens, nitrogen and oxygens you can't expect to use it to simulate something with sulfur or phosphorus So here this talk isn't this project is not focused on the black box the force field parameterization scheme, but sort of what molecules we want to build to put into it to develop this new force field So we want to focus on this orange circle of molecules and we want to identify particularly informative chemistry and here we're focused on Geometry structures that are not consistently represented by different force fields So this is a project that is mostly done by an undergrad in the lab named Jordan airman and Co-advised by Caitlin Bannon. So basically if you imagine all force fields in the limit of perfection Then they will minimize in gas phase to the same structure So in one case we have this dark green force field and then in force field T We have this light green and if you overlay they should be indistinguishable But in reality force fields are not consistent for all of chemical space so you might have these differences in specific origins and rings and These are the chemistries we want to find because force fields are not applicable for everything you want to simulate so the goal is to identify sets of molecules that are minimized to different geometries by different force fields and So we're not taking any specific force field as the right answer but we are comparing different pairs of infant force fields to see how they compare and And By doing this project we can develop better force fields by expanding the space of Chemistry that is covered So this is our brief workflow of what we're doing. So we're looking at two molecule sets drug bank and e-molecules We are limiting ourselves to molecules less than 200 heavy atoms and we are also not considering metals We are using the same big charge set for all of them with the a and one BCC partial charges And then with the same input structure. We have gas phase minimizations for a gaff I'm MFF 94 and 94 s OPL s 2005 and Smirnoff Thanks So two questions that you want one is How are you making sure you're getting the same minima right because because this only this is only true If you're in the global minima for the molecules concerned because that there's gonna be multiple different confirmations are available But you'll probably minimize too and then the second is is looking at the predicted optimized structure of a best option versus say Might it be better to compare the Hessian matrix? So so to look at like a predicted IR spectra or something between different force fields for the same molecule Okay, so to answer your first your first question was on How do you know you're getting to the correct local minima? So we're just starting with additional confirmations and we're just going to the local minima, so we're just basically seeing checking that the local minima will be the same and Seeing from the same starting structure if they will get to the same point I don't think that's guaranteed in some cases. They end up being different and we sort of pay less attention to those basically But we're looking for ones where they basically got to something like the same minimum, but it's different Well, would you I mean could you minimize for one falsehood and then take that as the input structure? But you then minimize would be other falsehoods to see if they move away from that minima that it's in Yeah, and then the second question was around. Yeah, is it not better to look at second derivatives rather than actual structures Yes, we could do that probably in sort of a later stage of this project Also part of this is about, you know, so the force fields aren't going to give us the right answer This is partly about figuring out which molecules were most interested or which chemistry were most interested in doing more work on in the immediate term So it's a quick way to check. I think this is a nice idea. And I Wouldn't think you'd need to worry so much about local minima at least with the sorts of structures you're showing here And also if you don't get to the same local minima starting from the same structure, that's informative, too I might suggest that you look at Allinger's work and include the MM X force fields because in his work first of all, he's he's Got a wide coverage of Interesting compounds so you could get some information from that And also he's documented With each of his force fields MM 1 MM 2 and MM 3 he's documented problematic systems So you might look you might add his problematic systems to your database and it would be very impressive if you could Eliviate some of the problems that he's been Thank you for that suggestion Okay, so the way we are scoring. Oh, sorry. It's also just came to mind the database of Minimize QM structures. I think it's GDP 9 or something like that that might be also useful as a comparison Okay, so the way we are scoring these Molecules are to you metrics one is Tanimoto combo Which is uses a combination of shape and cover color overlays for the molecule as well as something called torsion fingerprint deviation which Measures the basically the torsion finger torsion Deviation that is topologically weighted so that central deviations have higher weight than terminal torsions so we Use a normalized version of tanny of Tanimoto condo called tanny diff Where we want to look for a low tanny diff and a high TFD score So I'm just showing a few example molecules that we saw from e-molecules where These had a higher number of differences among pairwise comparisons of force fields So just showing the aromatic ring on the bottom first We see that aligning by the six-member ring. There is Variation in the planarity of the five-member ring. So here we have gaff in yellow gaff to in pink MMFF 94 which is the same structure as MMFF 94 s in gray and then Smirnoff in cyan so we see this range that is shown on the right here and With this double ring case, we have this cycle beetle connected to this cyclo-propyl And you can see that if we align by the four-membered ring these three force fields gaff MMFF 94 and Smirnoff Place the three-membered ring at a different orientation compared to the four-membered one and then finally with this compound here on the Middle image I'm turning off the hydrogens so you can better see that We again have a different Different represented planarity structure Of this ring compared to the two natural that are aligned So we are not we don't conclude. These are just preliminary results We don't really have much to conclude from them But the next step in this project is to group these molecules by chemical similarity In other words, can we say are there particular functional groups that are more poorly represented by These force fields and they're less consistent among these different force fields So from these molecules, we will build this set in order to That will lead to the next stage of generating perturbed confirmations of these molecules to Then interface with a third step with geometric and force balance to fit valence parameters So I think this is the end if anybody has any more questions The end of that part at least we can take more questions on at the end too Okay, so this is leaping, right? Yeah, so part of that is um, you know, if we're gonna do sort of a limited amount of A parameterized on a pretty limited molecule set to begin with we also like to add some value at the same time So if we can identify regions that are not well described by current force fields and prioritize those in the first round We're we're gonna know we're doing something interesting worthwhile in that first round All right. Um, oh john just a quick question. Are you saying vibrational frequencies? Is that loose there? Sorry Sorry, I'll wait till you're Okay, okay. Sure. Yeah, there'll there'll be uh, there'll be a slide or two on that um Okay, um, so uh, so in so in my portion of the talk I will describe a procedure for generating a quantum chemical data set to Fits a generation one valence parameters and um, and at least my initial expectation is that this plan should carry us into Generation two and generation three. We'll just generate more data. Okay, and um, and so um So we think valence parameters should be optimized to reproduce three types of properties from quantum chemistry calculations Well, at least these three so the so gas phase minimum energy geometries that a victoria just talked about and um vibrational frequencies from a vibrational analysis and And then potential energy scans along certain degrees of freedom where you expect the deformations to be larger So i'm just going to refer to these as soft degrees of freedom and um, and while these calculations are You know are ones that we can run on our machines or clusters at an individual scale Creating these data sets and fitting parameters on the scale needed to optimize the small molecule force field is going to Require a tighter integration of the software components. So the uh, so here are the software pieces that um that we're going to use that um were initially introduced yesterday and um And some additional coding and infrastructures needed to help the components really talk to each other and work together And um in the next few slides. I'm going to um, I'm just going to kind of highlight what still needs to be done using uh, you know using underlines, so um Um, so first for uh, um, so first let's take a look at gas phase minimum energy geometries force balance already implements a target to fit parameters to minimize the rmsd of a minimize structure to a quantum mechanical reference um and the uh, and the existing procedure is you basically you're preparing the target first which includes the um, which includes the minimize structure from quantum chemistry And then you go through the standard force balance procedure that you write the force field file with the current value of your parameters Round the molecular mechanics minimization with a very tight energy threshold Which is usually not costly because you're just minimizing the small molecule and then um, you compute the rmsd to your qm structure And you add it to the objective function And then you can compute derivatives of the rmsd with respect to your force field parameters simply with finite different and then um And there kind of is a problem here that rmsd does not properly penalize geometry errors evenly across parameter types Because an rmsd is kind of like a global metric of geometry difference If you have one degree of freedom that lead, um That that leads to the overall shape of the molecule changing like for example If you have a torsion in the middle of a large molecule that could lead you to a very large rmsd Whereas if you're wrong in a in a bond length that will lead to a very short rmsd As ill as illustrated by the by the two figures down here where On the left side you have a large rmsd and it's on the small And on the right side borrowing victoria's figure you have a small one So we want an objective function that is a little bit more representative of differences in individual Like valence type degrees of freedom so um So this target is not implemented yet, but um, but I have but I have a plan for it that is described here um, so the um, so the way the this target is going to be implemented is the following that um, The um, the top level organization is that um, is that each target had can have one or more properties and here um, and here we're going to have a have three groups of terms for um for bond lengths angles and if I grouped improper torsions into into the valence some vm properties is going to be a third group and Because deviations in bond length and angles and dihedral's all have different physical units. We um, We normalize each we normalize each term, right? So reasonable normalization for deviations in bond length being point one angstroms for example And and here I'm just showing you the first term in the sum But the second and third term are just strictly analogous to the first term So there's really no need to show them. Um, and um, and so the first sum is going to be over groups of bonds Where grouping is determined by the spin-off parameter assignment So we're going to use the force field to define the target in a sense that when we are measuring Errors in the bond length with respect to qm where you're going to put them into different bins as determined by the spin-off parameter assignment Um, and then um, and then the second sum that is inside the first one is over bonds that are within a group Okay, um and the reason why we Structure the sum in this way is that certain bonds are going to be assigned much more often than other ones like for example An aliphatic ch bond is going to appear many many more times than um, then I don't know a sulfur nitrogen bond for example You might not want your objective function to be dominated by a particular type of bond that appears many many times in your molecule set and The nesting structure of the sum here. It really is just intended to ensure that every bond type contributes by roughly the Same amount. Okay. Um, and that's and that's pretty much all there all there is to it. Okay, the Expectus implementation should be pretty straightforward. Okay. Um, next moving on to a vibrational frequencies Force balance does have an existing vibration target. Okay. Um, and the And the way and the way this works is that you still um, you start with your target preparation Which means you minimize the energy you followed by a vibrational analysis and um And and force balance is able to well has a has a script to convert, you know Different types of vibrational output from quantum chemistry programs into a common format that it likes to read and then um And then after you parameterize the molecular mechanic system You compute the molecular mechanics vibrational modes and frequencies And even though our ultimate goal is to fit the frequencies you have the problem that the vibrational modes are not exactly the same So an assignment step is performed first To assign a molecular mechanics vibrational mode to the closest corresponding quantum vibrational mode before you compute the objective function in the Which is a least squares quantity in the frequency difference Um, and and in practice using, you know a conventional by by conventional I mean the functional forms that you're that we're using we're able to to reproduce frequencies to within 100 to 200 Wave numbers and even though this isn't a very exact reproduction I think that the vibrational frequencies are mainly important to make sure that your force constants don't go too far away from physically motivated values. Um, and um um and so For the additional code that needs to be that needs to be added to make this part work Is that open mm does not natively support vibrational mode calculations? Um, and I think vibrational modes tends to be a slightly more esoteric feature in molecular mechanics codes Not all of them have it. Um amber does have the n-mode program, but I think it's pretty old Some folks in the audience there will have a better idea of that than me and then um And and so basically we need to implement a vibrational analysis for open mm, but that should be straightforward Force balance has a wrapper code around open mm in order to execute the open mm simulation So we'll just implement it in there. Yeah question if you're going to fit vibrational frequencies, um analogous to the a previous question you're obviously calculating the Hessian and He presumed so presumably you could fit the Hessian as was suggested And if your quantum program gives you analytical Hessians It may be far more efficient Then leaving them out you get a lot of free information. I mean it's taking Force balance one step further um Excellent point and we could and we could certainly implement a target to fit the Hessian The the choice to fit vibrational frequencies instead of Hessians was uh um I can I can kind of hand wave that justification, but um, but the point is that if you fit vibrational frequencies You're only fitting the minimum the Hessians Suggesting and I think the previous question referred to is you could fit Hessians all along the scan mm Those are and also just a philosophical point, but the The vibrational frequencies and Hessians are the curvature of the surface so they Are as relevant to the force field as first derivatives and energies I think chris has a coupled comment or question Uh different different comment different question. Oh, okay So do we want further discussion on this? I have a comment on that um, my my my quick my quick comment is um Is we is is we should include Hessians and there's no reason why? um Why we Would need to choose either Hessians or vibrations We could certainly do both and there's much added value to including a Hessian So we should do it I guess what I was going to suggest um is that it seems to me that by fitting to the Hessian Let's say at the at the energy local energy minimum one does avoid the mode Chorus having to decide which mode goes with which and that could get messy in some cases I imagine or ambiguous it can it can um It can get messy and I and I I agree that if you can perfectly reproduce the Hessian You've also perfectly reproduced the modes and the reason Why we do the mode decomposition initially is because it gives you some extra emphasis on the low frequency modes because the Low frequency modes come from small numbers in the Hessian Whereas the values in the Hessian are probably dominated by the stiffer degrees of freedom such as the bonds So we thought by doing the eigen decomposition you actually Um, you actually get to explicitly see what the low frequency modes look like and target them directly and that is what I think is the Added value of the of doing the vibrational analysis. So it's interesting. So I guess I wonder I never thought about this before but but if you diagonalize it, right? I mean then you can still see which You could still in principle. I suppose try to match The diagonalize components as opposed to doing mode overlays. I don't know um, if you if if you uh, if if you diagonalize the qm Hessian And then you diagonalize the mm Hessian and then you fit the diagonalize Components then we are essentially doing the and also the eigenvector We just put them in order then is that the same thing as mode matching? um Yeah, I think that I think that if we order the eigen values I mean, I think that if we order the eigen values, then we are basically doing a mode matching in terms of increasing frequency I think um, alberto had something and then chris and then we should come back to arnie. Yeah I'm not sure I caught your remark, but you don't need to diagonal if you're fitting Hessians You don't need to diagonalize and you have a direct one-to-one correspondence between Absolutely, absolutely. Let's go to alberto and then chris Just a comment. Are you looking at forces as well? Because I see only the second derivatives here and uh, yeah Yeah, there's uh, um, there's there's a there's a third target that I'm going to get to soon So what was really nice with your uh, uh previous slide about you can look at the equilibrium geometries And you can partition that out amongst the internal degrees of freedom where you're fitting the equilibrium values directly You get a direct comparison to a force field parameter With the vibrational frequencies and or even the diagonalized hessian You can know that your vibrational frequency is off or but it's much harder to attribute that to a specific parameter so How are you going to like this will give kind of a global readout on the quality of your force field It's going to be hard to focus it on the specific parameter. Do you have any plans there? Can you project that in somehow? Yeah, so this so this is not so so this is something I didn't directly think about before But there'll be a related slide coming up. I think you could translate your hessian into the internal coordinates And um, and we have a lot of the and we have most of the code in place. You're doing that I'll only we we just don't have the we just don't have the second derivatives of internal coordinates with respect to Cartesian coordinates, but um, but but but we can certainly do that and it might be a better idea I think there's also slightly a larger point Which is that like the vibrational frequencies will be coupled to the lener jones for example So you don't necessarily want to be fitting your valence parameters to get the vibrational frequencies, right? If the difference of vibrational frequencies is coming from a steric class, right? So so that's something that we have to just be aware of and keep coming back to especially you don't be this coupling And you're absolutely right if you transform Your hessian matrix into internals you can look at it and get the bun the bunch of the force constant That is the force constant matrix Except for the in some sense that it it's the force constant In a way that is coupled intimately to the the non-bonded parameters as well the lener jones or whatever Actually if you look at um, you're right and if you look at um crims work He does a very nice. He has a very nice trick Where given the non bond he subtracts the second derivative matrix of the non bond Out of the quantum hessian and then you're left exactly as you're saying With just the internals. So he reads off the force constants by subtracting off I need I need somebody on slack to write down this right down the name of this person to our valence channel. Who is it? Sam creme. Sam creme. Okay. The sdff force field. Okay, so we should look at that. That'd be great All right Let's plunge ahead then Okay, uh The the the third and let's let's see I I guess these I guess these animated gifs might not be might not be playing just because Um, okay. All right. Well, it's it's playing now. Okay. Um, so um, so the third so the third type of target is uh Is these energy scans along the softer degrees of freedom? So um, so the so the vibrational analysis is mainly intended to cover the majority of the I guess uh, the majority of displacements that you that you expect to be, you know Relatively relatively stiff where you don't get larger displacements So but you might have a but you might have anharmonic effects and coupling along those degrees of freedom Where larger displacements are are expected. Okay, so, um um So, so this is so this is just just an example of a of a calculation where uh Where jessica in in david mobley's lab provided this molecule and we use geometric to scan along an angle and an improper dihedral Degree of freedom producing this a two-dimensional potential energy map And this is the kind of thing that you might um that that you might be interested in fitting parameters to And seeing how important the anharmonic contributions might be And so um, so the initial plan is to use geometric to produce these one-dimensional or two-dimensional scans Working with qc archive to deploy the calculations at at scale and uh and retrieving the data But but if you have a large set of molecules save you have a few thousand It might not be straightforward to identify which degrees of freedom are soft and the ones that need to be skin And um, and this is where the idea of of the vibrational analysis might come into play again. So um So this is again an animated gif and um, and there and there it is. So, um Um, and and here and here the basic idea is that using the Hessian you can automatically detect the soft degrees of freedom Either by projecting your vibrational modes onto internal coordinates or by translating the entire Hessian into um Into the internal coordinates and in geometric We already have an internal coordinate system implementation that lets you do the molecular geometry optimization and um And putting in second derivatives is something that we wanted to do anyway because we want to optimize the transition states As well. So this will have a this will this will have some benefits in uh in multiple directions And this is just an example of a molecule where a vibrational analysis appears to reveal, you know, um, um A softer vibrational mode that you might want to scan along when you when you're automatically generating your your qm dataset um, okay, uh, and then um and then how are we going to work with the um with with the qc archive So um, so my so my interactions with daniel smith. I've I've developed a pretty superficial understanding of how a user interacts with qc archive so um, so so basically Um, suppose qc archive contains has defined a workflow called minimize then frequencies okay, um, and um, and so if this workflow exists then we can ask qc archive to execute it by providing a molecule and a quantum chemistry specification and then um, and then and then Qc archive will will do the computation or retrieve an existing one And then we will receive the minimize geometry and the hessian and vibrational data The next step is that you can is that we can locally run our internal coordinate code to detect the soft valence degrees of freedom um And we expect this will probably be easier than molecular fragmentation that hyah talked about that seem to be a much more challenging problem um And then in the uh, and then in the in the in the third step after we have identified Which degrees of freedom we want to scan that we can ask qc archive to do that step as well as um as long as Qc archive has that workflow precisely defined and then we will receive the minimize structures and energies um Um gradients are also really important. We will we we will get those on a grid. Okay, so for for fitting our parameters as well um And um and here are the needed uh software pieces to make to make that work um first uh First the qc schema, which is basically the standardized file format for storing vibrational data Still hasn't been fully designed then so um, and so this um And so we just need to finish the design of that file format and um and and decide what it needs to contain um We um, we also haven't really decided how to specify a potential energy scan because in a sense This is a lot easier than than a torsion drive which goes around Because the torsional degrees of freedom are Um are much more free right you can go all the way around but in But the procedure is a little bit different now because we no longer have the wavefront propagation And we are going to mainly be propagating around a minimum. So there's some questions there And then once we have precisely defined the workflows we want, um, they can be implemented into They they can be implemented into qc archive and the way I understand it is that the um Is that the definition of the workflow takes a long time because it takes a lot of meaning and planning But once it's been designed on the implementation seems to happen quickly. Um, and then um Um, and then there needs to be some codes that automatically downloads and converts this data into the force balance native formats That can be used for parameter optimization Um, and I think that is the end of my part.  We think it's going to be straightforward Here's one example of a sample Uh angle refit that I think leaping did in I don't know overnight or a few minutes or something the other day Um over break. So basically we'll be you know, pipelining whatever molecules that we end up with We'll go through conformer generation and and the workflow for Um scanning and then force balance for refitting And so we're going to be focusing on parameters that We haven't tested very much And or are identified as particularly problematic or informative um, so if if So both both things we draw from um Vicki and katelyn and jordan airman's work or things that you highlight in your own testing But because the the largest part of this first phase is just getting the infrastructure working So we want to make sure while we're getting it working. We actually provide some value at the same time um And so and we can already do this for for selected parameters This is an example in dimethylamine where basically the the original force Smirnoff 99 frost force field Around the nitrogen the angles are too stiff So you see that the mm energies are are high and then after a quick refit of those we end up with the orange um So that should be straightforward at the same time that we're also going to be working on concepts for fitting of improper's And this seems to have largely been neglected and at least the forcers were familiar with this is gaff's Improper's you might notice there's only two really There's 10.5 or 1.1 um And they're only used to ensure planarity and we think they actually should perhaps be used more broadly um, for example for partial planarity So we're starting to work through that One thing we're particularly interested in is nitrogen. So if you look at here's a small set We've put together and this is um, jessica mott. Can you wave your hand jessica says she's working on this aspect? um So she's working on this set and this is work actually vicki did just before Jessica started so this is looking at the sum of um Angles around the nitrogen center for this set of molecules and the nitrogen we're talking about is highlighted so You have pyramidal near the bottom And planar near the top and then you have this intermediate gray region where things are neither Paramidal nor planar and so depending on what chemistry you're looking at you can go relatively smoothly between Paramidal and planar And so we would like to do a better job treating that in the force field currently As at least the force fields were most familiar with All nitrogens are either pyramidal or planar never anywhere in between and they may disagree about which is which but you never have any in between And that obviously should not be the case And we have some longer term plans that can actually allow us to modulate more smoothly between pyramidal and planar um that we could talk about but Yeah, so there's a range of geometries and right now either That was exciting Force fields treat them as either or but really there should be a range that's possible in between And so we think with suitable improper as we can we can basically get the right planar for almost everything and go smoothly in between so We're going to be using geometric to work on that and this is something we're sort of getting this these concepts in place while working on The more traditional bond and angle turns So what we want to do is take a diverse set of small alkyls with a range of Paranalization we're perturbing the improper and the the corresponding angles compute geometries And then fit new force field parameters with force balance and To figure out what we're going to need to be doing on these more broadly So to give you just one quick sample. Here's one case. We've looked at already So we're looking on the left at the energy as a function of improper angle And so we have the quantum energy there in blue And what we get with serotonin and frost in red, which is rather similar to what you would get with Gaff or similar so this is a case where Gaff basically makes it planar And serotonin and frost is similar So we leaping just did a bit of geometric a scam with geometric and then jessica refit this with force balance And so after refitting we have the purple which basically Overlaps well with the mm. So this is an intermediate planarity case. So it has the two minima and Is indeed intermediate planarity after refitting So this is just one molecule and one molecule done in isolation So these aren't general parameters We aren't going to be using these parameters across a whole set of molecules But that's sort of what comes next is we head towards seeing how generally we how generally we can do this And so we think this is important because we Certainly care a lot about nitrogens So generation one is developing those concepts and then generate and fitting bonds and angles selected bonds and angles. So in generation two will be Fitting all the valence parameters we have while maintaining the existing connect chemical perception So basically the same thing is generation one but more broadly So we're going to need to ensure we have a broad molecule set that has adequate coverage of every parameter Which is an easy thing to check. It's a substructure query language So if we have Substrate have parameters that aren't being exercised. We just create a databases and pull more molecules Exercise that parameter and you can contribute your fragments of interest I think there's probably significant We could probably also use a lot of the torsion frag torsion drive fragments as molecules that we make sure we cover That's our logical place through synergy synergy um And so generating soaring quantum data for all the molecules and fragments and using force balance to refit And so at this point generation two we begin fitting a significant set of improper Because we should have the infrastructure built out for this And then generation three is a full valence refit with data-driven types. So katelyn talked yesterday about chemical perception automating chemical perception. So We're starting off with just having the types we currently have and then expanding those as needed Generation three would be starting over from basically scratch and learning which types we should have in a data-driven manner um And so the types will be now driven by data rather than either legacy types or expert generated types So to kind of summarize um we are Using minimized geometries vibrational we plan to use minimized geometries vibrational frequencies and our energy profiles potentially also hashins in fitting these and We can already refit selected parameters without any headache just using force balance directly and doing things locally But we're building out a lot of infrastructure to make this really large scale and automated and reusable and reproducible So and then there's a quite a bit of interesting science to do relating to improper I think so Um acknowledgement I don't have Jessica up here, but she's right there. So she did some of this And jordan is also not there, but he's an undergrad volunteer in the lab who's done this millions of molecules minimization project Um, and on slack. This is the valence channel if you want to talk about it So I'm happy to take more questions there. We have about nine or so minutes before break. Is it break? That's next Yeah I'm told it's on Oh, there you go Um harping back on the frequencies versus um Hessian the um, you're probably going to have what hundreds of thousands of frequencies And so there'll probably be thousands and thousands that are Uh improperly matched by accident or whatever And so if the goal is to get to something that is closer to a turnkey production of a force field from a given set of quantum data then that um Putting in the effort to fit the Hessians instead of the frequencies Sounds like it's going to be uh as a pragmatic matter If not for scientific matter is is going to be pretty important the other the other issue Leaping mentioned that you really want to fit the lower modes More carefully than the others, but you can probably achieve what you want there by uh by Increasing the weight on the one four Hessian elements Leaping you want to address that? Um Give your microphone hold your microphone Yeah, just uh, yeah Yeah, so I so I understood the the first part I might ask you to to repeat the the second part But um, so I guess so I guess we're regarding the first part. Um, my uh, the My experience using the vibration target is that the vibrational modes are never perfectly matched and then there's a Right and there and there are two ways to match them the the first way is you solve the assignment problem and then and every and every mm mode does get matched to every qm mode in the way, you know, it's the common tutorial optimum, but but um But you but you get certain modes that really overlap very very poorly And your and your objective function turns out to be very rough and hard to optimize the uh the other um The the other way is that you simply Is that you simply match the mm mode to the qm mode that it overlaps with most And in that case you get between 10 and 20 percent of the mm modes that get um that like get mapped to the same qm mode Um and and and again like that is that that's certainly imperfect But in but in practice of you know, we've we found it to work when we um when we don't rely on the vibrational data alone You know, we are doing this in conjunction with a with a lot of with a lot of uh other data such as the potential energy scans And it does tend to be along the soft modes where the vibrational assignment performs worst Right, which is which is where you're most interested, right? Uh, right So the so the other issue you're doing the um, you're calculating the frequencies on the force field version At the qm structure. Is that right? I know we minimize with the force field It is minimized. So you're so then you're comparing modes of two slightly different structures as well Which is maybe it is an additional compounding factor. Is that right? Uh, yeah, yeah, I I think that um, I I think that I think that a lot of um, I think that a lot of this Working is predicated on the mm and the qm giving you um giving you Optimized geometries optimized geometries that are very close together if you get um If you have very significant rmsd's between your um, um Like between your qm and your mm geometries and this certainly won't work Or like if you have like two different protein structures, you know, it's definitely not going to work Right, but again, if you if you were fitting the hessian you would fit at the same structure And then there's no issue of whether you're whether your uh structure shifted from one to the other And it's a you know, there's not going to be artifacts put in by that uh structure shift Like if there's a soft, especially if there's a soft potential Yeah, yeah, yeah, that's right. That's right. It makes sense that the hessian should be Should be fitted using the same structure or not different ones Um, I guess I've uh, you touched on this and I probably missed it at the beginning of the talk, but um What are your thoughts on integrating this with the Leonard jones terms because Everything's going to change I imagine when those change Yeah and so Maybe this is almost a question for john, but the the sort of big vision is that everything gets fit All together in some sense. So we have these sort of infrastructure building Phases and then there's a parameterization sprints where Everything's fit in a coupled manner And then test it Okay, so it sounds like, uh Take a shot with existing Leonard jones parameters for the uh, the first generation won't we aren't doing Leonard jones refitting quite yet So it's just going to be changing some of the existing valence So we could we could probably do a lot of Tuning of these methods independently, right? But then put everything together and the whole point of the whole project is to be able to integrate everything together And optimize concurrently all of the different components One thing we do have to talk about which we haven't brought up yet is what about one fours? Um What are we going to do are we going to do the same thing that everybody's done in the past? Because that that's one of the primary means of coupling between those and the more we can decouple them the more Easier it'll be to go back and forth between different components Yep, any thoughts or people passionate And I'm passionate about the one fours, but my question is not related to that So with regard to the improper's It seems like there's often a lot of overlap between improper's and other dihedral's that are in that ring already Yeah, right. And so when when I was fitting dihedral's I never really bothered with improper's thinking that they're they are kind of like Constants like you've pointed out But there's dihedral's on top of them that are doing pretty much the same thing So do you have any plans since the rest of Smirnoff is all about Culling excess parameters to reduce that dimensional space So, I mean, I think With the automated chemical perception, we should be able to Ask how many improper's do we really need? And and in fact, there's three right now in total In the whole force field So and you know because of well, they're mostly in aromatic rings. Although these are cases Notice none of these are cases where it's in ring Chris and then Arnie So going back to the one four's I think there's been very good discussion homing in on what's this issue of the coupling with the one four's And I just think we need to be aware that This problem is even worse than it initially appears because a lot of the Density calculations and bulk liquid calculations Focus on the distant part of the one four dominated by the one over the r to the sixth term Whereas the internal energies particularly for sterically congested structures, which I think will be instrumental in helping us tune The the one four's in the repulsive part because they will be going up this barrier in the repulsive part This is also going to be the dominant factor in the one with the one four Vanderbals as well. So I'm just you know, I think What I it is wonderful that we're aware of the issue and that we know we've got to try and solve this issue And we might have to think carefully especially since we're bringing such diverse bodies of data to try and use to co-optimize These these potential functions. So if we had unlimited time, we would clearly have a breakout session on one four's and You and Dave would we did Arnie you had something take several years to solve that problem But by overlap by overlapping the frequencies. I assume you mean taking the product the vectors One of the ways you might address the correlation between Is by using some of the more Some of the more rigorous forms of at a plane coordinate such as the um Wilson dishes And cross or the one we proposed where it's the distance if you take the distance out of the plane Of the other three atoms you're almost Yeah, I think I'd like the distance out of plane better than what we have right now. It's mathematically. It's easier too Just as far as test molecules, obviously you're going to have a whole set of these as you look at these so um I guess it's like in the urea case. You might want um, this aryl urias plus the mixed alphatic aromatic ones and then with a degree of um, electron withdrawing in your ring, so You can probably come up with a Unlimited set for those but still be able to cover it and make sure that yeah It works well across the mess. That's that's as well. Yeah