As for me, I'm Mike Hale. I've been in WordPress since around 2011 or so. I did a lot of corporate development before that, a lot of .NET, Stephanie, .NET developers in there? Never. Never. Never. All right. All right. I run a digital marketing agency called Stomp Gear. We do growth marketing for recurring revenue companies, but I'm also a freelance WordPress developer. I spend most of my time working on custom plugins for people, whether it's a direct client, or do a lot of work with other agencies. So this is going to be not necessarily plugin-based. This is a developer talk, but this is also good for people who are just implementing sites or troubleshooting sites. You don't have to actually write code, but you should be able to read it. So I said I'm a freelance developer, so this is how I spend most of my days. Just kicking back on, I think this might be women's feet, I'm not sure. My legs are not shaved, so I can't tell. Just kicking back on a beach, relaxing, having fun, doing nothing because freelancers, how many do we have? That's pretty much the life. I made a mistake this day and next to my beautiful little wooden tiki lamp thing that keeps the mosquitoes away, I brought my cell phone down to the water, and it's ringing and I said, okay, I'll get it, and it's my client. If you ever do support for a client and you just get that call and they say, hey, my site is slow. Can you fix it? You said, sure, that's what I do, I fix slow sites. You know, there are a million different reasons why that site could be slow, but you're going to have to figure it out. So WordPress, we're going to go look at it, see why it's slow. This is WordPress, by the way. Some gamers may know that's a gaming system, but no, this is WordPress.  You put something in, it spits something out. How it works, nobody knows. It's just you type in a URL and it does stuff, and it puts stuff on the screen magically, black box. For most users, that is a very good thing because that is what we want. We don't want them understanding, well, understanding how it works, but they don't have to know how it works under the hood. They know if I put in a URL, I click save, I add my content, I click save, it's going to show up on the website. Black box. Now, this is developer track, so I'm assuming people here have done some development. We know if we open up that box, this, my friends. This is WordPress. Badly coded old, this is, this obviously they have not updated these plugins. They're to the point of being rusty. Everything's just kind of tied together. The only thing, there's no duct tape in this picture. There's a lot of duct tape in WordPress. So how are you going to figure out how a site is performing? And this isn't just saying the site's slow, but that's the most common one. There could be some problem. Something might not be working. Something might not be appearing on the page where they expect it to. So how are you going to figure out in that? Well, yeah, I got a prop. What if I told you that there was a portal, a magical door, into WordPress, inside WordPress? No, it's a secret, shh, don't tell anybody out there. You didn't come to this talk, you didn't get to know it. I'm assuming this is the talk that has, wants nothing to do with WooCommerce, because if it is, you're missing a great talk next door by Lemma, so WooCommerce, come on. Who's going to buy stuff off the internet? The Debug Bar, the WP Debug Bar. So this is a WordPress plugin that's available. Just a quick show of hands. Has anybody used the Debug Bar on a site before? Okay, oh good, good. Usually, if I ask people that, it's like, the what? It is called the Debug Bar. It has nothing to do with stepping through code and debugging code, but what it does is it's going to give you some reports of what's happening inside WordPress on the page that you have it. Couple notes about this is one, this is not something that you're going to want on a production site unless you are actively troubleshooting it. If it's on the site and you're not an admin user, you're not going to see it on the front end. It will track both admin side and front end side, but there is going to be a little cost, your site was slow, it's going to make it just a little bit slower when this runs. So if you're not actively troubleshooting a site, this is not something you're going to want to leave on on a production site, because even though it's not showing it's still doing some stuff.  It'll tell you what page you're on, if there's a rewrite, some basic information like that. If there are any errors on your page, sometimes behind the scenes PHP will send notifications if there's deprecated code, or if you're using an array where string is, it doesn't cause a fatal error, but it's just not right. PHP will spit out those debug messages. If you do that, they will show up here, and you can look at those too. I do another talk about building developer friendly plugins, and one of those things is you should always run your code with debug true on when you're doing development, so that you see those things, so that when it gets to the end user, I shouldn't see any notices from your code unless it's something really strange is happening. The nice thing about the debug bar is, I said this doesn't really give you much, but show you notices, deprecation notices, fatal errors that were white screen, but there is a whole set of plugins that you can use for this as well. And I'm gonna run through a couple of these, and kind of where and when you might use them. First one is the debug bar actions and filters add-on. If you're gonna install one add-on, this is probably the one. And also just FYI, I will post the slides to the WCGR hashtag after, if you wanna see a few more of these. So WordPress, if you're coding to a lot of it, the actions and the filters are the big things. The actions are when the code does something, also run this code, that's all the hooks, that's the beauty of WordPress. And filters are saying basically the same thing, when I get this data, do something to it and then output it. So if I'm filtering a title, or if I'm doing something, when I purchase something and become or send an email, those are all the actions and hooks. I'm not sure the exact number of the average WordPress site, but it's something like 11 billion actions get called every page. Maybe not that high, but it seems like it sometimes. If you're developing plugins, it's a good idea to offer these actions and hooks, these hooks and filters to people that are using it so that they can customize your code or do things in reaction to your code. Having them in there is not a big deal. Saying do action isn't going to really cause issues, but when you have an action or filter, which are both hooks, I use those terms interchangeably, that starts causing an issue, that's where the problems come in. So the first thing this does is you'll see over here, now you have this action hooks and filter hooks part. This is not part of the regular debug. This will show you a list of all of the actions that are on that page. When that page runs, any actions that get fired are gonna be on this list. It's not the list of everything on your site, but whatever current page you're in. And again, that's on the admin side, if you're looking at it, or on the front end side. One nice thing about this is you could, if I do a lot of work with other people's plugins, and I have to modify them for a customer site or I have to add some functionality. They might do 80% of what I need, but I have to add some code to make it work for this one particular customer situation. One of the first things I'll do is once I install it, if I'm on, say it's a WooCommerce and I'm on a product page, I wanna see what all the different hooks are gonna be available to me on that page. It can just be a little easier sometimes than looking through all the documentation. It's definitely easier to looking at the code. There's one line of code that will put these on there, just do action. It works the same way for the filters. This will tell you what hook is there, the priority, which is nice. Sometimes you can see why is something firing before another. And then this is the callbacks that are actually called when that hook happens. So when something's being filtered, this is gonna show me what action is firing and then what code is happening to it. I had a site where it was a membership site and when you order a product, you get access to the restricted content on the pages and it sends an email and says thank you for ordering. And every time I was testing this, I was getting two emails every single time. And going through and I'm saying okay, I know it's happening on sale completed. I know that's where it's going. So you can put the debug bar on, find the sale completed, oops, find the sale completed and then look and say okay, what methods are being run on that hook? And then I found the second method and it was actually my fault. It was a copy of the plugin that had some test code in there but it took me a while to figure it out. But by being able to go through this, I could say okay, whenever this thing happens, what else is happening? And it was a way to get back and look at it. So that's the actions and filters plugin. Another fun one is the debug bar console add-on. This one's dangerous. This lets you edit and execute PHP code well on your site. It just gives you a little box and you type the PHP code in there. And it does format it the way that it does in the new editors now. It also lets you do the SQL server. It lets you run SQL commands in there and do database stuff. So PHP is just a simple PHP editor. You can run it, call it and then it just has an output. The only issue, it doesn't always work the way you'd expect it to. Especially when it comes to global variables. When I first put this on, I was troubleshooting another site and I had a custom post type and it had the global post. And I kept saying it here, global post and then trying to get it. That's not there. So I'm not 100% sure why. I may have been doing something wrong, I don't know. So it doesn't always work the way you might expect it to. But if you're on the admin and you wanna see what's the post meta for something without looking at a database, you can run something here. It's just a way to run code. I don't use this one much. Honestly, the SQL code editor, a lot of times I use a PHP storm as my IDE. So a lot of times if I'm looking something up, I'm looking it up in IDE. But this is a very powerful tool to do this. The other thing about this is you might have a test code page where you don't have some code written yet, but you need to add a meta field for a post type. You can do that here. You can put your update meta post field and put the value. And there's a lot of things you can do. You're probably not gonna be doing them a lot or too frequently, but it does let you. The other one with this very similarly is this gives you a SQL statement command. Again, I'll use workbench or the integrated SQL IDE in PHP storm and things like that if I'm actually looking through the database because this is very, not very friendly to look at. Oops, wrong button. It's not very friendly to look at. You can't really copy out of it, but you can't edit anything. So this will just run your SQL statement and show you the results. And you can do updates, selects, anything you need to do. The debug bar constants add-on. So in PHP and any language really, you can define constants. So constants is I could say define my plugin version equals 1.0.1. Anywhere I want to get, check that version instead of checking if it's this value, I can just check against that constant. I can't assign that constant a value after the define. It's immutable. It just won't let me, okay? So that's one where, and I know where if somewhere in my code I need to check, say I have a status code of, cancel, status code of active. I don't want to check cancel and active all the way because maybe I've changed those some days. Maybe some point in time it changes. You define the constant and then I just say check for my constant. Constants are usually all uppercase with underscores in between the names. So again, they're immutable. What this plugin will do is it will show you all the constants that are defined. This one isn't, I don't believe this one's page specific because it's basically all of them. It breaks them down into three pieces. The first one is built in WordPress constants that are defined. Apps path, cookie path, these are all built in WordPress constants that you might need to check. You don't, you know, there's not a lot of these that you're checking. Apps path, if you see a plugin that's usually the first check. If this not defined, it's probably not a WordPress site. And then you only see, you know, exit so that you can't access pages directly. It will also show you any WP class constants. So classes are just in an object-oriented way of programming, which is an option in PHP. You don't have to do it that way, but it does give you some benefits. I'm a very, you know, object-oriented development when I do it. It's just, it's a style thing. There's some things you can do with namespaces that give you some of the benefits, but it's just a kind of a clean and encapsulated approach to doing things. And this will go through all the different constants for all the different classes. And the class that debug bar constants, see, the debug bar is giving some of our own stuff. Accusement, that's, Accusement is a class. So these are all the constants that are defined within that class. It will also give you all the current version of your PHP constants. These are ugly.  The only time I've ever come across where I really have to go back and look at constants is to see if something is defined, and if it's not defined and I'm expecting it, I have to figure out why that constant is not in the list. The other thing with it too is sometimes different versions of things will define things differently. I can look back and say, okay, why is this, you know, I'm expecting this value from this constant, but it's not working. Let me see all of them. So probably not one that you would use, you know, a lot, but it is there when you need it. One of the other plugins is the debug bar cron add-on. And cron is an interesting thing in WordPress. It's basically a task runner that's at scheduled intervals or at a scheduled time. The WP cron, you can add that in your code, you say basically saying every hour do this thing or at this time do this thing. It's not always, it's not always sure fire. If you're on a low traffic site, the way this fires is if somebody comes to your site and it loads, it basically is checking to say, hey, do I need to run this cron job? If not, it doesn't run. If you have a very low traffic site and maybe you go a day without a hit, those crons aren't going to run. What does that matter? Well, what if you had a scheduled post that you had scheduled to publish that next day? If nobody's hit that site, your scheduled post is not going to publish and it's just going to sit there unpublished. And there's a plugin that you can get that will go back and do missed ones. This is good for a lot of things. It can be a little hit and miss. Some hosting companies have an external cron that you can run that will run a script for you within that. That turns it off. What it'll show you is the total number of events that are scheduled, whether it's doing the cron at that moment or not, when the next one's going to go, and then what time it is. So this is next, it's going to run this again in 17 minutes. And this, the lines that I hear what it gives you is what, or when this one's going to run. So it's hourly, daily, daily. What's going to run when we do it? Because you say add cron, you give it a time and then you give it the function that you want to run. This is the actual function that's going to run. And then internal val, I don't know what those mean. And then it argues if you're passing any arguments to it. The only time I've ever done this to see if something did run, or if it didn't run or when something is set next to run. This one is a very, very useful one. This is the list script style independence. I should take a step back and just say, when you, these are all available in the WordPress plugin repo. I'm showing about eight of them. There's a lot more, some are very specific. Some might be specific to WooCommerce or to a specific plugin they have, sometimes even the different plugins. So what you do is you'll install the debug bar and then you'll just install the plugins that you want. You don't need to run cron if you don't care about the cron. You're going to run the constants if you're not ever looking at it. I have a way I'm going to show you later where I have them all kind of installed at once. But this one is one that you're probably going to use. This will show you all of your include styles and scripts and their dependencies. So the scripts dependencies, the first three, obviously, you know, jQuery Core, jQuery Migrate. And then if your jQuery is loaded on this page and then it depends on these two. I don't know how many times, I'm going to plug in, register a script and maybe my enqueue script is just a little, you know, I have a typo in it or I have something wrong or the script isn't there or whatever. And yeah, you'll see those usually in the console, the console log in the browser. But what this is, is you say, hey, I'm expecting the script. Why isn't the script here? Or why is the script failing? Is one of the dependencies not loaded where it's supposed to be? This one is a good one to go through and say, you know, how many are there? One of the things when sites get slow, one of the big things on sites is when you have a lot of external requests. If you go to Google PageSpeed Insights for any WordPress site, and it's gonna say, you're loading all these different JavaScript files because a lot of plugins will put their style and their script and then another plugin will have their style and their script. And then you start getting 18, 20, 30 different external scripts that can be run. One of the best ways to go around that is there are plugins like W, what is it? Autopmize, auto-p-tomize, it's a weird one. But there's ways that you can put those scripts instead of the separate files, it will put them all together or concatenate all those scripts. That usually works, almost always works, but you run into issues if those scripts sometimes have things named the same or similarly, or there's a certain order or whatever it may be, there's a few times where those that concatenate a script will not work. Some of the caching plugins will do that as well. So you have to kind of try it on your local site, make sure everything works before you do any of those types of optimizations. But this will show you, so on this page, I have 22 total scripts and then 14 in cubed style. So I have 22 and 14 different CSS files on there, so that's a lot. And the other thing is the dependencies. If I know a script is depending on something and that script has not been loaded, then that script isn't going to work. The debug bar, reboot requests add-on, I like this one because I do a lot of code with third-party APIs. I write a lot of plugins that are calling some other service and if you're using what you should be like WP remote get, WP remote post, there's built-in functions in WordPress that let you call an API easily. They wrap up a lot of the plumbing around making an API call and then just give you back the results. This gives you a lot of information. If a site is slow, if a site is slow and I know I'm calling different third-party APIs, maybe their API is taking 20 seconds to return my result back. This one, it will show you that because it's gonna show you all the HTTP API, and these are, again, third-party requests on a different site on that page that are being run. Some details around it, what the address is, what the parameter was, how long it took, the total number of requests. There's also an option, it will show you all the parameters that you sent back and all the response headers that come back. So maybe if a call is failing, you can look at it and those response headers, maybe it's unauthorized. Maybe your API key went bad and you need to refresh. There's a lot of different ways you can do that. So if you are having code, writing code, or if you're just troubleshooting a plugin that makes third-party API calls, that debug remote request add-on is a good one. And again, I didn't have a good example, but this will just show there's one site I'm calling a thing on ConvertKit. It's actually coming back as a 404 and then there's all my information on it. The debug bar short codes add-on, I'm sure everybody here is familiar with short codes, it's just a little snippet in brackets that you put that gets executed when that page is run. This will show you all the registered short codes, what method gets run when they go in use as if they're in use on your current page. This is one where it'll show you a lot of them and then usage, so if you have a short code with parameters on there, it will show you how that one appears on your page. This is another good one. Sometimes you might, I mentioned before when I'm working with a new third-party plugin on a site and I'm not familiar with it, I wanna know, okay, how's that content magically appearing? Is it coming in in a short code? Yes, did they put a short code on this page? Is this the right one? Do they have the right parameters? This kind of saves me from having to go back in and look at the actual content of the page, even looking at the source code. This will give you some details on the short codes. The debug bar transients add-on, this is one that you're not gonna use a lot, but a transient is a way to save data for at most a certain amount of time. You can add it, it's volatile data. One of the most likely reasons you're gonna use this, I told you I do a lot of work with third-party API, so if I'm calling a third-party API and say that data is gonna be good for an hour, I don't wanna fill my database with stuff, but that data doesn't change. It's not like real-time stock quotes or whether, it's something that I'm just gonna save, maybe the next person's looking at it, maybe this person comes back to this page and wants the same result. I just gonna save it for up to an hour. The next time I make that call, I'm gonna say, hey, check the transients. Does this transient exist? Yes, use it. If not, okay, then make the expensive API call and come back. The tricky thing about this is you can set a time for that to expire, but it's not guaranteed to be there until that time. I ran into an issue where I thought I could save something for two minutes or five minutes or whatever it was and that it would be there, and at times when the WordPress crown runs, it can get rid of transients before the expiration. The expiration is a maximum. There's no guarantee it's gonna go up to that. Lesson learned, the hard way. Along with the debug bar, this is actually a separate plugin. And Kyle, if you guys were in Kyle's talk just previously, he talked about the query monitor. This is a super, super useful plugin. Even if you don't do all the other debug bar stuff, this is a really good one. It works like the debug bar and a lot of the debug bar plugins or the add-ons can be used within this. It kind of has its own UI. If you're using this and the debug bar, it gets a little goofy because you have the debug bar section and then you have the query monitor. But what this does, which is really nice, let's just show you how long it took total for the queries on the page. It'll show you the actual queries, what called them, so you can see things getting called, the get option is gonna be called a lot. And then the time, this was on my dev environment, but it will show you even number of rows. That's another key thing. I had a site, it was a membership site, and the homepage was really slow. It had a query that would return every member. And they had about 40,000 members on their site. So you don't need 40,000 members on one page if you're just checking for one user. So that was obvious. There was a goof in the code, but this was a good way to say, hey, why is that so slow? Okay, well, I'm returning 40,000 records and obviously I don't need them. One way to get this, all kind of all in one is I developed a plugin called the debug bar suite. It is not in the repo, it is on GitHub. It uses Composer to pull down all those different, all those add-ins I just showed you. And then in the admin, there's a debug bar suite. So you could pick and choose which ones you want on. And you can also add your own add-ons through additional hooks and filters that will put them in there. The reason I did that is in development, I would download the plugin, the debug bar. And then I would have to install this add-on, and that add-on, and that add-on. And I use about eight of them in development. I just have them there whether I'm gonna use them or not. So this is just a nice little tool that you can use. Pull it off of GitHub, put it in your development site. Probably not one you're gonna want to debug something on production ever because it does use Composer so you have to pull things down like that. But if you're interested in, this is a nifty little tool that I built. And there's a link down there. Start it, love it, I don't know. Make it better. It's open source. So now we've done all this. We've solved all of our clients' problems. That's their website.  I mean, it's so fast. I'm surprised his tongue isn't flapping in the wind there. And then what do we do? We go back to relaxing, sitting on the beach. All right, I think I have time for a couple. One or two questions? If anybody has any? I know that was a lot. I know we covered a lot. I didn't want to go into way too much detail around hood. I just want to give you kind of an intro as to what they can do. I would suggest install the debug bar on a local development site. Because we're all developing locally, right? We all have a local development environment. We're not developing on production. Just in, if you are, don't tell me. But play around with it. Add some and start getting used to it. You'll start seeing some of the cool things. This was all over the last year or two. I've really been trying to level up my development side. I've been doing more unit testing. I've been doing more code standards. Any tool that I can put in my arsenal, in my toolbox to make myself a better developer I've been doing. This one is very handy. And the more and more I get used to using it, the more time it saves me. So please check it out and try it. I covered it all. Yeah. Is there a reason you don't have your suite available on the as a workbook plugin? I didn't put it in the repo right now just because when you set it up, you have to run a composer command and what that'll do is I have it set to use the actual plugin repos once. I don't have it set up now to where like, if one of those add-ons is updated, I know because I have one that has them all. It's just not quite polished yet. It wouldn't be against any of the terms, probably next version or so. At some point, I would like to put it in there and just make sure it's cleaned up a little bit. But yeah, so I just hit, I have it on GitHub because you know, most developers can pull it down. It's really, the average user is not gonna put it on there. So, but yeah, but I might do that at some point. Hey, contribute, get it ready to go. Yes. Yeah, the debug bar should work on anything. And it helps you because sometimes if you're doing an older versions of WordPress that'll show you the PHP constants. One, that might help because it might be an older site that has an older server. It's not older than PHP. I'm sorry, this one helps too. Sometimes you don't always know on a server what version of PHP is there. You might not have access to the control panel. You just have admin access. If you put this on and you look at those PHP constants, you'll be able to tell the version and you'll be able to kind of get some information of maybe things that you're expecting to be there that aren't. But yeah, it should work across versions of WordPress. If you have an ancient version, maybe not, but yeah, it'll work on that. There's no issue there. Some of the other add-ons you might have an issue with because if they're newer and they're using something in their code, but those are kind of specific to different plugins or different things that you might see. All right, thank you very much.


You're good. Thank you, Mills. Good morning. This is a convening of the Massachusetts Gaming Commission meeting number 510. This meeting is being held virtually using remote collaboration technology. So we will do a roll call. Good morning, Commissioner O'Brien. Good morning. Just a quick congratulations to you, Interim Chair. Look forward to the insert by starting next week and all of us continuing to march forward. So thank you. Thank you, Commissioner O'Brien. Good morning, Commissioner Hill. Good morning. I'm here. Good morning, Commissioner Skinner. Good morning. I'm Jordan Maynard. I'm serving as Interim Chair of the Commission. I appreciate Governor Healy, Lieutenant Governor Driscoll for entrusting me with this responsibility. It is possibly 1104 on March 26th, 2024 and we have one issue on our agenda today but it's an extremely important one. To start, the commission has been clear that it will take all measures to ensure that a patron be 21 years old or older to engage in sports wagering in the Commonwealth. Responsible gaming is key to our mission and we want to ensure that our most vulnerable citizens are protected and educated. In October 2023, First Assistant Pat Moore called to ask if I could join a group of interested parties in working to combat this problem. It is my understanding that Commissioner Hill received a similar call from Marlene Warner from the Mass Council on Gaming and Health. On November 17th, 2023, Commissioner Hill and I met with the First Assistant Moore, Kathleen Ciliho from the AG's office, Marlene Warner, Dave Friedman and Claire Durant from the Red Sox, former MGC Chair Steve Crosby, and Mark VanderLinden was also present. Over the next five months, this group along with an expanding group, of stakeholders and interested parties, met six more times, including yesterday. This group led by the Attorney General's office, has continued to add members, and it's my understanding they will continue to invite powerful voices to the table to combat underage sports wagering in the Commonwealth, with the hope that we will be, as we often are, a national model here in Massachusetts. Given that the Attorney General's office is planning on a release and a conference to announce the group and its aims on March 28th, 2024, we thought it necessary to bring this issue to the entire commission. Brad Hill definitely pushed that issue, Commissioner Hill. Obviously, we must do our work in this public forum, given though from meeting law, but I also want to respect that this group and mission from the Attorney General's office is still work in progress. In short, the Attorney General's office wants the MGC at the table, understanding that we have oversight and regulatory authority over the legal sports wagering market in 23N. I will now kick it over to Commissioner Hill to describe where we are in the process, and to invite all of our fellow commissioners to ask any questions or weigh in in any way on this issue. Commissioner Hill. Thank you, Mr. Chairman. I don't think there's a lot more to add, except that we are at a point now where the MGC is being asked to be part of this coalition in ways that I felt very strongly, and so did the chair, that we should have a discussion with the full commission before putting our name to the coalition. The press release will be going out tomorrow. They want to add us our name to it. And I was not comfortable having the MGC be a partner of the coalition without a vote or a consensus of the full board. They will be using staff from the Mass Gaming Commission as we move forward for their expertise in problem gaming. And with the studies that have been done through the Mass Gaming Commission, which data is very important toward the goal that the attorney general wants to reach, being used and the fact that there may be opportunities for the MGC to be a voice with the coalition publicly. Again, I felt strongly that before we say yes to that, that this should be a commission-supported idea and not just the two of us making a decision on behalf of the Mass Gaming Commission. That's what brings us here today. What I'm hoping I can ask of the commission is that we will be involved with what the attorney general is doing and that the MGC be part of the coalition and allow the commission staff to be part of the coalition so that we can eventually get to the goal of ensuring that our youth are not being hurt by gambling and so much more. So that's what brings us here today, Mr. Chair. And I certainly threw you, if there was any questions from my fellow commissioners, I would be more than happy to take them at this time. Happy going on the floor. So I don't have a question just so much as just adding to the conversation. I mean, obviously this is something that in my time in the commission is of paramount importance. I think we all feel that way, particularly with sports betting, particularly with sort of the disconnect as well between MCA surveys and what we're hearing from the licensees. Commissioner Skinner and I did have the opportunity to get somewhat of a background briefing yesterday with a little more understanding, obviously in compliance with OML about the last several months. So this to me is a very exciting opportunity. I think that I absolutely, it needs to be all of us obviously signing on as a commission. I mean, that's how this works. So I appreciate the bringing of it forward. The only comment that I have that I do think bears stating publicly would be because this is in the working stages, there may be concrete steps absent of us either staffing, funding, et cetera. And I just want to make clear while I embrace this process and this joint venture, I don't want anything to be viewed as binding any future vote. And I'm particularly mindful of procurement loss where nothing in this and what I would expect will be a positive move forward by this commission today is anyway viewed as impacting anything that might come back in front of us on this. I just, I think for the record, it was clear to make that statement, but otherwise I'm in full support. And let me be clear, Commissioner O'Brien that if there's any request of funding or anything else moving forward, we will bring that back to the full commission for discussion and vote. We will not be making those decisions on the commission's behalf. We'll always bring it back for a vote. Open the floor, Commissioner Skeller. Thank you, Chair Maynard. I am also in full support. I don't know in what world here in Massachusetts, the MGC would not be signing onto this coalition. I certainly support what it stands for, particularly along the lines of consumer protection of our youth. And I think that MGC's participation makes eminent sense. And so I know that there are details that are still being worked out. Any advisory group that might be formed what the actual curriculum will include and like Commissioner O'Brien, I look forward to hearing additional details about those items and beyond as the work of the coalition moves forward. But I think it is a fantastic idea, fantastic opportunity for our leaders here in the responsible gaming room to make their voices heard. So thank you for your participation in this initiative and bringing it forward for us to consider today. Thank you, Commissioner Skeller. With that said, we did have this marked up for a vote. I didn't know if anyone has anything else to add to the conversation or anything they want to see in here, but happy to hear that. Mr. Chairman, I am going to request something else after I make a motion to approve the participation of the youth. I will be asking one more thing afterwards. Okay. So with that, Mr. Chairman, I move that the commission approve participation in the youth sporting betting safety coalition as discussed here today. Second. Okay, we'll do our roll call. Commissioner O'Brien. Aye. Commissioner Hill. Aye. Commissioner Skeller. Aye. And I vote yes or zero. Commissioner Hill. The second issue I'd like to bring to the attention of the commission is during the press conference that will take place on Thursday that the Attorney General's office is putting together, there may be a request for someone to speak on behalf of the MGC. And I was going to ask if my fellow commissioners would mind if I was going to be the voice, if asked. To participate in the press conference. Court down. Yeah, I think that's right. I also support it. I don't think we need a vote, but I just wanted a consensus if that's okay with everybody. And I will state that this is still very much in flux, right, the work in progress yesterday is when the decision was made that we might be asked to speak. So it was batted around for a while, but it seemed like it could actually happen. So I appreciate you, Commissioner Hill, and your willingness to represent the MGC in that capacity. With that said, anybody have anything else for this meeting? Any commissioner updates? I will entertain a motion. Motion to adjourn. Second. All right, let's do roll call. Commissioner O'Brien. Aye. Commissioner Hill. Aye. Commissioner Skinner. And I vote yes or zero. Thank you everyone and appreciate it and look forward.


First of all, I'm really humbled that you have a lot of good sessions to attend to and you decide to come here So hopefully you'll get The things I'm talking about and hopefully you're gonna understand how AI is going to be used in software testing my name is Rod Subramanian and a little bit about my company and what I do So I work for a company called testim.io TSTIM.io What we do is we have an AI based software testing tool which helps to do front-end testing UI testing and end-to-end testing and Coming to my role Just like Gil who was giving the presentation before I'm a developer evangelist Which means that I write about software testing speak about software testing and also Work with the founder of my company to make our tool better for our customers, right? And also work with the customers directly to help them use our tool, especially when it comes to code and stuff And I have a background of Selenium. I did seven years of cucumber Java and Appium So I kind of know the framework pretty well. So that's about me So today I'm gonna Cover these topics as part of the stock, right? So I'm gonna talk about what are the current challenges with software testing then how AI can help to solve these challenges and Before getting to how I can solve these challenges. I may cover the basics of AI. I Know in a previous presentation Gil mentioned a couple of stuff about Basics of AI, but I want to make sure when I use AI terminologies. Everyone is on the same page When I talk about AI deep learning machine learning and stuff like that and Then finally I'm gonna talk about the future of AI and there's some other things Those are some other things which we are currently working on as well So for let's set some context, right? So even before talking about what are the problems AI can solve It's really important to understand how software Testing has evolved in the past few decades, right? So I know for people in the end may be hard to see but In the 1980s and 1990s period, right? testing was predominantly manual testing and we had teams Which used to deliver which used to build the product for like six months and then finally they'll give it to the tester saying hey test this right there. So pretty much the development methodology was waterfall and What happened was there's a lot of wastage of time costs and effort because what the test will find a bug and Then once he reported a lot of rework Had to happen and then a lot of changes happened and because of that people started missing deadlines And the customers didn't get their product on time, right? So that's how software testing was in 1980s and 90s and Then from 1990 to 2003 we started seeing these bulky automation tools Which started cropping up right testers were super happy that and finally we have something which could help us in testing But the problem was they were so bloated and it was really hard to understand what they're actually trying to do right and it was a steep learning curve and At the same time We started seeing people experimenting with different development approaches right like rad which is rapid application Development and scrum XP and those kind of approaches. So that's what happened in the period of 1990 to 2003 Then we come to the 2003 2010 period this was the period of Innovation I would say because The open-source framework started during that time and there was this group of like-minded people who wanted to Solve different problems with software testing right and and what they encourage other people in the community to Help them solve problems with software testing. So one example of an open-source framework would be Selenium so that kind of started during that time right and and so We started seeing a lot of these Open-source communities forming during the 2003 2010 period at the same time people got tired of waterfall way of Developing software right so agile became huge. So we had scrum XP and people wanted to release Software fast, right? They wanted to write this fast Find bugs faster and really really really fast. So That was 2003 2010 period and then we had 2010 to the current period which we are in right now where Everything's about scale. So whatever I mean by that. So as The software is becoming really complex, right? there's a need for a lot of Server space faster processing parallel execution and And we have so many resources which you need right as your Testing complexity increases. So in this era from 2003 till now There were two things which have made a big impact in our software testing industry So one is crowd testing where people Regulates that collaboration was really important and what they did was they give their Applications to other passionate testers outside the company to get feedback about their application, right? so crowd testing when was one big thing which started and Another thing which happened during the same time was cloud testing because we had server space Faster processing need for parallel execution. We have sass left and other Amazon cloud and all those people have come into play So those are the two big things which have happened. So that's the current state, right? Now it's all about the future. The future is going to be autonomous testing using machine learning and AI It's not going to be about static locators It's going to be about dynamic locators, right? And I'm going to talk about that in a bit I'm going to talk about that in detail with the demo as well and and it's all about Increasing collaboration making automation simple smarter faster and more stable. So that's the period We are going to see and that's going to be the future, right? So it's important to get this context and it makes sense as and when I go through the slides So let's talk about the current challenges with software testing, right? So recently we did a survey in them we asked people how frequent how frequently they released software, right a majority of them They start testing early, but they keep testing, right until the last minute and because of that a lot of times Release cycles are getting affected and then they're not able to release software and time, right 57% were Responded to that and Finally at the bottom we see 2% of the people who are releasing every minute of every day, right now If you're going to transfer this into a graph this how it's going to look like So we have the laggards in one end Who release software every six months, right? And then we have the innovators on the other end who are releasing software really frequently every day, right? So let's talk about challenges in each of these phases and then you're going to understand how AI is going to help to solve These challenges in each of these phases, right? So the first challenge is the challenge of skill set so people Who are in the laggards phase who release only every six months? They have this challenge of skill set where getting skilled testers is really hard and and and it takes a lot of time and it's very expensive, right and That's because of that challenge Most of the testing is manual and it just takes so much time for them to release a product, right? So that's the laggards phase then you have the late majority phase where There are people who release product Who do the releases every month, but they have this huge challenge of authoring, right? So what do we mean by that? These are organizations who have testers. They have some skill testers and they start authoring tests But they've failed to notice and pay attention to good practices of automation like reusability low coupling Parameterization and all those things which you need to keep in mind while doing automation They fail to realize it and because of that there's a lot of problems and they end up when you're releasing once a month then you have the early majority of folks who Majority of them who have released once in two weeks and they have this huge problem of initialization state, right? So what I mean by that say You've written dentist right now and then it's all passing your super happy yay, right? Then next day you come only to notice that all the dentists are failing right now So what happened the problem was the state of the application actually changed and now all the dentists are failing, right? That's because a lot majority of those problems are because of the initial initial state of the application, right? So say for example of Amazon and you have an automation script which says you have to add items to the cart, right? so you write a automation script which add items to the cart and Awesome the test pass now you rerun the test the same exact test. What's gonna happen is It's going to add another item now There'll be two items in the car you rerun the test three items in the car four items in the car They keep going right people fail to realize that you need to make sure the app is in the initial state Like you have to have zero items in the car before running your test again, right? So similarly log in scenario log out scenario once you log into your application you may want to make sure you log out Right, so people fail to realize this initial state which causes a majority of your problem, right? So that is the third channel force challenge is maintenance and that's the biggest problem You're facing right now, but we just test maintenance, right? So Say now we have 500 tests, right? And it takes about five hours to run it and then only to find out that like hundred tests are failing Now what what do you do? You actually have to go look at every freaking test Which is there then troubleshoot it put a fix and rerun it wait for another five hours So that's this huge problem of maintenance, right and a majority of the people Have the exact maintenance problem and that's one thing which I'm going to talk about how we Are trying to solve it and how other companies and trying to solve the maintenance problem using AI So I'm going to talk about that and finally scale I already talked about scale where we need service space and you want to run more tests as frequently as possible So these are the current challenges of it software testing and test automation in general So I'm going to talk about how AI is going to help to solve all these challenges that I just talked about So before I do that, I wanted to cover the basics of AI just in three four minutes because People are mystified by this word AI they use AI machine learning deep learning And I know in our previous stocks that they mentioned a couple of differences between these two But I want to make it simple, right by no means I'm an AI expert But it's really simple if you understand the concepts. I'm going to just cover that quickly so You have this huge circle, right and The outer circle is called artificial intelligence this huge space. So what is artificial intelligence? It is giving the ability for machines to work and react like human beings Right, that's artificial intelligence Under artificial intelligence You have machine learning, which is a subset of artificial intelligence. So what is machine learning machine learning is the field of study which gives the Machines the ability to learn without being explicitly programmed, right? So that is machine learning and and and it evolved from the study of computational theory Which is study of learning algorithms and pattern recognition. So that's how machine learning evolved. So that is machine learning, right? And Now whatever you hear in the media and for the past 40 years about autonomous cars and everything related to AI a Majority of them is related to related to deep learning Which is a subset of machine learning and and so what is deep learning again? Some of you may already know but deep learning is mapped to the neural network of human beings So you have these different neurons which starts interacting with each neuron in your human body and then gives reactions and Makes you act in a certain way because all these new new runs are learning from each other So they took that took machines and that's what is deep learning, right? And all your autonomous cars and whatever you're hearing about right now is all deep learning to give you a very simple example of how it works so Say for example You want to you're giving the size Up a house you want and the price you have to pay for it and once so you give this input and then Price is the output. So what the input output combinations called a training set So whenever you meet any AI folks, they're going to talk about training set is just giving input and output combinations for the AI To learn to train the AI right so say yeah, you give the size and the price and you want to see what the AI comes up with, right? so First you give the size and the price and then you give the number of bedrooms, right and the price Then the AI starts figuring out, you know what family size is something you want to think about based on the inputs You have given and then you start giving zip code Then AI is good and determined AI is going to find a relationship called how walkable, you know that area is right It's trying figuring out based on all these input-output combinations Then you give the wealth and the price of the house then it's going to figure out the school quality so So all these things in the middle Those that's what the AI does and it's a black box and I'm just giving a really simple example of how we could figure something out But that's how it is So you give all the inputs and the outputs and then keep training the AI and then you're going to find a lot of Relationships, right? So that is deep learning and that's how it works and Your autonomous cars that's an example for deep learning and these autonomous cars you're hearing it in the past three four years, but The first research was done way back in 1989 at Stanford University and in fact they had a Video them training an autonomous car The car's name was Alvin. So this is a video from it to if you see Learned to steer by watching a person drive Alvin is designed to control the nap lab to a modified army home V equipped with sensors computers and actuators for autonomous navigation experiments The initial step in configuring Alvin is training a network to steer During training a person drives the vehicle while Alvin watches once every two seconds Alvin digitizes a video image of the road ahead and records the person's steering direction 30 by 32 pixels and provided as input to Alvin's three-layered network using the back propagation learning algorithm Alvin is trained to output the same steering direction as the human driver for that image Initially the network steering response is random about two minutes of training The network learns to accurately imitate the steering reactions of the human driver pushes the run switch and Alvin begins driving 12 times per second Alvin digitizes an image and feeds it to its neural networks each network running parallel Produces a steering direction and a measure of its confidence in its response The steering direction from the most confident network In this case the network trained for the one-lane road is used to control the vehicle as the vehicle approaches the intersection The confidence of the one-lane network decreases as it crosses the intersection and the two-lane road ahead comes into view The confidence of the two-lane network rises when its confidence rises the two-lane network is selected to steer Safely guiding the vehicle into its lane on the two-lane road This video is published in 1992 and we just hear about autonomous cars for for the past four years, right? So the reason I was stressing on the confidence level and The data we provide to the AI is because I'm going to show a demo of how People are using AI and software testing and you're going to see all these concepts the exact same thing reflecting in the tool as well So I just wanted to give you an idea that artificial intelligence was not something new So for you to To see how you can train an AI in live There's an open source application called quick draw has have any of you heard of quick draw from google So That's the best example of how you can actually keep training AI So based on the things you draw the AI keeps learning, right? And that's the best example for you to know how an AI actually works. So I have an example for that too so Once I get out of this so it's called quick draw and if you go So this how it's going to look like so you're going to draw different things and train the AI and anyone can do this this best example of deep learning here, so You click on let's draw now. It's a draw a guitar, right? Or garden hose or belt or speedboat. I see key. Oh, I know it's guitar So you're just praying you're training the AI right now. Now you have to draw a crab That's going to be a tough one actually. I'm really bad I see circle or blueberry or bracelet or teapot. I see frying pan or see turtle or spider I'm stumped See so yeah, my drawing is really bad Sorry, I couldn't get it Now you're let's try a pair I see moon or here or circle I see apple or cherry How do you do a pair of minks? I'm stumped. Sorry. I couldn't guess it. Okay, I can drop pencil for sure I see lime or hockey stick or magic wand or stream bean I see the Eiffel tower or wishbone or baseball bat. Oh, I know it's pencil exactly, right so So that's that's AI live in action, right? So that's an open source drawing tool and Google that so stumps you gets all this freaking data because now I'm using it for the presentation So they keep getting data for free and they're training the AI, right? So just want to give you an idea about how an AI works and what are the concepts behind it, right? So Now that you have this idea of the different challenges we have and how AI Works and what AI is so I'm going to talk about how we use AI in software testing, right? So Like um, there are different companies trying to do different things using AI But one thing remember at the start of the presentation I was saying saying the future is going to be moving from static IDs to Dynamic locators. So I'm going to talk about that for a bit. So I test them in my company. We use what is called dynamic locators and what that means is um We get multiple attributes for every single element using AI which is working underneath the hood, right? so for example say um, you have a login button and you write a selenium script and say I'll click on login, right? And uh, you're super happy and the next day you come and see the test fail because some developer Just decided to change the name of the login to summit and the test fails, right? And you're super pissed you have to take time to troubleshoot the issue Then you find the problem and then you rerun the test, right? And then there's so much time Waste over there. So that's where we bring the concept of dynamic locators And that's going to be the future of software testing and that's what people have started using So for the same example of login button, what happens is This AI underneath the hood It's going to go through the DOM and extract all the object trees and properties and create a list of location strategies, right? So for the same example of login button If you're going to change the name of login to summit the test is not going to fail The reason is we already have a list of location strategies which the AI created So it's going to say oh, wait a minute the name change, you know what? I'm not going to fail the test. Let's go to the next best location strategy Which could be ID, right? If the idea changed you'll say, you know what? Let's go to the next next look Next best location strategy, which could be tagged Then see I said keep going on and on because for every single attribute it gets Every single element it gets multiple attributes, right? That's what dynamic locators is And that's what we use on our tool So to show you a live example of how it actually works And this is how people are using dynamic locators in their testing, right? So Let me Take an example here Okay, so this this how our tool looks like let's just start creating a test, right? So I'm going to create new Um, I'm just going to start recording a test So I do refresh because I think I'm having poor internal connection. So give me a second and I know with live demos There's always There's going to be a hit or a miss. So I have a record of video too. So I come prepared But you know what? Let's uh Just try this out Because I know everyone is on the wi-fi connection right now. So It's taking some time Cool. Okay So right now we have the editor. Um, let's see whether it works now. So Cool. So I'm just recording a test, right? This is a demo application I created and I'm just going to click on login Enter username password click on the login button Um, I'm going to select dates from a date picker Just because I can and then um, I'm going to click on the select destination button and then In the bottom you have a book button. I'm going to click on that. Um, enter my name Email address and I think that should be Enough for now. So so as you can see, um The ai underneath our platform It actually records all the steps with the screenshots and actually displays it over here, right? But coming to the part which I want to talk about is the dynamic locators which I'm talking about, right? So say for example, let's take the select destination button so Here You can see the list of location strategies, which All the DOM elements have been Parse and then displayed over here for the select destination button, right? So what the ai has done is in real time as in when I was interacting with the element it just just Parse all the attributes of that element and displayed over here So you can see the text attribute um indexes then you also Have class tags and everything there's multiple attributes of the same element and Remember I was talking about the scoring in the video that because we use a similar kind of algorithm here I'm hovering over the stars next to each attribute and The ai says that the score for the text attribute is point nine, which is the highest you can get So it probably is going to use the text attribute to locate that element But say if we change the text to something else Then it's going to go to the next special location strategy Which could be class or type because both have the same scoring, but it makes the decision in real time so To show you a live example of Even if you change an attribute the test will continue to run because of the ai so i'm going to do something over here um So i'm going to put a break point next To the at the point where we click on the select destination button before we click on the select destination button And then i'm going to change the attributes of select destination And then i'm going to show you how the ai actually smartly figures this change. So i'm going to run this test It's going to run the test Until the point is going to click the select destination button. So it's going to click on login Enter my username password click on the login button Go through the dates from the date picker Cool and then now the test reach a break point because i haven't clicked the select destination button yet, right? and So now you see this loading icon, which means that okay, there's a break point. That's cool. So now Okay, so now i'm going to manipulate this attribute of the select destination so i'm going to do an inspect And don't worry if the demo phase again i have a recording too, but um I'm going to Whoa, what do you guys see? Okay, so i'm going to Move this button Say i'm going to move it to a location So if you see i just moved the select destination button from here to here and To make it Better i could do another thing I'm going to change the name of the button from select destination to Selenium conference 2018 right okay So if this was a situation in most of the Automation tools you use what's going to happen when you run the test It's going to fail right exactly because i pretty much did every freaking thing possible with this button so that It should surely fail, but now see what happens Then i run this test right so i'm going to click on play And there you go it just still keeps running It's going to continue the test enter my username email address that just keeps going right and Let's see what happened underneath the hood so Again going back to the select destination button So i don't know whether you guys notice but before it was 100 confidence but now the confidence level decreased to 83 percent because i manipulated the attributes and What actually happened was The since i changed the text attribute and the location It went to the next best location strategy which could be again class or tag and then it made the decision real time right and And it takes screenshots and actually shows you that hey You have this button but the attribute changed but the functionality didn't change so i continue to run the test But i just want to let you know so So it's telling you with screenshots so for example the baseline image was select destination, but What it found was selenium conference 2018 right so that's the beauty of using dynamic locators so when once you extract multiple attributes of the same element right so the test becomes more stable and and and When you have 500 tests and next day you come you don't have to worry about like oh man 250 tests fail and Now i have to put 24 hour days to actually fix it right so you don't have to worry about it So that's the beauty of dynamic locators and that's what people are using right now To and that's going to be the future of testing people are going to move from static locators to dynamic locators and Everything is possible all this is possible because the ai Underneath the hood so it's a good thing at least for this automation right so That's why i just wanted to show you a live example of how dynamic locators work So that's going to be the next best thing So this the coverage gap in software testing right now so the complexity Of different features of the software are increasing exponentially But the tests are growing only linearly right so we have this huge gap in between and that's what ai can help to solve right so That's something you want to keep in mind Now coming to the future of ai and some of the things i'm going to mention We are actually Trying to do in our company right now But again, it's a work in progress because as i said ai is a black box We need to figure out a lot of stuff about It's reacting to different put output combinations right so Let's first talk about autonomous testing so What's going to happen is and what we're trying to do in our company and i know a couple of other companies are doing the same is We're going to connect our production apps to our software testing cycle right, so what does it mean so The ai is going to start observing different flows which the actual production users Are trying to do with your apps and it's going to create tests based on the actual flows right And it's going to create tests for you and it's also going to identify Identify repeated steps like logging in logging out and it's going to cluster them for us it's going to create reusable components automatically for us right and We do it to a certain extent right now but we still Want to get to the point where we can start analyzing production User flows and create this and finally you have this test which are created Based on actual production data and that's autonomous testing and that's what everyone wants to get to right and and This what we are trying to do right now with using ai so the next one is ui tdd so everyone has heard about a tdd bdd all the dds out there, but there's one more Tdd to make it more complex which is called ui Test ui based test driven development. So what do we mean by that right? So what's going to happen is so we have these mock-ups and And developers so in general at least in companies have worked Worked in for the past 12 12 years We usually have mock-ups and then the developers will look at the mock-ups and say oh, this is a feature Okay, let me start developing right now with ai what you can do is while the developers are developing the feature based on These mock-ups the ai can look at these Images and start creating tests for us right based on the features so You're gonna so the ai is going to create the test based on the mock-ups The developer is going to develop the feature using the mock-ups and then finally Once the feature is developed You run the tests which were already created by the ai and make sure it's green right? So now you have ui based test driven development right and currently we are working on this aspect Of trying to do ui based test driven development right it's super fun, but it yeah, we again need a lot of data Probably I should do something like quick draw and make you guys do something so they can collect some data But the point is yeah, this is another thing which we are trying to do right now so Finally another big thing is automatic response. So every time You make a change in your ui say for example You change something in the login flow in the ui right you don't just go change just to test the ui you also want to make sure that Responds and request response to servers and database still continue to work because the developer would say No, I didn't change anything But of course we as testers Don't agree to that and we would want to test that whole end to end flow right? so See for example something changed the login in the ui right? Before what you have to do is you have to wait for the server To process the request it it will probably Access another database maybe and then give the necessary information and then in the response finally we get in the ui And that was a part of that was a little time consuming aspect of testing as well, but with AI what's going to happen is The a is going to record all the server responses and requests and you're going to mark it so You don't have to go all the way from A server to another server to another database. Can they get the request? Get the response back and back to the ui. You don't have to wait for that Even that time can be saved by marking Recording all these server responses and then based on whatever you do It based on the marked responses It you immediately start getting feedback right and it makes your test Much much faster, right? So that is something which People are working on currently and that's one of the things we are working as well using AI so How companies like us and other people are Trying to use AI to you know, pave the way for the future, right? So our main focus is on Solving the challenges which currently exists this automation. The main thing is the maintenance aspect of it So that's what we're currently working on but also Autonomous testing helps to eliminate the quality velocity dilemma. So what do I mean by that? So in terms of quality So you are now you've already created all the user flows based on the actual production data, right? So you're maximizing user coverage because the you're actually mimicking what the users are really doing, right? So Now the all of a sudden now QA has become a science and we make decisions based on all this data, right? We make risk based uh based be based on based our decision based on risks and data and and The biggest thing is you're already ahead of the curve because The what the AI does is it proactively? Finds problems and keeps fixing them in the background even before it comes to Comes to the screen or you see it, right? That is because of the self-healing mechanism of the AI because it it can it just keeps doing its own thing, right? And we can we don't have to worry about a lot of stuff So that's why it prevents bugs as opposed to fixing them Then coming to velocity aspect So we create much more scenarios in a short period of time, right? Because all these Tests are automatically created for us. Of course, we has human testers. We will go and check whether that's why you actually need it Right? That's why there's always a human who needs to be there. But the point is you can create much more scenarios um He can create this proactively by monitoring production apps and It's easier to do rootcast analysis as well. So Those were pretty much the things I wanted to actually cover um, hopefully now we got an idea about what the basics of AI is how people are using AI For software testing what are dynamic locators and how the AI actually Used the dynamic locators to make decisions in real time. So that's why you Everyone is seen right now if you need more information. I have my email address and everything over here and uh before the stock I already Wrote a summary of all the things I talked about and also have extra information about tools and programming languages people are using in this blog post so once if you just go to testim.io And click on blog you should be able to see that or if you have this Slide if you just click on it'll take you right there, right? And before I end the session, so I wanted to do a couple of things, right? um I know we have time so What's the time? Okay, we have five minutes left. So I wanted to do two things, right? um first thing is I Never realized that There'll be such a big crowd, which is phenomenal. So i'm going to take a selfie and you guys should say hey or something and just want to Prove the people that they were actually people in my talks. So Say cheese Okay That is one thing and another thing is I got some freebies Um, so we have uh testing t-shirts over here. So be free to take them and I didn't again Didn't know there were these many people so it's like a first come first serve basis And I have one request though once you take the t-shirt I would like you to just wear them and then we'll take a group picture and then we can blow up twitter as well so um So that's pretty much it and thanks for sitting there patiently and thank you and I'll take questions right now So Hi rash, uh, my question is that you were talking about the dynamic locators So in the application that I test what happens is uh, we have elements and most of the css the developers what they do is From the css, uh, everything is almost same right and the id is dynamic So in my application if I have two buttons with the same name just their position is uh different, right? Are you getting what I'm trying to ask? So, uh, if the if the buttons are placed differently, but the css and the x path almost it's same Just the position is different and the id is dynamic. So just as you saw in the demo, uh, somehow the developers somehow, uh, Interchanged the the location of the buttons interchange So the functionality of the buttons is different. So in that case, will it detect that the buttons has changed or will it click the same button? I mean because the the position of the buttons has just interchanged, no Right. So just to make sure, um, I got the question, right? Say you have four buttons called summit and all four buttons have Exact same css Tag names everything's the same even the id's are same. No the id's are dynamic. So we cannot use id's but and the names are also Same. Yeah, right and so somehow the two of the buttons got interchanged by uh, because of the developer or some Some mistake So in that case, so my question I Coming I'll answer that but my question was if you have four summit buttons Who actually cares if it actually changes it's the same button, right? No, no I mean like the name is same just it's just an example. The name is same but the functionality is different Yeah, the functionality of the button is different. Yeah. Okay. First thing is I would say that's a terrible coding there Um, I I don't know if they were in the last session, but we were talking about product managers and business people so First that has to be really fixed no matter what tool you use And next thing was coming to the ai aspect of it. So what happens is Yeah, as in when you keep running the test it starts gathering data if it sees that okay all these id's are changing dynamically then it starts learning, right and Again only based on the amount of data and amount of times you run the ai is going to learn So After a certain period of time it will start detecting the detecting the changes and start self-healing Which means that it's going to detect all these changes even though the id's are dynamically changing and everything is the same So it is going to becoming smarter It'll become smarter and start trying to identify all these changes and then work properly But even if you do use ai it's going to take some time for it to learn But I would first say is go to your developers and business people to actually change it But to answer your question is the ai can do it, but it will take a lot of Test runs and then more data to do it, right? So only once we have That scenario and keep running it then we'll start learning about stuff. So hopefully that Kind of answer your question because it's hard then is there any plan like if you If you click any button then the page that renders after it. I mean, is there any plan of you know, is there any way?  I mean what happens after the button is clicked the dom that is rendered Maybe that can also be used later on to identify. What is the functionality of the button? Great. So we already do that. So for every element we interact with we take screenshots and Ai already knows what's the next thing which is supposed to come So for example, you're clicking on the same example summit button and say a different page was supposed to Display it already has taken the screenshot and if it's failing it's going to keep looking Why is this page not coming right? So let's start learning. So the point is that is already implemented So it is going to detect that dom change and make sure the right page gets displayed based on the click Thank you. Yeah, thank you I'm sorry guys. We are out of time. Oh, I'm so sorry But raj is going to be here and we can definitely reach out to him Oh, yeah, and their t-shirts here and we're going to take a picture as well once you get the t-shirt So and I'm going to stay here. So feel free to ask me questions. So thank you. Appreciate everyone. Thanks raj