ime. Yeah, this is every time. Ideally, this is being reiterated every time. Yeah. We're losing, I'm losing it a little bit on the dialogue. So Eileen, if you could finish your question and then- No, I was just saying, so this is not, I guess when I first read this it was more, I thought it was a more one-on-one.  This seems more like this is a broad stroke to the table before the play or the role or what are the deal and it applies to anybody who may fall within outside the mid max. Anybody, anybody and then, and as I said, the dealers will, anything close will say it just to say it to make sure to cover that. You know, if they're, if they're, if they're, we all make mistakes, right? So if this is to cover the mistakes and that's why there's an A to the B for that mistake but as it is, it's just a pure mistake. So the best way I can say it and maybe I'm reiterating what everybody understands is that the announcement that is contemplated in 3A is made every game. That a wager looks or appears to be close. Yes. When it, when it looks like it's a big wager that could be over. So it's not every game. No, so let's say normally, like, let's say the table max is a thousand. It's a $25 game. You're seeing 25, 50, 100, $200 wagers. We all can tell that, you know, that's not close to the table max. So nothing is said, but then as Burke said, it gets hot.  It looks all fun. That's when they say pay or take the table max. Yeah. And then the default is if somehow he didn't, or she didn't see that and it was over a thousand and that was the limit and the patron loses, he loses the entire bet. And if the patron wins, patron gets the game. The entire, gets the entire wager. Yeah. And then there's also Eileen on the felt, I believe I'm hearing that there's also additional instruction as to what the max is. The sign, yes. That's usually an electronic sign, Madam Chair. It's, yeah, it's electronic. And sometimes is it on the felt too? Of course, I love it being right on the little box. Because it's in max depending what time of day and what the business is like. Oh, okay. I get it. So that's why Brad, you're pointing out it's electronic. Yeah. It changes frequently. It changes frequently. So only certain games are static. Yeah. All right. It's important to remember that the patrons that are getting up near a table maximum, they're pretty experienced. Yes. Betters, they understand what happens. They understand in a discussion that what's being said. And there's been instances, for example, what if a $500 chip got stuck in between a bunch of black chips and no one saw it, right? Afterwards, a patron could file a complaint against the casino. They would talk to the shift manager. They would make arrangements with it. You can't do that every day, but they would work with the patron if he's a good customer and talk to him about the problem. The experience of the gamblers that are up near the table maximum, they understand what's going on. It doesn't happen that much. We're just trying to find language. And as we were talking with Kerry and Judith, we were having a lot of fun talking about how to explain co-mingled chips, what to do with this and trying to get the proper words. Barber pole chips, for that matter. Well, yeah, yeah, that's all. You're pulling at a barber pole and it's just impossible to see what that total is. We're just trying to clarify it. The train dealer has the eye. They make this language. It's not for every game. It's only when it nears the trouble. I get this. The language, to Judy's credit, with all of your help, it's clear. Well written. It's definitely a team effort, Madam Chair. Brad, your clarify was really helpful. I don't see everyone. Do you have any questions? Commissioner Cameron or Commissioner Hill for Judy? I do know, but this was very helpful to understand it better. Yeah, there have been some questions that I've had earlier on some other matters that sort of raise this consumer protection issue. And I think, Commissioner O'Brien, you may have been aware of that too. So it is something for us to keep our eye on. But what I'm hearing from Bruce is that this is very much a standard practice and from Sterl and Burke. So. That's correct, Madam Chair. And then finally, it's just worth noting that some non-material edits have been made to 147.07 sections two and five, respectively. So they can adequately reflect the changes that we've made to section three and the deletion of section four. So if you're looking at page 67 of the commissioner's pocket, you'll note that section five has now been relabeled to become section four. And then the mention of section four within section two has been taken away. But it's all pretty straightforward. Yeah, questions for Judy and the team now really helpful. Commissioner O'Brien, thank you for framing it. Okay. You need a motion from us. Commissioner, if you don't have any further questions, I'll take a motion. I'll make the motion. Thanks. I move that the commission approve the small business impact statement and the amendments to 205 CMR 147 as included in the commissioner's packet and authorize the staff to take the steps necessary. To file the documentation with the secretary of the commonwealth and to proceed with the regulation promunication process. Commissures, do you, oh, thank you. Commissures, do you want to make any edits, any concerns? Okay. Commissioner Cameron. Aye. Commissioner O'Brien. Aye. Commissioner Hill. Aye. And I vote yes. If there's any hesitation on this discussion, I think commissioner O'Brien, you're noting our concern about consumer protections. I think as we go forward, yeah, that that's a conversation that, I think there should be sort of an overarching analysis on the right review too, in terms of the rules of the game, in terms of how it impacts that obligation we have. Yeah. So, you know, we're voting it today and I think what I'm hearing, and it's a very fair point, even if it's a practice across the nation, we can sometimes be leaders in change. And so looking at these rules and regulations through that lens is important. And so if we need to revisit, we'll revisit. As need be, commissioner O'Brien, are you satisfied with that? Yeah, yes, I am. Thanks. Okay, excellent, thank you. Thanks very much, commissioners. Thank you. I'm sorry, did I miss that? I'm sorry. I said thank you very much, commissioners. Thank you, Judy. Very, very well done. And I am only trying to imagine the tutorial that you had. It was extensive. Yeah, well, welcome. This is the official welcome. Once you start hearing Barbara Paul Stack, you know that you're in it to win it. Great, thank you. Todd, is that it for the legal division for right now? Yes, Madam Chair, that's all we have. Okay, so we had scheduled a break now for lunch before the rest of the afternoon. It's one o'clock, does that still make sense? Does that work for those who follow to be a little ahead? I'm seeing lunch is a good idea. Yeah. Did you enjoy that for 30 minutes though? It looked like it was only 20 and that's a little tight. Right, before we were gonna take the longer one, we took the shorter one before, so now we'll have a longer one. Do you want a full half hour? Yeah, I think that'd be, yeah. One to one 30, stretch our legs a bit. I'm getting pain that I'm supposed to be standing and breathing and all kinds of things, so. All right, let's go do that then. We'll reconvene at one 30. Everyone appreciate all the good work and congratulations, Judy, on getting us through that regulation. Appreciate it. Okay, thank you. We'll see you at one 30. Thanks, Dave, for the screensaver. So I think, Dave, we can bring the screen down when that works for you. Thanks so much. No problem. Thanks, Brad, for the helpful tip on the gaming. I knew that too. And I like that commissioner Cameron probably, even after 10 years, you're still learning. Right? Always, always, which makes it interesting. It does make it interesting, right? Okay. I think we can get started, commissioners. I'll do a roll call. Commissioner Cameron. I am here. Commissioner Bryan. I am here. Commissioner Hill. Present. Okay. Thanks. Before we turn to little copies still here, before we turn to item number seven, Karen does have an update on her administrative update report that she provided earlier to us. I think that was under 3D. Karen. Yes. So I just wanted to let the commissioners know, I was in contact with Jackie Perlman regarding the event on the 17th. They're not going to be having over 1,000 people at that event. So for now, there's nothing for the commission. I think to really act upon it this time, we are going to have for the conversation is about events going forward. But for right now, it's not really right for any kind of commission action. So we just stand by and I'll keep you posted about other further events. Excellent. Thanks. Okay. So I think we can continue on a really diverse and full agenda today, right? Now we're going to get an update from Mark. And good afternoon, Scott. It's great to see you. Chair, it's nice to see you as well. Great. Good afternoon, everybody. Can you hear me? I'm here. You can. Okay. And we can see you too. I've been having technical difficulties. So I don't want to assume anything. So we're going to give you an update about the implementation of play my way at MGM today. You'll be hearing from Marie Claire Fleurs-Peugeot who will be talking about the evaluation of the play my way long band who will talk about communications. And Scott Hellwig is going to represent the technical aspect of it. Who isn't represented here is MGM and her games since teen, though they are both incredibly involved in the rollout of play my way at MGM. So it's not that they're not involved and that's not why they're here. It's just that there's a lot going on. And so we're just going to take some. So with that, Marie Claire, do you want to pull up the slides for us, please? Thank you. Sorry. Okay, great. So I just, you know, as always, I just want to spend a couple minutes providing a brief overview and reminder of what we're trying to accomplish here. So Marie Claire, you want to go to... I was having trouble with driving the slides. So I asked Marie Claire to do it for me. So what is play my way? It's a budgeting tool designed to provide slot machine players and slot machines, specifically helpful information in order to inform their play. It's really about informed player choice. Give people the information they need so that they can make the decision about when to gamble and when it's time to stop. It's set up to provide daily, weekly, monthly budget, one or three options. It's completely voluntary. It's intended that patrons can enroll right there at the slot machine where they can go to the Game Sense Information Center, where they can get access and help from a Game Sense advisor and enroll from a Game Sense kiosk. It's at Claim Ridge Park Casino and has been for several years and we're getting ready to roll it out at NGM and on. Next slide. Two goals. One is to sustain recreational gambling by establishing feasible parameters. The second is to eliminate regret arising from loss of control. And what you'll notice here is that really, we have two target groups here.  And I would say at-risk gamblers. One of the key elements of at-risk gambling is loss of control or regret. While problem gamblers aren't necessarily our target audience, it's recreational and at-risk gamblers who we primarily want to target. Problem gambling is an incredibly important issue for us, but we want to make sure that we have other options available to meet those needs. Next slide. So believe it or not, Play My Way launched at Claim Ridge Park Casino back in 2016. So now it's been several years. It was launched as a pilot project in Commissioner Cameron. You may remember that discussion very well when that was decided. After a couple of years of Play My Way at Claim Ridge Park Casino, and including an evaluation of the program, the commission voted in 2017 to extend Play My Way cooperatively, not through regulation, to MGM and to Encore Boston Harbor. And it should be noted that both of those licenses were willing and ready partners to do this cooperatively. The idea was that it was going to be launched in September of 2020. However, COVID combined with some technical issues brings us to today, where we are now prepared to launch Play My Way at MGM. It says the week of March 28th, we anticipate a soft launch on the 28th and the official announcement launch on March 31st. Encore Boston Harbor is still on track to launch in September of this year. Next slide. So now for just a quick update about where we are at MGM specifically. I'm gonna be repetitive here, but please bear with me, because I would like to point out some of the differences in the system at Play My Way at Park Casino versus what you will see at MGM in it on. So as I said, it's, well, first off, it's intended and must be combined with persons who hold reward cards. And so at MGM, it's MGM Rewards, My Play, and the, I think it's still the red card at Encore Boston Harbor, but that's how the play is tracked. And fortunately, most players at our casinos hold those player cards. This system is completely voluntary, as I said before, and it tracks it on a daily, weekly, or monthly. A difference between MGM and Encore is that, whereas PPC, we can deliver a notification starting at 50% of the established budget at MGM and Encore, it will be 75% and then every 25% after that. The main rationale is that we receive feedback that beginning at 50% players were receiving too many notifications and that that wasn't a desirable feature. So we will track that and we'll continue to monitor that. Another key feature or change is that notifications can be enabled or disabled. And so there's a toggle button that I'll point out to you in the next slide that where players can turn off the notifications coming up, but they can still track their budget by clicking on the play my way icon on the slot machine or at a key ask. So the system does not stop individuals from gambling beyond their budget and this is intended to be informed player choice. And so a decision made very early on was that rather than having a hard stop, this is a soft stop. It provides a notification and allows people to continue to gamble beyond their established budget. And you can enroll, unenroll, set your budget, change your budget, check your spending all from the slot machine or from a key ask. When you make a change, when you unenroll, when you enroll, it takes effect immediately. And the slide that pictures on the right were providing screenshots from what, this is what it looks like when you're at a slot machine or when you see it on a game since key ask. It's, I know it's kind of hard to envision, so we look forward to having, showing you actually on the machine soon, but these are screenshots from the machine. Next slide. Budget settings, so these are the budget setting screens that you see right here. And again, you can see that it's a daily, weekly, monthly. The second picture there is when you click on the set button, it will pull up the screen that allows you to easily set it. And then you'll see that the budget is active. One of the key things that I really like about Play My Way, it's not difficult. You don't need an instruction manual to do it. It's incredibly intuitive and it's quick. There's very few clicks between initiating an enrollment and when you're actually enrolled. And while we wanna focus on training, the idea is that we wanna keep it as simple as possible too. And you can see the toggle button where there's from the first picture, there's an alert. You can turn that toggle on or off and that will enable or disable the alert messages. Next slide. So tracking your budget. So say for example, you don't wanna receive the alert but you wanna continue to track your spend. You can pull this screen up at any time and you can see what your actual spend is. And so you can see that for this individual, their budget is $100 for the daily and their actual spend is $1. You can also see the green buttons down at the bottom where you can easily adjust your budgets for an enroll from the program. Next slide. And these are the types of notifications that you receive assuming that you do want to receive those notifications. You see there's an approaching at 75%. There's a reached screen where you hit the budget that you set and an exceeded screen. Next slide. These are the unenrollment screens. So if you want to unenroll and at Plain Ridge Park Casino, I believe we're right around, I think it's around 14 or 15% of individuals who have enrolled, it's not for them, they wish to unenroll. We want just a little bit of information when an individual chooses to unenroll from the program. And hopefully this will help us continue to improve and enhance the program down the line as well. Next slide. So we also, let's see, I missed that there's a green button down at the bottom that allows you to access information about game sense too. So this is, it's an interesting series of slides or screens that you can kind of scroll through, but it really does tie play my way to the game sense program. Go ahead. And that is where we are. I'm going to turn it over to Scott for the, to give you a technical update followed by Lon who will provide you with information about our communication plan and Marie Claire. So Scott, I'll turn it over to you. Thank you. Great. Thanks, Mark. Good afternoon, Madam Chair and fellow commissioners. Nice to see everybody today. So I'm not going to go ahead and read right off the slide. I do have a couple of other notes that I'll go ahead and go over here.  So, you know, we have been working with IGT, MGM and Encore Boston Harbor and their technical corporate teams to develop Play My Way for launch in MGM and then eventually awkward properties later this year in Massachusetts. So as Mark said, we did experience some technical defects during testing in September. And during the last few months, those defects have been corrected by the IGT team. Those corrections then were tested in parallel and in the MGC Gaming Lab and in concert with MGM and IGT, the teams to ensure all function in audio Play My Way works as expected. Once that was completed, IGT then submitted the components associated with Play My Way to MGC certified independent testing lab, GLI for certification. The MGC has received those certifications with all components associated passing certification as well. So with that, we will continue to assess and periodically perform tests to ensure that the continued compliance is there for the advantage Play My Way module that will be at MGM and Encore invention. And as always, you're welcome to request demonstrations in the MGC Gaming Lab by request and you know, we'll be more than happy to give you a nice little demo of the software and show you around and kind of see what the players are looking at. Unfortunately ours is gonna be a little bit smaller because it's on the smaller TFT screens but what Mark is showing you is like what they call the service screen which will pop up actually on the game itself. So it's a little bit easier for the players to see and MGM has a mix of both of these types of screens at their property. So those screenshots are pretty similar but the layout is a little different between the two. And that's really it for the technical side of it. I'll go ahead and let Mark and his team continue one with going over Play My Way. And just so they installed Play My Way at MGM in a test environment earlier this week when they're Tuesday of this week. So if you're out at MGM, there's a way to test it. In fact, Long and I will be out there tomorrow to kind of take it through the paces in their environment as well. Okay, let's turn it over to Long for the communication update. Good afternoon, Madam Chair and commissioners. So today I really just wanted to sort of talk about the communication objectives. So we were tasked to roll out Play My Way. So some of the objectives that we collaboratively developed with our community stakeholders, the Mass Council and also the MGM. We really wanted to one of the objectives is to introduce Play My Way at MGM. And then the other one that we thought was very important was to educate the public on purpose and the benefits of Play My Way and how to handle the essence of Play My Way. And then the three other objectives that we put there was to promote Play My Way as innovative and voluntary. And I think that's a very, very important. He word is the voluntary pre-commitment budgeting tool at GameSense. And we want to increase awareness of the Mass Gaming Commission and MGM's commitment to responsible gaming with the brand and its program and update Play My Way branding by incorporating positive play, messaging in marketing and materials. And so now I'm gonna go to the next slide where we will talk about the strategy. So that's sort of our overreaching objectives. So some of the strategies to obtain the objective or to reach the objective, we sort of developed one to introduce Play My Way with the MGM rewards player. There is a highly targeted branding and marketing initiatives. So one thing that MGM is doing, and I will go more into details on the next one more of the actual tactics or some of the things but we're working on ways to target these highly targeted branding and marketing initiatives. The other one, we were really one of the strategies to educate MGM's casino employees about the availability and the functionality of the play management system. This one, it's a way to promote and also to educate the patrons as well because they will interact with the casino employees. The other strategy was to increase general awareness of Play My Way but developing and executing a strategic advertising campaign, targeting, visiting patrons, employees and various stakeholders. And then last, our strategy was to highlight the availability of this tool and our commitment, Mass Gaming Commissions Commitment to innovative responsible gaming strategies. And then I'm gonna get to the fun part, the tactics. What are we doing? How do we implement this next slide, please? Thank you. So with the collaboration with Mass Council on Gaming and Health and MGM Springfield, we're developing a series of marketing materials to introduce Play My Way to MGM Springfield with the MGM Rewards patrons. So some of the collateral includes email blasts, retractable banners, digital signage at MGM Springfield Geo-Targeted Campaign. So like I was alluding earlier, some of the strategies that we're doing for MGM is doing a staggering marketing approach where this week they're gonna do a social blast and then next week they would do an email and then right about the week of the launch, we're gonna do a digital display to let patrons know that Play My Way is launching or has launched. So it's just more building that excitement and sort of building the awareness. And then the geo-fencing as the Mass Council on Gaming, we're looking at different approaches on the cell phones where patrons are coming in and out and sort of having different campaigns depending on if they're coming onto the casino or if they're leaving the casino. And so different messaging sort of let them know Play My Way is available. So either if they're there or if they're gonna return. And then the other tactics we're doing is educating MGM casino employees about the availability and functionality. So we're developing the Mass Council on Gaming and Health has developed a really nice one page reminder to educate the employees what Play My Way is. We're just giving them a refresher and just letting them know because there's a lot of promotions happening but we want the employees to be aware of what Play My Way is. Then lastly, we're gonna do increase general awareness through a strategic advertising campaign which includes festive environment at Game Sense Information Center, distribute press releases and also distribute media advisory on lunch week and day. So those are some of the approaches that the tactics we're doing to reach our objectives. Thank you. And then I think I'm gonna turn it over to Marie Clark who's just gonna talk about the evaluation plan. If I could just chime in and just make a few points back in communication. So one of the lessons learned at PPC when we rolled it out was just how important the Game Sense advisors were as champions on the gaming floor to promote this as a responsible gaming tool. And so that was a lesson that we really tried to carry forward to MGM. Another piece of it is ownership of our licensees whether it's PPC or MGM or Encore in really taking this and running with it. And I just, I wanna emphasize the importance of both of them that being leads on this. While we are present, we are supporting this and also champions of it. The true success I think happens when it's led by those two entities. They will drive enrollments. They will drive the enthusiasm for it on the gaming floor all day, every day. And so it's really turning to them to allow them to run with it. Okay, that's what I wanted to tell them. Thank you. Marie Clark. Yes. Good afternoon, my dear commissioners. Now I'm gonna talk about a research study that is led by Dr. Michael Wall and his team at Carl's University in Canada. This is a study that is funded by the ICRG, the International Center for Responsible Gamling. And the goal for this study is to evaluate play my way and to build some evidence on pre-commitment oriented play management tools like play my way. And so for this, they'll be measuring the uptake, the selection of mandatory versus voluntary limit adherence options. And they'll also be measuring behavior change before and after introducing play my way at MGM. And for this, they'll be conducting the study in three phases. The first phase, they'll be recruiting 600 MGM patrons who play electronic gaming machines regularly. And they'll invite them via the MGM rewards program email to conduct some survey prior to the launch of play my way. And actually this took place in the last two weeks. And just last Friday, they actually hit the quota of 600 participants, which is great. The second phase, they'll be conducting another survey three months after launch of play my way. And the third phase will be six months after the launch. And I know that the team at Carl's University has been working closely with Daniel Miller and the MGM communications team. And they've been very happy and grateful for all the quick and efficient support that they've been providing for this research project. So this is pretty much a timeline overview of the three phases of the project. So like I mentioned, the first phase just happened at the end of February. And the survey questions included some demographic questions, some questions about the player behavior, what type of limits they're setting and to overall assess positive play. And then the second and third phases, the second phase will be at the end of June, third phase at the end of September. And the questions will be assessing change in gambling severity, limit setting behavior and overall perceptions of play my way. So overall the findings of this study will be very helpful to demonstrate the effectiveness of a tool like play my way in changing gambling behavior. And that's it. Is there anything else that the team wanted to mention? I don't think so. I actually see that Daniel Miller is on as well as Garrett Farns, who's with MGM International and from the corporate level has been supportive of this initiative. So thank you both Daniel and Garrett for your support. Hi, Daniel. Thank you so much. Thank you. If we could have the whole group, excellent presentation. I'm sure we have questions, commissioners. Commissioner O'Brien, are you leaning in? No, I wasn't. I'm just sort of taken aback by, I know all the work that's gone into this and I know it's been sort of delayed for various reasons. So I knew it was a lot of work but I did not fully appreciate until this exactly all the details that went into getting this up and running. So congratulations. I know it was a lot of hard work to get here. Thank you, commissioner. It's interesting. I mean, it's been, when you've worked on a project for several years as this entire team has really committed to that, you kind of lose track of it. And it really is sort of a pinch me we're almost there sort of moment to see this live at MGM, to see that we're extending the use of this responsible gaming tool. After this long effort is really special. So yeah, thank you. Yeah, yeah. You guys should be very proud of yourselves. Commissioner Cameron. Yeah, again, excellent presentation. Nice to see it rolling out again. I noted earlier the improved product, right? There were lessons learned and you are incorporating that into this new plan which I think is great. I remember the communications plan last time and this one looks like it's improved because again, you know how important it is that the licensee is embraced and be part of it as well as our game sense advisors who really did champion it the last time, right? I just wondered, Marie Claire, the evaluation plan, is that a little bit different than what we did the first time at PPC? Mark, do you want to take on this one? Yeah, Mark, I'm sorry. You weren't here with the first one. Sorry. So it is different. And I would like to point, the fact that this is the first time that we've had research funded, MGC research funded by something other than our gaming licensees through the Public Health Trust Fund. This is a fantastic step to have the International Center for Responsible Gaming stepping in and providing this type of evaluation. I would say that there is some similarities between the two evaluations, just in terms of, you know, we're wanting, it's both qualitative and quantitative. So I think that we will be able to compare the two surveys but there are also a few differences in that shape. Well, great work. Congratulations to the whole team. And it's exciting to roll it out, right? It is. Commissioner Hill? Oh, just a great job, great presentation. And as I've been coming in the last few weeks and a few months, they have been feverishly working on this program in their little cubicles. And it's great to see it finally come to fruition. Excuse me, fruition. Congratulations. And I do look forward, Scott, to going into the lab and taking a look and seeing how it works. Because I think that would be kind of neat to do. All right, just let me know. I do want to do a special shout out to Scott and Katrina that the rollout at MGM, our IT has been there every step of the way, driving to make sure that it's meeting those technical aspects. And the delay that we had last fall when we were supposed to launch, I think technical issues happened having the technical expertise to really isolate what the issue is, make sure that it's resolved so that when we launch on March 31st, we're confident that it works the way it does. It's supposed to. It's really, it's great. So thank you, Scott. And Katrina isn't here, but I thank her as well. Thank you, Mark, appreciate that. I think Scott's proved to be quite the utility player here on this front. So thank you so much. And Mark, you should be so incredibly proud of your team with Marie Claire and Long. Just, it's all coming together and gelling so nicely. It's a pleasure to hear from both of you. Mark, forgive me, because I'm realizing I'm just, my memory's probably not serving me well. Have I ever been, has there been a report from Playing My Way at PPC as to the percentage of players? It would have to be anonymized data, of course, that override their budget. Do you have that? We do have information about that in one of the evaluations that was done early, early on. Early on. There is a significant percentage of individuals who exceed their budget. And I think it's, got to put it into context a little bit and think about what the objectives of the program are. We want everybody to stick with the budget that they have. When you set a budget in the cool state of mind before you sit down to a gambling session, your goal is that you want to end there and stop when your budget is met. There are times where people are going to gamble beyond their budget, but are we giving them a notification? Are we notifying them at an important juncture that you've reached your budget or you've reached 125% of your budget or more and it gives somebody a moment to pause. What's really difficult, I think, in an evaluation and perhaps we'll see this in the current evaluation is without playing my way, would they have gone even further in exceeding your budget? Is it still a useful tool that is informing people's gambling behavior? Which to me is the overall goal not of playing my way, but of game sense too. Are we giving people a moment to pause to really think about their gambling behavior and where they want to go with it? Well, and that's really, really helpful. And we have, I guess I'm not hearing from any kind of a percentage. You know, is it? Yeah, I can track that down. I was just wondering, is it 80% or is it 20%? And so, and I think the Marie Claire, I just, I'm one thrilled over the funding mark, but I just think it's just really an exciting opportunity for you to be able to be involved with that research. I mean, it's a whole team, but Marie Claire will be looking forward to the translation of it. And I guess if there's any kind of lessons we can learn or innovations that could come out that still serve what you were saying, purpose of mark that, you know, these need to be individual choices, right? But I wonder if there's even another tool. I understand the notifications can get turned off. So we lose that tool, right? And, you know, just there might be another tool we're not even thinking of, right? That won't happen. I absolutely agree. I think play my way is a good tool. It really is that allowing people to set a budget. It's innovative, it hasn't been done. Is there other directions that we can go? Are there other tools? I know there are. And I think that as this technology develops, as we learn more, we need to just, you know, be diligent about listening to that, seeing what's happening elsewhere in the world, see where our research and evaluations and utilization and feedback is taking us with this and how we can continue to expand it. And then the final, and we don't have to go on on this, but have you ever, have we received feedback from patrons wishing that they couldn't have overridden their budget, you know, like, yeah, and that's what I figured. So, you know, of course we respect the choice and not the mandate, but I just wonder, so I'm looking forward to the, I'm really looking forward to the research that comes out of this and so excited. I don't want to, in any way, take away from what's been accomplished in the rollout. Daniel, we are eager for this and we thank MGM for all that it's done to partner on this front and all the work that went internally to get this done. It's really exciting, really exciting.  We're lucky in Massachusetts to have it limited to three right now. Very great licensees. So, yeah, good luck March 31st. Daniel, if I don't, we don't see you, you know, again, gonna hear from the folks shortly, but just congratulations on all the work that's been done. Thank you. Long, thank you. I'm missing seeing you. Likewise, can we, all right, Chair, for a minute. Yeah, my closest buddy. Office buddy. Okay, March, is that the end of our presentation from you today? That's it, that's all you get. Well, you know what? This was all part of the plan though. This is a big month for responsible gaming and play my way with Scott and Katrina and the IP team's help in all of your group. March 31st will end with a great product. So congratulations. Thank you. Thank you to the commission. So now we move on to item number eight. Crystal. And Crystal, thank you. She's been, of course, really nimble as our agenda continued to be evolving for today. So thanks. Never a problem. I will be really brief. I have an important update for us as our work continues. We will be allowed to continue to meet remotely as the open meeting law has been extended allowing a state and municipal entities to meet remotely through July 15th of 2022. Those are through the chapter 22 of acts of 2022 if you're looking for that reference anywhere. And in addition, just a quick note, the state house has also officially reopened as of February 22nd. So we have that ability now where we haven't in the last two years. But that's it. Really quick update for you guys, but an important one. Important one. It's July 15th. 15th. July 15th, yes. 2022. Yep. Two more months. Yeah, two more months. And it'll be interesting to watch this to decide how our lawmakers think about the ease of these virtual tools versus of course the benefits of being in person. So I think there's a lot for them to think about. Thanks. Thanks for that update. Any questions for Crystal on anything else that she and Commissioner Hill and Todd have been monitoring? Okay. We'll keep track of that for maybe the 14th. A more full sum report if there's any big developments. Okay. Short and easy. That means returning now to, what is your name? I was so embarrassed the other day when I couldn't come up with Joe's name. All I could think of was Chief Delaney. Good afternoon. Good afternoon, Madam Chair and commissioners. So first up really our only agenda item today for this meeting is MGM Springfield's fourth quarter report. We've got Daniel Miller, Arlen Carballo and Beth Ward from MGM Springfield. I think Gus Kim is also on, but I will turn it over to Dan to start MGM's presentation. Thank you, Joe. Good afternoon, Lady Chair and fellow commissioners. Thank you for inviting us back to deliver another quarterly report to you for the end of 2021. Let me just share my screen here so you can see our first slide. And then hopefully I can also, and I think I need to swap that over for you. Everybody see that clear link? Perfect. Okay. As usual, I will hand pretty swiftly over to Arlen as our executive director of finance to discuss our revenues and lottery slides. Take it away, Arlen. All right, thank you, Dan, and good afternoon, everyone. Can everyone hear me okay? I always have issues with the mic. We can. You're nodding. Awesome, okay. So I'll go ahead and get started for the fourth quarter, for October, our gaming revenues were 21.4 million, total taxes of 5.4 million. For November, gaming revenue was 21.7 million with total taxes of 5.4 million. For December, we rounded up with 22.2 million in gaming revenue and 5.6 million in gaming taxes. And in the quarter at 65.3 million in gaming revenue and 16.3 in gaming taxes, making it the strongest quarter for the year. Moving on to our lottery sales, total sales for the quarter were 335,217 dollars, which was equivalent to a 61% increase over 2020 Q4 lottery sales. Any questions on our numbers? Dan, back to you. Look, Dan, you're muted there. Yeah, you're muted.  I said some beautiful things. I really did. We could read your lips. Thank you. So yes, delivering a compliance slide for the miners who had sadly either gained access, gained or drank on the casino floor. Overall, you can see the change from, kind of going from Q3 into Q4. I know the numbers obviously have exceeded there from a gaining access perspective. The two things though that I conditioned that to, one is starting in November, we started most of our festive and family orientations with the ice rink and family items out on our plaza. The other part as well is with our operation of the mass mutual center, a lot more ice hockey games over there, family oriented programming, that just increased the number of people that were coming through our facility that were bringing children with them. And then as you can see at the bottom when we talk about the length of time on average that they were on the floor, the length or the longest time, the 18 and a half minutes seems long I would say, but I think that's because in total there aren't that many incidents. And so when you go from the hour and five down to practically zero or three or four seconds, it gets stuck in that group in the middle there. We had a few incidents where it was a child ran in from our plaza under the railing onto the floor, but then was gathered by their parents and security spoke to them very quickly. And then as you can see on the far right, we've got the number of parties that were prevented from gaining access that either showed up at our podium with fake or fraudulent IDs or identification that wasn't even theirs. And then speaking to the two incidents, the one that was drinking alcohol, this was not served to the party by anybody employed by in Jim Springfield. A person of age bought the drink past it to them. And then the two that gained, sadly we did allow them through the security checkpoint. And in those particular cases, those security officers were retrained and did receive a discipline based on that. Any questions on the slide? I think it makes sense to have the questions now with the slides in front of us. Commissioners, I can't see you all, but I just, well, if you want to just interject, if you have any questions for Daniel. I'm hearing none. I just missed your explanation. I'm sorry, with respect to the minor intercepted consuming alcohol, was it their own alcohol? Is that what you said, Daniel? No, a person of age bought a drink, but then passed it to the person that was underage. I see. We didn't serve directly to them. I see, but they weren't on the gaming floor. No, they were, sadly. They had made it through into the gaming floor and the, like I'd say, the person of age had gotten the drink and then passed it over to the person who I believe was 19 or 20 at the time. So they were an adult, but obviously not over the age of 21, so. So, okay. And the ones who were admitted for the minors intercepted on gaming and the retraining, were they just allowed in without being IDed or were they fraudulent IDs? One of each. One sadly wasn't checked and one was a fraudulent ID. That wasn't correctly handled. The second one, they were two incidents. The second one, I will at least say that immediately after security, we're there within, I believe it was two minutes and got the person off the floor within four. So although we had made the mistake of allowing them on, the reaction was very quick and then removed them, so. Thank you. Any other questions for Daniel? I do appreciate MGM's family friendly activities that we went out. Commissioner Hill and I, and we saw the skating rink. You know, we know about the bowling alley and the movies and that's the resort that we know that the lawmakers envisioned. It does present challenges. And I suspect Daniel, you'll be thinking about how to get these numbers down for the next quarterly report. Totally true. I think the only other thing that I'll pull from this because the most relevant quarter to compare it to would have been the same in 2019 and those numbers were a lot higher. So when we compare across those, these are only about a fifth of what they were in that year. So we definitely trended in the correct direction, but of course we can always attempt to do better without a doubt. Yeah, yeah. Okay, well, again, I do like those attractions.  Right. Thank you, Daniel. Very welcome. I'll hand back to Arlen for our operating and then local spend. Hey, Dan. For our total operating spend, we had $9.3 million identified in biddable spend. I know that's 7% or 0.6 million went to diverse suppliers. We have been working with NGN, no, G-N-E-M-S-D-C which is the Greater New England Minority Supplier Development Council to help us source more diverse spend. We've been doing it regularly as well as working with CWE, the Center for Women and Enterprise to also help us source more diverse vendors to bring these members up. And then for the, any questions? Do we have questions on the spend, missioners? That would be. Arlen, that's a tough slide, right? Is it a real change from the pre-pandemic? No, it hasn't been too much of a change. There has been a slight deficit. I think the challenge is now that on top of, really trying to bring this numbers up, meeting, we're doing everything we can, is we're also having us to play change for problem and challenges that everyone else is facing to just add to the layer, but we're not giving up, we're very focused on it. We're continuously meeting with suppliers. We have them come on property sometimes and we're looking for bids. We have regular meetings. We just had someone graduate from our mentorship diversity program. We're looking to start a new program and we have two to three local vendors will be submitting to our MGI mentorship program which also has extended, I believe, from six months to eight months. We just had the graduation Monday where I was able to attend. So we are all working together on this, not just us, but our corporate leadership as well. Thank you. And then for our total biddable spent was 12.4 million for the quarter. On that, 6.1 million or 49% was to suppliers within the Commonwealth and 38.5 or 4.8 million went to Western Massachusetts suppliers. Any questions? I'll start. Thank you. So this time I want to thank my pardon as I'm gonna play the part of Jason Randall for our employment numbers. He couldn't be here this afternoon, but for good reason, because he's actually conducting and attending a mass hiring event that we're having on site today. So that's why he couldn't be here. Looking at the first slide as we do regularly, you'll see the total up there of 1,189 employees total. As we go down, you can see that over that quarter period was an increase of 65 additional employees that we brought onto the payroll. Next. And then on this slide where we usually provide our separate minimum requirements of diverse employees, we understand obviously that there's a slight dip on the minority and the women numbers. And I can't give you an exact reason as to why that number went the way that it was. What we have looked into though, and that we're still more proud of is during the quarter and throughout the year, that 48% of our promotions within MGM Springfield were female employees. And I was actually gonna deliver, if you'll beg my indulgence, a little personal item on that, which was when I started within MGM Springfield, I was the risk manager. And I hired a young lady named Jasmine Rung as my coordinator. When I moved over to compliance, she became the risk manager. So she got one promotion. And then in the middle of last year, she actually was given the opportunity to become a director of risk management. Sadly, that meant that we lost her because she went back to Las Vegas. But the final thing that I'll throw out there is that when I hired her, she was a local Springfield resident. She was a female of color and she was a veteran. So I hit every single one of the requirements. And then she has continued to be a success story that I'm very proud to have been party to. And she's just one of many, clearly that we have here at Indian Springfield. I have one more slide, but after that slide I'll be handing over to Beth Ward, who as well, on previous occasions, we introduced her as our manager of community affairs. She is now director of community affairs as well. So there is definitely room for growth in moving up the chain. Here are some of the additional events that we've had to increase the idea of hiring, not obviously just regular frontline jobs, but also for people that look into leadership once they've completed their college journeys. And the final photograph there, the UMass leadership class tour and panel, I was party to that as well, got to deliver speech to those attendees. And they all seemed very eager to jump into the world of hospitality. It's what they're majoring in and obviously what they wanna do long-term. So any questions on any of those slides before I hand off to Beth? Are we sure there's any comments, if not questions? I had one, Madam Chair. You talked about at the very beginning your employment numbers, and I did notice a steady increase. Is that continuing? I mean, I know this is last quarter, but is that continuing as we speak, that steady increase of adding employees? It is commission at Cameron, clearly to speak to the fact that we've got a mass hiring then going on as we speak. And then because we really do want to try and expand on the amenities that we have available, we're running a, it's either weekly or five-weekly, and she best will correct me on this, hiring event for food and beverage outlets. We really do want to make this, the fully integrated resort that it opened as initially and come back blazing. And possibly if you do that, that could help you increase your number with women, correct? Yes, hopefully, yes. And you mentioned that you weren't able to identify why the numbers dipped or remain somewhat lower than you want them to be, need them to be. So I just wondered if you are, exit interviews with folks who leave or their childcare problems, are there, what the issues are? So I just wondered if there's work that could be done to kind of identify the reasons and see if there's anything you can do to assist with those numbers if you identify a specific issue. I'm understood. I'll get back with HR on that. I mean, clearly, although I said I'm playing the part of Jason Randall today, I don't do in very much justice, but we'll get back to you and let you know. No, no worries. Thank you. I appreciate that. Thank you very much. Good report. I like the story about the young woman, young woman who is now in Las Vegas. So that's a, that is a really good story. So thanks. Nevada's gain and Nevada's gain, right? Gail, but Commonwealth's loss. Definitely. Yeah. Yeah. It's a good story, Daniel. All right, Ms. Ward, take it away. Hello, everyone. Just checking to make sure you can see and hear me. Okay. Okay. Very good. Okay. I know I haven't had issues before, but this will be my one time. And thanks, Daniel, for that shout out. Very excited over my tenure so far to have received a recent promotion. So, and keeping, you know, the female representation here is near and dear to my heart. So you can be sure that I will be very much involved in that as well. So I get the fun part, as I always do, to talk about some of the fun things happening on property. And that includes community outreach. Madam Chair, you had mentioned coming out to the skating rink and what a wonderful, really great moment, community moment down there, which provided a perfect backdrop for our Toys for Tots drive. So once again, we partnered with Western Mass News, a local news affiliate here for a weeks long drive to help the Marines here, local Marines, in their quest this year. They received a record amount of requests for help from local families. We are also very proud that we partnered with the folks on site at Game Sense to make what we think was a record breaking and fantastic drive this year. We were able to fulfill two semi tractor trailer fulls of toys for local children in need, helping their Christmas a little bit brighter. That's over 24 pallets of toys. Those were collected through back of house with our team members and through our guests who were just so incredibly generous this year. And we thank them so much. We do have a lot more. I'm so excited to talk about for our next meeting. And that includes a canine send off, our security canine who is now retiring in Florida. So I have more on that. So just something to look forward to. Let's go on to our special events slide if we could. As I talked about last time, poker reopening in October, we have an update on that. We have gone from 13 tables to 14 tables. So we're excited about that as our poker enthusiasts. So we continue to reevaluate that. We know that, you know, people love their poker room and we are so happy that we were able to be the first ones to reopen that in the state. Also Bright Nights Ball, that was something we were very excited to be able to host once again, along with the spirit of Springfield. It is, if you live in Western Mass, you know that the Bright Nights Ball is the biggest event of the year, biggest charitable event of the year. Once again, we were proud to host and partner with the spirit of Springfield on that. Again, going back to that winter wonderland on the Plaza, and I'll tell you, it really, this sounds cheesy, but gave me goosebumps. And I know it did of everyone here. It's just such a, if I could coin the phrase or use the phrase, hallmark moment. We had the skating rink. And man, what a really great partnership with the community who, the response was so fantastic. Families were out all the time utilizing that, which was great to see we had the big tree lighting event, which Santa was here and cheerleaders from the Patriots were here, so that was a great time. We were very excited to be able to offer that and see everybody back again in full force. If we wanna move on now to the development update, just a couple of things on this. Very excited about the Mass Mutual Center garage that is scheduled for demolition in June. That is a $40 million investment by the state. So that is going to really transform that area by the Mass Mutual Center. And just so excited to see that happen and get that underway. Also want to mention the 31 Elm Street project. That is right behind me, whereas I look over it every day. Also, there's some progress to report on that. Of course, MGM is excited to be part of that project. Community partner both in support financially with a $16 million contribution and as a neighbor who was just thrilled to see that area come to life. I spoke with Tim Sheehan this morning who wanted me to send his well wishes. He says hello to the Fontaine brothers, that local contractor who has been chosen to transform this building. I have seen them in there. They are busily working inside to shore up the interior and the perimeter is now being, fences are being put on around the area. So that's exciting. We are psyched to see that beautiful building again be transformed. Also wanted to mention, Madam Tru, you had mentioned top golf and the bowling and we are excited to say just a little teaser that because of some of our hiring events and the response that we've had recently from the community, we are able to activate those for group reservations only and I will have more on that coming up next time we meet. So a lot of things happening, a lot of exciting things happening and honestly can't wait for to share Q1 with you next time, but just very proud of where we've come so far. Any questions? Any questions, commissioners? Can I ask Beth, do you know the timeline on the garage? On the garage? I really don't know. Yeah, they're scheduled for demolition in June and soup can not say I believe it's about 23 months. Gus, you might- Are you able to hear me? Yes, Gus, we can hear you. Again, like Beth, I have troubles with my IT system. Madam Chair, the project as it stands and I think it's an aggressive timeline by MCCA is about an 18 month development. They hope to break ground or not break ground to start demolishing the existing structure sometime in June when the Springfield Thunderbirds our local hockey team finishes his playoff run and hopefully with the championship. So the expected timeframe is about from June one to about 18 months. They are hopeful that everything will be done by the time the hockey season starts up again in 2024. I think that's a little aggressive, but that's the timeline as it stands right now, Madam Chair. Thank you. Thanks, that's great project too. As another field trip, Commissioner Hill, at some point will have to take you to C31L and we've only seen it from the outside. So that's just an exciting project. I think Commissioner Cameron has a deposit on one of the condos, if I remember correctly. That's correct, Madam Chair. The mayor challenged me to that actually. It's a great building though and they're beautiful views, really a nice building. Yeah, I think you kind of picked the one that everybody would want. Of course, overlooking the park. On the end unit, all the windows, of course. Yeah, yeah, so again, another tour, which we look forward to. Commissioner, is there any questions? And I think that, Daniel, does that complete the report? I don't want to, I thought that was the last one. I actually have a few more slides that are in. Okay, go ahead. So we'll just move on to those. There we go, yes, yes. To clear out, you know, so Q4, we had the Free Music Fridays that we had so successfully outside moved inside. They were inside of our Aria Ballroom. And then, as I mentioned earlier, within the More Family Oriented Programming, you had the Thunderbirds local hockey team playing quite frequently and winning quite frequently over at Mass Mutual Center as well.  Brian Regan, the comedian, was on site. And the Rock of Ages Band, that was actually our New Year's Eve event. Although we had a few around the property, that was the biggest one going on, again, up in the ballroom and welcoming in the new year, obviously, to the sounds of Classic Rock. So, oh, I'm sorry, it jumped past them. Oh, so, you know, we always talk about the Armory as well, that has continued to be our raw comedy club inside that facility. When Beth actually mentioned Bright Nights, the initial opening ceremony and reception was also hosted inside the Armory. So we are starting to program that a little bit more, varied as well, which is nice. And then the Commonwealth, I know sort of mid-year last year, you were a little concerned with it just having slot machines in it, would it return as the type of lounge that we intended it to be? And of course, it did return. And now we have dueling pianos in there and local artists that are duets and country singers. And I've been on a few occasions and seen the different styles of music that we have inside there. And then lastly, coming up, you can see some of the program we have mainly at Symphony Hall. I think there was a moment there, Chair Lady that you said you were a fan of John Mulaney. Big fan. At Mass Mutual Center. So yeah, so we really are trying to ramp up all of the entertainment that we like to consider ourselves to be top tier at now that we can spread our wings. So, and final slide, you guys have obviously seen this slide before. Although I'm glad to say this time around we are looking so on track. I don't want to jinx anything, but right now the two pieces of information that are on the slide, we will begin our social media campaign tomorrow about play my way coming. And we're all looking very much towards that hard go live on March 31st. So round out PGAN and just, this is our way of celebrating PGAN is through play my way. And that is. Thank you. And to that last point, if you weren't part of the meeting earlier, Daniel and team at MGM, we all recognize that PGAN takes really the village and we're very fortunate to have the collaboration and cooperative efforts from the three licensees that we regulate on all of those efforts. So, and play my way is going to be a great indicator exactly that. So thank you. Questions for this team, commissioners. Thank you for letting us interject during the presentation, Daniel, that worked out well. Not at all. Thank you. Keep it flowing. So Joe, I think that completes this part, but we do need to go into executive session if everybody agrees to do so. Yeah, so we need to, the separate invitation was sent for the executive session. So Dan, you and your team will have to log off this and then boot onto the next one. We just need a little process before we need to vote. So I'm going to interject on that. But before we go and leave this virtual room, commissioners, anything else? I don't know if we'll see all of the MGM at the next session if we vote in. Thank you so much. So I'll do my part, which requires me to read something into the record, Joe. So the commission anticipates that it will meet in executive session in accordance with GL, Chapter 30A, Section 21A7, to comply with GL, Chapter 23K 21A7 for the specific purpose of reviewing capital expenditures describing 205CMR 139.09 and any corresponding materials submitted relative to MGM Springfield as discussion of this matter in public would frustrate the purpose of the statute and associated legal authorities. This matter is further governed by 205CMR 139.02 as the information at issue is covered by a non-disclosure agreement. We'll need a motion on that. Just as a reminder, to those attending the public session of the commission meeting will not reconvene at the consideration of any executive session. So do we have a motion or discussion on it? Any questions? Joe, that's open meeting law. You have to actually, I had to actually read the word. So thank you. I'm happy to move that we enter into executive session for the reasons just stated by the chair. Second. Excellent, thank you. Commissioner Cameron. Aye. Commissioner O'Brien. Aye. Commissioner Hill. Aye. And I vote yes for zero. Todd, we got it done. All right, so Joe, your point is that we need to now go to our other invitation to get to the next virtual room. Correct. Thanks for that reminder to the MPC team. Thank you. We'll see you there. Thanks everybody from the team for today's very, very productive meeting. Thank you.


Hi. In this video, we're going to go through how to do a hypothesis test for a single mean in Python using a randomization distribution. And we're going to continue on with our record example from the 2021 Texas power crisis. And we're going to focus on wind here. The reason why we're focusing on wind is because, well, if you're looking at this visualization that we left off in the last video.  Natural gas is consistently below it. So the more questionable case here is, is wind, whether or not. Whether or not the mean power generation, the curve here is above or below the dash line, the forecast peak capacity. Okay. So let's get into how we would execute the code to perform this hypothesis test. So recall from last time we left off with Jen Piv and it's organized as such we've got date time as one column fuel as another generation as another here. So we're going to ask ourselves in this hypothesis test is whether the mean generation from when was consistently or statistically significantly below the forecasted peak capacity of 6.1 gigawatts. Now we could just calculate the mean with the solid curve and see whether it's above or below this dash curve. So if it was how much below it, or above it, would it need to be to be deemed significantly different than 6.1. If the actual mean was only say 6.01 is that statistically significantly below 6.1. If it was five, is that statistically significantly below it. So that's the real question here is, is how extreme was the actual power generation there. So if we were to do a hypothesis test for a single mean using a randomization distribution, we have to compute this shifted x sample. And so we're going to do that before we right before we do our randomization distribution step two here. The very first thing we need to do is calculate our sample statistic. So, since we're focusing on wind let's pull out our wind sample here from Jen Piv, just using a simple filter. We've got fuel equals wind. So now all of our wind generation values are just going to be in wind here. Actually, this will contain our three columns and then if we want to just have the generation values you can say wind. And then from wind we'll get our generation. And so that'll be our, those will be all the wind values. Our sample statistic though is the mean. So we want to then compute the mean of that. Okay, now there's other ways to do this we could have just done this in one line and filtered and only gotten the generation but but it doesn't hurt to do it multiple steps either. So, let's see what our mean wind is so 5.15 gigawatts. Well it's certainly less than 6.1. The question is, well is that extremely below it. Is it enough below it to say that wind, unequivocally fail as a power source during the Texas Colts net. So that's what we're going to answer with our hypothesis test here. So let's move on to the randomization distribution we need to import from NumPy. We need to import our NumPy package. And first thing we're going to do is do this shift. Now the whole point of doing this calculate the shifted wind speed is to see. Well what would our wind samples have looked like. If they were meeting the forecast to be capacity. In other words, what would the data look like if it were meeting the milk condition, if the actual mean was 6.1. What would that look like. Okay, so wind shift is basically the generation that we might have seen. Had or no condition to net. Okay. And it's preserving the very the same amount of variability that we actually saw in the real data. So how we do this is we take our actual wind sample from the data set. And then we're going to shift it. So we're going to add on the amount between our no value. So 6.1. So whatever is on the right hand side of the equal sign in our null hypothesis. Okay, 6.1. And we're going to subtract our wind mean value here. So the difference between the null and the actual mean. We're going to add that on to our wind samples here. Okay. So that's our wind shift. Next, we want to dictate how many randomizations we want to do. So capital N equals 1000. So we'll do 1000 randomization samples. And they're all going to have the same sample size as wind shift. So our little end is going to be the length of wind shift. Wind shift here is a series and we can use this link function. So it's going to have the same sample size as our original data set. That's important. And then we need an object in which to store our randomization sample statistics. So we'll use this empty function. Empty is just going to create an empty series in this case. It's going to be of size N. So it's going to basically have 1000 blank spaces here. And as we go through our for loop. Let's use J as our index here as we go through our for loop, we're going to fill in that blank series. So ran mean the J place. So we go through our for loop, the J spot in this empty vector or empty series. We're going to fill with something and that something is going to be a randomized sample. So we're going to use this random choice function. So NP random choice random choice is just going to randomly draw values from whatever we provide as the first argument here. So it's going to randomly draw values from wind shift. And we need to tell it how many values to draw. So the size should be equal to end. So it needs to be the same size as the original data set for a randomization distribution to hold true. And we need to do this with replacement. That is, we draw one value out of wind shift. And then we put it back in that same value can be drawn again. So we replace the value. Now this will, this alone will create a new sample of size N. But we want the statistic of that. So on this we can calculate our mean. So we'll have our new sample and we'll calculate the mean from it. So we just have one value and we'll store it in that single blank spot dictated by this J value. So we'll put it in the J spot of this empty random object here. Okay. And so that's our randomization distribution and go ahead and run that. We want to inspect this. Let's visualize it. Let's take our ran mean, convert it to a data frame so we can visualize it in GG plot. Take our ran mean series. And we'll just give it a column name of means here. So then in GG plot, we can take a look at this as say for example a histogram. And for the histogram, we just need to supply, tell it what what to use for X values in this case the means. So you take a look at that. So here's our histogram of the randomization statistics here. So this is our randomization distribution, all the different 1000 different mean values from scrambling up one shift. So this is the sort of variability that we'd expect to see in the sample statistic. If the null hypothesis were true. So if, if that forecast be capacity was actually being met to a statistically significant degree, we'd expect our mean values to fluctuate in this range here. Okay. That's great. Let's see how this compares to what we actually have so where does our 5.15 lie within this. And here we can visualize this with a vertical line. And so our X intercept is going to be mean. This is our sample statistic. Let's give it a color red, and let's make it dashed. Here we go. So this is our actual sample statistics. So this is the mean of our actual data and we see that it's well below the randomization distribution. What does this mean means that what we saw in reality is extremely different from what we should have seen if the null hypothesis were true. So we can also state the same result as a p value, or the probability of observing a value at least as extreme as this, given the assumption that the null hypothesis was true. Here we see that there's very little if no chance of seeing any part of the randomization distribution below this dash line. Let's write this out. We would then encounter calculate our p value here. I'll put it in a couch it in a print statement, although we could store it as a value if we wanted to. So our p value equals p value is going to be the number of our randomization distribution values that are at least as extreme as our dashed line. So we can write that as a length of the ran mean values that meet some criteria and that criteria is ran mean being less than or equal to our sample statistic. And so that'll be the number of the samples that are that are less than at least as extreme as when mean we want the proportion so we'll divide this by the big end. So there's a thousand values in here by that. Now the reason why we do less than is because that's the same as our alternative hypothesis. The mean is less than 6.1. So we need to make that agree down here we're calculating the p value has to be less than the reason why we include the equals to is because the definition of p value is that the probability of observing value at least as extreme. So at least so at least would include the equals sign. Alright, so let's go ahead and run this and we see that our p value is zero that there is zero chance of any element of our randomization distribution falling below our actual sample statistic. So, in other words, our, the data do not support the conclusion that the forecast capacity was met. So we can write this as a concluding statement. Always good to in the concluding statement. Okay, what the hot know what the p value is so we can say our p value is zero point zero if we want to be a little bit more precise about it. Because none of the randomization distribution falls below the sample statistic mean when right. So. So therefore, we can reject the null hypothesis. And we can say with null hypothesis we can reject null hypothesis that the four pasted peak capacity was met. But the key language here is that we reject the null hypothesis when we put that in bold. And the reason why we can do that is because our p value of zero is, well, it's less than point or five. We can make that clear we can say p value is less than 0.05. And we can, it's always nice to then put this into the context of the problem. We can say the average. Sorry, the average wind power generated during the Texas. Cold snap was statistically significantly less than the forecasted peak capacity. So here we have statistic proof that that wind also failed that we didn't meet the criteria here and I say also fail because well if the data for wind. We failed on null hypothesis, then most certainly that for natural gas would also fail since it is considerably below its forecasted peak capacity here. Okay. All right. So that concludes our test for a single me.


Hey guys, what is up? Welcome to my channel. For today's video, I'm going to be going over the 21 days of beauty at Ulta. You guys have requested this, so I'm just going to cover it. Let's get into it. Guys, if you're new here, my name is Morgan. I'm a product knowledge enthusiast. I just love knowing anything and everything about all the new makeup on the market and sharing my thoughts with you guys. Now, I'm warning you, this video is going to be on the negative side.  These are items that are going to go on sale and they're going to continue to go on sale based of what I've noticed of the industry lately, but I'm still going to go through each day. I'm going to tell you what's worth it. What's not of that. But again, like I said, it's rather disappointing. So let's get into it. If you don't know what 21 days of beauty is at Ulta, basically there are 21 days and there are certain deals each day. And I don't have the full rundown because they have deals that continue on throughout the whole month and then they also just have the daily deals. I'm going to go on by what is on the website. So yeah, we're going to start off with Sunday, March 14th, which is today. Maybe it's tomorrow, whenever I get this up. So the details are 50% of the cosmetics confidence in a cream anti-aging moisturizer, $24. This is a nice moisturizer, $24. It is worth it. Never pay full price for this moisturizer because it always goes on sale. It's nice. If you're in need of a moisturizer, sure, it's fine. Nothing special, but it works, especially if you have really dry, thirsty skin. I found that it worked for my skin in the winter. Also, one that is worth it for sure is the Stila waterproof liquid liner. It's $11. This was my college roommate's favorite liquid liner. So if you're like her and you love it and it's a staple in your everyday routine, by all means, please stock up on it. However, if you have liquid liners that you already love and use, there's no need for you to pick it up unless you are a true Stila liquid liner believer. It's not going to be worth it to you. There are so many other great liquid liners. Not saying the Stila is not bad. The Stila is really, really nice, but I think only really pick it up if you're a regular Stila liquid liner user. I'm going to skip the diamond plan. That makes it not worth talking about. Okay, anyways, let's go to Monday, March 15th. 50% off the Smashbox Original Photo Finish Smooth & Blur, whatever that thing. Yeah, I mean, if you use it every day, definitely worth it. 1850, but I have so many samples of these Smashbox, put these samples and everything, you don't need it. And quite honestly, it's not the best primer, in my opinion. 50% off this is online only on the Mac Studio Fix Concealer. I've heard good things about this concealer. It's only $12. I have so many concealers not interested in it, but hey, if you are, sure, go for it. So a lot of these deals, you'll notice a pattern. If you like them and you use them every day, you should get them, but that's the only reason I really feel like you should shop the sales, you know. Otherwise, you're spending money that you wouldn't have spent anyways, because for the most part, none of these are interesting. There are a few exceptions that I'll get into. Let's move into Tuesday, March 16th. Shiseido, now I love Shiseido. They are one of my favorite skincare brands. Their moisturizing cream is $24. I've never tried this product in particular, but I would push you in the direction of this moisturizer over the cosmetics. It's a bit of a better value if you ask me. And Shiseido is a lot more gentle on the skin with better ingredients. I personally think if you're on the market for a new moisturizer, this is the one that I would kind of push you towards. 50% off that Cover FX Luminous Tinted Moisturizer. I don't think Cover FX is currently doing very good sales. They took their products off the shelves of Sephora, and I think they're trying to make a little bit of money. I've never tried that before though. March 17th. Only interesting one is 50% off Beauty Bakery, Mented Cosmetics, Persona, and Sarah Hap. These are select women founders. I don't know if it's select items, but if it's literally 50% off the brands, that's a great deal if you've been wanting to try those brands. The Persona highlighters are awesome. So this is a day that I think is worth looking into if you've been curious about these brands like Beauty Bakery, Mented, or Persona. That's a day that's worthwhile if it is what I think it is. On the 18th, the only one that is worth it, in my opinion, is the Urban Decay Cosmetics Brow Blade. That's 50% off. It's $13. That's the brow product that I would recommend you going towards. Ulta usually has a good brow product on a really good discount every time they do this sale. This is the one that's worth it. I do really enjoy this brow product from Urban Decay. They also have 50% off bare minerals, bronzers, and highlighters. I'm not into bare minerals like that, so I can't say, but I'm sure some of you are probably excited about that, but that's all that kind of catches my eye. March 19th.  I personally don't use those. I have a Foreo that I enjoy, but if you're into using face tools, I mean, that's cool. That could actually really actually save you some money. Becca Cosmetics highlighters are also 50% off. I mean, this is a good time to get your favorite Becca highlighters, but Becca has a great formula and highlighters, but there are so many wonderful highlighters and with Becca going out of business anyways, eventually you're going to run out of the highlighters. So it's up to you. It could be a good opportunity for you to pick up, you know, your champagne pop, your Perseco pop, whatever, but that's $20 that you can put towards something else, you know? Saturday, March 20th is probably my favorite day of this sale.  I know that they have a lot of fans who enjoy this mascara. So that's something to be said for sure. Diamond Platinum Exclusive. So you have to be at this tier. It's the equivalent of the Sephora Rouge tier. 50% off the Tarte Shape Tape Concealer and Shape Tape Ultra Creamy Concealer, which is surprising because the Creamy Concealer is new to Ulta. So that's why I'm excited to see this on 50% off. So it would make it $13.50. I'm personally not a fan of their original shape tape, so I don't recommend that. I have the Ultra Creamy Concealer on its way from the Tarte website. I ordered it, so I'll let you know what I think of that. But that's pretty exciting for most people, I think. And this one is personally exciting for me. 50% off Select Eyeshadow Palette from NABLA Cosmetics. Now it's not specified what exactly is going to be on sale, but NABLA is an underrated brand, in my opinion, with really fabulous quality. So if you've been eyeing NABLA Cosmetics products, I would definitely recommend it for this day. The next day, March 21st, 50% off the Buxom Plumping Lip Creams. You don't need those. In my opinion, just skip. 50% off Smashbox Halo Healthy Tinted Moisturizer. Never tried it. 50% off Mac Powder Blushes. Now that is a Diamond Platinum exclusive. Mac blushes are really nice. They have really nice colors. And 1250 is a really great deal. But if you weren't already eyeing Mac blushes, you don't need to do this. You probably never even thought of Mac blushes. Now for me, as a makeup artist, I'm interested in making a little palette of some of these blushes, because that is a bomb.com deal. But again, I never thought of Mac blushes until I saw this sale. So not needed. March 22nd, the 50% off the Pure 4-in-1 Love Your Selfie Foundation Concealer. Personally, never tried it. But some people love it. Some people hate it. If you're not sure, I don't think you need it, quite frankly. 50% off Benefit Roller Liner. Again, same deal with Isila. If you're a fan of the Benefit Liquid Liner in particular, sure, pick it up. Other than that, you probably wouldn't have spent your money on it on the first place. Tuesday, March 23rd. Just so you know, I am skipping most of the skincare stuff because I don't know skincare like I know my makeup. 50% off Ofor Highlighters. Ofor Highlighters, they're really nice if you've been wanting to try one, but you don't need it. But they have a very nice super glowy formulation. 50% off Lime Crime Palettes. I do like Lime Crime Palettes. I think they're really pretty.  I'm not going to lie. So if you've been eyeing them, sure, if you think they're palettes that you're actually going to use. But remember, a lot of times these sales are just products that the brands are having trouble selling and want to get off their shelves. 24th, the only thing that looks worthwhile to me is Urban Decay Primer Potion. It's only $12. I'm a fan of Primer Potion. However, I always seem to get samples, so I never have to pay for it. So I wouldn't get it.  It is a great product. Everything else looks blah. March 25th, I see nothing. 50% off the salon at Ulta Beauty for a color gloss. I get my hair done there, but I don't get color. 50% off, bye-bye, under-eye, full coverage, concealer, never tried it before, but I've heard good things about that. But yeah, I've never tried anything for today, but it doesn't look interesting to me. March 26th, 50% off Benefit Give Me Brow. That's a nice brow gel. If it's part of your everyday routine, that's nice. 50% off Lerach Face Palettes. Lerach has nice products, but don't waste your time. Don't waste your money. I don't think Lerach's doing very well right now. Capari Coconut Milk and Coconut Exfoliant Scrub. Those are really nice and they really moisturize my skin. So if you're into bath stuff, I actually recommend that. March 27th, 50% off Becca Face Primers. Same deal with the highlighters. I mean, it depends if you want to stack up, but don't just buy it to buy it. Nothing else interesting. March 28th, Mac Prepping Primes 50% off. There are so many setting sprays now out on the market, but Mac Prepping Prime Fix Plus is really, really nice, but it's still water at the end of the day. Clinique, even better, all over concealers on sale. Never tried it. Not really interested. March 29th, okay. The only one that's really worth it and it is a Diamond Platinum Exclusive. All the good stuff is these Diamond Platinum Exclusives. 50% off the ABH Brow Gel. I enjoy that brow gel. I prefer Benefit Brow Gel, but that's been a staple in my routine for many, many years. So if I were running out, I would actually be excited about that. March 30th, Nude Sticks are on sale, $17. Okay, I'm excited about that. That one is really, really nice. 50% on Beauty Blender, Dream Swirl Blend and Clean Set. I personally find that I really only like the black beauty blenders. It's $10. That's a really great deal for what it is, but I only like the black ones. Nothing else really interesting, some bare mineral stuff, but the Nude Sticks is worth it and I would like to support Nude Sticks. I think they're a great brand. On the 31st, the Urban Decay All Nighter Face Makeup Primer. It reminds me a lot of the Milk Grip Primer. It's okay. Nothing great. I like it better than the milk, but it's not a necessity for me. I think if I were to buy it, it would just be buying it because it was on sale. April 1st, KKW lip liners are only $6. This is a really good deal. If you've been wanting to stock up on some lip liners, I like the KKW lip liners. I like the colors. I don't think KKW is doing that well either, but that's a really great deal. Also, if you're into Dermablend powders, those are also 50% off. April 2nd, no. I don't Tarte Shape Tape Foundation, Face Tape, whatever is on sale. 50% off. Nothing's even worth my breath. Let's just keep moving. April 3rd, Benefit Roller Lash. I know a lot of your fans of that. That will be 50% off to stock up on that. Then 50% off ABH Brow Wiz. That's exciting.  Am I going to run out to pick them up? Absolutely not. There are so many other great brow pencils on the market that I already have to try and I already own, but that's a really good deal. That's what the price should be. Was that it? Did I go through the 21 days? Wow. I'm good. Yeah, that's all we have. So there we go. Sorry, that was an abrupt ending. But there we go. Those are my thoughts on the Ulta 21 Days of Beauty sale. Lots of items that I'm like, okay, yeah, you could pick them up. Lots of items that I didn't even mention because I don't know. I don't think they're worth it. And then a very few items where I was like, all right, this is actually a really good deal if you want it. But for the most part, I find that the Ulta 21 Days of Beauty encourages me to spend money that I wasn't going to spend in the first place. So even though you think you're getting a good deal, you're actually spending more money than you would have. So normally with these sales, I'm quite underwhelmed by the 21 Days of Beauty, so I don't pay attention to them because I will buy stuff for no reason. So my advice, unless you see something that you actually need to stock up on, like a product that you actually use on the daily and you would repurchase it at full price, is to stay pretty far away from this sale. They're tricking you. Okay. Anyways, I hope you guys enjoyed this video. I hope this was helpful to you in some way. If you aren't subscribed to my channel already, I would love it if you would consider taking this time to do so, and I will see you all in the next one. Bye, guys. Have a good one.