All right Dave and we're all set great. Good afternoon. This is a convening of the master's gaming commission. It's our third convening Which is pretty good given that it's only noon we had a meeting at nine and then a genocidal meeting at 10 and now we're turning to some sports way during division Reports before we continue. I'll take our roll call because we are holding this meeting virtually Good afternoon commissioner Ryan. Good afternoon. I'm here Good afternoon commissioner hill Good afternoon. I'm here Great. Good afternoon commissioner skinner. Good afternoon and good afternoon commissioner maverick Good afternoon Okay, we'll get started with our public meeting number five On three and I'll turn right to our sports way during division On director band. I see your video is off because you are relying on manager boscheman. Good. Good afternoon crystal They're him. He's nodding his head. There we go Good afternoon. I don't have a lot by way of introduction but um the scandal team did say they were ready to Provide their quarter for reports. So I'll just turn it over to them Thank you Good morning or good afternoon chairwoman and commissioners. I will go ahead and share my screen Um Give me one moment. Are you able to see my screen appropriately? Also Okay, great We have this morning a the the crew that you are are used to seeing maybe you've now gotten Used to seeing rich cooper in my place, but we have key to young back and gelat comes as well Our agenda is as follows as it has been in prior months We're going to talk about revenue and compliance talk a little bit about de and I both workforce and our vendor and suppliers spend and then responsible gaming and charitable impact The revenue numbers are here as as we've submitted them You can see that we had a little bit of a dip in margin in the november time period Uh, that was based on customer friendly results generally speaking and I think was seen industry-wide In in during that period, but otherwise these are largely in line with our expectations Happy to answer any questions about these as I or I'll keep moving Um and Cory a point commissioners Why don't we just chime in as Cory goes along if you have any questions because that will help him anything And if we hear none, Cory, we can always return. So thank you. Yeah. Happy to take anything as they come up Um next is our miners and underage report and again, you see that it's zero Obviously Vandal had discussed this issue an executive session with the commission. Happy to answer any questions on this as well Not hearing anything I will hand things over to keyda young to discuss some exciting progress. We're made. Oh madam chair Yes, commissioner hill. So I was just having issues getting off my mute button So I apologize. It took me a second to to get to this compliance Screen so yes, we did discuss this in an executive session But I I just want to reiterate and hope you understand That there are some who believe these numbers, you know, aren't realistic And we understand during our executive session, you know, what's going on and how you Are dealing with this but I just want you to understand that for some of us on the I think all of us I can speak for but for some of us for sure That underage sports wagering is still a very very big issue for us And we are still concerned when we see zeros coming up or at least I am as one commissioner So I just want to keep that in the back of your mind that as we move forward in the next Few quarters that you know, you also make this issue You know a top priority in ensuring the best that you possibly can That our minors and underaged folks are not getting into your Site in order to do, you know gaming it's still a very very big issue for us Well received and and we do take this very seriously I I don't anticipate unfortunately that they're or fortunately depending on your point of view that these will always be all zeros I think though there will be numbers that come up here And we will continue to look for ways that we can enhance Our our prevention of underage wagering Enhance and identify. I think I would like to add to that It is in the front of our minds and and we we take our obligation here very seriously Thank you, madam chair. Yeah, just for the record commissioner hill You're alluding to an executive session we had recently with individuals sports wagering operators who offered um tactics and strategies with respect to kyc matters and proxy Wagering and because of the nature of those tactics and strategies. They were conducted in executive sessions. So Thank you, cori. I know it's hard. Um, let me really Have to we're we're straddling that careful mind there. So thank you Thank you So with that I'm going to hand it over to Keita young to discuss our diversity equity and inclusion program If I can Go ahead. Good afternoon everyone. It's I'm pleased to be here again in the start of 2024 Keita young head of de and I So each time I come before you I'm even more excited. I think this is my third time Here before you to talk about my job our company and the continued efforts we're doing in the de and I space So I think as you all recall a little every year ago It was hired to develop the de and I framework and strategy And I can tell you that this company after a year later I have learned is not check the box company when it comes to de and I And it continues to be intentional and invest in our de and I efforts. So when I started a year ago It was just me by myself. I think y'all remember but I'm here over a year and a half and I've hired A program manager a de and I program manager and recently just hired a de and I specialist to assist with our further implementation of this strategy So as you know, we have four focus areas Diverse source and hiring talent and development community and culture and employee engagement and communications So within these areas, I'd like to highlight and showcase for you some of our 2023 initiatives It's it's not a comprehensive list, but I just wanted to kind of bring you guys up to speed really on what we've been doing So first in our diverse sourcing and hiring So in this space our team my team has been working side by side with the talent acquisition team understanding that representation matters and we've dotted a couple of initiatives to bring intentionality to our intent and really to create impact We are establishing a diversity recruiting committee to make sure all of our businesses Are aligned with our efforts our de and I efforts and are implementing these initiatives We've implemented a diverse slate process For our leadership roles when we talk about diverse slates We are diversifying the pool of applicants that are considered for key roles And bringing forth candidates with different backgrounds Experiences races and gender classifications in an effort to diversify our workforce We believe here. You can't hire who you don't see We're proud to share about our newly created pipeline projects. We have a veterans internship program We have a uk returners program, which is really special to my heart because that's my story I actually am a lawyer by trade I took 11 years off and never thought that I would get back into the workforce But at another company returns or a returners program and here I am Testifying I'll be in front of all of you really this changed my life And I'm really excited that fan duel is actually taking us on with the uk returners program We've implemented diversity outreach initiatives We've got a hbcu community connections event that we hosted this year And we also did a bet on black tech summit for black black technologists And these are just to name a few of our initiatives that we were able to accomplish in 2023 In talent and development. I think last time I was here. I spoke about our partnership with McKinsey So we graduated over 30 black employees that participated in a three month leadership development program with over 30 hours of learning And I'm here to also share we just launched our Asian leadership academy with McKinsey with over 40 participants And we're gearing up for our Hispanic leadership academy to start in second quarter In our community and culture and employee engagement pillars We actually have had a year long last year of cultural moments celebrations activations to bring awareness to the diversity of our community And how special that is we celebrate diversity in all different forms One super proud moment of 2023. I would like to share today is our expansion of our erg program When I started we had four grassroots organizations and now we have seven official erg groups That are formally recognized with charters executive sponsors budgets And there's a member of my team solely dedicated and responsible for the oversight and support of these groups Last month we flew in all of our erg chairs to new york for a curated erg summit We set them up in the board room here We brought in an external subject matter expert and we had a full day dedicated to upscaling and setting them up for success Our ergs also have professional coaches that we provide to them separately to further demonstrate our investment in critical nature of their roles We support our ergs wholeheartedly because we truly believe that they're a critical pillar of fandal to support and amplify our diverse communities You can see on the next slide our proud logos of our seven erg groups That was something that we worked on they all have representation and that's something that is shown all across our community So they know our who our seven energies are And the last initiative I'd like to call out is our huddle series Which is our d and i listening sessions would provide an opportunity to hear from our colleagues On often very hard topics, but they provide insightful information for all of our community We've had things on identity, allyship, cultural fit versus cultural ad, inclusive language And last week we had a session on code switching These discussions allow us to drive forward our d and i efforts in a way that allows our community to listen and be heard Hundreds of people join this our discussion And I can tell you that my email is full of thank yous from those who come to listen Those who want to be heard and those who walk away with open eyes and new ways of thinking So we've talked a good number about our initiatives and what we've been able to establish But I think what's more important is the impact of all of these things that we've done So the impact in the in the focus area of diverse sourcing and hiring Is that we've had last year 51 of our overall new hires have been diverse and that's because of our efforts We've established stronger ties with With diverse pipeline organizations that we're really happy about The impact in our focus area of talent development Is that we've had over 100 participants in our women's leadership development program, which I talked about last time I was here and we have over 80 diverse associates and manager level employees that participated in our mckinsey leadership program Our impact in community and culture like I talked about we revamped our erg framework and structure And we've created a culture that openly discusses d united topics to create a more inclusive environment And then the fourth pillar the impact in our employee engagement and communications We now have quarterly metrics reporting that I give personally to our elt our most senior level employees That provide benchmarking and they measure our progress in the d eni area We've got d eni training and education which brings forth d eni to the forefront and assists all of us in our journey So the next page i'd like to turn to is our workforce numbers Um last time I reported to you guys at our minority level we were at 35.6 percent Of our employees at itself disclosed As of today we're 38.5 percent With respect to our representation in for women last time I was before you it was 29 percent and today we had 30.5 percent You can see our veterans um is unknown right now But we I can tell you we are launching I promise a self ID campaign Where we're going to get the information of our veterans population as as well as other diversity classifications Which we currently are not tracking So the last thing is our supplier diversity Fandall last year committed to developing and executing on a plan which involved identifying and engaging minority-owned women-owned and veteran-owned businesses In support of our external diversity efforts We outlined our comprehensive plan with phases as you can see on the screen And we're pleased to announce today that we have contracted with a vendor supplier I o who's going to be assist us in our efforts Who is currently right now in the process of assessing our vendors building dashboards and I can assure you by our next Meeting that I'll have more information about where we are with with respect to our our vendor supplier efforts So that's um in a nutshell the report of what we're doing in the d and i space. I'm open to questions if anybody has any pressures Any questions on No question no questions madam chair just a few comments I'm key to hi I I always it's it's nakisha skinner I always like to Hear from you because I think that you are the real deal fantastic DEI agenda It's it's a full agenda and Your intentionality here It is not unnoticed and I think Fandall made an excellent investment Um because you know your excitement Just shines through and and I as one commission I think I can speak for my fellow commissioners as well truly appreciate Truly appreciate your your energy and I think you started by saying Um, this isn't a check off the box situation for for you folks and and and you've really demonstrated Uh that it's not and and I just want to thank you Mr. Skinner, thank you so much for that and um everything you said is is totally accurate and I will tell you You know my I report to the chief people officer and she says to me all the time You're doing all this great work and you've really been able to make an impact And I tell her I'm the same person that I was at my prior company But this company allows me I think to really have wings on my back and I really and I can truly say that from the bottom Of my heart they're really truly invested and I love working here And I love all the work and the investment that they're putting in the d&i space and thank you for recognizing that Oh, thank you. I think uh for sure Skinner summed it up for all of us um Cory Yeah, we can we can move on to to Jill who's going to address our responsible gaming program Thank you I chill first up. I just want to make sure I said thank you to kieta, but I think you might have an Okay, perfect. I think we're okay now. I hope yes. Okay, perfect. I Love you to see you all again and also commissioner skinner. I agree. I think he does an amazing investment And we're very happy to have her here as well Um, so last time I came and spoke to you guys about responsible gaming. I will do the same here today Um starting with our tool usage. So last time I came and spoke I talked a little bit about how we were sending it monthly notifications to customers to try and get them to Access their pay activity statement Which for us is our way to try and reduce this cognitive distance We see arranged people wanting to use tools and get people to Look at what they're spending going to give in month and take appropriate actions So those monthly pushes have continued and we have seen from the back of that continued growth in people Accessing their player activity statement. So For the last quarter in massachusetts about 45 of our active player base in massachusetts looked at the player activity statement We are still trying to think of strategies this year for how we want to increase some of our core tools Across the board and we have some exciting product work coming out In q2 and q4, which we hope will address this and we are looking forward to be able to talk to you guys about that in upcoming sessions But we feel positive about our plans for our tool usage across the board And our hope is that by the time we get to the end of the year We have reduced this statement that we know exists around people using tools We have more people engaging with these tools In a way that aligns to how they want to play Moving on then to touch on people who have Taking advantage of the voluntary self exclusion. We had 77 people in massachusetts in q4 engage with this We've seen that about 0.2 percent of registered massachusetts accounts have also elected to take a time out in q4 With the average length for that time out being about 69 days I believe last time I came and spoke that was around 70 days. So It has remained fairly consistent over the last couple of quarters in terms of length of time out And then we continue to direct customers to the game sense website to find out more information about the voluntary self exclusion Program and what is available there for people to access to If we jump on again corey, I think if there's no more questions about that, I'll jump in for an update on operation hope Um, the last time I came and spoke to you all we were about to roll it out I'm happy to be able to say that in january we did and we launched our partnership with operation hope Which is america's leading nonprofit dedicated to financial literacy The program is currently live in massachusetts and it's giving individuals in massachusetts access to financial well-being Coaches giving them virtual and in-person financial health sessions at no cost to them We currently have about three coaches Working on this and that means that any one time we're able to roll the service out to a few hundred people With the aim being that around five to six thousand massachusetts residents will benefit from this over the course of the partnership We have been able to promote this. So we have Um videos here. We've also promoted this on meta snapchat Radio in massachusetts are in the boston area Out of home and we've put an awful lot of financial spend behind this In order to support it and really make it known to people that's there So hopefully people might have seen it at bus stations heard it on the radio We've gotten loads of really nice imagery of it being out in the wild which has been amazing for us Um, and we've been able to work with operation hope to build it It's a really nice creative to hopefully really drive people towards the landing page where they can sign up To do the various different workshops I'm hoping that we come back In q2 we'll be able to give you more Numbers in terms of how many people we've been able to serve so far The way we're looking at it is we're trying to gauge What good looks like based on credit scores and stuff like that So as people engage with the course the hope is that their financial Well-being metrics will be able to increase and we'll be able to really determine if this has been the success And how we then expanded and rolled out further So jill, I um I'm really thrilled with this partnership Just this morning. I was speaking with commissioner maynard about An initiative we we tried It was very informally done not didn't gel enough to really survive change administrations, but uh The idea that this could help those who are in debt that's been Sort of at the top of my list since my arrival You know, we noticed more sports wagering was was even legal That there were those who found themselves in debt in in some publicly and resulting in some uh, you know criminal behavior And I just feel as though there should be opportunities for folks to understand how they can get out of that crisis and not have to know in any way Turn to illegal measures to get themselves out of debt so I'm really thrilled to see this and I hope that your efforts to advertising are Are successful I think um commissioner skinner. This is something that perhaps treasurer billberg would be interested in And that's an opportunity for you to forge more relationships here, massachusetts treasurer billberg does have a financial literacy and Just a way to to spread this this particular Effort that I understand it's being run in partnership with this nonprofit It's exciting Yeah, we're excited about it. And we think there's It's just a really good way to help empower people as well Um, it's really kind of help them understand and hopefully teach people life skills that will carry on far past Like we're we are there. Um, so no, we're really really excited about this And it's one that we're very very happy to be able to say we've officially launched And we have lots of plans and operation. Hope we really enjoy our work with them So our plan is very much trending into this and and really make this a success because the general The general vibes are infundal is that this is something we should be doing and doing more of and it's it's just a good thing For us to be involved in the industry generally Madam chair. Yes, thank you for your comments Regarding the treasurer's office and and the programs offered there Mark and I kicked around a couple of ideas Not that long ago, but not very recently on ways we could take advantage of A partnership with the treasurer's office around the financial literacy programs. And so you're absolutely right. This is an opportunity to forge new relationships Going down this path. I have a question for jill as well Um, I know it's a fairly new partnership just launched in january I don't know what the volume is and I look forward to hearing those numbers From you at the next quarterly report You have three coaches Is there an opportunity to to expand that number to And to to accommodate, you know, any demand that you might see Beyond what you originally anticipated. And I guess I'm assuming That the program is available to anyone that wants it right like there's no cap on the number of enrollees Correct. Thank you for those questions. Go to your center. Um, we can flex that off I believe the last time I spoke to you guys we'd originally hoped to launch At the end of the year But we had some issues with the hiring of the coaches that took a little bit longer What the plan is that if we see that the demand is there We are able and we are willing to lean in and hire more coaches if we deem that like We need more in order to be able to facilitate the volume that we're seeing so Our plan is to run it for the first six months Then review and see do we have the right amount of coaches for the volume you're seeing? How many more would we potentially need? Are we seeing volumes less than we anticipated? And if so, how do we lean in heavier and get those volumes up? But we are very much Willing and want to lean into this as an opportunity. We just need a couple more months to really understand how it's performing Thank you Thank you I think the only other things I have left are how we've been promoting Operation hope we have a video here and then just some visuals around what we have been putting into the market Hopefully this actually works. Perfect. And then We just have some visuals of what it looks like and introductions to coaches then on the follow on the following slide I'm trying to get there Super so here we just have some an introduction to who the coaches are in massachusetts Um, what the visual is that we're going out with what our taglines are and what our kind of log looks like with operation hope And how we're leveraging it. So on the far left when um, people go onto the landing page This is what they see they can read a bit about the coaches Understand who they'd like to work with and then go ahead and set up a one-on-one coaching session Um, or they can opt into attend one of the workshops that are on offer as well So more to come on this in future meetings. We're very happy to be able to tell you all that we are live Um, I'm looking forward to our future and our work with operation. Thank you all Questions commissioners, I think the the lack of questions only indicates Everyone's satisfaction and and appreciation for your presentation While setting commissioners, uh, chloe. Thank you for bringing your team here today Thank you so much for your time today. Thank you guys. Let me see. Oh the two of them are together. That's right. Yes, we're together We'd like to see that you're a tag team. Exactly. That's excellent. That's excellent. Great to see you all take care Thank you so much Okay, crystal. Um, next up We're gonna hear back from wind back I think that that mgm is up. Is that clear? I have wind when that Okay, then you might be right. I can't pull up my Agenda faster. Oh, hope so. Well, no, I've got that mgm in front of me. Um I've got the full so That mgm my apologies that mgm my apologies. Yeah, okay. No problems. We know what I went to I went to to my apologies Um, I I should have corrected on the record. We did it in the earlier Meeting that we started at the end. Oh, and thank you to the mgm and wind back for accommodating their schedule appreciate it. So the mgm. Thanks, crystal No problem. I think I see sarah. Do you have everybody in the chair? Are you ready to go? I see sarah Hi Hi, how are you madam chair? Thank you commissioners for having us this afternoon Um, I am joined by rich taylor director of responsible gambling josh wiseman director of licensing jilliel morton uh head of derisine inclusion for bet mgm and we've got rich eggy Uh, regulatory senior analyst or regulatory admin who'll be running our deck for us So rich if you can share your screen you'll get started I let crystal know already, but I have a Child who just came home sick right before this meeting. So she makes a cameo Meeting madeline as well again. Yeah, we look forward to meeting madeline. You don't have Roll here. We like cats dogs and yeah, especially small children. So thank you. Well commissioner O'Brien. It looks like she's in the office today. So I won't see her Cats today. No mango today. No mango today. Yeah Um, okay, but rich when you're ready you can pull the deck up Rich eggy, right sara. Yeah rich eggy Oh, there he is I'm sorry. I thought I was sharing but I guess I'm not here. I go Okay There you go. I apologize. Thank you All right, um, as I said, this is our q4 report. Obviously rich you can go ahead and switch revenue, um As anticipated revenue went up 10.3 million Was just over 2 million collected in taxes. This was anticipated with the organic sports calendar Next slide Julio will present on human resources Hi, everyone. Can you hear me? Okay Yes, we can Awesome. Hello everyone. I'm julio for those commissioners and of course madam chair who I haven't had The pleasure of meeting yet. Um before I get started. I just wanted to begin by saying that Up with a foreseeable future, um, I'll be here to serve as head of dei for A bed and jam in the short term and I'm looking forward to working with everyone Of course in our business and with all of you to ensure we stay at the forefront of dei best practice Uh as well as remain accountable to all of you within context of a diversity equity inclusion Uh, so looking at our q4 diversity numbers We've split the data of course along three identities and by level in the business Um as a q4 last year, we had around 1332 employees 48 percent were minority and underrepresented populations 34 percent were women And around 1 percent are veterans and um as its residents respectfully Respectively, um Our bed on vets erg is currently working on a strategy that educates our people leaders about veterans Who are looking for civilian roles in the workforce? What those sorts of resumes look like? Usually they can look a little different than a standard civilian resume So that we can try to organically and systemically see more veterans in our employee Our data relies on self-id and so i'm in discussion with our executive team Our erg leadership and other internal stakeholders To conduct an updated self-id campaign this year so that we can have the most updated data available to us Any questions at this time on this Also Next slide rich vendor utility Uh global vendor utility you can see our spend increased By roughly 60 million quarter over quarter And then the breakdown of those expenditures Is below Next slide And our local vendor utility spend Went down on the whole where 2.2 overall quarters 2.2 million And excellent compliance. I'll hand it over to richard taylor director of responsible gambling Thanks, sarah. Good afternoon commission. I'm madam chair In q4 we had one reported instance of underage patron gaining access to a bedmgm account We filed a suspicious activity disclosure regarding this incident With the mgc in in the in q4 Essentially what had happened was the father Reached out to bedmgm to report that his son had gained access using the mother's registered bedmgm account An investigation was conducted and this was confirmed and so Bedmgm took action we closed the account and filed that that sad with the mgc Next slide, please In addition to the one confirmed Incident of suspicious activity involving underage access. We also had three other Suspected instances The suspicious activity team conducted a thorough investigation of each of those and we determined that these were Instances where individuals had lost money Wagering and they were making claims in in an effort to recoup those losses As a result of these claims and our concerns regarding their ability to play responsibly and sustainably on our application That mgm made the decision to close all of the accounts and provide resources For problem gambling in massachusetts In addition to that the one suspicious activity filing Account was also closed due to the the nature of the issue Involving that that uh that account holder and the family Any questions regarding Yeah, rich, maybe the other commissioners have questions first anybody I Maynard and I can hold for your question madam chair My apologies, I can hold for your question madam chair. No, that's fine. Go ahead commission maynard I was going to ask is rich. Thank you for walking through those on the Last situations you brought up Is it standard to close those accounts when a parent? reports That this has happened It is um and the reason for that is if the patron I'm sorry the underage individual was able to gain access using the parent's account one time Our fear is that that can occur again in the future and for us It's not worth it to to take that risk and allow that that event to occur again in the future So we make the decision to be You know as safe as possible here and to close the account because we just don't feel comfortable Allowing it to stay open and to do business and for that risk to exist within our platform Thank you rich. I'm going to throw something out for food for thought for both my fellow commissioners the commission and for you Which is at one point are we disincentivizing parents? Saying this is an issue and are we potentially cutting off Another stream to get this information because as commissioner hill said earlier, you know, no one believes it's zero and um You know, I won't go into what we talked about it what the chair referenced earlier our Our executive session last week, but You know I just want that to be thought about what you know at what point are we saying to parents if you report that this happened Um, and we turn off your account that they say well We may just not report that this happens and that's something that I've been thinking about and chewing on myself Uh, I as well and and I think that's a fair fair thing to consider Um and something that I'll take back to the suspicious activity team and leadership here at bed MGM to have a discussion about um, you know, really it's about how comfortable we feel assuming risk and Um, you know, what kind of possible outcomes may occur by allowing us by allowing that that account to stay open um But I I see your point I mean, it's it's really the the issue that we face in general when it comes to problem gambling We want people to be forthcoming and to come to us if they have any issues or challenges But with that information we have to take certain actions to not only protect The customer but protect the the business as well But at the same time we don't want to be in a situation where that action is discouraging people from coming forward So it's I wouldn't say it's a catch 22, but it's very complicated and challenging Um, and and I will take this back to leadership and we'll have a discussion around if this is disincentivizing The very behavior that we're looking for which is for people to be forthcoming when something illicit is occurring So my question is probably not for consumption in the open meeting But I'm curious as to the three suspected instances that were deemed not to be Underage access. I'm curious what the factors were that made you conclude that But I'm also mindful of the fact that I don't want you talking about investigative techniques necessarily in this setting I appreciate that commissioner, um, and I would offer up to have a executive session about that We could go into detail about how the team investigates these and what information we're using To determine um, if it's a false false claim or not. Um, but I I agree. I would not want to share that information in the setting So, but I'm chair. I'm wondering if maybe the next quarterly we could mark the agenda To have that option. Um, because I Get print on it Um, can I do a follow-up on the three rich And I may have misheard I'm hearing commissioner brian wanting to understand More about the investigatory techniques Did I hear you say that the three? um Were concerned about their losses. Did they allege that it was their children? Did I hear that right? That that is correct. Um, and and this is common in other jurisdictions as well. I shouldn't say common This occurs in other jurisdictions as well Where there will be a substantial loss. Um, it doesn't even have to be a substantial loss there will be a loss and the player will write in and say that their Their child had gained access to their app and had placed wagers that they didn't mean to To place and they would like to refund And usually the ask for the refund is is one of the bigger Tip offs that that this is likely A false claim, but we go and we go in we check the data. We look at a lot of different Data points and from that we're able to to draw the conclusion if it was a legitimate Underage access or if it's a claim for for recouping losses Thank you Ready for the questions Next slide. Thank you rich I'll be doing the responsible gaming tools update Rich if you can go to the next slide Um, so in q4 we had 1089 use our cool off 187 Voluntary self-exclusion uploads were submitted to our back office system Um, I'd like to ask the commission. I noticed in vandals presentation that you're they're using percentages Is there a preference in how you want this data presented from the operator? I I'd be happy to To to be more consistent so that you're you're seeing apples to apples versus Thank you for that rich. Um The there was a percentage in the last one commissioners should return to crystal to see is there A template that you're thinking of crystal um So we had originally asked for this by solid numbers. I think over the course of the last few Quarterly reports certain commissioners had asked what percentage that was and so some operators are now also giving percentages or giving percentages instead I'll note that the data That is on this slide actually all of it goes to long and he puts all of the numbers together In full so that we have a an overview of the entire market and that's where a lot of this data comes from so We have a better picture in some of the reporting he's developing That would be more of an analysis percentage wise and for the whole market So it's really up to you guys, but that's how this is progressed over time Yeah, chair if you don't mind I I would like to in future reports will provide the percentages as well as the full numbers just so that You have a good comparison across operators. Um during these meetings. Thank you. Did you do that math today? Not not in my head. I haven't had enough coffee. It's still the morning time here in los vegas Well, and I appreciate that it is. Um, it is a good morning still for you Commissioner mander and I were pointing out that we were both having an afternoon coffee and it may come back to us Okay, thank you Uh in in closing uh for the rg section. I want to highlight in next meeting. We will be doing a um A good recap of our problem gambling awareness month activities as i'm sure you all are aware March kicks off problem gambling awareness month and we have a lot of exciting things planned including direct emails to customers social media campaign Supporting other initiatives such as the cambridge health alliances problem gambling screening day And so it should be an action packed month, but highlighting all of that is um our commitment to Re-engage with epic global solutions to provide lived experience sessions to all of our employees during the month which i'm very excited about these are very impactful and um, you know I'll extend this invitation to all of you if you're interested in attending one of these I'd be happy to send you a link so that you can observe what these sessions are all about for our employees in addition to that And this is really the the most exciting thing to share and we'll share a version of it once it's ready During the next quarterly meeting. We are just now finalizing our very first responsible gambling television commercial Which we intend to play in markets um around north america In the not too distant future and it's using one of our top ambassadors I'm very excited about the the way it's turned out. I don't want to give away too much as we haven't aired it yet But it's a a fresh take. I believe on how you can promote Responsible gambling tools to the public which I know is a big concern of the commission as it is a concern of ours Um, and it's the first of many We all we all already have plans for additional ambassadors to get in on the fun of promoting responsible gambling So it's going to be an exciting 2024 for betmgm and rrg program Any final questions before I turn it over to josh wiseman to discuss lottery No, I'm excited about that though rich. Thank you. I am too. The next meeting is going to be a lot more fun than this I promise All right josh all yours Thanks rich. Uh, good afternoon. I'm chair and commissioner. It's nice to see you again um for our update on the lottery Um, this quarter is just to say that we betmgm is still having active and meaningful conversations with the lottery Um, and we're still still very engaged and excited to to explore our partnership with them Um, we haven't got to that point yet But uh, but I'm looking forward to hopefully being able to provide a more substantive update and I'm recording the update next quarter Thank you any questions josh I'll set. Thank you. Thank you Awesome. I'll be presenting on our community outreach initiatives Madam madam chair. I'm sorry to interrupt. I'm having trouble getting on my um getting off of my Um Mute button, so I apologize But I just wanted to to bring it to betmgm's attention That we are watching very carefully What the legislature does for the lottery during the budget process this year where they may allow some Uh They may allow the lottery to be able to for us to be able to go online and do our lottery through Through different means and I think at that time that's when there may be some opportunities if that is adopted Uh for for all our operators to be more involved with our lottery than they currently are I just wanted to let them be aware of what's going on In terms of what could happen in the next three months in regards to our legislative Um actions when grad to the budget That that's very helpful and I'll make sure that the distance is the right amount. Thank you commissioner Thank you Community outreach to heal Awesome. Thank you. Uh, so in terms of community outreach our erg's employee experience team contributes to a lot of the work That betmgm is doing As a q4 last year, we actually established our first Functional erg strategy and embedded in that strategy or pillars that each erg circle One of the major pillars across many of our erg's is community development and community outreach So this is a priority for us and we're looking forward to having a great relationship to our communities and serving them Um as far as our erg's go We have seven groups all equipped with leadership teams composed of employees across all levels of business Uh as well as executive sponsors that champion and advocate for these groups Additionally, each group has a robust membership of employees who voice their concerns Share resources with their colleagues plan celebratory educational and thought-provoking programming and On this slide, we have the mission statements of each group and this year We are looking to have these groups continue to serve their communities but also to collaborate together In an effort to bring intersectionality to the forefront Rich can we go to the next slide? In q4, we had a few community outreach events that we just paid it in We were pleased that these were able to take shape given the flexibility of course needed during q4 Specifically with folks traveling and the holidays things of that nature Firstly, we had our latinx heritage month school supplies drive And this was a community led initiative via our latinx alliance erg In which people in our business donated school supplies or bought school supplies for our amazon gift basket And our actual latinx alliance erg leadership brought all of the donations to the school It is a school with a relatively high at-risk student population And we wanted to be intentional with the school we selected to make sure that this initiative had as much impact as possible Um secondly our adopted family program This initiative was led by our employee experience team and they partnered with volunteers of america during the holiday season to provide essential items to families in need This year we were able to serve four families donating food gifts toys Clothing et cetera any other essential items that they needed so we were very happy to participate in that as well And then finally our bethams and usos salute the troops initiative So this is an initiative that is organized by our regional marketing teams And due to the heavy military and veteran presence in the communities we operated They wanted to develop a relationship with those who serve our country We serve breakfast to Around 170 service members at this event specifically in support of mgm resorts larger salute the troops event And that team also partnered with usos for a surprise and delight initiative for those service members That conduct the funerals at parlington national cemetery. So a very cool initiative as well um I would also like to highlight that we are ramping up with urg programming in 2024 Including just had robust black history month programming that is taking place now As well as a thorough slate of programming for women sitting month in march Some examples of these would be our representation panel that we had With the business for black history month and a couple more conversations in the same vein And we had a store in march that will be powered by women's urg to this quarter as well that we're looking forward to sharing at our next meeting Any questions can be shared on this slide or anything that i've talked about to this point awesome Hearing no questions, uh rich moves the the next slide So lastly we have our our mentorship program how this really came to be Was that we were hearing from the urg's and the representative employees that they needed more opportunities to grow their skill sets develop And connect with business leaders. So we responded with our empower program Basically, we identified people leaders across the business who satisfied the criteria That could serve as mentors in our program And for mentees we were intentional in having the urg members participate since they represent A good majority of our underrepresented employees and so we really wanted to make sure that they were able to participate in this Um, we encouraged mentor mentee pairs to meet at least an hour per month And we actually found that many meant for longer periods of time and have actually kept in touch for the other monthly bi-weekly things So that was a great long-term outcome of the program that we observed Um, moreover, we were to graduate all 32 of our mentees from the program, which was fantastic Um, our second cohort will be kicking off in a few months And right now we are putting our program plan in place to make enhancements from last year's uh cohort So we'll have to look forward to and more to come in our next quarterly fall any questions on our empower program commissioners Fishers Also, I see from pressure hill and don't care from anyone else Awesome. Thank you so much good work That closes um our presentation if there's no questions then Thank you very much for your time from the bed MGM team Sarah, do you just mind um just pulling down the um The slideshow please so we can just say good bye Thanks so much. Well, thank you since Sarah everybody else Madam chair, just I want to say thanks to to the bed MGM team and uh A welcome to these meetings to julio Nice to meet you Thank you very much. It's nice to meet you as well. Looking forward to working with you Julio has been behind the scenes for us now for a while and does an excellent job. So we're very happy To include him as a spokesperson for bed MGM. So he'll continue to do excellent work for bed MGM. So thank you Well, welcome to this public forum. Yeah, no pressure, Julio Then you know what there should never be any pressure excellent job today Welcome and thank you, Sarah and rich always great to see you. Thank you and josh. Thank you so much. Thank you Okay we will Then now turn to win that and jennifer you you were with us very early I'm presuming you are in las vegas I am in las vegas 9 a.m. Lots of coffee so We appreciate you hanging in there for of course. No, it's it's nice to see you again So, uh, good afternoon, madam chair members of the commission and commission staff jennifer roberts E.p. And general counsel of win bet and I will pull up my slides so because We were operational within the fourth quarter of 2023 Uh, we are presenting our quarterly report And we will start with the revenues that have been Already reported to the commission So Yes, jennifer. I'm sorry to interrupt you. It might just be my screen, but I don't think it is It's yours is really small what you're projecting here on the screen Okay, let me It actually looks like an iphone Like it's Oh, that's strange. Okay. It's on my monitor, but let me try another route Jennifer happened to be looking at my monitor Where I bet it projected from our package. So that was just a little small Okay, let me Okay, try this again I have mine up as well, but just for people tuning in from the public it might be helpful to Show it on the screen Absolutely, let me just Trying to get this to work Okay, I'm going to try a different route Nobody's nervous so Are you able to see this Right now it's black up there. It's from there we go. Um, you do have you know these full slide, um I tried that in it and it shifted it over to the other screen. So I'll just leave it as is to give you And it'll be easy for you to go right through Perfect. Thank you. Okay. Thank you Okay, so we have the revenues that have been reported. Um, this is a snapshot of the entire year so And it includes the fourth quarter revenues As well as the taxes collected and the amounts wagered So our workforce diversity, uh windbed has always, you know promoted a Diverse workforce, uh are Obviously with natural attrition Due to the status of our operations. We've had some decline in numbers, but we've maintained pretty consistent numbers with our female representation and our Minority representation within our workforce This also includes numbers and representation at our Executive and management levels. You can see here that we've had have As of q4 2023, we still have maintained pretty consistent female representation at the Executive levels and similar numbers as well in minority representation And again throughout our operations our initiatives for employment has always Has always been to recruit higher promote and retain diverse talent across all levels within the company and uh, we believe that we maintain a workforce that reflects the representation and Of our initiatives and includes Again has always included several women in executive leadership roles For as far as vendors We have a breakdown. We did the percentage breakdown of diversity spend Again with the Announced closure of our business, you know, we've seen The diversity spend some diversity spend decline, but And it's been challenging through windbed in that, um, you know, we Did not require our vendors to disclose their diversity certification But we had always been dedicated to Identifying and contracting with diverse vendors during our operations From the compliance perspective we, um, also similar to our the other operators in the space We did not identify Or discover any attempts to access our sports wagering platform by minors So we did not have any numbers to report And through our responsible gaming initiatives, we had during the fourth quarter 88, uh, voluntary self-exclusion enrollees Um, obviously you're going to see numbers increase with the launch of football during q3 that we have, uh, maintained our self-exclusion And responsible gambling tools for for all of our patrons throughout As far as the lottery, uh, we, um, with the assistance of our affiliate on core boston harbour We made attempts to meet with the, uh, lottery to help ensure that the lottery was not negatively impacted by sports wage during we did not, um, Uh, Have a meeting that was able to make any meaningful steps before we announced our closure, but uh on core boston harbour Still continues to Collaborate with the lottery And finally on our community outreach and charitable, uh, impacts, uh, all wind bed employees Are invited to participate and contribute to, uh, community engagements that are sponsored by the wind group of families In fact, several wind bed employees participated in feed the funnel, which is a very Which is held at both on core boston harbour and Wind los vegas. It's an event that helps prepare meals for local food banks and, uh, Several of our wind bed employees have volunteered their time and have made charitable donations The majority of which are nevada centric as our headquarters Happy to answer any questions about wind bed I saw there's only questions for jennifer, um And we can go back to slides if you wish commissioners And you can take down the um, this This uh slide show then jennifer so we can i'll see each other I know it's interesting that, uh When that your some of your vendors are less than cooperative around their diversity numbers I think in massachusetts perhaps it's just You know, maybe that's there's more openness to that crystal I don't think it's mandatory that vendor share right, but we just maybe it's a culture where that sharing is encouraged and Uh, I just haven't really thought about that as a nationwide challenge perhaps Crystal knows a lot about this area. So that's why i'm turning to her jennifer Yeah, I do think that's um, there's a little truth in that chair We we hear this sometimes that Um, you know, it might be a capacity thing and sometimes the smaller vendors do struggle to identify this, but it's also Because it's so built into a lot of what massachusetts does we we hear that there's a lot of um, you know more camaraderie around it so to speak and celebration of it and uh just familiarity with it So I think it probably depends it could depend on where these vendors are I'm I'm not familiar myself, but um, it may just be a cultural component there Interesting. It was some of our operators. I think I think you've seen and some for reporting today, right? They can get some of the the information and I think Actually commissioner skinner had asked some of this in the very beginning that we when When that and others were reviewing their diversity plan it was a little struggle in the beginning So I think we're seeing that some of them are coming on board because we asked But for some entities, it's just a very unfamiliar process Well to the extent that We influence and impact that that's that's okay for the mgc But I appreciate your you're mentioning it. Um, or reminding me I guess Of this challenge. So thanks jennifer Thank you Anything else? Okay, jennifer. Thank you. That's been a long day for you. We appreciate it Thank you very much Okay commissioners Uh returning to item then three on the agenda Any commissioner updates? Okay We've had uh reproductive meetings. It's a 111 three months seems like a winning number So I have a motion to adjourn Quick before it turns Move to adjourn Too late. Did I get did I catch it? One one two. Anyway close enough commissioner skinner you got the motion and that's all we needed Uh commissioner brian, did you second it? I did I want to make sure um Just watching the clock everybody else that Okay All right Commissioner brian I Mr. Hill I Commissioner skinner I Commissioner I I felt yes five zero Thanks and pistol. Thank you very much. Um next recording and we'll convene tomorrow For another group of three more three more tomorrow


Hey guys, I'm Abhishek and I'm a senior product manager at booking.com based in the Amsterdam office. So today I'll be speaking a little bit about my journey of applying some of the product craft skills from some of the previous domains I worked in to what I do today, which is marketing. So and hopefully there are some insights for you guys. So I believe this should be relevant for the experienced product managers who are planning to move to a new industry or also for early stage PMs who want to get a perspective around product craft while working in marketing industry. Yeah, so I'll start by giving a brief on the product journey I've had so far and I think this is quite relevant to the topic itself. So I started working as a product manager almost nine years ago with a fintech company in Mexico. So we are building mobile payments and mobile banking solutions. So as you can tell, there is a big component of B2C because we are building these services for the end user. So there's a component of building the checkout experiences or the payment experiences. But then there's also a large component of platform, which is about integrating with the banks, integrating with the payment service providers or integrating with the telco partners. So I operated primarily in the B2C and platform space. Later on, I moved back to India, which is my home country and started working for a healthcare company, but again, focusing primarily on printech. So I was building a marketplace payments. I was building the insurance services there is still part of it. Again, similar to the previous experience, there is a component of B2C, which is again around payment experiences, how people pay for, let's say medicines online, how do they pay when they book an appointment for the doctor and so on. And then there's again, huge component of platform, which is integrating with the services, integrating with the insurance providers and so on. And four years later, I moved to Netherlands. And I've been working here since almost four years. And as you can see, things changed quite a bit since then. So right now, I lead the product stack for performance marketing, primarily for the search advertisement business. So the general term that we use for this is at tech, which actually encompasses all the different marketing channels, but we use the term at tech. People also call it marketing technology or mark tech. So either way, essentially, what we are doing here is that we are building technology that support and scale the marketing activities that your business or the company wants to do to achieve certain business outcomes. So that's the core problem here. And then depending on what scale you are operating, there can be a component of machine learning, which is basically how do you use machine learning to optimize for the content you want for your ads? How do you target your customers better? How do you optimize for the budgets, let's say? So this is the domain that I'm currently operating in, at tech and machine learning, which is very different from what I've done before B2C and platform. And that's what I'm going to cover a little bit today. So yeah, so let's touch upon the agenda for today. So I will be broadly talking about two areas. First, we will try to define a broad landscape for this domain, the tech domain itself. We'll talk about what the industry we are in, what are the typical business goals for this industry, and we'll try to get a sense of product stack.  So we'll talk about product discovery, we'll talk about product strategy, and we'll talk about people management and how that changed when you move, let's say, from outside marketing to within marketing. There could be other craft areas, which are critical for a product manager, but I've chosen these three because I feel these are the three that are most impacted specifically when you move in marketing. For example, there is always a big component of product delivery or product execution, which can change. But in my experience, I felt the learning there was not that high. It's basically how you engage with your development team and how you build the process around it. So yeah, we'll limit our discussion to these three. Yeah, so first about the industry itself, right? So it's like I said, it's called advertising technology. What we are essentially doing here is to scale marketing activities for particular business outcome. You see there are different channels within this whole domain. So organic marketing is always very critical, and there can be a big component of tech in that, how you enable that. Then you have search advertising, which is primarily what I'm doing, brand marketing, email marketing, and then there are different channels within social media marketing like Facebook, Instagram, and so on. So we will not be talking about the specific product strategy for each of these channels, rather we'll take a more general approach and try to build a general framework on how do you approach either of these, right? So how do you adopt the product mindset needed to target any of these channels without going into the specific customizations of these channels? Yeah, so the key product goals, right? So if you take, let's say a B2C product in mind, some of the goals could be around increasing the number of transactions, let's say if you're an e-commerce product, or maybe increasing the NPS score, or improving the customer satisfaction, and so on. But in a marketing universe, typically you see there are goals around how do you drive growth, which is how do you get more customers? Then there could be specific goals around retention. If your company wants you to focus on marketing activities to get existing users to transact more, or maybe engage with the product more, then there could be specific goals around revenue. So this is beyond just getting more customers or getting the customers to do more transactions. This is about getting the customers over food, generate positive return on investment. So you bring in more revenue for the company by using marketing channel. And then there could be specific strategy goal, right? So how do you use marketing channels to differentiate yourself from the competition? How do you use it to launch for a new market or differentiate yourself into a new market? And there could be platform specific goals, right? If you want to drive that option of your app or get more customers to book through app and so on. So this is basically, this could be a very high impact area for the company, because you can operate into these different areas of business outcomes. So let's speak about a product stack, right? So once we have a good understanding of which industry we are operating in and what are the boundary of business outcomes that you can operate in, as a product manager, your job is to identify the product stack, what is the ownership that you could have? What is that you can influence in terms of product? Because compared to a B2C product, you might not be able to influence everything, right? You need to operate within the boundaries of your product. So you need to draw a rough idea or rough boundary for what those products stack look like. So this is how I see marketing technology and Act Tech in general. There is a base layer of data, right? So and this is, let's say, even if you're launching a B2C product, you need to track the metrics, right? So you need to get the data in. In similar fashion, once you're running marketing campaigns, you need to be able to measure those campaigns, right? You need to be able to measure the performance and the success. And if you're doing that at scale, you need to develop skills around how do you work with data engineers, how do you work with architects, right? So how the storage of data works, how the governance of the data works, because in a privacy-centric world, data governance is really critical. So all that can be really like a domain in itself. But definitely the base layer for a product stack for any marketing technology product. Then comes the reporting and tooling, right? So imagine you have all this data and you have figured out a way to successfully store and manage this data by leveraging your engineering team. You need to be able to provide access to this data to your users, internal users who are marketers who are actually implementing those campaigns, right? So and you will basically user-friendly access for this data to whoever is going to use it. So that's where the reporting and tooling aspect of the products that comes in play. And this is where you can potentially apply some of your previous skills around UI and UX. And you can really shine in here, right? So this should come as a second nature if you have done a lot of B2C work or done a lot of platform work. How do you basically build experiences for the users who are then taking some decisions using that data? Now, depending on what scale you are operating it, this could be limited to just these two where you're just leveraging data and building some tools and reporting around it. But it can also go beyond that, right? So for example, how do you automate some of the core user actions to run any marketing chart, right? Which is about creation of new campaigns, which is around optimizing for particular languages, optimizing for particular geographies, getting access to some of the insights, right? So then you can really drive some of your product thinking around automation. And lastly, production, which is where the ML part comes in play. So how do you basically leverage the data that you have to make some predictive analysis? So let's say if you want to get an understanding of potential return for a particular set of customers, particular set of traffic. In here, again, as a product manager transitioning into marketing, you need to develop certain skills on how do you work with machine learning engineers, right? And this is a really good opportunity for anybody who probably does not have that experience. And when they move to marketing, they can potentially earn that experience, right? So in here, you basically get an opportunity to define a business outcome or define a business problem using the capabilities that you have at your disposal. So this is broadly how a product stack is shaped up for any marketing channel. It could vary a little bit if you're, let's say, working for brand, then you also have a lot of huge component of media buying and creatives working with partners in a different way. But by and large, you can see how this spans out from the base layer data to the advanced level of predictive analysis. Cool. So keeping that in mind, I'll try to touch upon three craft areas and see potentially try to draw a comparative analysis, right, between domain. So yeah, so the first thing, like I guess the starting point for any product manager when you move into a different industry is to develop an understanding of product discovery, which is to understand who your user is, how do you gather feedback, how do you gather new insights, and what is the process of iteration, right? Even if it's not there, you have to develop some framework from scratch. So see if in a typical B2C framework, you start out with a business problem right in a particular industry. So let's say you want to solve problem in travel industry. So you go out into your market research, you try to understand the feasibility of that problem, size of that market, the competition, and so on. And then you take a step further and understand different user demographics, different user types, and probably zero down on one user segment. So let's say you want to focus on solo travelers, right? And that's where your product building stage comes in. Once you understand the pain points of solo travelers, you build an MVP, you define the success metrics, you develop some sort of feedback loop, so you understand how those users are using your product. And then finally, you develop a framework to iterate on it. If you want to scale it or tone it down or take a different direction. So that's a standard lifecycle for product discovery. Let's see how that changes, right, for tech. So in terms of market research, there is very limited scope of direct market surgery, right? Because you are primarily dependent on the ad platforms you're working with. So at an advertiser, you need a platform like Google or a Microsoft or a Facebook to run your ads. So the research or the road mapping that you do comes from those partners, right, on what those partners are building, what are the capabilities that they're offering. So it becomes really crucial or as a stepping stone for you to build those relationships with that partners, right? In a similar way, you might be engaging with some research agencies to do the research. You can probably transfer some of those skills, but in a way, this is about engaging as a product manager, taking the lead and influencing the planning of your advertising platforms. User segment analysis, again, in marketing universe, there might not be a direct user level targeting always, right? Again, it's a privacy centric universe. So a lot of the analysis you do is at a data level, anonymize data level, not always at a user level. You understand the user from the perspective of your product. Let's say if I'm doing marketing for travel, I do understand the broad landscape of who my travel, who the traveler is, but may not always be from the marketing perspective, because not all channels are content where user is telling you what exactly their needs are, right? So there's a lot of understanding that you need to develop around the user, which becomes a challenge itself, right? So how do you leverage that, right? So rather than doing user segment analysis, you do capability analysis. So which is to say, how do I run my campaigns more efficiently to get the outcomes I need, right? So let's say if you're working with Google and you're working on text ads, what are the new capabilities that they're building that I can test and maybe try it on my customers or my potential customers? So let's say, can I extend my ads to add more information, right? Can I try and experiment with different budget strategies, right? Can I adopt different strategies for different geographies? So your discovery process should involve understanding of those market capabilities, and that's how you take a next step around building the product. So in terms of MVP, yes, again, the difference here is that in a B2C, you have a product workflow, right?  So let's say e-commerce product, your user is coming to book a product or like buy a product online. So you have a life cycle of that product, but in marketing, you don't necessarily have that, right? So your roadmap or your backlog will have certain projects that you carve out based on the capabilities that you want to try, right? So that's again a shift in mindset, rather than working on the same workflow and iterating on it, you are working on different projects. The thing to keep in mind is that does not necessarily translate you into a project manager. It's more about building a backlog of initiatives that eventually make up your product stack. Defining success criteria should remain the same, always the same, because you do not want to drop your product mindset, even if you're running projects, you have to have specific success criteria, specific metrics that you can measure that against. You still need to develop the feedback loops. So how do you measure those projects? How do you measure the user interaction around those products, let's say ad campaigns? And finally, iteration, the only caveat here is that you iterate with partners. Similarly, the way your research is revolved around the partners. If you test out a new capability and you want to make any changes, you have to influence the partner rather than doing directly, because again, you are dependent heavily on the platform you work on. So hopefully that gives you some sense of how the product discovery could change a little bit when you're working in marketing. Yeah, I would also like to touch upon a little bit on product strategy. So here I'm just have a few high-level insights rather than a framework. So what usually I have tried based on my understanding of on the tech stack is that the first thing first is you have to hold the foot, which is you have a default or the base set of product stack, which your users are using to generate these campaigns at scale. So your goal should be to maintain and evolve that. Other than trying a lot of new iterations, your first goal should be to make sure this is running very well because at the end of the day, you need to empower the marketers who are actually domain experts. So somebody could be an expert in a particular language or a particular type of content. So there has to be a constant effort involving that stack. And that's where you really leverage your engineering team and your skills around team management, your skill around development processes can really shine and give you an edge here. Second thing, I think I briefly touched upon in the last slide as well, that you need to develop some skill around road mapping without direct user input because not all marketing is direct content. So there's a lot of push because you're trying to capture the market, you're trying to achieve certain business goals. So you try to roadmap by leveraging your ad partners, you try to roadmap by leveraging other industry experts. So basically your roadmap process could not be entirely internal. It has to evolve into influencing people outside your company. And lastly, since your backlog is not purely product workflows, but a set of projects that you identified, you need to be able to focus on experiment velocity, because not everything might work. So if Google launches a new capabilities, it might not work for your customers. So in order to bypass that or in order to do that more successfully, you need to build a process around experiment velocity. What is the framework that your team has adopted, but you to be able to test on these things faster? What is your turnaround time if you want to test a new ad feature? So that's where the product manager skills come in. This will not purely be done if you're just letting, let's say the marketers or the techs running the show in terms of the course stack. So you have to focus heavily on driving experiment velocity and also you have to be heavily driven by metrics. Only then you can make the right suggestions to your partners. And lastly, people, I think this is probably one of the most critical aspects that I've realized in terms of the mindset shift. So as a B2C customer, the idea of a product owner really resonated with me, where once I take ownership of a product, I take ownership of the workflow or the product lifecycle end to end, which is right from understanding the customers, right from influencing the behavior, right from measuring the data. But that might not be the case when you're operating a marketing channel because you are working with, again, marketing experts who are domain experts, specialists, business leaders who understand the channel or the finance leaders who are determining some of the goals. So you have to understand your spirit of influence, right? What are the aspects that you can directly influence, right? And what are the aspects that you can't directly influence? And what are the aspects where you need to leverage other teams or empower other teams, right? So in the slide where we mentioned what products tag, you need to do that exercise for yourself or whatever channel you operate in, right? Whether you have full ownership of the data, can you actually make upgrades on the way you store data? Can you make changes in the way you do productive analysis or the content, right? So really critical to understand your spirit of influence, you cannot assume that you will own the entire lifecycle of the user or the channel or the business activity being done. Second, once you understand that, you need to own that. You need to own what you influence, right? So you need to own data into it and you need to become an evangelist for that or the other stakeholders internally you're working with. So this is one of the key learnings, I realize that not everyone would probably understand or have a high level view on the tech stack and the product stack. People might be operating into their specific spears of influence themselves in which they are operating let's say for content, maybe for images, maybe for a geography and so on. So you need to own the product stack that you have for all those users for all those activities and wherever you can't own, you need to influence, right? So let's say you're working with a marketing expert who has a who's a domain expert on a particular type of creators, right? You need to be able to influence the strategy that they adopt of the scale content, right? For example, in product stack, be leveraged to scale that or help them out. Again, if you translate this from a B2C to a mark tech perspective, as a marketer, you are only owning the incoming traffic, right? You're only owning the inbound traffic. Once and your goal is to make sure that your user sees an ad wherever it's relevant and you get the click at a positive return, but you can't really influence what happens to that user afterwards, right? So once user clicks on an ad, lands on our website, you are dependent on the other product teams to take that journey forward, right? To show them the right experience to make sure that they convert, right? So you don't always have a direct ownership on the rest of the product funnel and that's where you need to influence. And lastly, manager, like I said, being a product manager in a mark tech would be misleading for several people, especially sometimes even in the leadership level, right? So for example, you are by default subsumed under the tech team and you might be, your focus could might very easily devote into being a technology leader or being a data manager and so on. So you really need to make sure you identify your spirit of influence and you basically convince the leadership in order to make sure that you are adding the value at the right places, right? So because you will still need the tech team to maintain and grow that system, but you also need product managers or the business leaders to understand new opportunities for the same business outcome. So managing up is really critical and could be a, especially when you transition fresh into the domain, it's something that you should be careful about making sure that everyone is aware of where you can add value because you do not want, it can, there might be a case where you, over time, you are translating yourself from being a product manager into something else into some sort of marketing manager or some sort of technical manager, right? So you have to be careful about that. Yeah. So I feel these are some of the insights that I had to share with you guys today. Hope it was insightful, hope you learned something out of it. And I hope to see you guys next time at Prague School. Thank you.