int we were trying to highlight in the letter that we sent. Finally, I think there is a lot of work we can do with the RAN data. We have some internal expertise that I think together with the board staff we can get to a better place of understanding how we may be different, how we may not be different, but I guess I'll just close in saying that no benchmark data is perfect. We're not upset with the benchmarks that are being used. I think with more time we can get to a better place, and that was the whole point of the letter that we sent to you was just to kind of highlight and clarify where a benchmark may not be perfect and we still have some more work to do. Thank you for those thoughts, Mr. Vincent. I agree benchmarking tools really helped us, and your letter helped us think about your perspective on those data points, and so that was valuable for us in evaluating it also. Thanks for putting the time in to do it. I'll turn to the health care advocate for any questions they may have. Thank you, Chair Foster. We'll do public comment after just for I know folks are waiting, but sorry to interrupt, Mikey. I have a few questions, and Sam, vice for my team will have a few questions. So first off, thank you. Been a long day already and promises to be even longer. And also thank you. I should pause for Mittens and express a thank you through you to all of the vast number of providers under you who are providing care. I think I know others feel the same. We have challenging questions at times, but that doesn't mean that we don't appreciate the work that is represented here. So first off, I have a question about your Part C partnership with MVP. And as I launch into it, I'm sure I trust you will alert me if I stray into questions that should be answered in a confidential setting. In answer to a question that was asked by the board about the UVM MVP partnership, you answered that UVM does not currently have an ownership interest in the MVP Part C plan. And so I guess I wanted to ask maybe maybe in follow up to that. So are you saying that if this plan does well and is profitable that that will have no impact on UVM's bottom line? That's correct. Right now we have a co-branded product in the market, but MVP is solely responsible for the performance of that plan. We don't have any connection to the financial performance of the UVM health network advantage plan. Okay. I was going to maybe I'll ask it anyway. I had a sort of a version of the case mix question for Part C versus quite a lot of questions already about case mix. But I guess I wondered whether you have an analysis of your case mix for even though you don't have an ownership interest for the Part C plan in comparison to other Medicare? Sorry. No, we don't have that comparison. Okay. While I'm talking about Medicare and Medicare costs, I do want to just pause for a second. This is a comment more than anything. Whenever I hear people talk about Vermont being a low cost Medicare state and referencing the Medicare benchmark data, I believe we're talking here about traditional Medicare. And I just think it's really important for us all to remember that at least up until recently, Vermont has had a low uptick in Part C. And I think there's been a lot of analysis that Part C plans tend to take lower acuity, lower cost people out of the marketplace, out of the rest of the traditional group. So states with a lower uptick of Part C often will look like they cost less. Been some a lot of analysis on that and I could certainly point you to it. There was one, it actually done in state as part of the Act 99 report recently. So follow up on member Merman's question about the medical school. In addition to the costs that he explored in his questioning, I also see, I think it's an addition if you could clarify, I also see in your audited financials what appears to be a transfer to the medical school annually of a range of $20 to $30 million. Is that in addition to the dynamics that member Merman explored? No, the transfers that we, that they're going to UVM are to pay for specific costs. So some of that cost is the cost of physician salaries that a portion of their salary is at UVM. But for that time, they're actually providing clinical care. So UVM is reimbursed for that piece of the salary. Some of that is benefits with our physicians. They essentially have a choice where they get their benefits, the ones that are duly employed by UVM and the UVM medical group. They can choose to get their benefits through UVM or they can choose to get their benefits through the UVM medical group. So that cost is part of that transfer. Plus we do have some space that we occupy at UVM. So the transfers are all connected to an actual operating expense that we should be paying for as an academic medical center. Okay. I will leave that topic with it with, you know, hey, by the way, no one's questioning the importance of the medical school here. I think the question really should be, what's the right way to fund it? And then so I think that it would be an interesting question to evaluate whether other states have found other ways to fund it other than through rate payer dollars. That's the place where I find the need to push back as to whether the rate payers is the right way to fund that part of the medical school. In follow up to Mr. Vincent's earlier comments about the bond rating, about your rating, I wanted to ask you a little bit about the obligated group. It was curious for me to understand that the entity that is rated here is not the whole UVM health network. It is less, maybe Porter and Alice Hyde and a couple other small entities. Can you explain why that is? That's just based on history. And actually on October 1st of this year, those groups will also, those two hospitals will also be in the obligated group. But to be clear, the rating agencies have always looked at us as a health network. The obligated group to them doesn't doesn't matter when they're looking at financial performance that are looking at car and they have been looking at our entire health network. Okay, that's interesting to me. I thought I was understanding that that the rating was only on the entities in the obligated group. You're saying it's everyone. So I want to note that it's similar to my question about the medical school. I want to focus on the cost of the health systems across the lake. And similarly, I'm not questioning whether those hospitals should be supported and really meaning to ask, is it Vermont rate payer dollars that should be supporting them? I recognize from their audited financials that Alice Hyde was down 13 million in 2022. Champlain Valley Physicians was down 54 million dollars over the last 22 being a particularly bad performance more over there. What's your biggest plan of how to manage those costs? And I don't know, maybe you could just respond. Is that those people need care just like they're people just like Vermonters, of course, but is it should it be Vermont rate payers that are footing that bill? To be clear, Vermont rate payers are not footing that bill. So in our budget submissions, I think we've clearly laid out that the rate increases, both from Medicare and Medicaid and commercial are driven purely by the cost inflation at Porter UBM Medical Center and CBMC. So that is the direct connection to the Vermont payers. The same avenues that we have here in Vermont are in place in New York. It's not the same regulatory body, but we deal with the New York payers there for the needs of the New York hospitals. We work with state government there. Obviously, it's more difficult in New York than it is in Vermont because it's a much larger state and the North country is a very small portion of the overall state. But to be clear, the Vermont rate payers do not subsidize New York care. So I may be a little confused and I don't have the numbers in front of me, but I thought I had seen transfers over to New York. You're telling me that there's no net loss to the network in terms of supporting those hospitals. There's a net loss when you look at the overall performance of the network, and we have provided the Green Mountain Care Board over the years. The net funds flows, both the revenues that come into Vermont, the provider tax revenue that's generated by New York patients to come coming to Vermont, the charges that we charge New York hospitals for their share of shared services. But yes, that is correct. And maybe just one more on the same line. And the having those entities in your operating group, obligated group, does that have a negative impact on your bond rating? Certainly. You see the financial results and the struggles there in New York are quite great. The hospitals, particularly in the North country of New York, are all struggling. So we're working hard to improve performance. Some of the service conversations, obviously, you may have followed. We've had to have those difficult conversations and made those difficult decisions. So we had to close OB services in Malone as a way to consolidate services and try to improve financial performance. And so we're focused on improving that performance. But they do, as you see in our financial results, they do have a negative impact on the overall UVM health network finances. I think my last question for today is sort of in follow-up to member lunches, maybe it was member Holmes's question about the sale of investments. Actually, the combination in your other financials, you show a profit and loss statement, I think you show what was budgeted as a $16 or $17 million sale of investments and you're projecting a $61 million income.  So two places where, at least on that page, it looks like 23 is looking good for you. Can you reconcile those numbers for me with, you know, the story you're telling about how tough 23 has been? Yeah. So when you look at those, that revenue piece and looking at our total margin, those are all market returns. And as we've all seen, the market has done well this year. That is realized income. That doesn't mean that we actually have sold investments. It means that the value of those investments have gone up. Some of them have been sold and has generated some realized income. But that's what you see there, 95, you know, 90 to 95% of that income that you see in that number is all market returns. Is all unrealized because you haven't sold it. Is that what you're saying? It's a piece of both. Some of it is realized and some of it is unrealized. And it would be a strategic decision for UVM as to whether to realize those unrealized. And it's a funny sense, but you could sell them, right? You could. Absolutely. And when you, but we don't, we had to last year with the, and even some a little bit earlier this year, that's where you see for any of the unrestricted both cash and investments that we have. That's what has contributed to the days cash and on hand decline that you've seen. So we've had, we did have to sell some investments just to fund routine operations because the routine operations were not generating enough cash to pay the bills. So you don't want to be selling investments. You want to keep them in the market so that you can use them at a later date for capital projects for unexpected events. So you never want to be in a position of having to sell your event investments because you're essentially, it means you're not generating enough revenue in your core operation to fund all the expenses that you have. And the other half of my question, I know I asked both about the sale of investments and the NPR numbers. Did you have a comment about the projected 15.5 percent when you had a budgeted 10, 10.7? So you were looking at the total margin? The net patient revenue line, yeah. Oh, okay. The net patient revenue line, 15 percent versus a budget of 10 is what you're, yeah, that is, is, as we've shared, I think throughout the, throughout the day, a big piece of that is all our access improvement efforts. So we've worked through a significant amount of backlog in our cases. We've improved our wait times for radiology. We've improved our wait times for clinic appointments. So this is, again, the balance we're trying to meet all the access needs. And that certainly has generated more NPR than what, than what we budgeted. Okay. Thank you.  Thank you. Can folks hear me okay? Great. So thank you for the opportunity. And if any of the, as Mike said, finding these questions move into territory that'd be better addressed in the executive session, please let me know. Just looking through the last four or five years of your audited financials, I'm wondering if you could describe a little bit more detail some of the purposes of the subsidiaries that exists underneath the parent, particularly UVM Health Ventures and the Medical Center Foundation. Just wondering if you could describe what the purpose of those entities are. So the UVM Health Ventures Corporation is, there are times when we make investments in private equity companies. So part of our investment strategy to be well diversified is we invest in publicly traded investments, bonds, and privately held companies. And for some of those investments, due to tax, tax reasons, we do run some of those investments through the Vermont Health Ventures, very, very small amount of investments that we ultimately have to run through there. And in terms of the foundation, sorry, that kind of predates me. So I might need to call on one of my colleagues here to explain what the purpose of that. Steve. So the foundation is basically our arm for philanthropy. So we have the UVM Medical Center Foundation that's philanthropy. It's a separate board that raises money for the medical center. Thank you. It does not just to be clear, it does not have a balance sheet of its own. There are no assets that sit there. Thank you. I appreciate that. Are you able to quantify the amount? I used to do a small, Mr. Vincent, the amount held in because I believe the network is an equity holder in the Health Ventures. Are you able to quantify that amount? I will follow up with you on the exact amount. I'm shifting over a little bit following up on some of the board questions on the Population Health Services Organization. I'm wondering, and it's possible that you mentioned this and I missed it, how you plan to evaluate progress towards the outcomes and costs reductions that it seems to be the primary purpose of the organization? How will you evaluate that over time? Do you want me to go ahead, Rick? Please do. Yeah, so approach to evaluation for the PHSO is really in terms of initially implementation metrics of things like hiring, care managers, assessments, care plans, then moving into process-based metrics. So looking at utilization year over year for populations. So ED utilization, for example, or inpatient utilization, admits per thousand for those types of, and then moving into outcomes over time. And so those outcomes will be cost-driven metrics, and they'll be benchmarked of course. And we'll use literature to drive some of that. And also our partnership with Health Services Researchers, which is one of the things that we're looking to build in the PHSO to have robust evaluations for all of the programs that we built across the PHSO. Thanks. I appreciate that. Do you have a time horizon, short-term, long-term goals for when some of these metrics we can expect to see or hope to see improvement? So typically, if you want to talk about a particular program and we want to talk about the care manager program, which is, I would say, the one that is really most mature at this point, although still really growing. Typically, literature tells us that you need 18 to 24 months of longitudinal care management before you do see change and true outcomes. However, you can see short-term gains pretty quickly. And so some of that we are realizing and seeing now. A lot of the reporting we're actually starting to think through. So I would say that it's reasonable to think that within the next year, we will have initial reporting to be able to provide from the PHSO. And then that will obviously grow over time. Thank you. I appreciate that. Shifting gears a little bit. A couple years ago, the CEO of the Mayo Clinic made a controversial comment that they at the Mayo Clinic planned to prioritize commercial patients over patients that have public insurance and then walked it back. And we've heard a lot of discussion today about the cost shift and the impact on commercial rate payers and the importance of commercial revenue. I'm wondering if UVM Health Network or any of the entities within it, is there any prioritization for non-emergency inpatient or outpatient services for appointments for patients? Do you prioritize commercial patients in any way? Absolutely not. And I think we've got, we have some physicians here on the screen that can also add in from a provider perspective, from a clinician's perspective, we're insurance blind. We take care of everybody that comes to our door. And we essentially, obviously there are front-end things that we have to do in terms of prior authorizations and other things to get patients financially cleared. But we don't restrict access to care based on insurance in any way. Steve, I don't know if you want to add anything. Couldn't have said it better, Rick. We absolutely are pay or blind to all the care we deliver. Okay. Thank you. Last question, focus on race equity particularly. As I know, that's been an area of investment in particularly in your DEI department. I'm wondering what metrics you've specifically identified to hopefully make progress on in the coming years and how you plan to evaluate progress in that. So for that, I'll turn to probably wondering which one I'm going to turn to. So I'll start with Sunny first. Sure. So it's a great question. We have a number of metrics that we're looking at. But I'll just start with the really basics around education. So we have this year come up with a unified education plan for our entire organization. So all 15,000 plus employees plus our board members with a goal of getting 80% of our staff educated on just simply the basics of what it means to be diverse, how to have those conversations with each other so that people feel safe and can bring the sense of belonging to the organization. And then for all 100% of all new employees going through that training. The second is how do we actually increase diversity in every way that we look at it across our organization. So whether that's race or ethnicity and non-English speakers or sexual orientation, gender identity, ability, poverty. How do we bring a more diverse group of people into our organization? What we realized is that our metrics today are not great in the way that we've captured them. By that I mean probably somewhere between 10 and 15% of all of our staff, if we look across all of our institutions, have left that question blank around race and ethnicity as a simple one. But it's even higher that is that they left it blank if we ask about sexual orientation, gender identity. And so before we can identify growth in those areas, we need to start by making sure that we have an adequate capture of the data. So starting in September, we're starting a new campaign across all of our entities just to simply create safety so that people can adequately and and safely feel like they can report on their own identities. And so that's the two big pieces across the network. And then when we look about health care equity, that is, is the quality of care that we deliver across our network equitable. We know it's not today. Meaning in the metrics that we're looking at, so we're looking at a number of them, and David probably has a much bigger idea of this, but I'll give you the ones that that I'm focusing on, breast cancer screening, colon cancer screening, diabetes treatment. So hemoglobin A1C as a measure of diabetes and hypertensive treatment. We know that if you're black, if you live in a poor area, your every one of those measures is lower than if you're white or live in a zip code that has a higher level of wealth. And so we started in this year, FY23, to go forward on trying to identify what do we need to do to improve those and focusing on those particular metrics to try to move forward. So it just gives you an idea. There's a lot of work going on and a lot of work on across our organization just to bring those to light and to show the particular areas. So if you're practicing in a particular zip code, we want you to know what your numbers are and what you need to do. So that's the work that's going on this year. It ties into the work that Jessica was mentioning around our population health services work that we need to go out and really drive this out, not just keep it inside, but first starts with just being able to make sure that we have adequate data on that. So thanks for asking. Thank you. Appreciate it. Thank you, Chair Foster. Thanks. A quick follow-up on the health ventures. Is there any University of Vermont Medical Center health network money that goes into fund the investments that makes? Yes. And are there policies prohibiting health ventures from investing in companies that UVM health network executives have a stake in? I'll look to Eric. I don't know if you're able to answer that. There are no investments by UVM health venture health network ventures that are in companies that to my knowledge, Chair Foster, are owned by UVM health network executives. Part of the reason of having health ventures is to support research that may be commercializable by our physicians who are also employed by the university. And so in that sense, there may be investments in, for instance, a new invention that's being worked on and partially owned by a professor physician, but not on behalf of leaders at the UVM health network. Yeah. What I'm getting is just making sure there's policies where the money that's being funded into health network, sorry, from health network isn't being used to invest in products that executives have, obviously.  No, it's not the case. And as you know, as a 501c3 organization, we also have very, very strict conflict of interest policies and controls in place that prohibit us from engaging in interested party transactions. And of course, we abide by those scrupulously. Of course, right. I'm going to just draw your attention to a video interview with ZDoggMD and Marty. I forget his last name, but it interviewed Chris Jones from health ventures. It's from January 15, 2020. And there is some discussion of the JPMorgan conference and what hotel rooms cost and how much it costs to get a cup of coffee and things like that. That segment's in the first early five minutes or so. And I just wanted to flag it. I don't know what they were really saying, but it sounded expensive. And I just want to make sure that that kind of expense. I didn't say they were spending that money, but it's worth looking at. Please take a look at it and we don't want to have that kind of expensive hotel rooms and the like. And I don't know if that was actually going on in that video or not. It was really clear. I will turn it back to you, folks, for any closing statement if you have any. And then there's some hands raised and we'll turn to that after. And thank you. Chair, I think we've been doing the public comment before the closing statements. So I think that would be the appropriate. Oh, great. Yeah, great. Thank you. Yeah, even better. And Ms. Brown, thank you for waiting. Your hand was up first. Hi. Can you hear me? Yes, we can. Okay. Thank you. My name is Betsy Brown. I am a lead at the UVMMC Provider Access Services Call Center. Thank you for this opportunity to speak. And before I start my comment, I'd like to say that I've been on this Zoom since eight o'clock this morning. I have been patiently waiting for hours listening to all of you. And I presume that all of those other people from the public who are here to speak today will get the opportunity to say what they have to say. I have worked at the past call center for more than 14 years. I am a member of the UVMMC Support Staff United, the newest and biggest union at UVMMC. There are over 2,300 of us in over 140 different titles. It is a diverse group at the bottom of the pay scale at UVMMC. We are the backbone of the hospital. We clean the rooms, cook and serve the food, draw patients blood, help take images, help dispense medications, register patients, transport patients, bill patients, distribute supplies and equipment, schedule appointments, answer phones, page providers and staff, call the codes, care for patients, many of whom are challenging and sometimes violent. I could go on and on. Without us, the hospital simply cannot operate. Providers, managers, directors, vice presidents and those at the very top are paid anywhere from 3 to 32 times more than the vast majority of our union members, comparing 31,000 dollars for a full-time union member versus over a million dollars for someone at the top of the hospital. We all pay the same premium for health insurance regardless of our pay scale. The members of UVMMC Support Staff United should be paid a livable wage and health insurance premiums should be on a sliding scale based on a pay scale. If UVMMC is allowed to increase their fees, they should be directed to first use their money to support their most important resource, the staff who work for and with them. Why build new facilities when they are unable to hire and retain staff to operate the facilities they already have? If the staff is respected, paid a livable wage and have fair access to healthcare themselves, the hospital will be able to retain staff and serve the community safely. Thank you. Thank you for your comment and your patience, Ms. Brown. And we will take every public comment for as long as we need to. I appreciate everyone waiting. Let me see who's next. Ms. Gutlin, please go ahead. So I just need to turn yours up. Okay. Thanks. I'm Sharon Gutlin. I'm the owner of a physical therapy practice called the rehab gym. I'm also a member of the Green Mountain Care Board advisory committee. I appreciate UVM Health Network. I have gone there for care and will always be grateful. I am my comments are not as much for the direct hospital care, but more of care that can be delivered in other non-hospital-based locations, more in communities. I think UVM Health Network provided a formidable presentation. But what I felt is lacking is any awareness or appreciation that they are not the only business in the Vermont healthcare system. They're a large part, the largest part. But the best system requires collaboration, not autocracy. And I feel that the sense of autocracy or being in a bubble, and I understand. I mean, everybody representing UVM Health Network is here to only advocate for UVM Health Network. I understand that. But it shouldn't operate in the bubble. And if it operates in the bubble, that's the core of what I think is negative that stimulates the growth that is becoming more of a monopoly instead of a collaboration with the other healthcare providers within this Vermont healthcare system. And it's a small state. We all know each other. We should be able to work in collaboration and cooperation and not be adversaries. The reduction actually in healthcare expense is less about being more efficient or reducing the delivery of care, but reduction of healthcare in general. We need less hospital care, because that is the most expensive care. Of course, it's most expensive for all the reasons laid out. But we can avoid having as many Vermonters going into this expensive care that is driving up the insurance premiums. So as long as focus is on financing a hospital-centric healthcare model, healthcare expenses can only be expected to rise. And at a higher rate than the community-centric care model. And I trust that everyone that is in this discussion, over 100 are listening to this, that we really, really, really, really all want to be healthy Vermonters having a happy life. That's fundamentally why we're all here. I want that. I believe people in the hospital want that. I don't compare my business, the rehab gym, to a hospital because it's not apples to apples. But I can compare my business to the outpatient PT model that is owned by the hospital. And I can speak about that, and I want it to be on public record. We face the identical inflation rates, hiring challenges, the need to upgrade, the repair, the pay of bills. I even have an elevator in one of my locations. I just spent $9,000 to repair that and listened to a whole lot of complaints while it was under repairs. Our pair mix is identical. We are also pair blind and we have on comp care. Our budget shows actually a higher revenue percentage going to employee comp than 60%. The major difference is, as Owen alluded to, that the reimbursements are up to four times higher for same services in a hospital-based outpatient clinic. And we know this because of the CMS transparency ruling that both hospitals and commercial payers are not obligated to contribute to. So it's not yet user-friendly, but my daughter, who is not needing computers to be user-friendly, has dived in. So all of this, all of the CPT codes that we are under can be compared apples to apples. So when I say it's four times higher, there's evidence of that. And I provided it to the Greenmount Care Board. The rules are known. What did confuse me was it sounded like people that were representing the hospital didn't really know what the rest of the healthcare system was facing. So that's kind of why I'm speaking. I'd rather not. I'd rather be outside enjoying my day. But I think that that also speaks to the point that has already been mentioned, that it's not positive. It doesn't shine a positive light on the hospital. So we'll pretend it's not there and maybe it'll blow over. Well, my business has reached the critical point where the lead balloon has gotten heavier and it's falling. So after being at near flat rates for since 2015, we're at the breaking point. And what reality presents us is is after reducing our CEO, CFO and owner, and it is as low as we can go, I'm on 24,000 a year, I can't find any more money to squeeze. So this is where we're at. If we cannot get rates from the commercial payers, which have already testified, Tom Weigl from Blue Cross and Blue Shield said that because the hospital demands what the Green Mountain Care Board provides, they have had no choice but to have that come from everyone else. And I'm speaking for every, probably every other non hospital based business and possibly even hospitals in Vermont that compete with you. I don't know. But I do know this, I'm not alone. And I am having to make a very painful decision after 20 years in business and in the major provider of physical therapy in the state of Vermont, who is at the forefront of reform in preventative care, trying to keep people people out of physical therapy, out of hospitals, out of doctor's offices, and can provide plenty of evidence that that's effective in this medically oriented gym. I have no choice but to go out of network and get the higher out of network rates. This goes against my very reason for going into business. And if I go out of network, there is a potential for the cost shifting to be even more extreme. Speaking of cost shifting, I've heard nobody speak of the actual cost shifting of the private community based business towards the hospital to meet demands. The money has to come from somewhere, it's not the trees. And so if there's a limited amount of commercial money, we are not getting it if the hospital gets it. A couple more things. The margin and volume, they do matter. Costco shows how low margins and high volume do cover the bills. And the rehab gym has had the pleasure being the size it is to be actually strong because we have high volume. So the pennies we make on the dollar can go farther. And then last thing is hospitals and other healthcare systems. Oh, I'm keeping the finger in my role as an advisory committee to the Greenmount Care Board. I'm keeping my finger on the pulse of what is going on in the nation. Certainly what Vermont is facing is really not unique. It's nationwide. And there are systems and hospitals that are reducing the administrative roles along with salaries out of some out of the fact that they're forced to because they don't get all the money they want. But others strictly because they're being proactive. They see it's not sustainable. When I heard the term, what was it, the dartboard number? 24%. I was in my living room that came on the news and my jaw dropped. I mean, everyone in this call and on the zoom should like stop and think 24%. So I want us to stop thinking of individual bubbles. And I'm 66. I'm not going to be in this profession that much longer. I would love to see before I leave this profession, an actual paradigm shift where we're starting to focus on where we can actually reform healthcare. It's not from a hospital. You can't put the fox in charge of the henhouse. It's like Kentucky fried chicken working on people to stop eating chicken prevention health wellness fitness mental physical is best applied in communities in a variety of settings as universal as individual mankind presents. So I am hoping although it's a big ask that the hospital can actually focus on what it does best where everybody agrees the hospital needs to take care of the sick and injured and do a fabulous job. And you should get paid for what you're doing to save people's lives and create health when people are sick. Let me let other providers that have absolutely nothing to gain from a sick person and only everything to cane from a healthy person that's basically plugged back into an active life. That's physical therapy. That's social work. That's primary care. That's occupational therapy. Pediatrics that that that speech therapy there there there's there's there's those professionals along with the fitness industry industry health coaching. If we pay anywhere close to 24% even over the 8.6 that the board has originally tried to stick with that is money not being spent on the healthcare care. True health care health care of remontal of remontures and it's investing in a sick care system that can go so long before it implodes. Thank you. Thank you Miss Gutwin. Mr. Hoffman. Hi can you hear me? We can yes. Chair Foster I have if it's okay with you I have a technologically challenged guest who I'm helping. Would you allow him to go and then I'll follow him here on teams. If you unmute line<PHONE_NUMBER>. I have David Taffer and then I'll follow his call on teams. Absolutely that's no problem. Miss Lodge, do you know how to unmute Mr. Taft on that number? Not sure I do. Microsoft Teams does not allow us to unmute individuals. I'm sorry. Okay so you need me to unmute my our phones. Star six nine I believe. Hey Mr. Taft it's Owen Foster from the care board. Can hear you can hear you fine. How you doing? Okay I'm doing fine. The floor is yours if you'd like to make a public comment for us. Oh okay unfortunately my aspect that I'm concerned about is this all started for me back in 2019. I ended up with a medical situation where I was brought to the emergency room and they did some blood work on me and unfortunately my PSA level for prostate cancer was elevated in the emergency room. My doctor who I'm sorry but I can't remember his name at the time was I don't know if he was notified of it at the time when I got it done but unfortunately like I said it was in 2019. I ended up with a new doctor in 2022 who did some blood work on me and she diagnosed me with prostate cancer and I was given over to a cancer doctor, Lester Cole who determined that I did have the prostate cancer and when I asked him how long I've had it he said that apparently from what he could see in my paperwork I was showing numbers back in 2019. Unfortunately I didn't show any signs illness wise to tell me something was wrong but I just unfortunately from 2019 to 2022 I went all that time without knowing because no one took the initiation from the emergency room area to either notify my doctor or my doctor didn't notify me so I went two years with the cancer which could have probably been taken care of by surgery if it was caught in time and unfortunately by the time they noticed it it already spread so the surgery was not an option for me so I ended up having to go through numerous radiation treatments and my biggest concern is what I went through but I could have I mean right now I'm doing fine but the aspect of the whole thing is why wasn't my situation with cancer which is more of a priority of anything brought up to the point where someone should have notified me right off the bat. I know back then they had the ordeal with the COVID and I feel like even though you know they were trying to do their best my cancer situation got put on the back shelf and by the time I was realized what's going on it was too far and my procedures were more extreme than they should have been. Me and my wife had been through a very emotional time because of it and I've been having problems with health insurance to the point where one time I got it one time I don't because I didn't have Medicare at the time and I had Medicaid but they've been bouncing me back and forth on my habit I don't have it but the issue of it always how much money was spent out of my pocket and the hospitals on the procedure that could have been taken care of a lot sooner and like I said being my wife's been through this um financially and emotionally were unfortunately I didn't say it this way but I feel myself that on my behalf the hospital dropped the ball and notified me to let me know and now I gotta fall down on the front of it and make the best of what I got. Mr. Taft um yeah I wasn't sure if you were finished I didn't want to interrupt if you have more that you'd like to share about your experience um it just uh it was very frustrating and uh I just feel like I think the hospital could have been able to step up a little bit more on their prior torsion whether it's the emergency room or whatever with their communications with each doctor and each patient for the the seriousness of their illness um if I only had uh you know uh an infection in my throat okay yeah no problem you know that can sit back a little bit if need be but I think anytime somebody comes down with any form of cancer whether it's a minimal type of cancer or a major I think that should be a priority one into someone's health to get taken care of more immediately and not going on to the back shelf like mine did until a new doctor decided to do blood work and that's how they found it so I went two years on the back shelf with cancer not even knowing I had it were prioritized they shouldn't know about it right off the bat and I just feel myself that somewhere in the hospital someone dropped the ball has anyone reached out to you about financial assistance issues last few weeks um they did but um they were telling me that uh because uh supposedly financial situations um the Medicaid that I did have they took from me um they said I didn't qualify for it and yeah I'm on Medicare right now but um even though it only covers so much um with me being on disability now because of my COPD that happened in 2019 I'm on a fixed budget um my wife has insurance but she can only afford insurance for herself and not for the two of us because it's too expensive and even with the insurance that I have I'm taking more money out of my pocket to take care of the medications I need um and even though I got my insurance it's still not enough to I don't tell it for me financially I'm still struggling trying to get my medications because my insurance won't kick in until I meet this requirement every month and by the time I do it starts all over again because it goes monthly so in all long run all my medications coming out of my pocket so it's like I'm paying all this money for health insurance I'm paying out money for my medications and it's like why am I spending on this money for on insurance and things if my insurance ain't gonna cover anything and it's gotten to the point where I'm almost to the point of okay if I get sick can I go to the hospital I really don't know because that's the money that I can't really afford to pay um my medication can I afford that well um if I'm in good needs with the good ward upstairs okay I might be able to get by without taking it but do I have that choice um it's pretty sad when you have to take and balance your health over a money situation and it's just that's the part I'm getting at where like the other woman was saying you guys want to raise your system by 24 percent I'm on a fixed budget I cannot afford to have my insurance my medication and everything go up just to help satisfy you guys um and I think I speak on behalf of the water list of who are on fixed incomes um it's to get to the point where unfortunately we're gonna have to start make a decision on um our medication and also being able to go in and get ourselves taken care of through our doctors because it can to that point where we can't afford to do it no more and where do we draw the line enough it's enough chair chair foster this is um Dr. Leffler if I can get Mr. Taft's name and phone number I can have patient advocacy calling first thing tomorrow to see if we can help um okay um I have talked to um the hospital they put me on a payment plan I was like okay um I received two bills from a collection agency because the hospital sent two of my bills to the collection agency because supposedly I was taken too long to pay on my medical bill I can only give you guys so much money and if it's taken so long that you guys have to put me in a collection agency and ruin my credibility I mean I'm sorry I mean I called you guys we made plans the do a payment plan and then you pull this on me it's like I might as well talk to a wall you guys are going to do what you want and you don't care about us you just want your money and that's how I feel I'm sorry um Mr. Taft I'll get Dr. Leffler's number to provide you or somebody that he can in case you want to reach out again um maybe it'll help and I acknowledge what you've been through is excruciating um you can keep speaking yeah yeah yeah okay David did you have anything more to share um I'm pretty much on what I need to say um I think I got my point across that you know I can only do so much and I can't do anymore and there's people out there like me who are in that same boat and all we're doing is asking for a little bit of consideration and a little bit of help from the hospital to make things easier on us that's all we are asking and uh oh I just have to sit back now after saying what I had to say and uh see what happens and to do the best I can and hopefully I don't have to file a bankruptcy situation because of paying all my medical bills and also uh taking the chance and having to jeopardize my health because of my medication and uh getting the help that I need so so that's we're on that right now Mr. Tapp thanks for taking the time to participate and share your experience because I think it's really important for all of us to it's sobering and it's important so thank you for speaking up and participating and um being here um Mr. Hoffman did you have a comment as well or yeah I have a comment some comments I'd like to make if that's okay of course thank you um I want to point out that uh Mr. Tapp's trying to diligently pay down this debt that really isn't his to oh as a result of a misdiagnosis and with all the investments in health information technology and care management and population health and on and on this should have been caught and so my hope is that uh instead of destroying his credit and possibly causing him to go bankrupt this hospital can help him not only retire the arrears but provide him the care he needs free of cost going forward but I want to make some other comments um the type of diagnosis and treatment David needed in 2019 is it's the fundamentals of medicine and that requires sufficient staffing since 2018 the network has coordinated the redirection of hundreds of millions of dollars that would have otherwise been available to direct patient care to the sexier work of population health IT execs paid at 15 times the median income of Vermont from honors who fund their salaries IT projects at three times the national average cost some of the most costly per square foot vanity building projects in the country and concurred to expanding its revenue and balance sheet Mr. or Dr. Epon's predecessor went on a more than decade long spending spree and acquisition endeavor hospitals presently reminds me the state of retail before I left it for health care massive condo consolidation transpired from the late 90s through the great financial recession enormous sums were spent on expensive but low margin acquisitions and handsome executive salaries to manage them and then the business model broke after the great financial recession when tremendous deflationary pressures particularly with the entrance of online retailing shareholders raised to justify every line item of expense returning conglomerates to the fundamentals and eventually those conglomerates either went bankrupt or were chopped up into what was valuable similarly hospitals post COVID faced tremendous inflationary pressures which have broken their business model their model is now broken and they have to respond Vermonters are the health network's shareholders and the Green Mountain Care Board is those Vermonters Board of Directors as with retail after the after GFR the Green Mountain Care Board the state of Vermont must examine every line of cost examining its return on investment the chair fosters repeatedly asked academics and hospital executives how to balance controlling costs and not reducing access to care which is routinely held out as a threat if the budgets aren't granted there are four main areas where nonprofit and I say that in quotes hospitals plow profits executive compensation vanity building projects excessive it spending and unproven population level endeavors with poor outcomes the four top executive seated before you today shared an aggregate of four million in C-suite compensation averaging to one million per executive or 33 times Vermonters median income paying their salaries many more were on this call today Dr. Eppin's predecessor went on unbridled spending spree on executive recruitment snatching up a former regulator at a cost of 800,000 annually another former regulator from Vermont AHS presumably at 15 times median Vermont wages a former U.S. attorney who's on this call today at 610,000 or 20 times Vermont median wages a former blue cross blue shield Vermont executive on this call today again presumably at 15 times median wages and the list goes on and on there are members of this green mountain care board regulating these very executives equally qualified to perform some of their roles and yet you work for many many multiples less as does the governor of the state of Vermont the health network even accounting for their proposed change in math which would have reduced their admin the clinical comp ratio from 2.1 still remains at the high whisker mark or more than 71 percent above the national median demonstrating as member Walsh said economies of scale have not been realized and I will renew I will numerate further momentarily how this is a top heavy organization reducing the network from 24 percent to the median of 14 percent in this category would yield some of the savings necessary for the hundreds of frontline staff who have written into all of you this week crying out for help it's the type of move shareholders would demand after leadership abandoned management fundamentals for a decade the health network proposes an outpatient surgical center that on a cost per square foot basis is the most expensive proposed build in the nation at three to five times the cost of similar builds no less than two hospitals and a nurses union have petitioned for interested party status to protest the ill-conceived nature of this proposed hundred and fifty million dollar investment to be made by vermont tax and premium payers such profligate spending results in facility fees and outside consumption of scarce health care dollars in this system this is exactly what miss gutland was talking about which every year risk to put folks like her who offer higher value lower cost community care out of business forever exacerbating the current well documented access to care crisis vermont faces and while the network may claim it's necessary to do these things to recruit top talent go down to ruttland and ask our state the state of vermont's highest paid physician melboyton if he stated ruttland region regional medical center all these years because of its lavish atrium the health network has spent hundreds of millions on it infrastructure since 2019 but has never told vermonters what ROI they've received for it in 2024 alone the network proposes to spend a hundred and fifty million on it that's three times the national average of three of three percent of operating expense for similar hospitals on this line alone a hundred million in 2024 exceeds the national average spend for the same top heavy management for this division includes an sb svp of network it is six hundred and fifty five thousand annually or 20 times vermont median income a network svp network vp of health informatics at 450 000 or 15 times vermont median income achieve medical informatics again 415 000 50 000 and 15 times median income a vp of enterprise info management and analytics presumably at similar comp the network has led the state's health reform efforts since 2018 hundreds of millions of scarce health care dollars have been poured into their aco and now the network proposes to spend another 23 million this year alone on a duplicative population health services organization and only provides very high level aspirational language around its aims and accomplishments over the past 24 21 months this is eerily reminiscent of the same language its aco used for years before former a hs commissioner mike smith said it needed to move on from being aspirational to being operationalized top heavy management in this sector includes two million spent on aco executives 590 000 on an svp of high value care 510 000 on svp in cheap cheap population health and quality an assistant general council of population health whatever that is i've not seen that at other hospitals but apparently it's necessary for vermonters to pay for as well as a vp of managed care contracting who was on the call today i cautioned their aco in 2018 it could not achieve its aims with the analytics it possessed they were not actionable or reliable despite continuing to represent otherwise to this board diva the legislature and the public otherwise for that i was terminated by my supervisor that supervisor is now the vp over analytics i just mentioned a minute a moment ago for the health network four years later blue cross blue shield vermont would cite lack of actionable data and consequent interventions as part of its reason for withdrawing and even the nork report assesses that providers find the analytics similar insufficient or irrelevant since 2018 the network's leadership of health care reform has seen its three most important clinical quality measures hypertension depression screening and substance use disorder screening underperform as hypertension scores bounce between the 60th and 70th percentiles and depression and substance use disorder weren't even benchmarked and rates of uptake and screening showed little improvement in them from 2018 to 2021 what cdc data tragically shows us is that hypertension suicide and substance use disorder related deaths grew far in excess of national average for age adjusted mortality and likely resulted in over 500 additional age adjusted deaths beyond national averages over the same period this isn't the high baseline that vermont already has over the national average for those same measures this is just the growth of growth from 2018 to 2022 2021 4000 vermonters died in excess of the national average at baseline but just the growth of growth yielded 500 additional age adjusted deaths on these important quality measures that hundreds of millions report into time to get back to the fundamentals of medicine every line item matters the time for pie and sky is over the public would make the following requests accordingly first and foremost immediately clear the credit history of david taff after it was destroyed reimburse him for all he's paid to treat stage four cancer that metastasized because of a missed prostate cancer diagnosis in 2019 immediately pay for him to receive a second opinion on his current treatment course and reassure him that the prostate that wasn't surgically intervened in addition to the cancer found in his ribs will remain in remission with the health network's chosen course of treatment provide him free care for all remaining course of treatment deny the current certificate of need for the surgical outpatient center until and if its projected costs are brought in line with national cost per square foot averages it can be shown that the uvm health network will provide the better quality and cost of care than current care in the region or by those who like the green mountain surgical center have had their request denied to expand specialties in the region while offering higher value lower for lower cost care reduce the 2024 budget by an amount equal to the excess of three percent of operating expense for their it investments currently a hundred million dollars and demand an ROI accounting for all epic investments made to date finally we'd ask Dr. Ethan to perform a thorough accounting and evaluation of the executive leadership that has been rolled up into the stable over the last decade including their total compensation as compared to national median averages for the same thank you thank you normally comment on public comment because they're all really valuable and so I don't get a chance to do that I don't think it's appropriate but I will just thank you for pointing out the care board staff that is really talented and works really hard and I think you know they take state salaries and they do great work so thank you for pointing that out I appreciate that they've had a really hard couple weeks doing this work so thanks um miss snel please go ahead thank you um I did want to uh first I wasn't going to speak up at this meeting I was just here to observe um and then there is a few comments made that I just felt like I needed to respond to in particular Mr. Miller's um comments about executive compensation because I found that very disturbing that that kind of numbers and that kind of information is being withheld um I do hope that in executive session that when it is brought up that the possibility that the Green Mountain Care Board could sign an NDA to get that information we have had to do that in union negotiations in the past to get that information I do hope that whatever hospitals the UVMC and network administration is using to benchmark themselves against is the same hospitals that they are using to benchmark and market their own employees when we bargained back in 2018 um they kind of had the audacity to compare us to a 100 bed rural hospital in Maine when they were looking at our salaries so that has always stuck with me that that's how they look at their employees unfortunately um and I do want to thank Dr. Lefler for somewhat standing up for union negotiations and being an important part of our work at the academic medical center that we do bring value and we do bring employees there because they know that they have certain rights and rules that they may not have had at other hospitals my other comment and I think that Mr. Hoffman before me spoke much more eloquently than I would ever be able to is on the network this shared administrative services um I agree with him that the expenses the 407 million dollars the share total expenses is kind of outrageous and especially in my mind when I was looking at it and granted I'm looking at it in very broad terms but when you look at the managers there is one manager for every six employees average at that level and that is a little mind boggling to me that those employees would need that kind of oversight and if they truly do then something needs to change at that level um I will I will stop there um and I think the Green Mountain Care Board for all of their work and I think the UVM Medical Center for being in the hot seat for these what seven eight hours now and I do appreciate everyone's time and attention on this very important matter. Thank you Ms. Snell and I believe the care board scheduled a hearing with with you and the nurses soon so thanks for doing that um we didn't we didn't ask a lot of questions around the nursing shortage because I I know we have that scheduled um yeah um so on this yeah yeah it's a good call on you sorry yeah yeah you got you got your hand went down so Ms. Sirle Schrader go go ahead hi um thank you for the time for me to speak today um my name is Becky Sirle Schrader and I've worked for UVMMC for over 11 years I first worked at what was originally when I started um PPS and then it became the Regional Transfer Center I left there in July of 2020 when I was laid off along with two of my co-workers because the massive epic project made our jobs absolutely obsolete. In January of 2021 I started at Provider Access the call center um I actually overnight worked a 12 hour shift from 7 p.m. to 7 a.m. and have been on this teams meeting off and on since 8 a.m. this morning I've heard a lot of numbers but what I want to talk about today right now is people is human beings not only have I worked for the hospital for 11 years over 11 years I've been a patient of this hospital for much longer than that I am here today to implore the board to hold UVMMC accountable for the care given to its most valuable asset and resource the human beings that make this hospital run we are the invisible workers that until we formed our union were easy to ignore we answer their phones we clean their rooms we draw labs for their patients and process them we page out their codes we cook their patients meals we fill their prescriptions and so much more no employee working at not only Vermont's largest employer but also its largest healthcare provider should have to worry about homelessness food insecurity or the inability to afford afford health insurance healthcare or medication due to being paid poverty wages right now the taxpayers of Vermont are funding UVMMC refusing to pay a livable wage in the form of food stamps housing subsidies and Medicaid and doctor taking the taxpayers money and robbing their hard working employees of their dignity my office recently started a food pantry in our break room because so many of our colleagues and coworkers are so food insecure they were coming to work working full shifts without eating because they were deciding between rent and healthcare and food I would also like to point out that there are multiple executives in this hospital who make more in one week than I make in one year another way that UVMMC has chosen to save money is by not having 24 hour security and multiple sites that have 24 hour staff at least two of these sites have been breached by people not associated with the hospital when I asked about adding security they locked the elevators instead and said well they can't move from floor to floor if we do that I asked for a keypad they said we'll see what we can do I demanded a keypad rather than buying a keypad they went and found an empty office and stole it off of that door we also received an email giving us tips on how to keep ourselves safe it's my job to be safety and security for myself I was there one of the nights that people broke into the building I was working in I don't like feeling unsafe at work that's not a way to save money as a patient the care that I have received from UVMMC over the years has been a mixed bag I have received wonderful care I have also received care that has been affected by work or retention rates that are terrible major medication errors caused by new staff major scheduling errors with surgeries and testing that nearly cost me going to my grandmother's funeral because nobody was communicating with each other and people didn't know what they were doing because they aren't being retained if you don't pay people a livable wage they won't stay and you constantly have a lack of knowledgeable skilled workers I just I know that UVMMC can do better than what they're doing right now they need to pay staff enough to get their loyalty and to get them to stay I know of people that are looking at jobs in retail in the food industry because they're paying more at those places than they're paying people who help page out and run the codes that save patients lives I just that that's not an okay way to run a health care industry I stay at my job because I believe in the work that I do I love helping patients when I help a patient it feels so good but I also deal with a lot of frustrated and angry patients that aren't getting the care they need and deserve and I try my best to help them and I don't want to have to leave this hospital I don't plan on leaving I join the union and I'm involved with the union because I think we can do better but that starts with making sure that the hospital is paying the lowest paid workers in their facility enough to live on so that they can stay and they don't have to leave thank you very much for the time to speak today thank you very much for your patience um I'll quickly apologize all the public commenters who waited I think I mismanaged this time a little bit um it's my first UVM hearing so I'll try and figure it out a little bit better for next year but thanks for sticking around thank you for your work thank you thank you for your work um mr davis thank you mr chairman I've got four or five comments but they're going to be really fast um I thought the discussion about what people from ours can afford to pay for health care didn't really go anywhere it might be of interest to you to know that in 2013 um the green then green mountain care board hired a consultant from Johns Hopkins to ask to ask that question directly and what he came up with was the idea that the that what brahmanas could afford to pay on a given year more than what they paid the previous year would be three percent now I'm pretty sure it's not they I think they put in an extra five percent just that because hospitals at that time were trying to go to single payer but in any event if you want to know what they can afford then that's a really good way to find it out and use something like the uh st louis fed to figure out the data like what is the actual what is the actual um uh growth uh growth state product anyway number two um the the whole discussion about about market power it strikes me it's it's significantly unrealistic for this reason the it's no there's no question that market power is being is rampaging across the whole united states and the entire in the entire um healthcare industry but not here you cannot uvm on anyone else including blue cross nobody can it can charge the public or the the payers any more money than the green mountain care board allows them to do they have total control over the commercial ask third the um the question of whether uvm is a financially valuable organization and really a financially efficient organization you can talk all you want about how how uh how the the Dartmouth Health Atlas just doesn't mean anything of course the Dartmouth Health Atlas means a lot okay but what I would suggest is okay put that aside for a minute I understand that it's important to some of the members to to lay down this um marker so that they can so that they can get an answer that they want but I would say this the rate if you if you if uvm is making too much money okay then what they're going to that money is going to really drop to um day's cash on hand okay and what and if you look at us if you look at um six years of day's cash on hand you'll find that there were 18 data points there's three rating agencies six years that's 18 data points and 17 of those data points saw the uvm uvm's financial situation okay was below any of the uh any of the rating agencies uh uh numbers that they would use to to uh charge uh you know uh interest rates for anything that the company does um the uh the uh the fourth is um one of the things that I've talked about here and that hasn't gotten anywhere but but this this all kinds of data in the in the in the one care archives that never never never appear one of them is the pqi readings okay that show that that uvm compared to all the other hospitals in vermont okay is so far ahead of them in quality that it's really just sort of in a way embarrassing uh so there's another not another piece in there that does not involve uvm or darkness which is a potential available avoidable utilization which shows that of the non-network network the small the 11 small hospitals that 20 to 30 20 or more than 30 percent of their admissions out of their er um are not justified that the uh none of that surprises me it's it's very clear the case that the board is trying to make to give the give uvm a huge haircut here and that that's that's the way it is that's life okay here's what surprises me that so that doesn't surprise me here's what does surprise me I can't see that the uvm leadership is really defending themselves if you can if somebody if somebody can call any of this a defense I mean I'm just amazed because uh because um the uvm network now the uh the uh the the academic medical center is actually at risk for survival I know really important people that know this thing backwards and frontwards who think that we're gonna that the uvm that the uvm mc is going to have to sell the uh sell the academic medical center to somebody who will play a different game thank you um thank you for your comment mr davis um i have an email address um hl bowman vt hey how are you hey i'm well my name is heather bowman thank you for this opportunity to speak to the green mountain care board um like betsy and Becky I am uh a member of uvm mc's newest union and the support staff united i'm a phlebotomist here at the hospital our union formed back in january of this year and we've been in negotiations with the administration since may you've heard from Becky and Betsy and you've no doubt seen the 118 comments submitted by staff from uvm mc they came from members of our bargaining unit and as you read through them as you listened to um to my colleagues i'm sure you got the gist of it as a group we are not being paid livable wages we struggle every month to make rent and to afford groceries and we cannot access the wonderful care offered here at the hospital because uh the insurance offered by the medical center the um the part our our contribution is too high for many of us to afford so most wednesdays for the past couple months before and after our regular shifts sometimes instead of spending time with our families or getting rest hundreds of us gather in person and on zoom at the bargaining table to do the work of bettering these conditions our working conditions we're proposing changes that would bring up our wages and make health care more affordable for the thousands of vermoners who are a member of our bargaining unit and um i really appreciate that dr lefther said um that we need to pay our people well we appreciate that and we look forward to being paid well um but so far the administration has been unwilling to support our efforts to make sure that the support staff is earning a livable wage and has access to affordable health care we understand that traditionally the board doesn't get into the weeds on how specifically the medical center spends its money but we implore you please please take a very close look we are asking you to help ensure that the state's largest medical center the region's level one trauma center and the largest private employer in the state does not balance its budget on the backs of its most marginalized and lowest pay staff there are more than 2000 people in the support staff union and we are the backbone of the university of vermont medical center for far too long we have been an afterthought i'm here today representing our union and who you've heard from in the submitted comments and who you've heard from earlier speaking i chose to speak up we choose to speak up because we are no longer willing to accept being an afterthought it's time that the people who power uvm mc are a priority we are the hospital's most valuable resource and the budget should reflect that and again i appreciate that dr eepin stated a commitment to providing care to everyone who walks through the door and i just really wish that that included the employee entrance so thank you for your time thank you for your public comment um i i see no more hands raised we'll take um just a quick two minute break and dr eepin do you have closing remarks you'd like to make today we have each of the hospital presidents who are going to make the closing remarks today okay all right we'll take a quick little two minute break and then we'll do that and shift gears thank you


In 2014, Amazon started working on its experimental ML-driven recruitment tool. Similar to the Amazon rating system, the hiring tool was supposed to give job applicants scores ranging from 1 to 5 stars when screening resumes for the best candidates. Yeah, the idea was great. But it seemed that the machine learning model only liked men. It penalized all resumes containing the word women's as in women's softball team captain. In 2018, Reuters broke the news that Amazon eventually had shut down the project. Now the million-dollar question. How come Amazon's machine learning model turned out to be sexist? A. AI goes rogue. B. Inexperienced data scientists. C. Faulty data set. D. Alexi gets jealous. The correct answer is C. Faulty data set. Not exclusively, of course. Data is one of the main factors determining whether ML projects will succeed or fail. In the case of Amazon, models were trained on 10 years worth of resumes submitted to the company, for the most part, by men. So here's another million-dollar question. How is data prepared for machine learning? All the magic begins with planning and formulating the problem that needs to be solved with the help of machine learning. Pretty much the same as with any other business decision. Then, you start constructing a training data set and stumble on the first rock. How much data is enough to train a good model? Just a couple samples? Thousands of them? Or even more? The thing is, there's no one-size-fits-all formula to help you calculate the right size of data set for a machine learning model. Here, many factors play their role from the problem you want to address to the learning algorithm you apply within the model. The simple rule of thumb is to collect as much data as possible because it's difficult to predict which and how many data samples will bring the most value. In simple words, there should be a lot of training data. Well, a lot sounds a bit too vague, right? Here are a couple of real-life examples for a better understanding. You know Gmail from Google, right? It's smart reply suggestions save time for users, generating short email responses right away. To make that happen, the Google team collected and pre-processed the training set that consisted of 238 million sample messages with and without responses. As far as Google Translate, it took trillions of examples for the whole project. But it doesn't mean you also need to strive for these huge numbers. I Chang Yeh, Tam King University professor, used the data set consisting of only 630 data samples. With them, he successfully trained the model of a neural network to accurately predict the compressive strength of high-performance concrete. As you can see, the size of training data depends on the complexity of the project in the first place. At the same time, it is not only the size of the data set that matters, but also its quality. What can be considered as quality data?  Feed your model inaccurate or poor quality data, and no matter how great the model is, how experienced your data scientists are, or how much money you spend on the project, you won't get any decent results. Remember Amazon? That's what we're talking about. Okay, it seems that the solution to the problem is kind of obvious.  But it's not that easy. Say you need to forecast turkey sales during the Thanksgiving holidays in the U.S., but the historical data you're about to train your model on encompasses only Canada. You may think, Thanksgiving here, Thanksgiving there, what's the difference? To start with, Canadians don't make that big of a fuss about turkey. The bird suffers an embarrassing loss in the battle to pumpkin pies. Also, the holiday isn't observed nationwide, not to mention that Canada celebrates Thanksgiving in October, not November. Chances are, such data is just inadequate for the U.S. market. This example shows how important it is to ensure not only the high quality of data, but also its adequacy to the set task. Then, the selected data has to be transformed into the most digestible form for a model, so you need data preparation. For instance, in supervised machine learning, you inevitably go through a process called labeling. This means you show a model the correct answers to the given problem by leaving corresponding labels within a data set. Labeling can be compared to how you teach a kid what apples look like. First, you show pictures and you say that these are, well, apples. Then you repeat the procedure. When the kid has seen enough pictures of different apples, the kid will be able to distinguish apples from other kinds of fruit. Okay, what if it's not a kid that needs to detect apples and pictures, but a machine? The model needs some measurable characteristics that will describe data to it. Such characteristics are called features. In the case of apples, the features that differentiate apples from other fruit on images are their shape, color, and texture to name a few. Just like the kid, when the model has seen enough examples of the features that needs to predict, it can apply learned patterns and decide on new data inputs on its own. When it comes to images, humans must label them manually for the machine to learn from. Of course, there are some tricks, like what Google does with their recapture.  But labels can be already available in data. For instance, if you're building a model to predict whether a person is going to repay a loan, you'd have the loan repayments and bankruptcies history. Anyway, it's so cool and easy in an ideal world. In practice, there may be issues like mislabeled data samples. Getting back to our Apple recognition example. Well, you see that the third part of training images shows peaches marked as apples. If you leave it like that, the model will think that peaches are apples too. And that's not the result you're looking for, so it makes sense to have several people double-check or cross-label the dataset. Of course, labeling isn't the only procedure needed when preparing data for machine learning. One of the most crucial data preparation processes is data reduction and cleansing. Wait, what? Reduce data? Clean it? Shouldn't we collect all the data possible? Well, you do need to collect all possible data, but it doesn't mean that every piece of it carries value for your machine learning project, so you do the reduction to put only relevant data in your model. Picture this. You work for a hotel and want to build an ML model to forecast customer demand for twin and single rooms this year. You have a huge dataset with different variables, like customer demographics and information on how many times each customer booked a particular hotel room last year. What you see here is just a tiny piece of a spreadsheet. In reality, there may be thousands of columns and rows. Let's imagine that the columns are dimensions on the 100-dimensional space with rows of data as points within that space. It will be difficult to do since we are used to three space dimensions, but each column is really a separate dimension here, and it's also a feature fed as input to a model. The thing is, when the number of dimensions is too big and some of those aren't very useful, the performance of the machine learning algorithms can decrease. Logically, you need to reduce the number, right? That's what dimensionality reduction is about. For example, you can completely remove features that have zero or close to zero variance, like in the case of the country feature in our table. Since all customers come from the U.S., the presence of this feature won't make much impact on the prediction accuracy. There's also redundant data, like the year of birth feature, as it presents the same info as the age variable. Why use both if it's basically a duplicate? Another common pre-processing practice is sampling. Often you need to prototype solutions before actual production. If collected data sets are just too big, they can slow down the training process as they require larger computational and memory resources and take more time for algorithms to run on. With sampling, you single out just a subset of examples for training instead of using the whole dataset right away, speeding the exploration and prototyping of solutions. Sampling methods can also be applied to solve the imbalance data issue involving datasets where the class representation is not equal. That's the problem Amazon had when building their tool. The training data was imbalanced with the prevailing part of resumes submitted by men, making female resumes a minority class. The model would have provided less biased results if it had been trained on a sampled training dataset with a more equal class distribution made prior. What about cleaning them? Datasets are often incomplete, containing empty cells, meaningless records, or question marks instead of necessary values. Not to mention that some data can be corrupted or just inaccurate. That needs to be fixed. It's better to feed a model with imputed data than leave blank spaces for it to speculate. As an example, you fill in missing values with selected constants for some predicted values based on other observations in the dataset. As far as corrupted or inaccurate data, you simply delete it from a set. Okay, data is reduced and cleansed. Here comes another fun part, data wrangling. This means transforming raw data into a form that best describes the underlying problem to a model. The step may include such techniques as formatting and normalization. Well, these words sound too techy, but they aren't that scary. Combining data from multiple sources may not be in a format that fits your machine learning system best. For example, collected data comes in .xls file format, but you need it to be in plain text formats like .csv, so you perform formatting. In addition to that, you should make all data instances consistent throughout the datasets. Say a state in one system could be Florida, and in another it could be FL. Pick one and make it a standard. You may have different data attributes with numbers of different scales presenting quantities like pounds, dollars, or sales volumes. For example, you need to predict how much turkey people will buy during this year's Thanksgiving holiday. Consider that your historical data contains two features, the number of turkeys sold, and the amount of money received from the sales. But here's the thing, the quantity ranges from 100 to 900 per day, while the amount of money ranges from 1500 to 13,000. If you leave it like this, some models may consider that money values have higher importance to the prediction because they are simply bigger numbers. To ensure each feature has equal importance to model performance, normalization is applied. It helps unify the scale of figures from say 0.0 to 1.0 for the smallest and largest value of a given feature. One of the classical ways to do that is the min-max normalization approach. For example, if we were to normalize the amount of money, the minimum value, 1500, is transformed into a zero. The maximum value, 13,000, is transformed into one. Values in between become decimals. Say 2700 will be 0.1 and $7,000 will become 0.5. You get the idea. Up until now, we've been talking about working with only those features already present in data. Sometimes you deal with tasks that require the creation of new features. This is called feature engineering. For instance, we can split complex variables into parts that can be more useful for the model. Say you want to predict customer demand for hotel rooms. In your data set, you have date-time information in its native form that looks like this. You know that demand changes depending on days and months. You have more bookings during holidays and peak seasons. On top of that, your demand fluctuates depending on specific time. Say you have more bookings at night and much fewer in the morning. If that's the case, both time and date information have their own predictive powers. To make the model more efficient, you can decompose the date from the time by creating two new numerical features, one for the date and the other for the time. A machine learning model can only get as smart and accurate as the training data you're feeding it. It can't get biased on its own. It can't get sexist on its own. It can't get anything on its own. And while the unfitting data set wasn't the only reason for the Amazon AI project failure, it still owned a lion's share of the result. The truth is, there are no flawless data sets, but striving to make them flawless is the key to success. That's why data preparation is such a crucial step in the machine learning process, and that's why it takes up to 80% of every data science project's time. Speaking of projects, more information can be found in our videos about data science teams and data engineering. Thank you for watching.