Awesome. Well, this is very good timing. All right, hello, everyone. It's so nice to see you all. I guess just the first thing to say is I assume that most of you work remotely most of the time. And it definitely gets kind of quite lonely. So coming to events like this, being in-person with you all with my lovely team from Red Hat and old friends from previous events and the new people you're going to meet today, it's just, yeah, it's really nice to be back in-person. So I guess, sorry to take this off now. Yeah, my name is Ian Billett. I'm an engineer at Red Hat where I'm the team lead for Red Hat's internal observability service. And today we're going to talk about Prometheus exemplars and specifically how you can use them to accelerate your debugging to warp speeds. And being serious when I say warp speed, I'm not using this term lightly. It's a very appropriate use of the term and not just a great excuse to include lots of Star Trek memes, which it is, by the way. Exemplars are one of these features that if you're not, if you're kind of earlier on in your Prometheus story or you're not kind of paying super close attention to the project, it could be a feature that kind of slips under the radar. But kind of when you see it in action, it looks like magic and I want to kind of show you the magician's tricks today. So the goal here today is threefold.  And kind of as with any change that you make to production systems, it absolutely has to be worth it. It has to be worth the investment in time and effort and energy. And I'm going to construct that argument with you. Secondly, we'll then go on to talk about the actual mechanics of exemplars, how they work, how they're ingested, how they're stored, the real nuts and bolts of how exemplars work. And then finally, we're going to see some exemplars in action. And I sacrificed the pano-chocolata, the demo gods, so fingers crossed for that one. All right, so I want to kick off just by kind of painting you all a picture which will kind of give us all the shared context with which we can understand the value of Prometheus' exemplars.  You are responsible for a system or a set of systems. And then the system that you are then responsible for starts behaving in a way that is unexpected. And we receive, and then you receive an alert or you observe some symptoms of the system not behaving. So as the responsible human, it is then your job to figure out what is going on and to return the system to a normal working order. And this is exactly why we have observability tools, right? They are tools that collect and emit and store streams of data, your metrics, your logs, your traces and more recently your profiles. And then with these streams of data, it is then the human's job to ask questions of the system using these streams of data. And it's these streams of data that then provide the answers to the questions that you are trying to answer. And every question you ask and every answer you get narrows down the space of where the problem that is afflicting your system lies. So this is kind of a very roundabout way of just describing your normal debugging workflows, but in context of your observability tools and your observability data. And yes, that is a picture of Mr. Zuckerberg giving his congressional hearing that was photoshopped to look like data. It was too funny not to include the internet's amazing, isn't it? All right. And I would say that it is the speed with which you can repeatedly go through this cycle of asking questions and getting answers that really is the making or breaking of your observability tooling. And when I say speed here, I don't just mean like mashing through like the questions and answers quickly and quickly as you can. I mean, I'm more talking about the speed with which you can home in on relevant contextual information. And kind of, then we can then use the mean time to detection and mean time to recovery metrics to kind of capture just how effective we are being here. And these are metrics that are kind of most commonly used in incident response like crisis response like situations, but I would argue that they are absolutely applicable any time your software is not behaving in a way that you're expecting. I think these are applicable. And another point I want to make is that the time you spend debugging is not productive time. And that feels like a slightly controversial thing to say. And I would add a caveat to that by saying that from my own experiences, time that I spend debugging actually has some very useful kind of side benefits. They're the times when you kind of pick up your service and you kind of look at it from every angle and you start to see it in ways that you didn't before. So there's definitely a big learning process that you get from the debugging. I guess the point I'm making here is that the opportunity cost of that debugging is always quite high. There's almost always something better you could be doing with your time, something more productive, but instead you're debugging this problem. And the final point I want to make here is that your time is incredibly precious. You know, like in engineering organizations, I'm going to assume this is true of most of you that the human cost is the biggest cost that those organizations pay out. Thank you very much. And I read a kind of, I've seen an article that said that on average, the average engineer only spends, only has 30% of your time in your kind of working week is focused engineering time. A disclaimer, I didn't know if that was backed up by any data, but it kind of feels intuitively correct. So the overarching point I'm kind of making here is that finding and eliminating inefficiencies in your debugging process is not only good for you, it's good for your organization, it's good for everyone, okay? So next question is, how do Prometheus exemplars help us achieve a better efficiency in our debugging process and what is the magnitude of this efficiency gain that we can get by using them? So when you're in this debugging loop of asking questions and getting answers, very often the piece of information you require to answer your question will lie in a different stream of data than the one you are currently focused on. For example, you see that a metric that shows increased request latencies, so you go and you need to find some tracing data which kind of breaks down these latencies. And it helps you answer the question of like, where is this latency increase going from? And I've been there like many times, you kind of looking at Prometheus and you're like, okay, where is the tracing bookmark production URL? Okay, it's there. Okay, how do I log into Yeager? What was the Yeager syntax, blah, blah, blah. Now all these steps you have to go through in order to get the answer to the question. And it is this process, this switching of context that I think is a small period of time but is actually incredibly costly. The human brain can only hold four chunks of information in working memory at any one time. So if you have to go through this costly context switching process, you have to take your current debugging context, kind of unload it into your longer-term memory, load in the Yeager context, find the trace, and then unload the Yeager context, and then go back to your debugging context. And it's this context switching that kind of ruins your debugging flow state. It's a very costly switch that Prometheus exemplars can help eliminate. The documentation says that exemplars are references to data outside of the metric set. I would kind of go further and kind of almost use more vivid language and say that exemplars give you the necessary pieces of data so that you can hop directly from your metrics to the relevant contextual locations in different streams of your observability data. And that is exactly why this talk is called warp speed debugging. You don't have to faff around context switching, remembering things. You can just warp directly from your metrics, directly to the relevant location in a different stream of data. Final point I wanna make here is like, okay, that sounds cool, that sounds helpful, but just how helpful is this? So this is gonna sound silly, but I think of this almost in terms of swimming pools. Bear with me here, right? You take the average time it takes you to perform one context switch, then you add, say, 20% to kind of load and reload that context in your head. You then multiply that by the number of times you do that in a given time period. And then you multiply that by the number of engineers in your organization and you kind of defining a volume of size of inefficiency that your organization goes through. And it's like a net negative cost that you're paying. So it's like a volume that is underground, so kind of swimming pools. And this just kind of gives you, that's a way to think about just how useful Prometheus exemplars can be to you and to your organization. All right. Yeah, and there's always a relevant XKCD. Okay, captain's log. Convince everyone to use exemplars. I'm gonna mark that one as done. So now let's talk about the actual mechanics of exemplars. All right, first question is, how do you create an exemplar? Okay. And this is done when you are instrumenting your application code. So effectively it's like the client libraries make it very, very, very simple. When you are making your observation in the instrumentation code in your application, the functions that are provided, it just observe an example, so you just make your normal observation, but you also can supply a Prometheus label set which you can think of just like a, almost like a very simple JSON dictionary. And kind of in this instance, this is taken just directly from the Prometheus Golang library. We are kind of observing this value with the label of dummy ID and like a random integer. Okay, so that's how you create an exemplar in your application code. Yeah, there it is. So once we've defined an exemplar in our application code, how do we get that into Prometheus? Well, if you think back to your Prometheus 101, you remember that Prometheus operates on a pool-based model of data ingestion. This means that Prometheus kind of goes out to each of its targets and the things that it's configured to scrape and then request the metrics page, which is a text page which kind of looks like this in the normal case, which includes the metric label set, the set of data that uniquely identifies that metric and then the value. So that's standard Prometheus without exemplars. So after you have instrumented your code just as we saw that previous example, what happens is that in this format that the application presents to Prometheus for scraping is that it depends some extra metadata. So this is the exemplar label set that we defined in the instrumentation code and then you have the value that was observed and the timestamp that it was then observed at. So that's what an exemplar looks like in terms of the exposition format, but this model has a couple of nuances that I just wanna call out here. The first one is, let's assume that your server is serving, say, 10 requests per second and every request is being observed with a set of exemplars, but your Prometheus server is only coming around to scrape that metrics page every 30 seconds. So what's gonna happen here? And what you find out that happens is that only the most recent exemplars are captured. So this means that if you have 10 requests in the time it takes for Prometheus to come and scrape you only the very last one is going to be presented to Prometheus to then ingest into its storage. So just bear that in mind. The second nuance you should be aware of is that exemplars are not supported in the standard, bog-standard Prometheus exposition format, but they are supported in the open metrics format. And like these formats are very, very similar. This isn't something to be like worried of, it's just something to be aware of. And there is some more peculiarity in their differences that I do not feel qualified to talk about, but the thing is that if you configure your Prometheus this is just gonna ingest exemplars, this is just gonna happen kind of automatically without you thinking about it, you just need to be aware of that.  What you actually need to do is just tell the Prometheus, the server that's serving on metrics page that explicitly that you want open metrics format, which will include your exemplars. All right, so we've instrumented our application, we've ingest, we've kind of gone and grabbed the exemplars from the application. So the next question is how does Prometheus store exemplars, like what's going on on the back end side? So within the TSTP, our exemplar data is stored in a fixed size in memory circular buffer is the terminology used in the documentation. And the size of this buffer is controlled by the max exemplars configuration option. What this means is that Prometheus will just hold on to a set number of exemplars at any one given time. And when a new one is added, you'll lose one from the back end. So you just have the most recent ones in memory at any given time. And things being in memory then also mean that, maybe troubles, maybe difficulty when your Prometheus restarts. So exemplars are not persisted in a long-term fashion like Prometheus blocks are. You do get the persistence of the wow, which is the right-to-head log, which then what this effectively means is that the most recent two to three-ish hours of data is like there on disks so that if your Prometheus server restarts and starts back up, you will read all that data back into memory. So if your Prometheus restarts, you will get the two or three-ish most recent hours but not the longer-term ones because they were just in memory and they're kind of lost. All right. Don't forget. Exemplars are not enabled by default in the Prometheus server. They've been in the master, the main branch story since version 2.26, which was about March last year, but there's still a feature that is marked as experimental so you need to explicitly enable it.  Okay. So once your exemplar data is in Prometheus, the next thing we want to do is actually get hold of that data. So how are exemplars queried? So very, very simply Prometheus exposes on get and post methods the query exemplars endpoint. Super, super simple. And to this endpoint, you provide a query, you provide three things. You provide just a standard Prometheus query. Any query doesn't matter. The start timestamp and the end timestamp. So what's kind of going on behind the scenes here is that Prometheus is taking this query. It's then with this query, it's parsing out the selectors, like the label sets from this query. And then with those timestamp, it's going through its circular buffer and kind of pulling out all the exemplars that kind of match those criteria and then returning this to the user. It's very simple. But the final question that I then want to address when we're talking about the specific mechanics of Prometheus exemplars is how can you walk between the data streams? Because it's all very well having a HTTP endpoint that gives you JSON back. But in reality, that's not going to give you a real first class debugging experience. And so the answer to this question, like at this point in time, is simply Grafana. No, this is Prometheus days isn't Grafana cons. I'm not going to go into it in loads and loads of detail, but, and the point in Grafana specifically is in your Prometheus data source settings. And when I show you the demo, we'll kind of show you what I mean by this, but like that is the point that you tell Grafana that you're interested in exemplars and you're interested in linking them into other data sources. And Grafana does kind of, I said I wasn't going to like talk about Grafana, but it kind of does two neat things with the data sources. It lets you define an internal link and an external link. So an internal link will basically allow you to hop from that exemplar data to another point, like within Grafana, but it also lets you basically template into URLs, your exemplar data, which will link to a place outside. So we'll see that in action, hopefully. Okay, internal versus external. Yeah, yeah, and I've been, I was bumping into a bit of trouble with the URL templating in Grafana. It wouldn't let you have all of the global variables, like the timestamps. It just seemed to be just the value rule, but I'd be happy to be corrected. Otherwise, if other folks know better. All right, and I guess the final point is that all of the code kind of I wrote, kind of for the demo that we'll kind of show you here is just available in that repo. It uses a really neat framework that allows you to run Docker containers from your Go test that my esteemed colleague, Mate, is, and Jessica are giving a talk on, so I definitely can check that out. Yes, all right, demo time. Time where it all goes wrong, okay. So for this, awesome. Okay, so for this demo, what we've done is we've written a very simple server that basically just calculates Fibonacci numbers, is it? It just picks a random number and calculates that Fibonacci number and then kind of exposes metrics and traces and logs, but we're not gonna go into the logs. And then these are then ingested into Prometheus. I'm using for metrics, obviously. I'm using Grafana Tempo to ingest the metrics and then obviously using Grafana to visualize this all. So I'm kind of looking at it from here. So these are, this is my Prometheus data source in Grafana and I've done this in configuration but we'll show up here. This is the crucial bit here.  And we have our internal link that goes to Tempo using the trace ID and then we have our external link that's linking to parser, which is the continuous profiling tool. So then we'll just kind of just pretend like we are going to query for a, oh wow, this is not good ergonomics if I'm just like. Okay, so let's then just do the very simple example of wanting to look for the 95th percentile of my demo metric that I'm exposing and you'll see, and you'll see here what this data looks like. So all of this is obviously, this is just the normal line that kind of you draw but what's really cool is that after I've enabled exemplars and I've ingested them and I've configured all the things to the right level, these little green triangles are that exemplar data. So what's really neat is you write this like quite high level aggregate level metric but then it gives you the possibility of taking individual specific points that relate to relevant contextual pieces, relevant contextual locations in other streams of data. So just take for example this data point and it kind of gives you the option to first query with tempo. So let's just see what it looks like so and then you'll see straight away that trace. You can see here this is us calculating Fibonacci numbers that is almost as pointless as Bitcoin mining but perhaps not so much. That was actually probably the most controversial thing I'd say in this talk, all right. And then looking at these traces we were then calculating and the steps I would have had to have gone through to get to this point would have been such a pain in my neck but it's just a one click and that is that one click hop, that one click warp is the prime debugging experience here. But the other cool thing, so that was an internal link to tempo but the other cool thing that I will show you is just by templating, you can template into this external link which then just links me to a totally different data source that then lets me dig into where's my, where's the icicle graph? That's the real, there we go. And then like it's a contextual warp to another relevant place in a different stream of data but for those of you that are hawk-eyed you'll notice you'll realize that this is not actually the thing that was running because this is just on the parsers demo but the templating works and you can link to an external source so that is how you would do it. It looked a lot like that. It's really cool. It is warp speed debugging. Cool. And with that, that's all I have to talk about today. So that's all folks. All right. Thank you Ian. Are there any questions in the room? I saw someone typing on Slack, oh yeah there, question from Slack for you. All right. What magic was that? I was about to see the demo and it went to the, oh that was a streaming question. Sorry about that. If anyone has any trouble viewing the stream or wants to watch the sessions again, everything is recorded. Everything will be up in a short period of time. Thank you. It's possible today to send an exemplar by remote write? Good question. I don't know off the top of my head. I assume so. Does anyone know off the top of their heads? I assume it is. I may be looking at them. It is. Three thumbs up. It is. There you go. All right. Any more questions on exemplars? Thanks for a nice talk. Is it always like this that the client libraries would expose the last exemplar? Should it be always like this or can it be through some magic of detecting the very interesting ones? That's a really good question. I think the short answer is that's the only way that it works. I think the clue is in the name, right? Exemplar. It took me a while to figure out why that was chosen, but it's an example of something. It's not everything. It's an exemplar of the data. So I guess just to follow that logic through then is that the exemplar that you observed that Prometheus scrapes, if you have a system like tracing that you can't hold everything, there's no guarantee that that's... Exemplar you've observed is going to link to anything that your tracing system can serve. So yeah, there's nuance to this. But about the most interesting thing, I don't think you would know a priori like before Prometheus has come to scrape it if it's going to be interesting or not. So, and that would get a whole complicated... I think it's probably just the easiest just kind of knowing the limitations and kind of working with them. All right, thank you. Good question. Embrace the randomness. There was another question. Thank you for the talk. You mentioned that only the latest exemplar is stored, right? So is that the latest per set of labels that like for a particular label set, the latest value will be stored, the latest exemplar. And if that's true, let's say a particular metric was raised with the exemplar and for the next five minutes, there was no new exemplar. So will that exemplar be sent out in each scrape to Prometheus? Okay, I think I get what you're saying there. So we're talking about in the code that the application, the instrumentation code, it's just, so it's on a, the label set basis. So if you have like a really broad histogram that has really, really long and really, really short and it's only very rare you get a really long one, that will persist like indefinitely between scrapes until another new one comes along to kind of knock it off its perch, if you see. So, and I believe at like, at least until the process restart or whatever. But on the Prometheus side, like the backend side, I believe that the exemplars are identified by the label set that you ingest with. So it's similarly to the application side, yeah. Cool, really good question. Another question, wow. Very interesting topic. Are there any plans that Prometheus will persist those exemplars? Another excellent question that I do not know the answer to, that I look towards the table of maintainers. I'm gonna, I would think probably that the, the overhead required in terms of storing and serving those would be too, too high in a similar way that the kind of, that tracing kind of can get very, very big, very, very quickly. I would imagine that there'd be a similar constraints felt on the Prometheus side. And you know, like what is the use case of this? Is it like, it seems to be geared towards like a very, like it was recent two or three hours. And if that's good enough and you don't have to go down this long winding path of storing them in the TSDB and sowing them from the TSDB like blocks, then you know, that just feels like we kind of getting 80% of the goodness for 0% of the work as it were. So I would raise an issue on the Prometheus repo and kind of see if any smarter folks have anything, any smarter things to say. Cool, great question. Thank you.


Thank you. So, Senate government operations. It is Wednesday, April 21st. I have to look at my computer to see what the date is. And we're going to be looking today again at the retirement pension. Bill that came to us from the house, it is H449. Yesterday, we spent some time on the governance. Issue and today we're going to be looking at the. The task force that is set up to review the benefit. Package as we go forward. So, I think. What I'd like to do is. If you might as with us now and I'm. I think I'll start. Because this, this. Most. I think I'm right here. I think it has a huge impact on the. Plan participants. So I think what. What the way I'd like to start is to hear from. Michael Neil, Jeff. Fan and, and. Steve Howard first. Around some of the issues that they have around it. And then I have the conversation there. Does that make sense? Because I know they have some real concerns about it. So that's, I think I'd. Like to start there. And Mike, I guess since you're the only one here. Right now, I guess we better start. With you. Thank you. It looks like I lucked out. Yeah. The director of the Vermont trip association. Thanks for the chance to testify again on this issue. To start with, we appreciate the decision made by the house and Senate leadership to go in the direction. Of creating a task force to study this issue. As you know, there was a lot of concern from our membership. We had a lot of real fear over what was coming with some of the proposals that were put out there. And, you know, one of our hopes was that we would have the chance to study the impact of this. And fully understand exactly what was being proposed and what it would do to the workforce. So to start with the makeup of the task force. We had a lot of concern with that. There wasn't an amendment made the gave the VTA a seat there. Originally we didn't have a seat. So we do now. We are comfortable with that. And the concern we have is with the. Charger of the committee, you know, what, what the. Committee is supposed to be looking at and accomplishing. And, you know, the way this is worded. As we think about what the. The impact of the work that we have done with us is that we have been able to get back to the annual required contribution. And the unfunded liability back to the level required in FI 21, what they have what the unfunded liability was then. What they ate at was then. It certainly seems to me that this language is pre determining the outcome. Of the task forces work. You know, it may not be telling the task force. But if the task force is being told what the savings need to be, we're definitely predetermining what is going to be what we're going to have for an outcome. And how does the workforce have any faith in a report that's issued by a task force that's told what to have for an end result? You know, we really feel what should be looked at very closely is what the appropriate benefits structure should be and determine how that is paid for. You know, should some of those savings come from the employees? The answer is probably yes. But what is the right level to be coming from the employees? And what is the right level for the employer to have to absorb some of this cost going forward? You know, this is a problem that neither side is directly responsible for. You know, the change that we saw in the unfunded liability in ADEC was due to changes in assumptions from the actuaries. And it had a big impact. We all understand what that did to the numbers we're looking at. But why should that all be put on the backs of the employees in diminished benefits? These are benefits that they have been paying for in contributions since their first pay period with the state. When you're hired, you're not asked whether or not you want to pay into the retirement system. You're obligated to pay into the retirement system. And I know your committee earlier in the session had some discussions about whether or not there is a contract between the employees and the employer. There's no discussion of that in any of the duties of this task force. Yeah, I believe that's something that should be another piece of what the task force does is look very closely at what that contractual relationship is. And with that relationship in mind, what can be changed with employees that are already paying their contributions and what can't be? That's something that needs to be fully understood before any changes should be proposed to benefits. It's a big piece of what we're dealing with here. The other concern I see with this is just the timeframe. And I testified to the same thing in the house. The work of the committee will start on June 15th and a report be issued by September 1st. And I think probably everybody here has been involved in some type of summer study and seeing how difficult it is to schedule some of these when you're working with this commission will have or task force will have 15 people on it. And it's not easy to schedule those meetings over the summer and get it done in a period that's that short. So if the work of the task force is really going to be meaningful, they need to have the appropriate amount of time to get the work completed in issue a report that isn't rushed or cut short in some way just because we're trying to meet a timeframe that has been put into this language. So I think that covers the concerns I have. I think that's a good question. I don't know if anybody has any questions. Does anybody have any questions right now for Mike? And then just, I think what we'll do is hear from everybody and then try to have kind of a general conversation. About it. Because. I think that. That will lead us to better thinking and better results. I think that's a good question. I think that's a good question. I think that's a good question. Thank you, Tom. Well, I just wanted to ask my, my understanding is that there's a language in there that alludes to looking at. Defined contributions as opposed to defined benefits. And I just like everyone who speaks to have an opportunity to say how that, you know, is landing with their members as well. That isn't something our members would have any interest in is it's a big part of why people come to work in the public sector. Yeah. These benefits are important to our members, but they're just as important to the state of Vermont as an employer. And if not there to attract people and retain people. It's a problem. We're all going to see down the road. It's especially important to a department like the state police. Yeah. Our ranks, structure is built on longevity. To move up through the chain of command. We need people to stay in employment. For an entire career. If we were to go to a defined contribution, it doesn't keep people there. They're retirements that are portable. People could leave at any time. And it would do damage that I don't think the state police could recover from. I'm not sure how we would maintain a rank structure. If we didn't have a benefit structure that kept people. In employment for an entire career. Senator Palmer. Thank you. Madam chair. So Mike, I think I raised this. The last point you made about the timeline. When we had the members of the house committee in. I also felt that June 15th and then the end of this. September was a pretty compressed time. I think they're called to meet maybe nine times. If I'm remembering correctly. Something like that. And the answer that was given at least at the time. And I, I understand this part too. Is that every day that goes by that we don't have a solution. That. You know, we continue to sort of lose more money. If you will. But I'm wondering whether, if you gave another month, and I don't know the answer to this, maybe. Beth or Tom. No, better. I mean, how much of that? I don't know. I don't know the answer to this. Maybe. Beth or Tom. No, better. I mean, how much are we really talking about. In a month or an additional six weeks of just, or do you have a sort of an end date that fits better into your view of this? If I can just respond to that for one second. My, these, whatever is come up with by the task force has got to be adopted by the legislature anyway. So I don't think anything is going to happen until January. So what doesn't make any difference if they come up with a benefit structure on June 20th. Or December 20th, the legislature has to act on it. My understanding from the house. Was that they wanted it done because they have to start dealing with. Redistricting. And they're going to be busy with that. But I. And I have sympathy for that, but I also feel that this is a really important. Issue and they have 11 members on their committee. And maybe they have to divide up the work somehow. So that they can start redistricting. I don't think we should. Set an artificial date here. To accommodate that. That was my understanding of why they set that date. And I stayed correct. Madam chair, you are correct. I think. The representative of the chair of the house committee did mention that. Yeah. I think we would also know from the treasurer's office and the retirement division, can they accomplish some of the work that will meet need to be done in that short of time period? Because a lot of it will rely on. Work being given out to an actuary and getting the information back. And it may be very difficult just dealing with that piece. And I think there is scheduled to meet. They're allowed to meet 15 times, but it's only 11 weeks. So. It is pretty compressed. Well noted. Any other questions for Mike right now. Steve, you weren't here when we first started and. Yeah. I think the way I decided to do today was to start with the, with the VTA, the VSEA and the NEA, because you have. I believe the main concerns. About this section of the bill and then, and then here from others who may, may have some suggestions, but. Have not expressed as many concerns. So. Mike got to start because he was here. And now I'll turn it over to you. Thank you, Madam chair. I'll try to, to. It's a hard act to follow. I'll just put it that way. Okay. Well, you're done then. Yeah. Thank you.  That is what I'm concerned about. I know that the key awareness and equity and. And equality. Are sort of the key balance or the key buzzwords. That I would use to describe this. Our concerns about the current house proposal. I would add a word and that is credibility. Which is really important. For our members. the task force outside of the members of the three labor unions. Our concern is that the number of members in the three labor unions be equal to the number of members who are not in those three labor unions. So whomever the non labor members are, meaning whoever is not in one of those three labor unions, you know, we don't really have any concern about we're used to sitting at the bargaining table across from folks who have different philosophies and different views of the world and we work out an agreement. We're used to working collaboratively with the creation of whatever party happens to be in power to govern our healthcare plan and we have one of the best managed healthcare plans in the country. What we're not comfortable with is not having equal voice at the table. So that I think is important. I also would say, and the timeline is really part of this, you know, the speed at which the leadership in the house tried to move the speaker's proposal really caused a vicious backlash from our members who thought that they, that the political leadership was trying to shove something, you know, sort of through the process without vetting it, without allowing them to be heard. So we don't want to repeat the same mistakes that were made in that process. And I think that the truncated process of June to September is really hard. We are still managing a pandemic. We hope that, you know, around July the governor has said that he thinks we'll probably be turning the corner on that. And you know, our members are the folks who led the state through the pandemic. And they're still doing that. So a little bit more time would be very valuable. And I think what you said, Madam Chair, is accurate. I mean, we do understand the speaker's concern in the house government operations concern about redistricting. But really tell you the truth when it comes to comparing the financial impact on our members of the changes to retirement, you know, that is not really something that we consider a part of our problem. And we think it's manageable that they should be able to do both at the same time. And we told them so when we testified on the House side. One aspect of this, and I know the teachers share this concern that's very important, especially for the government operations committee, and I'll repeat it because I did say it last Friday, but this whatever comes out of this committee, however it's done, and that I think one of the things that needs to be deeply investigated is the impact on staffing in state government, particularly in the in the places where with a with a with the current pension that is 7% let the benefit is 7% less than the national average. And when you could just go to New Hampshire and either work for the federal government's Department of Corrections or the New Hampshire Department of Corrections or the New York Department of Corrections and have more pay and better retirement. We really could have a serious impacts on all on all classifications, but particularly in the classifications where we already have high vacancies. And I don't have to tell the chair who represents a board of county or Senator Collin Moore or Senator Clarkson, anybody who lives, you know, within a border that if you can get if you can work in another state and get better benefits, you know, the community is not that bad for a lot of folks. So that's a concern that I think really has to be looked at the other thing that there's some very prescriptive language in there about almost, you know, about what the goals of the task force have to be. And we really believe that the task force has the resources within its own makeup to decide what the goals should be. And, you know, maybe the legislature doesn't know what the right goals should are in the time that they've looked at it. So I would recommend that that language be removed and that and that the task force decide, you know, how much of the unfunded liability they think really needs to be eliminated, how much of the ADEC needs to be eliminated. That's really something that the task force should decide and they shouldn't be told what to do. I mean, this bill goes as far as to tell you who the witnesses should be. So it's really it's I think everybody sort of knows this, but one of the big lessons that one of the things that we continuously try to share with your colleagues from the other body is that the process really matters. It really, really matters. And if people don't feel as our president Amy Town says of every opportunity she has, if they don't feel heard and valued, then they give up and they become cynical. And we don't want that in this case. I know that's something that is a major concern for the members of the VSEA. Senator Rahm's question also is a very important point. We agree with Treasurer Pierce that a defined benefit plan is a more secure, less costly plan for the state of Vermont. And that it, you know, 62 cents on every dollar comes from the good decisions that Tom Blanca makes. And we are pulling money out of Wall Street and we are spending it on Main Street in Rutland, in Fairhaven, in Woodstock. 78% of our members retire and stay in Rutland and stay in the state. And they are investing in local businesses. This is the most important economic development program in the state. And so to even have this in the bill as even a possibility is I think a bad omen for the future of the retirement system. And if you look nationally, the state, and I know Treasurer Pierce has made some efforts to try to fix this problem that we supported, which is that Vermont has very little, Vermonters in general have very little retirement savings.  So do we want to have money pulled in off of Wall Street and spent in the private sector on Main Street? Or do we want to have a less stable, more costly retirement system that encourages the use of the social safety net for people to survive? And, you know, the social safety net is not funded by Wall Street. It's funded by taxes that Vermonters pay. So if you for the affordability agenda, you are for a defined benefit plan. So I'm going to ask committee members if they have any questions right now. But I would first, so I would like us to think about a couple of things here. If there is an imbalance, how do we correct that balance? Are there suggestions as to how to correct the imbalance? And I understand when I first was thinking about the imbalance, it seemed to me that the six legislators that were on here are not either labor or non-labor. But when you and Jeff both were talking about the balance, you were talking about plan participants and non-plan participants, and that that's the balance you're seeking. So how would you correct that balance? And then I just have a very specific question for if somebody can just help me understand this in the charge of the task force. It says, including a review of whether or all or part of retirement income should be tax exempt. Is that really part of the, isn't that for the finance committees to determine what kinds of retirement should be? And are we talking about here all retirees in the state? And why is this even in here? Can somebody answer that? Are you asking me that Madam Chair? I'm asking anybody who has an answer for it. Alison? I absolutely think that does not belong in here. And it is in the purview of House Ways and Means and Senate Finance. That is not appropriate here. That's what I, I just, that just leapt on at me and I, so I just wondered if anybody else had that concern. May I, yeah, I'd like to know what the recommendations are on balance because it seems to me that six legislators are pretty heavy. So one way to look at balancing this would do four legislators seems just fine. And then three professionals who actually know what they're talking about and six participants. And so I think that the expertise of the three identified non-participants who may be participating actually in this, they may be participants in the pension system. No, they can't be according to the bill that we have in front of us. But don't all state employees pay no matter whether you're a VSEA or not, pay into the state? Oh, the legislators. No, I'm not talking about legislators. I'm talking about the HR. Oh, I see. I see. I see. Aren't they all, aren't they, don't they all pay into the bench, pension? There's a, there's a small Senator, there's a small group of people and it's a shrinking group of people who are very wise. It's because most state employees are in the defined benefit plan. And those are the folks who really looked at it. There is a defined contribution plan that is available to some exempt employees, but the numbers of participants in that plan are declining. Right. I've heard that. So just to go back to balance it, I would propose that we, you know, unless we hear from Steve, Jeff, or Mike, you know, that I think it's a pretty legislature heavy and I'd rather see experts like HR, DFR, and the retirement with, at the Treasurer's Office serve more than six legislators. So it's my turn. So I am going to ask, I'm going to ask again, put Commissioner Pechak on the spot. And ask why, why you're on this one in particular as a member, as opposed to a, there are, it seems to me that whoever is on this task force, you can't get all of the expertise that you need on this, on this task force and there's going to have to be outside expertise that, that the task force relies on to come and give testimony. So I'm just curious as to whether you think you should be on here or somebody who is consulted and gives advice. Yeah, thank you, Madam Chair. Nice to see everybody again today. So, you know, I think probably similar to the testimony that we provided yesterday, I can talk a little bit about our department's background and expertise, but really leave it to the committee to determine, you know, if DFR makes sense in terms of a member of the task force.  DFR has a seat on, on Beasters, which is the, as you all know, is the teacher's retirement board. And we also regulate VI, which provides health insurance to that retiree group. So we do have that experience and knowledge when it comes to the issues facing retirees and the teacher system and their health care costs. Generally, obviously, we're familiar with health care costs. Also, as we regulate the commercial health care industry, separately, you know, we often have, we often work very closely with actuaries inside or outside of our department about things like, you know, expected returns, expected longevity of individuals that are being insured, you know, what the amount of money that needs to be put aside, what's the solvency of the company. So, I mean, those kind of technical skills and financial acumen is something that, you know, comes second, second nature to us. So I think that's another skill set that we bring.  So leave it, leave it to the legislature. We're happy to serve if that's the decision. Thank you. Any questions for a commissioner at this point? Okay, I think that Jeff, you were not here when we first started. And what we thought is we would go first to the people who have raised the most concerns with us. And so we've heard from Michael Neal and Steve Howard and would like to hear from you. And I do know the notice that you've posted. We have some documents that you've posted. And we've talked about suggestions for how to solve the imbalance. And then the timeline, those are the things that so far have been brought up as, as the major concerns, I believe, in my, by your other two colleagues. Well, if I, if I may, please, please, for the record, Jeff Vannon, Vermont and EA. Thank you for letting me speak to you today. I'm Pyle Judge. Can I just stop you for a second? Sorry, Mike, maybe if you mute, it would help, but there's like a lot of feedback. Yeah, sorry, I'm in Winooski and it's probably plans. Thanks. All right. It's always fun on Zoom. No, it's not. No, I'm a big sarcastic. So thank you for giving me the opportunity. You know, in all essence, we support the the bill as it's going through. We have a couple suggested tweaks. We appreciate the good work. The house is done and doing and they made some changes yesterday, as I understand it, haven't fully digested those. One of them was changing the report out from the task force. I think they made it till September 15. But in my written testimony, I'm suggesting October just to give a bit more time for the task force to accomplish its work. It's a serious undertaking and significant.  So certainly the house understood there was a problem with timing extended, but just two weeks. I think it could go a bit further. And I don't know what Mike or Steve said. Senator Rom, did you have a? I do. And maybe it's related to a misunderstanding that I've had. I thought that there was a limitation on beneficiaries sitting on the final governance committee, but not on the task force. And it relates to a question I would have about teachers. And was there any, was there thinking that if there's a teacher on the committee that have a really hard time, you know, ending in October when they've just been a month doing back to school, work at well, but maybe you're just still mistaken. I might have miss spoken on that when I said that the, what it says here is that the legislative appoint appointees to the committee cannot be members of the retirement plan. And the, and members of the retirement plan that are appointed by the unions cannot be legislators. That's the, I think that I think I misled you there a little bit. But Senator Clarkson. Madam Chair, I think Keshia may also be referring to sort of some of the timeframes that were discussed in relation to when people were working. But I think I point out that everybody on this task force is working. Yeah.  So we, but, but I think originally we heard that the task force would start mid June because of the teacher schedule. But in all honesty, everybody's going to be working and the, and the meetings are going to be have to be scheduled thoughtfully to, to, to deal with that. So I think everybody's working in some capacity or another. So it's always a scheduling challenge. Yes, absolutely true. So I just, just to, to clear, I'm in my written testimony, I talked about October 15, just to be clear, understanding that yesterday, the House Appropriations Committee said I think said September 15, I extended, I thought another month was sufficient. So I say October, but what I meant Senator Clarkson was October 15, just to be clear. That's what I'm suggesting. And if I could just say that, that's kind of where I was going is the first of October, that's, that's, that last month is going to be really intense decisions while teachers are also getting kids back to school. So October 15 feels more reasonable to me. Okay. But every, everybody's working in it. Yes, everybody is going to be working. The state employees are going to be very busy. The legislators are going to be very busy because they all have other jobs. Everybody's going to be busy. So my guess, my question is, why would we not make this report due December 15th, like we do all reports? So anyway, that's, I would not, I would not stand in the way of that Senator White. But I know that people wanted this earlier rather than later. So being respectful and trying to meet a balance here, we're suggesting October 15. But if, if you want to push out to November, December one or something like that, that, that would not be objectable to us. I would just remind our chair that our drafting deadline is the first week of December. Okay.  I don't want to be I was just going to agree with the need to desire to push it out further into December. The other option would be if we would write this into the plan, but to put an interim report out sometime in the fall, to give people a chance to comment on it and then come back to the final report this in early December. Oh. The way they did the higher education report, whether they like it or not, they basically did a preliminary report that they released so people could respond to it. And then they did a final report later on. It was interesting. Yeah. So, Jeff, that's an intriguing idea, Senator Polina. It certainly may have merit and give people time to see where the, get a weather report, see what's going on and then a final written report thereafter. Right. It's intriguing. Because it would make it be easier then to prepare for the upcoming legislative session because we would have some discussion over the fall but not have the actual final document until early December. Anyway, it's just a thought for now. Good thought, perhaps. Let's let Jeff finish this. Yeah, I'd say let's be open and weigh our House colleagues challenge with reapportionment too. So, I mean, I think we have to be thoughtful about everybody's pressures. Absolutely. So, I'll start, I go back. I started at the end of my written testimony with the timeline because it seems like you folks were talking about it and I wanted to get there. And obviously, it's a bit of a conversation. So, composition for us still is a need of balance to convey any work and recommendations that come out as one of being from fairness. So, we think that it's important that plan participants, as I talked about yesterday, IE employees and non-plan participants, they need to be balanced out. And whether you add to the union, Vermont and EA, the SEA or VTA there or delete on somewhere else, it does not matter to us. We just think that driving at fairness and balance is what we're looking for. As to the charge, one right out of the gate, one of the charges that I think needs to be adjusted is found at section 10 C1A, which is on page 22. Of 27, my world. And it's talking about establishing the targets back to the pre-June rate of last year. And for, you know, the Treasures proposal, the House initial proposal a few months ago, neither one achieved that target. So, it seems like we're sending the task force on a fool's errand. And I don't think that's advisable. It wasn't obtainable by the others. I think we ought to let the task force hear from informed entities, people, and then establish the targets that they think are reasonable and practical and achievable for the plan, keeping in mind the targets themselves and how they impact on the workforce, on employers, and the system as a whole. So, I think that's really, you know, that target right out of the box is a problem. And to meet the target seems to undercut the work of the task force, right? You've given them marching orders. It's, it preordains a finish that they've been asked to examine truly. So, if you're going to ask them to do the hard work, and I think you are, we ought to let them do that hard work and not preordain an outcome. I think that's, you know, that's right out of the box. The charge needs to be adjusted to let them find the target that works for the system at large. We still have concerns about the workforce recruitment and retention. And I think everybody's heard about, certainly I'll speak to the teachers and I won't speak to the troopers and the state employees for Steve, but I know they've heard the same thing, that finding people and retaining people to be employed is a challenge right now. And we ought to let, in fact, the Secretary of Education, Dan French, just two weeks ago, spoke to the shortage of teachers, qualified teachers in Vermont right now. So, the task force should examine the effects of any changes, benefit structures, contribution characteristics, and on the workforce recruitment and retention. And we think that's important to include in their work. The impact on other state benefits, and I think I've said this here, I certainly said it in House GovOps. In the early 2000s, the state had to step in, and I think I remember being in Senate GovOps many years ago, and the state had to step in and actually increase teachers' pensions unilaterally because they were so low and that people were significantly below the poverty line. And so the state stepped in and increased teacher pensions. And I'm not suggesting we do that, but you ought to look at how these changes might affect people of color, women in particular, who make up 77% of the teaching ranks, and public state employees as well, to make sure that we don't drop anybody below lines, poverty lines, or otherwise, that we really don't want them to be dropping below, adding to the cost of state government and other places. And the impact of underfunding, certainly in the teacher system, and I'll speak to that only, the underfunding is clearly a challenge for the teacher system. If you look at the difference in funding, full funding for the state employees, for example, versus the teacher, it's about 16, 17% difference, if your point difference. And that's because of underfunding, and I think underfunding only, because we've been investing with them, as we know with VPIC for more than a decade. So that seems to be an underfunding issue that ought to be accounted for as well and looked at. We still believe and think that there needs to be a dedicated revenue and proper use of the one-time funds. So we think that S59, which is the bill that would actually increase taxes on the wealthiest among us, ought to be incorporated, at least mentioned in the bill here, to have the task force look at that, whether there should be a dedicated revenue source, as well in separate depart, we think included in the bill ought to be used to the $150 million one-time funds right now. Using that money now seems to be a wise fiscal decision. I don't think anybody really objects to that and thinks that it isn't wise to do. So why are we all reserving the money? Let's use it now so that we can begin to address some of the underfunding and the ratio problems that are we're trying to address. Use it now, use it early, sort of the reverse, it's the compounding effect. Let's use it. We have it now. The economic arm has been given a good boost from the federal dollars and we've increased our revenues. Let's use those wisely now to put towards the pension. And finally, the intersection with health care costs and teacher system. The governor often talks about the OPEB four buckets and we're requesting some language added to the bill to talk about health care costs that should be enhanced to study health care benefit, design innovations in the state education system. Not in the state employees, but in the education system. It's a little bit different in the education world. We're now finally bargaining at the state level for that. But look at that more holistically and take a look at what we could change there in the health care arena because it does the OPEBs, the other post-employment benefits do affect the state's contribution. So we think that ought to be added into the mix as it relates to educators. So I've got draft language that I've sent along includes all of those and some specific wording and we hope that you will consider those and I'm happy to answer any questions you might have. Does anybody have any questions for Jeff right now? And I apologize because I didn't read your before this and I No need to apologize. I just sent it, you know, a half hour ago, whatever it was. There's no apology necessary. Okay. So committee where? Okay, let's, I'd like to hear from Mr. Golonka and Treasurer Pierce about whether about there where they are with the task force. Thanks, Madam Chair. VPIC did not take a position on this section. We were more concerned about governance, obviously. I will say a couple of things. I think, you know, we've had informal meetings. Beth and I with members of legislature, we had a study group with the governor's staff year and a half go right for COVID, which we got together with and they, they were positive, but they never really went anywhere. I think formalizing it through some type of task force like this is positive. And I think it will move it forward, I think better than we were able to do a number of years ago. In terms of the specifics of the task force, I won't, I won't, you know, in terms of makeup, you know, I think I'll leave that to you. That seems to be a beehive I don't want to go into. The rest of the topics in regards to to be overly some, some seem to be overly prescriptive in regards to it has to meet certain benchmarks. I'd also put in the documents, something to say, and any other ideas that the task force can come up to that haven't been thought of at the legislative level, because there may be other good ideas. I know Oregon did a study years ago where they tasked a task force similar to this, and they did more broad goals where it was like reduce the unfunded liability by give us ideas that we could then prioritize in terms of cost and benefit and the legislature would prioritize that going forward. So generally I think it's good idea to have a task force in terms of structurally that up to you, but anytime you address this unfunded liability issue will help the pension board meet our obligations, which in particular for the teachers, you know, we're running a 40 million deficit every, every quarter. So we need to come up with 40 million dollars every quarter, basically 15 million dollars a month. And that's not going away anytime soon unless you address some of these issues. Benefit changes have an immediate impact on actuarial changes, investment returns take five years to smooth into the, to the puzzle. So these type of decisions will have more of an immediate impact. So I'll turn it over to Beth if you have any. I am just going to say one of your things about the about whether or not we define we have are so prescriptive around the the savings. But it does say including the following and our drafting. The way drafting is in the legislature is if it says including the following and it means not to exclude other things. So they've started to stop dropping that from there because including always means anything else that you can think of. Just thought I'd throw that out there just to make sure that we all understood that that is there. Thanks, Beth, you are muted still. I'm better with numbers and technology. I just like to point that out. So thank you very much for the opportunity to speak. And I've been taking notes because I've heard some very thoughtful input and I've tried to incorporate some of that in my recommendations. And I appreciate the opportunity to to listen to those discussions. And as I said incorporated, you know, when we started our process, we had for the most part four meetings a week, two meetings with the with the VSEA and the and the Troopers Association and two meetings with the NEA. I also did a town meeting with the numbers I hear Jeff go from 800 to 1000, but it's a large number. And again in the state system about 350 employees. We had 100 plus people on the on their council. I met with another very large group, the Chittenden Democrats, excuse me, sorry, the Chittenden VSEA and as well as the Troopers. So, you know, I think that that's very important. And I think that that will be one of the comments I have about balance and about time frames. So let me begin with the creation.  I've got a lot of suggestions here. And the creation of the committee, this really follows what we did in 2009. In 2009 was a very different process. That's where we were writing recommendations and doing some of the research that has already been done by the Treasurer's Office and the General Assembly. Now we need to do more. There's no ifs and buts about it. I think that the employee groups are going to have some ideas that we need to research. I think we should research the revenue issues. I think that's important to do. But I think that that we should take a look at this as how do we get to over the finish line. And rather than, you know, create a report that may or may not have a lot of dust on it or may or not may or may not end up at the end of that process, we're, you know, helpful to the to the legislative session as we move forward. I would think that the focus should be on creating legislation for the for the upcoming session. We've done some studies. We need to do more and be informed. But I think that getting over the line is extraordinarily important. Now in 19, 2019, we did convene a committee. Tom mentioned that. And there were a couple of problems with it. One, that it was in the fall of 19 and that it was too big. And I'll get to that point as well when I talk about recommendations here. And the other is that it was interrupted by COVID. But we were late to we were very late in getting information over to the general assembly, working primarily with the speaker. And we just ran out of time. And I think that we need to give this one the appropriate appropriate amount of time. But getting back to creation, I think that it's extraordinarily important to have in that creation a recognition that ultimately we can produce recommendations. We did we can produce options. Joint fiscal gave you some options to take a look at. But ultimately, the decisions that are going to be made are by the general assembly. When you look at the statute, you make the decisions about benefits. The Treasurer's Office, the retirement boards can give you recommendations. They're not always followed. Senator White, you remember in your committee we've made different comments over the years that have or have not been accepted. I know the Treasurer Spaulding had some specific ones when we were creating group C, for instance, and we've now done a study of group C that you were participating in. So I think that ultimately you folks are the creator and the owner of the solution. And that should be very clear in the creation of this this committee. Otherwise, we're not going to get there to be very frank. I will sort of disagree with Jeff on the issue of the goals. We set a goal to get to those numbers. And frankly, that was a bit of a reduction in our original goals, because in 19, we were trying to get to reducing the prior year valuation results. And that kind of went by the wayside and we said, we'll take those and start with the impact on the 2021 budget as opposed to the, excuse me, the 2022 as opposed to the 2021. Now, we did not get there. We did roughly 78% of the target that the Board of Trustees gave us. The Board of Trustees said, go out and try to find a way to get to lowering it to the previous year's level. Ours is roughly $474 million of the total, about 78% of the goal. And on the ADEP, the actual actuarially determined employer contribution, which the legislature needs to appropriate. There are some that comes from from the locals in that process, some federal grant dollars and some other dollars in there. But we got to about 88% of the total. And that's where we were very clear that that's where we ended. You have other options. Senator Rahm in my first conversation with you folks talked about revenue and that may or may not be part of it, although I do think, and I think it should be explored, but I do think that some structural changes and contribution changes are going to be necessary. I think having a goal in there is important because it gives you a target. And whether or not you meet that target, it gives you something to work toward. So I would recommend keeping that in, recognizing that there may be some shifts when you get to the actual work. But I think that's very important. And again, making sure that the charge is very clear that you folks are the lead in this. I believe that again, there will be more options. I would point out also that the 2009 committee, even though I think that it had a slate, a very different focus, had two legislators on it. Senator White, you were a member of that and appreciate the work that you did there. When I'm looking at this, I have similar concerns that the employee groups have mentioned. One is that the committee is too large. And I think that we talked a few days ago about the, and yesterday, about the committee on the original proposal from the House that had a 15 board for VPIC. And it had a commission that had 15 members and then a second investment committee. And that it was both too unwieldy, which we've seen studies in other states, the BC study, that again, having something in that seven to 10 range makes a lot more sense. And having that two tiered structure over there was cumbersome, although that's not in here. So I'm going to talk a little bit about a recommendation on how that committee would look. And I would, I'm going to suggest that it's targeted about eight and that four members of that eight be members of the employee groups. So we have balance. And I've heard that loud and clear from the employee groups over in the house over here yesterday. And I think that's something that we should honor. And it's something that ultimately is very important. When we started our report, I worked primarily with the employee groups. And while we did not get to a conclusion and a consensus, it was good work. And I think that that was the first and foremost on our agenda. Madam Chair, very ethically, can Madam Treasurer, can you just say that again, four of eight? I just, I was trying to make sure I understand exactly what you're saying. So I would recommend a board that has the following structure. I think it should have four members of the, of the, of the employee groups, one from the VSA, one from the VTA, the Troopers Association. So that would mean two out of visas. And I think that that, and then two out of the NEA so that you, you're representing two from each system. And you're recognizing in the, in the state system that you have two bodies. So I think you would have four at that point. I would recommend, you know, we had two legislators the last time that we, that we kind of stay in that kind of framework. There were seven people on that committee. And again, it had a very different focus, but it was manageable. And it did not, by the way, have employee membership. That was a complete mistake. And that we don't want to repeat that. We had Terry McCague and, and again, your, your, your, your chair here, Madam Chair on the committee. And I think that was successful. And again, since I believe that ownership begins with the legislature, I would suggest two co-chairs, which I believe you already have, and the Senate and the House, and that they should be charged with getting to the finish line and that they should be appointed by the speaker in the Senate pro tem. And again, that creates a, a, a different level, I think, to, to where the 15 member committee is. And then I would recommend the, that the administration be, be represented in this, I think that as we move forward, that we're going to need them to be in this in terms of buy-in. Now, I would also recommend that you talk to Suzanne Young about this, as well as others. It could be, it could be Commissioner Petschek. I think he's great. You know, I, I, I can't tell you the number of interactions we've had on healthcare primarily, not so much pensions, but healthcare have been excellent. And what he's done in terms of COVID, this is a very smart guy. It's possible that the, the secretary would appoint him as her designate, but I think it's important to have someone at the secretary level being recognized as, as, as a member of the, of the task force. And it brings in the administration. I would, you know, I do have not had time to vet these ideas with either the employee groups or the, or the secretary. I briefly mentioned it to her. I've had conversations about other pieces, but not this, you know, about with the employee groups. It, I actually came up with this particular structure at two o'clock in the morning, last night, and I decided it wasn't a good time to call, call Madam Chair or any of you members. And I don't think Jeff or Mike or, or Steve would appreciate that call as well. But I think that, again, having that balance is very important. You would have the chairs, the co chairs, recognizing that they are the ultimate owner of the solution, having the administration and the treasurer's office involved and having an equal number of employee representatives. So you have balance in the process. Now I will tell you that on the 15 task force committee process, I, I said that if that was where we were, I don't think I would like to be excused from that because I did not think it was a productive way to go forward. It did have our director of retirement on it from a practical purpose. That's July is when we essentially process 500 retirees with four counselors. And, and, and two folks that are doing the administration and folks that are answering questions at the front desk, I believe we have 14 people in the retirement office and they're dealing with multiple people, not just the 500, but the others that are asking for information. Practically speaking, she would not be available. She could certainly testify on benefits because I, when we talk, you, you folks are frequently asking about that. She'd be happy to testify. She's very, very smart. But I think that that having a designee from the treasurer would be a better approach. And I would probably pick, I don't know if he's listening, Michael Kloss in our deputy treasurer for two reasons. One, he's familiar with the issues on, on that, that I've advocated as is Erica, but he's also was previously the deputy retirement, I mean, excuse me, the director of retirement. So he has a good feel for the benefit structure as well. So I would, that's the structure that I would recommend. It's lean, but certainly not mean it's balanced. And I believe that it brings in the expertise and you will have witnesses. I mean, I think that that's extraordinarily important as we move forward. The timeframe is, is something I'm very concerned about. I don't believe it's workable. It's, especially with a 15 member committee, but even with the eight person committee that I would recommend, it does not get you to a point where you have that decision making. And as I, as I did, meet with a thousand teachers, meet with 350 employees, meet with a hundred people on their council. I think it's extraordinarily important that you give time to do that. I haven't thought about the interim report. That was a great idea. Maybe there's some interim preliminary, maybe not a whole report, but some interim preliminary discussions about ideas with those groups. I think that it's important to have that, that discussion. As much as I appreciate the, the public hearings that the house did convene, I don't think they gave a full opportunity for, for employees to participate in the process. And I would advocate that, that we do that. And there was one employee that did some investment numbers. Tom and I talked about that. And Tom, Tom kind of verified his numbers. And I hope he's a math teacher because he did an extraordinarily good job in, in looking at those numbers and the opportunity costs that we lost back in the, in the nineties when investment rates was inflation rates were really high. And when you look at investments, it's a combination of your real rate of return and inflation. And we lost a lot of opportunity there getting to, to Jeff's point about coming into the recession at a lower level. It did not impact the 604 million, that total increase. That was a, a separate issue that said coming in at a lower level, you know, into the great recession and coming out at the other end from a funding perspective does put more pressure on the teaching system. So again, I would recommend a, a different timeframe. We talked about November, December, I would recommend that it be someplace in late, late November, early December. And the reason for that is timing with the actuararial data. So the actuaries will produce a report in October, based on the current act, actuarial assumptions that we have prior to any changes. And that'll give you a good measure of where we are. That will be on November 1st, October 31st. We generally have the meetings in that third or fourth week of October. And by statute, we're required to submit the resulting ADAC rep recommendation of the boards of trustees and submit those to the governor and the general assembly. So that would give us an opportunity to do that. It doesn't preclude you from having that impact on the 2020, 2023 budget. And Senator White pointed out that you've got a very long timeframe, well, not a long timeframe, hopefully you're not here to June, July, September, but in your legislative session, Senator, but that gives you some time to make some decisions.  They wouldn't do a full experience study, but they would calculate the changes there. The boards would then have to adopt those changes and assumptions that are a consequence of the benefit changes you would make. Then recalculate an ADAC, approve it, and then put that into the mix where appropriation purposes for 2023. And I'll get to the benefits issue, Jeff, in just a minute. But that's the process we used in 2009. Essentially, the boards went back to the drawing board, took a look at the changes that were made. The buck consultants at the time made those changes and it worked from a timeframe. That does not preclude delaying some of the implementation, and I agree wholeheartedly with folks that people need time to make their decisions. You can't put something in place on April 30th or May 5th or May 12th or whatever it might be and say, by the way, it's effective on July 1st. That does not give employees a time to look to assess what it means to them, what their options are to perhaps talk to our retirement staff about that. So people need time to work that out. And our director of retirement would say that from our software going through and making the changes there, verifying that, that would be a practical problem. So that's a practical problem. The principal one is people need time. This is a decision that's very, it's life changing when you make a decision to retire. And we see that already when people come in, as much as they're looking forward to retirement, they're anxious too, you know, am I going to have enough money in retirement? Am I going to make the mistake in terms of a survivorship benefit or something and walking through those decisions? And we need to give people the time to do that. And they need to hear the message now that they will have time from the legislature as well as the treasurer's office. We've been very clear about that. We've said that on our website, you know, trying to get that message out. I think it would be very helpful if the General Assembly got that message out as well, that we will give people time. And I know that the employee groups will help us with that as well. So again, I think that the composition is an issue. The timeframe is an issue. I think that the scope or the charge needs to be adjusted to include that the charge that the legislature has again, I'll repeat it as the creator and the owner of the solution ultimately. But I do think that skin in the game should be there from the treasurer's office and the administration. And I get I would invite you to talk to Suzanne about that. I don't want to throw something out and not give people an opportunity to testify to that. I also would like to comment on some of the scope pieces and some of the things that were raised today. As I said, I took a lot of notes on this. The question of DC defined contribution plans. We have debated that at length over the years. I did a session in House Appropriations with, I won't say the gentleman's name. All you folks do know him, but that is a big spokesperson on the issues and defined contributions. He's agreed to debate with me any day I want to. There you go. And we actually did a full six or eight hour piece on retirement. So 101 on pensions and included another debate between this gentleman and myself. Many legislators, I remember Donna Swaney, the previous House GovOps was there, Mitzi, Johnson, the speaker. I'm not the best person at scheduling. I scheduled it the Friday before Labor Day. So I really know how to impact people's schedules. Jefferson, you're crazy. And I was, but we still nonetheless had great attendance. And we had the same question at a business round table meeting. And I've done this in PR with them. And I've done this in multiple, multiple presentations to the GovOps Senate House Appropriations, the money committees across the board. Over the years, I've got more PowerPoints on this issue. Beth, I'm not gonna, I don't mean to interrupt you here, but I think you're going into a little too much detail about things. I think that what we need to hear is on these points, cross-off number F, because we don't need to look at defined contributions again. Sure. Okay, that's, we don't, we have heard the presentation. So I would just ask you to to kind of focus on your suggestions for the charges with, because, because we have some limited time and we really need to get to committee discussion. Sure. Okay. So number one, change the charge. Number two, I think there's a need to make that committee smaller and still have representation, a balanced reputation representation. Number three, change the timeframe. And I think that those are very important. Leave the rest of the scope alone, but I think you have some issues there. I would leave the goal there, you know, in terms of fiscal year 22. The last piece that I would talk about is administrative support. Right now it says that the, the Treasures Office would provide the technical administrative and legal support. Legal we can't do directly. We have one staff person that's the attorney, that's actually an employee of the attorney general. The attorney general may be in a position where he has to opine on, on what you'd folks do and hopefully not but possibly litigation. We did do a contract, an RFP process, a procurement process. We'd be happy to do that. We can provide some of the technical. I would recommend that joint fiscal has an individual who has some actuarial experience that, that Chris be involved in this as well. So I'd like to share some of that with the, with the joint fiscal office. And as far as administration, you know, taking the minutes, warning the meetings, the agenda, I would strongly ask you to, to give that to, to the legislature and one of your bodies because this is a time period that we just don't have the person power to get that done. So I think that's covered our points. I would be happy to answer any questions. I, I do have one comment. Our pro tem doesn't appoint anybody. It's the committee committee. Thank you very much. Good technical point. Thank you very much. Any questions for Beth? All right. Yes, Senator Rom. Well, I mean, I don't know if this is a general question for sort of all who have spoken, but is anybody, does anybody want to speak up for this part of the charge of this committee looking at defund being to look at defined contribution? I mean, it said, it's going to say, including anything else they want to look at. So if the task force decides they want to look at defined contributions, they're willing to do that. I don't know that we need to put it in here. Right. I, I hope it doesn't seem like I'm advocating for putting it in there. My understanding is there's language in there. Yeah, there is. And so I'm wondering if anybody today wants to defend the specific inclusion of that before we move on. I thought that's what the chair had said. No, I'm suggesting, I suggested that we take it out because they, they will be able to look at it whether it's in there or not. And by putting it in there, it sends them in a very specific direction. Right. I agree. And this is my speak now or forever hold your peace. Like, is this important to somebody? No. Okay. So what I'd like to do committee is look at a couple of things here. It seems to me that there are two issues that we can start to look at that are pretty clear timeline and makeup of the committee. So I'd like to start with timeline because that seems pretty simple to me. Yes, Senator Clarkson. Senator Clarkson. Yes. And you know what I'm going to ask you before we dive into that. Okay. Could we have a moment? I mean, could we take a stretch and go to the loo? We've been sitting. Let's take a vote on it committee. Yes, we can. No, it just we don't build in and it would be great if we I know we I tend not to but okay, five minutes. Great work. Okay. Five minutes. Yeah, we'll be back in five minutes and


And his continued efforts to ensure adequate and comfortable accommodation for personnel of the Nigerian Army to enable them to discharge their statutory duties efficiently and effectively. The Chief of Army Staff, Lieutenant General Farouk Yahya commissioned some infrastructure projects in legal state. The Army Boss said the Nigerian Army would continue its projects on barrack renovation and reconstruction, procurement of new platforms and rehabilitation of hospitals for effective medical care. Among the projects commissioned were the Finance Corps Soldiers Transit Accommodation, Ordinance School Students' Hostel, and the Seventy Command Supply and Transport Transit Officers Accommodation. This newly constructed Niger Army Finance Corps Soldiers Transit Accommodation, Ojuk. This accommodation is one of the most important ingredients of good administration which is a cardinal pillar of my common philosophy. You will therefore understand how pleased I am to be here today to commission this new accommodation for my soldiers. I have clearly followed the activities and performance of the Chief of Army Accounts and Budget Army, Major General A.A. and Chief of Army, in his area of co-competency and other assigned tasks and room command and tool and his staff last year, and I am in no doubt of his capabilities. This piece we have seen here is one of many. Therefore, let me commend you co-commander and Chief of Army and Budget. We are looking at your responsibilities which include improving welfare of your personnel.