Good morning. Good morning, Craig. Good morning all. This is the sub regional technical advisory committee meeting. It looks like we have a forum. Do we want to wait another minute or two? Craig or Mike, put your thoughts. I don't think so. I think we can get started. Okay. I'm going to promote Ryan Crawford, who's representing Sebastian from just having a little difficulty. Okay, I'll give you a little. We'll just wait a minute and see if that can happen. How are you, Jennifer? I am good. How are you all? I think we're good. It's been a good year for flood protection so far. This calendar year, I should say. Not so good for water supply. No, not so good for water supply and good for the treatment plan, though, in terms of not having to discharge. So depends on what you want to have on. I know. Well, that's the thing about our business is that there's never, there's very few good years for everything. What are, do you know, or maybe Emma might know, what, what are flows into the plant right now? Emma would know. We're looking about 16 MGD. That's kind of a summer flow, isn't it? I think we're at 19 for the average overall of December. Okay. Yeah, it's pretty low right now. All right. It looks like we have Ryan Crawford. Good morning, everybody. How are you doing? Good morning. Ryan, I don't know, before we start cleaning, did you want to just introduce yourself? Sure. I'm a senior hydrogeologist with GHD. I've been there for 15 years, originally with the Windsor and Kelly days before that, starting up in Eureka and down here in Santa Rosa from Sonoma County, originally out in Occidental. And that's where I ended up coming back to. So I do a lot of wastewater, groundwater, mostly groundwater work, but all things water integrated water management. Feel free to discuss anything, but this is my first go in one of these meetings. So just give it a speed on what you guys are doing. So thank you so much. And you'll be representing Sebastopol. That's right. Okay. And if you're able to, it would be great if you could turn your camera on. Do the rest of the tech members, do you want to just introduce yourself since we have a new sub-tech member? Can start with Katadi. Ryan, this is Craig Scott, City Katadi. I also worked with Windsor and Kelly back in 2005. Mary Grace. Yeah, I'm in it as well. Excellent. I did, I did find in our kitchen closet here. It was interesting, a couple of Windsor and Kelly champagne glasses. Coming out of the closet. I love that. Just deep in the closet. One's going to tell you that's great. Thank you. Okay, those were different days back then. So yeah, so I'm with Public Works. Thanks, Craig. And I'm Mary Grace. Hi, Ryan. I'm the city engineer and director of development services for the city of Rona Park and another Windsor and Kelly GHD alumna. So, you know, like, watch out, you're on your way to getting hired by some public agency. I'm Mary Grace. How you doing? Good. Well, thank you. Didn't get to say hi to Emma earlier either. All right, and South Park. Yeah, hi. Hi, I'm Mike Thompson with the Sonoma County Water Agency. I have to confess I never worked for GHD or Windsor and Kelly, but my son does now. Evan. I put him to work and all kinds of stuff, so. Well, he's like right there. I can go say hi to him right now. But you really don't want to see him. He doesn't go on camera first thing in the morning. So, but anyways, I'm with the Sonoma County Water Agency. So, we managed the South Park County Sanitation District along with Occidental County Sanitation District from sounds like we're your neck of the woods. And I've been there for coming up on 27 years and assistant general manager over the maintenance division, but also do pretty much anything that's asked to me there, which is, I think probably anybody who works in Mary Grace or Craig and probably, as we all know, we, we get into things you'd never think we would have gotten into and when you work in the public sector. So, why are engineers doing this. Okay. Sure. So, Michael, if you mind, were you part of the tie in the emergency tie into the Greaton facility for Occidental in the last few years I worked on a bunch of that stuff. I'm, I am, I'm like this deep in it. Yeah, actually, I'm kind of the, I guess the, it's the mad scientist who's working on that. So you're, you were working on the, I guess the truck fill. Yeah, some of the geotech and pipeline work to get it up there at 116 and then down the road. Yeah. I've been working with that. Jose and I were been on about three or four hours of call with Jose Artisa last two days. So we're, we're, we are, we're, they're finishing up feasibility study on that and it's looking really promising. I worked all the financials and where Occidental becomes self supporting with a little money left over and Greaton could get an extra 25, 30% of net revenue a year. So it's, it's, yeah, so the next thing to see is if we can find, I was going to come to Jennifer just saying, Hey, do you have an extra six and a half million dollars laying around for this sweet little pipeline that someday I did to the Santa Rosa system. I'm just throwing that out. Not because there's any plans for it but anyways, the, our next step is we're looking for funding. Looking for cats, maybe pretty cats that will help us get that but luckily I'm sorry, last little thing on that it. It looks like there's there actually is grant funding available for it so we're fingers are crossed. Thank you. We better do real business now. Yes, thank you, Mike, and I've been a pervert I am the director of the Santa Rosa water department and also the chair of the sub regional tax. I just want to remind everyone we are a brown act body we have been live streaming since the start of this meeting. Yeah, so, if you could please mute your microphones when you are not speaking, I'm going to go ahead and call the meeting to order and I'm going to ask the secretary to do a roll call please. City of Santa Rosa. Jennifer Burke here. The city of Katari. Great Scott here. City owner park. Mary Grace possum here. So you suggest Paul, right Crawford here. Sonoma water. Mike Thompson here. All right, thank you. It's nice to have a full. Full board for a change. So, let's see, we are now going to go to item number two. Item number two is adopting a resolution finding the proclaimed state of emergency and that meeting in person would present imminent risks to the health or safety of attendees and authorizing meetings by teleconference of legislative bodies. We are still dealing with a health order that is encouraging shelter in place where possible, encouraging folks to remotely work where possible, due to the search that we've seen. So, at this time we do believe it's still appropriate for us to be meeting in this manner. So you have a resolution before you if there's any questions or comments. We have a motion. Oh, well, before we do that, I'm just going to open it up for public comment real quick. We are now taking public comment on item 2.1. If you wish to make a comment, please raise your hand. If you're dialing via telephone, please dial star nine to raise your hand. And secretary, do we have any live recorded or voicemail public comments. We have no public comments. Oh, there's a motion. I'll move the resolution. Okay. Great Scott second. So we have a motion by runner park in a second by Katadi. If we could please have a roll call vote. The city of Santa Rosa. Yes. The city of Katadi. Yes. City of our park. Yes. City of sabbatical. Yes. No more water. Yes. That motion passes unanimously. Thank you. We are now on to item three 3.1 approval of minutes before we accept the minutes. Are there any questions or comments by the sub tech. All right. So we'll now open it up for public comment. We'd like to make a comment on the minutes, please.  Secretary, do we have any public comment. We do not. All right. So hearing no objection, we will go ahead and accept those minutes. We're on to public comment. Item number four. This is the time for any comments by the public for items that are not on the agenda. So we're now taking public comments on item four. You wish to make a public comment. Please raise your hand if you're on zoom or dial star nine if you're on telephone. Secretary, do we have any public comments for this item? We do not. Okay. Great. Thank you. So we'll move on to the new business and the first item will be item 5.1, which is approval of the 2022 meeting schedule. I think you all received that hopefully. So it has our meetings for the year. Any questions or comments by the committee. All right, seeing none. We'll now open it up for public comment on item 5.1. If you would like to make a comment by a zoom, please raise your hand. If you would like to make a comment by a telephone, please dial star nine to raise your hand. Do we have any public comments for this item? No, we don't have any public comments. Okay. Then we will. Consider the meeting schedule accepted. And we will now move on to item 5.2. So item 5.2. We're going to do a presentation on our virtual water reuse tour. Something we're very excited about and been working on for a while. Elise Miller, our communications coordinator will be making this presentation. Good morning. Assuming you can see my presentation. Thank you. Thank you. Well, it's my pleasure to be here today to share a new digital education tool that we've recently launched for the regional water reuse system. Now, the region has a long history of water reuse. And in fact, a few years ago, we celebrated the 50th anniversary of the Laguna treatment plant. And since coming online in 1968, we've been using the regional water reuse program, producing billions of gallons of recycled water and thousands of tons of biosolids each year that are beneficially reused. And each year pre COVID. The plant welcomed visitors from around the world, students and interested individuals of all ages. But due to the pandemic, we currently have our in person tour program on hold. But we do plan to restart that soon. In the meantime, the team produced a virtual tour video that provides an online learning opportunity and can be used to enhance our in person tour experience once we resume. So what we created was a 10 minute video. And as you can imagine, creating a video like this requires a bit of planning and coordination to impactfully tell our story. And this included building and editing scripts. We used the three main steps of filming at various locations with a variety of our staff. And we used high quality video and drone footage to try and capture the magnitude of the service that we provide to show the people that work around the clock to operate the system. And to showcase the value of water and the importance of water reuse. So in December, we launched this new outreach tool to the community. This video has been produced in both English and in Spanish. And both versions have been shared using the city's website through city connections newsletter that reaches over 100,000 subscribers, our YouTube channel and the city's social media accounts. And to date we've had over a thousand views of this video. So what I would like to do is just show you a short little segment of that video. So bear with me while I pull this up. This video produces a remarkable 7 billion gallons of recycled water per year.  The maturity of our recycled water is pumped 41 miles to a geothermal energy production facility called the geysers. Hi in the myocomus mountains. There it's pumped underground to recharge steam fields which are used to generate electricity for nearly 100,000 homes in the North Bay. By recharging and replenishing the underground steam fields, our recycled water helps to support the generation of clean renewable electricity. Approximately one third of our recycled water is used for agricultural or urban irrigation. So if the plant produces more water than can be reused, the extra recycled water is released to the Laguna de Santa Rosa and returned to the Russian river. The solid component of sewage is processed and turned into nutrient rich biosolids. Each year over 30,000 times are used as fertilizer and soil amendment for the city of Santa Rosa's farms and other agricultural lands. So it's just a little sneak peek of it. Let me see if I can get back to my presentation. So like I said, this was a 10 minute video and we've broken it up into three sections so that we can easily share it on our social media accounts and through our website. And all the videos are available online at srcity.org forward slash water reuse. So I hope you enjoy the preview that you're interested in watching the full 10 minutes of it. And I'm more than happy to answer any questions if you have them at this time. Thank you, Elise. Are there any questions or comments from the sub-tech? Yeah, just real quick. Thanks for putting this together, Elise. I think this is, I think this will be good. Not just during COVID, but at any time. So I look forward to seeing that. Thank you. Thank you. It's good to, is it, I'm sorry, was it Denise Chadman? Denise Cadman. Cadman, I'm sorry. Yeah, it was, it was great. I didn't know she's still working and it's great to see her. Out there, but I know she's always had a lot of, a lot of passion and a lot of dedication for her job there. So anyways, it's great. It looks great. And I look forward to seeing the whole thing. Thank you. And Craig looks like you have your hand up. Yeah. Thank you for the video. Looks really well done. Very professional. I just had a question. Do you, does the city of San Rosa have any objections to us posting a link on our city website? Okay. We'd absolutely love it if you share it. This is for all the partners to use. You know, this is something we've been wanting to do for a while and COVID really motivated us to get it done. And then it's really meant to enhance our tour program, especially as we get back, because it's showing you things you wouldn't, may not normally see on an in-person tour. And so we wanted to get that, that bigger picture. So please share what love for people to see this product as much as possible. And to that point, at least if you could get the Lincoln information out to Roberta and we'll send that out to the sub-tech. Thank you. Thank you. Any other questions or comments? All right. From the sub-tech. I will just note, Mike. Yes, it's lovely to see Denise. It's a lasting legacy. She has since retired. So she is enjoying retirement life. All right. So I will now open this up to the public to see if there are any public comments on item 5.1. If you wish to make a comment by a zoom, please raise your hand. If you're dialing in by a telephone, dial star nine. And Secretary Aitha, do we have any public comments on this item? We have no public comments. All right. Thank you. All right. Thank you very much, Elise. We are now moving on to item 5.3. And this will be a presentation on the start of our fiscal year, 2022-23 budget process and timing. And we will now move on to item 5.3. And Kimberly Zanino, our deputy director of administration will be presenting. Kimberly. Good morning, everybody. Let me share my screen. All right. Can you all see that? All right. So I don't have a lot of information for you yet just because we are just starting our budgets. We are just starting our budgets for each division, but I put together as much as I could for you today, just to give you a little bit of information about what will be coming for the next meeting when we're together. So I will go over the preliminary increases that we are expecting. The annual flows, cause I do have that. I have completed the miscellaneous revenues. So I can show you what those will be. And then we'll just talk about the budget schedule. And then we'll move on to item 5. And then we'll move on to item 6. First of all, we do have some increases that we know are going to be coming our way this year. We're going to have a little bit of labor increases because we are having salary increases that were just approved. We did not have any increase in July of 2021, because most of our bargaining units had open MOUs. And so we needed to negotiate. We did not have any increase in September. So in September, the city gave an increase by 3% for this fiscal year that we're in. And then they also agreed because of all the delays and how long it took to get negotiations completed. To implement the next year's increase early. So we'll have the 3%, we had 3% in September, and then we'll have 2.5% in April of 2022, but then there will not be one. In July, which would be the typical timeframe for increases to go in place. We do know, sure everybody's aware that the cost of everything seems to be going up right now, but we have seen a significant increase in the cost of chemicals. So we expect to see an increase there. I think in our, in your last meeting, which I'm sorry, it was not at Emma talked about some of the projects that have been going on out at the plant. There have been a lot of emergency. So we expect to see a little bit of an increase in operations and maintenance projects. And then also we did. A restructure. We moved people around last year. With that change. It was discovered that we left the plant a little bit short. So we need to bring a new wastewater operator on. That is scheduled to go before our city council. In a couple of weeks. So it will be a mid-year. If it gets approved. And then also we are looking to add with the new fiscal year, a supervising lab analyst. This position is needed due to new regulatory requirements and just keeping up with those. And I think Sean is here. If there's any questions about that position. This is just showing you your annual, the annual flows and how the allocation will look like. This is just showing you your annual, the annual flows and how the allocation will be spread out this year. Pretty much everybody stayed fairly stable till last year. So nothing. Nothing earth shattering. Very, very consistent year over year. As I said, I did already complete the miscellaneous revenues. This is good news. We actually are seeing an increase in miscellaneous revenues. So this will be nice. This will come off of our total O and M allocation. The biggest increase is coming from the high strength program. We are seeing a significant increase in that program. So that we're going to see those tipping fees. Increased by quite a bit. And then just as a reminder, I'm showing you what the allocations were last year. Obviously the big increase came from the new debt service for the UV project. I also provided for you what each of the partners who used fund balance, how much they used. Last year for the allocations. Here's a picture of the refund reserves. You're going to see there is a big drop in the total dollar amount in there, but just so that you're aware it's only, it only reduced because Santa Rosa was going to use some of their funding to cash fund the UV project. And when we decided not to, when we decided not to use our funding, we were going to use our funding. We were going to use our funding. And the funding was so reasonable when we went through that. We just moved that money back to our wastewater fund, which is where all of our funding comes from. And so we just moved that money out. So that's why you see a big decrease there. And then finally, this is our schedule. So we will see you again on March 7th and April 7th. We will, I will be meeting with the budget. Subcommittee for the BPU. We will be meeting with the city council. We will be meeting with the city council. We will be meeting at BPU on April 21st to get the preliminary budget recommendation, which will include the preliminary approval of the sub regional budget. And then city council, we will be on April 26th. And then our budget final approval work with the city council, the study sessions will be in May and then adoption of the budget on June 21st. And with that, we'll see you on April 16th. And then the last item that we want to present is a sub pack of. Thank you. Kimberly. Are there any questions from the sub pack I see. Craig. Go ahead. Just timing. When, if the city, if the agencies. Want to present to their. Governing bodies. What would the timing be for that? Where we'd be looking at April. An April meeting. Yes. Okay. It should all, we'll be, So we had to go through all of the processes. And we don't have weeks that happened at that point, but water gets their budgets done fairly early because we have to go through all these other processes. So it would be pretty finalized. If, if, if the city's had a, um, had input at that point. Would it be too late? Then or. We can still make adjustments. Um, but it would be a problem for city councils, Now we have to have things submitted so early in order to get on their calendar. So we could do something more preliminary, possibly in March. It just wouldn't be up, you know, the finalized numbers that would be going to city council, maybe. We can talk about that though. If you want to. We will let you know as we get closer. Okay. So we're going to have a meeting on the, the April sub-tech meeting is where we're looking for a recommendation from the sub-tech. Got it. So it would have to be a March. Council meeting. Okay. If you, if you could have, keep that in mind and have some materials ready for us to present. We would need for the potati. We would need something. The first. First or second week in March. I have, you know, multiple funds that I work on, but this one always is the first one I work on. Just because it has to, it has so many approval processes. It has to go through. I have to get, I have to get a jump on this one. So. Thanks. That's great. Mike. Yeah. Thanks, Kimberly. Just a few questions. One. And they may be questions for Emma, but first up, one with regard, what do you attribute the tipping fee increase to? Is it, um, that's just my question. Um, actually that's under Sean's group. Okay. We've just seen an increase. I think, um, Sean can speak to that. But yeah, we've had a lot of, a lot of new. New folks and a lot more tipping going on. Yeah. So, um, you know, we've had one industry moved into town and to runner park. Um, and we're handling a lot of their waste, um, uh, through the truck to waste program.  So we saw some increase there. Um, you know, just takes a little bit of rain and then everybody's got extra water to get rid of. Um, so, uh, those, those are the two things that we're seeing and we're also, um, I'm not prepared to, I didn't present on this yet because it's kind of preliminary, but I did mention earlier that we are doing an evaluation of our truck waste program, uh, to make sure that it's optimized for our community's needs. Um, and ensure that the, uh, tipping fees that we are accepting are covering the costs, um, of the program. Um, so we have enough time in the program to, to really be digging into those, um, details. And being open for whatever other opportunities are out there. Great. Um, thanks, Sean. And then, um, let's see the next question was the, you indicated that plant had a lot of, a lot of emergencies and there's going to be more equipment and more equipment. Could you maybe shed a little light on what the emergencies were? Um, I can take that question. Um, I, uh, that's a great question, Mike. Um, and on the agenda later is a regional operations update item. Um, and I'll be going over the activities that we've been undertaking. Okay. Thanks. Then the last question, Kimberly was on, you had a chart showing the change in the reserve. Was that a net change in reserve or ending fund balance or ending balance of reserves? That's ending fund balance as of last fiscal year. So there were, I thought I saw a negative negatives in there. So there were negative fund balances. Yes. There are, there is, there is a negative fund balance in there. So I mean, really it's a running total. So we've talked about the reserve before it's kind of a complicated piece of our budget, but. Um, as far as our auditors in the finance department goes, they just care about the bottom line of the fund balance, right? So it's just as long as that's in a positive status, it can't go into a negative status. They keep track manually of each different partners amount that is in the fund balance and it's a running total. So what we do is we obviously set our budget based on previous funds. So we set our budget based on previous funds. And then we set our budget based on previous funds. And then we set our budget based on previous flows. And then they true that up with the actual flows. And then they distribute whatever turn back there is based on that. So there is the ability to go below. If your flows are higher. Then, then what you're, what you were budgeted by. So that's the mechanism that gets it. Into a negative. And then I know you had mentioned this, but. I don't know. I don't know. Did I read that right? No, that was just here. You want me to put that back up? Yeah. I just, I just, I didn't understand. Let me get to it. So that, that 12 million there. That's what we had sitting in the funds last year. So Santa Rosa pays their portion of the sub regional budget. We pay for it with wastewater rates. So what we used to do is we used to always just move that to the waste water fund at the end of the year. We had not done that for several years because we thought we were going to put some cash towards the UV project. And when we decided not to put cash towards the UV project, we took that whole 12 million, 12.8 million, and we moved it back to the wastewater fund. So it doesn't, it wasn't a reflection of there being more expenditures or less turn back. We just moved it out of the fund. And my question was, is this. Is it, that's what was my question. Is this show a net change in the reserve? Or is Santa Rosa really have a. So our overall reserve there is, you know, so that's what your orders look like. We're still in that positive. In the reserve, which is good, but that Santa Rosa. Has a negative 12 million reserve. So we have, so the, with the running total and the turn back from when this was true to have, we have 592. That's our total. So you had the total. Okay. Okay. So, yeah. Okay. So the, I'm sorry. So that, that, that was my question. So the bottom line is the ending fund balance. Each year just shows the net. The, the, the, the, the net change. Correct. Okay. Thanks. That was just my question. I didn't answer. Ask it correctly. Thanks. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you for the button. Any other questions from the sub-tact members. Our comments. Thank you. Kimberly. All right. Thank you. Nice to see everybody. It'll be nice to see everybody in person someday. Maybe as March. Yeah. Wouldn't that be nice. We might want to decide when we're going back to meeting in person for this body. So. We have a public comment on item 5.3. We wish to make a comment by a zoom. Please raise your hand. If you're dialing in by a telephone, please dial star nine to raise your hand. And secretary. Do we have any public comments for item 5.3? Not. All right. Thank you so much. Thank you, Kimberly. We will now move on to item 5.4, which is the presentation on our regional operations division committee. So we're going to be looking at the presentations. Emma Walton will be making this presentation. And Mike, you will hear about all the issues that we are having. So Emma, please take it away. Right. Everyone see my screen? Good afternoon, members of the tech and welcome, Ryan. I am also a former Windsor and Kelly GHD employee. So I'm going to provide a brief update today on, operational ongoing since last, since we met last. Oh, here we go. I will be covering our recycle water and the supply and availability, some of the operational activities that we've undertaken since October, as well as briefly touch on some upcoming projects that we have. So our recycle water production, as I had mentioned previously, for January was about 19 MGD, which is lower than average, but higher than what we saw last year. You can see there the historical flows from 1986. Just kind of give you a sense of the variability that we see here. Along the middle there in gray is our average production. Last year in the dotted line along the bottom was our lowest production in our historical records. And we are currently trending back towards that direction as well. Given the rains we experienced in October, our recycle water storage is higher than average. But you can see that it's starting to flat line given the lack of rain that we've been seeing. We anticipate this to continue. But we are in a much better situation with our recycle water supply than we were last year. We are already have already started looking at our anticipated deliveries for this year. For geysers, we're looking at 4.2 billion gallons, which is our average. Last year, given the extensive shutdown that we undertook for maintenance needs, we supplied geysers less than average. And we gave them for we sent 4 billion up the hill last year. Agricultural irrigation on average we supply about 1.7 billion gallons last year. If you recall, we only had 600 million available for ag irrigation. So that was a pretty big hit to our ag program. This year, we've already established allotments and we established a minimum allotment for allocation of 900 million gallons. We decided to establish allocations in January this year, one to allow water to be used for frost protection, and to kind of give our users a sense of what the minimum that we would provide for the season would be. As we move through the season, this number will only go up. So this will be the minimum allocation that we provide for the year. If we don't receive more rain, we might not be able to free up more water, but we'll at a minimum make 100 million available for our customers. Our urban users, all the contracts we have in place there, we have to allocate 500 million for urban use for a total of 5.6 billion that we're looking to allocate at a minimum. So now I'm going to talk some about some of the projects that have been going on out here. The October 24 storm brought a lot of water our way. We had a the picture on the left there shows Chris Ward, one of our mechanical techs working on a valve actuator. He we had a valve, a critical valve fail during that storm, a valve that allows water to move from our primary clarifiers into our aeration tanks. It failed closed. And water, we were starting to gain elevation in our primary tanks. And we risked overtopping our primary tanks. Luckily, our mechanics were able to respond very quickly, remove the actuators on this valve and manually operate this valve and get water flowing again. We also experienced some localized flooding, not too severely. But you can see there our parking lot is a little underwater. As a result of the October 24 storm, we also ended up going off spec. It just was more water coming into the plant than our capacity allows us to treat. And it's also more water than we were able to pump to the geysers and to brown farm, which is our off spec pond. So we did end up storing water in our metal lane complexes and both C pond and D pond, a very large amount of non spec water that we have since been returning to the plant for retreatment. So on the left is our crews constructing a pipeline so that we can pump water from D pond into C pond and get it all isolated into C pond, which we have already completed and done. And then the photo on the right there shows pumps being placed in C pond. So that we can pump water over the levees and back into a manhole that runs along a Lana Road. We are nearly complete with draining C pond and returning all of that water, which is quite a big effort given the amount of non spec water that we generated during that storm event. So in the next couple weeks, we will have completely retreated all of that water. The other big project that's been happening out here, you may remember last meeting, I showed you a video of leaking aeration piping. This is piping that provides air to our aeration tanks. It was severely leaking. And our plant cannot go more than a day without air being provided to our aeration tanks. So we got an emergency project together. That video was taken on September 28. We quickly got an emergency project together, got an engineer to do a temporary design to do bypass piping and got a contractor out to complete that work. The work was completed on November 30. So we now have this temporary above ground air piping that will provide air to our aeration tanks. And the underground piping has since been abandoned. We are in the middle of design for a replacement of this piping. And we will, we're going to replace it with permanent above ground piping. And we'll utilize this temporary bypass during construction as well. So it'll help facilitate the permanent construction of the above ground piping. Let's see. We had a Christmas present come early on December 24. We had a pump fail and break and had a very large polymer spill that our crews quickly responded to. Not only did our regional crews respond to this, our local sewer and water crews also came out to assist in the cleanup of this spill. A little after Christmas, we had other gifts. We had a large sinkhole appear over the failed aeration piping or the leaking aeration piping open up. The picture on the left doesn't really do it justice. This sinkhole is about 15 feet deep. You can see on the right after we pulled it back and got some shoring in. You can see kind of the magnitude of how big it is. We were very concerned that these conditions may exist over the entire length of the leaking aeration pipe. So we mobilized the CCTV crew to video the inside of the pipe to verify the integrity. And then we got, we brought out geotech engineers to do ground penetrating radar to see if we could identify additional voids. Luckily, nothing through those efforts, nothing was identified. We also did additional exploratory excavation to just verify that nothing else had opened up that we were unaware of. And the extent of the sinkhole really was just isolated to to this area. And since December, we have backfilled and repaved the area. January 8, we had a plug in digester number three, we were unable to get waste material out of digester three. Typically, this would not be an issue since we do have excess digester capacity. But we have a digester currently offline, because it was just cleaned. And we typically have a digester offline because we, on average, clean the digester every two years. So we have four digesters. And so we, on average, have a digester offline. And our crews were able to get a bypass line installed to keep material flowing, and to ensure that we were able to continue to make class B biosolids. Our mechanics and our onsite crews being able to do all this work has really prevented a lot of failures and a lot of us going off spec and keeping operations going. So it's really great to have such amazing crews. Just a couple weeks ago on January 22, we had a recycle water main break. It was a 12 inch line, the main line that feeds recycle water to the plant. As you can see, it was a very extensive impacts and flooded the area. We were able to isolate the line. But because we use recycle water to supply water to a lot of our processes, we had to go shut down equipment that utilizes recycle water prior to shutting down the line, so that we didn't further damage equipment. So more spilled than would have otherwise. Again, the response for this work was not only from our regional crews out here, but our local crews came out to assist as well. And ultimately, the break ended up being so extensive that we needed to bring in our on call contractor to make the repair. This repair has since been made and we are back in water. Those are kind of some of the big projects that have been going on out here. We do have some exciting upcoming projects. Our disinfection improvements, our UV project finally went out to bid on January 28. Very exciting and long waited event to occur. So it is currently out to bid we're getting RFIs in the process of responding to those requests for information. We are also scheduled to rehabilitate two of our belt presses here in February. So that work is going to be starting soon. We are have a project to replace the our flow equalization return pumps. So the parts for those pumps have been ordered and they're on their way and our internal maintenance crews will be doing that work. We have a project that was awarded to replace our emergency generator radiators. That product that work was stalled last year given the early rains we received. So we were unable to complete that work last fall. But we're going to be kicking off that work here soon. And many, many, many, many more projects that I could go on and on about. But at the next sub-tac, you'll be receiving a presentation from our engineering staff to talk about all the exciting big capital improvement projects that we have going on. So with that, I'd be happy to open it up for any questions. And if you're interested in what that is, then you can ask that too. All right, thank you, Emma. Yes, a sobering look at a 50 plus year old treatment plant. So and I just have to say staff is doing an amazing job keeping everything together and working through this. And you know, we're trying to be as proactive as we can. But we continue to have difficulties with having enough labor and resources to get projects done. So that's that's our biggest issue really. So any questions from the sub-tac? Yeah, I would say no questions. But you know, just thanks to your crew. I mean, I thought we were the only ones that had to go out on Christmas Eve to prepare emergencies. I'm glad we're we've got company here. Or I guess I'm sorry for you, we have company. But anyways, those can be really trying time. So how did your crew respond? Our crews are absolutely amazing. You know, the redundancy that our infrastructure should be supplying is just not there right now.  And they just do incredible work. And the fact that we have very highly skilled mechanics and highly skilled electricians and highly skilled instrumentation techs and operators has just really kept this plant going. It's wearing, obviously, with COVID and emergency after emergency. It's it's difficult. But I just can't speak highly enough about our crews. I guess I would say if you can pass, I guess on I'd look if I can say on behalf of our our tack here, can you pass on our appreciation to them? And I think we, you know, I'm sure Craig and Mary Grace, we've all been in that position where we're really at the mercy of our staff. When things go wrong, I mean, there are things that when, you know, I'm looking at your recycled water break, you know, I think most would look is like I have no idea what to do. And, and you have folks on your staff who just immediately jump in. And you know, they're the ones who go running into fires, then away from things. And I know we all can't, you know, can't thank them enough. So if they can on again, I say on all our behalf, just say thanks. I will definitely pass that along. Thank you. Any other questions or comments from the Subtack? Alright. So if there are no other questions from the Subtack, I will now open it up for public comments on item 5.4. If you wish to make a comment by zoom, please raise your hand. If you are calling in by a telephone, please dial star nine to raise your hand. And Secretary, do we have any public comment on this item? You've done public comments. Alright, thank you. And thank you, Emma. And thank you, Mike. I second the appreciation on behalf of the Subtack and just continue to always acknowledge how amazing the staff is out at our treatment plant, but in our department in general. So thank you. Alright, so we will now. Oh, yes, Ryan. Oh, I was gonna say, Emma, you didn't. I guess you didn't tell us really what that that I'm gonna give a quick guess on what that last picture was. Yeah, is that was that the pandemic phenomena, inorganic, wipey monsters that are hearing it all the treatment plants around the country? I'm sure there's some non flushable wipes in there. That is a rag ball a rag ball that was taken out of the bottom of our digester after cleaning. So Christian, our supervising wastewater operator had the pleasure of removing it. Because you see that picture again. I missed that little detail. So good eyes, Ryan. Let me see if I can make that work. Here we go. And he looks so happy doing it. Well, who wouldn't be? Oh my God. Wow. Okay, well, that's a that seems like for Elise Howard, something for the the video of like, why not put flushable wipes down the toilet? Yeah, it's true. Actually, there's a very interesting display at the treatment plant. So when we finally get back up and running of flushable wipes and toilet paper that have been sitting in water for months now, right? Since October. And there's just no breakdown of that flushable wipes. So all right. So we will now move on to thank you so much, Emma, we will now move on to item 5.5, which is a presentation on our environmental services division update and Sean McNeill, our deputy director of environmental services, will be presenting Sean. All right. Good morning, Chair Burke and members of the Subregional Technical Advisory Committee. I'm excited to be here and share with you what's been going on in our environmental services division. So I'm Sean McNeill, the deputy director of environmental services. So I just want to give a quick update on what's going on with the Laguna Environmental Lab. As Kimberly mentioned, we are adding one new position to aid in the adoption of these new requirements to comply with the NILAC Institute standards. These are the TNI standards that all environmental labs in the state of California need to comply with. In general, we think it's really good best practices that are included, but it's a lot more detailed tracking of which aliquots of various chemicals that we're using and data tracking. So and we need to have certain procedures in place. They gave us three years to update our procedures to meet these new requirements. We're a little over a year has passed. We have a consultant online and adding a new staff person is a part of that process to get us over the hurdle and then to keep us in compliance as we move forward. We did just have our first external audit from a third party assessor. In the past, the external audits were done by by the the state. And this year they've they said that they're no longer doing those and that we'd have to hire our own third party assessor. So we did that auditor came in that we had nine findings, which is fewer than average for the Bay Area and for laboratories of an equivalent side size. And many of those findings were relatively easy to address. Some of them were issues related to COVID that we were able to clean up. So we're moving forward with that. Also, our lab is also aging. And the instruments that we use in our lab are highly technical, and they utilize software and that software kind of gets dated sooner than the technology does. So eventually you get hit you get in a place where you have to have software to operate these instruments that you that are no longer safe to have on network. So we've had to do some update of our instruments. And we're methodically going through and changing those out. We had budgeted a little bit each year. So that it's not a huge hit to the budget to replace these instruments. So in October of 2021, we installed our quick trace low level mercury analyzer. And this will help us monitor mercury and drinking water, recycled water and our effluent, as well as solids. And then in January of 2022, we finished the installation of our new inductively coupled plasma mass spectrophotometer, also known as the ICP MS. And this conducts trace metals analyses, and can quantify the metals in parts for the trillion range. So it's very high high tech sophisticated instruments. And it's not just the purchase and installation of these instruments that we have to worry about. Actually, our staff needs to prove that they they can implement the EPA methods using these new instruments. And we're going through those special studies to ensure that they do it. So so they each get a little test to ensure that the instrument and the user of the instrument can detect known quantities in their samples. Also want to just kind of give an update on our safe medicine disposal program. This program was started in 2008 to prevent unused pharmaceuticals from being either flushed down the toilet and getting to our wastewater treatment plant, or disposed at the landfill where they make their way into the leachate and get to the wastewater treatment plant. So so we've had this program to do that. Senate bill 212 was passed in 2018. And that requires manufacturers or suppliers to establish a safe take back program. Yet that implementation has been delayed because of COVID. And some of the programs that they've proposed are a little underwhelming to the state. So they're looking at increasing that so that increasing the effectiveness of that so that before we take away our program, we want to make sure that the other programs will make it easy for people to get rid of their old expired and unused medications. So since 2008, our regional program operated from our pre treatment program has collected 93,589 pounds of pharmaceutical waste. In in the the whole watershed, we've removed 100, almost 148,000 pounds. And this includes the regional pro our regional program program run by Sonoma Water, City of Petaluma and the Russian River Watershed Association. So it's been a very effective program in getting those medicines out of people's cabinets and out of our waste system. And in the last year, we found here's the collections from each of our locations for our regional with Roner Park having collecting about 500, a little over 500 pounds, Katadi Police Department, over 100 pounds, Sebastopol Police Department, over 1000 pounds. And then the Santa Rosa Police Department, 1800 pounds for so for almost 3500 pounds of pharmaceuticals were removed from potentially getting into our local waterways. So moving on to our annual report, we're just about ready to file it. It'll be due about February, it's due February 20, 2022. Some highlights from this annual report is we had no discharge this year. We treated almost 6 billion gallons of wastewater here. And that was 100 percent reused. And we produced over 5000 dry metric tons of biosolids. And that's all at 100% reuse as well, almost 100% reuse as well. So our phosphorus compliance, just want to kind of give an update here. As you're probably, well, many of you are aware, but Ryan, you are new. So we have to quantify the amount of phosphorus in any of our discharges. And then we need to have a mechanism to offset those discharges. And so to do that, we've had in the past a nutrient offset project. We developed a number of projects throughout the region to remove phosphorus from the watersheds to meet our compliance. We're now in the process of migrating those credits from the nutrient offset project into the water quality trading framework. And we're also working with Sonoma Water, the town of Windsor and the Sonoma Resource Conservation District, to develop two creek restoration projects on different segments of Golden Creek to remove sediment, restore that area, and to get more credits so that we can bank enough credits for at least one very large discharge, or possibly two medium sized discharges. Because sometimes projects, it's now with the new rules in the water quality trading framework, these kind of credits have life of imperfectivity. So it's a good investment to start banking these credits, because sometimes projects are not easy to happen in one year. So having multiple years worth of discharge credits is a good way to ensure our compliance with this regulation. And with that, I'll take any questions. All right, thank you, Sean. Are there any questions or comments from the sub-tech? One quick one if I can. And I know I'm sure the answer is it depends. But what it do, do you have a sense of what your reuse capacity is? I know it depends on the water year, it would depend on when the water comes and all that. But in general, do you have a like, is there a sense of how it looks like, you know, obviously 100% reuse last year. Do you know if your reuse capacity is probably like seven or eight billion gallons a year, would you say? Or I'm just just a good question. You know, I'm going to see Deputy Director Emma Walton can feel that one. Yes, I have those numbers.  I want to say it's 6.4 billion, but I can I'll verify that and send you that information. Yeah, okay. It's not I was just curious, because just was curious how close you were to your capacity last year, your reuse capacity. You were able to do 100%. And if you had a sense of how much more you could have done, if there was more water, I know it was just because there was such a water shortage. Yeah, so obviously, we did 100%. We could have done much more. And I, I want to say it's 6.4. But I will, I'll double check that. Okay. And obviously, it depends on when the water comes in. Of course. Yeah, of course, on average. Yeah, we're able to sustain. Yeah, and I'd say that just also want to thank Sean for working with us at Sonoma Water on these Creek Restoration Projects. These are projects we wouldn't be doing otherwise to restore these creeks if we weren't for the credit trading project. So it's, it's I think there's, there's many wins here. So I just want to thank Sean and everyone that's at Centro as a water for working with us on that. Well, you guys, Sonoma Water makes it pretty easy to work with. So appreciate I want to appreciate you and your team as well. Thanks. One, one quick comment or question. Those are pretty impressive numbers of pharmaceuticals collected. I'm assuming this, that huge number from Sebastopol would be because they're the West County sort of hub. So they've include sort of the river and the west, west of there coming into Sebastopol to be collected. That kind of would make sense to me if that is what's happening there. And then I guess do you have a sense of how much is actually still really needing to be collected?  I've always wondered that. So thanks for showing that that number. It's just sort of a wow moment for me seeing imagining that much in our waste stream. I don't think I can answer intelligently about how much more there is out there. But I will say that, you know, we do look at whether or not we want to continue the program each year because they're supposed to there are other mechanisms now. I'm I'm a Kaiser member and they'll send you an envelope and then you can just put your medicine in there and mail it back to them. So there are more pharmacy options than used to exist back in 2008 when we started this program. But collecting 3,500 pounds in our our various police departments was is it's convenient. And as long as we can find convenient ways to make this the medicines not get to the flush down the toilet, which, you know, not even that long ago, EPA was advertising. That's what you should do. And then regulating that we shouldn't be allowing that in our areas. So, you know, there's a lot of mixed messages, people aren't very clear. So we want to make sure that there's an easy option getting rid of these pharmaceuticals because we do add our treatment plant except the leachate from the landfill. And so even if it goes to the landfill, it can make its way to our treatment plant. Yeah, I was just wondering if you had a sense of within the waste stream, if you were seeing a decrease overall in residual pharmaceuticals or emerging contaminants generally associated with pharmaceuticals or if that's still sort of an increasing trend like you see in other parts of the country. Yeah, so we in our priority pollutant testing list, we have not seen a change in that getting into our influence or effluent. So we haven't seen a change there. If I'm understanding your question, how much is coming into the treatment plant? But we don't, we do not, and there do not exist EPA methods for every single pharmaceutical. Yeah, it's more of just an organic range, you'll pick up, and you can assume with a lot of its pharmaceuticals, it's a tough one. I'm not trying to put you on the spot. It's just a really interesting subject. Thank you for talking about that. Sure. And we could also dig into the data a little bit more and see if we can find any trends. But we just, we mostly are non detect on almost all those priority pollutants that we have tested for. Thank you. Any other comments or questions from the subject? All right. Seeing none, then I will go ahead and open it up for public comment. We are now taking public comment on item 5.5. If you're on Zoom, please raise your hand to make a comment. If you're on the phone, please dial star nine to raise your hand. And Secretary, do we have any public comment on this item? We do not. All right. Well, thank you so much, Sean. I think that was a very good presentation. Thank you to all the presenters today. And Subtack members, as I mentioned, we'll we'll see if we're going to be virtual or in person. We'll see how this all plays out. And I do just want to offer to Ryan and frankly, to all the Subtack members, if you haven't been on a while, we are always happy to tour you around the treatment plant. So if you would like to have a tour, please let us know. So you can kind of see everything we're dealing with. So with that, unless any other comments, I believe we are adjourned. So thank you so much. And we'll see you in March. Thank you, everyone. Take care. Thank you. Take care.


So today I want to talk a bit about the different use ways in which data is actually important for your machine learning pipeline. Probably most of you are aware that data is super critical, like data quality. Preparing your data is super important to get proper output for your machine learning models and proper quality for your machine learning models. But on the other hand also just capturing metadata is very crucial to actually run and operate a machine learning pipeline in a production environment. So we're kind of going to talk about those different ways and another way at which you might see that we're going to talk about the way in which no SQL databases such as graph databases, document databases, full-text search engines are having an impact on machine learning and basically those two fields first of all for feature engineering. So how can I actually use it as part of my machine learning pipeline? So this is probably more appealing to data scientists and then on the other hand we're going to look at as the title promises like what does it take for machine learning infrastructure and here we're actually going to take a look at the metadata we might want to capture across our metadata across our pipeline. This is actually captured under an open-source initiative called Arango ML, Arango Machine Learning and we'll learn a bit more about the heritage about this project throughout this talk but basically it's an open-source project which actually allows me to do both of that, leverage no SQL databases for feature pre-processing or data pre-processing for machine learning and on the other side capture machine learning metadata across various pipelines you might want to build. I'm Jörg, I'm the head of engineering and machine learning over at Arango DB and previously I always have been switching back and forth between the database world and like machine learning infrastructure or large-scale infrastructure world so switching back and forth between databases in my PhD then I was over at SAP for HANA then I built large-scale machine learning frameworks over at Mesosphere and Suki AI which is like a healthcare start-up and what I kind of like about my current role is that it's combining both those passions so I can both work on the database side but also integrate and merge that together with the machine learning world. Alright maybe just why is machine learning taking off nowadays so we always imagine like hey just with TensorFlow everyone can actually become into superhero and program really cool models. This is mainly due to the fact so why is it the right time right now that we actually have large amounts of data. Secondly we got to computing capabilities available like large data centers, cloud environments where you can easily get large deployments and also recent advances especially in like deep neural networks from an algorithmic side. This actually allows us to train like self-driving cars. There's this like a deep Bach which actually is composing music which I find pretty indistinguishable from like real Bach music so it's pretty cool what we can do all with machine learning nowadays. So just kind of seeing that if you and I always hear that when people are starting with a new machine learning project all even in big companies it's like I just going to hire a data scientist. The data scientist is going to sit there leverage his laptop together with TensorFlow. I give him some data he's writing this cool model we need to train it obviously but then we're done and we have this cool model and this data scientist can actually approach the next big challenge. Unfortunately in reality this looks very different so this is from a paper by Google so those people actually have a lot of experience with machine learning but even for them the machine learning coding is actually just a small box in the middle and actually the real challenges of productionizing machine learning models so not just coming up with like one model but actually putting that in production is basically all those bigger boxes around the small black box here and so I believe what we should actually do as a community and I believe the metadata aspect is like one part of this is actually work together as a community and define like data science principles. Similar as we have like software engineering principles like CACD testing patterns and everything else I believe if we work together as a community we can actually structure this picture much better and work together whereas currently I actually see a lot of people reinventing that from scratch and trying to come up with their own way of doing such things. There actually other people like Ian Goodfellow like agreeing with that and I think this is something where we as community should really work together and hence move the AI or actually more the operability of AI forward with best practices we share and I guess the first best practice I see is as mentioned like people start hiring the super hero the data scientist but actually the super hero isn't very powerful if he doesn't have like side gigs which can actually help him manage this entire big map here because for a data scientist often coming from a background mathematical background dealing with large-scale distributed systems scaling computation often is like not their core strengths and this is why actually we need to work together as a team in solving this challenge and I guess most prominently what we've seen over the past probably two years there's been this new role it's kind of like the DevOps role for data science and this is basically like data ops data engineer there are different names out there but this is a person actually combining data science skills together with like distributed systems engineering so kind of a mix of computer science and mathematics and this person this persona can actually help to build this entire infrastructure operated and make sure we actually are efficiently implementing that so this is probably like the first takeaway I would like to give people really think about operating your platform and not just hire a data scientist and yeah if we actually divide this maps and between our different personas the data scientist can actually really focus on the data itself so feature extraction feature engineering writing machine learning models writing analysis on like how efficient different models are whereas the other two people like a more traditional system admin DevOps person can take a care of the infrastructure and our data engineer data ops can actually take care of the data aspects around our yeah machine learning platform so what typically comes out if we start such initiative we end up building a machine learning pipeline with more or less those different steps here so on the left side we got data basically how do we store how do we manage our incoming raw data and then comes the next step this is the feature engineering and this is actually where data scientists are probably spending most their time this involves data cleanup extracting core values out of the raw data basically transforming the raw data into something which we can actually leverage and input into a machine learning framework then comes the actual model training so there is what we actually expected there's a small black box here where we have something like tensor flow spark py torch mx net or whatever our favorite machine learning framework is and we basically train a bunch of different models so also to keep in mind we typically not only train like one model for a given problem we train up up to like in the range of tens to 50 different models for different hyperparameters for different ways to try out things and the next step is then typically I end up with a library of different models so also keep in mind this might actually improve over time I might get new input data I might get new training data I might get a newer model by my data scientists so actually I end up with like a large library of different models I've trained and the big task is then how do I select the one I actually want to use in production for serving so which one do I want to leverage to actually solve the real business problem I'm having on the right side in model serving so there multiple actually open source pipelines available probably the most prominent one is a tensor flow extended or short tfx which has been built around by Google around the tensor flow ecosystem and you can already see this pretty much matches to what we saw earlier we got data on the left side we got some transformations or feature engineering in the middle we got our training in the middle and then we got model management and model serving more going to the right side very similar and actually related in many aspects it's a cube flow pipelines so cube flow was originally a way to deploy tensor flow on kubernetes but it actually has grown into a very large ecosystem of different projects around machine learning model serving around feature stores so there we actually basically are trying to build a similar ecosystem as in tensor flow extended and they actually there's a lot of interaction between both communities as well also here we actually I think they have a boost here it's logical clocks hops it's also an open source platform pretty much again following the similar steps we got data ingest data preparation experiments and train and then deploy on the right side so we actually got a number of different open source options out here to actually build our pipeline and now as promised we want to talk about how no sequel databases kind of fit into this picture and I guess the kind of obvious answer is like on the data and streaming side taking care of our data storing out our data so probably not so interesting to actually go into much more detail here but what's actually more interesting is the feature engineering aspect as mentioned earlier this is actually where data engineers are taking most of time taking raw data and turning it into something tensor flow or something else can actually start doing something with what are the typical steps here first of all we have to actually clean up our data there might be missing data there might be different for example time or date formats different units for distances and so on and so on everything you learn when you live in the US to love and hate and the other thing is I might actually also have to transform it into something which can be understood by my machine learning framework so typically tensor flow and others they work with for example TF records which is basically like a tabular format of input data so if my data is actually a JSON document or a graph or a key value store I probably have to do something to turn it into something meaningful which can be understood by tensor flow and actually processed in an efficient way so for example if we take a graph we then turn that over into for example extracting the number of movies made by each director if we want to predict something like hey which director should we actually choose for our next blockbuster movie so we need to extract features from a graph which can be easily understood by humans but not so well yet by machines so we'll also see a bit there's research in that area later on but this would be for example a typical step in feature engineering talking about different data models why does it actually make sense to have different data models such as JSON graphs key value and I guess the most important aspect here is that we also have to think about how data is represented for us as humans and so for example it's very easy for us to think about graphs to think about different connections between entities rather than like a big table and it also often makes it much easier to express certain queries for example graph traversals finding out everyone who's connected with us finding out all the models we will see later on related to a data set this is an easy graph question but it can end up being becoming like a really annoying join if we want to express it in sequel and this is kind of the one thing we're working on with a Rango DB is that we have actually seen in the past their their specialized databases probably most of you you know Mongo Neo4j Redis for example which basically help us to solve like one of those data models here the challenge in real life is actually for most use cases that one data model isn't sufficient I typically I need kind of a mix of both of them and then I actually end up having my engineers write a lot of code basically merging data from my graph database with my document database and I often even end up re-implementing a lot of database functionality on top and that was the kind of motivation behind the Rango DB it's an open source multi-model database so it natively supports all those different data models inside one core engine it's distributed so graphs can actually spend multiple nodes and also for usability there's just like one sequel like language which actually allows us to query across all those data models so I have a uniform access to all of that I promise is the only a Rango DB slide so I'll also move forward to the next slide as mentioned this especially the topic of graphs and machine learning this is where the connection of no sequel databases or different data models and machine learning probably is most obvious so we just search like Google we find a lot of different things about like knowledge graphs and how we can connect deep learning and knowledge graphs or graphs in general and I think this is coming from this fact that for us as humans a lot of things can be easily expressed as a graph so if we go back to this movie example this is actually a Kaggle data set so Kaggle is like a platform for data scientists for like competitions and you can actually also learn a lot by just playing with those data sets so if we have this data set here and this is originally here in a tabular format but for us as humans if we want to think about those different entities so for example there's there are different records like for example a director is appearing like multiple times in here but if we as humans are thinking about we're actually thinking about the connection of a movie if you think of a given movie it's connected to a director and that director is actually its own instance which again can also be connected to other movies or even other actors by that because for example Tarantino is always choosing like his favorite actors so there might be also different connections in there so for us as humans it's very natural to think in like a graph like structure and what's also really interesting is basically how can we kind of like the next forward step where a lot of research is happening right now is how can we actually combine this world of deep neural nets together with graphs and this is the kind of area of graph neural networks and this can then also involve like how can we leverage machine learning to predict edges in a graph how can we predict machine learning to leverage labels so basically how can machine learning deep neural networks natively deal with graphs so I think this is very interesting because then we don't have to do this feature step in between where we actually need to manually extract features but we can leave it up to the machine learning framework to extract the valuable information from a network as mentioned we also wanted to talk about not just graphs but actually the other ways in which we can leverage data or which in which we need to leverage data as a data scientist and this is where is in different processing capabilities come in so we can't do for example graph queries we can do graph traversals but on the other hand what's super powerful even in just feature engineering is for example natural language processing going back to our movie data set imagine we want to find all the Batman and Robin movies so we want to issue a full text query and actually find all the different movies which concludes there and then we could for example leverage a graph query to find all the connected directors again for those movies another aspect about features as we talked about how we can derive features is feature reusability and keep in mind that it takes up to like 60 70% of the data scientists time to actually clean the data and develop those features so actually reusability is a very important concept and here is the idea of feature catalogs or feature stores has become quite prominent so feature store basically allows me to reuse features which a previous data scientist has worked on imagine I'm at Airbnb data scientist one is coming up with some kind of user representation some kind of transformation he uses to clean that data set to extract the feature describing a user and then I'm the next person the next data scientist who needs to work on a similar problem like predicting which what might be booked by several users then it's probably a good starting point at least to take that feature and not derive everything from scratch so good feature store and this is a screenshot for example from logical clocks again we're outside is I can actually discover features which have been worked on by other data scientists before I got versioning consistency and I can also have a caching so I don't have to compute that always from scratch which can also take a lot of time for large data sets okay let's actually move to the second part where or the third part actually where databases are super useful in our machine learning framework this is actually by managing this entire pipeline in one go and for that let's like one motivating example for me and that actually brought me to back to join a Rangudibi a while ago was I was working in building machine learning pipelines for finance and health care we often had problems like we needed an audit tray like how where did this model come from which data sets were used to train it but also the other way around like we were dealing with patient data and we needed to be able to identify which models were impacted by this one patient record for example because that patient was withdrawing his consent for us to use this data and then under GDPR it's a bit unclear whether if it's concerning the original data set it actually translates over to the model but there can always be information exposed for example here this is shown in this paper show out in the final model from the original data set so at least we wanted to be able to identify which models were impacted or trained by this like one single record and this actually turned out to be a pretty hard problem because each of those steps had its own metadata we knew where a data set was stored we knew when doing feature engineering like which of those stored data sets was used we knew then when doing model training which features had been used we knew then when storing a model like from which training rounded that come and then also on the model serving side we knew like hey we use this stored model but this actually turned out to be like us manually joining data from five to six different systems to actually figure out like hey those models were impacted and we might have to withdraw those models from our production environment and as mentioned like this actually is related to a number of other challenges so for example just understanding the provenance the lineage of a model is important for a lot of audit trails especially in finance this was like a very crucial thing and also just being able for a data scientist to see like hey which reusable steps were used there we already saw this concept about a feature I want to reuse things which other data scientists have done before so I also need to be able to discover that so just being a data scientist and saying like hey those this Jupiter notebook those features those data sets were used to train this is a very powerful tool and this is actually when we started into looking looking into building a common metadata store so as mentioned like there's often like individual metadata stores for each of those pipelines but we were at that time really missing this common layer which we could join across all of that and maybe just to get like a common term here let's just define what we mean by talking about metadata so metadata so this is the cube flow definition is actually just describing information about runs models data sets so it's not the data set itself but it's for example a description of this data set has this size has this name is stored here and has version xyz and thereby I can figure out that there's a newer one of that similarly with like training runs with training runs of course a really important metadata is like training performance test performance and also with like notebooks features and everything else so it's basically like a description of different entities but not the entity itself and this actually led us to build a Rango ML pipeline so a Rango ML pipeline is a common extensible metadata layer we basically have seen that it's not really good if we would build it for like one particular pipeline so for example just for tfx or just for cube flow simply because most people actually vary a little bit in how they implement their pipeline so we wanted to be both extendable and flexible and not being tied to like one particular model and so we ended up creating this open source project based on a Rango DB and why did we choose a Rango DB here so a Rango DB we talked a little bit about this idea of multimodal data and this actually turns out with machine learning metadata this is a really good use case for me also to explain multimodal why is it important and what we actually saw is when we started out we started using a document store and this was worked really nicely because the individual metadata items for each pipeline stage for each entity really nicely fit in like a document like model training different machine learning frameworks have different outputs so I got different metadata from like my TensorFlow framework from my MXNet framework so I could easily model that into like adjacent document and store that just turns out that a modeling the connection between different entities became a lot harder that way so we actually wanted to model the structure in between as graphs we wanted to say like this JSON document describing my experiment my training run from yesterday is related to this feature this feature is related to this JSON document describing a different transformation this is related to a different data set so basically we wanted to bring up this graph structure and this is why we then in the end ended up choosing a WranglerDB because it allowed us to be flexible here and leverage both the document store for storing this unstructured data but on the other hand also have a structured connection in between by leveraging a graph and using that actually queries like which models are being impacted by like this one data set actually turns out to be a simple graph traversal or reachability we start with one data set and then it's basically a graph query which models can be reached here and for that just keep in mind that it's not always this linear chain but a feature might actually also depend on a different feature or a model might actually be also trained on top of another model by using transfer learning so you actually also have a lot of loops there which makes like just writing like a long sequel statement for this if you would store that in a relational store pretty pretty difficult and yeah so basically just leveraging the graph structure a lot of those things we needed to answer also like audit trails for a given model turn out to be a simple graph query which I can easily express and this over time actually allowed for a number of different use cases for actually different personas so originally we were actually mostly targeting like the data ops engineer who needed to take care of like hey this is the audit trail this is the data lineage for GDPR because he had to answer those questions in our environment but it actually turned out that this metadata was pretty useful for the other personas as well so both for example our data scientists here he could easily find relevant entities for a model so we had like a new data scientist we onboarded and we told him like hey can you please look at this model and improve it and just by looking at the metadata you could get like the entire view of how that model has been trained in like one go whereas previously he would have to go through like all the different systems or ask the people who had implemented that to tell them like which dataset was used for example it also helps to like search whether someone already has done like a user feature and also it really helped them in the end to have like compare performance differences across different models because they can again issue like a simple graph query and then compare like training or test performance across various models depending on very flexible criteria then obviously as mentioned like the data ops engineer or data engineer was our main persona he could much easier do audit trails he could track this lineage of data identify what's impacted by what and also it helped us to actually reach reproducible model builds because we would store like random seeds and other things as just attributes in in that graph structure and that actually helped us to be able to rebuild models more or less with the same outcome which is was a pretty hard problem and then we always got like from compliance they were pretty annoyed that we had different model performance in the end and also it really helped us in production there was this problem of a data shift so data shift basically occurs if your production data ends up having for example different distributions over time there are new attack vectors or simply like just a new usage pattern you haven't seen in your training data and this often means that your model you have trained is not applicable to that data anymore because it just has different characteristics and this was previously pretty hard to detect and by keeping track of that metadata also for the serving environment basically like which requests do we get we were able to always compare the distribution and characteristics of the training data set to the data we would be seeing in production and this would also for example trigger like a rebuild of models once we saw this was getting out of hand or out of his like a certain range also for the more traditional dev ops or system administrator it proved to be very useful simply because we could do resource accounting we would know like how many teams how many hours training for example models teams had spent because that's also something we would simply monitor and hence also translate to like cost like how much had they spent on training this one model how much had the team spent on solving a particular problem which was very helpful also in allocating different resources also for permission tracking we can basically see is the data scientist for example able to see all the different entities involved in like training a model or he actually can't work on a certain problem because he is not allowed to access the data set all right maybe just diving a bit into arango ml here so how did we solve that problem in the end so basically this is the schema so to speak we came up with for arango ml and schema so to speak is mostly because there's no fixed schema we came up with a number of entities which we felt useful were for most machine learning platforms but in the end everyone can simply like define a new feature a new entity in that schema and actually also change connections in between because it's simply like a graph structure so i can for example directly move if i'm not using notebooks i can directly move from like a feature to a model function depending on my workflow at my company so as mentioned there is no fixed schema but we have some entities which just have better support in the UI or are kind of like first-class citizens arango ml it comes with different apis so we have a Python package available you can just leverage from your like your favorite notebook or your favorite data science environment we got an HTTP environment for cases where you are not working in a Python environment and we are also just finalizing our TensorFlow extended integration and i think i have a slide for that upcoming so basically if you're using tfx it's all automatically stored in into arango ml if you want to leverage that this then is a first screenshot of the UI it allows you to for example find models or deployments it allows you to tracks and also lineages so once i've found something in particular i can actually see everything which is related to an experiment related to deployment and the next version will actually allow you to then also double click on model and then also deep dive into there so again this gives me a really easy insight into everything which is involved there and again we as humans like graphs make it really easy to comprehend how certain things are connected so this is why we wanted to have another graph representation in here making it easy for people to understand the connections between different entities as mentioned like tfx so TensorFlow Extended kind of has its own way in adding metadata storage so they actually announced that after we started that project and originally i was pretty annoyed because we were like hey we tried to solve that but in the end it turns out to be a pretty good fit because this is very targeted to the TensorFlow Extended to the TensorFlow ecosystem those APIs you don't really want to code them by hand it involves a lot of grpc calls a lot of generations and in the end they actually defined an interface which allows us to easily plug ourselves down there so this is basically just imagine that last box being a ranko ml down here and we'll release that probably within the next months and hence we can easily integrate into that ecosystem as well the other projects are also catching up with metadata i think here there's alpha version notice and we're actually working together with that community as well on the metadata tracking but this problem of metadata is actually coming up on a lot of those different machine learning platforms or pipelines because it's typically required for production grade scenarios and also for anything involving audit trails or something similar so whenever i got compliance coming into the picture i somehow need this metadata and be able to trace back of what has happened for building this model all right thanks for listening i actually just i got six minutes left i can just briefly show you because this is all open source so if you just go to like a rango pipe slides are will also be online after this talk so links are going to be in there so just jump in the notebooks you can run if i reconnect to my session let's see if it reconnects but basically that entire example is here for you to run from from a jupyter notebook here's the ui we can search things we can explore whatever we have done in in our models and then the last thing i briefly wanted to show is kind of the implementation details for those entities and this is basically the way in which a rango db is dealing with those different entities so you might see their collections are kind of like the table acronym for inside a rango db because it's not just tables it's collection of entities so collections and here we have the overview of all collections with all the default entities involved and you can see there are two different kinds of ways in which things are represented so those are kind of like documents and so for example here's a document describing a deployment and then we got those kind of like edges so edges are describing how things are connected and because those two entities are actually independent of each other i can have this flexibility of always adding new edges of adding new entity types without being constrained to like one particular schema to one particular setup all right with that i think because we are also at our planned limit yes shortly i would just leave it up for some questions otherwise feel free to just find me somewhere afterwards and thank you so much for listening three two one no questions okay i'll be down here thanks


From a sheer production standpoint, Kenneth Walker looks like an obvious hit for any team looking for a new running back. He rushed for 1,636 yards and 18 touchdowns from Michigan State in 2021, ranking second in the NCAA in yardage and eighth in touchdowns. He paired that however with only 13 catches, actually a career best for him after transferring from Wake Forest. The mixed production also leads to strange comparisons for Walker and the most similar profile to Walker's is Lamar Millers. While a lack of receiving work is problematic historically for prospects, Walker does have elite size-adjusted athleticism after he ran a 4.38 second 40 yard dash at 5'9 and 211 pounds. A path to production in the NFL is there, but few with Walker's type of profile have become three-down featurebacks.


Some 75 states have laws that actually criminalize same-sex activity, with punishment ranging all the way from imprisonment to the death penalty. In numerous states, structural and societal discrimination is deeply entrenched, including in education, health care, employment, housing and families. May 17th marks the International Day Against Homophobia, Transphobia and Biphobia. This day is a reminder to all of us of the violence and discrimination experienced every day by lesbian, gay, bisexual, transgender and intersex people all over the world. At IOM, two of our fundamental principles as you know are equality and non-discrimination. These principles apply to both our staff and to the migrants we serve. The world has made significant progress and advances in promoting and protecting human rights. And IOM is very much part of that practice. Let us commit on this day to serve all migrants and staff members with respect and dignity, regardless of their sex, their sexual orientation or their gender identity. IOM counts itself as an LGBTI ally and rejects discrimination and persecution against LGBTI staff members and migrants. I am myself an LGBTI ally and I stand for human rights for all.


It's 1926, your money doubled. Commander Fleet Activity Sasebo held a million dollar sailor course to teach sailors about investing and financial stability. It is part of a Navy program. The goal is to really get people to start thinking about investing and thinking about their retirement plan because in the future there's going to be a huge change with the military retirement system. So we have to be prepared for that and part of that is understanding the basic steps. One thing you can start off doing is putting a little bit away, understanding where the money is going to go, have a savings account somewhere where you don't see it and where you're not going to go back and tap into it to use it. Petty Officer Zachary Hale, Fleet Activities, Sasebo, Japan.


Hi everyone, Joe for jazby's casebreaks.com coming at you with 2022-23 Panini Prison Basketball 4 box picker team number 13 all card chip in this Big thanks to and it's a bounty break as well big thanks to this group for making it happen Michael ended up at last spot mojo with the Rockets If you got prism nine next year and that means you won that team in that filler pack that we did as a reminder Here are the numbers we're looking for in the bounty Gotten close on the number of We've gotten pretty close on a number of breaks, but not actually quite hit yet. There's prism basketball right there Take this out of the inventory system Don't do it now. I'm gonna forget. There's a set of four or a set of three now. We're doing four There another set of four Here Four and we'll keep this set of four in here just for the just for dice roll purposes. All right So we'll go one two For this column three four and then the four in there five six on the dice roll and it's four one two Three four Well, there's just scored. All right, so we're gonna do that and we'll save these For next time so put some squiggly marks there And we'll put some squiggly marks on the bottom right there Let's grab this box See the cards on the table. All right. Good luck. We're basketball Kings doing here. That's the baseball game lot going on today 435 left in the fourth And the Kings are up 99 to 95 Steph Curry on sitting on 26 points deer and Fox 21 7 and four steals 21 7 assists and four You can switch back to the basketball game. There's still 10 minutes 1030 left in this hockey game and stars and wild headed to OT OT and hockey always exciting Basketball is on commercials. All right, we've got all the depot The red waves are not numbered by the way We've got Trey Murphy the third. Let's check the number on that after we check out Bob McAdoo Nice old-school Laker Going to Roger. All right, let's take a look at Trey Murphy the third 130 out of 199 No, we need 100 out of 199. That's for Pelican. That's gonna go to Colin. Here's a Kenneth Lofton, Jr. Rookie hyper for the Grizzite Joe with that We got Obi-Toppin to 75 We got Jalen Williams and Santa Clara Jalen Williams for OKC. That's for Bob and Obi will go to the Knicks at OB for Greg Red waves once again not numbered. Mr. Barry Smith, Jr. Rookie Silver. That's one of that someone that Michael Ruzz looking for with the Rockets There you go Maybe find some ink later on. It's Carl Malone. 29 out of 149 Logan and his jazz The mailman for the mailman All right, and he guesses on the redemption for bragging rights David Roddy red wave Rookie red wave for the Grizz for Joe Max Christie Rookie hyper for the Lakers and Jeremy Sohan 121 out of 125 blue ice Nice one for William James is thinking that redemption is gonna be a rookie Be nice Scott thinking Christian Braun since he's a common redemption His red Cameron Payne 44 out of 299 Sons we need we need 196 out of 299, but I'll still go to Nicholas And Phoenix won that team in the filler. Oh, we got some gold coming up Wow luck of the lottery Gold Chet Holmgren 10 out of 10 Nice luck of the lottery insert there. That's for Bob and Okay, see Congrats and a base Jade Navi for Greg. Yeah, nice Chet for Bob. All right Redemption is Peyton Watson sensational Signatures Peyton Watson Peyton Watson is a Denver nugget. That's right. He was with us with UCLA for a season That's Matthew Matthew Nielsen with the Denver nuggets. All right. That was box one good start. No bounties though Good cards Let's see what's in box two So we'll chat have another rookie card next year since Uh No, I don't think he's not going to get a second rookie card even though he retains his rookie eligibility at least for the NBA But yeah, he could he could win rookie of the year Yeah, it's as soon as you play a single second In the NBA you were no longer a rookie according to the NBA remember I don't know if some of you remember Julius Randall's rookie year with the Lakers. I think in the first quarter He was playing you I think he was even starting I think he He just He did his knee. I think he landed near did his knee and he was out for the year and he hadn't even played a full game And that's it. No more rookie eligibility. I think baseball has you play like a certain X number of innings or At bass or something like that before you lose that eligibility. All right. There's a Jalen Green Orange 47 out of 49 That's for Michael in the Rockets. That's Christy rookie hyper. We got a Keegan Murray For the Sacramento Kings, I'm watching my LA Kings trying to Try to tie this game up. They're only down a goal. I might have to switch over to that basketball game Warriors are down by six or Sacramento Kings are up by six Depending on whose side you're on and we got Anthony Simons 64 out of 299 That's what we're looking for back there and we got a triple J Jaren Jackson, Jr. Nice. He's pretty good. I have a nice game against my Lakers Roger not Sir Roger has the Lakers. It's Jake Joe with the Memphis Grizzlies Nicola Jovic Rookie silver for the heat and will be from Michael P. I mean there's Still two 150 left and with the way Steph Curry can shoot The lead can swing pretty quickly, but oh, I'm going to say shooting free throws Trying to sleeve Jay crowd out necessarily need to there. There's Jalen Williams thunder Not free throws made there's Alec Berks 42 out of 149 and Willie Green for the Sixers Oliver with the Sixers No, it looks like a fun finish here There's Jalen Williams red wave for the Thunder. This is Santa Clara Jalen Williams Although Arkansas Jalen Williams not too bad either He was the higher graphic The first rounder Big three there's Bryce McGowan to 125 and Steph misses a three Ten to run the last minute 40 minute 150 Wow There's the other Jalen Williams Jalen Suggs 158 out of 299 no bounty But that'll be for Orlando. That'll go to Jonathan. I think this game is over We're just passing it around. Yeah Warriors really not going for the ball I'm just gonna let the clock tick shot clock tick shot clocks out of the game clock Are they chanting light the beam? Sacramento Michael is gonna go to oh on the Warriors. I Like the beam thing. I think that I think that's pretty fun. I think it's a that's that's a fun thing What's the game? Sacramento Kings win 114 to 106 taking a two-nothing lead in This series Wow light the beam Go back to different Kings hockey Kings Mark Williams There's Frank Jackson 150 out of 299. We need 196 Pistons that's gonna go to Greg And we got old nugget Danice out. It'll be for Matthew Nielsen and Denver We got Marjon Beauchamp Rookie hyperparallel for Milwaukee. They'll be for Russell the Carl Anthony Towns double oh one out of 149 and Keegan Murray rookie card and We've got a TJ Warren The variation or something maybe maybe Thunder Jalen William Bob And then there's been and behind Davion Mitchell is Christian Braun rookie autograph for Denver That's gonna go to Matthew. There's Davion Mitchell to 43 Out of 299 Dash, what's going on Greg? How are you? Nikolai Jovic red wave. Oh Is that the Kings with a delayed penalty? There's Desmond Bain to 49 This is the LA Kings chance right here Desmond Bain for the Grizzlies and shake Joe Patrick Ewing orange wave 56 out of 60 Red waves are not numbered and the next box Mmm. Was there only one auto in there? That's how they get you on average Yeah, I don't know what that random that might have been it I don't know what that was but I sleeve that anyway in case that was like a variation or something like that I don't know if they have that but the LA Kings have a power play with a Only a one Just a minute left in the game. I'm tend not to be a ref blamer Greg Even if they're bad, but from what I understand that I'm hearing to former players and Announcers from multiple different teams and stuff. I think NBA officiating has been pretty bad this year Overall, we're gonna empty the net. There's Patty Mills 203 out at 299 Brooklyn Nets That's gonna be for Kenneth and there's Josh Minot Rookie auto for the T-Wolves Colin with the Timberwolves Jaden Hardy for the Mavs hyperparallel Jabari Walker for the trailblazer rookie silver for Matthew Nielsen There's Giannis 65 out of 75. It sounds like he'll be okay Russell with the Bucks 30 seconds left. Oh, that's a goal Yes, oh my god Wow with 16 seconds left La King score a goal ties the game Let's go. I was just kind of rattling around in front of the net You know one rebounded and then it slipped right through wow Nice little jade and ivy luck of the lottery silver for Detroit That's gonna go to Greg. Come on. Come on.  Let's go OT now saying earlier Some exciting OT a hockey OT pretty exciting. Here's a jam around red to 13 out of 299 Grizzlies Joe and there's Nikola Jovic Miami Heat autograph Yeah, Copa tar has been around for a while, but he's still playing well Real well and Greg was talking a little bit earlier about refs gave us six thousand three minutes. I feel like I Feel like the feel like you're not gonna get a lot of sympathy from other fans because I feel like people have been saying especially during the Any of the Warriors play like deep playoff runs How many call Steph Curry gets on a day on a nightly basis? There's Josh Minot for the Timberwolves. That's gonna be for Colin Red rookie red wave Nicola Vucevich 33 out of 125 That's for Greg and the The Bulls We've got a Jalen Williams for the Thunder For okay, see that's Bob and the Thunder and that is that nice Well, no bounty, but I think a nice little break especially with that silver jade and ivy Red wave Jalen Williams Jaren Jackson Jr. Jeremy Sohan Bob McAdoo and that luck of the lottery 10 out of 10 Chet Holmstrom Nice sharp-looking card right there.  I'm Joe for Jaspy's case breaks calm And I'll see you next time for the next one. Bye. Bye