Well, thank you guys for coming to my talk. My name is Alec. I am a core product engineer at Gray Matter and also one of the active maintainers of Envoy's Go Control Plane. And I want to take this little 10-minute talk to bring everyone up to speed on what exactly has actually happened this past year and where we intend to take this project as it's become a pretty critical part of the core infrastructure for the Envoy ecosystem. So let's first start off with a little recap of what's actually happened. So if you're following the XDS lifecycle, V2 has been pulled out. So we're pretty much strictly on V3 and focusing more on the UD, which is V4 and forthcoming. So in 0.9.9, I believe, we pulled out V2. So if you still are dependent upon that, you're going to need to go ahead and pin back to an old version. In June, we introduced ECDS. So as the project's growing, more XDS services are coming out, more discovery. And we are continually integrating those into Go Control Plane as there are quite a lot of projects that do rely on us. So that came in in June. This I'll talk more on in a little bit, but profiling has become a big target for Go Control Plane. We know that, again, this project is required in a lot of places upstream. But none of us really know what actually is happening under the covers. We've never really profiled. We've never really benchmarked. We know it works, but we don't know how well. In August, we actually reworked the snapshot API. So I don't know if you guys are familiar with the code base, but this is pretty much the core interface with the cache. And we found that it was really difficult to continually update for those rolling discovery services that kept coming out. So we readjusted, we changed the API, and we haven't reached 1.0 yet, but I'm going to talk about that in a little bit. So we just merged those in, and we called it another minor release. Incremental went in also. That was a pretty big milestone, but I won't talk too much upon that. Since last year, I gave a talk on this. So if you guys want to go learn more about that, I would recommend checking that talk out. And then we just actually merged in the scoped route discovery service, and that was a pretty big change to the core internal logic, because it obviously changed a little bit how we handled route associations to listeners. So that's pretty much what happened in 2021. And then I'll go ahead and talk about where we want to go with the repo. So what are some goals for Go Control Point? We've had a lot of requests for a 1.0 release cut. We've had some pretty major projects now become dependent upon GCP, so like GRPC Go, their XDS implementation requires on us.  So we can't just continually break. If we just keep breaking, it's going to become a huge nightmare and a big pain for all those control planes that are dependent upon our code. So what does that entail? The 1.0 stuff kind of involves a lot of protobuf upgrades and also a little bit changes to the Bazel system that Envoy uses. We want to focus on benchmarking. So like I said in the beginning, we don't really know how this is performing. There could be optimizations out there we don't understand, and we want to investigate into that. And we're going to continue to bug fix. Again, this is a pretty popular project. A lot of people use it, so we want to make it as stable as possible. And then a reference implementation.  So for the 1.0, we decided to go with a multi-module release plan. So this means that if you're familiar with the code, we have the generated protostubs from the actual data plane in the code base, as well as the actual reference implementation of the XDS server and cache. So we have to be able to version those independently and asynchronously. So by doing that, we're going to treat that the protostubs version separately, and then, et cetera, et cetera. So we have a plan. I'd recommend downloading the slides and checking out that proposal to see how that's actually going to be versioned. It's a pretty complicated process, but we are working to get that done. We are dependent upon some deprecated libraries. But we want to move to the latest, which is what Envoy's pointed out. So that's going to require, again, some of those Bazel build upgrades. But that's pretty in-depth. And we are lacking a little bit of bandwidth on there. So if anybody wants to help out with that, please, please make an issue. And we would definitely reach out and like to collaborate. So for benchmarking, this past year, we integrated Pprof, which is Google's Go profiler, where we started investigating how the code's actually performing and optimizations that we can make. So the reason this was done is because incremental was merged in. So we wanted to see the differences in actual compute load for the XDS types. And we are actually working on building a full end-to-end throughput benchmark. That's not done, but it is something that I think a lot of people will benefit from as they'll be able to see what the actual benefits of incremental from state-of-the-world are. So the reference implementation. Previously, Go control plane was not really looked at as something that a lot of people would really use or depend on. So with the ecosystem growing and maturing so quickly, there's been a lot of issues that have occurred around replicating common functionality across multiple projects. Matt actually wrote a really good blog post in last year's EnvoyCon about greenfield deployment and the code replication, et cetera. And I'd recommend you guys go read that. But that replicated code is a problem. People are solving the same solution over and over again. So how do we fix that? Well, we bring it all upstream. And because Go control plane is a Envoy-sponsored project, it's a perfect place to do it. So we break down some common functionality across these projects. We bring them up, and maybe in Go control plane is a really good place to do that. We could even potentially have something like a reference service discovery mechanism, because what are all these projects do? They're all tackling the same problem. They're all performing service discovery. They're all doing health checking, circuit breaking, et cetera, et cetera. So is there really need to replicate this over and over again? So that's basically where Go control plane is going. That's our thoughts and ideas on the project. And again, we would love for people to help out and also maintain this project. We have a pretty small bandwidth for the maintainers. I think there's only four of us. So the more help we can get, the better. But yeah, thank you guys. And I will be in the slack if anybody has any questions.


doing the recap for the Sunday Funday NFL 50 Box 2017 National Prism Treasures football mega mixer, random team style. We'll start off here with our jerseys. We had the Sony Michelle, Nasty Patch, Josh Jacobs, Alshon Jeffery, that's Christian Kirk, Lamar Jackson, Leonard Fornette, Matt Ryan, Phillip Lindsay, James Morgan, Richard Sherman, Mike Williams, Tony Dorsett, Marcus Marriota, Bryce Petty, and then a quad running backs that went to Lions by random. More jerseys. We had a printing plate of Emmett Smith, we had Cooper Kup, Jake Plummer, that is Don Tavion Wicks, Damian Harris, Trey McBride, Deontay Johnson, Kway Walker, a Tyquan Thornton, Brian Robinson Jr., Kareem Hunt, Trey McBride, Duel for the Jets, Chris Olave, Garrett Wilson, Dak Prescott, we had a DJ Moore, that is John Mechie, Sam Ellinger, Nico Collins, Kenneth Gainwell, Derek Carr, Isaiah Spiller, D'Anthony Thomas, James Winston, James Winston again, T.J. Elden, and Gameuse Jalen Ramsey. So with our first stack of autos, we had a McKee and Tanner McKee, Michael Wilson Duel, Hedden Hooker, Jalen Hyatt Duel, Mecca Aguca, Trayvon Henderson, Xavier Worthy, Caleb Williams, Sam Heward, Max Johnson, that is Jameer Gibbs, Jordan Battle, Miles Murphy, Jackson Smith and Jigba, Trey Tucker, Drake London, Devin Witherspoon, a Rakeem Jarrett, Zach Harrison, Josh Wiley, Kendra Miller, Tanner McKee, Jordan Battle, Rashi Rice, Miles Murphy, Chase Brown, Hedden Hooker, C.J. Stroud-Plate, going to the Texans, that is Bryce Young, Traylon Burks, Damien Pierce, Kate Aten, Joe Montana, Tyler Beatty, Carson Strong, had a Tyler Goodson, that's Cam Acres, Damien Pierce, Christian Watson, Noah Sewell, Cordell Flott, Drew Sanders, Quentin Johnson, a Raheem Moster, Trayvon Walker, Kennedy Brooks, Breece Hall-Rookie, Traylon Burks, Sam Howell, Kenneth Walker, Marcus Colston, David Ajabo, Louis Seen, Nick Muce, that is Chris Alodicon, Louis Seen again, John Mechie, David Bell, and Zemir White, next stack of Autos, a Desmond Ritter, Bailey Zappy, Fadarian Mathis, Trey McBride, Amon Ross St. Brown, Christian Barmore, Brendan Knox, Kellan Mond, a Racy McMapp, Rhonda Elmore, had Darren Hall, Jonathan Cooper, Darnell Mooney, Cornell Powell, D'Yammy Brown, Cole Comet, Trent McDuffie, Pierce Strong Jr., Gardner Minshew, Zemir White, Max Williams, Jeremy Langford, Ty Montgomery, that's a DeAndrew White, Drew Locke, Riley Ridley, Isaac Bruce, Kalevon Cheson, Cam Akers, Jacob Eason, Gabe Davis, Carson Wentz, Jared Cook, Marlon Mack, is that game used? Not player worn. Matt Ryan, Malachi Dupri, Matthew Daze, Corey Davis to 25, O.J. Howard, Joe Williams, had booklets where Dalvin Cook, Breece Hall, as well as a Trayvon Walker, we had a Jameer Gibbs, one of one, and a Will Anderson Jr., one of one, and our Mags now, that's actually pretty solid. Good lord, dude. Bailey Zappy, George Pickens, Jalen Tolbert, Kenneth Walker, we had Autos now, we're a Chris Olave to 11, a Kenneth Walker, one of one, R.P.A., a Jonathan Taylor, R.P.A. to 99, had a Kenny Pickett, Auto to 20, a Tua Tagovai Loa, Rookie Auto to 99, a one of one, Nebula, Odell Beckham Jr., and then two of 10, Josh Allen on the MVP Contenders Auto. That'll do it for the break, guys. Thanks again for the fill. Appreciate it.


It was February 13, 2009, when Amber Dubois set off for school for what would unknowingly be the final time. As she arrived, she was approached by a boy who appeared to be a bit older than her and certainly much larger. The two chatted outside for a few brief moments and then Amber disappeared. Just a year later, in February of 2010, Chelsea King was out jogging on a local hiking trail in Rancho Bernardo Park. She planned on going on a five-mile hike, but at some point along the way, she too disappeared. Investigators would later find pieces of her clothing scattered all around the trail and soon after, detectives came across a crime scene. As a quick side note from today's story, did you know that your personal data is actually being sold online by data brokers right now? I believe we all have the right to keep our data private and protected, and that's where DeleteMe comes in. DeleteMe is a service I trust to remove all my personal data from the internet with the simple push of a button. DeleteMe's proprietary system and team of experts will review hundreds of data broker websites and remove information about you and your relatives, such as name, age, address, email, phone number, photos, social media links, almost literally everything. After your first seven days of joining DeleteMe, you'll receive a personalized privacy report detailing all the various places where DeleteMe has found your info and what they'll do to get it removed. Personally, my favorite aspect of DeleteMe is that I'm getting far fewer robo-calls and scam calls since signing up. They've removed my phone number from so many databases and it shows. I used to get four or five calls a day and now I'd be surprised to get more than one or two in an entire month. I'll admit I was pretty skeptical about the service at first, but I can tell you firsthand that it works and it works well. I've been using DeleteMe for several months now and the results couldn't be better. You can join DeleteMe right now using my 20% off discount code TI20 or by clicking the link in the description. That's discount code TI20 or by clicking the link in the description. Give it a try for yourself and let me know how it works out for you. I think you'll be impressed. Thanks to DeleteMe for sponsoring today's video. Today's story involves three victims in total, but we'll specifically be focusing on the two victims that led to the criminal's eventual conviction. Remember, if you guys know of any strange or interesting cases that you'd like me to cover, be sure to leave your suggestions in the comments. I'm always looking for new stories and most of these cases come from you guys. But with that said, let's begin the story of Amber Dubois. Amber Dubois was a very interesting girl. She was just 14 years old in 2009, but she had already made up her mind about what she wanted to do for the rest of her life. Like many young girls her age, Amber had a loving interest in animals and wildlife. She'd spent a lot of her life infatuated by farm animals, and she'd always speak to her mother about her dreams of owning a few animals of her own one day. See, Amber was a very academic young girl. While many of the other kids her age were worried about what they were going to do for the summer or where they were going to go shopping or go hang out after school, Amber was extremely focused on getting good grades, taking as many courses as possible than trying to graduate early. And she was well on her way to doing so. Amber had been attending Escondido High School in California, just a short trip away from the heart of San Diego. When it came time to choose her classes for the year, Amber knew that she wanted to become part of the school's future farmers program. This program would allow the younger generation to become acquainted with farm animals and all the ins and outs of living life on a farm and taking care of the animals and livestock. This program was everything Amber had dreamed of, but there was one catch that her mother seemingly didn't see coming. This program would allow Amber to get a head start on her future goals of becoming an animal behavioral scientist, but in order for this to happen, Amber would need access to a lamb, particularly a lamb of her very own. Her mother wasn't interested in letting her buy a lamb. After all, lambs are a lot of work, at least for someone who doesn't really have access to all the necessary supplies and whatnot. And Amber certainly fell into this category. But as the weeks and months passed by, Amber continued to beg and plead with her mother to allow her to buy a baby lamb. Eventually, her mother felt obligated to agree and even offered to pay for the lamb if it meant helping Amber achieve the future that she had always dreamed of. But while she was very hesitant about the idea, Amber's mother wrote to her check for the lamb with Amber becoming ecstatic, securely hiding the check inside of her backpack so that she could go to pick up her new baby after school. This all took place on February 13, 2009, a date that for reasons unexpected, her mother would never forget. Amber left her home that morning with a skip in her step, overwhelmed with excitement by the idea of finally having a lamb of her own. She couldn't wait to get to school and tell both her instructor and her friends about what the future had in store for her. But tragically, she never got the chance to do so. As soon as Amber arrived on the campus of her local high school, a witness spotted her talking to someone who was described as being much taller than Amber. The witness couldn't make out the boy's age but seems to have believed that the person she was speaking to was a student at their high school. To clarify, the witness didn't say this outright but it's just the impression they gave during their interview. They described the boy as being doughy looking, quite tall and dark complexed. After Amber was spotted talking to this person, she would never be seen again. This particular day was a Friday. It had been raining on and off throughout the day but by all means it was a day like any other. That is, until Amber failed to return home that afternoon. One thing to note is that Amber's parents had separated at some point during her childhood, leading her mother to begin dating again. While Amber's mother appears to have been at work on this particular afternoon, her mother's boyfriend had been at home and was anticipating Amber's return from school, but she never came back. Naturally, the boyfriend, named Dave, assumed that Amber had just become caught up in her newly purchased lamb and had lost track of time. But as minutes and hours eventually ticked by, Dave became extremely worried. He decided to drive over to Amber's school and try to find her. No sooner than he arrived, he spotted one of Amber's teachers and asked if she had seen Amber. In a disturbing twist, she replied that Amber hadn't been in class that day. This was particularly odd because not only should Amber have been at school for the entire day, but she'd also never missed a day prior to this. Her mother recalled that Amber was hyper-focused on her school attendance and her studies and she wouldn't miss a Dave's school no matter what. So where had she gone? Dave grew incredibly concerned at this point and called Amber's mother to report what had happened. Soon after, Amber was officially reported missing and the search was on. Hundreds of volunteers caught wind of Amber's disappearance and turned up to help search for her. The search area spanned for miles, but no matter how far or wide the search was, no one managed to find any evidence that would explain where she had gone or who could have been responsible. When police began to dig deeper into her disappearance, they realized that Amber's phone was still active in the area a short time after she had disappeared. There was no mention of what time her phone was specifically turned off for good, but we know that wherever she had gone, it had likely been within a span of just a few miles, as Amber's home and school were both within the proximity of the same cell tower. And there was no indication that she'd been anywhere near another cell tower during the beginning stages of her disappearance. Now it's obviously possible that her phone could have been disabled before she left the area, but police didn't seem to believe that this was a likely scenario, though it's unclear why. When police began looking further into her cell phone data, they quickly learned that Amber's voicemail had been accessed shortly after she had disappeared. This voicemail again pinged off the same cell tower that connected to both Amber's home and her school. Investigators had now begun to feel confident that Amber had been abducted, but the only real evidence they had to go on was the eyewitness account from earlier that morning, describing the unknown male that she'd been speaking with just moments before she vanished. Police, now taking the case far more seriously, began to interview all of the registered offenders in the area as well as Amber's friends and family. A total of 1200 tips were followed up on, and police conducted at least 500 interviews, but none of these interviews produced any leads or additional evidence. The captain of the police force recalled that no matter how much they investigated the case, they only ended up with more dead ends. Just over a year later, things would only get worse. Chelsea King was just 17 years old on February 25, 2010. She was a straight A student at Poeway High School in California. Much like Amber, Chelsea lived just a few miles from the heart of San Diego. Chelsea was also a girl who took her schoolwork very seriously. Outside of her usual classes, she was a member of the school's cross-country team and would spend a couple hours each day training, running, and hiking. One particular trail that she often visited was in the Rancho Bernardo Community Park. The trek would have taken her about five miles from beginning to end, and it was a trail that her family said that she would take time and time again, a trail she knew like the back of her hand. Chelsea had spent her last few months researching more than 100 colleges. Considering she had excellent grades and a remarkable work ethic, she'd been accepted into a few of these schools already, but it seems as though she was trying to decide between the University of Washington and the University of British Columbia. But unfortunately, she wouldn't attend either of these schools. That's because after she was out hiking her favorite trail at Rancho Bernardo one day, she wouldn't return home. It was February 25th, 2010, almost exactly a year after the devastating disappearance of Amber Dubois. Chelsea had been following her typical five-mile route, with her parents expecting her home at her usual time. But that time came and went with no signs of Chelsea. Chelsea's family and friends searched all over the park for any sign of her, but there was simply no trace of her. Before long, the police were called in to investigate. The police worked alongside a group of countless volunteers. With their combined efforts, they began to make a bit of progress in Chelsea's case, but it wasn't the progress they had hoped for. Rather than finding evidence that Chelsea was alive and well, all of their discoveries proved the exact opposite. First, volunteers came across one of Chelsea's socks, found about two miles away from where she had parked her car. But then came the most shocking evidence of all. A volunteer came across her undergarments, combined with evidence that Chelsea had met with a terrible fate. A short while later, one of her shoes was found as well, less than a mile away from the two previous discoveries. A police sergeant says that the location of this evidence proved to them that foul play was almost certainly involved. The FBI was called in to investigate soon after, with them interviewing people at at least 300 homes in the surrounding area and following up on at least 600 tips. And again, registered offenders were interviewed, but no one seemed suspicious or gave any indication that they may have had ties to Chelsea's disappearance. But investigators would finally have a breakthrough in the case when they sent in the underwear that they had found for a DNA analysis. When the results came back, they found that the DNA obviously matched Chelsea, but they also found DNA that belonged to a man that was known to the local police far too well, a man named John Gardner. John Gardner was well known to the police near San Diego. That's because back in 2000, he had been convicted in another case involving a minor. In this particular case, the victim made it away with her life, but just barely. He had ambushed a 13 year old girl and taken advantage of her. In a truly chilling twist, the young girl had actually been his neighbor. John had been about 21 years old at the time of his offense. He was sentenced to serve just six years behind bars, being released early on good behavior. The prisons were known to have been incredibly overcrowded at this time and it's likely that this played a part in his early release as well. John was eventually released from parole in 2008, but a later investigation found that during the stint of his parole, he violated it at least seven times, including one time in which he lived too close to a school in 2007. John was also given an inklet to wear during his parole, during which time he was documented as having violated that 168 times, but nothing was done about this. When the GPS data was looked over later on, it was found that he'd spent countless hours in close proximity to several schools. He was also spotted in front of a daycare, as well as visiting prisons, with police believing he may have been delivering contraband to inmates. When police connected John to the case of Chelsea King, everything began to fall into place. It didn't take a genius to connect the dots and understand that John had stalked Chelsea along her jogging route that day, striking when she least expected it. It was reported that he took advantage of Chelsea and also claimed her life, but police never revealed the order in which these events took place. But as investigators were digging up all the details that connected John to Chelsea's disappearance, they began to realize that Chelsea's case had striking similarities to the case of Amber Dubois. Police would eventually track John down to a bar, where they found him to be soaking wet, covered in mud, and clearly drunk. They brought him in for questioning regardless, and it became apparent that he wasn't telling the truth about his knowledge of Chelsea's disappearance. But things got a lot darker when, without being provoked, John brought up Amber's disappearance as well. We have no idea why he would have spoken about Amber, and police say that her name came up at random during their interview. During the process of John's interrogation, it became clear to him that investigators had enough evidence to pin him to the crime. He eventually called in an attorney, and the attorney knew that they had a very solid case against him. It also became clear that prosecutors were going to do their best to secure the death penalty. Throughout this process, John quickly began to realize that, by all means, his life was essentially over. This realization set in even more when he found out that just five days later, on March 2, 2010, a team of FBI investigators had located the remains of Chelsea, buried near Lake Hodges. This was the moment that John knew the jig was up. He had been found out, and there was little he could do to stop his conviction. Unless he got crafty, investigators obviously had enough evidence to pin him to the case of Chelsea King, with the DNA match on Chelsea's clothing being the most crucial piece of this. But the evidence connecting him to the disappearance of Amber Dubois was trivial at best. They believed her case had many similarities to the case of Chelsea, but they had virtually no evidence tying him to her disappearance. Just because he mentioned her name in an interview doesn't prove he was guilty, it just proves he watched local news back in early 2009. But that's when investigators came up with a plan. As mentioned, John was staring down the barrel of capital punishment, but they offered him a plea bargain that he couldn't afford to refuse. If he would lead them to the grave of Amber Dubois, they would agree to stop pursuing capital punishment, and instead simply pursue a life sentence. The only problem with this is that Chelsea's family would need to agree with these terms. Up until this point, Amber's family had been living in utter turmoil, hoping, begging, and praying to learn what had happened to their beloved daughter. The only thing worse than finding out that your loved one passed away is finding out that they're just missing, and you have no idea if they may still be out there begging to be found. After hearing about the heartache that the Dubois family had been forced into, Chelsea's family agreed, without hesitation, to waive the possibility of capital punishment in exchange for a plea deal. Everyone was in agreement, and John decided to open up about what had happened to both Amber and Chelsea. He told police everything, and he led them to an old dirt road, taking them to a rusted-out water tank. Nearby, police found the shallow grave that belonged to Amber. When John explained the details of her final moments and what he had done to her, they confirmed these details after digging up her remains, and it appeared that John had in fact been telling the truth. He was later convicted and given two life sentences, one for each of the girls, as well as an additional 25 years for assault and an additional 24 years for other convictions. Needless to say, John Gardner will never be leaving prison again. But one of the headlines and comments that I continually see in regards to this case is that the system somehow failed these two girls. Now, I'll admit at first, I didn't understand what these people were talking about. After all, the criminal was caught and put away for life two times over. The system worked as well as it ever has. Now, I'm often the first one to criticize law enforcement when they make ridiculous mistakes, but this is a prime example of an investigation firing on all cylinders and putting a killer behind bars. But there is a bigger problem here. If John hadn't been released from prison in the first place, or maybe if he'd at least served a longer, more beneficial sentence, these girls may still be here. After all, the whole point of being put behind bars is to be rehabilitated and learn from your mistakes, but that didn't happen with John. If you remember, John was arrested back in 2000 for what he did to the 13 year old girl. That girl's life was changed forever in the blink of an eye. He took something away from that child that can never be given back and left her with a lifetime of fear and resentment, a lifetime, yet he served just six years behind bars. In response to this outrageous sentence, the family of Chelsea King worked alongside attorneys and law enforcement to instate a new law, now known as Chelsea's law. This law is the first of its kind to allow criminals to be placed behind bars for life, for crimes that didn't involve claiming the life of another person. If John had been given the proper punishment for his crime against the 13 year old girl all those years ago, both Chelsea and Amber would likely still be here. To top this off, if investigators had done something about the 168 violations of parole that John committed, these girls would still be here. If they'd arrested him when he was found stalking a children's day care, these girls would still be here. If they had arrested him for moving to a home that was within walking distance of a school, well, you get the point. The most important thing is that John Gardner is now locked behind bars, where he will remain for the rest of his days, but I can't help but shake the feeling that if police had actually done something about this man years prior, then the King and DuBois families may still be sharing holidays with their beloved daughters. Thanks for tuning into another episode of True Crime Stories. If you want to see more true crime documentaries like this, be sure to hit the like button and subscribe. If you'd like to help support the channel, the best way you can do that is simply by leaving a comment below, any comment at all. It helps out the channel a lot more than you may realize. If you want to help out financially, you can do that by clicking the blue join button below, or by picking up a True Crime Stories mug from TyNauts.com. But with that, my name is Ty Nauts, and I'll catch you guys in the next video.


OTAN, Outreach and Technical Assistance Network. Welcome, everyone. We are so excited to be here. For those of you who may have attended the CAPE Summit, you might have seen a session with Jay, Carol, and I earlier, later in 2021, previously, where we've started doing this kind of collaborative conversations and really talking about how do we make data and data usage a collaborative effort? And so we're gonna dive into that a little bit more today and we're gonna talk about the purpose of data collaboration.   So my name is Sudie Whalen. I'm with the California Adult Literacy Professional Development Project. I'm the deputy director of CalPro and also a major tech nerd. This is literally my favorite conference every year, so I'm really happy to be here with you. Also on with me are Carol Heroda. Carol, would you like to introduce yourself real quickly? Sure, good morning. My name is Carol Heroda and I'm a retired former administrator in Stockton School for Adults and I'm currently working as a coach with the targeted technical assistance with Cape Tap and I'm happy to be here this morning. Thank you. Thank you, Carol. And also on with us is Jay Wright who most of us probably know but we still wanna have him introduce himself anyway. Hi, I'm Jay Wright from CASIS and yeah, I think I recognize most of these names but I'm Jay Wright from CASIS. I do a lot of trainings and network meetings up and down the state, mostly on Zoom presumably the last couple of years. Of course, good to see you here this morning. Thank you so much. So I'm gonna go ahead and jump in. And so let's talk about kind of some of the reasons why we focus on data collaboration. First of all, it's really informative and helps us utilize data to inform our instruction, to differentiate instruction when needed and really have concrete evidence of where we need to go in those areas. It also helps us improve instruction and programs based on data-driven decision making. You can also utilize data to create actionable steps for your data teams and Carol's gonna talk to us more about what data teams are and what they do. Data also helps provide opportunities for teams to support each other if you're working in a professional learning community or a data team or just having collaborative conversations and wanna develop shared meaning. Collaborating on data is really helpful to add that additional level of support and understanding. Looking at the data specifically steers teams away from ineffective practices of procedures and moves us away from assumptions and into what does the data actually show us. It also improves consistency from one classroom or program to another when we're being intentional about focusing on what the data is showing us and closing gaps where we see them. It helps to guide future practice and informed teaching of specific skills if you're identifying areas that the data is showing you that certain competencies aren't being met and things of that nature. Lastly, it really supports a culture that uses evidence of student learning as an essential element for continuous improvement and not just I know my students or this is what I think is happening but utilizing that evidence. From an educator perspective, it really helps us in terms of student learning outcomes to differentiate instruction, provide interventions when needed and promote when students are ready to move up. But it's also super helpful in terms of curriculum alignment. If you're aligning curriculum within your program from one level to the next or if you're aligning curriculum within your consortium, looking at that data closely really helps us do that. So at this point, I'm gonna go ahead and turn us over to Carol and she's gonna share with us why it helps to collaborate with colleagues on your data. Thank you, Suti. So with my experience at Stockton School for Adults, the question was why would we want to collaborate with colleagues and not work in a silo? Well, the answer was basically accountability and continuous improvement. It was the theme of Kaizen, always improving, continuous improvement. So what we were expecting is that we wanted to collect accurate data. So that meant our data technicians as well as our teachers and counselors needed to ensure that the data that was put into the system was accurate. You wanted to identify evidence of student outcomes. You could also identify gaps, barriers, trends, equity issues, student needs, and learner persistence. Those were some of the things when you were evaluating your data. You wanna identify program and more importantly, your student successes. And creating a culture of data is really critical so that people are not afraid to share the data from one person to another, teacher to teacher, and the data technicians just saying, we want the most accurate data when we get it over to costs as TE or making sure it's correct in our attendance system, but also developing a system of sharing and utilizing that data. So let's go to our next slide, Suti. This is a chat.  How does your agency currently collaborate on data? Can you just place it in chat? We'll give you a minute to do that. Any kind of collaboration? Okay, thank you. Gather assessment data on a shared spreadsheet and use that data to inform planning for future units. Thank you, Richard. Quarterly data meetings. Awesome. There you go. Data-driven dialogues at staff meetings, you share your data informally, fairly frequently, tied to reports, analyzing reports. Okay, those are all good ways where your agency can collaborate on data. Thank you for sharing all of that. Perfect. So let's go on to the next slide, Suti. It is, should be in your chat. Now, indicate your role and your responsibilities at your agency with data collection and analysis. So based on your assigned role, what is your responsibility to collect that data, data analysis, data collection and analysis? You can place that in chat, please. Maria is doing data collection distribution of reports to appropriate staff. Thank you. That's really critical. I know, Jay, you're always asked some questions with some of your data managers. I think Maria Elena's comment is very applicable. Managers are our team.  So we get together to distribute the data and the responsibilities. Thank you. Okay, so let's go to the next slide. So whether you're working as a consortium or with your separate agencies, the administrators, like was reported, you're in charge of the whole system, creating that culture, creating a system so that data is shared. You also have to make sure that your colleagues have all of the tools and resources necessary to be able to share their data. In other words, updated, you know, hardware and access to the software, you know, that it's not just one person has access, that maybe several people have access so that they're able to share that. And then of course, a reviewing and analyzing the data with the team would be the administrator's responsibility. So the other one is the data managers. The data managers need to make sure that the data is input into the system accurately. And they're also responsible for sharing that data. So you will find different ways when SUTI continues this conversation on different platforms that data managers can share their data. In Stockton, we had an internal system within the school district where we were able to post student data on a drive so nobody else had access. The teachers also used Google, you know, to have their data team meetings and their professional learning community. So they definitely used Google, but the data managers need to be able to share that data with everyone, with the administrators, with the teachers, the counselors and the transition specialists. So that's really, really important. Now the teachers, when they look at this data, they're really looking and analyzing their data because they want to make sure that they have, the students have access and that they've created a lot of different opportunities and strategies for students to demonstrate learning gains. So that is the responsibility of the teachers. And I've already seen some things like data team meetings. That's really important that teachers meet regularly to talk about their data, share their data and have time to review and analyze how their data can improve. And if you create that system at the beginning of the school year in Stockton in August, the teachers develop all of their data systems then so that at the beginning of the school year they are able to continue on. They also have their data from the previous year and they're able to just import that data with them because with the open entry, open exit, you'll want to be able to have access to data from the previous year. So creating professional learning communities was the best way in Stockton to take a look at creating their data teams. And then of course they went through a whole training session on how to do a data team process under the umbrella of a professional learning community. Our counselors and transition specialists are essential in making sure that the data and the conversations that they have with students are documented in the system that your agency is using as well as making sure that your data goes to the next agency. So in this place, it would be San Joaquin Delta College. So there is a coordinator from Delta College who helps with this transition. And so they're able to share this data of students who are going from one agency to another. That was essential and critical. And then the other part is making sure that our students are part of the data system, that they are responsible for their own learning and that they know what their student assessment is looking at. What does that look like? And that they have access to their cost of scores or they have access to their modules on EL Civics or with their whatever it is with their GED prep classes, their high school diploma classes or their ESL classes. Where are they showing evidence of learning? So you wanna make sure that you include all of these folks in your data collection process. Now you can start this at the agency level and then you can expand it to the consortium. But you need to make sure that your agencies are comfortable and you've created that culture of data. I think that's really key. So thank you. Thank you so much Carol for sharing your expertise and experience with leading data teams, developing those PLCs and data teams and things like that. We're gonna transition a little bit into talking about the tools that we use. So really quick before I jump into the tools I wanna share with you, I wanna know from you all, what tools do you currently use technology wise to collaborate with your data teams or those you're sharing data with? And you can respond in the chat or if you'd like to unmute and share. We love hearing voices and seeing people on camera. Hi Suti, it's Peg from New Jersey. Hi Peg from New Jersey, I love Jersey, go ahead. Okay, so I just want to give an update and I can give full credit to Jay from CASIS for this. At our college in New Jersey, we use individual skills profile, personal score report, student competency report and student content standards all from CASIS of Tops Pro Enterprise. Great, awesome, thank you so much. We're gonna talk a little bit about those CASIS and Tops Pro and TE tools in a moment. So I'm glad you brought that up, wonderful. So let's go ahead and, yeah, Jay mentioned we'll be looking at all of those. We're gonna go ahead and talk about some other tech tools that can integrate with those reports and things that you have. So one of which is really it's actually the number one most commonly used platform for collaboration amongst colleagues and that's Microsoft Office 365. It's cloud-based or desktop-based. You can have real-time collaboration. So if you're inserting your data reports into say an Excel spreadsheet or an access database, you can manipulate that and share that with your colleagues at the same time and do that data analysis and looking at it in real-time. It's also very, very data import friendly, like I mentioned, it's browser friendly too. So if you're using a Chromebook and you don't have Microsoft Office installed, you can still utilize it on the web. It has the ability for teams to collaborate directly within the same documents, including on spreadsheets, which is really big when you're looking at data. It has Microsoft Teams integration. So if you're having your data meetings through Teams and you're in Microsoft school, it's really cool to be able to be in Teams, demonstrate what you're looking at and be able to see each other face to face and have a real conversation. Microsoft Office is also browser, Office 365, I should say, is also browser computer and mobile friendly. That means it works on Chrome, Firefox, Edge, Windows, Mac, Mac, Android and iOS. If you're using Internet Explorer, I don't understand why, but if you are, you might run into some difficulties with Microsoft Office 365, even though it is a Internet Explorer, is a Microsoft product, it's no longer being updated. So I encourage you to shift over to Edge if you really wanna use a Microsoft browser. The interesting thing about it is it does require multiple applications to utilize the full suite of programs. So for example, if you have a presentation you set up to share some of the data information and you also wanna be able to utilize the spreadsheets and other functions, you do have to have multiple applications to do that unless you're using the browser front. It's also not free. So that's the kind of a downside to if you're using Microsoft, if you wanna use Microsoft Office 365, but you're not currently an Office user. And so I wanna show you kind of what it looks like on the front page of it. So this is just things I've been collaborating with colleagues on. This is an onboarding tracking chip thing that we use. And it allows us to kind of see where people who are coming in are at in the process, what kind of supports they're needing and things of that nature. We collaborate on all kinds of things at CalPro. So it's really helpful to be able to have everything in one space on the browser, even though I have other applications I can use too. It just helps me get to the most frequently used or currently being used collaborative spaces that I'm utilizing. So that's Microsoft Office. Then we also have Google, which is the second most popular cloud-based collaboration tool. I know no one is shocked by that. Google also allows real-time collaboration on a lot of different fronts. You can use multiple applications to utilize the full or mobile or mobile versions if you're using outside the browser. If you're utilizing the browser tools, then you don't have to download anything extra. But if you're trying to use it on your phone, which I do from time to time, then you do need to have all the different Google Docs, Google Sheets, all those things downloaded and all reported to use it. It is data import-friendly though. You can import any piece of, any kind of spreadsheet or CSV file or things like that directly into Google Sheets if you wanna be able to look at that in real-time and collaborate on it. You can also use Google Meet with your Google Sheets so you can look at your data and collaborate in real-time. It's browser and mobile-friendly. So it works on Chrome, Firefox, Edge, Android and iOS. But remember it's not desktop necessarily-friendly because there isn't a separate application you download to your desktop. You do need to have an internet connection to utilize this. And of course, if you're collaborating on this, you wanna be able to use it in a collaborative space which requires online connection. There's no paid subscription so you can use this pretty much for free unless you're utilizing Google Workspace or Google Wine. And all those really do is give you a lot of extra space for storage because there is a limitation on storage on the free version. However, you get a pretty good amount of storage for free. So you don't necessarily have to purchase anything and you can still use your data collaboration here. What I love most about Google is just the fact that it's so easy to get to it. You can simply go to, I am on the wrong page. Sorry, I was playing around with this. You can simply go to google.com, click this little icon here with your Google Apps and you have all the tools right there at your disposal, easy for you to access. You don't have to go far and dig for it or look for it including your Google Drive, which I was just on a moment ago that you can quickly access and utilize. I use Google Drive really regularly with the CalProtein because that's how we do a lot of applications and tracking and things like that. And it really makes it easy for us to pull information externally without having to have access to SharePoint and our Microsoft products and still be able to collaboratively look at that data as a team. So that's also really convenient. So on to our next tool, this is Slack. We used to use Slack quite a bit before we switched over to Teams. So unfortunately, I can't show you my Slack spaces but I can show you the setup in just a moment. Slack allows for is really great for collaborative conversations and for being able to integrate into other areas. So it allows for sending of direct messages and files to an individual person or a team. It's not expensive to set up. It's actually the free version. You can add a bunch of different people to the team. It doesn't limit the number of team members you can have. There's no real-time collaboration on the documents but there's a workaround for that we'll talk about in just a minute. You can organize conversations into various channels. So for example, let me show you what that looks like really quickly. So if I'm in Slack, I can set up channels so I can have things set up so I can have conversations with everyone. I can have specific people, in this case, group by interest or I can just have a randomly selected group of people for whatever reason if I was trying to do a social thing or something like that but it lets you set up different channels. So if you wanted to make sure that only specific people had access to specific documents you were sharing you can easily customize it that way as well and keep that into a closed group. It does support... Yes, thank you, Richard. Slack is a great antidote to endless and confusing email threads. It really is. It really helps you really kind of group that conversation and make it really easy to navigate. It supports video calling so if you wanna have a quick video call with your team that's in that channel you can do that, you can have file sharing it's also compatible with other services so in the event that you don't that you do wanna be able to collaborate on a document in real time you can leverage your Google Drive, Dropbox or Box which is also a file sharing source in order for you to jump in there and kind of do the real time collaboration while you're having your video calls. So even though it doesn't have it built in it can be leveraged in other ways. The free version does have limited limitations which include the number of messages it keeps and the file storage space. But if you're integrating it with something like Google Drive or SharePoint, Dropbox or Box. Then you can easily get around that whole file sharing that storage space issue. Additionally, it does clear out the number of messages after a certain point which is kind of typical a lot of our internal firewalls and security setup automatically delete messages after a certain date or after a certain amount of messages. So it works kind of similarly. There's no data import so you will have to use a third party to trying to collaboratively look at the data and import it. But again, Slack is compatible with so many different products that it's really useful in that way. It's a browser computer and mobile friendly. So it has Chrome, Firefox, Edge, Windows, Mac, Android and iOS compatibility. The apps are super intuitive and very easy to use. So I really like Slack. We stopped using it because we switched over to Teams but I still kind of like and I think it's fine and so I wanted to include it here. So let's switch to some kind of planning and process tools. So when you're trying to look at your data and you're setting goals especially if you're doing SIP planning right now and you really wanna make your SIP smart goals and smarty goals if we're doing inclusion and equity which I hope you are that we're also thinking about the process of doing this and we're setting actionable steps to work towards those goals. And so the next few tools I'm gonna show you are really great for project planning to look at our data. So Asana is one that's really great for data project tracking. You can create a to-do list for ongoing projects and set reminders from upcoming deadlines and send requests to team members. So for example, if you're wanting your teachers to look at some specific data to identify areas of growth that they have or say for example your individual PLC teams to figure out what their areas of focus are you can give them a specific deadline to look at the data and get back to you on where they feel like the areas of focus are. Of course you wanna be involved in that conversation too and confirm with your own eyes looking at the data that aligns with what you're seeing and that we're not going by assumptions. You can also use Asana to organize your data projects and list them board format which I'll show you in a second. It includes a search function which I find super useful because if you have a lot of data products going or if you're the data manager or the administrator and you have teacher teams looking at data and doing their own projects it makes it a lot easier to locate those data sets and those specific projects just by being able to utilize a quick search function. And it works way faster than our file explore search functions that we have on our typical computers. There's no real time collaboration which is a little unfortunate but you can't integrate it with some other tools. You have to have other tools if you wanna work collaboratively. So for example, you could provide a link to where the document is stored in your Microsoft Office 365 or your Google Drive and Google Files areas and things of that nature so it can easily integrate but the collaboration doesn't happen within the documents within Asana. There's no data import but again, you would link it with your other tools in order to make it work. It's a browser computer and mobile friendly so it also works on Chrome, Firefox, Edge, Windows, Mac, Android and iOS. So let me show you what Asana looks like. So this is Asana and you can select your and have your different tasks and teams and things like that set up. Have your to-do lists and things like that. I was playing around with this and notice I had my completion of my TDLS slides due by February 24th. I did finish that. So I probably should have marked that as, oh, I did mark it as complete. So I did that, it's all done, see. So then it disappears but it's really quick and easy to use. It's very intuitive, doesn't take a lot of time to learn and the free version does a lot. The one downside about the free version is the number of people you can have collaborating with you. If you're a small school, it's super useful because you don't have to have 10, 20 people in your data sets but if you're a larger school, Asana may or may not be the choice for you depending on your budget because you do have to have that you have to pay to have a lot of people added to your data sets and your data projects. So the next tool I wanna talk about is called Trello. Trello is also really good for data project tracking but I like it a little better than Asana because of its ingenuity with its compatibility with being able to pull stuff directly into it. It's great for project tracking just like Asana but and you can also organize your spaces by teams or tasks or roles similar to Slack. So if you wanna have all your things going in grouping things based on who you want to have access to it and those kinds of things that functionality is already built into Trello. You have the options to assign comment cards which is a way to give feedback on individual things. So for example, if you assign something to one group but you want a separate group to be able to give feedback but not manipulate the data or play with it in any way you can allow them access to comment cards that they can use to just give you some feedback. It's super compatible with a lot of different services including Evernote, GitHub, Google Drive and Slack. So if you wanna be able to integrate things it's really great. It has free version limitations which includes storage space and file size limit. It does not limit the number of people you can have on there on the free version. So that's pretty cool. The storage space and file size limit again you can get around that simply by linking to wherever you're actually storing other documents and then be able to get to it that way. So if you're using Google Drive for example or GitHub or Slack or Evernote you can quickly and easily integrate that without any real difficulty. There's no real-time collaboration on documents which is kind of unnecessary with Trello because it does integrate with so many other things.  So while there is no real-time collaboration on documents within Trello it does integrate really nicely for real-time collaboration on documents. It is however only browser and mobile base which means you either have to use it on your browser or you have to use on your Android or IRS. IRS, not IRS it's taxis and sorry y'all. I-O-S, but that's also not that big of a deal if you're trying to use this in a collaborative space. So I wanna show you what Trello looks like also very user-friendly. So this is Trello and what Trello has is all of these really cool ways to set up and look at the status of your different documents and areas. So you'll notice that on this particular project versus marked as blocked and they have this cute little cat gift you can make it as creative as you want but you can also have your to-do blocks, pending blocks and things like that. And this is on my project management board. You can select a bunch of different types of boards and you can customize who you wanna allow it in there. And again, what I really like about Trello is just the fact that I can add a bunch of people and identify who specifically I want to have access to things without having to break the bank. Another thing that I use and this is one of my personal favorites is Monday. Monday is another online collaborative project management tool which allows you to have unlimited boards, unlimited work that WorkDocs, which I'll show you with that is in a second and real-time collaboration. It does a lot of things. It's data import friendly and compatible with almost all other collaboration data services like Microsoft, Google, Asana, Trello and Facebook ads. If you're using that for marketing and you're trying to look at your marketing data. So it's really great for integrating all these different things into one space. It has a very customizable workspace which is super intuitive and really easy to use. It is, I cannot stress this enough, ridiculously user-friendly, very easy to utilize with no issues. The free version does have a significant limitation, however, you can only have two team members collaborate at the same time without having to spend extra money on it. So you wanna make sure if this is something you're utilizing that you do have budget to add in extra team members if you're trying to use this as a collaborative space. Again, it's browser, computer and mobile-friendly. So Chrome, Firefox Edge, Windows, Mac, Android and iOS compatible, super user-friendly and works with so many other tools that it's really great to be able to integrate. And if you're someone like me who works on projects and various products use different tools, Monday is really cool for being able to pull all those into the same space and to limit a little bit of confusion. So I really like Monday for that reason and most people don't like Mondays but I like this particular Monday. So let me show you what that one looks like. So if we're looking at my Monday workspace for CalPro, we have some modules that we're updating including our motivation and persistence for adult learners to massage in the necessity to focus on motivation, persistence and online and blended learning. And that'll be piloting this summer just in case anybody wanted to know. We're also up to our IET module too to pull in some new resources. And so one of the items that we are stuck on here it's not that we physically can't get it done but there's some issues with 508 compliance. And so we're having our publications department look through everything to make sure it's 100% compliant before you move further any farther. And so we can see what was done, when it was done and where we got stuck and what's happening there. And I looks like there's a new comment. And then our IET one, we are started working on it on February 28th. So that's where we are doing a content review. And then once we get further in there we'll be able to do some more, we'll add in extra steps and mark things that are complete. It's just really intuitive and really useful for tracking and being able to utilize things. You can also add in so many different types of documents and that when I said work doc those are the different types of documents and dashboards. And you can also import your data from so many different locations. So if you have these wonderful data reports that we get from costless we can totally import that so we can utilize that filter it to figure out how many people are passing certain competencies and not others identify equity gaps and those kinds of things. It is really, really super user friendly. So I think that's why I like it so much because anybody can kind of pick it up on one day and start playing with it and be able to utilize it. It's very user friendly. So with that, I wanna turn us over to Jay who's gonna talk with us more about what costless resources are out there and the different types of data you can download and other resources that are available to you that you can use within these collaborative spaces. I've uploaded the PowerPoint PDF into the chat a couple of times just in case anybody missed it. The links to all of this are included in there so make sure you use those. Okay, thank you, great. So what we're gonna do is we're gonna cover two broad areas. Obviously we only have so much time so no, we won't go forever but wanted to cover kind of the two broad areas covering a few resources related to curriculum management and instruction then we'll kind of turn it over more to the quote unquote hard data side and look at the data portal and then we'll talk a little more esoterically about TE. That's kind of five or six hours we don't have getting into all that but I wanted to talk a little bit about how you might go about TE. So if you don't mind Suity, I don't know if you wanna give me access or whether you wanna just go to that web page, your choice. It's up to you, I've got it open for you. Okay, go ahead and go straight. That'll probably make it easier for everybody else. So we're on the curriculum management and instruction here first. This one? Yeah, so go up to just click curriculum management and instruction right where it says, yeah, right there click that link. Yeah, there you go, great. So this is the page I'm starting. So this is our COSIS website. So I guess, because we were talking through that a minute for a few seconds, I'll get everybody split on the rail. I assume we've all been here. I shouldn't assume too much but has everybody here been to the COSIS website before yes or no? Do these guys look familiar to you all? Yes or no? Just kind of checking. All right, thank you, Lorena. Okay, it looks like I'm seeing a lot of yeses. So great. So we've all been to the COSIS website to a large extent that's, because you've been there whether you like it or not. I'll say this is the section that perhaps you haven't been to, because this is the section that kind of is completely out of that like it or not part. We will go there here in a few minutes but this is the section called Curriculum Management and Instruction. To be clear, when you go to our homepage there's a section called Product Overviews that you click. And then when you get to Product Overviews you then select Curriculum Management and Instruction. That gets you to this page. That's where a lot of our instructional resources are kind of the underutilized part of the COSIS website, so to speak. So what I'll do is like all the other sections of our website, there's that list of the left of all those little blue breadcrumbs with all those typical categories. So I'll just sort of talk through the categories and then point out a couple that are of specific interest. So just going down the list we have our two different kinds of content standards. You have what you might call the old school COSIS content standards from 10 years ago that are still embedded in our life and work tests. We've got the newer college and career readiness standards that relate to our goals tests. We have COSIS competencies that of course have been there since 1980. We have skill level descriptors. One that I'm not gonna dig into but one that has gotten a little air play lately and is one of the most underutilized COSIS resources, period, are our low level literacy modules. That is something we did with developmental services for adults with disabilities and or low level ABE and ESL, lots of resources to provide teaching, using a lot of picture symbols and categorizing it by different survival symbols rather than requiring reading of text and so on. Then we will dig into quick search for a few minutes. That's another heavily underutilized resource that we used to promote a lot more than we do now. So we will look at that. Then sample test items, information on TE and some agency stories. So the one I wanted to look at first, I think was that college and career readiness. I think that's when we discussed as a team that would be good. So this is what's new. You can see it's been around for a while. I think the resource we use in our TE reports is 2016. That was when we always started. So that's when we're relating the official use of college and career readiness standards for all of adult education. On this page, it goes over a lot of information about those career readiness standards. And what we obviously, what we like to emphasize at CASIS is the fact that the college and career readiness standards are so closely related to content standards that we've been emphasizing for much longer than CCRS has been available. So we have some cross-watch that kind of allow you to link one to the other. As well as how a lot of our CASIS system are related to those college and career readiness standards. So lots of good resources for you here. The next one I wanted to show is right below that CASIS competencies. And so this is the oldy, but goody. We've had competencies, six just about our inception in 1980. A lot of you probably use these, but this gives you the everything you wanted to know and more about our competency system. You have the PDF of the official list. There's content standards again. It gives you more information about those different competencies categories and competency content areas. So you can dig deep here and get that list and look at more information about that. The final thing I wanted to touch on in more detail here is right below that where we have quick search online. And so this is what I wanted to touch on maybe a little bit more than the other stuff here is we've got the links here to quick search. So I'll just stop here for a minute and just kind of curious how many of you have used quick search before or are familiar with it? Yes or no? Is this something that you've worked with before or is this a complete, you've heard it here first? Just kind of wondering. Okay, thank you. I see, okay, so we're a mixed feedback. Right now it's looking split down the middle, about 50% yeah, it comes down, about 50% yes, 50% no. So I'm glad there are a lot of noes because obviously that was why we wanted to talk a little bit about it. So this has been around CASAS for a very long time. It used to be called the instructional materials guide many years ago. It was a big clunky binder that we updated every year that we used to present instructional resources. About 10 years ago, we got out of the clunky binder for a year or two. It was on a CD now for the last eight or nine years. It's been on our website free of charge. You don't need to pay a dime for this. You do need to have a login account on the CASAS website, but you don't need to pay. But this has been a constant resource at CASAS because we of course, they're more of like a smaller mom and pop organ organization. We're not one of the big guys like McGraw Hill or Stefawn or those larger conglomerates that have lots of money where they can publish lots and lots of published materials and training materials for relating instruction to their products. So we say, we just are never gonna be able to go there at CASAS. So we have the next best thing. We have quit search where you can search for instructional materials. There's lots of different ways you can do so. But the most common way is, back to what I heard Margaret talking about earlier, that is things like competency reports and context standard reports and TE. You have your students take CASAS tests, of course. You have those reports related to competencies and context standards that show you what the individual student or the class or the agency level, some areas of strength and areas of improvement for your students. So once you get that information, as Sooty is demonstrating, you can find those specific context standards and or you can find those specific competencies that your students might need to improve in and find instructional materials specific to those competencies or context standards of need. So Sooty's selecting a sample context standard now. There's another page here where you've got that program level and skill. This allows you to further the search and kind of look by modality if you want only web materials or only old-fashioned paper copies, whatever. You can select that. You can taper it for workforce or taper it to AVE or taper it to ESL, different skills of course, different levels. And so after you do that specificity, then of course you click quick search. I'm not sure what we had here. This is my search scenario. We'd probably mixed and match some of our different standards. I'm not sure what, but you know, okay, so there we go. So now you can see we select the specific competencies or standards and then we refine our search by modality or level and so on. And we've got a nice, tidy little list here of all the different resources we might consider for that specific area of need.  You can click the title or the publisher and that gives you a little bit more information. You can see some of them have been around for a while, but especially you can see some things here. You know, things like computation never goes out of style. So maybe not a biggie, but you get specific, a big list of all the different standards and competencies. They cover some information about the publisher if you want more information about that publisher and so on. And the last thing I'll say about all of this is we don't just kind of sit there in our ivory tower and pick our favorites or whatever. There is somebody who works part-time that reviews all of these materials, kind of like an inspector 12 or whatever. We don't publish any of our materials in quick search until you know, that person takes a look at it and gives it kind of a seal of approval. Yes, this really is what they say they are. Yes, this really does apply to adult education and looks like it should be something that would be productive in an adult-ed classroom. Okay, I'll just say any questions there, probably a time to back out of that and move on to the next item here. So back to the slide, we'll go ahead and take a look at the classes data portal. So here's our segue. I guess I'm sorry to have you keep hot scotch in, but if you can hop scotch to that link, put the data portal on. Just say I was kind of hoping we're right here where we're just about ready to post the most recent data from the 2021 year. I was hoping that would be done. I gotta say I was really hoping it would be done Tuesday when I did that statewide presentation with Carolyn. I was also hoping it would be done today, but oh for two, but nonetheless, we can still show it with the 1920 data. So here is the data portal. I'll back up a step. How many of you have been here before? Have you used the data portal before? Yes or no? Just kind of checking a little headcount constantly. All right, okay. Thank you, Lorena, Elaine, anybody else? Yes or no? I'm interested in those too. I'm kind of curious. Okay, some yeses. So sort of mixed feedback on the data portal, roughly 50-50 on this one, just like the curriculum management. So this is our class as data portal where we basically post end of year results from all we owe a title to agencies every year. We've got the top part I don't wanna get into, but I'll just show it because it's on the screen that California State, I guess you can click it, doesn't hurt, but this is not where you can get agency results, but it has our California statewide federal tables. You can see it goes all the way back to the inception of WIA back in 99, 2000. So you can look at all the fed tables, including the ones we never really talk about and nobody really knows about, but if you're like a researcher and you just what the official documentation of how we did in California for a given year on any of these tables, you can do that aforementioned research right here. Back, if you don't back up a step please, but yeah. So, but what I really wanna show is California table four, yes I know. So click California federal table four if you don't mind. So this is what it looks like if you select table four B or the persister, the look and feel is basically the same. It's just the different data, table four to be clear is where we would go to look at performance for these federal levels. You know, persister is where we would just look to see what percentage completed a pre-post test pair regardless of their actual performance. More on that later. We're not gonna cover that here, but I will have a slide that refers you to this information if you want more of it, but just to stick with, you know, what's here, you obviously start by saying start here. If you don't mind, we'll just do a sample. We'll select agency. We'll keep the example as basic as possible. Let's just select ABC. Again, we can't get any more basic than that. And then we'll select them. Oh, wait a minute. I'm pleasantly surprised. I gotta say, I actually looked at like seven this morning and this wasn't here. Some little angels looking out for me. Well, let's click and see what happens. Click submit, sorry, right there, but look, yeah. I mean, I am what clicks submit. I clicked it. His bottom portions came up. Oh, that's all that did. So click submit. Oh, there we go. I think it did. I'm sorry, I missed that. So, hey, there it is. Sorry about that. I made you do it twice just because I wasn't looking obviously or whatever. So you can see the 2021 is there. Thank you for pairing down my error. Anyway, that's our 2021 data.  Well, they're killing me. I'm the one killed here because I'm saying we didn't make it.  I'll just say that's what you can get. So I'll just say you can search by agency, by any of the 58 counties, by CDE region, by big or small, whatever, pair it down to, you know, if you select agency, you've got a list of all the agencies. If you select county, you have a list of all the counties and so on. And then program year any year from 2004 or five to now 2021 and anything in between, you can drum up the data. So you can compare logitunally. So if you wanna look at your own agency every year for five years, go ahead and knock yourself out.  So you wanna compare yourself to San Francisco City College or Riverside versus San Bernardino or whatever. You can do all those kind of comparisons with your neighbors, with your region, with your county and see how you stack up. We've talked a lot about SIP goals, Cape Goals. This is the best resource we think to be able to do those sort of performance goals. If I can just add really quickly. Fire it up please. And at the very bottom, don't miss these downloadable files that are here that you can import into your collaborative spaces to look at with your team. That's right. What SUNY brings up is there's that clip you can download to a CSV file or to Excel at the bottom if you wanna kind of save it. Yeah, just share with staff and do all of those great things that SUNY just said. And yes, Elaine, it's here. We'll have more here later. Hold that thought for a sec. Let's do the last slide though, just before I forget for goodness sake. So a little bit on TE, there's a couple of things. I mean, there is mostly just, I didn't wanna dig into TE too much here because it did feel like five or six hours. We don't have, there was a little bit of, hey, we wanna access the data in my mind if that's how we're discussing it. And I do think that was the seamless way with what Suti and Carol presented to talk about our website, not TE, but I will at least speak to it because it does feel a little bit like the pink elephant in the room or whatever. So what we say at Costas is we really want you to have control of your data at the local agency. So fittingly enough, we have each agency managed user access. Is there an element where we don't really wanna be responsible for all four or 500 TE agencies? Of course, we'd never get there from here if we had to manage it for everybody, but there is also a need for each individual agency to have 125% of the authority as well as 125% of the responsibility for your own data. So at each agency, you have that one data manager. Hopefully you have two or more, but you have at least one higher level manager that has that access and control to things like payment points and fed tables and all those wizards for data submission and so on. That person decides with your management team how many people have high level access, how many people might have lower level access. So to tie it to what Kara was talking about at the beginning, we would say that's a perfect start for some of those staff discussions about how to manage data, who should have access at your agency and to what level of access. What I'll bring, there's a lot of ways to do it, but I will cite one example. This is from a recent TE meeting we had, I think in January, Fabiola from Sweetwater did a video talking about those managing functional roles in TE, which is the term we use for different setups you can use in TE to enable administrators more access, they under-enable teachers more access, other support staff more access and so on. So that's one video about how she does it and at Sweetwater, they've kind of gone way out of their way as an example agency to make sure their teachers have access to the data and can review progress on her or his own students at that individual level. So I'll just make a plug and kind of tie it to that importance of having staff data discussions to talk about who has access and whether you want to be sharing that data by allowing more people more access to your agency's data. So I'll just stop on this one. I'll just say anybody with any comments or questions, including, you know, Suti and Carol of course. No, I think you make a lot of really relevant points and I think that's a discussion you want to make sure those who are listening that you're having with at your agency is who needs access to data? Who needs what level of access to the data? Does that mean they get reports supplied to them electronically or in their boxes? I know some schools are still doing that or does it mean they need to be able to download their own data? And so we want to make sure that all staff that are impacting students have access to look and review data in a part of the ongoing data conversations from data managers, administrators, to teachers and that are directly in contact with students. So make sure you're having those conversations. If you're a teacher who doesn't have access to your data, then go back and ask about how you can get access to it.  Thank you. Okay, so there's one or two more slides to try to kind of tie it up, I think. So this first one, you know, I'm referencing that presentation. We did it the summit. There's one of those nifty slides where we did the reverse panel discussion. So a few of you, I think are doing this for the first time, but there's a few familiar names where I think you have been at one of these previous sessions or at least heard us talk about it at network meetings. So we did that presentation at the 2021 CAPE Summit. Again, the reverse panel discussion where Sudie, Carol and I kind of had you grill us in a way rather than the other way around, you know, we kind of went both directions, went inside out, kind of got a lot of feedback from you. You know, those were our two basic questions that dug out a lot of feedback that were a lot of areas of need, one of which was more resources. You know, that wasn't the only thing, of course, but we do think it was one of the biggies. So here we are at TDLS thinking this is the perfect opportunity to take care of this bucket. So we're bringing up online resources. There's also a panel we'll be doing sometime in April, you know, we'll be having a panel with best practices, you know, we'll focus on, I'll just for the easy way out, I'll say it'll focus on a lot of the things Carol brought up at the beginning, what are some best practices with engaging staff, you know, having more internal mentorship and collaboration will have some panelists to address that. And then we're hoping to have something more interactive at the classes institute. I admit we're still kind of chewing on exactly what it is, but we're looking to have something more face-to-face and interactive where we can start hopefully getting into more active, you know, working on mentorship and collaboration, you know, maybe externally across agencies. You know, I guess I don't have anything past that. We'll have to see what that yields before we'll move past that. But I'll just say, I think, you know, if this works well, we'll hopefully keep moving forward after that as well. The one more is a little more same idea, but more brass taps that are coming up. So the first one, you know, doesn't really relate to what we were talking about, but it does relate to data. That is the illustrious employment and earnings survey. Hey, it is our official CDE action plan with Octay. So, hey, can't get more important than that. So we are gonna be doing a training and a panel discussion next Tuesday there. The ones I mentioned for the 9th, 21st, 23rd called NRS Performance Goals, those are ones that I do. In particular, there's a couple sessions labeled part one that are the most basic ones where we get into the specific mechanics of how to access data on the data portal. There are other sessions where we do a deeper dive and talk about performance improvement at agencies, but I really wanted to reference those so if you want more information about how to use the data portal, those will include sessions where we'll really get into that in more detail. We also have a panel on workforce collaboration here toward the end of the month. We'll be doing some goal-setting things with Cape on those dates and then some panel discussions on, you know, like we talked about one that will be creating a culture of data. We'll also have one that really focuses on student focus. We've got some really good panelists with kind of in the final process of confirming panelists for these two. So here in the next hopefully weaker cell we'll have nailed down the panelists and we'll be able to post these, but gotta say I've been talking to a lot of people that's really shaping up good right now with some, you know, definitely some superstar panelists for these occasions. So I think it's 929. Hey, there's a minute left so I better turn it over just in case hopefully a minute is all you need and I didn't need to have turned it over sooner. Nope, that was great. Thank you so much, Jay. And thank you, Carol, both of you for bringing your expertise to this panel and sharing this information about why do we do this? The data that's out there and the tools we can use to look at it. Thank you both so much.