the, what is it, August 23rd, meaning of the Act 65 working group of the racial disparities advisory panel. Let us introduce ourselves. I will go around the screen. Evan, you're first. Okay. I'm Evan Menon with the Department of State's Attorneys and Sheriffs. Good evening, everyone. Thank you. Jeff, Jeff, you're muted. Some people would like that always to be the case, but I'll turn mine on. Jeff Jones, I guess one of the original appointees at large, more or less, at this point. Right. Thank you. Tyler. Good evening, everyone. Tyler Allen, Department of Children and Families. And this is actually my first time joining the working group. I'm sorry I haven't been able to join the past couple of iterations, but I am glad to be here. Thanks. Rebecca. Hi, Rebecca Turner from the Defender General's Office. Hey, David. Hi, everybody. David Chea, representing the Attorney General's Office. Good. Okay. Now, my computer has decided to do something weird. Which is that you? It is me. It is. Yeah, hello. Yeah, we try to pronounce him his name there's Athens resident. Yes, that is in Vermont. And data warehouse specialist. Thanks. Great. Well, for some reason, floating over letters right now is not bringing anything up for me. I-L. Hi, Ian, Laura. I'm Beton's summer assistant. Got it. Hi. T.A. Oh, we are- I think that might be me. Mm-hmm. Yes. Zuzana. Hi, good evening. Zuzana Davis, Racial Equity Director. Sheila. Good evening, everyone. Sheila Linton. She, her, hers. Root Social Justice Center, a pointy at large. Great. And O2. That's the work of people. Oh, okay. Work of media. If I have forgotten anyone, please let me know. Or like- Hey, it's on Robin's on the phone. Sorry. Okay. Thank you. Hi, Robin. Hi. All right. Hi. It's Monica. We talked earlier, but I'm here too. Oh, I'm sorry. It's just- It's okay. It's okay. Karen's here too. Hi, Karen. Hello. Yeah, the computer's just not doing it tonight. Thank you. Yeah. Well, to begin. I, as you know, I took the thoughts that were forwarded to me by all of you and started in on writing, on writing a draft that would be specifically focused on questions two and three of Act 65 or that particular page of Act 65. I can't remember what the page was somewhere in between 18 and 23 of the Act. That was what we had decided to do was, I mean, not last week, because that was the holiday, but the week before was to focus on questions two and three. There was broad feeling that that would help us answer the first question where the Bureau should be situated, taking into account the necessity for independence and the advantages and disadvantages of being a standalone body or being housed in state government. Four and five, I didn't touch. And nobody else did in what they sent me. So I sent that along. It dawned on me at the very last minute, like literally half an hour ago, that I didn't resend Rebecca's drawing, but I figured we all had that. That might have been really a mistake on my part. And if so, I'm really sorry. But I remember that from three weeks ago that we all had that drawing. And so I just made an assumption which may have been very wrong. But then I also, as you know, put in Monica's drawing. I did not try to synthesize any of that, the drawings. That is something that I'm going to leave to someone else to do because I'm really good with abstract thought. If you give me a picture of something, I don't know what to do with it. It baffles me. So for those of you who are visual learners, this is your moment to shine because I fail miserably. So I was just hoping that we could have a discussion based upon the draft of what's been written, talk about its failings, talk about things we like about it, things that are left out of it, things that need to be put in it, so on and so forth, and then leave the discussion on two points one. And after that, I'm hoping to talk about points four and five, which I think needs to bring in Karen and Robin. Those are the points that Mo Stein had been talking about very early in this process, where we had discussed that we might well very need, we needed to have these questions that we're working on now finish before we got to those. Does that sound like a reasonable strategy for this evening? Okay, well then, given that I came up with the contrivance of what people have written, I'm going to stop talking now and let you all weigh in. Well, I guess someone's got to go first. And I will preface my remarks with a, I think you've already received at least one, Tom, but a big thank you for taking the first crack at this. It's usually a thankless job, but thanks to you. So I appreciated it. You're welcome. And, you know, overall, I thought there was some good language in there. But, you know, as I was reading it again, I had a question which I know I've asked before, but I'm going to put it on the table again by citing a couple of statutes that I went back and revisited. And I guess my point really goes to the first question we're supposed to answer, which is where should this be housed? But some of the language that you drafted, Eitan, was, is really, really similar to some language in the enabling legislation for the executive director of racial equity. For example, there's a statute that says there is created within the executive branch the position of executive director of racial equity to identify and work to eradicate systematic racism within state government. And then there's also a statutory requirement for the director to manage and oversee the statewide collection of race-based data to determine the nature and scope of racial discrimination within all systems of state government. And then a final statute that requires the director to work collaboratively with state agencies and departments to gather relevant existing data and records necessary to carry out the purposes of this chapter and to develop best practices for remediating systematic racial disparities throughout state government. And then there's, you know, even some language regarding the confidentiality of information, which I think is something else that we're supposed to make recommendations on. So it just seems to me like there already is an entity in state government that is supposed to be doing some of what the bureau, I think, is going to be designed to do. And I know that there was some concerns expressed last meeting about whether or not that entity was independent enough or whether or not it had sufficient resources. But it seems like, you know, we could make recommendations that those two things should be adequately addressed so that it could carry out its existing statutory obligations, the obligations of the bureau while maintaining a sufficient amount of independence and a sufficient amount of resources to get the job done correctly. So that was sort of my initial overarching thought. Okay. Thank you. I need to put on chat, don't I? Because I won't know who's got their hand up or anything. Excuse me. Okay, there I am. Thank you.  Did I surprise you? No, no, no, it's good. I'm finding the buttons. I actually wasn't going to address seven your point. We have thousands of questions here. If you have any questions, you can go to the chat and the chat is up there jumping in and going to number one in terms of, I took, so your comment or your suggestion as we think of the entity being placed, the bureau for calling this simplistically, the bureau should be placed in the, in Susana Davis's director. Is that what you're saying here? Do you want to start a discussion on this, or do you want to go? And I was kind of making that suggestion, but I don't want this suggestion to be perceived as saying that Susanna's office should just house this entity and should just do the work in its current configuration. If her office needs to be reformatted or retooled or it needs more resources, then that might be a worthwhile consideration to avoid having multiple entities with overlapping or conflicting jurisdiction that could make resource allocation less efficient and more costly. Sorry, I'm trying to write on this as you're talking and put notes in. In terms of where to go, I feel like start wherever. If you want to talk about number three or number three, let's just go. I don't feel like there needs to be a particular order to this because it's all sort of intergenerative, if you will. Oh my God.  I'm seating the floor for now. Okay. Thanks. I think part of the reason I did that drawing that Eitan sent out was mostly for myself was to sort of understand the way I was thinking about it because to me it does hopefully help us get to where the bureau should be situated. And in the drawing what I was pointing out was that there is data that's going to come from the executive branch that's going to come from the judiciary and that's going to come from a bunch of outside entities or standalone entities and that at least for those departments that are within the executive branch, there's going to need to be some very strict rules around the sharing of data and how the data goes from DCF, DOC over to the bureau. And when I was thinking about that, that means ADS has to be involved from a structural component and that ADS could provide the support that they provide. Again, not that it should be housed there, but they would have to be involved. So that just kept leading me to needing the bureau to be housed somewhere within a state's government organization in order to also help provide the things that you were talking about the last meeting Eitan, like who is going to sign the bills or order whatever needs to be ordered and all that. That administrative function has to happen somewhere unless the legislature is going to really create a very large standalone entity which I just don't see it doing. And so, you know, in some ways I think question one helps, you know, they're all linked together. I think we would write the mission slightly differently depending on where the bureau was housed. So that's why I was connecting those things as well and let me to the drawing. But overall in terms of the way you put everything together, I like all the concepts that are on this document and I think we could really refine it if we kind of got a little bit more involved in the other question. Yeah. I agree. Witchie. Yeah. So I kind of really, as soon as the office of Rachel Equity got mentioned, it kind of really made sense to me that that would be where it's housed. I feel like, you know, that office is really in charge of making sure there's racial equity in Vermont and administrating it and I feel like that sort of sounds naturally in there and I like the idea of ADS having a say in how that governance goes, how it comes in, where it goes out because that's their skill set. I do also feel like there needs to be some legislative representation given that this is forming because the legislature wants this to be formed and I feel like it could be very helpful to have the legislature be able to pose some research questions. So I'm wondering if there can be some type of multifaceted working group, like we were talking about an advice group that included stakeholders, what about of those stakeholders included folks, a representation from the legislature and I don't know, not to show how the branches work, but maybe some representation from the judiciary branch also, so like it's sort of like a collaborative effort in making sure that we have research questions being placed by everyone in Vermont and also that the administration is being done by the administration. Oh, I'm sorry, I'm just, I think I just got lost with you and I'm just sitting here thinking about it and not running the meeting. It's okay, I'll get it, thank you. Rebecca, do you want to go now? Sure. You know, Evan, when you started the conversation with going directly with where should this be housed, I was sort of finding my footing and hearing others' comments about it. I think really where I zeroed in on both of these documents that Etan sent out for us this week and again, Etan, I do also appreciate you pulling all these thoughts together. And I love it. Here's where I highlighted a key phrase that I think can center us in the discussion where whether we want to go to the entity placement and I think Bono for someone else, it depends on what we're doing. I'm looking at, if you guys have the first page of Etan's document and it's the first paragraph, you know, he starts saying the charge of the bureau is to collect, blah, blah, blah, right? If you go to the second and that's the last sentence, it's the last phrase with the intent, right? So now we're getting to what is the purpose of this bureau and the way I read how this is written, I think that then gets us to looking at back to our constant theme, which is independence. A reminder, my recollection of two weeks ago or the third week, Susanna shared that there was draft legislation and then the legislation that passed creating, you know, her directorate and how she shared that it wasn't as independently conceived as it was in draft form. And so I thought that was a useful red flag. Again, just avoiding sort of, let's go to the easy place to where this lands and thinking first, what is our intent for this data? For me, it certainly is more than just having access to it. I don't think there's enough safety for the people who's represented in here and the interest by just that too big of a statement. Okay. Thank you. Evan. Thanks, Aetan. I just wanted to follow up on what Wichee said because, you know, one, I agree with him. But two, it was actually the mention in Aetan, your draft about having, what was it? Maybe it was called an advisory panel or a governing body that made me first think back to Susanna's office because there is a racial equity advisory panel and it has to include a member appointed by the committee on committees who's not a current legislature, legislator, a member appointed by the speaker of the house who's also not a current legislator, a member appointed by the Chief Justice of the Supreme Court who's not a current legislator, appointment by the governor and then an appointment from the Human Rights Commission, both of whom cannot be current legislators either. So it seems like some of the entities that he identified or contemplated, the legislature, the judiciary, so on and so forth, you know, maybe that, if Susanna's office is going to take on the bureau, whatever it ends up being called, you know, maybe that panel, how that panel is put together needs to be changed or modified because my recollection was the same as Rebecca's that Susanna said the independence language could be beefed up and I think we could certainly make that a recommendation if this is where we're headed. But again, that was sort of another similarity that I recognized, you know, a governing body for this entity and there's already a panel for Susanna's entity and, you know, maybe we need to make her entity a full department, you know, within another agency and her position would be a commissioner. I, you know, I don't know how the final format would be, but I imagine it could be beefed up in some respect to address her concerns. So that was my thought. Okay. Witchy. Mine is less of a comment right now, more of a question, which I think maybe Evan was starting to answer, but sort of, you know, Rebecca's introducing. And it's like, how then would, you know, if we're trying to learn lessons from what created the office of racial equity and we want to make it more independent, how does that look like? Like, how would we, if we were to redo it again, what would we do differently? That's a pretty darn good question. And my thought would be, let's talk to Susanna about that a little bit and see if she has any thoughts because she's living and breathing it right now.   No, no, let's let, I'm sorry, Susanna, that makes sense. I'll wait for you. You should go. Okay, Susanna. Good evening. So I've been here, good evening. I've been quietly listening here. So, yeah, I have a few thoughts. Number one is I would say that if we want to seek a greater degree of independence than what exists now for the racial equity office slash director, then one of the things we need to think about is fiscal support and how that's going, how we're going to make sure that that's firmly in place. Because one thing that I found is that with independence also comes kind of like, oh, you're, you're out of sight or you're less in sight. And then you might be, I'm trying to put this really diplomatically, but when budgeting time comes around, you may not necessarily get the same attention that you might get if you were under someone or something. So, and that's not a reflection of anybody in any position currently in government, but it's just a reality of how our budgeting process works in this state. So one thing that I would think about it, and I'm thinking about, you know, like the Human Rights Commission is always under staff and is always telling us that they're always understaffed and yet, you know, who's factoring that in when we're building a budget every year. So I would caution us about that. The other thing I would say is, you know, I agree with Rebecca's point about it's not just about creating access. It's about what is that access supposed to get us or get the public? Is it just proof that our hands are clean because of transparency? Or is it actually going to serve a greater purpose by helping to, A, inform the public's understanding of what we're doing and B, inform policy itself? And if it's about informing policy, then theoretically, we don't need the access, right? Because if we have our own data, we can just use it to make good decisions. So clearly, I think it has to be both, right? It's a yes and so. So I agree with that point. And there were a there's a whole lot that you all talked about that. You know, made my eyes flutter, but I can't remember any more of it. So I'll stop there. OK, thanks. I guess. Hey, Tyne, could I? Yeah, could I go ahead? I can wait. Evan, when you said like what you rattled off the stakeholders who are involved in and and, Susanna, I have some crazy mind block over the name of your director. I'm so sorry. I avoid I just I just result and just call it, you know, your office. And I like that. Anyways, but you were rattling off by statute, some of those names. And then you said, how can we make this better? Right? What are some thoughts? I think for me immediately, what comes to mind, whatever we call it, I'm thinking of it as the governing board, right? The governing board is the core. It is driving the direction of what the executive director or essentially, you know, Susanna, whoever it is who's running it, right? It's directing that person. So then who's on this board? And I think it's critical that it's not just the people or those organizations and the leaders within those organizations, state organizations that you listed. We need also, of course, I didn't hear the defender general's office in there. We didn't hear any sort of the stakeholders that are involved in our depth, but critically, we need community members at that table with equal, with equal, unequal voice and also people with lived experiences, which are not necessarily one of the same, right? And we need those numbers to not just be a token number, right? I think that we should think about how that board will be sitting, voting, thinking the questions. It's the checks and balances in and of itself to make sure that what we're hoping people do with this data. And I appreciate, Susanna, what you said, you know, going further, it's not just transparency. It's building more informed policy decisions and I hope we can get to an end. Like, what are we hoping these policy decisions will be doing, right? Like, our whole point is addressing white supremacy in these systems, right? We certainly don't want this data and the policy to be used to further entrench, right, to add a different nuance to the same old, same old. So I think that's where I'd hoped us to go. Very bold statement, similar to where we were when we were reporting earlier. It's certainly what we do. Keep the focus on what the point of this is. I mean, certainly to me, why I am here on our depth, I feel like we're doing something real in terms of trying to address this squarely to the front, sir. Again, that is about being honest with how these systems are our systems of, you know, of entrenched white supremacy, right, institutional racism. So how do we want this access to data to address this, right? We should have that as guiding principles of this governing board. OK, give me one set. Thank you. I just wanted to write that down, Rebecca. Karen. Yep, I've got my hand raised because Robin can't raise her, so I'm doing it for her. Oh, Robin. Hi. Yeah, so to Rebecca's points back a few on intention and also what Rebecca just said, one thing that was missing out of the the great stuff that you guys put together so far is exactly what's going to happen to the data once it's released. And, Rebecca, I'm going to tell you, you're not going to be able to control what happens to that data once you put it out there in the public. Someone's going to use it for whatever purpose they want to. But what you can do, and this goes to something that Aiton asked me to kind of bring up every once in a while, is somewhere get either the advisory panel or someone within the budget to have a certain amount of dollars that they give to independent researchers for analysis. I would ask, and I don't know if Susanna Shop has like attacked the whole contracting with the state and how that might not be an easy thing for everyone to do, but to limit the barriers that independent scholars and scholars of color have to accessing data for research and funds, making that process easier and to have that kind of grant process be responsive to the board of directors or whatever you want to call them to a specific question. Right. So like what we kind of talked about before, you want to know our communities of colors being overpoliced. So here's how here's the data we need. The Bureau works with the ADS to get that data. The the funding body that has a certain amount of money to give every year to outside researchers, that independence that you want answers this question specifically related to dismantling white supremacy in the criminal justice system or, you know, 10 years from now with this advances beyond the criminal justice system to whatever housing, employment, etc. So that's one thought. A technical thing that we haven't really talked about yet, but I want to put a footnote in is any this entity is going to have to wait a few years before it gets enough data to do analysis. And the reason for that is that every day we expunge records and those records are expunge from the court, from from Monica, from GPS. And to do analysis on records that have been expunged is already giving you a bias that we can't really quantify. So this entity needs to not be touched by the expungement statutes. It needs to not be the official record holder so that they can collect, identify the data, de-identify it, but merge it all again, you know, across so that you have a clean data set that isn't touched by expungement. So while you're doing that, I just also want to point out that that years for all of this to happen, for this legislation to pass, for the funding to get forwarded, for the data architecture to happen, people are still in pain. And to have the the advisory panel until you can fund statistical analysis to fund qualitative analysis, to really start to collect information from participants in the criminal justice system about how they experience discrimination for the idea of policy change, because it's going to be a long time before you get a math answer. And I don't know that we want to wait. That's it. Thank you. Thank you very much. Evan. Yeah, I certainly appreciate Robin's comments, but I wanted to I wanted to go back to this question of the composition of the governing body, because I was struggling with it when I was thinking about it, to be honest, because I was trying because in my mind, it's also connected to the question of independence. And I think if we're going to have a governing body, you know, if we're going to have one, I think it's, you know, there's sort of two questions. How do we make it sufficiently balanced? And, you know, is it going to be an entity that can direct the bureau to do something or advise the panel to do something? Because I think that one of the things that could negatively influence the bureau's ability to influence policy is the perception that it's actually beholden to someone else, whether it's an entity in state government or, you know, stakeholders who are who have some sort of disproportionate representation on the panel. So, you know, and I don't have the I don't have the answers to that. It's because I was I was struggling with with it myself. So I'm more flagging that I think that the issue of independence should also be woven into our conversations about the composition of of the governing body. I I'm going to I just want to throw something in here that has been sort of creeping around in the back of my brain about what you're saying, and then for for, I don't know, a few months now, actually. And that is, again, I'm not sure that we need to reinvent the wheel here. And the reason I say that is because the wheel that has already been invented would be the what used to be the Criminal Justice Training Council, now the Criminal Justice Council that went into I mean, there was a radical restructuring of that body. And what, David, two years ago now? It was only nine months ago. Oh, sorry, I forgot the pandemic. But there was that, you know, where it had been 12 people and all of a sudden it became 24. And then there were all these subcommittees which advise it that cannot make decisions on their own, but really exists solely to provide information for this newly grown, frankly, body that and I think we're all still, I mean, I've certainly sit on one of the subcommittees and I'm like still deeply confused.  Who wants what? I mean, I really I'm not trying to be funny. I'm serious. I have moments where I have no idea what is going on. But people are liking that. And so I guess what part of me is thinking is it might be good to look at the legislation that reinvented the criminal justice training council as the criminal justice council, because certainly no one is standing around going. There's not enough representation from community members. No one is doing that. That for everything I have heard go on about the council right now, that is not a question. That's not coming up. So I'm wondering whether it might not be useful for someone to deputize themselves to go after that legislation and bring it forth to the working group. Because I think there may be a lot there that could be quite useful for us. The other thing that came up for me listening to Evan earlier was it may be very useful to, you know, in someone's free time when they're not lying on the couch eating bonbons to look at the statute regarding Susanna's office and apply it to what we're trying to do here. That's all. Rebecca. I wanted to something that Robin said about. That you can't control the data once it's out there. Now, of course, I understand what you mean. And of course, I didn't suggest that we could cross think that we could draft some legislation to prevent how people would interpret the data. But I think that we can do something more to ensure the responsible use of the data, recognizing that the ultimate policy decision makers are the legislators themselves. Of course, we're not trying to step on the separation of powers. We talked. I just want to make sure we don't lose a thread from one of our subcommittee meetings. Susanna, I don't think you brought it up directly, but someone referenced the work that you've been doing. What is the language essentially like ensuring that any new law that is passed and I saw this quickly and I was just looking in my email queue that it should be analyzed and considered what the impact is on furthering racial inequities or correcting racial disparities was something affirmative in the statement. And I don't know is that oh, thank you. You see you put something in the chat. And again, that's one example I thought of Robin. Like there was yes, of course, we can't let we can control it once it's out there. But we can guide the legislators in terms of what we want this data to be used for. We can also minimize the collection of certain data to make sure it's not too much, right? It turns into a surveillance situation where we can't protect the people who are actually people behind the data in terms of their privacy. I don't disagree with that. I mean, I've been a big proponent for racial impact statements for well, years now, and I think that they should be used. Again, putting a footnote there that it's going to be a while until you collect enough data that isn't touched by expungement. Oh, right. I had another comment for that comment. Yeah. So it really it does make me nervous about how much how like the comments about how long it will take to get to what we're talking about building, right? And I remember I think it was Abby Crocker from UVM at some point talked about we and we recognize this in our report. We identified all those crazy, all those decision making points in the system. Then we had we narrowed it with some priority decisions within that huge basket. Abby Crocker and others have suggested, I know, the Connecticut guy who came to talk to us about data. Like, start with what you know. There's certainly that AISP document toolkit was was saying, yeah, let's let's start with a low hanging fruit. What's currently available? What are the high priority questions we want answered within our priority list? Right. And so we have like the short immediate and with someone which we were talking about the immediate, the intermediate, immediate, intermediate, long term and in terms of the data collection and analysis. I feel like Robin, that could address those issues because I don't want to just be we got to wait until we're at the end and have it all there before we can produce anything. Sure. I guess so there's only a few sources of data that it aren't touched by expungement. And so if you start, so there's already a a data set that the judiciary gives to me, Monica, DCF and VCIC. That report is already run. That report can start going to the bureau once you set up all of the permissions. Right. So that report is already there. You can start with what you know. You know, if once it goes to the bureau, if you say to the bureau, don't expunge anything here, you can start to build a data set to do some analysis. However, so if I refer to those of you who get to sit through justice reinvestment as well, right, so their statistician has five years worth of data. And she's telling the group, we're not going to be able to do racial disparities on a county level because there aren't enough people in the data set. And so she's going to do disparities on a statewide level, but that's not going to help help people down the southern part of the state experience versus the northern part of the state or to get to the very simple question of how counties look to things differently. So, yes, there are data sets that you can start building on that judiciary extract can be merged with the DOC public data on a regular basis to avoid some of the expungements that happen there. There are things that can happen, but I do want to be realistic about the time that it's going to take for an entity well established and well funded and with the directors to do this. They won't start at least for what almost another year, right? So if you get this past next session, goes into effect July one, you can hire the people to do it. They start it in a year, right? So that's where my timing is coming in. And I just want to be realistic because there is the idea that asking for more data is a very wide supremacist act in and of itself and not just listening to people. And that if you push for listening to people while you are building up these data sets to do an analysis for the questions you want answered, then I think you are hearing the people who want to speak instead of waiting for the data to be worked upon. Thank you. Yeah. Witchy. Yeah. So I have two items that I want to sort of follow up on that Robin's been mentioning. I think the first is about timing. I think regardless, you know, whether we deal with because of expungement or otherwise, I think it'll take at least two to three years to actually build what you all want or what we want. Things like this infrastructure like this just takes time to create a foundation for. Once that foundation is built, then it's a different thing. And, you know, Robin, I don't know the work from your organization. And maybe you already have a foundation that the engineers or architects can can sort of like crossover and use. But in my experience, it takes it takes a few years in order for things like this to be even queryable. The second note that I wanted to make is that I really agree with Robin about collective people's narratives on that qualitative data while this thing is being built, not just so people are heard while these experiences are happening, but also because that is going to inform the data that's collected and how data is going to be analyzed. When we look at how data is traditionally analyzed and reported on it, it almost always tells you what, right? It'll tell you like, there's a correlation, right? Or there's like this that happens or there's this disparity, but never tells you how or never tells you why and that qualitative data is always so super essential to be able to tell that fuller story. And then it'll tell you, oh, we should probably get this data that we're not already collecting in order to help inform the why or help inform the how. So that's how I think that also can be useful. OK, that's important. Thank you. Oh, no, your hand was up. That's OK. Never mind. Anyone else or Rebecca, you've got. Yeah, which I appreciate it and Robin, and it's something we should it's the language like my my familiarity with this world is the earth is limited, right? Because when I think of data, this whole project, it absolutely has to include the qualitative side and that's sort of shorthand phrasing for the stories, the people who have lived experiences in these systems to be able to explain and contextualize the data and to me, absolutely. And maybe we haven't been clear or even come to a consensus on that point, but I think we cannot just go forward and push the sort of just numbers only, please, even what what we're talking about in terms of the governing body. So I appreciate that. I almost am thinking about how many different ways and places again, referencing back to the AI ISP toolkit, how many different places and ways do we build in assurance of that voice, that it's not lost and not just that it's heard, that it's waited, that's included, right? It's not just considered, it's not just noted in a report. It's actually integrated into the whole thing. And I think that's another part of that a key part, which is I love that. Here's another question. I appreciate the two to three years. What if we is it possible that if we agreed within our priority group, if we nail down one, two, three questions, right? With assuming that data was available now, Rob, in acknowledging the expungement issue, maybe that will be a different issue, which is that there might have to be some tweaking with the expungement legislation to allow for this to work in the future, right? But not getting bogged down in the mechanics, even though I understand that's a very huge issue. Does it make sense that we identify, again, referring to the experts here? Because I've heard a little bit different, which is, again, seen Connecticut's example, which is not obviously the same what we're proposing. They don't have their own set up with entity. But using existing data, can we find that falling within one of our set of priority questions or decision making points that we were hoping to collect it on, like what if those match and what if we were like, oh, then let's focus on those two. Could that be even faster within a year? Is that realistic? Could could we use one tangible question as a way to build this? So a year after, so let's start so because in my mind, I'm thinking let's start the let's start the clock a year after the legislation passes. Can we start the clock there? So that's what can what can be happening a year after the legislature authorizes this? So let's just start with July of next year. And no, I mean, so when I went through about what data are being collected, one of the things that ADS has been working on and will need to be done and in which you may have a better way to explain it than I will, is and the sounds awful. But it's what's called a master person index. And in Vermont, we've been very opposed to that for a very long time, that one number tracks us across all governmental systems. But we still need to identify that when I come in with, you know, somewhere into into the system that I was the same person that came into the system before or the same person that comes in after and that when I get arrested under one name that I'm the same name in the in the you know, in the court system. So that master person index doesn't exist in Vermont. We have different numbers everyone's identified with across the system where we use what's called a string match a technical way to match names and dates of birth across systems. So you have to build that connection. I can you know, I gave you guys the same thing that Connecticut had based on the court data. I don't know how useful that will be, but you guys can go through it in your report and what I you know, there are some interesting things in there. I think about how many people who go to the system who never end up with a conviction, right? So that's an important number there. But what you do with it, I don't know. So you can, but it's still going to take time. Right. And Wichita, you understand what I'm trying to say and maybe you can explain it better in English? Yeah, the word that I use at least in my technical environment is conformance, which is what you're talking about. It's like in order to be able to analyze things, things need to look the same. And they need to be comparable. And a lot of the foundation building is making sure that everything that's going to come in now and later has the ability. Yeah. And just to give you an idea of how things change and why you need governance. Many of you know that when the court system went to Odyssey, you change the way you record the statutes, right? So now I've got data from 2005 when charge codes were introduced to now and now you switch to something else and I still have data systems that use the old with the parentheses and things like that. So even just how you record the statutes that somebody has been charged with sentence to is different in every data system that's in place. And that's right. And so that needs to be formatted across so that you know what everyone's doing. It's not an impossible lift. It's just not something that's going to be done overnight. David and then Monica. I think Monica might have had comments that were more directly relevant to this issue of data gathering in time. It's all defer to her. Monica and then David. Well, I don't know, David. That seems like a lot of pressure. Honestly, I have a few things going through my brain right now. And really, I think for me, the question of the question of the question actually is sort of like a rabbit hole. I don't honestly think it matters too much. And to me, it doesn't help answer the question as to where the bureau should be housed. And I was looking at the bullet points that are on the second page of the document that you sent out eight times, because I think, you know, in my mind, it's like, what do we need to answer? And what needs to get answered after the bureau is really formed? You know, and some of the things that are on that bullet section are things that could be potentially answered like in a five year strategic plan that was presented to someone that approved, right? What are the questions? What are we going to start with? How are we going to follow these best practices on sharing integration and analysis that have come up in multiple comments that people are making? Because maybe when the board or the bureau gets created, that people that get hired to do this and can spend like 40 or plus more hours a week, you know, with full-time employees doing it could actually figure it out with guidance that we give to them. And so that that's sort of the level that I was at. So I don't know actually if I did have a real data related point to make David. But thanks for letting me go, David. We've obviously been discussing a bunch of different things today that are kind of along different threads, which I think is totally fine. But I say that only to announce that the thread that I was going to pick up here was really more related to the definition of the mission. And Rebecca's talked a little bit about that. And I wanted to say first that I really like actually what you've done here and the second sentence of the first paragraph after the summary of Act 65. I think that's a that's a nice sort of general direction with enough removed to allow for a lot of a lot of work to be done within that. I think Rebecca's point is a valid one in terms of adding in one more clause and I have not wordsmith this at all. I'm just thinking out loud here. But one more clause or clause related to the fact that we are doing this for the purpose of ameliorating racial disparity. That still leaves a huge amount of, you know, discretion within the the Bureau and the Bureau's governing panel. But I think that's a valid point. I may not be. I hope I'm getting at it fairly. But I think it's a valid point that Rebecca is making about what the why behind this and I think encapsulating that in the mission is wouldn't even add that many words to what we've written. But something along those lines would be reasonable. And then I also do think that this is a slightly different point, but but related. I think that after we set the mission and set up some of these guidance things, guidance, ideas, I do think we may somewhat painfully have to be have to step back and say, all right, now this is the Bureau's job and it is the job of the governing body of that Bureau, which hopefully will have. And I think that is something we need to make clear in our message to the legislature, hopefully we'll have a needs to have community representation and representation for effected individuals. But it will be that body's job now to start setting priorities and deciding what are the first questions we're going to look at within this mission and things like that. And I think that's appropriate. You know, that's that we are we are here to to set that up to provide principles that will allow it to input from the people who it needs to have input from and give it a mission and then. Let it set sail, let it take flight. So just a sort of generalized thought about I think the mission is good. I think a little bit of an addition will help give it a sort of positive purpose driven idea to it in addition to what's already there. And then the fact that I think there is going to have to be some allowance for this this governance body to make some some big decisions. OK. Which is your hands still up and I just forgot to take it down or do you have something else to add? I just have something else to add. Oh, good. So I think first just just trying to fully answer Rebecca's questions here. I do think that two to two to three year is like kind of a hard fast rule. I mean, you could really try for one year, but I think we'll be shooting ourselves in the foot. I think we would just end up creating a system that's not scalable. It's just it's just really difficult to create a foundation without the appropriate time. The qualitative data collection. I agree that that is definitely something that needs to happen. That takes resources. So like time and designing and especially if we don't already collect it. And if yeah, I just it qualitative data is a lot more complicated to store and for analysis because it's like, how do you store a paragraph of somebody's story? Like, you know, you have to break it down. You have to like create some type of structure and architecture. And again, talking about conformance like I said earlier. So it's just it's doable. It's just like the more you ask of it and the more complicated you want it, the longer it takes and the harder it is. Can I just address that point, Aiton, for a second? Sure. Yeah, thank you. So as many of you may know, we put in for an earmark with Senator Leahy's office to do qualitative analysis of people in the southern part of the state who've experienced a criminal justice system of color. And this was done in conjunction with Curtis and the NAACPs. We put in our budget for that and we're at the top of the list. We just are waiting to see if we go into into a bill. Was I think what Karen, 170, 180,000 just to give you an idea. And that was for the data collection, for the software to analyze it. And the state may get a cheaper deal on the software to analyze it than we do. And the time to recruit the human protections that are involved in. We want to make sure nobody can subpoena our records. So we're going to apply for an exemption from the from the National Institute of Health to say racism is a public health crisis. It manifests in the criminal justice system. Nobody can subpoena my records because researchers are never exempt from you guys, from lawyers subpoenaing us. No matter how like, right, we don't have a an evidentiary privilege like attorneys do or counselors do or something like that. So those are the types of things that you have to do and consider when you're looking at qualitative research and why you want to pair with a researcher who can go to the review boards and and do all of that stuff. And independent scholars can again, to find points that it can't just be me and UVM doing all the research in Vermont that you want to expand this. But that's what you're looking at for a study on a small population in part of the state was about 170,000. Great. Which in the grand scheme of things is not actually all that expensive compared to the amount of dollars that flow out of the feds. But that was what we put in for it's a tight budget and that also includes paying the NAACP to help us recruit and other right. So there's all of this kind of stuff that goes into recruiting participants. You know, paying for child care, if they need child care, well, we talk to them. All of those sorts of things go into that type of budget. Right. And food.  Oh, I very, very abhorring addition to the conversation. I just wanted to point out, Aiton, because I know that you're monitoring lots of different things that I put a link to the the act that reorganized the Criminal Justice Council in the chat. So if folks wanted to pull that later and take a look at it, it's it's there for those who would like it. Thank you. Oh, thanks so much, Evan. Yay. All right, I have to go back to Rebecca. Robin, your your comments earlier and just now sort of reminding me, I didn't follow up with this question for you earlier. So I appreciate you sharing with us your your pending proposal on Southern Vermont closer analysis. You also said earlier this evening that and you've said this before at other our DAT meetings, and I fundamentally don't understand this, which is that we have too little data to analyze it at a county level. We can only do this at a state level, yet you're able to take certain analysis at a smaller than state level. To me, when I hear this and it just may be how you're explaining it as someone. And again, I hearing what you talk about how it is possible is just a different complicated animal that we have to separately source, which is the qualitative side. Isn't that where we can still analyze what is happening in Vermont, that we cannot exceptionalize Vermont just because of its rural low population matter, that we can never trust them. Because to me, when I hear you say things like that, it's like we can't ever trust the data, we can never get to the bottom of it. Therefore, we can always point fingers to elsewhere to point to the blame of the source of the of the disparities. Now, maybe you can explain it for me, but that's how I hear your your references that we just are too small with state to understand racial disparities at a smaller than state level. So that was language from Sarah, the PhD who's doing the. Research for justice reinvestment and so the way she explained it and it's recorded, that was recorded. And I think that can get it to you if you haven't, you know, in all your free time, listen to it. Yes, that is also something that C.R.G. has said in other in other report meetings. Sure. So it's right. So if you go back to a few presentations ago, when I was showing you the numbers in various counties of non-white defendants for a five year period. And I said, what happens is that Chintenden kind of puts a foot on the scale and we don't collect a lot of data that will help enlighten as what she was saying about what's the what the data actually means. So one example, so the reason why we applied for the earmark to study Southern Vermont is that Southern Vermont's people of color and the data are smaller numbers than what we see in Chintenden County. And everyone says that Southern Vermont is fundamentally different. I don't live in either count in either part of the state, so I don't know. But right, we hear this and you use qualitative analysis, this kind of interviewing and coding of stories. To get at policy solutions that wouldn't be there in quantitative data. So, for example, the one that I always give is when I interviewed victims of domestic violence, survivors of domestic violence, what makes you unsafe? What makes you feel unsafe when you when they violate the restraining order or you have to go to court? And what makes them feel unsafe in the county that I worked in to do this was Washington County. What makes them feel unsafe is the bus stop in front of the courthouse because the abuser waits in the bus stop where he's allowed to be legally. The only way she can get out of the courtroom is to walk past him. That's what makes him feel unsafe. So that type of policy solution, move the bus stop, is never going to be available in the what did you plead out to? What was the circumstance of the plea? What were you arrested for? None of that, right? You only get those answers by talking to people. And you can get important answers and meaningful policy solutions from people that don't involve collecting the actual numbers or that numbers can't ever quantify. How do you feel safe? We don't ask that question in the data, but we can ask, how did you feel respected or did you not feel respected when you went through the criminal justice system? That data we can ask in qualitative. And if you do your regular study, you can make policy decisions on it. So I don't think that I'm saying anything that's inconsistent with what I said along, I think that it's the idea that when qualitative analysis is useful is when you have small numbers, when you want to get to really important aspects of the data that you can't collect regularly or want to. So that's when you would use that. Does that make sense? Thanks, Robin. Yes. Monica. To give another example, you know, one of it, I wasn't at the Justice Reinvestment meaning last week, but I've heard what Sarah's had to say and what CSG said, but also, you know, in the data that we've started to put out in corrections and we've been trying to disaggregate it as much as we can at a race level. Sometimes, as Robin says, the more we break down and do crosstabs and break down this by this and this by this, the numbers in the cells are so small that we're essentially identifying who that person is. And so we're talking about the whole Department of Corrections population. And so we're starting to combine BIPOC into one category so that we're not identifying people because the numbers are so small. And I think that's similar to what CSG is finding, too. Because people are curious and they're asking to break it down in ways. And the more and more you do that, the smaller and smaller the numbers are going to be. And from a, you know, from the research perspective, that's difficult because then the protections are in place. And that was something I know you brought up, Rebecca, so that's part of it, too. If I said that wrong, Robin, just let me know. Yeah, that's also part of it, is that if you go back to a few presentations ago when I pointed out that there were so few Indigenous people captured in the data that, yeah, I mean, every time I have an asterisk, it's because I have to protect that person's identity. And there are a lot of asterisks in the data, even just for how many people were sentenced, how many people were charged with this crime, how many people were sentenced to this particular sentence. It does get, you know, when you start slicing it smaller and smaller and smaller, you risk identifying people, but you also can't make any policy suggestions without, you know, hearing the stories and and and how people felt about their experiences. OK. I have, oh, wait, Rebecca, you've got it. I've got like a big thing, which is a term. So, Rebecca, you go. I just want to just emphasize what I'm hearing, so I just understand the very small data presents the problem of assuring privacy, right, which I understand. And that it that we need the we it makes it absolutely necessary to have the qualitative analysis to contextualize it. But here's can I just clarify this? If we could build whatever measures are necessary to ensure the privacy of these individuals who are subject in our greater risk of being exposed because of the share numbers, if we could build that. Could then then there's the fact that we just have these small numbers doesn't mean we can't analyze and understand it. Again, assuming we have this qualitative analytical piece to it, is that right? Some researchers are not going to want to publish data with cells that are that small, so it really just, you know, it really just depends. And you're saying that you wait, you're saying that you can't, you can't make it protective enough? It's not even so much a protective enough. It's that can you draw any conclusions? OK, but here's a I started in a rough. So that's that's why the qualitative. And again, there are two different things. I think so one of the benefits of the quantitative data is you can do over time, right, metrics over time. How many, you know, so the data that I presented to you, for example, a few weeks ago was a five year snapshot, but what's the more interesting snapshot is year to year, you know, for some things that you want to measure. Sometimes you want to say the legislature makes a change and you want to measure the impact of that change. So they begin to, I don't know, they do something we want to measure it. You can measure it over time with quantitative data for the most part. The qualitative parts gets at different questions and it makes a more complete policy picture for people to act on. I don't know that you need to prove with numbers what people can't tell you for themselves on their experience. Because one of the things that is not going to be captured in administrative data is the feeling of procedural justice and the feeling of being respected by the system. But certainly you know that there's like ten who had that same experience. And I just realized like where I'm having a disconnect with hearing you know, we are always working with single people, right? A single particular fully contextualized set of facts relevant to the legal issues raised for the challenge below that. And we're asking the court to make a decision based on a single person. I think I see this use of data a little bit differently. Like I don't see that there needs to be a trend before it's significant enough to act upon. I think that if there is one or two or three or five. And again, and we contextualize it to provide some meaning, then isn't that how we give voice to all of these very few? Relatively speaking, to other places, people of color in the state who are receiving various forms of injustice going through these systems. And so I guess I'll stop there. So qualitative analysis doesn't need an N, a particular N. It does need the way it's analyzed. And I can send you what I don't know who would be helpful if people saw like computer software that we used to analyze these data or some qualitative studies that have actually changed public policy. It gets to things at the list. Let's say Washington County has actually not moved that bus stop. That was how victims of domestic violence experienced the bus stop in front of the courthouse in in in Barry. Let's say they did move it. I know one of the reasons why they didn't move it was because wheelchair access. So you can ask for a policy based on a few things and then you move it and find out you didn't talk to everybody possible. So by documenting things in qualitative analysis, allows this kind of broader policy discussion to happen about how. OK, so if we can't move the bus stop, how do we get you out of the courthouse safely and we have further qualitative analysis in the state that we've done that have had that policy decision? You can. I don't know why you wouldn't want to do qualitative analysis or why there's like this hesitancy about it. It's been used for. I mean, I'm thinking some of the really great. It's used a lot in education to change education policy, especially around race and marginalized folks. So, yeah, I don't know how to make it any more accessible. Now, I'm not I support qualitative analysis for sure. OK, witchy. And then me. I sure you don't want to go before me, Ethan. No, go for it. OK, Rebecca, I guess I just have a follow up question on something that you said. You mentioned that. You don't need to see a trend in order to bring something to. And I I'm messing up the second part of this, but in order to bring into policy or something like that. I think and I should have clarified, let me I know you guys have or you've been using an example of victims of domestic violence and what would make them safer in the bus stop. Let's let's bring it to an actual suggested discretion and collection point, which was how many I think the charging decisions of a prosecutor, right? And let's say in that county with those prosecutors for the similar amount of cases, I don't know how we would control it, right? Again, how many are getting what types and levels of drug related offenses? Just misdemeanors or how many are getting them to the maximum felony aggravated trafficking? Let's say then the numbers are in that county. You know, out of a hundred ninety eight or weight, two or black are those number. And of the two that are black, those are the two that are charged with them with a maximum maximalist mindset for prosecution. But it's only two, right? But it was all men who went through that prosecutor's office for whatever time period you choose versus 98 from a data statistical analysis. My understanding is you would discount that because there's not a sufficient amount of data to conclude anything, right? From my perspective, I think that's absolutely significant, right? And so I guess that's where I'm having the disconnect with the discussion here with where the data is too little. The I think in not a particular example, and I'm really glad to use that example, because I I understand a lot clearer now what you're what you're trying to say, I think in that example, what becomes dangerous is who's looking at that and who's allowed to look at that. Because if you're you're saying that only two were prosecuted to the maximum, right? And you're like here, prosecutors are here, people. It's like, all of a sudden, OK, well, those people now know, know who those two are, right? I think it's like all of a sudden it's the it's identifiable and then it becomes a privacy and security issue. Yeah, I wouldn't say that that would be the case in in in all cases, but it definitely that would definitely be one of the biggest risks in having such a small amount of of of a sample. OK, it's my turn now. Couple things. First, I want to point out that it's up. We're getting up on seven thirty and we have to talk about action points for next week. OK. I put a few out there earlier in conversation and I didn't know whether people liked them or didn't like them or had others or what. But I think we have to work on that. The other thing that I want to raise is that points four and five, how the Bureau should conduct data collection and analysis and the best methods for the Bureau to enforce its data collection and analysis responsibilities is something that a group that Karen, you were and I'm invoking you because there's a group that's ready to go on that that would include, I believe, most time. And I am what I mean? My first question, I guess, is is it time to get that group going? Well, it's on. I think that's why we were going to meet with you briefly. The few of us that are that bridge both groups. So we could talk about that and then present it to this whole study committee. I think we actually have that scheduled and I was going to invite Kristen. I was going to invite I think we do. I can't remember. I was trying to remember if I actually ended up rescheduling it, but I was going to see if I could invite Kristen McClure to that meeting just for a few minutes to get her engaged in the whole data integration conversation. That would be lovely. Yes. And so I will keep trying to get that small group together to talk.  As I already said, yes, it's it's all aligned together. We're happy to have you guys doing the data integration work. And I think one of the questions we had to that I did send David an email was and I've checked with a couple of the judges to for the Sentencing Commission on a different issue was do we have to notice the meeting? Because it's a subcommittee of a subcommittee of a legislative body. So if we do it as NCJRP, we don't have to. If we do it as our DAP, we do. So that was that was my thought and I don't have the I mean, maybe the AG's office can notice the meeting. I don't have that capability. OK, all right, thank you. So that's going to be taking place. Everybody should know that that's going to be taking place separate from this. Not that you're not invited. It's just no. And it will invite people. I mean, it's not it's not an exclusive group at all. In fact, we'd like you to be there. He's very savvy about all this data integration stuff. So it'd be great to have him there. Right. All I was saying was I just wanted to point out it's this is an exclusionary. It's simply this is where other work is going to be happening on this. So you all know that. That's all I wanted to say. The other point I wanted to raise is do we want to look at, first of all, the statute that created the office of the executive director of equity? And look at how that plays into what we've already written for today. Do we want then also to look at the enabling statute that Evan has put into the chat that designed the God, I'm sorry, it's been a long day. The Criminal Justice Council and see how that interfaces with what we've already proposed here, because we've been discussing the governing body, the makeup of the governing body, the fact that we didn't want to tokenized people on the governing body. All those issues, that's in there. I remember that being in there. So I guess what I'm saying is if we're going in that direction, I'd like people to volunteer to do that work. I can't do all of this is what I that's what's coming to here. I just wanted to say I'm willing to do a lot of this, a lot of putting it together, but I also mean, I need other folks to to look at some of that stuff and do some of this. Accretion, as it were, that I did with this particular thing. And if that's where we're going, I'd like us to I'd like people to volunteer. Having said that, Evan, you have your hand up. Evan, you are muted. I was saying, I'm glad you repeated those two tasks, because I couldn't remember exactly what they were when you were soliciting volunteers. I also had another suggestion and, Monica, I'm not I'm not trying to steal your idea. I'm trying to agree with it. But, you know, I think I think one thing that might help us is trying to figure out what level of detail we want to include in our answers to these questions, because I think Monica was right when she said at a certain point, the entities that the people that are going to staff this bureau are going to have to provide more complete answers to these questions. And I think David was right when he was saying at a certain point, we're going to have to be willing to let go of the reins and hand them over. And it sounds to me as I attend these meetings that there's a lot of common ground and principle amongst the the participants in this group. And if there's a way to answer some of these questions based on those principles, maybe that's a productive use of our time. But then also, Eitan, to your specific question, I'm happy to help look at either the Criminal Justice Council legislation or Susanna's legislation and try and provide some type of analysis. I'm a little hesitant to promise that I can have something done by our next meeting on Monday, because I've got a public records request I need to deal with. But I can I can make an effort to do some of that and report back. You just let me know what you want me to focus on and and I can I'll do my best. Evan, why don't you focus on on the statute for the executive director? Sounds good, I can do that. I can do that. I am trying desperately to unmute someone right now and I am having a horrible time. I don't know what is happening with the computers tonight. Rebecca needs to talk and I can't unmute her. I think it's Henry. Henry's happening on re.  And now my hand is up and I don't want it up. Oh, God. All right. Rebecca, I'm working on it, but I'm not done. That was OK. So that's one thing is to look at the enabling statute for that office. The other one would be around looking at. The why do I keep doing this? Criminal Justice Council. Good God, I'm blocking every and looking and seeing if that has any relevance to talking about what the governing body that we're talking about does. Because it and I guess I'm making that point because it already exists. People spend an enormous amount of time on this. I just remember that being a huge lift about not tokenizing, not bringing enough people in who actually are not insiders into any of the criminal justice system, who would be able to stand outside and look at it and ask questions that people within it would not necessarily ask. That was really critical in its design. I I don't know. David hasn't been working. I was sorry. Go for it, David. My apologies. I was just going to say, you know, having been on the council before and in its new iteration, I think it's been a huge improvement in terms of bringing in a lot of voices that should be heard in these deliberations. So, yeah, I mean, is it is? Is it working? Might be too to more to I may not be able to give an answer to that definitive. But I do think it's a it's a really significant improvement and it's a much more holistic set of voices that we have in that room, which I think it's been very valuable. Great. OK, what I was going to add, A-Ton, is if I'm going to be sort of taking a global look at Susanna's enabling legislation to see if it answers any of these could help us answer any of these questions or how we might be able to adopt it for our purposes, since it has a section on an advisory panel, I can't imagine I won't be looking at the Criminal Justice Council recent legislation as part of that exercise. They're somewhat intertwined. So I can I can try and do that as well. Great. Thank you, Evan. And bless you. Rebecca wants everyone to know because I can't unmute her. She's going to try to connect with the Secretary of State's office on the fourth point, how the Bureau should conduct data collection and analysis. And she also wants to make sure that we can bring Abby Crocker into this discussion, which is critical. She's one of the people that Act sixty five says needs to be brought in. And we really ought to be able to mine some of her thinking there. And she asks also, Karen, can you send out notice of the Parallel Pop Project that we were talking about earlier? Yes. OK, great. She wanted to know if someone from O.D.G. Office of the Defender General, right? Great, I got it. OK, is someone from the Office of the Defender General going to be there? They're not, but Marshall is on the NCJRP team. He hasn't been attending in quite some time, but he is on that team. And I think Matt gets the notices, too. OK, Monica, you had had your hand up and then it went down. Was that was that the storm or was that you? That that was that was me. OK, I decided I didn't have a fully formed thought, which sometimes doesn't stop me, but it did today. Oh, OK, all right. It just this all just feels like it needs to be in sort of in the direction that we need to go to sort of solidify some of the stuff that this policy, this this thing I contrived from everything that people wrote and sent to me. You're all kind of going, all right, it needs to go in this direction. So I'm just trying to give it some shape here and sort of say, all right, so for next time, let's go in this direction and start giving it that shape. I will myself look at. That sentence that both Rebecca and David brought up, the second sentence of the first paragraph and speak more directly to what the reason for collecting data is. Basically, to get more specific, which was what Rebecca was asking for, right? OK, yeah, this is ridiculous. I'm sorry, Rebecca, I. What is the intent for the data? I'm going to do all that. Yes, because Rebecca has really hard time speaking for herself. So I will I will get on that for this week. I will do that. Susanna, you have a ton of you want I can help you synthesize that. I would love that, Karen. OK. I would love that. OK, I have about 12 hours of teaching to do this week. OK, let's let's touch base and just find a time to talk and we can get that done pretty quickly. I would love that. That's great. Tyler, haven't heard from you a lot. Hi, I've been I've been quiet. I've been trying to synthesize a great many points coming from a lot of directions and catch up, but it seems to me that so much of what we're talking about does boil down to that second sentence that I agree we need to add a little bit more to and it seems like that second sentence really gets to question number three, which is what should the scope of the bureau's mission be? I think once we I think that's a thing that our group it feels like we're relatively in the same place on and that's a that's a thing that can be pinned down first and then the other pieces will make more sense with that. And I think we're actually quite close to a solution on that one. OK, I do too. From my perspective, I also want to point out that just to make us feel better about ourselves, because I think that's good. About five months ago, we had eight possible locations for this thing. Eight. I can't remember all of them, but I want to say it was eight. I remember it was eight. We are now clear that it needs to be a state government organization for housing and we have very clear reasons for why we believe that. And we all came at it from different directions. And I think that's really interesting that it came from different directions, but we all got to a very similar place for similar reasons. So just think about that for a bit. That also says to me we've done a lot of work. So I just want to put that out there. Good. We've got forward motion here. I'm happy. You know me. But I don't feel like we're going somewhere for the next meeting. I get anxious. Other comments, questions. Seriously? OK. I'm thinking of, you know, who always has something. Sheila, Sheila's, there's something we're missing that you're going to like it's going to occur to you at two in the morning and you're not going to sleep. And then you'll send me a text. Well, I can't guarantee you about the two in the morning text. Right now, I'm feeling like like you summed it up really well that everybody came to this at a different angle. And I think multiple people addressed a variety of things that I was thinking about or might needed to have had said. And so I don't need to repeat the wheel. Thank you for giving me the space. Got it. OK, then. I do. I, John, before we go, I had one other thought that's been bubbling as I've been thinking about this that might be helpful or might not. So discard if it's not helpful. I kept going back and reading what the bill is asking us to do in the development of this bureau, and I kept comparing that to what the scope is that we're talking to. And that and I understand also wanting to make sure we're guiding that purposefully. But I guess I guess the thing that I keep thinking about is remembering what is the differentiation between what the bureau does between what our DAP is supposed to do, what Susanna shop is supposed to do. Because I think sometimes we start mixing those things together and that makes our projects complicated. And so I don't know if it's helpful at all to think in terms of being able to differentiate is this bureau have a specific and clean purpose that is different from our DAP's purpose at large and and where how do those relate to each other? Let me just point out here and I don't know whether this will help you with this, Tyler or not. When Coach and and and Martin LaLonde were working on this, I got very protective of the our DAP. I had a lot of moments when they were when Coach first came and said, I'd like you all to be the advisory body. I like got really hot under the collar and said, this is not the our DAP that does data. It is the our DAP. You asked us to do some work on data. We did work on data. We are not a data body.  And he agreed. And we went to this point and I was a little like uncomfortable with it, but I'm working with it, you know. I think I think this is just where we're at right now and we shouldn't take it as anything more than contingent. I guess what I'm getting at, Aetan, is I hear that. Thank you for that. What I what I think I'm getting at in my head is if this body is people who have the capacity to work with data, to do research, to make sure questions are asked in a thorough purposeful, thoughtful way, that I see are the Bureau, right, as that body who's doing that. I see the our DAP as a body that says, I am going to ask you this question. How do I get to these answers? I'm going to pose this thing and our Bureau of Racial Statistics is able to say, here is how I can work with you towards getting to the answer or something along those. That's where I see a little bit. I see the Bureau is being a little bit more living in the data side of the equation. The R DAP is asking, how do we get this policy question answered? Do we need qualitative data or quantitative data to get to an answer to this question, so on and so forth? Well, that makes sense. Sure. And if you look at the last bullet point, it's directing the mission of the Bureau or the data entity or whatever we're calling it. We've got to get there, too.  Review the reports of the advisory panel on racial disparities in the primal and juvenile justice systems to incorporate its recommendations into the strategic plan, that plan being for the Bureau itself. Does that help? It does help me. Thank you. And I'll stop talking while I start formulating this because I know we're we're close to the finish line and I stopped us. Oh, God, you're right. It's 10 minutes. OK. But you also should know, I mean, in case you don't, Tyler, we are technically supposed to write.  It's been every year. But theoretically, it's supposed to be every two. So I but yeah, that that's there. And that maybe that needs to be strengthened, in fact, because I don't want this to be the our depth of those data. I just it's well, it's just ridiculous. I mean, it's absurd. So. I think we all know where we need to go this week and what we need to do. Thank you very much for yet another very, very productive meeting, even though I couldn't unmute Rebecca. Thank you. And I'll see you all next week. Bear in mind, by the way, that the 6th of September is a Monday and it is also Labor Day. Just bear that in mind. We're going to have another holiday. And let you know, that's also that's also Arab Rosh Hashanah, because you said you forget these things. Oh my God. Well, there it is.  But if you have me for. Thank you. Yeah. At Rosh Hashanah and you would think I would.  In any event, I just wanted you all to know that's a holiday for a lot of reasons. So we won't be meeting then. I will talk to you all shortly. Thank you all for volunteering to do the work that you're going to be doing. And please be in touch as that goes on. I can't believe I'm going to say this, but I cannot ever receive too much email or too many texts. I know that sounds crazy, doesn't it? It's a need to, frankly. But I mean it. So thank you. See you all next week. Be well.


Mae'r hun cyhoraeth am y Gymwe Deillol. Aуд ydym mor hemugi fel ym 308, health guidance awr nesaf, umd hacerlo am 250,000 ysgr棒, 150 nôr f weight, 310, nesaf, nesaf, nesaf, nesaf, nesaf, nesaf, nesaf, nesaf. Over a huge range of organisations, enterprises, have a look at the website if you really want to know more about us. We do the tool, the tool is kind of the main product, but we do services as well. So I'm a coach with Riley, so I go around, we don't just want to sell a tool and help people that the tool is going to solve all their problems. We want to help organisations to be successful. So we provide coaching as well so that when they're using the tool, they're using it more successfully and then we try and be really active in the community as well. So that's why we're sponsoring this conference and we have a whole lot of stuff on the website. So lots of content on there, lots of community support. That's kind of the combination of the three things, the tool, the training and the community that we think are a really powerful combination. Because I've only got 10 minutes, I'm going to really focus in on this demo and I'm going to talk about what we're doing with Dean Lethingwell's Scaled Agile Framework and what we're calling Agile Portfolio Management. So this is really a kind of a new market. There aren't any other Agile Portfolio Management tools after we've got Portfolio Management tools that are trying to work with Agile, but what we think we're doing is taking Dean's Scaled Agile Framework and how he recommends doing Portfolio Management and Dean was involved with Riley from the start. So we've been working with him on this over the last 10 years. We've kind of been involved in getting this to where it is today. Putting this. I can't really read that, but essentially there's three layers of this and I'll kind of try and go through it. I'll try and give the quickest scale Agile Framework training ever. The other portfolio level, this covers the entire business, anything in the portfolio. And this is this is Apex, really big things. Things that are going to cross-releases. And we've got architectural Apex in there as well, because when we're talking enterprise scale, you do need to do intentional architecture at the architecture just in time. There's this portfolio allocation. So how are what our strategic goals? How are we investing for Apex? Those Apex then break down into the program backlog. So we're now getting down into the features. What are the features that are going to get that deliver those Apex? And there's this concept of the Agile release train. So at least every quarter we want to do a release of features. Ideally more frequently, but most organisations are not able to do continuous work. So we're looking at features. Features should fit in one of these quarterly releases, but they may be bigger than fitting in a normal strength. So features then break down into stories and that's where you're into your traditional Agile teams. So this is where we started, where most teams are and then we're working our way up and now have functionality from the top down. So this is a dashboard. This is the information that we want to generate because what we're trying to do with Agile portfolio management is get real-time visibility, real-time reporting of the data that the actual scrum teams, the Agile teams are putting in. So instead of a project manager, a program manager having to generate stuff in a PowerPoint slide, we can create a dashboard like this which shows the reality of what's actually happening. Then we can use that. So we're getting feedback and the same way that we value rapid feedback at the team level, we want the same sort of rapid feedback at the portfolio level. So the things we have here, we have a Kanban board. So this is showing the features. Features have a workflow, so they just get created, they don't have an entry. Then they go through a proposal stage and this is just a sample. This workflow can be configured, but somebody will put some effort into put a proposal for a feature. If that proposal gets accepted, then we go into some more discovery work. Probably going off and talking to customers, finding out what actual details we need to build. Then we go into developing it. Once it's developing and employed, we want to go to a validation stage. We want to check is the thing where you built the right thing. This shows which features in a nice Cardwell style are in progress with some other information. You can see that once we go into development, at 56, we've got some progress indicators. So we can work out the progress of a feature based on its user stories, how many user stories have not started, how many user stories are in progress, how many user stories have done. And now we can either do that on account of user stories or we can do a sum, a roll-up of the estimates of user stories. That's our first visualisation, just looking at the Kanban board, seeing what work is in progress, what's safe to work with. Then we've got some little reports here. The active epics. So epics, those things are the portfolio level. Which ones are being worked on? And we can see this four here. This one's already completed. This one's about 50% burn, this one's 6% burn again. So we can start looking in progress and we have the investment category here. So in this demo environment, we're using Jeffrey Moore's investment categories of optimise, neutralise, differentiate. Optimise stuff that we're currently building now, customers are using and we just want to make it better. Neutralise is something where, there's the competitors out there that's got something that we don't and we want to neutralise that complexity. That or differentiate, there's just something for us. Rally portfolio manager is a differentiating investment. We invested in this to differentiate ourselves. So we can look at the epics and again these on track to these look healthy. There's some little algorithms behind here because we can put some anticipated, some plans start and end dates in. If we're getting towards when we plan to end it and we haven't started it, that's probably going to be a red flag. We can put some simple calculations in there. One question. Yeah? Portfolio management in this sense is the same as managing programmes or projects as a portfolio? Yes, so the portfolio is all your programmes and then your programme will break down into what we've made up of features, features that are delivered by agile teams. Yeah. Now, epics are the first two features. So epics are things that are delivered at the portfolio level. So let me just kind of scroll through. So epics are these things here. Epics and your portfolio. Epic backlog is your whole portfolio of really big things. So deans you. How epics can also mean what are the stories you've heard about? Yeah, so this is deans using epics and this is how deans using the word epic is. Diagram. I want to understand the larger uses. Because on a project level we understand uses what we may refer to a feature of a product which we develop. On a portfolio level we may talk about multiple effects. Yeah. A scaling across programmes. So what we found is that every organisation has its own terminology. Deans picked on. We can, you can configure. You can have as many levels of these as you like and you can call them what you like. So epics basically is a user story on a portfolio level? Yes. Very basic. Yeah, very simple. Yeah. It's something that's going to take months, maybe years to do. Really is really epic. So we got epics that are in progress and this is our epics that have not started and what you've come to see here is this is showing the plan style date. So we can give some idea and we can sort this so we can see which were the ones that we think we're going to be starting on next. Yeah. Can we get a similar view at the team level? Yes. Yes, so teams. I'm kind of focusing on portfolio but when we get down to teams, teams can have their own Kanban boards as well which show user stories if I workflow by user story. And then once I've gone through this I'll show you how you can drill down from an epic to features to user stories and then user stories. So if you do decompose user stories iteratively you can do that as well. Okay. I mean I actually meant to ask maybe part of my accent that I didn't pronounce it well. Team in the sense of logical group of user stories or a group of user groups. Yes, you can. So one you can group by your investment category. So what some organisers do is they have an extra layer in here that's just that grouping for you. So rally we have an initiative layer. So you've got an epic. So actually your initiative is your top layer and your epic. So yeah because you can have multiple layers in here you can have that's because we found some organisations, huge ones had five or six layers in that portfolio because that's how they did that sort of group. I mean one week of slice and dice the backlog is look at some of this cross cutting concerns like performance or you know turns out. Okay grouping. So we have so there's a couple of ways we can you can define custom fields on these things with a drop down if it's a finite set and then you do reporting on that grouping by that filtering by that or we have just basic tagging functionality. So you can just use tags. So you've got nearly the right to say you one would be able to view the progress based on the speech groups. Yeah, yeah and yeah exactly you're going to everybody's going to have their own personal preferred view of subsets of that and the others various ways you can do that. It's not like hardware for just no no no nearly everything in here is configurable. There are applets and then again features we can see which features are in progress with data, center information and then so this is actually grouped by so this is PSI 2 I think. So this is a potentially shippable increment. So this is a release train. This is something kind of three months worth of work. Again we've grouped it by that all the features that we think can go into this release and then this back level of work that's been PSI 3 so again that's that's that's one way of grouping things. Pretty much all of this stuff you can you can customise as a query language behind it. So either worth thing noting here you can't really see it. This column here is slightly shaded because you can see we have a weapon in a six here and we've we've got seven things in there. It's just giving us a warning that we've exceeded our weapons on that. And then at the bottom so this is the investment allocation. So what we've said is that this kind of demo scenario we want to equally invest a third of our budget into optimise, neutralise, differentiate. Differentiate, optimise, neutralise. That's what we want to be doing. This is what we've got in the plan. So this is what we've planned in and you can see that this is very different from that. So we can see that what we've done is we've planned a whole load of differentiate stuff in there. We haven't planned any. So that's feedback to us that our current portfolio of work that we're working on doesn't matter what we want to be doing. And then you can see what you're actually doing. So as Mark gets work gets done at the moment there's nothing completed here. This will get filled out. So again you can be constantly checking in. What's planned, what have we done and how does that match on to what our strategic plan is. So very much strategic. So what we're trying to do here is we talk about linking strategy to execution. Portfolio level is very strategic focused. Down at the team level it's execution focused and we're trying to put a link between the two so you can check whether your execution is actually delivering to your strategy. And then the last thing on here is this timeline view. With these portfolio items these epochs and these features you can put a plan start and end date. So when you think it's going to start and end and that's you know finger in the air but what this is showing here so this hatched line here is our plan start and end date. This is when we've actually started so it's looked at all the user stories under a feature and it said the first user story that a team planned into a sprint and started working on was started here. This is just based on real data based on what the scrum teams are doing and we can see that we now we've started later than we thought we were going to see. And then we've got a little progress indicator and that might say well that's at the moment showing 50% complete. So well we started it late but look when we were going to finish here and we're already 50% complete. So it's not a problem. Yeah. So there's quite a lot of stuff. No. This one's not interactive. This one is so this one here but I didn't give a one that the Wi-Fi is bad which is not good when you're trying to get a solution. It's back up again. Okay let's see. That's where I loaded all the so okay yes so I can expand this one out and I can say okay underneath that so this is actually a theme at the top this is even higher than the epic for that theme what are my epics what's the progress of those epics this is epic 21 I can expand that one down so yeah you can play around with this and interact with it absolutely so this is this this is just going to quickly show you the hierarchy so at the moment we've got these two themes here this top level of our portfolio and I can in the same way on that timeline it's going to work yeah I can start drilling down these and expanding them out and this should show that I go down for the story level so this is a theme that's an epic I know you can't read this that's the feature and then we're down into user stories here and you can see the difference here because here you've got at the portfolio item level you've got this nice simple cent complete but it's calculating once we get down to user stories you've got slightly more traditional is it just planned in progress completed accept it so you're more traditional kind of scrum schedule sense and we could keep drilling down and then this one just the same thing but it just stops at the story level so development teams might be more interested in that sort of drill down so I'm kind of out of time what I've not shown you then is that once you get down to the stories you're into traditional agile territory of having team backlogs team scheduling work into sprints getting them done marking them done lots of the usual team dashboard like burndowns things like that any other questions capturing more details around with the these portfolio items and user stories it is like a but a visual focus of various of various I guess visualise from a more scrum perspective that we have a we have a way to do these the state code you know the posting nodes yeah so you can get a a come down bold view at the team level yeah you can do that and then again you can define your own workflow each team gets its own view of its data there's a whole load I mean I've kind of just scratched the surface here once you're getting down at the team level there's a whole load of extra data you can put in there you can have your acceptance criteria you can associate tests and test suites with user stories so when tests fail the user story gets marked you can mark user stories as blocked and again that will ripple up as a report so you have it down on your dashboard easily see blocked work so where do you want to go and help teams because mostly I have seen and that we all the charts get generated yeah but typically you know the work and progress and stuffs like that but we don't have a diagrammatical representation we get more from a a tabular column level the load but not on a diagrammatical basis okay so here's a yeah the wi-fi is working here so here's a kind of a more team-based dashboard again you can see all the block to work here so program manager scrum master and I have that sort of view just kind of a summary for the release burn downs I wonder if I can find a team camp on board so this is showing for agile team one they came up with an original name again they've defined their own kind of fairly complicated workplace space and yeah these are all their user stories but they're they've been prioritized it's probably stuff that they've they've sprint backlog ready for dev dev and visualisation this is blocked if something's if I want to mark something as ready so I can say that might be a different way of saying this I've finished development on it so yeah that's a this is a more visual way of looking at it and we're actually moving in this direction moving more towards this card view base so even I don't think I've got it set up on here we have our iteration planning board which again is all just dragging cards into across into iterations rather than the list view it is probably going off in the middle yeah so I was looking for this because he has come asked us to typically handle a work of you know handling all the posted nodes on the white board yeah and we can't help the rest that's a external maintenance work apart from what we enter the data now right so we do things twice you've got better duplication there but that keeps the spirit on because people basically see the flow but still data needs to go on I was thinking if they would have any data like this yeah this makes it a lot easier to update in the tool if you're doing it physically dragging stuff on there yeah I mean this is yeah this is drag and drop I can pull that over there and actually we have a even newer version of this that hasn't made it onto the demo environment and yeah it's just kind of pre-released within so even I can get at it in our with our logins but nobody else can yet which has all inline editing and even it's even nicer than this oh you're trying to we're trying to really kind of up the ante on the usability so that's that should be available to the public weeks months yeah it's not really my job to say that sort of thing but it's going through just final testing and tweaking and maybe maybe this quarter is that safe if it doesn't don't blame me it's that's not a product or that's not a promise yeah I say Steve can say that we don't have like fixed release release dates or stuff we release it continuously so we just have to open final testing yeah so we do weekly releases and then we have a kind of a it's not quite continuous deployment but when we release stuff we then turn it on subscription by subscription so the first subscription gets turned on is for rally employees and then we'll kind of slowly roll it out so that gives us some some better testing some gradual testing there's a problem we can roll it back or fix it or just turn it off again so it's kind of featured toggle type technology yeah you can either so the iteration is is an object in itself you can add notes to that so sometimes we'll then actually have a that on that dashboard or they'll just have a little panel which shows the retrospective notes for the last retrospective so it's going to be visible every URL is unique so you can email a URL that's um as a you know if you want to just send the retrospective part um understand what is happening well what we'll probably do there is we have an API so you can expose reports outside of rally and it's you just create a read-only user that log in to rally pulls the data out and then you can embed that in a wiki or SharePoint so that's typically what we'll do with that sort of thing or do you have reports do you have reports within the I mean pretty much yeah yeah there's a whole load of um so there's there's two kind of ways of doing that one we have some standard built-in reports which are things like burn release burn down spin burn down as cumulative flow but then there's a whole lot of apps so we've kind of gone down the app routes but you can create your own custom pages and embed any old apps and whatever's in the catalogue yeah so some of the reports again we'll kind of roll slowly rolling this feature out you can you can either print directly from there or you can save it as a jpeg and then you could email the jpeg yeah so that's the other way all some of this stuff you can export to excel and then generate stuff in excel if you want yeah we can give you a trial yeah contact so I think you can sign up for a trial just from the website is that right yeah oh community edition yeah community edition is the free version yeah that's probably worth we have a we have a kind of a how would you describe a kind of scaling set of products from the community edition which is free but limited functionality so you know this is the top end you get everything so but you could start with a free one if you just want to play around with it yeah so we can set you over the trial subscription and after trial subscription if you choose to buy you just extend it so you don't have to start from scratch again so we yeah so in and in a trial we want to get you using it for real oh you know we'd come and we'd give you some training and coaching and whatever help we need free version is great for real simple yeah yeah yeah the free version is really just targeted a single small thing all right thank you