ular vote based on the certification by these issues of state in the various states and say that's the finale that's the end of it. When Michigan says this is what the vote is, and the Secretary of State, and or the governor depending upon the rules that state certified. That's it. And we shouldn't have all of these other steps where you can have abuse I'd like to think that going forward, we won't have actors that will abuse the process, but it's clear that in this case we did. It's clear that the guardrails just simply don't exist to prevent that from happening again in the future. Yeah. Yeah. Councillor Katie you want to hop in on that. Well, I will just say, I do agree with Attorney General of on abolishing the electoral college, but I also want to say that our, our democracy was stood the challenges. While I have concerns about the opportunities that there were for attack at the various steps along the way, we are strong and we withstood those challenges. I, and I think we learned a lot of lessons that will prepare us in the future of, should we face these challenges again, we, we know what to do in the face of them. Yeah. Secretary Napolitano, can I just briefly, I mean, really interesting points raised, but I think we have to deal with the fact that you have to get to certification by the states first right and you raise the time aspect secretary which is totally fair. But, but dealing with the facts and this is probably always going to be the case but viewing this as a horse race rather than than getting to the end of the process drives that, that urgency and, and what anxiety about it instead of understanding and opening up the process to those ballots, the count didn't, it wasn't delayed, for instance, the counts in these states weren't delayed. They were going through the process to ensure that every valid vote was counted and counted correctly, right. And so understanding that as we look at the process as a whole that states have a job to do in certifying the election that includes the necessary controls and checks to ensure the integrity, the access and the security of the process that's needed to get to that certificate before we ever get to an electoral college or anything like that. And how much detail goes into it if you look at Michigan and what Secretary Benson and the local officials over 1000 local officials in Michigan do they check the township state then go up to the county and check the totals to the county then check the totals all the way up to the state. And so I think what we need to do as a profession as election officials are now put in the position to have to rebuild trust unfairly put in that by the lies we have to begin to look at what can we do to build greater transparency and open up the process so that folks can see all of the controls that are in place and I give a lot of credit I had the pleasure of serving on Secretary Benson's security committee that that looked at this it can't just be a post election audit like we saw in Georgia. It needs to be full opening of chain of custody and understanding ballot processing and registration records and how we maintain lists so that voters don't have to take anyone's word for it but in fact can see the evidence have the evidence to have confidence in the process and that means more resourcing for election officials. It is untenable to ask them to go quickly to do it effectively efficiently and do it transparently without supporting them investing in the infrastructure in a way that allows them to do that and they'll embrace that role election officials have long said it's our job to convince the loser they lost I think that's changed now it's now the job to convince the voters of the accurate result even if the loser doesn't accept that they lost and that's a higher bar and a greater challenge that we need to accept embrace and invest it. Right so we've got some audience questions and one is pivots right off of your comment Matt which is even given all of these checks in the systems and the accuracy and the confidence you had in the integrity of the count a significant percentage of the population don't believe it. They think former President Trump won the election. What ideas do you have for how we build trust and not have this kind of a situation which is really counter to our democracy. Yeah I'll go first just just very briefly and then open it up but it's got so the advantage one of the large advantages we have is the fact that we run elections at the state and local level and I think that's a real asset in this case because voters can engage directly in the process engage directly with those who run the process in order to understand to have their questions answered to gain transparency. The reality is in all of these states represented here in all the states. There are opportunities for voters to get directly engaged obviously the most productive way would be as a poll worker but there's a number as poll watchers and pre-election testing is public all of these things and then as election officials finding new and creative ways which I think election officials did during covid to open the process up so that voters can see exactly how each step is done how those reconciliations the amount of detail and minutiae that goes in election officials are detail oriented professionals that have these controls and so whether it's a post election audit and opening that up or offering the types of evidence around chain of custody ballot storage ballot custody. We have to continue to engage with voters and it's it's going to be a slog and unfortunately election officials have unfairly been put in that position to have to begin to do that and it's it's going to take real time investment and sort of just being the water just constantly providing that steady stream of information and evidence but the other part of this that that I think several brought up that's critical there has to be accountability for the lies the reality is if that those were who are pushing the lies would just stand up and say this election was fair it was accurate it had integrity that would go in incredibly long way but but I don't think any of us anticipate that happening and so we've got to deal with the reality that we have now. I think I'll add two things on that one I agree on the accountability point and I'll talk about that in a second but first. The issue actually is really quite heartbreaking you have citizens who were lied to by people they trust elected officials they supported that they voted for then turn around and lied to their supporters about their votes and the process in a way to further their own political agenda and and their own partisan gains and their own egos. So I think you know citizens that have the facts and the evidence that secretaries like myself and secretary Hobbs and others have worked to provide and is the court cases actually did demonstrate as the attorney general attorney general Shapiro can can underscore and so the evidence is there to withstand any amount of scrutiny the truth is is available for anyone who chooses to see it. But I think first recognizing I think citizens need to recognize that these folks these leaders lied to them about the election it was actually the most secure and accurate and and highly participated in the election that we've seen in recent history. And that gets to the second point of accountability because what we really need is for leaders to stop lying to their citizens to voters about the truth of the elections. And the only way we get there is by adding you know making sure that they don't they not only don't benefit by those by perpetuating those lies but there actually is some accountability there. And so in Michigan we proposed an anti deceptive practices act actually criminalizing the intentional spreading of misinformation and lies with the intent and so the knowing spreading of that information with the intent to disenfranchise someone to and there's multiple ways you can narrowly tailor it so that it withstands any strict scrutiny under the Constitution and doesn't infringe on You've got a First Amendment issue. Yeah, but but we have to I think really dig deep and look into how to protect our voters just the way that we protect customers against false advertising. We can protect voters against those who would seek to lie to them about their rights intentionally so and that's the type of legislation that we proposed and we hope other states do as well. I want to pick up on a few of the points that you're on mute. Josh Apologies, I want to pick up on Jocelyn's point and and Mads and come back to something Katie said a few minutes ago. I agree with the points that that were made by Jocelyn and the type of legislation that needs to be put forth and hopefully adopted. I also think it's critically important through our political process that the next nominees for Republicans forget president but the elections that are going to come up in a couple years for Senate and governor and Secretary of State and all these other offices that the Republicans were nominated aren't rewarded for perpetuating the lots I think that there is a political component to this as well that that really needs to the needs to still play out. But I want to come back to something Katie said which you know I tend to be a glass half full guy but I feel a little glass half empty I listened to Katie talk about how our democracy held with which it did if you sort of look at the bottom line which is on January 20 Donald Trump walked out Joe Biden walked in right, but there was just such an extraordinary amount of damage that was done to our democracy. And furthermore, I fundamentally believe and this is one of the catalysts for me that led to me reaching the conclusion we have to abolish the electoral college. I fundamentally believe that if Kevin McCarthy were the speaker and Nancy Pelosi was not and you combine that with Mitch McConnell at the time who was the majority leader majority leader. I have no doubt in my mind that they would have attempted to steal the election and not certify the votes from places like Pennsylvania, Wisconsin, Michigan and Arizona and try and hand the presidency to Donald Trump.  And so, but for the fact that Nancy Pelosi was the speaker and had a majority and thus was able to run the floor and, and, you know, control the votes and the apparatus there in the house. This could have been a very, very different outcome. I do not believe that there was enough control within the Republican caucus in the house or the Senate of the United States to prevent the type of stealing that that I know they wanted to do the guys like Holly and Cruz and other people. And that's what the voters attempted to do and so I think we have to come to grips with that reality. We have to educate in our schools and through our politics, we have to, you know, carefully nominate certain people and we have to make the kind of structural changes that are about a moment ago. You know, yeah, well said. And Josh, after the election there was, you know, so much litigation but one case that caught my attention was the case filed by the Attorney General of Texas. And he had, I don't know how many Republican attorneys general on the case they wanted to throw out your ballots and Jocelyn your ballots and and can you explain to me what possible issue they had and and the role of the state attorneys generals. I was a state attorney general. In 1999 to 2003 so it's a while ago. And there's an association called the National Association of attorneys general nag, not a great acronym but not a great name. No. But while I was attorney general and when I while I was attorney general when I started the idea was that attorneys general were somewhat politically neutral. They weren't enforcement officers and prosecutors, but we weren't particularly partisan. During my term, the Republican AGs broke off into what has now become raga. A few years later the Democrats had to do the same thing and we had daga so kind of fundamentally changed the job of being an attorney general. And so I was sad to see but not surprised to see that with with the lead of the AG from Texas that the Republican AGs from across the country were trying to throw out other state balance what do they do. Can you explain what their theory was. Well I can explain their theory but I can explain what happened. I don't I don't think they had a particularly honest theory. Look, this. This was one of the saddest days of my time is as the attorney general Pennsylvania, a job I'm just honored to hold when that lawsuit was filed and in a nutshell what the Texas Attorney General was saying, was that the Supreme Court should step in and invalidate the votes here in the Commonwealth of Pennsylvania, because he didn't agree with the choice that our citizens made that was effectively what he was saying.  And ultimately we concluded there was none.  In this case. I just kept going back and forth with my team saying that I really thought this was an active sedition that they were really trying to undermine our representative democracy and end it as we know it, because if they were successful take it to its logical conclusion, any state couldn't validate the votes in any other state, just because they didn't agree with their political choice. I was sad that, that they did this because, frankly, I've worked in a really bipartisan manner with my colleagues, especially on the opioid litigation so many other things that we've been dealing with over the last number of years. But I was especially sad when 17 of my colleagues signed on to that lawsuit, demonstrating their lack of respect for the rule of law, and I've spoken to them privately about that and been very direct and pointed with them in terms of how I feel. The idea that they would go through with that was very sad. I'll tell you I was a little bit concerned given the makeup of the new United States Supreme Court that somehow they might entertain it but I just kept saying to myself, there's just no legal rationale for this they can't possibly take this up. And at the end of the day notwithstanding the new look of the US Supreme Court they still adhere to the rule of law, and they threw the Texas suit out fairly quickly. I will just tell you from kind of an insider perspective, it's really poison the well amongst the attorneys general. It's very, very hard to work with a colleague that believes that the residents of your state don't count, and that they somehow know better than in my case the 7 million Pennsylvanians who cast votes for Donald Trump and some for for Joe Biden. And I thought it was seditious. I thought it was, it was unethical, what they did. And I think they simply are not fit to serve as attorneys general anymore given the role that they play in trying to undermine our democracy. Yeah. It may also been the way I looked at it is it was more for the press attention than the than the legal validity. Even in those cases I mean we all recognize that the power of the media to amplify our point of view. But I think we all recognize, I mean there's certain lines you don't cross and one of those basic lines you don't cross is you don't violate the oath you take to the Constitution. In this case the Texas Attorney General did and encouraged others to join him. He is absolutely unfit to serve and and certainly violated his oath in an attempt to merely suck up to a corrupt president. Yeah, yeah. Well thanks for that and while we're talking law Katie, we've got a question from the audience. The Supreme Court upholds the Arizona voting statutes at issue and Burnovich versus DNC. How will that affect future elections. Are there ways of improving voter access, even if those statutes are upheld. That is a great question and let me tell you my concern about this case is not necessarily the impact on these laws and Arizona these are this this lawsuit is old it was brought in 2016. These are laws that have been on the books in Arizona. I would like to see them go away and would be happy with a ruling that did that I'm not hopeful that that is going to happen but what is more concerning. That was just a little more background the oral arguments it's it's DNC versus Burnovich and the oral arguments were last Tuesday in front of the Supreme Court. And in the context of all of this voter. The legislation we're seeing across the country in unprecedented numbers Arizona leading the way in that that would restrict voter access and rollback voting rights. The arguments that were made to try to convince the Supreme Court to weekend section two of the Voting Rights Act will impact and across the country, people's ability to bring court challenges in the future on discriminatory vote voting laws. And that is to me what's more concerning and about the court case because that impact obviously is far beyond Arizona. Now I think that the court could decide to rule narrowly and just uphold the Arizona laws and not tackle the attacks on the Voting Rights Act. I am and I don't know what they will do and so that should be concerning to everyone in terms of future ability to use the court to challenge discriminatory voting laws, because we are seeing them being proposed and unprecedented numbers right now. Yeah, yeah. Well we'll find out when the when the court hands down it's an opinion. No question this is for you Jocelyn. In Michigan where any devices in Antrim County hardwired to the internet or otherwise connected to the internet. We've actually done a entire, what happened in Antrim County in Michigan was actually the result of a clerk not updating the software actually for this particular technology. But, you know, knowing that it was then utilized to spread this sort of false information and calling the question the results of the entire election not just in Michigan but nationwide. We, we actually did a full audit of the of every ballot every paper ballot and our citizens vote on paper ballots in in Michigan just like the majority of citizens in the country. And so what that then enables is for an audit to happen of the paper ballots to affirm that they were accurately tabulated by the voting machines and indeed that's exactly what the audit showed. And it's interesting I think once the all is said and done that that sort of case study emerging out of Antrim County will be very illustrative of just how much bad actors can try to leverage different pieces of misinformation to really create an entire conspiracy theory about an entire election. But we simply just responded to all of that with the again the truth and that's again why a verified paper ballots which we have in Michigan which most states have are so important to protect because as CISA's former leader Chris Krebs and many others says you can't hack paper. And through those paper ballots were able to verify the results and the accurate results of the Antrim County elections but again, it was and I'm sure Matt may have additional to add on this whole saga, but it was really illustrative of how really innocent administrative errors can be falsely and wrongly extrapolated and used in a way that just furthers and feeds false information and conspiracy theories and really is wrongly used to undermine people's faith in the process. Matt, you want to add anything there. Yeah, no secretary Benson is exactly right and it's sort of fascinating slash depressing to look at sort of how these conspiracy spread that the one related to Dominion voting systems was really the combination of three or four very frankly in my experience being in elections since 2005 have sort of been around the fringes for a while. This idea behind Hugo Chavez and Venezuela being involved in voting systems that that's been around since 2005 2006 that the theories around site L and results reporting in Barcelona and the relationship between President Obama and and that company and George Soros has been around since 2008 2010 all completely false all completely wrong all not based in fact what we saw in this election was incidents like Secretary Benson described in Antrim County incidents in Georgia that happened in every election by the way where humans are involved and mistakes are made and quickly rectified Antrim County quickly rectified once identified to them what what the issue was had the auditable records right there could correct the answer but we saw this combination into this broader conspiracy that included at times the CIA included DHS putting isotopic watermarks on ballots to be able to track them to look for the fraud and to me perhaps the most disturbing part is these took hold and spread very very quickly as the attorney general talked about and despite the best efforts of all the officials the loudest voices a recent report I know I'm not supposed to push a report from Stanford on a cal broadcast but a recent report from the election information project which includes Stanford Washington Graphica some other partners shows that that really 20 to 25 major influencers just jumped on this misinformation pushed it broadly and the result was you know death threats threats to election officials and then eventually January 6 and to me the loss of a transfer peaceful transfer of power and that's a loss of the very core of who we are as Americans and we we're now going to have to come to grips with the fact that we we didn't have a peaceful transfer of power and what does that mean for us moving forward to begin to deal with that all based on just insane theories that that word you know every good piece of dis info or effective starts with a little nugget of fact it answered county have a problem yes answer county had a problem but none of the rest of the facts around that were correct we had the paper ballots I mean the easiest example I can give you 5 million plus ballots in Georgia have been counted then hand recounted by hand without any machines and then recounted again all coming to the same result and we still have people saying that that the result in Georgia was incorrect and we've got to come to grips with that and begin to really ask ourselves some challenging questions both about our information ecosystem our election ecosystem and what evidence we can continue to provide and how to hold people accountable and I'll just chuckle a little grenade in the middle where the previous conversation about the courts we've got to come to terms with the fact that the courts were used for dis info as well and the way that the courts were used on how we can begin to respond and recognize the use of the courts in that way as well. Yeah, Josh do you agree with that you think the courts were used as a dis information device. I think the the courts were used by attorneys who were willing to act in an unethical manner as a way to spread their dis information I'm pleased to see. You know, Jocelyn's colleague in Michigan and my good friend and colleague Dana Nessel, you know seek sanctions against some of the attorneys who were involved in those cases and we're certainly looking at that as well and in Pennsylvania. The good news that at least from our perspective is both the state and the federal courts, all the way up to the US Supreme Court and our state Supreme Court did hold I mean they they saw through that misinformation. And they sided with the truth, but we have got to, we can't let that become a norm where the courts are used as a way to spread this misinformation the way actors like Giuliani and you know some of those, you know the other you know the other nutballs that were associated with him did, you know when it came to the, you know, the federal and state courts. Yeah, yeah. It was amazing to watch as a as a as a lawyer. I must say it was almost it was it was embarrassing actually to see fellow members of the bar. I'll do that. And by the way, I think, I think there's serious questions not just about those lawyers who were acting on behalf of the president but some of the attorneys general we talked about this before when it came to the AG of Texas but you know at what point does you know that the that the AG's are on they probably have a little bit more latitude than a private attorney but at what point do they go too far and can they be sanctioned or disbarred and I know that there are some in Texas seeking to disbar general Paxton I would encourage them to continue to move forward with that process but you know that's another issue these these government actors, including some who are working at DOJ and trying to work directly with the president former president to overturn the election at at what point you know where they held accountable as well it's not just the private attorneys, some of the government attorneys as well sorry to interrupt you. Yeah, no, no good really good point. And I'm going back to an audience question it's somewhat related. One of the panelists on our first panel this morning. I brought up the that in other countries. Those who are responsible for conducting the election are not themselves partisans. I gather civil servant types. And if, if we had that sort of system in the United States, do you think it would increase the perception of fairness of our election system. Two things one, and I wrote a whole book on this where it really you looked at the secretaries of state as guardians the democratic process local election administrators as well. And also whether someone should be appointed or elected as in Pennsylvania you have an appointment of secretary of state in Arizona and many other states, like Michigan you have an election. And one nonetheless, local level clerks are elected in Michigan to administer elections. And really what I think a lot of the research and what my experience is really underscored is just it's just all about the person. And yeah we exist in a political ecosystem where there are pressures, but at the same time. I know for myself as an attorney and as really someone who's been a voting rights and election attorney throughout my entire career. My allegiance is to the Constitution and to the voters and that's why I ran for this office and I think I intentionally, you know, I don't endorse candidates in any election. I don't even endorse appointed people seeking appointments because they're asking elected officials to appoint them, and that also creates a potential conflict. And I think one voters demand that elected secretaries of state or election administrators in particular act in a non partial impartial manner. They don't elect those who seek to do so, and, and don't elect those who use their office to put their thumb on the scale and there are some that do. Then we'll see, you know, be in the best position possible but I don't think the infrastructure itself there's no infrastructure that really can lead to a place where you have non partisan election administrators you just simply have to put them in place and Now I do think and I have talked folks about having a code of ethics for election administrators secretaries of state much like we have a code of ethics for judges that they must abide by or perhaps lose their ability to perform that job I think it is important to get those administering elections do so as referees not as members of a particular team that's on the ballot, but again it all gets back to voters selecting and electing good people in these positions and advocating and supporting them and those good people can be Democrat or Republican they simply have to be beholden to voters first country above party. Yeah, well said, well said, anybody else want to chime in on that. Just real quick, just to drive home, I agree. In my experience at the state local level, working in election administration is as soon as you become an election administrator. You stop caring about the results and you just care about the integrity of the process and that's just not that local election administrators don't want to see themselves in the news and want to know that they carried out their job with fairness. And we need look no further than this last election. You had Secretary Rathausberger in Georgia, a Republican, you had Secretary Sigarski in Nevada, a Republican, you had Tina Barton in Michigan standing up Republicans taking accountability and owning the process because that was their job that was the oath that they took. And that's consistent with my experience in elections over over now 15 plus years I'm getting old in in working with these folks as soon as they take that oath their commitments to the process to our democracy. And we saw that play out in this election that doesn't mean we shouldn't have checks we shouldn't recognize that a bad actor might not end up in that position and that's why the process has checks balances and ways to hold people accountable. But in the end, I think we saw in this election that the goodness of election officials was critical to us having a successful election. Yeah, just wonder Katie were you in touch with some of your Republican colleagues in the aftermath of the election. Yeah, we talk weekly. We have weekly meetings every every week. But in addition to that, particularly myself, Secretary Hobbs, Secretary Sigarski, and Secretary Rathausberger were in close and sometimes nearly daily contact with each other as well as Secretary Bookfar. They're then Secretary of State in Pennsylvania, where we talked often about our unique challenges supported each other and several of our colleagues did as well.  Well, we're, we are almost out of time. This has been a terrific discussion. I'm going to do a last lightning round as it as it were. Based on your experience and what you saw last November. What would be the number one change to improve our election system you would recommend. Matt, we'll start with you. We'll start with the feds. Well, not having the feds tell the states no that without question having full audit ability votes at this election had 9495% of ballots cast were on auditable paper records. We need to ensure that 100% of votes cast in the upcoming election has that and not only that that we're doing good, efficient, transparent auditing of those results so that voters can interact and understand, look at the evidence to have confidence. And then the second one I'd say is elevating election officials voices. We need platforms like the social media platforms and Google to ensure that the trusted info from election officials is prioritized is pushed if I do a Google search about Michigan elections. I should get secretary Benson's websites and the facts, not some of the conspiracy theories so that those are the two that I'd start with. Katie. Well, I think Matt brought up some great points. I would love the ability to just make voters listen to what I had to say and to listen to the truth. In the absence of that I think I'm investing robustly in public education campaigns, giving election officials the ability to communicate directly with the people that need the information. You're good, Josh. I would note for the record you've asked all of us for one thing and Matt gave at least two or three so I'm not going to be bound by, by the way, I agree with the, the comments my colleagues I think above all else. Leaders need to speak and act with moral clarity, and we lost that for quite a bit of time and that's not about your perspective on one issue or another it's, it's about speaking with moral clarity adhering to our Constitution and so I would say that is above all else the most important thing. There are certainly some changes to our election laws here in Pennsylvania that we need to make, but overall I think we need to get back to, you know, a common set of facts and actors who are willing to act with moral clarity. Yeah, I couldn't agree more and I of course agree with everything that's said and to me it also underscores the need for sustainable funding and resources to invest in our elections, whether it's about educating voters, countering this information or just supporting the infrastructure of a secure and accessible democracy. It is to me, ridiculous that the most important thing we do as a country hold elections to ensure and select who has power to make every other decision on our behalf is so severely underfunded. And every so many of the challenges from educating voters to auditing the election and everything in between that we encountered this year could have been significantly avoided or or eliminated with appropriate funding and we asked for it from the federal government. I don't work from our states but I do think every single leader who has a state in ensuring our democracies are accessible fair and secure must put resources behind that and really invest in our local and state election administration so that we can do the job that we've been elected or selected to do of educating citizens about their rights making sure they have the tools and resources to exercise their vote and then supporting and investing in our election administrators as they tabulate and secure the process. Very good. This was excellent I'll just close by saying on behalf of a grateful nation. I am so glad you were in your roles. During this turbulent period. You were really needed and you stepped up to the plate beautifully and I'm grateful. And I know lots of other folks who are very grateful so thank you for joining the panel today. And I hope our paths cross again. Thank you. Thank you everybody. Everybody. Good afternoon. Welcome everyone. I'm Janet Napolitano. I previously served as the president of the University of California Secretary of Homeland Security. But now I'm a professor at UC Berkeley's Goldman School of Public Policy, where we are establishing a new center for security and politics. I'm excited to host today series of events on the timely and important topic of free and fair elections. Our third panel today focuses on what new or existing tools and strategies can be deployed to further improve the integrity and security of us elections. And I'm going to be moderated by Henry Brady, dean of the Goldman School of Public Policy at Berkeley, a leading expert on the practice and politics associated with the US election process. Henry, thank you for leading us in such an important discussion. Over to you. Thank you, Janet. It's great to be here. And it's wonderful to have you as part of the Goldman School. It's great to have this new security in politics center at the Goldman School dealing with such important topics. Before I introduce our distinguished speakers, let me note that you can ask questions for those out there watching. And you can do that by submitting them using the form on the Goldman School's website that you use to register. So please submit questions. For the distinguished panelists, let me start and go through them alphabetically. Our first panelist can be foot is currently serving in her third term as the clerk recorder and registrar of voters in Inyo County, California. And during her long tenure in public service, can he has testified before the California legislature and the little Hoover commission about improving elections. I mean, I think I'll go to you and then I'll introduce each speaker as they come up. So please tell us about your experience there but before doing that actually let me just do a short introduction and talk about trust in elections. First of all, who am I, I'm the Dean of the Goldman School of Public Policy. I was involved with the butterfly ballot episode in Palm Beach, California and Palm Beach, Florida in 2000. In 2003, I was involved with a court case to get rid of voting addict style punch cards in California. And that case temporarily delayed the 2003 recall for a little while because of worries about the voting addict style punch cards. And then on April 18 2005 I testified before the Carter Baker Commission to try to improve elections in America. So I've had a long experience for the last 20 years with elections. Let me just say what the basic problem is here. The basic problem it seems to me is loss of trust. We've seen a loss of trust in government. And now we're recently we're seeing a loss of trust in many different kinds of institutions, including science, higher education, the press, religion, medicine, and now elections. And it's not that there's been suspicions about elections in 2000 there were certainly those who worried about whether the election results were fair and right. But if you go look at the data from that period people were much less untrustworthy, they were much more trusting of the election result than they are today. That's the polling firm morning consult in late January of this year, found that only 60% of voters trusted the election, either some or a lot 40% basically didn't trust the results. And that was 82% of Democrats trusted the results, not surprisingly, President Biden one and 33% of Republicans only 33% trusted the results. And this demonstrates what's even worse it's not only that trust in institutions is declined. It's that it's become polarized. One side believes in particular institutions, and the other side doesn't. Democrats tend to believe in science higher education in the press Republicans believe in religion the military and institutions of that sort. And now we have this division between the two parties with respect to elections, and a really deep seated distrust on the part of Republicans. Let's say another word about what is an election. We sometimes think well that's just people going to the polls, or they're filling out their mail in ballot. In fact, it's much more than that. It also involves a period after the election generally called the canvas, or the certification of the results. And in this period, all the results are checked, corrected, and finalized. And the period for this can be quite long. The states do it in a week. Vermont, South Dakota had deadlines this last election, a week after the November 3rd election, but some as much as five weeks later, Maryland, Missouri, New Jersey, and California is one of those And what's the reason for this. The reason is it's really difficult to count all the ballots on election night. The only reason we even try is because the media have historically wanted results quickly. And in fact, television really pushed for this. And television is one of the reasons why we introduced automatic style punch cards, which allowed computers to count the votes, and why we have continued to invest in technologies that will count things quickly. But there's no reason to think that counting quickly is necessarily going to get the right result, especially now when we have so many mail in ballots, and all an early votes, and all sorts of other modalities by which people vote. So in fact, it makes sense to think it'll take a little while to figure out exactly what the right count is. Now, what do we want to do with the right count. Well, first and foremost, most you want to convince the loser that he or she lost. Typically, the winner is convinced that he or she won, rather or not, that person did for obvious reasons, but it's the loser that has to be convinced. Now increasingly, we also have to convince the public. And that's a much bigger job. And how do you do that. Well, one way is just to invoke the authority of the government, but we see that that's no longer probably enough. Another is the cultural belief that the society would not let fraud occur. That's been destroyed by communications, which have undermined trust in elections. A third way is that norms have been met, such as lawsuits have been submitted, they've all been rejected and therefore since the norms of the legal system have been met, the election must be fair and correct. We see that that didn't work so well. So now we're going to talk a lot today will not exclusively about the performance of this system and how we can prove it to be adequate. And the question is, how can we do that. How can we reestablish trust in elections. And let me start with Kami who I just introduced a few moments ago, who is the clerk recorder and registrar of voters in in New County, California. And then we'll go to our next speaker. Thank you so much for having me. So a little bit more about the area where I administer elections in New County, the second largest county geographically in California with over 10,000 four miles. And we have a little less than 11,000 registered voters. So we're very dispersed in nature. And as an election administrator, I've directly overseen 22 elections, including this past November election. Prior to that I was a poll worker for several years. We also have five major American sovereign nations within our county boundaries, the made up of the Bayou Shoshone and the government. And we are the one of the most purple counties in California, I would argue maybe even in the nation, their 14 vote difference between President Biden and former President Trump and the presidential election. So while the accusations of vote reading and machine manipulation are fresh in our minds. In this last election cycle, I had more voters also ardently believing that Russian voting directly led to the outcome of the 2016 election. So because of the nature of the area that I served of small men and politically diverse communities who are often remote and distrustful the government in general. It's necessitated that I seek out ways to demonstrate to voters that the elections were connected properties, regardless of trust. So essentially taking trust out of the equation completely. With that as a backdrop, I'd like to talk about the information pathways and data consumption. This is something that we've been talking a lot as an election community and as a nation in the form of disinformation. And then I want to talk about a few additional steps that we can do now across the nation and every election jurisdiction to actually secure the integrity of election administration itself. The reason I think it's important to start with how society consumes information is because we hear the same thing brought up over and over again about misinformation and disinformation. But there's not a lot of talk about what we can actually do to solve the problem. But addressing this issue is really the largest security threat to elections as far as I can tell. So it's important to understand the mechanisms that are manipulating the flow of information. Because this problem permeates broader society, as we just heard it's not exclusive to elections, it's infected public discourse and policy in all areas of modern life. It's important to look at it from a holistic and systematic approach rather than just focus on election specifically. Because until we address this larger problem of partisanship and divisiveness that has gripped our culture on both sides of the flow while it's going to be almost impossible to have honest conversations about meaningful election reform. But we all know free and fair elections should not be a partisan issue. We should all agree that if you cast your vote for candidate A, it should be counted it for candidate A. So I personally believe that the first test to fix this problem is going to be something akin to a internet bill of rights. Something that we can use to protect voters in the general populace from predatory behaviors of nation states and campaigns and domestic troublemakers. All of the people and organizations that have been data mining, what we like, what we love our habits, who we communicate with and taking that information to exploit us so that they can feed us. This and disinformation that's convincing to us personally that feeds into our own biases, fears and anxiety. I think that we also need to be looking at things that surround the FCC regulations, things that were repealed in the 80s during the Reagan administration under this libertarian mindset. One of the things that was repealed during that time, I think if you grew up in my generation, you maybe saw commercials on television about schoolhouse rock how bill becomes law or the way that the government structured. That was because regulation required that every media outlet that aside certain amount of time for public education. And those are the kinds of reform. But I think we're going to be needing to address in order to focus on the misinformation and the way that we consume information, the way that we share information, and the way that we communicate with each other in this world of instant, instant thought and instant connectivity. So now I want to pivot to what I will call my top 10 practical things that we can do an election administration now in every election jurisdiction across the country to help secure the administration itself. Improve to the voters that the election is secure. Again, taking trust out of the equation. To begin, because voting systems are certified at the federal and state levels, we need to demand the highest level of security and transparency around the point of certification, including exploring ways to incentivize open source publicly on voting systems. Because even though the software may be perfectly fine and all of our voting systems now it might. As a local election officials, I can't prove it to the public because they're proprietary. So I really think we need to start focusing on the point of certification so that we can use the public will accept and trust. At the local level all election officials, they can become members of the election infrastructure information sharing and analysis center that's a lot of words, we call it the EISAC for short. It's free to join. And this is allows information sharing across federal, state, tribal and local government. We have access to credit information in real time, including resources to help mitigate the stress. The 11th election administration, excuse me, election systems have been designated as critical infrastructure. We now have access to things like penetration testing, phishing testing, cyber hygiene, Albert sensors, all kinds of information and resources from the Department of Homeland Security and no additional cost accounting. So there's no excuse for any election jurisdiction in the country to not avail themselves of these resources today. Every staff member should be trained in cybersecurity from beginner to advanced. That's something that my staff and myself have been through that those courses, we need to be able to recognize appropriate ways to behave online. Anyone who works with transports, counts ballot should maintain strong chain of custody documentation to prove that procedures are being followed. And these documents should be made public. They should be published on our website if possible. They should absolutely be made public to anybody who asked for them. This should also include inventory tracking of election equipment, fund drives, all components of ballot counting. Also with so much reliance on technology and the growing threat of rolling blackouts, fires, power safety shut off, any possibility that can take down the power grid. Power backup generators should be the standard, not the exception. We installed them all of our polling places in the last election and they will remain. We should also be forensically monitoring our networks so that we can prove that no intrusions occurred. And if they did, we can discover what was accessed when we did that in Año County through the entire 2020 election cycle. We knew when there was any issue with our internet connected systems, any of our networks we were informed immediately. I also believe that hand-marked paper ballots with ballot-marking devices only for those that need them are the best method to ensure voter intent. You're going to be hearing from another panel today who's probably going to talk a little bit about that. And instilled some work to make sure that election officials like myself understood the threat with anything other than a hand-marked paper ballot. Scanners should be equipped with imprinters that stamp the identifying information on ballots such as the date, the time, the batch number, and the ballot order within the batch. As they're being counted, we did that in Año County. This prevents even accusations of the same batch of ballots being fed through the scanners more than once. And finally, at the culmination of documenting every other area of the election, once the candidate is complete and every voter record has been reconciled, every voter ballot has been counted, we should be adopting risk-limiting audits, which are the gold standards for audits. So this is a non-exhausted list. Funding is always an option, but if we can do it here in Año County, you can do it anywhere in the United States. We're a very cash-strapped county, we're very rural in nature, but it doesn't prevent us from having the highest level of securities that are available to us. And I just will leave you with one final thought, that although I'm speaking to you today as a person who conducts elections, the most important position to hold in a democracy is that of a citizen. We as election officials serve you, the people, and without your advocacy and engagement demanding evidence-based election systems and the highest security standards and funding priorities, we're going to continue to struggle with these questions about legitimacy, like we did in 2000, 2004, 2016, and yes, most recently in 2020. So let's work together to improve election security to avoid a happy to go through this in the country again in 2024. Thank you. Thank you, Kami. That was great. So we're going to talk more about risk-limiting audits and train of custody and the regulation of various machines and how those all interact and why they might be important. So if a lot of that seemed a little like some jargon, it is, and we're going to do some of that today here, but we'll try to explain it as best we can. Next speaker is Wayne Williams. He's the former Secretary of State of Colorado, ending his term in 2019. Wayne also served as a county commissioner in El Paso County and was then elected to the role of county clerk and recorder. And by the way, this is a role that's sort of standard across the nation, and often what happens is the person who's the county clerk and recorder does elections but also records deeds and things like that. That's a fairly standard thing. Wayne, please. Thank you for the opportunity to participate today. I apologize for being a little bit late like Secretary Napolitano. I am a former Truman scholar and was chairing our regional panel and it went a little bit late today. But let me talk about why the elections matter. Let's, and then I want to talk about some of the things we did in Colorado and I'll try and emphasize what I think are the most important parts of that. I first got actively involved in elections in 1997 when I served on a canvas board, I was one of our county's three members, and we had the punch card balance. And we had specific standards that said what counts and how many chads and how closely they have to be hanging at all of those things. So a few years later when I am watching the process in Florida, where different people were advocating to change standards mid course. It began, what I think has been a series of things that have undermined the confidence in the election systems. And so, you know, one of those things is that as election officials, we want to make sure those standards are set well in advance so that everybody knows what those rules are going to be. And then I've got six. So, so Cammie is, you know, from California they count higher than I count in Colorado, I've got six general things but they're pretty broad. The first is something that impacts anyone who's ever walked precinct or reviewed a voter list and that's that you need an accurate voter list. That's particularly true in a state like Colorado where we mail ballots to people. And so we use a host of different processes to make sure those are accurate from gathering information from other government agencies such as when you get a driver's license update, or your car registration changes. From looking to make sure that folks who have passed away, whether they're from the Colorado Department of Public Health and Environment or the Social Security Death Index that they are removed from the voter rolls. We work with the Department of Corrections for those who are currently serving a sentence for a felony conviction. We also check to make sure non-citizens are not registered. So these are objective things that counties and states can do. We actively participate in a cross-reference of our database with the Electronic Registration Information Center. So we're comparing our information to other states so that we can first be accurate on our list and second ensure that that person who moved gets the correct information going forward. And that's critical because that's something that's publicly available. And so if you get a publicly available voter database and you see that it doesn't match who lives there, that undermines that confidence. And so that's one of the reasons why that needs to be there. The second part of it, and Kami already talked about this, is having voter verifiable paper ballots. And that means a ballot that the voter can actually look at and ensure. And this was one of the reforms that we adopted when our Secretary of State of Colorado, ensuring that we had a voting system that actually provided a paper record. Even if you voted on an accessible device, there was still a paper record that could be verified by you, the voter. And by someone who wanted to audit results later. Absolutely critical. The idea that was at one point popular, that you would trust the ether, or what is on a computer card, is not something that provides that evidence you need to ensure that the voter gets the correct information. What is on a computer card is not something that provides that evidence you need to ensure the election was conducted accurately. Ultimately, I adopted standards that required that voter verifiable paper ballot. Not everybody was happy. Some companies that didn't make sufficient machines sued a couple counties sued the legislature reviewed it. And when it became apparent four or five years ago now that foreign governments were seeking to interfere in elections, all of a sudden other people began to recognize that it's there. Those of us in the election community are dealing with this on a daily basis. Some people only pay attention when there's a crisis. And so we have to make sure that we are engaged and that we can explain what we're doing. The third thing you need is a secure ballot return system. Someone needs to be able to assure their ballot gets back there because and this is one of the challenges in a state like Colorado that does vote by mail. We had predominantly, you can only mail it. When I was secretary we began to have drop boxes around the state. These made it much easier and ensured that the geographic disparities and mail service deliveries were evened out. We also made it so that you could have total confidence because those drop boxes are under 24 seven surveillance and the ballots from them are picked up by bipartisan team of judges. And so that secure ballot return for a mail ballot state like us is important. And that's the perspective by the way I'm talking about in Colorado everybody who's a active voter is meld a ballot. There are some in person options. And that's the perspective I'm talking about as I go forward. The fourth item in a mail about election is some form of verification at the present time I believe signature verification is the strongest one, where we actually are able to say this was the vote cast by this person. The fifth item is a series of procedural protections and one of those I alluded to already which is at every step of the way we have bipartisan judges. Notice of whether the clerk and or the secretary is from a particular party. The judges need to be across from across the political spectrum and you need someone, you need Democrats and Republicans who are both there, verifying the process every step of the way. And we also have watchers in Colorado in addition to the judges. We, we changed the regulations when I was secretary to ensure that the parties were the ones choosing those judges so it's not. So so an election official can't just choose a friendly person from the other party to them. The parties are actually part of that, so that they can have confidence. I had a discussion recently with a Republican Party executive committee and, and they were raising questions about the election and I looked at the chair and said, who did you appoint to test the machines. And pointing out that yes, this process is one that is conducted by the election official but always with bipartisan teams of judges throughout that process. It also means that those those procedural protections include not connecting machines to the internet, and ensuring that the security and can we talked about some of those issues is there, so that those are not tampered with. And then this the sixth item, and can we refer to this as well is making sure that those results can be and are audited. And the purpose of all of these protections isn't just to make sure it's done right. It's also to be able to show people that it was done right. And as Henry was talking about in that interest in those introductory remarks. It's not the winners you have to convince it is the people who lost the election and their supporters, and you need to have the evidence to back that up. One of the other changes we made in Colorado is we made all of our ballots subject to the open records loss. So, we aren't playing a no you can't see that no it's hidden trust us we got it right it is a process that is open and verifiable both through in a formal audit so we adopted the nation's first full statewide risk limiting audit. But it's also available. If you wanted to request every single ballot in the state and do your own hand count by golly you could do that in Colorado. And so we want to be able to make sure that that process at every step of the way is verifiable you've got bipartisan judges you've got video surveillance, and you've got voter verifiable paper ballots that are audited and are open. So those are some of the things we've done in Colorado. And certainly for all of us around the country, we're all going to conduct the elections a little bit differently. And that's fine but all of us need to be able to show and demonstrate that those are that that process is accurate and we need to be able to prove it not just say trust me, or as President Reagan used to say trust but verify. Thanks. Great. Thanks, Wayne. Jennifer moral our next speaker has alerted me to another bit of jargon. I think Wayne when you say judges you would, you're really meaning what we would call in this state at least poll workers is that right. That is correct so thank you. Nice. We sometimes luncheon so poll workers, or people who help count the ballots, or who do any of the processes in Colorado, a county, you know, our largest, our largest population county has about, you know, 7800,000 people. We've got an election staff of a dozen. And so there's no way they're doing it so all of the work in the election. So whether you're called poll workers or election judges, or whatever the case may be those individuals are citizens from the community who come in and help. That's what helps make the election work and none of us want a system that has so much bureaucracy you don't have that help. Plus, it's an opportunity to make sure that both parties have representatives at every step of the way. Poll workers, election judges, ballot judges, whatever the term may be so thank you for the clarification Jennifer. And I would just also add that good training for poll workers is a really important thing too. So that's something maybe we'll talk more about our next panelist is Jennifer moral, who has served as a local election official in Utah and Colorado. And as the partner in the elections group where she consults on election administration and auditing. She works as a consultant with the democracy fund, leading the election validation project, which seeks to increase trust in election. Take it away Jennifer. Okay, well, let me start first just by saying thank you, Dean Brady professor start it's really an honor to be here participating in this conversation. What a great group. A little introduction just to tell you all how I got here and maybe set the stage for wherever at now. In 2009, I was a city council candidate on the ballot for a local municipal election in Utah. Before the election was to take place things went a little sideways, and I was asked to withdraw my name as a candidate and oversee the election. I knew nothing about running elections and I spent the next 48 hours immersed in Utah's election code. And on the phone talking to anybody who was willing to give me advice on what I was supposed to do. I promise there is a much better way to be introduced to election administration, but this crash course in a long night of manually counting and recounting to ensure the totals and tallies were correct. So I left me convinced that there were opportunities for improvement, both in how elections are administered, but also reinforced that this is something I wanted to keep doing. It was, and it continues to be an exciting puzzle to solve with an endless array of challenges. So I spent several several years working as a municipal clerk wearing many different hats besides election official as Dean Brady alluded to. I was in the elections in larger jurisdictions of both Utah and Colorado. I was during my time as the deputy of elections in a Rappahoe County Colorado that I was introduced to the idea of risk limiting audits or RLA's as we often refer to them and we'll probably refer to them from here on out. The fact that no one was able to easily explain to me exactly what an RLA was, and what it did made me immediately wary of the idea. In 2018, after playing your role in the first statewide risk limiting audit in Colorado, which you just heard former Secretary Williams talk about, and Professor Stark assisted with who we'll be hearing from. I was really hooked. And I decided to leave running local elections to focus full time on pioneering election audits and evangelizing the benefits of our liaison. Some of you probably aren't surprised when I tell you that well meaning colleagues called to say, Are you crazy? No one is going to be interested in election audits. How is that even a job? I saw the impact it had not just providing a way for me as an election administrator to validate that the election had been conducted fairly and accurately. But the requirements necessary to store and organize an account for ballots as a precursor to audit meant that I had documented evidence of the number of ballots in our possession, and I could identify where each and every one of those ballots was located. More importantly, and Kami mentioned this, I could balance that against the number of ballots issued in cast. So it provided this whole new perspective to election accountability and transparency. And I also found it fascinating when I first started thinking about the broader concept of election audits. This has played a role in US companies since the creation of the Security and Exchange Act in 1934. So essentially providing a way to ensure financial records and transactions are reported accurately and a system of internal controls is established and followed. I think that's something that allows all of us to feel confident about our financial and banking institutions and we know many organizations today both public and private rely on audits to evaluate processes and internal controls. And so I started asking the question, why didn't we have similar audit standards for elections. So I set out to write some guides that would help state and local officials better understand what an RLA is, how it works, the various methods that can be used when conducting an RLA, and what is required from both the voting system and operationally to conduct an RLA. And I tell everybody that 80% of the work in conducting a risk limiting audit happens prior to the election. There's a lot of front end work that has to be put in place. So I ended up with four guides and they cover everything from policy to terms and definitions, how to conduct a pilot as a way to provide a hands on experience for learning how the RLA process works, and a guide to ballot accounting. And this is an area where there's still a significant need for improvement in most jurisdictions. And in addition to these guides I've had the opportunity to organize and conduct RLA pilot projects in several states. So speaking for a minute directly to the academic community watching this if you want to see a policy or a tool or strategy implemented. One of the best things you can do is connect with a practitioner with an election administrator and develop a pilot program where you can see firsthand what it takes to transform theory into practice. So moving forward to 2020, I along with my business partner formed the elections group and we partnered with state and local election officials who were looking to implement new programs or improve processes. And our team of election experts, all former election officials were able to quickly provide guidance and resources and direct support for those jurisdictions in need. But this new avenue of work really gave me a front row seat in 2020 to experience how elections are run across the country, but also allowed me to see firsthand some of the challenges. And it became more clear than ever that good election practices set the stage for successful audits. Despite the challenges of 2020 elected and appointed officials from both parties worked together to mitigate what really could have been a crisis, as we heard in an earlier panel today. They did this by instituting health and safety protocols recruiting and training hundreds of thousands of new poll workers and expanding early in absentee voting and other alternative voting options and processing and tabulating a record number of million ballots. And I bring this up because I think it demonstrated a remarkable amount of resiliency, which should reassure all of us. As we're thinking about the challenges that we need to tackle around mis and disinformation and improving election security and integrity that the ability is there to meet those challenges. They also became expert crisis communicators, many for the first time with limited staffing, they did a remarkable job working to inform voters and the media and other stakeholders about the facts around how to vote and where to vote and when to vote. And they were really at the heart of a collaborative effort to amplify trusted information, which is really vital for us to keep in mind is we're exploring better ways of providing evidence that the outcome of an election is correct. Jurisdictions of all sizes provided us with virtual tours, their ballot processing facilities, and created graphics and short videos explaining how each stage a ballot processing worked, how ballots would be secured and accurately counted, and what we could expect while we waited for results. So I think in addition to this being the most secure election we've ever had in many ways, it was the most transparent of any election we've ever had more local jurisdictions turned on a camera, so that the world can watch the work they were doing in real time. And I think that effort revealed a couple of things. First, for those of us that have re-elections, this is no surprise that counting ballots is tedious monotonous work. And second, and maybe more importantly, that this added transparency alone was not enough. And that we still have some work to do in educating the public on how the whole process works. And more importantly, we need to continue pursuing ways to provide evidence that the work was done fairly and accurately. So I want to end my introduction by asking a question. Why should we audit elections? And my answer is election audits and not just tabulation audits have the potential to detect voting system errors, whether that is a human error or misconfiguration or hacking. They provide accountability to voters. They can deter fraudulent activity. They help assure us that votes were issued and counted and reported accurately. And just as important, they provide feedback per process improvement, something I really valued as an election administrator. And so the question we all should be asking is why don't we have a uniform set of practices for conducting election audits across all states and jurisdictions. So there are three quick areas I'll just touch on where I think there is opportunity to further improve the integrity and security of our elections. First, every election operation should have documentation. So this includes things like standard operating procedures, chain of custody forms, ballot reconciliation logs and ballot manifest. So having a trustworthy paper record is essential and I'm sure Phillip's going to talk more about this. But having a trustworthy paper record starts by having written procedures and logs for every task that gets performed. Second, pre and post election testing and audits can provide a way to verify the work performed by staff and the many temporary workers and poll workers that are brought in to assist, as well as verifying how election audits work were functioned. So those are things like that might include, you know, regular maintenance of the voter registration databases, as Wayne mentioned, maybe someday systematic audits of those databases for anomalous changes, GIS audits of the voter district and precinct assignments, compliance audits around security procedures, both physical and cybersecurity pre election testing of the voting equipment to ensure it's been programmed correctly, and it's operating as expected. Tests of the ballot layout and design and resource planning and allocation audit, so essentially ensuring enough equipment supplies and people have been allocated to meet demand. And finally, we need to improve the way we use this documentation as a way to continue to inform how procedures work and improve how we present those test or audit results as evidence, as proof that the outcome of an election is in fact correct. And I'll leave it there and I'm really looking forward to the questions and the rest of the conversation. Thank you Jennifer that was great. Nice overview of the issues. We're now going to go to Philip Stark he's professor of statistics at UC Berkeley. He's also associate dean for mathematical and physical sciences. He's received the chancellors award for research in the public interest for developing election auditing methods, risk limiting audits that have been incorporated into law in eight states. He currently serves on the board of advisors of the US Election Assistance Commission that was set up by the help America vote act that came after the 2000 election. And he's also on the strategic board of advisors of the open source election technology Institute. Phil knows lots and lots about the details of how these methods work statistically and electronically. Phil. Thank you so much. It's a delight to be here and a delight to share the virtual podium with some of my favorite people in election integrity, people I both trust and respect and enjoy, I guess that's trust and respect and enjoy more than two things. I'm going to share my screen now.  Let me see if I can make this happen. Oops, sorry. All right, I hope that's visible. So, Henry mentioned earlier that an incredibly important goal is to improve public trust in elections. I'm not good at that. The piece that I'm good at is something that I think should come before the trust, which is the trustworthiness. And my focus in this area has been trying to figure out how to make elections more trustworthy and then leave it to others more skilled than I to to improve public trust. So I actually think that there are aspects of the way we conduct elections in the US right now that make them less deserving of trust than they could be. In particular, many of the aspects of the procedures and equipment that we use are neither tamper evident nor resilient. That is, we can't always tell whether something has been altered, whether it's been misconfigured, whether they're bugs, whether there's been hacking. And if we do discover a problem, we're not always able to recover what the correct outcome of the election would have been but for the problem. We rely heavily on computerized systems for various aspects of elections, including voter registration databases, electronic poll books, the systems that are used to mark ballots in some cases or actually instead of marking ballots, touchscreen interfaces for direct recording electronic voting systems, and then systems that tabulate, aggregate, report the election results. All of those systems can be misconfigured, they can have bugs, they can be hacked. In order to provide evidence that the reported winners of the election really won, we have to have a way to check whether they won without trusting software computers that were used to conduct the election itself. So one of the most important things is actually paper. And all of the speakers have mentioned this before, but it turns out that paper has some unique security properties that are not shared by electronic systems. It's tangible and accountable. Both Kami and Jennifer talked about this. You can keep track of how much paper you sent to a polling place and how much came back, voted, unvoted or spoiled. It's tamper evident. It's not that you can't alter it, but it takes some work to alter a voter's mark without leaving discernible traces. And finally, and kind of most important is that if you want to change a lot of votes that have been recorded on paper ballots, that requires physical access to the ballots and it requires accomplices if you're going to change many, many votes at all. That's very different from the electronic world where a single individual can change an arbitrarily large number of votes without any accomplices necessarily. But how the paper is marked curated and audited is crucial. And if we create the paper trail using untrustworthy technology, then it's not clear what the paper trail is really providing us. Handmark paper ballots are the best choice right now because they're a record of what the voter did. There are a lot of states that are moving to universal use ballot marking devices. So that's where voters interact with a touchscreen and the touchscreen then prints a record that's supposed to be a record of their selections. The problem with that is it's a record of what the machine did, not what the voter did. Voters by and large don't notice if what's printed is different from what they intended. And even if they do notice there's no way they can prove to a poll worker or election official that the machine misbehaved. And if the election official is convinced that the machine misbehaved, there's nothing the election official can do to figure out what the right outcome of the election actually is. So there's a lot of security problems with those. They do provide additional accessibility for voters with some kinds of disabilities. And that's an important thing. There's some evidence that many of these devices in fact are not very accessible. And I think this is an area where we really needed to note a lot more research and attention to building accessible ways of marking a paper ballot for voters who can benefit from technological interventions there. If we don't keep the paper trail demonstrably secure, it's not clear why we should why having that paper trail should increase our trust in elections. Jennifer and Cammie both spoke about this actually, Wayne as well. And of course, if we never look at the paper trail after tabulating the votes, it might as well not exist. And so that's where the auditing comes in. So this is kind of like the carpentry slogan of, you know, measure twice cut once. This is sort of, you know, count twice, certify once. You don't really need to count twice, but you do need to count a little bit more than once. And that's what that's again, what auditing is about. So I would categorize the way we run elections in the United States right now as procedure based rather than evidence based. So what the way the rules work, the way things are is states have various procedures for certifying equipment. Many rely on federal certification by the Election Assistance Commission. And then there's rules to follow for how to conduct the election. And at the end of the day, basically the assertion is I use the certified equipment. I follow the rules. Therefore, the outcome of the election is right. And I liken this to a brain surgeon saying I used a sterile scalpel. I followed proper surgical procedure. Therefore, the patient is fine. I think we need to look at the patient to figure out what the patient is fine or not. And similarly, I think we need to look at the way the elections are conducted and look at the paper trail in order to determine whether the outcome really is correct. Some years back 2012, David Wagner, who's in the ECS department here at Berkeley, and I published this paper called evidence based elections. And the basic idea is that it's not enough for elections to identify who really won. It also needs to provide affirmative evidence that it really did find who really won. And that evidence basically is a function of having auditability that is having these paper records, keeping track of them and so forth, and then actually looking at them, auditing. So what is a risk limiting audit? The term has come up a number of times already today. It has a relatively simple definition. It's any procedure that has a known maximum chance of not correcting the reported election outcome if the reported election outcome is wrong. And moreover, it never makes a right outcome wrong. So that's a straightforward idea, I think. I mean, there's a lot of words there. It's a little bit tortured, but big chance of correcting the outcome if the outcome is wrong. Only a small chance of not correcting the outcome if the outcome is wrong. What's the risk limit in a risk limiting audit? It's the largest chance that the procedure won't correct the reported outcome if the reported outcome is wrong. Typical values used as risk limits in actual election audits are 10%, 5%, some cases smaller than that. It turns out that the amount of additional work you need to do to attain a smaller risk limit is not always that much in addition. So what does wrong mean? Wrong means that if you accurately tabulated the trustworthy paper trail, you would find that someone else had won. Figuring out whether the paper trail is still trustworthy at the time of the audit involves other processes.  Jennifer mentioned a number of the things that one would want to do to ensure that the paper trail is trustworthy. And these, of course, are not the only things that you would want to audit. I think Jennifer also mentioned auditing things like ballot layout. We want to make sure that ballots are usable, that the instructions are clear, and so on. Making sure that eligibility determinations have been made correctly. This is related to things that Wayne spoke about, making sure that you have accurate voter lists and that you check things against them. Right, so risk limiting audits have been pretty widely endorsed. The National Academies of Science, Engineering and Medicine, Presidential Commission on Election Administration, the American School Association, legal women voters, common cause verified voting, and others. There have been on the order of 60 pilots in a bunch of states and Denmark. They've been routine in Colorado since 2017. It was under former Secretary Wayne Williams that they were actually implemented. There was a tortured time of, I think, more than five years between when a law was passed and when they were actually rolled out. There have been statewide audits in a number of states that were actually orchestrated by Jennifer. And there was most of a statewide risk limiting audit in Pennsylvania quite recently. There are laws in a number of states and there are methods that can be used to do risk limiting audits of all the social choice functions that are currently used in U.S. elections, including plurality, multi-winner plurality, supermajority, instant runoff voting, and things like that. So to distill this all to a relatively short list, what do we need to have evidence-based elections? What do we need to be able to produce affirmative evidence that the reported winners really won? The voters have to create a complete, durable, verified audit trail. The local election official, there's another acronym there, LEO, needs to care for the audit trail adequately to make sure that it stays complete and accurate. We need some compliance audits to check to confirm that the paper trail really is trustworthy because auditing an untrustworthy paper trail can't tell us whether the reported election results are right. And then at the end, we need risk limiting audits that will check or correct the reported results. I'm going to stop sharing my screen now. And there, turn it back over to Henry. Thank you, Phil. That's great. That's a nice distillation of what's actually a very, very sophisticated technique to make sure that in fact risk limiting audits, for example, can be done most expeditiously and efficiently. Because the whole point is that it doesn't require that you actually look at every ballot, you can just look at a certain number and then at that point you say, well, there's almost no chance that we got it wrong and therefore we can go forward and assert that we've got it right. So we have a great set of audience questions. One of my favorites here is, given the election security concerns raised by Professor Stark, will we ever be able to have voting by computer? That was a big word, but will we ever? I'm sorry, I missed the preamble to the question. I mean, we do use computers to tabulate votes and we use computers in many parts of elections. But we can't trust them to do the right thing. So we need to check whether they got the right answer. And that's where having this trustworthy paper trail comes in. It lets us do some manual work, visually inspecting some often relatively small number of ballots to confirm that the reported winners really won. By looking at just a random sample of ballots, we're not going to be able to tell whether the tally is right down to the last vote. But that's not the point. We don't need to get the tally down to the last vote. What we need to know is, did the reported winner actually win? It's a much lower standard of accuracy. And that's what makes it possible to do with a relatively small inspection, by manually inspecting a relatively small sample of ballots. Yes, so let me follow up and address what I think also might be part of the question is, can you, at the present time, we do not have the technology sufficient to ensure that the person at the other end of the screen is who they say they are, and at the same time be able to ensure the anonymity of their vote. So this is different in elections than banking or anything else, because in banking, they're perfectly fine knowing who you are and what you did. In fact, you want a report of exactly the transaction you did. So doing banking online is a different thing. In elections, we need two things. We need to know it's you and that you did it, but we also can't have the government know how you in particular voted. And right now, the technology does not exist in a way that is secure and verifiable to do both of those things at the same time. The question was ever, look, my grandfather was born in 1899 when there was horse and buggy and by the time he died, people were landing on the moon. So certainly things can change, but it's going to take a lot to convince and it should take a lot to convince that something can be completely secure because computer security is a never ending battle. As soon as you if look, you've got a computer it every few weeks or sometimes more often it gives you an update because they found some security vulnerability that needs to be addressed. You can't be doing first that computer can even be connected to the internet if you're doing it right in an election. And so we just do not have that technology or ability to provide that completely secure system. We're comfortable with a few errors made in banking. It's part of the process they use and they ensure against that and they plan for a certain amount of loss and when you call up and say hey that wasn't me and they reverse the charges on something. You can't do that in an election and so we're not at the point yet with our need for 100% accuracy and verifiability at the same time as we have anonymity. Those are at least my thoughts. Thanks, I'm trying to think if there's any votes that I'd like to have reversed historically as I think back through my voting career but nevertheless. Let's go to the other end of the process. How can we make it easier for people to register and what role does computer do computers play there and what are some of the security concerns we have. When we use computers for people to register online for example, is that doable is that sensible and can we use that as a way to make sure we expand voting instead of constricting it. So, so it's, and I apologize for jumping in here again but it's absolutely doable and it's being done now. There are several security precautions you have to ensure one of those is that ultimately no changes are made in a voting record unless a live person approves it. That's the model that Colorado uses for that live person to use it they're using two factor authentication where you can't just get in through a hacked password. You've got steps along the way that verify frankly the fact that voting roles and most states are open to the public is another verification source so that anyone can look at it and say, yes, I know Bob Smith he lives next door to me, or they can look and go, Bob Smith, that's a vacant lot and so there are a host of processes you use. Most states have some form of online voter registration. It can be done. There are precautions you need to take in place and to ensure it actually is that person, but those same precautions need to be in place with respect to a paper signature verification that submitted. So it can I think it can be done and most states are doing it at the present time, but you have to make sure you've got those precautions there were in the 2016 presidential election. There were efforts to get into the voter registration databases of multiple states that continued through 2020 and 2016 I think there were two that had a potential vulnerability. One was in Illinois and one other. And so we want to, you know, there are precautions you have to take us one of the great things that Homeland Security's efforts in this area have provided to help states and localities make sure that they're secure. Jennifer and coming you have any additional thoughts on that. A quick thought there. There's a real opportunity still, we talked about, well, first of all, let's just answer the question. I think, as we mentioned, having online registration is essential. And I'm a big fan of the states and jurisdictions that have adopted automatic voter registration but one of the one of the nice things is when you're able to tap into other state and federal agency data. That kind of adds to your ability to sort of verify or validate that data. So there's a really nice effort. I think Wayne mentioned it, the electronic registration information sharing, but I would just add the convenience is great for those that are nervous about that are nervous about this idea of online registration or registering registering at different agencies. The more we sort of can share data on when people move, death records, marriage records, incarceration, those sorts of things. That's where we can improve the integrity. And how about same-day registration, by the way? Is that something we could do? Cami, you were going to jump in. We have a version of same-day registration in California. It's called conditional same-day registration. It allows us, it's sort of like a provisional ballot process. I think the ability to register on election day is important, especially for people who have moved. We don't want to disenfranchise voters. We want to invite voters into the process. But to just talk about a little bit of the caution on online registration, although I do think that it has expanded the number of people in our voter rolls. There are issues that need to be addressed still. Historically, this is done through Department of Motor Vehicles signature match. Signatures under license are not very readable.  And there have been security breaches. There have been additional issues that states that are considering adopting it do need to look at. And we should be continuing to look at security in that area to make sure that we're not unintentionally causing more problems for voters than they would if they registered with a paper ballot. And then your very first question about how we can make it more accessible. We also need to be really careful that there are a lot of people in this country that still don't have access to the internet. There is still a very real digital divide between rural and urban. So we need to ensure that we're still providing those in-person opportunities to register to vote in the traditional methods. And signature verification. Is that an issue? Phil, looks like you might have something to say about that. I was going to chime in briefly on the question of online voter registration and online voter registration databases. So they are vulnerable, but there are best practices to secure them, including very frequent backups and then checking for differences between today's record and yesterday's record to make sure that you didn't have 5 million records changed and so on. So there's no way to get rid of the vulnerability, but again, there's a way to make end runs around the vulnerabilities to ensure that you can detect problems if they occurred. Signature verification is really troubling partly for the reason that Cammy mentioned that when you capture signature on a tablet, when someone is renewing their driver's license, that doesn't look like a wet signature. People's signatures change over time. There are not good standards for comparing the signature on, say, an absentee ballot to the signature of record. Some of those, some of it is done in an automated way. They're automated signature verification things. There's no standards for those. Often the people who are doing this aren't necessarily trained well. And there isn't a lot of work that's been done to even measure sort of the type one and type two errors. That is the false negatives and false positives. So I see that as an issue. There are states in which it is now illegal to check the signature against a reference signature. I guess because it had been done so poorly in the past. All that officials are allowed to do is check that there is a signature. And then similarly we're talking again about absentee voting rather than voter registration. Georgia is poised to pass a new legislation involving taking a photocopy of driver's license and says isn't getting an attestation to this and that and it's just, it's not, it hasn't been well thought out. It's probably going to reduce security rather than increase security. There are just complicated problems around around all of this. Signatures are just they're not a great way to verify who people are, but requiring photo ID can have the effect of disenfranchising some groups. It's, it's, it's problematic. I don't think we have a really good answer. The best answer historically that we've had is in person voting in local polling places where it is your neighbors who are checking you in in the poll book, and they know you. They recognize whether you were the person who goes with that name. As we move to things like vote centers and whatnot we're actually losing that security that comes from, you know, eyeballs familiar eyeballs on other people. So just, let's talk a little bit about the overall security of mail in ballots that of course was a big issue in this last election. Mail in ballots, of course, to some extent were made necessary by coven, but nevertheless that's clearly been one of the claims of the Trump administration and those in league with him. And what do you think I mean is it possible to make mail in ballots, sometimes called absentee ballots, actually secure. I would say absolutely. There do. Again, we need to be able to make sure that we are going to mitigate training of the vulnerabilities, but people who want to vote by mail should be able to vote by mail. We should be able to come up with systems that we can verify and that the voter can verify that we receive the ballot. And if there's an issue with the validation, they should be contacted and given the opportunity to correct what that issue is. There are systems that we have in place, at least here in California. There's even if a voter received two ballots in the mail, oftentimes it's because they re registered during that six to eight week voting period. We can only check in one ballot per voter. So we do have protections in place for all of the vulnerabilities that we know about. And I think people who choose to vote by mail should be able to have confidence that their ballot is going to be counted and counted. And I'll just tack on to that. Agree with that Kami. And in addition to those, some of those things that she talked about in terms of like the front verification of a voter's identity, the signature matching that we just talked about. This is where that that chain of custody, the standard operating procedures, those reconciliation logs, those batch control logs really come into play where there's an opportunity from the time that that ballot arrives in the election facility, or is put into a ballot box as the election official, you can track that account for that. Wayne mentioned the bipartisan teams. So all the way through we have we have the ability to sort of verify as ballots move right we take some out of the system because of signature issues. Some have to be removed to be duplicated. There are different reasons why those numbers are changing. But there actually is some pretty easy sort of basic principles to account for those as we get all the way through the casting and counting of those ballots. So from from the back end from the operational end, I would say, absolutely. It's a it's a pretty secure process and just one little note on the signature verification piece states that have been doing mail ballots for a while, Colorado, Washington and others actually have done a pretty fair job I recognize all the problems. And I don't discount those but they've done a great job in sort of thinking about tiered verification audit so taking samples to something Colorado did. It requires you to sort of pull a sample of signatures to make sure that the people or the equipment that are verifying those are doing that accurately, creating some sort of standards when it comes to training, working closely with local FBI and other partners to sort of understand what that training might look like knowing that again signature verification, sometimes less of a science and more of an art but I think there certainly are good practices that should help voters feel confident that once that ballot reaches the election office or is put into a box that it's going to be secured and accounted for. Let me let me follow up with what Jennifer just said so Colorado is one of the three states that mails a ballot every voter. So we've been and we've been doing it for longer because we had no excuse absentee and then we have the option to become a permanent mail and voter. And because of that we did adopt standards and Phillips right if you don't have standards if you don't have training, then it's not as secure as it needs to be. So Colorado does have those standards we do have that training. We also require election judges or clerks rather to update that signature so after every election. As Philip pointed out signatures do change over time so one of the things we do is we keep a running record and election judges can go back and review those past records. So all of that are part of what we have in our database that makes our elections in Colorado work. I will say that if you do not have some way of verifying who sent that ballot a system that simply says as long as there is a signature on the signature line is not secure. Because at that point right that's where the stories come from of hey there were a bunch of ballots at the post office someone grabbed the envelopes and turned them in. And if you don't check signatures, that's a huge issue. But in Colorado, we have our processes in place and it is there and as Jennifer pointed out, we established audit standards for both people and for automatic machines that do. And we limit the number that can be accepted. And we verify that those are accurate. Is it possible that someone can forge a signature. Absolutely, so it's not perfect. But it's also very possible for someone to walk into a polling place and say that they are an individual. And while the wonderful idyllic world of I know all my neighbors in a precinct might have once been the case in the United States. And they'll tell you there's lots of places where people don't know the other people that live in the apartment building and the person next to them pulls into their garage. When they get home and they wouldn't they don't even know who lives there. And so that that that idyllic view of the way elections used to be conducted. And we have verification right if this is my brother I know him he's here. I'm related to everybody in the precinct I know all of them. But that's not the reality of American society today. And so we as election officials can't. We have to adjust to changing times, but we have to have standards that are followed up on and if there are improvements we can, we need to adopt those. Are there ways going forward that we may look at different things sure there may be some other biometric that can be used there are other things that may be there at some point in the future, but whether you walk into a polling place or return a ballot. We do need to have some verification process that takes place. So let me just talk a little bit about this issue, which is that some people are asking whether we need national standards in this area. We could even think about going as far as places like India where there's a national agency that takes care of all the voting systems. I want to talk a little bit about national standards. And at the same time whether it's a good thing that we have such a decentralized system, and how much more centralized should it be should it be just at the level of standards, or should it even go further than that. And what are the strengths and weaknesses of this system that we have, which is 3000 counties and maybe 10,000 election agencies in the country. I can weigh in here. So there are sometimes areas that would lend itself to federal standards. I think the health America vote act with guaranteeing provisional ballots, and making sure that we request and need to verify driver's licenses and social security numbers on the front end of voter registration. I think that's a good example of the compromise that was made in 2002 after that contentious election that you were involved in Henry. But as far as national federal elections, it's just way too difficult to be United States, because even in California, voting in Los Angeles is very different than voting in a new county. And so trying to come up with a system that will work for all of us in the different areas that we live, the different ways that we communicate, the way whether we're very rural and we're living close knit in an urban community, it's going to require flexibility. And so it is something that the country should be looking at for certain standards, I would say, but it should not be federalized as well as the decentralized nature does lend itself to this issue of trust that we talked about in the beginning. My voters know me. I can go into my community. I see them in the grocery store. Our kids grew up together. So when I have heard these accusations of vote rigging in the 2020 election and the 2016 election, oftentimes what I hear is, well, we know that we trust you. We know that you did it right. You could prove it to us. I'm not sure about everywhere else in the country, but we have a lot of faith and confidence in you as a community member. So if all of the election officials could cultivate that relationship with their communities, that's how you're going to build trust across the nation. Yeah, and I don't, I've been involved with the National Association of Secretaries of State and a host of other organizations. I don't think you want the federal government trying to micromanage how things work in each community. And I'll give you two really contrasting examples. Colorado, where we mail about to everybody in New Hampshire, where it's really hard to get an absentee ballot normally and everyone votes in person. Both of those states are amongst the nation's leaders in voter participation. And what works in New Hampshire would not work in Colorado and perhaps vice versa. But we have complicated ballot issues in Colorado. We trust our voters to make debt decisions and tax decisions. And so people like to get a ballot, they can review those on ahead of time, whereas some states don't, you don't have complicated ballot issues. I was at the Alaska elections office in the midst of one of their primaries, and they were getting ready to send out ballots, and they had to do it on a particular day, because the community only got mail delivery once a week. And so that's something that most of the rest of us aren't having to deal with because we have roads that go to our towns, you don't have to get it into the weekly plane that goes there. And so I want to get people in here, by the way, because he's been trying to get in as well. Okay, so, but that's, so there are some minimum standards that are appropriate. But I will point out, for example, HR one currently, that I think just passed the house would strike down Colorado's mail ballot process. In terms of signature verification and some other issues, it would cause some excessively long delays before you knew results, which as we know, the longer it is before you know results, the more concerned that you get. And so those are some of the things that that make it so I am not interested in the federal government is telling me everything I need to know. But when I look at the voting rights act and help America vote act, there are certainly some appropriate levels for the federal government. So, I'm sorry, so I was going to say I think there's ways in which we're airing too far in the other direction. For example, the election assistance commission is reluctant to even enunciate best practices for use of equipment. Because that's perceived by states is infringing on states rights, even if they don't say you must do this and they just say, here's a really good idea about how to use this equipment and this would be this other way of using it wouldn't be as secure. I hope that we get some federal legislation around auditing and around ensuring the trustworthiness of the paper trail. I mean, I completely agree with everything Wayne said that we need flexibility to run elections in a way that accommodates the logistic restrictions of, you know, the locations in which we run the elections of individual jurisdictions. But that said, we need to make sure that at the end of the day we have elections that we can trust. And that may require so an unsteadying principles at a high level maybe the way to get there rather than micromanaging. How about this question. You may want to also discuss what we've been discussing but are we risking voter suppression in our effort to make elections more perfectly secure. What's the trade off here. We all want to make sure that we have a real democracy where everybody gets a chance to vote. We already know that the disparities are really unconscionable. Don't we have to worry about making them worse. Yeah, I think we absolutely do and I think it would be on wise to say that best adopting national standards for best practices means we have to give up local control or our decentralized model I think both can exist together. You know, in a really good example of that talking about, you know, making sure that we don't disenfranchise voters is it's generally accepted across all 50 states that if a voter comes into vote in person, and their name can't be found in the pull for their information. There's an issue with their information that they'll be issued a provisional ballot and that that will be researched and determine determination made at a later point so I think there are certainly ways that we can continue to put voters to ensure. And I'll say it people cringe when I say easy access but I think we need to make it easy to vote. I don't think that we have to sacrifice that for better security for audit ability for trustworthiness. We certainly have really good examples of entire states and jurisdictions across the country who to do both really really well. I want to ask a big question here, which is fill usefully made the distinction between being trusted and being trustworthy and argued that in fact he's not sure that the systems right now are as trustworthy as they might be and we have to improve them. On the other hand, trust is a complicated thing and it's not just a function of trustworthiness of the system. It's also as others have pointed out like me. It's a function of the trustworthiness of the people involved. My question is, what's the right mixture of things that we've got to have here in order to increase trust. I trust Kami. Obviously, she just looks so trustworthy. And I frankly don't care that much about the procedures because they're too complicated. And so, maybe that's enough. I am here since you invoked my name. I, when I, when I say that my community trust me, I've run 22 elections here. I, and I do it with that concept of evidence based election that's a new language that I'm using thanks to Dr. Stark and others, but that's, that was the idea all along is, you don't have to trust me, let me show you. Usually the people who are the most vocal of my community that are concerned about whether or not the election is going to be conducted properly. The first thing I do is invite them in. Come on down, use my process, be a poll worker, help me count on election night, come and help me process absentee ballots behind the scenes. And that is how you cultivate trust over time. So my community, hopefully the majority of them think that I'm trustworthy because I've proven to them over time that the elections have been conducted properly by showing them through evidence and then eventually they build that, that relationship to where then they might not have to check the evidence that anyone's new in the community or anyone outside of the community. This questioning why there's only a 14 to vote difference and did we actually read the election here and in your county. I can show them that we didn't. So these factors interact. Phil, what's your thought. Well, I think that the gold standard would be to for elections to provide public evidence that justifies trust. We shouldn't have to trust individuals in to in order to have trust in the outcome of the election. I think I should mention at least the existence of end to end cryptographically verifiable voting systems which I think are not yet really ready for prime time. Generally, there haven't been found to be terribly usable but they do provide another way of making an end run around the trustworthiness of the system to have some trust in the result. I think you exactly the same kinds of guarantees that a paper based system would but they're very similar. So one thing I think it's important also to mention is that transparency doesn't necessarily inspire trust and radical transparency isn't needed to justify trust that there are key things that one needs to know at various steps along the way. So I think that the evidence that the paper trail was really kept secure. How do you do that. Well, you have two person chain of custody rules and maybe a video camera on the cage where the ballots are kept.  That doesn't mean that that a member of the public has to accompany the ballots for every stage of the process. I think that the Cami piloted very recently is a way of convincing the public that the electronic vote records that are used as part of the audit as sort of auxiliary data for the audit are and do indeed reproduce the outcome of the election and are indeed the ones that were used in the audit. Without revealing the an entire cast vote record like the whole list of selections that any individual made because there's a possibility that revealing that could compromise voter privacy. So she used a novel cryptographic approach to doing that that may that that again means that anyone who wants to check the audit can check can check it without necessarily having access to all of the underlying data. We don't need radical transparency to have enough information to justify a trust. So we're going to go to Wayne in a minute but let me just first say that I think one of the goals here might be to involve as many people in the process so that if you were really trying to do something to manipulate it. There might be at least one of those people who would speak out and say, I was approached and asked to do something that I really can't do because it's not right. We would have a human who would like testify as to what the difficulties were. Wayne, your thoughts. Yeah, so I wanted to follow up first. Phillips right. Absolutely. The EAC should not be afraid to tell people what best practices are. We need that certainly we've had Homeland Security do that in terms of a number of security issues we need the EAC to do that. I am not opposed and I support them providing that information. I just don't want the mandate. You asked a question though about vote suppression that I think is really important. There is an impact with respect to vote suppression if people don't trust the election system. So one of the critical reasons why we're having this discussion about evidence based elections is we want people to have confidence in the elections. They are more likely to vote. And the easiest way to suppress somebody from voting in this day and age is to tell them their vote doesn't matter. It doesn't matter how you vote it's it's already been determined what's going to happen. And so we have to have processes that provide things and so in response to your question. I would say that not providing election security is actually what suppresses votes, because people are less likely to vote if they don't think their votes can account. Jennifer we're going to end after hearing from you and what are your thoughts to end us. Okay, I love that you brought up. This is a citizen based process, and I think you're absolutely right. I think when we have citizens and we saw this sort of remarkable recruitment effort this year, who are willing to step up and whether they're hoping in a central count facility, working and as a poll worker administering voting in person. Those are those hundreds of thousands of witnesses to the credibility and the reliability of the process is at 100% perfect. Because it's a human process, and we're always going to have issues or errors. That's why we need some standards around audits, but I certainly would echo what you said and and the other panelists that people are concerned if they legitimately feel like things aren't running the way they need to be step up and volunteer step up and help. This has really been a terrific panel I wish we had more time. It's partly I'm an election group groupie and I love this kind of discussion and I think it's extraordinarily important. And I love the way you ended there, General I think the idea is that this is about people getting involved seen it up close, recognizing that in fact, it is a process where people are trying to be as careful as they possibly can be. And that will create more trust in elections, and hopefully more trustworthiness as bill would hope for. So thank you everybody let me thank Kami foot Jennifer moral Phil Stark, and Wayne Williams. Thanks so much. I want to thank Janet Napolitano and her Center for security and politics, which has had just an absolutely terrific set of panels which really have run the full gamut of what the is going on in elections in America right now and actually in the world because the first panel did that as well. And thanks to the audience for tuning in. Thank you for being here. A great set of questions you put forth. And then we just finally say there will be a recording of this event, and the others included in this symposium and prior events sponsored by the Center for security and politics, and they'll be available at www dot uc TV slash public dash policy. So that's www dot uc TV dot TV slash public dash policy. Thanks for tuning in. Thanks to our panelists. Thanks to Janet Napolitano Center. Thanks for all the great work you're doing. And thank you for trying to make, and helping to make elections more secure in America. Thank you. Thank you all. Great panel.