Continuing to populate the webinar, but in the interest of giving our amazing panelists the maximum amount of time to discuss these important issues, we'll go ahead and get started. And as is our tradition, we like to start with a statement honoring place. The University of Washington acknowledges the Coast Salish peoples of this land, the land which touches the shared waters of all the tribes and vans within. It's a Suquamish, Tulalip and Muckleshoot nations. So on behalf of the University of Washington School of Law, I'm Mario Barnes, the Tony Rembideen, and I wanna welcome you on behalf of the Law School and its faculty, staff and students to this gathering for what I think is both an incredibly important topic where we will be really fortunate to hear from an August and well-informed group of speakers. It doesn't take me to point out that we are living in an unprecedented time, a time which includes next Tuesday a historic election, which has many unusual contours and that yes, it's an election to consider president, vice president, elected officials locally in the state house and the national government, but it's against a backdrop of events that I think most would agree that in our lifetimes, we've not seen this confluence of events. Just a few days ago, we had a Supreme Court Justice confirmed without bipartisan support and in a manner that raises controversies about the independence and legitimacy of our highest court. At this point, we have multiple states embroiled in litigation that is headed to or recently entered into federal courts related to state and presidential elections. And when we look at the national election hanging in the balance, there are larger questions about equality, democracy, fairness and rule of law. That will affect every corner of this nation and beyond the results of which next Tuesday will. And all of that against the backdrop of an unprecedented global pandemic. And so we could not be in a time where we needed a more informed discussion of these issues to include a discussion of voting rights of how the election results won't be contested and what it will mean after Tuesday to our nation when we think about all of the kind of issues that have been raised related to voter equity and access to ballots in the polls. And so I'm really excited to have our panelists engage in a rich discussion over a broad set of topics and I will introduce our panelists and get out of your way. And so I'm so pleased that Zar Saeed, our Associate Dean of Research, Faculty Development and Charles I. Stone, Professor of Law will be your moderator. We have joining her on the panel, Naomi Ishisaka, the Assistant Managing Editor for Diversity, Inclusion and Staff Development and a columnist for the Seattle Times. Lisa Marshall Mannheim, the Charles I. Stone, Associate Professor of Law at UW Law. Peter Nicholas, the William L. Dwyer Chair in Law at UW Law and Brianne Schuster, Staff Attorney for the American Civil Liberties Union. Thank you all for joining us. Thank you for giving your time and energy to a conversation that I know our community really needs at this time. I turn it over to Associate Dean Saeed. Thank you, Dean Barnes. Let's start with Professor Mannheim. If you could provide us with an overview of where we'll be going today. Thanks. And thanks for everyone who's joining us today as well as everybody of course who's joining us on the panel. So when it comes to the laws surrounding the 2020 election, one way of thinking about them is as falling into three general categories. So there's the legal issues that are relevant in a sense or at least most relevant before election day. Then there are the legal issues that become relevant after election day. And then finally, if we look forward, there are the legal issues and the legal discussions and controversies that are likely to arise, not with respect to this election cycle, but going forward. And so of course, to talk about all of those would take a very, very long time. However, we're trying to touch on all of them today. And it's just in terms of a basic overview. I was hoping you have just a minute on each. So this election cycle, the 2020 election cycle, when it comes to the legal disputes and the legal issues that have arisen before the election, there are, I mean, there's a ton of them. But if we were to summarize them, most of them or at least a large percentage of them relate to the pandemic and the sorts of accommodations that jurisdictions either are willing to and able to make or are not able to or willing to make in light of the pandemic. So what we've had across the country is we've had jurisdictions changing their laws in any number of ways. One way of confirming that this has had some effect for what it's worth is in the really dramatic rise in early voting we've seen so far. So we have already surpassed 50%, I think we're at 60 plus percent of the total voting that occurred in 2016. So we've already surpassed that in a place like Texas. There's actually been more people voting in Texas already than there were in the entire 2016 election, which just helps to confirm not only the interest in the election, but also some of the changes to the laws. A number of jurisdictions have not made changes that people have requested and in response to that people have brought lawsuits. Summarizing all those lawsuits would again take a while, but one way of understanding the gist of what the courts have done with those lawsuits and in particular what the federal courts have done is they basically said, again, this is very broad, but they basically said, number one, we're generally going to defer to the states and specifically the state legislatures in terms of how the laws should change, number one. And number two, they've said also it's getting too close to the election. So even if these laws are not constitutional, even if they are themselves unlawful, we're not going to change them this, or order them to be changed this close to the election. So that's some of what's been going on pre-election. When it comes to the legal disputes that are likely to arise in a sense after election day, it's really hard for us to know for sure what those are going to look like. And this is true every election cycle. You might think about it this way, every election is imperfect. If we insisted on perfect elections, then we would just not have elections. It's impossible to run a perfect election. That being said, usually we don't care that much about at least innocuous imperfections. And the reason why is because it all sort of, it generally doesn't matter in terms of the outcome. But when the outcome potentially turns on what one might argue is an imperfection of the election, then suddenly it's like you're taking a microscope and you're putting it right at that election and specifically at those specific issues that potentially may affect the outcome going forward. And because we don't know what the votes are gonna be yet, we don't know which elections are gonna be close. We don't know where that microscope is gonna be aimed. And so as a result, it's hard for us to know now exactly what those disputes will be. I have plenty of predictions about what they might be, but we don't know for sure. The final category is sort of reforms looking forward. And I'm not gonna try to summarize that now. Instead of what I'll just say is right at the outset, I wanted to say that we don't have to run elections the way we're running elections right now. Other countries do not. We in the past have, as a country, have run elections in different ways. And from my perspective, what's going on right now on a number of different levels is highly troubling. So while I would say that I predict and expect that the election will go forward next week and things will proceed, to me, a lot of what's going on right now does not feel sustainable. And so it seems like the push going forward for reform is likely to be pretty strong. Thank you. That's an incredibly helpful overview. And it teased up nicely Ms. Ishisaka's comments. We'd love to hear from you about what's going on before the election from your unique vantage point. Yeah, I really appreciate that intro and also just a caveat on the journalist not a legal person. So I just say I'm very excited to learn from all of you because you have a lot of expertise that I don't. So I'm very excited to be part of this panel but definitely not a legal person, I'm a journalist. So I'm a journalist on a lot of topics. But I would say one of the things that is really important to remember when we think about this election is that we've had hundreds of years of ways in which certain people were advantaged in the electoral process and other people are disadvantaged and that continues today, right? And so the things that you're seeing now, I mean, we certainly have a pandemic and that's impacting people's ability to vote for sure. But we also have things that are a continuation of efforts that have long been in place to make sure that some people have access to the vote and some people don't. And so what we've seen is for people that wait over an hour to vote, for example, people of color are more likely to wait over an hour than white people by a ratio of seven to one, right? And you see that throughout the electoral process, right? The people who get purged off of voter rolls, the places where there are fewer ballot locations, all those things, if you overlay race, you'll see them tie together. And I think that's a really important thing to always remember and to keep our focus on. And that said, I wrote about this in July that we were kind of looking down the barrel at some pretty dismal voter disenfranchisement coming into this pandemic election. And I think what's been kind of heartening to watch is people kind of took matters into their own hands and decided to enfranchise themselves as I wrote about this week. The number of early voting, as Lisa said, has just exceeded all expectations. And I think part of what's happened is that people have seen the ways in which the law has been used to kind of keep them out of the process and they're really not having it. And they've also seen that anything can happen in this year and in this life and that they didn't wanna leave anything to chance. And so I think all those things happening together paint a somewhat hopeful picture, but then also keeping an eye on the fact that, as Lisa said, we don't have to do things this way. We've kind of carved out this hodgepodge of systems, this sort of Byzantine set of systems that I think an outside observer would kind of look at if they came from another country or another planet and say, why would you do it this way? I think it's a really good question. I think particularly us in Washington, I think I've heard a lot of people say, well, we don't have any of these problems in Washington. Why should we care?  What happens in Texas affects us, right? We can't just kind of sit here in Washington being what was just regarded as the number two easiest state to vote in and just say, well, we're good. So good luck with that. We're all interconnected. And so I think it's important to keep all those things in mind at the same time. Thank you. Let's turn to the other two panelists we haven't heard from just yet to weigh in on this issue of voting rights and any other thoughts you may have. Maybe first to you, Professor Nicholas. Sure. So one of the things I'd like to talk about is the potential aftermath of an election, especially if it's a closely contested one and in particular the precedent of Bush versus Gore, which many people know from a number of decades ago, was a bit of a surprise opinion in terms of the doctrine. And why do I say that's a surprise? Well, you had a conservative leaning Supreme Court set of justices who tend never to be super excited about the Equal Protection Clause when we're talking about the rights of marginalized groups like sexual minorities, women, African-Americans, and so on. And all of a sudden we get a really robust interpretation of the Equal Protection Clause in a narrowly decided opinion. Now, to be fair to the majority, as a con law professor, I would have given them an A for their clever use of Supreme Court precedent to come up with that decision. They basically took a number of cases that liberal Supreme Court justices had written over the years dealing with the fundamental right to vote and assembled them in a way that came up with this issue about the right to have your ballots counted equally. So the precedent is out there and what it suggests is that if we have another close election in a state like Florida or somewhere like that, to the extent there are any differences in how different parts of the state on a county by county basis are counting the votes, we're gonna see something bubble its way up to the court. So there's the legal precedent, then there's what we might call the political precedent, okay? So that was what we would still have called a fairly moderate conservative Supreme Court, but even people like justices Kennedy and O'Connor who we often think of as these independent moderate justices came home to roost so to speak and voted on the side of the Republican candidate for president over the last few years, we've seen Chief Justice Roberts serve as a swing vote, but he is at the end of the day also conservative and he's no longer a swing vote with the way the court has moved. So I think that the most important thing really is that the vote not be as close as it was back in Bush versus Gore, because I think that precedent is going to be very difficult for a Democratic candidate to overcome. Oh, did we lose your audience? I think we lost your audience in the last couple of sentences. Okay, so what I was just saying is that I think the ultimate takeaway is just the hope really that this election is not close because as I think Professor Mannheim pointed out and as I've seen in studying electoral voting privileges, our elections work great when things are not close and most of the time they're not close but when they're that close and when they're that close on a national election or a statewide election, it's really hard for our voting rules to work well. Gosh, I'm glad that we got your audio fixed so you could share a note of hope. Before we turn it over to Ms. Schuster for her thoughts, I just wanted to encourage the audience to drop questions into the chat function or the Q&A as it might appear to you. We'd love to hear those from you. Ms. Schuster? Yes, thank you. And I'm glad we could end on hope because I feel like I just have more bad news to pile on but we'll try to find some hope too. There's a lot of wonderful work that's happening and we'll continue to happen and I do have a lot of hope that both our election systems are going to be changing after we kind of see how this is playing out both federally and locally. One of the things that and I think that Naomi touched upon in other panelists is that we've in some ways, in many ways, this is an unprecedented election but the way in which we've seen the efforts to suppress people's right to vote have been really rooted in the history of our country. In the founding of our constitution, we forbid black people from voting, we forbid women from voting and we're still seeing a lot of the suppression of the same communities. We're just seeing different tools being used. So now instead of having an explicit provision of you can't vote based on race, we have you can't vote unless you have an ID or you can't vote unless you have a specific address that you can prove that you live there. And so we're still certainly disenfranchising people based on race and while Washington in many ways is a better place to vote or an easier place to vote, it still has a lot of problematic policies and one that really stands out to me is we still forbid people with felony convictions from voting and that's really clearly rooted in racism and in preventing, well, at the time, black men because women couldn't vote but in preventing people of color from voting. And so that's something that we still I think really need to work on here in Washington among other things to make sure that people on tribal land are able to vote, that people that are houseless and don't have a stable place to receive mail necessarily are able to cast their belt and have information to vote. In addition to the election process itself and voting, we also have a lot of really problematic elections system. So like even if you can vote, is your vote actually being counted in the same way that everyone else's vote is being counted or not even in the same that everyone else's but that is your vote being counted equally. That's a huge problem nationally and in Washington state and I think a lot of times people toss out gerrymandering and that's super important to be aware of but here in Washington, we also have the majority of elections for like city council or for counties are at large election systems which means in a county for example, like Yakima County or the city of Yakima where it's majority Latinx citizens or like almost majority that are not or sorry, almost majority voting population, there's still because of racism and voting and racial voting patterns, never able to elect candidates with their choice to the council. So the white majority voting population is still electing all seven candidates or all however many candidates there are. So I think there's still a lot of work to do with that both nationally and locally in addition to protecting our electoral process. Wonderful, we've got all sorts of questions coming in. I wanted to come back to Professor Nicholas to weigh in on what the post election landscape looks like and see if you could weigh in on that and we need. Thank you, yes. So I mean, for me thinking about the post election area there is the how do we resolve this vote but I'd also like us to move forward to thinking about what do we do next? Because one of the things we've seen in this election is in my mind an unprecedented effort to really deny people the right to vote. And I'm not talking about, we can agree or disagree on felon disenfranchisement it either is or is not the law but now I'm just talking about people who are legally qualified to vote. We are seeing all sorts of tricks popping up to try to prevent that whether it's things like in Texas where suddenly there's one drop box per county and I have to be honest with you we have those drop boxes in King County and I thought that they're too far away from me and I looked and there's like 70 or 80 of them so I can't imagine what it would be like if there was just one of them in the county. So that's one problem is that and why is this happening? Why is the wire States doing that? Well, guess what the Supreme Court did a few years ago they gutted the Voting Rights Act by striking down the coverage formula. Now, I'm blaming the Supreme Court in part but I'm also going to put a little bit of blame on prior democratic majorities in Congress. What do I mean by that? They had an opportunity to do a clean job of reenacting the Voting Rights Act with updated data and they didn't because they didn't feel they needed to. And if there is a democratic majority in Congress and a Democrat in the White House the first thing they should do is reenact the Voting Rights Act coverage formula with new data and there is plenty of data from this particular election. So that's one thing. The other thing though that I think is concerning and that I think is truly unprecedented because we've seen states do this before that's why the Voting Rights Act came into being. I don't trust the US post office anymore to deliver my ballot and that is never a feeling that I have had in any election with any president in the White House. And so that is very concerning the lack of independence in agencies like that that are partly responsible for protecting the right to vote. So in terms of other reforms it's not just about looking at the states but looking at the independence of other branches of the federal government and hopefully a new Congress can look to that as well. Wonderful. I have a question I think is aimed for the all of the panelists Peter Professor Nichols after you mentioned the Supreme Court here is the Supreme Court likely to decide this election? Question many of us are wondering and watching with bated breath. So I'm happy to jump in on that one. So I'm a lawyer I'm not, I don't think I can predict future events. So I don't know, it's possible. But in my view and in my view it would be extremely problematic for that to happen for a number of different reasons including as Professor Nichols suggested we are not at a time right now when the, we're not in 2000 right now the country is much more politically polarized and the idea that the Supreme Court would sort of step in at a certain point and appear to the public to be deciding the election is highly problematic just in terms of the legitimacy that's likely to accord both the Supreme Court as well as to the election results. And so while I share the concern that the Supreme Court may at some point weigh in I think that's a relatively low probability event at this point. The Supreme Court has been kind of pulled into a lot of these disputes thus far because in the pre-election phase like we talked about at the beginning the courts can often have a lot to say here. There's various claims that people can try to bring based on federal law, based on state law based on US constitutional law based on state constitutional law arguing that a jurisdiction must change its procedures in some way. But once election day hits the sorts of legal arguments shift into something new. And for the most part, definitely not entirely but for the most part those disputes are based in state law, not in federal law. And what that means is that you're certainly gonna have various state actors being involved in the recount process if the dispute continues after that it goes into an election contest you're certainly gonna have state actors involved and state courts possibly involved but you don't have a lot of direct involvement of federal actors or of federal courts. Now the big exception here that in a sense kind of proves the rule but also calls into question what's gonna happen going forward is the Bush versus Gore case. And Professor Nicholas talked about that a little bit and what I would say with respect to that case actually a surface back to what one of the audience members asked, which is what kind of presidential value does that case have going forward? And essentially it has the same presidential value as any other case. We know that the Supreme Court in the decision suggested that it's holding was narrow but that doesn't mean it can't be used as precedent going forward, number one. And number two, just to remind everybody prior to the Bush versus Gore decision Bush versus Gore was not itself a president at all. So the court can also take a step further and create its own precedents. In terms of what legally speaking that's likely to look like essentially there are I would say three broad federal claims that may be relevant to a post-election dispute. The first is a Bush versus Gore claim which is essentially that the way the jurisdiction is counting the ballots is so sort of inconsistent that it violates equal protection of law that's Bush versus Gore. The second is a due process claim and that claim essentially states that jurisdictions are not allowed to change the rules of an election after the votes have been cast. Okay, so we haven't seen that at the US Supreme Court level but that potentially could arise. And then third is a sort of claim that has gotten a lot of press recently which is the idea that when it comes to figuring out a state's elections rules the state legislature controls and maybe it controls even when it's in conflict with the state constitution. That third legal theory was in a concurrence in Bush versus Gore and now in various sort of concurrences and dissents justices of the US the currents of US Supreme Court have been bringing up that theory a little bit. So this gives rise to essentially concern that based on one or more of these theories the United States Supreme Court may be the one that ultimately decides the election which then I would say brings me back to my initial point which is yes, it's possible that that would present itself but that's a pretty low probability event. A lot of things would have to fall into place for the US Supreme Court to have any sort of federal question to answer in the 2020 election. Thank you. As I'm listening, I'm thinking about the majority of our audience is our lawyers or law students and can probably follow a good amount of what you've just been saying. And also there may be some folks in the audience who for whom that doesn't track and or the sort of issues are larger than the kind of narrow presidential potentially narrow presidential questions. And I wanted to turn to Ms. Ishisaka to ask how you think about your role in the democracy as a bearer of information and an advocate because you clearly are an advocate for voting rights and for sort of sharing accurate information. And I just wondered if you would weigh in on how as you're seeing these facts and cases unfold what your role is in this or what you're learning about how to translate and how to kind of share information. Yeah, I mean, that's a great question. I think my role at the paper as both the an assistant managing editor and as a columnist is to really try to make sure that the voices of those who have been excluded are included, going forward and so in terms of my column what I really try to do is translate the things that folks like you and have a lot of expertise on to the lay person and try to get them to understand how it impacts them on their day-to-day life. And so I think my role is to really kind of filter out that information and make it as accessible for people as possible and if we're gonna get back to the question about what can be done but I definitely have thoughts about that if you want me to just say something about that. I think that'd be great. And then we can turn to Ms. Huster to hear about her efforts on the ground and locally and that might be a nice transition, so thank you. Yeah, so one of the things that I wrote about my first column on this topic in July was that even though we might've gotten used to the way things are in Washington I think it's important that we remind ourselves that it's not like this everywhere. And so I wrote in my column in July about this idea of the four pillars of fair absentee voting and you could call it four pillars of fair mail-in voting since absentee and mail-in seem to have some loadedness attached to it now. But those four pillars are things that we do in Washington but I think it bears repeating which is that one of them is allowing ballots to be postmarked on or before election day and then not having that be the barrier to whether or not the ballot is accepted. And you can see that the impacts of that playing out across the country right now. Allowing or giving voters prepaid postage so they don't have to scrounge around for a stamp or figure out how to pay for the postage and eliminating that barrier. Allowing community organizations to help return sealed ballots and I know that that's a controversial one but I think having that conversation there's a lot of uproar that people have had around seeing people return other people's ballots and it's a strange thing people are really fixated on but I think having some conversations around that as a community as to the pros and cons of doing it different ways I think is really important. And then having standards for signature matching that's another really controversial one that people are talking about across the country and it's almost like going back in time to hear folks and the consternation around a lot of these issues happening in other states because we've dealt with this for so long but I think it is important for folks that are from other states or folks that are thinking about how we could change things going forward that we keep those principles in mind. Thank you. Ms. Schuster, over to you. Yeah, thank you for that. So our office is, or our affiliate is focused largely on voter protection and election reform in Washington state specifically. And so there's, I just say that upfront there's lots of things that ACLU and other affiliates in the national office are doing in other places but here because we do have mail-in voting have for a while, it's pretty secure. And as was mentioned earlier don't have quite as many issues with voter suppression as other places again, still have issues. So we're working on that. We're working on making sure that the Washington Voting Rights Act is enforced and that allows cities and other jurisdictions to change their election systems to make sure that communities of color having an equal say in who their candidates of choice are. But I think, and also enforcing the Native American Voting Rights Act which allows folks on tribal reservations to use non-traditional addresses to name like a building and also request ballot boxes for their communities. So that was just recently passed and we're still trying to make sure that, it's worth more than the piece of paper it's written on. In terms of what I'm also really excited about I think we will see legislation this year to expand voting to folks that have a felony conviction that are no longer incarcerated. And so while we still have, building blocks from there to go to really further and make sure we're not disenfranchising anyone that I think that's a good step in the next direction or yes, a good step in the right direction. We've been working a lot with officials with respect to this election for folks that are in jail a lot of people don't know that can still vote. So making sure that they know they have a right to vote and have access to the ballot. So we've gotten some ballot boxes in jails which is really exciting. So people can drop their ballots off there and not have to worry about mailing them. I think, but I think the biggest thing and this was alluded to is really working with communities and being driven by what community needs are and making sure that folks know what their rights are. I am a lawyer, but the law is sometimes good and sometimes bad and so we need to work together to change it and it's certainly not just gonna come for me. And so I am excited to be collaborating with folks to see where we can go from here. Thank you. That seems like a great segue to head over to Professor Nicholas or Professor Mannheim for their thoughts on reform and what should come next. Who wants to go first? There you go, Peter Nicholas. As I started to say, I think that really the first step has to be to reinvigorate the Voting Rights Act. Professor Mannheim talked about changes, right? And some changes have been positive. Some changes have been designed to try to make it easier for people to vote. I think that sadly, a long history in this country has shown us that, especially in certain areas of the country, they are absolutely willing to disenfranchise people, to trick people to make it harder to vote. And one thing I do wanna pause and say is it is actually quite heartening to see that this election, people are not letting it happen. Now, it's hard, right? And I have, my hats off to these people I see standing in lines during a pandemic. People are saying, I want my vote to be counted and hearing about all of these organizations that are out there trying to help people vote. I mean, that's really what's gotta happen at this point because one of the things is, we as lawyers are used to kind of falling back on law. And a lot of the population thinks the law will help us. And sometimes it does, and sometimes it doesn't. And I mean, one of the realities that we've started to see elections matter, right? You know, I remember four years ago, people saying, well, there's not much of a difference between the candidates. Well, I'll just say, imagine if Hillary Clinton had appointed the last three Supreme Court justices. If you told me four years ago that Donald Trump was gonna be appointing one third of the US Supreme Court in a single term, I would have not believed you. So that's one thing, but I think in general, efforts have to be made to make it easier to vote. You know, I was just looking, and it looked like they finally changed things. But even in what we might call progressive states like New York, where my parents live, I tried to fill out an absentee ballot form or try to help them with doing that a couple months ago.  I think they've just now made a change there. But, you know, we are all used to in this state voting by mail. It works. And so I really think we need to see more of that, but I wanna hear from Professor Mannheim, who thinks about this stuff all the time. Yeah, so, I mean, there's lots to say about this question of sort of reform going forward. One of the questions from the audience asked, what sorts of reform can be effectuated through constitutional amendment and what sorts of reforms can be effectuated through a statute? So I have an answer to that in case it's helpful. And that's gonna actually circle us right on back to the Supreme Court. So with respect to what sorts of reforms that might go to some of these issues of voting, what sorts of reforms could be done through statute? One is, as Professor Nicholas says, is the amendment to the Voting Rights Act. So in Shelby County, they struck that down the coverage formula as unconstitutional. That could be re-implemented. The whole act could be sort of looked back over and amended. The democratic members of the House have now repeatedly and pointedly offered bills that seek to make these sorts of sweeping changes to the preexisting Voting Rights laws that we have. So that's one place where folks could look. And I'm kind of stalling here a little bit because basically I'm about to go into things that they may not seem voting related, but I think they're voting related. So let me tell you what they are. So what else? How about a new state? New states, Puerto Rico, Washington, D.C. Maybe there's other states out there too. I don't know, but those, my understanding is that new states can be introduced into the country through legislation. And to me, that's relevant to voting and to elections because right now with when we have the system we have with the Senate and we have the Electoral College. What that means, among other things is that we have a real disconnect between on the one hand what a majority of the voters across the country want and then on the other hand what we actually see in Washington, D.C. It is also the case of course that the people who live in Washington, D.C. or at least most of the people live in Washington, D.C. And same thing, most of the people live in Puerto Rico are American citizens, all of them, but they are not able to vote in some or all of these elections. So that would respond to voting rights issues right there quite a bit. Another thing that could happen is expansion of the U.S. Supreme Court could happen through statute. And again, this may not seem to directly relate to voting rights but we'll get back to that in a second. As related matter, and this also goes to a question in the chat. Through legislation, Congress could also change the jurisdiction of the various federal courts. So Congress could take jurisdiction away from the federal courts with respect to certain types of issues. It could move it around to different courts. These are things that Congress can generally deal without a constitutional amendment. What sorts of things would require a constitutional amendment? Well, here's one. The U.S. Constitution does not include in its text a generally applicable right to vote. So it includes, it refers to the right to vote in various amendments and in other places. But when it refers to the right to vote in a sense what it says is, this is again the text of the constitution. It says to the extent there, I'm paraphrasing, to the extent there is a right to vote, you can't, the estate is not allowed to implement that right to vote in a way that discriminates against people on the basis of race or on the basis of gender or on the basis of age, if you're over 18. So in a sense, the constitution again at least based on its text does not set a baseline. It just says once there is a baseline, you can't discriminate on these factors. Now I think that there actually is a generally applicable right to vote in the U.S. Constitution. Half a century of case law from the Supreme Court really clearly confirms that understanding. But my sense is that that's not necessarily shared that understanding the U.S. Constitution shared by everybody in the federal judiciary. And so if one wanted to have a generally, textually bound right to vote in the U.S. Constitution, that would require an amendment to the Constitution. And there has been now some push toward getting a right to vote in a text of the Constitution added through amendment. Finally, there's been, I know some talk of turn limits associated with Supreme Court justices. And it's hard to know whether for sure that would require a constitutional amendment, but by my assessment, that probably would require a constitutional amendment, even if you tried to kind of be clever about it and just move the justices around, that likely would require a constitutional amendment. Now, some other things, if you want to change the electoral college, that's going to require a constitutional amendment. If you want to change the way the Senate and the House is constituted, other than just by adding states, that's going to take a constitutional amendment as well. And so to quickly kind of circle back to why I keep talking about the Supreme Court when it comes to voting reform, the reason is because at the end of the day, the US Supreme Court plays a pretty significant role in election law in our country. And what it has done recently in the last 10 plus years is essentially, it's been very hands-off. It's basically said, look, states, you do what you feel like you need to do. Unfortunately, what we see in not all states, but in some states is we see the people in charge not focusing, I'm trying to sort of be sort of neutral about this, they don't seem to be committed to facilitating the right to vote in a way that makes it easy and accessible for all voters. And the US Supreme Court doesn't have to take this hands-off approach, but that's what it's doing. It is also the case that in part, due to the legal theory I was suggesting before about this idea that you have to defer to state legislatures, the Supreme Court is now giving signals that maybe it's actually gonna start interfering with what state courts are doing to try to, in a sense, promote the right to vote. Okay, so this is just a long way of saying that when it comes, if it goes back to that sort of the three things again, what are the legal disputes when it comes to the 2020 elections that are going on right now, those are pretty sort of turning to resolve themselves and the election will go forward. What are the disputes gonna be after election? Well, we're gonna have to see. When it comes to a longer term, however, I think that's when at least in my view, you start talking about these really big picture issues. What about the Supreme Court? How many states should we have? Things like that. Thank you. We've got five minutes left. And I wondered if I could ask the panelists, if you are a law student or a citizen concerned about this election, what would you do on election day and thereafter? So kind of advice for the concerned citizen or a law student who'd like to be helpful in the democracy at this moment. Oh, I have plenty of thoughts. I would love to hear from you all. I mean, obviously vote, vote, number one vote, right? Number two, if you've got the extra time, you can obviously volunteer with any of the advocacy organizations that are trying to help other people get their votes in. And three, really just getting yourself informed on the issues. Some of the things, especially that Professor Mannheim has talked about are complex specialized areas of law. So taking the time to take courses like that so that you can become informed and then later on advocate for those particular areas that you're interested in. And I would just say to piggyback on that in a plug for local journalism, I think there's definitely the law, but I think we only know what's happening in our own communities and our own regions because of the work of journalists on the ground who are making sure to observe and report back any irregularities or things that come up. We have a whole team of folks that are working on that as do other newsrooms. And that says not anything to be taken for granted. I think there's a lot of states and localities that don't have any local journalism anymore and that's a very real threat. And as we think about kind of this whole ecosystem, an informed public requires that we have journalists on the ground doing that work. Yeah, I would all of that. And in addition, vote for the president and vote for everyone else on the ballot. I think that's something that often gets lost every presidential year. And there's a lot we can do to protect Washingtonians that with our state legislatures and legislators and judges. So vote and see council and mayor and all those things. And then I think the other thing echoing what's already been said too, but is regardless of who wins that we don't become complacent, right? Like we, there's some really unprecedented things, horrendous things that this president has done, but there are terrible things that democratic presidents do too and we need to be holding all of our elected officials accountable and not become complacent. So if there are issues that you care about, there's lots of ways to get involved politically, but there's also ways to get involved non-politically to be pushing people to be doing the right thing. So everything that people are saying makes a ton of sense to me. To circle back to the point about journalism, I can't emphasize enough how important that is. And I'll circle back to this theme in just a second, but I think that there is a lot of work to be done all the way down. And when I say all the way down, I mean civics education for my kids all the way down. And again, I'll come back to that in a second. So if I were lost in right now trying to figure out what to do with respect to concerns with the election, I think I might consider the following. So first I have to pay attention on election day, what's going on, that's my job, but if you don't have to pay attention to what's going on election day, consider taking the day and going off on a hike and just enjoying yourself. The news will wait, it will be there when you get back. And speaking of when it comes to the news, if you hear things breaking quickly and they seem very dramatic, very certain, coming out really quick, those are probably not correct facts. As somebody who is often called on and trying to do her best to explain what's going on, to reporters and to others, I can tell you it takes me time. I can't look at a court's opinion in 10 seconds and start commenting on it. It takes me a while to get to it and to process it and to explain it. This is certainly that's gonna be true generally on election day and the days thereafter. And speaking of when it comes to the days after election day, folks, there's a lot going on right now, but we have laws and this is an election and we have a process to follow and we have to follow it to the extent that anybody right now from the top on down thinks that we can just bypass these procedures, bypass these laws, that is not correct in my view. So we have rule of law, rule of laws in place and we have a procedure for resolving this election. And then what I'd say when it comes going forward, you know, there's a lot going on right now and it's really, really hard to sort of grapple with it all, but there are a number of signs for hope, I think. And one is that I said before, I talked before about the need for civics education. And in 2016, I think it was the Adenberg Public Policy Center or I don't quite remember the name. They have a study that they put out regularly testing citizens knowledge, civics knowledge and the results are appalling. I mean, it was something like over a quarter, only a quarter of the population could name all three branches of government or something like that. And so this is in the 2016, this survey went out and we got those results. In 2020, they did it again just recently and you know what, all those numbers skyrocketed. People are paying attention. You guys are not the only people who care about this. We're not the only people who care about this. There's lots of people who care about this. And frankly, and I know I'm talking too much but I'll end with this. A recent study by some political scientists looked at what they called the costs of voting and they found that when they compared all these different things, these impositions to voting, Texas was the most expensive state. If you think about it like that to vote, it's really, really hard to vote in Texas. And when I see that and then I see that there are nine million people who have already voted in Texas that warms my heart and it confirms that again, we have a lot of energy out there. I'm a lot of people who care about this. Thank you so much. I wanna thank all the panelists for joining us today. It was really a pleasure to hear your perspectives and your expertise and I wanna thank Professor Mannheim in particular for all the work she did in organizing this. So thank you and if we were in person, we would be clapping and be well. Thanks everybody. Thanks for attending.


Hello guys, one greetings from Germany. I'm from Morocco and I've been living here for a while. My name is Mohammed Shahid and I just finished my my tip-of-course and I'm really glad about that. I found the course to be informative, rewarding and useful despite that I've been actually teaching for a while in China before I ever decided to take it. I still benefited a lot and the help from my teacher was was was so useful. As an educator this course helped me so much in numerous unexpected ways from creating lessons to how to approach a class of foreign students with different cultures. More significantly this course gave me so much more faith in my ability as an individual and as an instructor. Therefore from my perspective and assuming that you would like to begin a job as an English teacher take this course you won't be disappointed at all and they guarantee that. To sum up the course is competitive and it is definitely worth the money. I would suggest this course to any individual searching for a good Tesla program which could get you ready to become a good teacher and also to get one of these certificates. Thank you so much.


I mean, I guess, I guess if we get stuck in Minneapolis, that's where we're supposed to be anyway, so. Right, Evan? Is that why you were late? You were scouting locations? Okay, hold on, I gotta do this real quick, okay. This is actually for expenses, to prove I was here. Computer, don't we? Is that on? Are we on? On? Okay. Hold on, one sec, hold on. No, one sec, one sec. Geez. Okay, okay, all right. Okay, there we go. All right, let's start, let's start. So this is a just-in-time presentation, not to be confused with just-in-time. I'm Aaron. Okay, so I'm an extremely, extremely horrible procrastinator and instead of working on my presentation, I kept putting stuff and doing actual work and putting stuff in Slack and Eileen suggested that I title my talk Things I've been doing besides preparing my talk because I get a lot done when I am supposed to be doing something else. Anyway, I get extremely, extremely nervous about giving presentations and I'm always paranoid that I'm going to go way, way too fast and I won't have enough slides. So the other day, DHH made this tweet. He said, I'm just going to say it. I hate foo, bar and baz as prototypical variable names is time for a change. And this made me laugh a lot.  This is what I'm doing, working on my presentation and then, and then, and then DHH liked it. And I'm like, yes, there's two slides done. I'm getting like four or five slides out of this. So I think the presentation is going okay so far. Anyway, oh, hello there. My name is Aaron Patterson. I may look different on the internet than I do in person. This is what I look like online. If you don't recognize me, if you don't, some of you may have noticed maybe a cat picture coming up on your screen. I don't know if you recognize this, but Jason and Kevin, I'm not sure why. You should have accepted my airdrop. I'm not sure. I had sent this one too, geez. Gabrielle, come on. Just trying to send you photos of my cat, please. Anyway, I work for a very small startup company called Github. I don't know if you've heard of it. It is the only legit company I've ever worked for. I really enjoy using Gith, but I will not force push it on you. Now, I make this pawn at every conference and my coworkers told me that I really need to branch out. But I had to say to them, look, look, I'm just really committed to these Git-based puns. Not even flogging me is going to stop me. In fact, even if you reflogged me, that wouldn't stop me. Anyway, if you don't like these Git-based puns, we could switch to SVN, but I just don't know what the Git-diff would be. How long do we have left? All right, so, I love titles. At Github, I'm a level five engineer. I am level five, and despite the fact that I have played Xenoblade for over 90 hours, I still don't have enough X-speed level up to level six engineering. So I'm going to be working on this every day and my boss is here in the audience. I'm going to be working on Xenoblade every day until I can get up to a level six engineer. So, I guess I should go to real content. No, not yet. I love local businesses. I love local businesses. Before I visit any place, I look up the best local businesses for that place so that I can go help support them. And I was inclined to learn more about Pittsburgh.  Anyway, so you may or may not know this. Hines is a local business, so I bought some ketchup, there's hashtag local, yes. These are really exciting. I wanted to support local businesses even more, so I picked up some Hines Texas barbecue sauce. That is also hashtag local, this is exciting. Then I went and picked up some Hines beans and these are actually amazing because there's 57 varieties of beans. You may not know this, there's 57 different varieties. But anyway, I thought this was really neat, but unfortunately I looked at the back of the can and they're actually made in England, so they're hashtag not local, so I apologize for that. I was trying real hard, I'm not sure what the deal is there. So anyway, I love local businesses and I hope you got the point. Pittsburgh people, yes, yes? Come on, I know there's like five people in the audience that got this one and I'm sure you're laughing real hard. Thank you, Evan. The end of Rails comes where I waste one of your hours. So the day before yesterday I learned, I've got 99 problems and I guess they're all yours. Anyway, I really, really love coming to RailsConf and it's really exciting for me to be here. I like to visit everyone here. I feel like it's kind of a homecoming for me. I get to see all the old faces and new faces, meet new people, see other folks on the Rails core team, do a lot of mingling. So I want to catch up everybody on what I've been doing over the past year. So this past year I actually, I became an uncle, which is exciting for me. I've never been an uncle before. This is my niece. This is me touching her face. Surprise, my sister, let me do this. Since this past year, I actually became extremely famous in Japan. This is true and if we have time at the end of the presentation, I will tell you exactly why. But I made this tweet right here. It is one character, one character and it garnered 81,000 likes, which from a social media perspective is like a really good deal. Like characters per like is like very, very efficient here. This is worth my time. So any of you social media managers out there, look at this, look at your likes per character. I mean, this one's like great. Anyway, this is well worth your investment. I decided to become a thought leader again. Every year I want to be a thought leader but it just doesn't work out. But the only role model I have for thought leadership is DHH.  So I've got this new head shot. And you may notice the resolution is a little bit off on mine but that's because I've got an older Mac. It's still a 480p camera. So I'm sorry, not quite there. I'm still working on it. But anyway, I saw he like, I follow him on Instagram and he has this amazing desk. So I thought, well, I want to have an amazing desk like this too. I'll set it up like I can become a thought leader too if I have the right desk. So I want to show you, like I put together mine, like this is DHH's office and here's mine. It's exactly the same. You can see it's perfect. In fact, one thing I noticed when I took this photo, there is actually that photo. But really, I'm actually really, really good friends with DHH. Just check out this selfie that we took together. In fact, he invited me over to his house the other day so I went and I took a photo and this is it. Okay, okay, that's a little dark, isn't it? All right, this is just a test slide. So I was doing a test slide here. I just wanted to see if this would actually work. I don't know if you can see it. There's something right there in the slide. It's right between the S and the W. I'm not sure if you can see that right there but it is a zero-width space that is character that's between those two and we're actually gonna talk about that. We're gonna talk about that in a bit. This is just a test slide. Anyway, so this tweet really annoys me. So I have really weird conversations in my head. Like, I think I have a problem focusing or something. I don't know, I have weird conversations with people in my head. I think, okay, we're gonna talk about this and this is how the conversation is going to go and I kept replaying it over in my head and I'm like, I have to get this down somewhere so I made a video out of it. This is the video I made. Anyway, so this is what's going through my head. I'm having this conversation in my head. Unfortunately, DHH is the last one to say something. So anyway, I'm thinking about this. I'm thinking about this foobar and baz as prototypical variable names and then I'm thinking about code in Ruby and I don't know if all of you know this but a valid identifier in Ruby is anything that is not a space basically. So this code works. I'm defining a method that's named smiley face and I'm calling the smiley face method there at the bottom. So if I run this, it'll work and print out neat. This also works, I've got a method with smiley face and it takes keyword arguments and the keyword is a green heart. Defaults to a train and you can call it and passing keyword arguments and it prints out that little trolley thing there. So this is what I was thinking about is foobar and baz because they're really just representing something and I thought, okay, well what else can we do with this code? Here is another example. Can anyone see what's going on with this code? I'm gonna wait for you to type it out on your laptop. You can give it a try, it actually works. This really works. No? Okay. All right, so I'll explain it. Right here at the very top, the name of this method is actually a non-breaking space. So there's a non-breaking space right there and down at the bottom there's another non-breaking space so I'm calling the method with a non-breaking space and if we look at this in Vim you can actually see it. On the right is the source code in Vim. You can see the non-breaking space character there but when we cat the file and actually run it, it totally works fine. So let's extend the program just a little bit. We're gonna do one extension to it here. In this case we have a method with the name, the name of the method is a non-breaking space and we have a parameter given to the method that is a zero-width joiner. So right there that's our name, our parameter we're calling puts. So here it is again in Vim, you can see like right there this is of course this is Seattle style, there's no parentheses. So you can see in Vim we've got the non-breaking space and the other space and it totally works fine. So what I'm proposing today is that instead of using foo and bar, we replace foo with zero-width joiner. We replace bar with non-breaking space. So all right, with that let's write some production ready code. I dare all of you to put that in your app code tonight. Wait, don't do that, don't do that. Don't, don't, okay. All right, so today, surprise, surprise, we're going to talk about performance. I always give performance talks. We actually just had our performance reviews at work. My boss is in the audience. We, you did a great job on my performance review. Thank you. Anyway, so we're going to talk about performance today, but I don't want to talk about the speed of your code. I want to talk about the speed of your development, like your actual development getting features done and actually making that website that you need to ship. Now the thing is I was thinking about this and we're always getting faster. We're learning new tools. We're learning new technologies. We're learning new techniques. But unfortunately, it seems like as time goes on, your team velocity gets slower and slower. It takes longer to add a new feature. It takes longer to ship your code. So why does this happen? This seems paradoxical. Who is this mystical man? Why are we concerned with his months? But seriously, why are we slower? Why are we slower to ship these features over time and what can we do about it? Well, there's different ways that we can deal with some sort of situation like this. One of those ways to deal with this situation is to rewrite. We want to rewrite our code. We want to chase that dragon, that dragon that we felt, that high when we wrote Rails new and we can get all of those features out the door really quickly. We want to throw away all that code and say, okay, I'm going to begin again and feel that exhilaration when you can ship new features quickly. But if you're doing this all the time, if you're just rewriting something, all you're doing is going down the same path over and over again. Maybe the weather has changed, but the places you're going are exactly the same. What good is this velocity that you get if you're just running down the same path over and over again? Now, the day before yesterday I learned that if you don't want to deal with these hard problems in your program, you can just hire someone to fix it. Hire someone better than you to fix that scaling issue in your application. Now unfortunately, I cannot afford that. So I have to pick a different solution. I choose to refactor. We could choose to refactor our code. We could make performance improvements and pay off technical debt, et cetera, et cetera. But the problem is it seems like we have these two different arcs juxtaposed against each other as application developers. It seems that we either have to be software writers or we have to be a PhD in computer science to work on an application. Now in my opinion, there has to be something in between. We can't just only be software writers. We can't only be PhDs in computer science. We have to go deeper into the application. We have to go down this cliff so that we can actually solve these more difficult problems. Now, as we learned a day before yesterday, I am the only one who can fix this. And we've recently learned that that doesn't actually work out. The truth is that you're the only one that can fix your application code. So the generic features that Rails provides gives you a great bootstrap. You can get going early. We give you all the things that a generic web application would require to ship a feature, but as time goes on, your application is getting more and more unique. You know your business logic. You know your business. I don't know your business. I can't help you with that. So I think this is one of the reasons that application development starts to slow down over time is we have to get better at our particular domain. So those generic features in Rails give us a huge, huge boost. And I think this is why I like to focus on Rails a lot. This is why I speak about profiling and speed improvements with Rails is because I want people to be able to take their Rails application and live with it for a long time. I want them to have a successful business, not have to run into these scaling issues as often. So I love Rails and I want Rails to continue for a very long time. So this is why I study these particular things. So today we're gonna be talking about profiling and tuning. We're gonna specifically look at runtime and memory management which sometimes I like to call time and space because it makes me feel like I'm on Star Trek. And it makes me feel a little bit more important than I actually am. So let's look at some stuff. We're gonna look at types of profilers and how to use them, how to build them and basically how to improve our application performance with them. So there are two types of profilers that I typically deal with. One is called an exact profiler and one is called a sampling profiler. And to figure out which profiler you need when you're profiling your application you just need to ask yourself a couple of questions. If you're asking, the thing that you're trying to solve if you're asking is how many, how many of something, how many method calls, how many object allocations? If you need to know an exact precise number then you wanna use an exact profiler. Now if you don't care about an exact number like let's say you need to know is this what is the slowest method? What percentage of time does this particular thing take? You wanna use a sampling profiler. So if your question is how much, use a sampling profiler. So in order to figure out which one you need just ask the question out loud. What is the thing that I'm trying to do? What am I trying to solve? So let's take a look at exact profilers. We're gonna actually implement an exact profiler so that we can understand the way that it works. This is an example of an exact profiler written in Ruby. It uses the TracePoint API. This TracePoint basically what it does is it hooks into the call method and every time a method gets called once we enable the TracePoint our hook method will actually get called. So we listen for the event and every time we get an event we just keep track of that and increment the counter. And you'll see at the bottom of this code here it outputs the result which says, okay, benchmark me was called once, fast was called a thousand times and slow was called 10 times. Now if this code wasn't so simple you might look at this and think, oh, the place I need to optimize is the fast method or the method called fast. That's the thing I need to optimize, it's called the most. But obviously when we look at this code we can tell the slow method is a slow one. So this is the particular case where we wouldn't want to use an exact profiler. This is misleading us. If our goal is to speed up this code we don't want to use an exact profiler in this particular example. Now the other profiler is a sampling profiler and this sampling profiler, what it does is it stops your program at intervals, regular intervals and it asks the program what you're thinking about. So every so often, it asks what you're thinking about and just records it. So every so often we have a program here that's running so time is along the bottom, our program runs, it stops, we record what it's doing and we keep doing this at regular intervals until we're done sampling it. Now the idea behind this type of profiler is that let's say you have a slow method. If your method is slow then the probability of being in that method any time you stop the program is higher. So we can tell that if we're in this one particular method frequently, if that shows up frequently within these samples that's probably where our slow code is. So we can write a sampling profiler and Ruby here's an example of it. What this profiler does is it starts up a thread and it asks the main thread at an interval, what are you doing, what are you doing, what are you doing every so often and you can see at the bottom we have results from that and in this particular example we have almost 6,500 samples inside the sleep method and maybe one sample inside the slow method. So we don't know how many times the sleep method was called but we do know that when it's being sampled it's very frequently in the sleep method. So 99% of the time we sample, we know it's inside the sleep method so that's the thing that we need to target. Our fast method didn't even show up in this profiler. So when to use exact versus sampling? It depends on what you're trying to measure. Again if you're trying to get absolute counts using exact profiler, for example the number of objects in the system or the number of times a method was called or API or whatever. If you're trying to reduce time or space then you want to look at a sampling profiler. If you're not sure, use a sampling profiler. Sampling profiler is what you need about 99.963% of the time. That is with one sample. Check my math, it's wrong. So let's take a look at some runtime profiling and tuning. We're going to talk about tuning your runtime. Profiling your runtime and tuning your runtime. So my favorite tool for doing runtime profiling is a tool called Stack Prof. This thing uses MRI's built-in sampling capabilities so it doesn't spin up its own thread. You can actually use a timer inside of Ruby. Ruby has a timer inside the C code, not inside the Ruby code, that lets you sample every so often. So Stack Prof just uses that built-in capabilities and it looks very similar. The API looks very similar to the one that we wrote earlier. We just wrap up our code inside this block, execute it and when you execute it, it saves a profile to a dump file. So I've specified profile.dump here. And when we're done with that, we can just run Stack Prof on that and it dumps out all the sampling information. And this looks similar to the information we got in our profiler, just much nicer. So we get kind of the same information, 999 samples. Again, we can't tell how many times the slow method was actually called. We just know that it's in the slow method very frequently. So we can use this to benchmark Rails boot time. So let's benchmark the Rails boot process. Let's talk about the Rails boot process. So let's take a look at how a breakdown of the Rails boot process. If we look at it like this, where time is moving along to the right there, we can divide it into a few different sections. So at the very beginning here, that's where we hit enter. We hit enter right there. Then we start requiring files and once we've loaded and required all the files, we start what's called compilation. We're compiling the app. It's not really a compilation phase. Essentially what it's doing is taking all of your rack middleware and putting it all together and then handing that middleware off to whatever your web server is. In this case, we're using Unicorn. So after compilation occurs, we hand off the application to Unicorn. Now after that, when you get your first request, it has to compile the views and then actually execute your business logic. So if we look at this from a little bit higher level, we can say that these steps here, these requiring files and app compilation, we can say that that's our time where we're not ready for requests yet. We're spending time here and we're just not ready to respond to a request. And now the first request that comes in, it's gonna spend time compiling our views and then executing the business logic for that page. And then the second request is only going to execute the business logic. So if we wanna profile these things, now we know what to talk about, where we're actually looking. So knowing this, we can figure out what we need to benchmark depending on the task. So what we're gonna do now is we're gonna look at benchmarking a script, just a startup script. This is an example here. We have, we just load config environment. This is how you would benchmark that very first slice before we're actually compiling views. This is going to load all of your applications, information, all of your, I don't know, models and stuff if you're using production mode. Maybe it's lazily loading and development. But this is how you can benchmark that overhead for your scripts. Now if we look at the output from this, you'll see, okay, it looks very similar to our previous output, but I wanna call out one thing in here. You can see that we're spending some time in garbage collection, right there, GC. So we can see what percentage of our time is spent in GC and we're gonna look at how to improve that time a little bit later. So runtime tuning, now talk a little bit about tuning your runtime. Now the thing is Ruby will try to execute your code as fast as it can. So there's not really much runtime you can, tuning you can do. The best thing that you can do is to write faster code.  Obviously we want the Rails framework to be as fast as possible, but at some point we can't make the framework any faster. So you need to look at your own code to see what's slow about it. And the most advice I can give about that is say like, okay, use these tools, find where your bottleneck is, look at it and say, well is there a faster way to do the same thing? Can I do this thing in a faster way? Is do we need to even do this thing so much? Maybe I can only do it, I only need to do it once or twice. Or maybe I can defer this until a later moment. Maybe I don't need to do it right now. So these are questions that you should be asking yourself as you're trying to tune your application. All right, let's take a look at memory tuning. We talked about tuning runtime, let's take a look at space, actually using memory in your application. But before we get to memory tuning, we need to talk a little bit about memory, how memory is handled in Ruby. And I hate this so much. This made me so mad. How did David know that I am going to talk about garbage collection in this presentation? Am I that transparent? Why? He said that in his talk and I'm like, no, how do you know? Anyway, so let's talk about a garbage collector. The GC has two responsibilities in your application.  It's freeing up your memory. But the other thing that it does for you is it actually allocates memory. I think people don't think about this as much and I think the reason is because of the name.  But it's actually in charge of allocating your memory too. So what is MRI's GC? I'm going to tell you what it is. We're not going to go through all of the algorithms and stuff but I just want to give you a high level overview so that when you have time later you can Google these words because that's really what's most important is knowing the key words to Google. Seriously.  It's generational, has incremental marking, lazy sweeping and uses a free list for allocation. I feel really, really bad for the translator. I'm sorry. Anyway, so the generational part of this just is a optimization on top of mark and sweep. It makes your GC faster. Incremental marking and lazy sweeping mean that we have shorter pause times. So when MRI's garbage collector is concurrent it is not parallel and what that means is as your program is running every once in a while it has to stop and do something. Now these incremental marking and lazy sweeping mean that those pause times are very short and what that does is it increases the throughput of your program. So you can actually run stuff through it faster. So one question I think is interesting is when can the GC collect garbage or when does the GC collect garbage? When does that happen? Well, there's three main places. If you just call GC.start it will collect garbage though it's not guaranteed. And none of these are guaranteed. These are just locations where it could collect garbage. That's why I said can. So if you call GC.start it will do a collection. Maybe it's not guaranteed but it will, don't worry. Now another case is when you allocate an object. If you need to allocate an object the GC may do a collection. Or if you allocate memory via malloc the GC may do a collection. So allocating objects and allocating via malloc are two very different things and we're gonna talk about the differences between those in a minute. So this leads us to what is an object in Ruby? In Ruby an object is a 40 byte chunk. So it's a chunk of memory that's 40 bytes wide. So let's say we have some code that looks like this here at the bottom. It's a string that says OMG, this is a very long string. Okay, now this string is obviously too large to fit inside of a 40 byte chunk. So how do we actually store this? How can we have a string that's this long? The way that it actually works is rather than storing that string inside of the Ruby object itself we have a string object that points at something else. So we have a string object, a Ruby string object that points at a C string object. Now the garbage collector is in charge of allocating these Ruby objects. And malloc, the system malloc is what's in charge of allocating these strings. So we have a Ruby object that points at a C object. So these are, in this particular case we have two places where a GC could have happened. Now Ruby doesn't allocate one object at a time, instead it allocates a page. So an object is a 40 byte chunk and when you say give me an object it pulls that object out of a page that contains many objects. So Ruby won't actually, or the GC won't actually allocate one it'll allocate a page and then it'll give you one of the objects inside of that page. So in this particular example we allocate our Ruby string. Our string is just a Ruby object inside the page and it points at a different string that was allocated via malloc and in this case we have two places where the GC may have run. So slot is a location where a Ruby object may or may not be and this slot term is important when it comes to tuning the garbage collector because we allocate in terms of slots. Well in terms of pages and those pages have slots and this is the way that we tune it is by the number of slots that we have available. So we can control the total amount of free space in the program and the way that we control it is via this term slots. So let's look at some tuning parameters. GC tuning is done via environment parameters and we're gonna take a look at some of them and what they do. So this is the first one we'll look at is called Ruby GC heap init slots. I don't expect anyone to remember this. I promise I will put these slides online so you can just Google it later. But we can see how it impacts the process here. If we run Ruby you can see the number of available slots with no tuning is about 1800. We have about 1800 slots available. Now if we say init with 600,000 slots and we run it it has about 600,000 slots available. So with this particular parameter we can say I want this many objects or this many free spaces declared when the script boots or when the program boots. So let's take a look at that profile that we looked at a little bit earlier. This is an example here we have our app boot. This is the default case that we saw previously no changes you can see how I ran the script at the bottom of the slide there. Now it's spent about 16% of its time in allocation. 16.5% of its time in GC. Now if we tune the slots and say we wanna init with a bunch more slots we can do this. We just say let's set that environment variable to 600,000 you can see the command at the bottom and in this case we only spent about 8.5% of our time in the GC. So we can see kind of an impact on that when we do go to boot our application. In the top one there's no tuning and it took about 4.1 seconds and in the bottom example it took about 3.5 seconds and this is a default application, default Rails application where I just randomly chose a number and the nice thing about this is that we didn't have to change any of our application code and we were able to decrease the boot time just by saying hey I want you to prepare a bunch of room for allocation. So we didn't need to change our app code at all. You can use this if you measure the objects that are allocated in your system you can come up with a good number for this and then decrease your boot time just by tuning this particular parameter. The other thing that we can do is we can control growth and these are very long, very long. So we can control the growth of our heap via these particular parameters and again you don't need to remember these the important is that we're applying a ratio to the amount of free space that we have in our application. The GC tries to maintain a certain amount of free space for you to allocate into and we can specify those ratios so we have a minimum ratio, a maximum ratio and a goal ratio. So we have our min and our max and we're trying to hit the goal so kind of a window there. Now let's take a look at a test program. We have a program here, this is our test program and it just allocates a whole bunch of objects. It allocates 600,000 objects.  So each time we allocate a new object we may have to expand that heap and we want to look at how many times we actually had to expand the heap. So if we set our goal to particular values we can look at or we can, excuse me, we can study how many times it had to expand the heap by checking the GC count. So if we tweak the goal numbers a little bit and then graph it we can see a different behavior and this is an example of that behavior. Our goal, the blue line, our goal is set to 0.2 and our green line, the goal is set to 0.7. Now the interesting thing about this is we're able to hit that 600,000 mark in fewer GCs with a larger goal. We're essentially telling the GC okay I want you to be aggressive about allocating memory, then allocate it and then expand into that. Since we're aggressive about allocating memory we're able to hit that goal or hit that 600,000 mark sooner. So unfortunately this means that we have more memory growth. We're not being conservative about our memory growth. The blue line, we're more conservative about memory growth but we have to spend more time in GCs or using more CPU time. So it really depends on what your application is doing and what you're trying to tune for. If you want to tune for fewer GCs you may want to increase your goal. If you want to tune for lower memory usage you may want to decrease your goal. And you can actually try this at home. Again this slide is literally just for being online later so don't write this down or anything, please. Like you wrote down the example code with the non-breaking space in it. All right so let's take a look at some memory profiling. I saved this for last because I actually have a story to go along with it. At work, somebody pinged me at work and we have an application at work it's called the GitHub Enterprise. And basically what it is is it's a version of GitHub that we ship out to clients and they host it in their data centers. It's just I guess stored in a container or something. I don't really know all the details but one of the people on the team came to me and said Aaron I've noticed that as time goes on the application that we're shipping is consuming more and more memory. It's getting larger and larger and do you know, can you help with this? Do you know why this is? So I said okay well I'll take a look at it and see if I can figure out what the problem is and the first tool that I reached for was Stack Prof because it was my favorite tool to use and this is an example of using Stack Prof with object allocations. We can use a sampling profiler with object allocations. So I ran this code and I required our environment and took a look at the output from that and this is what the output looked like. It told me that we were spending, this isn't actually our application, this is a sample app but it said oh we're making a lot of allocations inside of the require method. Now the problem with this type of profiling is that even though we made a lot of allocations in the require method, those objects aren't necessarily retained. They may not live for the life of the program so if they don't live for the life of the program we don't really care, it's not contributing to our bottom line so to speak. So in this case we actually needed to switch to exact profiling. This is a case where we said I need to know how big is the heap, I need to count exactly the number of objects that are in memory right now. So this is a case where you want to use exact profiling and to do that I use object space, this is my favorite tool for doing exact memory profiling and the way that I use this is via heap dumps so this is a way to get a heap dump and I will describe it a bit here. This dumps your heap to a JSON file and in this case I'm just dumping it to a file named mydump.json. This line up here, the second line, what it does is it enables allocation location recording. That's fun to say. Any objects that are allocated before this line we don't know where they've been allocated. The garbage collector does not keep track of where they're allocated. After this line when we call this method it tells the GC hey every time an object is allocated I want you to record where it got allocated. So anything that's allocated after that line we know where we can find out where it was allocated and that information will actually be inside the heap dump. So after I enabled that I said okay let's require the application boot it up. After we've required the allocation I say okay I want to do run a GC, I want you to clear out any objects that are temporary that aren't going to live for the life of the program. Then dump out the heap. So once the heap is dumped out we just have this JSON file and each line in the JSON file represents one object. One line is one object. And I know this is hard to read so I've made it a little bit nicer here. So this is an example of an array. We can tell it has the type, it says array. It gives you a list of all the references so those are all the objects that it's pointing to. We can actually see the allocation location of that object. And finally my favorite is the memory size of that object. And these JSON files make for a very extremely easy analysis. So for example let's say I want to know how many objects are alive? Just how many objects do I have in the system? Very easy I just count the number of lines in the file and I know. So this example had about 185,000 objects. Done. Okay so how about I want to know what file is most popular? This is one tool I really like to use, JQ. If you don't have the JQ tool, get this, it's amazing. Basically it's like a really great grep for a JSON. So I can say hey I want to extract the file, just the file field. I'm gonna sort it and get unique them. Then I'm gonna sort those by the number of counts and then just get the top 10. When I run this I can see okay well the top allocations came from activesupportdependencies.rb. So I know it came from that file. Okay I focused in on that file. Now I can use JQ again and say well I only care about allocations that are inside that file. So I want you to filter out all the allocations for that file and then give me the line. Then we run through the same, basically the same command again, sort them, get the unique count, sort by account and then get the last 10. So we can see here okay the most popular line inside that file is line 283. So I know okay I need to go look at that file, look at that line and now I can kind of get an idea of why we're allocating so much stuff. So we can also answer other interesting questions like how much memory does activesupport allocate? So I just say okay give me all of the records whose file contains activesupport, extract the mem size from that, pipe that to a little awk command and sum the total and we can see that okay activesupport allocated what is that like around three megs. Okay so back to our performance problem at work. We ran into a thing, I want to talk about hidden references, this is the thing that we ran into. We're trying to find what has references to what. Basically this heap dump contains all the information of the entire object graph, we can make a graph out of it so let's talk about references. This is very simple, we have an array, we have references to things, this is obvious. We can see okay we have an array, it points at two symbols. Now here's another example of some references in a program, we have a lambda, this function foo it returns a lambda and that lambda sums the two values that you passed in. So the variable m contains two references and I promise this isn't a trick, there are no non-breaking spaces in this code. So all right here's a question for you. Here's a similar code, the foo method returns a lambda, how many references does this lambda have? You don't need to answer, I will tell you the answer, just think, how many references does it have? It actually has two references, the reference is the number a and b and there are two reasons why this lambda has references to those variables, one Ruby doesn't do what's called escape analysis, so it doesn't look at the lambda block and say oh I'm not actually using those things, there's no reason to keep track of them and the other thing which is basically an implementer's nightmare is a thing called binding. So if you ask the lambda for it's binding, that binding object is the environment in which the lambda was created. You can actually ask the binding, hey what local variables do you know about? It says hey I know about a and b, they are local, when I created, when the lambda was created and then you can say hey binding, give me the value of a and b. So we actually have these references, this lambda contains references to a and b but you can't really see it when you're just reading the code. So I was able to find these particular locations, unfortunately I didn't understand this code, like I could point to it and say hey that's the problem, we're allocating a bunch of stuff there, but I couldn't really fix it, so I worked with other folks on the team and they actually rolled out fixes for this that eliminated those hidden references and we were actually able to drop 200 gigabytes from our memory from our production application. So this was like really awesome and I praised the person who did this a lot, it was amazing, a really amazing change. So I wanna talk about some tools that I want. And then maybe some thought leadery stuff and then we'll wrap up with another story and maybe two stories depending on my time. So let's look at the Rails boot process again. Unfortunately so far all we've been doing is looking at analysis of scripts, like maybe running DB Migrate or other scripts that you run with your application. We haven't looked at anything else. What about app compilation time? I mean when we run this it's easy to know, okay this is how much our scripts are spending inside of just loading the application, but what about that actual compilation time? I wanna know how long does it take to restart a web server? I wanna optimize that, how do we deal with that? So how do we deal with benchmarking the rest of these things? We only looked at that one. So today we're gonna take a look at some new rack up options that I've added in order to deal with this particular situation. So rack, we merged this into rack a couple of weeks ago. This is one of the things that I was doing instead of working on my talk. In newer versions of rack you'll be able to say, hey rack I want you to dump the heap before you or after you compile the application but before you give it to the web server. Or I want you to profile the application up until you give it to the web server. So we're going to build this tool interact for you so that if you wanna know why is it taking so long for my application to restart or be ready for requests we can actually have this built into rack and give you some information. So you know how to identify problems in your code and we've written the benchmarking code for you. So this just leaves us with two other issues, our warm up time and our actual and our run time. So we're able to profile script boot time or able to profile compiling applications but we're missing two components. Now last year Eileen spoke about system tests and this is a really important thing. So besides the fact that now you can do entire system tests this is an important refactoring on Rails itself it means that we can actually run real requests through the application. So something that looks more like a request that you would see in production. Before we had system tests most of the stuff, I mean we on the Rails core team knew this and maybe most people did not but a lot of the integration tests were basically just BS. And I wanna say that, I'm trying to say that in the nicest way possible. They would actually run the data, run these requests through only your application it would not consider any middleware that you had in your application and this is the advantage of the system test is that they actually run the data all the way through the middleware into your application and then back out. So you can get a more real world idea of performance of your application. So now that we have that, now that we have that system in place for running these system tests. Sorry, I said I'm a little scatterbrained, right? Okay, so now that we have the system in place we can run real requests through and actually benchmark those. So we have a script for doing this at work. It's called this, this is bench app. We use this for benchmarking our actual requests at work and this is what the output looks like. It's very easy to read. So I wanna tell a little story about using this tool. We had a, this is amazing. This was amazing. We had a page at work that would allocate 13 million objects per request, 13 million. And I mean, this page, this page would return in like a second. So anytime anyone tells me Ruby is slow I'm like, what? We did 13 million allocations and we're able to render that page in a second. Like this is pretty good. Anyway, I was able to get it down to about 300,000 objects per request. Using these tools so we could monitor an actual request going through the application and get a real benchmark with it. And I wanna share the graph with you because it's extremely satisfying. So this is the graph and you can see where I pushed the production there. It's very, very nice. So the thing that I wanna do, the thing that I wanna do with Rails 6 is that I wanna make performance easy. So we make, we Rails, we make generating applications easy. We make connecting to databases easy. We make writing routes easy, generating views, working with forms, writing tests. All of those things. But I think that now it's time for us to do the same thing with application performance analysis. I think we're falling down in that particular area of Rails. Yesterday, yesterday Eileen said something that I really, so first off, her talk was amazing, amazing. I said to her, your slides look really, really great.  And I was like, whoa. Look, I, yes, I did drop out of college, but that like, that hurts. But yesterday, she said she wanted to make Rails scalable by default. I think that this is really important. This is a really important thing, but when I think about making Rails scalable by default, I don't wanna make just the application itself scalable by default. I wanna make your team scalable by default. I wanna make it so that not only is it easy to get started, it's easy for people who don't know much about Rails web applications to get started. That's the way it is today. We can easily start a Rails application, but I don't think that the Rails team is providing as much support as we can mid-career. So that's where I wanna start improving is making sure that we can actually provide tools for you to move from beginner on to more advanced. So for those of you like me who cannot hire somebody to fix their performance issues, you can just, instead of rewriting everything in Rust, you can say like, let's take about five minutes. Let's just take like a couple minutes here and run this one command and see what it says our bottleneck is and maybe we can just fix that instead. So I wanna do for application generation or I wanna do for performance what Rails has already done for application generation. And I think the first thing that we're going to do is start by taking the tools that we use internally for doing at GitHub for doing performance improvements and pushing those into open source, pushing those upstream. So I wanna tell, I have two minutes left. I wanna tell one more story. I have one story to end with and this is not serious at all so you can sit back and relax.  So last year, I gave a presentation that was much more of a waste of time than this presentation.  I talked about doing analyzing this data and different types of analysis that you could do and 40 minutes later, I revealed that I had acquired this data by a Raspberry Pi and a motion sensor that I had put together where I had mounted the motion sensor above a litter box as such and my cat would get into the litter box and then come out and I recorded all this data and made a log and this is, well I guess the cat made a log. Poop jokes, yes. So we had about 100 grams left over.  But anyway, so the reason that I built, the actual reason that I built this thing is because last year, around this time, we were moving, we bought a house. Like the day after I got home from RailsConf last year, we had to pack up all of our stuff and we moved.  If you do not use these electronics, we will throw them away. And I said to myself, what can I do with a, what can I do with a scale and a Raspberry Pi and a motion sensor? And this is what I came up with. So let's talk a little bit about home automation because this is the first thing, I mean the first thing that you do as a new homeowner I think, well at least this is the first thing I do as a new homeowner is I'm like, wow, I've got a house, I need to automate everything. I need to automate all of the things. So we have a toilet in the house as you do and unfortunately we have that like blue water in it. Okay, it's got like that blue, clean water in it and the thing that's really bad about that stuff is it's basically tied pods for cats. They like, except that unfortunately my cat isn't as smart as a 13 year old and he tries to drink that stuff. So I'm like okay, I need a way to solve this issue. The way I did it was I decided to automate my house. I bought one of these. This is a door sensor. Basically it's a magnetic sensor and on the inside it has a little reed switch. So that's our magnetic reed switch on the inside. Now what I did is I took a tilt sensor, this is a tilt sensor. I put my hand there for scale and basically I bypassed the reed switch with the tilt sensor. So we have this Z-wave sensor where I've mounted, imagine this is a side view. I've mounted the tilt sensor on it like that, bypass the reed switch such that when the Z-wave sensor tilts like this, oh this is our on position. So when it's like that it's on and then if you rotate it like this, it turns off. So it's as if you've opened the door, for example. Then I decided to hook it up to a siren and I took a video to demo. I think it's running, we'll see. So that's the siren there. There is a toilet. When I lift the lid, you can see the siren will come on. When I close it, of course, yes. Good job editing Aaron, yes it shuts off. So now I can tell when the toilet is open and closed. So if nobody's in the bathroom, we know to close the toilet so that the cat does not go in there. Now aside like one happy accident from this is that I actually have a log of all the times the toilet is open and closed. So I decided that Cambridge Analytica doesn't have enough of my information so I'm quite, I'm quite. All right, anyway. So I mean, I wish I had implemented this earlier so that our cat didn't drink any of that blue water and get sick, but the truth is hindsight is 2020. Thank you very much. I hope I can see you again next year.