don't really think they did you know like man that was a bad baby so i think those records um could be thrown out oh right i'm not ready to show you that all right let's go back crime by area right so yeah this is the data um you'll see there's a latitude and longitude provided and that's generalized to the nearest uh intersection just to protect people's privacy so if you see that that's not actually landing on someone's house and i think it says it says that in the read me but yeah so i'm carrying out data that is coming from 2010 to oh 12 31 2018 and it contains uh these pipes are just separators with pandas so if the string contains aggravated murder rape burglary larceny vehicle theft and a couple of entries then that's that's what i'm including in this analysis got that aggravated murder rape larceny kind of important stuff not not that all crime isn't serious but all right this is just like data wrangling well we're so self-conscious showing people my code i'm like ugh but anyway so then we have this file and we're seeing um this division or area as i call it in areas per square mile and i did that just the gis software i got the area this is more wrangling so yeah this is fun when i first joined these two okay so i got the count by area name for these different areas and i joined it with the csv file i had which shows this area in square miles of these different divisions the first time i did it well i didn't have it specifying that it's uppercase so then i was like really freaked out because it was total so you just have to be careful about all this stuff so i'm saying string to upper so it joins correctly here so i don't know just want to point that out yeah so this is a select crime by area normalized per square mile this is from 2010 to 2018 in los angeles so this is like decently difficult to look at then i said it as a plotly plot and um in the repository you're gonna have to get your own username and api key that's just how it goes i'm not i didn't share it so yeah i divided the count by the area in square miles and just rounded it that's why they all ended zero yeah i'll run this i think it already popped up but we'll just take a look yeah so this is it in a plotly plot so it's a little nicer this kind of visualization right because you can actually scroll over it i know you're like i can't say anything from where i'm sitting but yeah this is it this is that first chart as a plotly plot cool so i'm going to close a couple of these and we'll go into the next notebook okay so we'll go into series crime and this is going to harken back to that line plot that we saw in my um powerpoint and here i have i was trying to think of it what's it called in candyland you had those shortcuts where you don't have to go all the way around what's that called bridges yeah great that's a good word so i built a little bridge in this we don't have to go through all the cells because yeah these are like more heavy duty scripts so i'm just going to very carefully because if i start running it no bridge for us i'm going to run the first few importing the libraries yeah i just have this here printing the versions of plotly and map plot live because these are in my estimation these are rapidly developing libraries you want to make sure you're using like the current libraries and then i'm just defining file out data and then i'm skipping again these are the codes i have it in bigger font on this aggravated murph rate burglary etc we can ignore that so run from here so this is a simple chart you don't like have to look at it this is just trying to show you the initial path and these are my comments it says x's month y is count hue is year because i was really i don't know just really trying to grasp how to make this kind of a busy line plot because i knew i wanted it but i wasn't sure how to do it and then so this is just my comment on okay hue is year oh no it works cool so whatever so this is cleaning up the plot so run this cell all right so this looks this is a little better i think it's a lot bigger colors are a little more different and then i'm going to set as a plotly plot all right here it is in jupiter notebook it's looking a little stretched here's 2019 but it's saying like underscore line seven underscore line eight instead of the year so it's not too good we'll run this last one all right this looks cleaner just hold off on that previous one i'm going to run a flask app to show it looking a little more cleaned up this is select crime by area from 2019 to january sorry 2010 to january 2019 in los angeles so this is less busy yes a larger spike what was that that's november i don't have a good answer but you're making me think about my friend was robbed someone came in through her shower window because she like lived in a basement and it was like right around the holidays so someone stole her maybe it's a holiday thing but i'm just conjecturing yeah but what i find interesting is like why what was so good in like 2014 that's like really low comparatively and i do have january 2019 so you can see that and i also i just just one more thing before i forget i'll come to your question but do you remember reading like that mayor garcetti and chief more that said that crime is down from 2017 as compared to 2018 do you remember reading about that i was reading about it but that's what i saw as well for this serious crime and i'm not saying i have like the exact same analysis as they did but yeah so that's good uh yes that do what yeah so i'm working to like kind of create that position for myself but i'm not there yet um they definitely do do analysis of crime uh everyone in that department is interested in that kind of stuff um but i don't really have like a ton of details on like how it's done specifically just because like i'm a civilian i don't you know kind of on the outside with some of that stuff but it is yeah i i think this is how they should be doing more of that analysis like this is the right way some of the technology they're using is kind of old so this is like better all right now we're gonna look at this in a flash cap and then i'll just have like one or two other things to say all right it's just running on local hose yeah so this it doesn't it's not fitting crazy go to blame myself for that but you can see the months as well so it's kind of cool and yeah 2019 is the yellow dot there sure i gotta clean up the visualization even more but yeah that's kind of these are some examples of analyzing the data and i mean would you just to step back a little bit but it's like yeah it's cool to have all these charts and plots but would you want to actually create a different 100 different charts for all the different ways you could look at the data like i wouldn't so then the next step would be like actually calling from a database so you could query by area looking at select crime and if i'm invited back next year that's what i'll do i'll have a database put together yeah yeah so the question is is yeah this is just crime data there's also a publicly available data set on arrests but i'm not sure we have any data on convictions i just i don't know but i'll put on my email address and if you can find it i will let you know yeah yeah you're right there's definitely more to this story than just looking at it like by division and it's also another way to normalize like a normalize the data in that first notebook by area but it also be something could be maybe done from analyzing by the population because there's probably more people living in holand back than um i don't know west l.a. oh no i don't know it's panga maybe so yeah yeah that's a good point any other questions yeah yeah i know i know yeah so you know in my effort to make this kind of particular analysis more prevalent i'm i was showing some of these plots to my co-workers ones and they were like can you just put that as two bar charts i forget what the exact request was but they wanted two bar charts for like 2017 and 2018 and i'm like that's pretty simplistic but i i don't know so yeah no i definitely agree i mean there's these even like this is line charts bar charts and nothing scatter plot nothing fancy but then people ask me just for some totals for two years i'm like oh great that's fine it's helpful so yeah yes yeah no i don't know i'd say a little more yeah yeah i mean i'm honestly at the point where i am it's pretty like general so yeah i do think normalizing is important though to just kind of get a better sense of like the actual area that you're looking at um yeah it's pretty general what i am doing i mean it'd be good to make like a bell curve and just look at the distribution of the data that would be good as well but yeah it's pretty general yes yeah so the first thing i can say is if you want to do data science like it's like super popular everyone loves it right now i would start i mean i started with doing automation and then everyone's like python school and i'm like what like i've been doing this for years now and just because i've worked at like all sorts of city departments and they they all need automation let me tell you um yeah so i will say the course where the ta is called trilogy um trilogy education services that's a six month course it was three days a week pretty intense and i also on my repository i put up like i really like the o'reilly books and there's a um like sort of a self study course on python three but i think like once you come up with a question and let's hopefully you can find the data or it's not too hard like accessing the data you know is a big part of it so if you either find a throw open data or web scraping then come up with a question you just kind of like it'll just take off i think yeah plotly okay so i'm not a pro but it's uh it's sort of like a way to make it more interactive visualizations so when i was scrolling over the numbers were displaying and you can also like choose if you want to display a single point or all the data at once does anyone else have anything more to say about it's pretty new plotly anyone yeah html yeah yeah it does and you get a javascript file um it's still pretty young i think so i look at the documentation i'm like why is no one asking this question but that's also kind of cool so oh right i just wanted to put up this slide yeah so um my email address at work is horribly personal but n5875 at la pd dot online so that's where you can reach me and if i can't answer it i can try and find someone who can and that's my github i'm on social medias yes and another lifetime i also work for the federal government but i'm not really interact interacting with them but maybe i can meet them at some point yeah thank you thank you patty and our next talk will be up in about 12 minutes pretty quiet yeah i am talking quietly what if i talk louder louder louder yeah no that's good that's good it's working yeah all right welcome back uh we have our final session of the day for the open government track thank you all for coming here we have the open precinct project team up here and i will let them introduce themselves hi everybody um the title of the talk is open precinct a collaborative resource for redistricting reform we wanted to put gerrymandering in the title to really drive the crowds but then we saw that all of the talks in this track had alliteration in it so we stuck with the theme okay so um we are the Princeton gerrymandering project and you're probably wondering what that patriotic bug is uh well we like to say that gerrymandering is a bug in american democracy and we're going to squash it uh and today we're going to talk about a data project that we hope will help us do that so our director sam wong is a professor of neuroscience at princeton but he also studies elections and after 2012 he and some others noticed a pretty strange result there was a major distortion in the relationship between seats and votes in us house elections so i'm going to plot a series of us house elections on this plot on the x-axis i'm going to plot the national vote share that democrats won higher numbers to the right being more democratic and on the yx i'm going to plot the number of seats in the house that democrats won so if you plot a dot for all house election each house election between 1948 and 2010 see they're correlated that's good um and now if you plot the four house elections that have happened this decade which is all the elections that have been held since districts were redrawn in 2011 you see something pretty interesting which is that these dots are all significantly lower uh indicating a new disadvantage for democrats in the range of maybe one or two dozen uh out of the 435 house seats so you can see that for all of the vote shares that they got um in elections this decade there uh are previous elections that had the same vote share where they got more seats so uh princeton gerrymandering project was started to measure the effect of gerrymandering and you can go to our website if you want you can browse through the years you can see the results of a few tests that we have up there that can indicate for which years and which states does gerrymandering seem really bad um note that this is from 2012 um some of these states some of these really red states michigan ohio pennsylvania and uh virginia have been fixed or sort of fixed by state level reforms or court cases um so for instance michigan this year voted to implement an independent commission to redraw their district um ohio passed the constitutional amendment um pennsylvania had a court case which i'll talk about a little bit later but now with 2021's redistricting on the horizon we're trying to uh switch a little bit from uh showing you know retroactively where gerrymandering was bad but instead we're putting energy into a new open data project uh that we're excited to talk about but first i'm going to take kind of a meandering path before we get to the project so first i'm going to describe uh this bug in democracy with a bit of a primer on redistricting and gerrymandering and then talk about two broad ways to fight gerrymandering namely measuring it and also empowering citizens and by then hopefully i will have convinced you that precinct data is an essential missing piece here and i'll hand it over to my colleague hannah who will talk about the meat of the project so redistricting every 10 years the census is tasked with counting all of the people who reside in the country and that kicks off a whole as soon as they release the data it kicks off this whole process where lots of maps have to be redrawn to ensure that the number of people in each one is about equal meaning that each person vote has about equal weight there are lots of maps that get redrawn during the process from congressional maps all the way down to school board city council and so on um but since we mostly focus on partisan politics there are three really important maps the state legislators draw i'm going to show california's maps we've got the congressional map which has 53 districts we have the um state senate map and we have the state assembly map and the way these maps are drawn has a huge impact on the makeup of the body that gets elected through it so who draws these lines well it varies pretty significantly across states so in a handful of states there are independent commissions composed of citizens recruited through various processes and those states are shown in green here from most reformers perspective this is kind of the gold standard of redistricting processes uh arizona was a pioneer here arizona was the first to implement this sort of commission in 2000 um california followed suit in uh 2010 with proposition 20 did anybody vote for this nice cool good job and uh and michigan in colorado voted to create one uh last november but for the most part if you see all the gray maps are done by state legislators who usually just create and pass a map like any other bill and by the way some of these gray states only have one at large district like montana so they don't actually have to draw congressional maps but all states have to draw state legislative maps so who draws those well the same states that have independent commissions draw congressional lines also have been draw state legislative lines um so again those are in green um and there are also some sort of like for both of these there are some sort of in between models like commissions where politicians appoint the uh the commissioners to draw them um but in general you can see that for the most part districts are passed just carte blanche by by legislators and for state legislative lines it actually creates kind of a feedback loop that's hard to break where legislators draw a map to try to achieve a certain partisan outcome and then if it works then they have even more power to do it again the next go around um one thing about this is uh i i know from from hannah who's a californian that california is very famous for citizen initiatives and long ballots um but in many states besides california it's there are no there are no citizen initiatives so to implement redistricting reform you have to convince legislators to do it you have to can there has to be a lot of public pressure on legislators to to give up the pen and it's really hard to do so um it's it's a pretty big lift to get um really solid redistricting reform in a lot of states okay so what's gerrymandering well it has a pretty loose definition uh it generally refers to just some distasteful line drawing practice but our concern is primarily partisan gerrymandering which is pretty specific so let's say you have a um a partisan map making body what are some options available to them uh well the goal of of partisan gerrymandering is you just take you take these votes as a constant just where your voters are and then you draw the lines to try and get as many districts as possible to vote for your party by a small but reliably positive margin so i'm just going to take this arrangement of blue and red voters as are given and show you the variety of outcomes that you can get through uh map making so first gerrymander for the uh red party tries to pack the blue party into uh the blue voters into a single district that they win by a huge margin and then the remaining blue voters are divided among three districts that go reliably red a blue gerrymander works the same way you just artfully arrange the red voters in the suburbs into one district and try to get three more blue districts um the most common kind of gerrymandering is a bipartisan gerrymander where everybody in the state legislature says you know i don't think any of us really wants to have a competitive election right and then you you jointly pass a map where everyone gets to keep their seat and has pretty safe general election uh and then you can also um try to make a map that is competitive and has in this case has two districts that go 50 50 so a partisan gerrymander aims to to skew the distribution of vote shares so let's say that um you have a fair map you hold an election this is just a hypothetical um you have a fair map you hold an election so then you have a vote share in each district which is just a number between zero and a hundred percent democrat we're ignoring third parties for the most part it makes things a lot easier um and in a fair map you might have like a roughly linear distribution like this where you have some in the middle districts some you know some democrat districts some pretty strong republican districts um so and the the vertical position here indicates the percent of the democratic vote share so if it's above that 50 percent line that's a district that voted for a democrat um but if you look at what happened in ohio 2016 you see this like really curious distribution uh this is a map that was drawn by a unified republican party and you can see what the what the aim is here it makes four really really strong democratic um seats and then keeps all the other districts pretty reliably republican so what happened in 2018 so again this is 2016 in 2018 there was this this blue wave where democrats got a pretty significant increase in vote share pretty much across the country um so in ohio i think it was about a three or four percentage point shift towards democrats and you can see that um this gerrymander was designed really well um it was designed to hold up even in the case of of a blue wave um and so even though the democrats got more votes um they didn't actually convert any of that increase into an additional seat and so democrats won the same four of 16 seats in all the elections that were held um this decade since the redistricting in 2011 um i should say by the way it's that gerrymandering is a bipartisan offense um later this month there are going to be there's going to be a court case before the supreme court about gerrymandering in mariland where democrats tried to um convert one reliably republican district to a reliable democratic district so um bipartisan but the republicans had a lot more opportunity to do this in 2010 so how do we fight gerrymandering well the fight has been raging in courts for decades and a big part of the fight has focused on how to measure it the idea being that courts need a way to say that this map is worse than that one and so was there was this high-profile case against the wisconsin state assembly map last year called gilvy wittford used a metric that you might have read about called the efficiency gap the idea behind the efficiency gap is to measure whether one party wastes more votes than the other where a wasted vote is any vote that doesn't contribute to a win and to to compute a metric like that you don't need to know anything about geography all you need is a series of numbers so like in that last one i showed you it had 16 dots those 16 numbers are all you need to there's an equation that you plug them into um and you just get out a metric with the efficiency gap so in oral arguments for this case uh chief justice john roberts referred to these metrics as sociological gobbledygook so of course my colleagues who are not here uh wrote a paper just for him an antidote for gobbledygook um and it kind of what it tries to do is sort of summarize is when these different kinds of metrics come in handy and just tried to hold the chief justice's hand and say it's not it's not so bad you know these they come you know all these metrics do different things um so there's also a really great comparative study by greg warrington at the university of bremont that takes a more quantitative approach uh computes a bunch of metrics for a bunch of elections and it does various cross comparisons and tries to find the scenarios in which they agree or disagree and that's a really good paper and we also have a python package that you can use to apply about uh 15 of these metrics to any sets of votes that you have um to see if they seem indicative of gerrymandering so if you're interested in in doing this kind of thing you can run your own metrics um but there's some issues with uh with standard gerrymandering metrics like these um even though this is a great package of course um for one thing they might have a false positive problem so let's say you have a state where let's just imagine you have a state where urban voters vote for democrats like 100% and uh people in the suburbs vote kind of mixed democratic and republican um and let's say that that state also has a law saying that a district map should try not to split cities as much as possible now you might have what you're going to have is you're going to have um a situation where it looks packed right it looks like um you might have a metric that says oh the result here seems really packed it seems like democrats are really disadvantaged even though that math was just drawn according to the rules so to to really understand to really measure gerrymandering you need to compare partisan performance against some baseline and the way you get that baseline needs to take into account uh the rules that govern redistricting the state but also the um the geographic layout of voters basically like where are those blue and red dots that I showed in the in the previous one so um that's why in recent years we've seen the development of a number of algorithms to generate large ensembles of maps that are reasonable according to some rules so here you can see a map of virginia I think we started off with the real congressional map and this is an algorithm that just makes lots of random tweaks and if that tweak results in a map that's still like reasonably good like population equality is still good it keeps it and then it does it again so this uses um markup chain money carlos sampling um and if you do this millions of times and uh uh then you you have a you can get a big ensemble of lots of pretty reasonable maps and then if you have that you can then say okay all these reasonable maps if we plug in these election results how do these maps perform what kind of partisan makeup do they produce and then you can go back and compare that against what the original map is doing and you can see if that's some crazy outlier among the whole distribution of reasonable possibilities so this is kind of like the next generation of measuring the effects of gerrymandering um and this approach was actually used um effectively in the pennsylvania supreme court uh last year so a researcher at carnage millen named west pegden uh did something like this he started with the pennsylvania congressional map uh did lots of tiny changes and found that um the original map compared to this whole ensemble was in the top 0.00058 percentile of of uh partisan bias and then he also came up with a theorem showing like what are the odds that you would get a result like this from a reasonable map um and the court was actually convinced by this um which was like uh it was landmark ruling in gerrymandering and they actually they struck down the map as an unconstitutional partisan gerrymander ordered the implementation of a new fairer map which was used in last year's elections so this was this was a really big triumph for all the the gerrymandering nerds um and a modified version of this approach is also going to be used in the upcoming case about north carolina's map this is um a slightly different sampling technique uh this plot is produced by a group at duke led by a professor john mattingly um this histogram shows you they constructed some gerrymandering index this histogram shows you all the reasonable maps um the judges map is some fair map that a panel of judges came up with you see it falls right in that distribution and then the 2016 and 2012 map that were actually used in north carolina are way out there so um this was actually used in the district court to strike down the north carolina map which is ruled as unconstitutional but there was no time to replace it so an already ruled unconstitutional map was used in november's elections actually um but this is actually coming back before the supreme court as well this month along with the maryland case and if this goes well at the supreme court then you're probably going to be hearing a lot more about this technique in the future you can also do some of this sampling yourself thanks to an open source project released by the metric geometry and gerrymandering group which is a group uh led by let it it's at tough led by a tough professor named moon duchan um and this is a really cool program and so so that's kind of the latest and greatest tech for measuring gerrymandering and uh depending on how things go at the supreme court you might see more of this um so in addition to using metrics and measurements to fight gerrymandering in court we can also try to fight gerrymandering by giving citizens the tools that they need to have a say in the redistricting process which is going to happen again in 2021 so what's going to happen in 2021 well in some states like california there's going to be this uh there's a commission and there's a lot of public input and there's a whole process about that and periods of public comment and that's great uh you guys are very lucky congrats um but in most states it's going to like state legislatures are going to craft these maps mostly uh mostly in private using expensive software and well trained consultants to try to achieve a particular objective um in many cases these maps are going to be released not long before they're scheduled to be voted on so how are voters or how are journalists uh supposed to understand the likely effects of these maps well since 2011 there's been a huge increase working of in people working on software to try to address this issue and one is this website called plan score which is really cool it allows you to upload a map file so if you go to so they have this page where you can upload a shape file that you might get from the legislature or you can click on a pre-existing plan and it scores the plan for you and it says okay how bad are these metrics according to this this hypothetical map and you can you can browse through it you can see for any hypothetical district you know at least achieve population equality like how is it predicted to vote and this is a this is a really great tool um and it'll allow people to have a much more rapid response to a map than they would have uh 10 years ago hopefully um another way that citizens can be involved um is by being able to draw um maps of their own so um there are a few pieces of software to do this including district 3 which is by the same Tufts group that I mentioned before and it has a really fantastic interface for drawing your own district and this way that the public could have could be able to say to their commission if they have one or to their legislators I don't really think that's how my community should be represented I don't think that's a logical district I have this other map that I think is a lot better and this is really cool you can you can go to this website you can and it's just like painting so you pick we're going to be painting this district here you can change the brush size and such as you're you're drawing you can see how your population equality goals are changing in real time um and you can also see partisan performance here so here we're we're looking at the 2016 presidential election how these districts be likely to vote um and so you can see if we're you know we make this district one as it edges into Philly um or now we're drawing district two as we edge into Philly we just picked up a lot more Democrats so you can see it go blue this is a really cool um this is a really cool interface for this there's also some other good software like Dave's redistricting app um is a pretty popular one and in the past various organizations have held even held contests to draw maps and raise public awareness so you can offer prizes for different objectives like a prize for the map that best achieves partisan fairness or best represents some community um and that's that's a really cool thing to do but um all these things that I talked about uh all these avenues for compacting gerrymandering have one thing in common they are thirsty for data they really need data um so uh and because they all need uh geographic election data um you'd think that there would be like a really solid resource for this some kind of national resource for this but you'd be wrong it's really really hard to get this data and that's what this project is trying to address and Hannah's going to talk more about uh about the meat of this right now all right so everyone else has been talking about uh things to do with the open data but nobody has talked about how hard well everyone has mentioned how hard it is to get their hands on this open data but we're really going to dive into that so um first uh I want to transition into talking about well eventually I want to transition to talking about this tool that we're building because we really want feedback from everyone in this room but I think we need a little bit more background I'm sorry for so much background in this talk um but first of all we need to talk about what is a precinct so a precinct is the smallest unit of geography that votes are reported in in general there's a school or a church or something that's designated as a polling place and everyone in a certain geographic region votes at that same polling place in California you know it it gets a little more complicated especially here in LA but uh around the rest of the country that's generally what happens um so when we say precinct level data that's what we're talking about uh everything that Will just described runs on this precinct level data we're referring to two pieces of a big data set we need the boundaries on the map and we also need the votes that go inside them because without the boundaries we don't have any way to place the votes on a map so a big nice open source uh precinct level data set for a state isn't really that useful without the the map so elections are generally administered by counties and we usually have to contact the county officials to get this map info and the US has more than 3000 counties or county equivalents so that's a lot of raw files that we have to sort through of course we don't have to do it for every single county in every state some states like Minnesota publish the data after each election but that really isn't the case for more states and like Will said there's just no national database for this um some counties publish it on their websites some don't even do that so it's really uh all over the place uh there's a few entities that have stepped in here to fill the void I put up here a picture of a New York Times feature of precinct level election results for the 2016 presidential election and I'm just curious how many of you guys had seen this data set I think there was a similar one in the LA Times how many of you guys have already played around with it okay that's a pretty pretty uh good size audience um and with with data sets like this you can really see how you could powerfully gerrymander when you have access to this data um but the thing is this data it's not really it's the most comprehensive and user friendly data set out there so far but it's not accurate enough for our purposes so for example this is an image that Will made where the New York Times map of Manassas Park City Virginia is on the left and the physical map that we got from the county which was originally colored in crayon by the way that's that's what I'm talking about um so these discrepancies are just too big to ignore we can't be saying that voters live somewhere where they don't um so so the other data sets they're great they're useful for people to get a feel for the political geography of where they live but it's not something that you could use in court um but there is a lot of energy out there to do this there's currently multiple groups working on this um Will was mentioning uh the group MGG and Boss MGG in Boston I was part of a summer program called Voting Rights Data Institute that they had this last summer and we collected and compiled Ohio so that's what this top picture is this is what we had to do to get that state done we estimate it took about 400 man hours and we had to have in-person trainings and then we had to have in-person work sessions where we had some of us walking around to help we had to buy lots of pizza and most importantly we had to have air conditioning when the dorms didn't have air conditioning so that's how we got everyone in the same room to actually do this work um we've also been in touch with an individual in Kentucky who's just started taking on this project for that state there's a team um at the University of Florida that works on this there's a website called the Harvard Dataverse with lots of historical data and there's a great collection of files called the NV Kelso election geodata repo that's this bottom picture and it's really been the the main source for this information so far but sometimes it has very little documentation it can often be really messy and there are also just big gaps in this data set there are states where we don't have anything so it might be surprising to many of you that this data set is actually in such a terrible state but anyone who's worked in local government probably understands why this is so difficult um and because like a local election administration varies so much from state to state and even from county to county within the same state we know that we aren't going to be able to do this alone especially since so many other groups have been trying to do this and we still aren't where we want to be nationally so we want to build a tool that's going to try to coordinate all of these grassroots efforts that are popping up all over the place and we want to lower the bar of entry to actually contribute to the project because right now you have to be fairly technical we had to have these in-person trainings and we had to have someone walking around to answer questions so we we want to come up with something that's going to make this all easier and one of the the hardest parts of this this project is just getting our hands on the precinct maps in the first place there's no consistency for the the precinct maps across counties in one state and then when you're going across multiple states sometimes they don't even call them the same thing anymore um so we've received broken shape files from county clerks hand-colored maps like the manassas park one we saw um once i got a serial killer style map that was made of like 50 pieces of paper cut into funny shapes and they were all taped together um and once in for virgini we even had a county clerk say his old boss just walked around the county a long time ago and wrote down which addresses went into which precincts and that no one really knew where those boundaries were and yes this is actually coming from a county clerk um so sometimes the the county makes you send money for the maps or sometimes you even have to go get them in person we had this fun with highland county in virginia where our legal analyst ben who's not with us today um he had to drive over an hour from Harrisonburg which was already out of his way uh down this windy mountain road to go to an unmarked double-eyed trailer where they had a single copy of the map on the wall for him to take a picture of with his cell phone and i think this is like the original data scraping now this is just like the outlier this isn't going to be every county but i i feel like in every state there are going to be a couple of counties like this you can see where it is way up in in virginia know very rural they didn't even have cell phone service in the trailer for someone to take a picture with their phone and text it to us so we had to go um so collection that's the collection part that's just the first step once we've collected county contact info we've contacted the contacted them all and then gone through whatever hoops we needed to to do to get a hold of that raw map then we digitize clean and correct these maps and then we match them up with the election results so we can they can be visualized like in that other map and we have a github repo full of all the python scripts that were developed by two undergrad researchers last summer one was jake jake of watchpress and he's at princeton and another was connor moffat from right here at caltech so he spent all summer at princeton working on this and those tools will tackle most of the the tasks you might encounter when trying to clean this data um but as i was describing the form of the raw data you receive can vary quite a bit and that's going to change what you're going to have to do there and in reality you're also collecting election returns at the same time to determine what precinct names you need to match up with and sometimes this requires collecting those election results directly from the county officials but there's actually an entirely separate project called open elections to handle that so don't worry don't let that scare you but um you also have to process the the maps differently depending on what type of raw file you get and it can even change with the quality of raw file um and then we might want to include historical elections and i'm just gonna stop talking before the font gets too small but i hope you guys all get the point that the it seems like a simple process but it's really really complicated and these middle steps get really messy um but but groups are tackling it uh and and they're doing it in different states they're tackling different aspects of this uh but we really need something to bring it all together and make it easier so we need a platform that's going to organize it all and it needs to be something that promotes cooperation across groups and teamwork within a state and it also needs to coordinate every step of the process uh which is going to be different depending on the the source of your data it has to have very high standards for accuracy because we can't have any uh errors or a case in the supreme court might be affected and we need to have it consisting of the most up-to-date information we need we need to be collecting the most accurate precincts because precincts can change between every single election and currently there isn't anything that checks all of these boxes so we started development on open precincts in collaboration with many organizations who have already been doing this work like open elections which I just mentioned plan score which you saw um will show an image of the florida election science team at the university of florida does this work and we've we've also been developing it with individuals like nithaniel calso from the calso repo i showed and the project leader in kentucky and and a ton more that i can't even think of right now so we're trying to build open precincts as a custom web app um where we'll we'll bring all of these together lower the bar of entry so more people get involved and hopefully fill this void for good um so it's written in jango and it helps organize and coordinate the process for an entire state and it also has to manage it on a higher level across all 52 jurisdictions um it naturally naturally breaks into three main sections you have the collection of the raw data which is really tough you have that cleaning and matching of the raw data which again was really messy and complicated and then you have the display of the final product which is not that bad and and that's what a lot of the the other softwares that you've seen so far are doing but they just don't have the data yet to display it um so here's the links to the github where the jango application lives and the website where you can get a feel for the tool it's in it's very early stages right now but this is where we need your help we want everyone to look at look at this and and give feedback so let's take a look at the website all right so if you wanted to to volunteer you're gonna go ahead and sign up here and I have to try to type backwards or I can log in right now so you would go ahead and sign up for a new account it sends you an email and then oh do I have to one step so I forgot I wouldn't be on my laptop you guys can all take this as an opportunity to make your own account if you'd like oh yeah we can answer a question while I'm yeah so we're we're technically employed by the neuroscience department which is a bit weird I'm actually amateuring neurosciences but that's just kind of a coincidence um but yeah we're we're um funded by uh foundations and private donors but we just happen to be in the neuroscience department so I got hired um for the job because some of the computational stuff that I was doing for my neuroscience degree is actually the same sampling technique as is used to to do that kind of the math sampling thing but our team is actually very diverse we have um go ahead uh so yeah we are pretty diverse we've got um uh James here is a software engineer Hannah's from math we've got um we've heard to Ben before he's a lawyer um yeah a lot of skills okay I got the website up so it's just hard for us to do because like I said it takes 400 man hours to get a state kind of done and we need to have perfect data in order to have a have an impact in like a supreme court case but in order to gerrymander you can see how you could have done that with that new york times map pretty well you just need a basic idea of of how people are voting so they don't need to go and get perfect data and they also have the money to to pay someone to to put in those 400 man hours yeah but also the people who draw the maps are usually in government and if they have more levers uh to work to get the data that they need so actually some data that has just uh some data that you get just by kind of like working your connections and like talking to someone who's in government but it's easier to do that if you are a legislator there's already so much money that is being put into gerrymandering that like I don't think that this is gonna I don't think that this is going to aid people aid the people who would otherwise be gerrymandering I think they already have the tools that they need or have access let's get to questions later at the end oh yeah so let me quickly go through the website now that I have it up so let's pretend that you all want to help out here you just signed up for an account and now you're in open precincts for the first time you can get a little general idea of the different steps or you can go up here to the about and read about the project in general but the the fun thing is to go to the state view and see the state of states right now across the country um and say you wanted to like help out on something like california you'd click on it to get a more detailed view and for most of those states all of those gray ones it's going to tell you something like this where we want you to get in contact with us before you to initiate the process because in states like california you aren't going to have to go to every single county some counties already have it available online and some counties have already compiled it for region so um in every state there's going to be a little bit of a different uh path to action so to start a brand new state you contact us and we will figure out what that path is um but if you click on a state like colorado that already is in progress you'll see the steps that it's currently uh working on right now so right now we're collecting county officials for colorado uh let's say this is the state you wanted to help out with you've already um contacted whoever's in charge of colorado click on on the county you you live in or you're interested in or you have connections to and now you can add officials um to our database directly uh you can work with us to contact them and and try to get back those raw files and then the log will come up right here and when you start getting the raw files in you can upload them right here um directly to our database and sometimes they're available on the county website um which means you could download it and upload it right here for us um so that is a state in that process in that stage of the process let's see but once this is done for a whole state it's going to look like pennsylvania where we're currently cleaning and that's where my diagram sort of got really complicated and the linearity just breaks down so what i'm going to do is show you a clip of the general process where did the slide go is that oh no it's not we were too ambitious sorry okay sorry about that okay this is what we're looking for so this is uh what you would generally do if you need to what we call digitize a map if you get an image of a map um that that isn't already a shape file go ahead and and press play so it's gonna take take your map as input and it's going to ask you what color are the boundaries that's what it's doing right here and then you're gonna you're gonna tell it the color and then it's going to take every area that's surrounded by that color and fill it in with a distinct color um that's what it's doing right now and then it's going to take every every region that's a distinct color triangulate it because we know that for math that polygons can always be triangulated and we know how to randomly uh sample from a triangle so we randomly sample pixel color we take the census block file with underneath which is the smallest unit of census geography that exists we assign every census block to a precinct based on the pixel value above it um and then you can see that you're able to make a precinct file directly from the the image above so this wasn't a shape file before this is a brand new shape file and before this existed what people would do is open up a GIS program and trace the boundaries and that's why it took us 400 man hours because we we had to not only do you have to trace the boundaries you first have to georeference that image um which means giving that image coordinates on a map so there's a lot of manual work that this is um getting rid of so uh once we've done this for every single yeah once we've once we've done that for every single um county in the whole entire state it means we have a shape file for every county in the state but we don't actually have names on those uh those shapes we have to go to the election results pull out election names match them correctly because you really can't have precincts mismatched or you are incorrectly representing those people um so you'd think that a dataset that you got from the local government might match with a shape file that you got from the same county but really there's typos uh or just name changes uh it's a really big mess so that's another another thing that takes a long time but once you've done all of that um you're gonna end up with a state in the final stage and we've only successfully gotten there once so far but that's Virginia we're really proud of Virginia um so at the end you don't need to focus on all of that behind the scenes stuff you can really just focus on the data which was which is what we cared about from the beginning um so we'll have we want everyone to be able to download this with this data we want it to be ported into all the other softwares um but this is just a fun little way for you to get a feel for your state you can turn counties on if you want to orient yourself a little bit and then you can zoom in and and see what's pretty good granularity um how people vote uh and you can move your mouse over different precincts and see exactly what per percentages uh the election went for that area and and yeah this is pretty pretty powerful um and and very accurate compared to stuff that already exists out there and eventually when we have multiple election years you'll be able to select which election years you want to look at um all right so that's the final product and like I said that's what we want every single state to look like and we need help getting there it's going to be a really long process so um if you feel like this project would be something that you'd like to contribute to or if you want to spread the word if you think you know people who would would be all about something like this that's exactly the stage we're in right now uh we we need everything from folks to gather that raw data um to those with more technical knowledge who might work with us to coordinate volunteers for a state or even just a group of counties for a state um we also we need a really diverse skill set here where we need people who have experience in GIS or also user experience and interface or even just requesting government information I heard the word FOIA being thrown around earlier we need people who know how to do FOIAs because you have to do that a lot here um and we also need really really technical help as we make changes to the web app and that's why we brought James up with us he's our open precinct developer um so if you want to talk to us more specifically about that or ask questions about that he's he's gonna be your guy there um so yeah that I think I'll end there because we went through a lot today I hope that everyone knows more about redistricting and the role that data has there uh than you did an hour ago and if you don't if you already knew all of this then you really need to stop and talk stay after and talk with us um so with that yeah I'll open it up to some general questions and then Will and James and I were happy to stay here for as long as we need to have more specific discussions and you can see our our specialties here um but yeah thank you so much for listening uh and coming and we'd love to hear any comments you have about this are the uh are the models that you you were showing of the maps taken into account demographic changes what models were you referring to like the maps that that are being suggested like the the samplers yes uh yeah so that's a good question there are um there's not a lot of like federal law as it relates to how districts need to be but uh there is in terms of the racial composition of districts so um the voting rights act and a lot of the cases that have flowed from the voting rights act um are very important for considering when drawing districts and it's very hard to do because usually whether um whether a map satisfies the voting rights act is a pretty long drawn out court process and has various things that I think the kinds of people at this conference would think are pretty subjective um so uh but there are ways that these samplers deal with that so one is just to say um is just to freeze certain districts so say this is a district that was required to give a minority community the opportunity to elect a candidate of their choice and then just say we're not going to mess with that we're just going to freeze it it's already been approved by a court let's just not touch that one so that's that's one method for these samplers you're so you're talking about Ken if you have a map how how easy is it to forecast how yeah so that's kind of it usually when you when you're trying to figure out how a map is going to vote you just use some past election and like all the forecasting and the predicting is about the map and you just hold the elections constant election prediction is just a different ball game entirely so the question is do we use census data yeah the the whole reason that our digitization process is is going from a random map down to census blocks is because census blocks are the only way that you are going to know um the population of these precincts and when you're when you're drawing if we're telling people that they should use this data to draw their own map you need to be able to apply population to it because these maps need to be within like really close in population so that's that's how we use it there and also in in some of the tools like like district or um you can you can see a breakdown of how people vote in every district but they also show a breakdown of the demographics in each district which which becomes useful yeah it really is and that's why we need something that's going to be able to maintain always collecting the the most accurate precincts because it really depends where you are there are some rural counties in virginia where i'm sure the precincts are the same as they were a really long time ago but in LA the precincts change every single election so there's there's not even a map that that will be the same if there were two elections in the same in in a few years of each other um but generally what we see is a map like uh the state of Ohio will have 10 of the precincts across the the state changing from election to election so so yeah you have to this is one of the open questions that we're trying to think about how are we going to handle this are we going to uh make sure we have different data sets for every year we're going to try to make sure that our most up to date data set somehow encompasses all of the others very very interesting uh what you were just saying kind of confirms my my my question in italy we don't have a general mandarin but every election basically the majority or the majority the leadership just redraws the the electoral law and basically achieves the same principle the same the same objective is to maintain their power at the next cycle even though they're losing it right um so what are what do you think is the end game for this for this project is this um like reducing gerrymandering freezing having more objective uh laws rules inside the laws yeah so um we're pretty involved in redistricting reform so we're pretty involved in um doing what we can to change laws um around the country to implement independent commissions like california so i mean that's really like the main there's that uh and there's lawsuits um but right now it's time to think about 2021's redistricting so that's kind of like right on the horizon so we're kind of focusing towards the tools that um can be used there uh which i mean that's what that's what we're talking about today but i mean really in the in the long run it's you know just getting just getting able to um get citizens to be involved in the the redistricting process is really great um but you really do want to change the laws and there's definitely there's been some really good like this is an issue that's on americans radars now this is on the ballot in four states in 2018 and past in all of them so hopefully things are changing but um yeah uh yeah i mean we yeah we want maps to be fair and and we want the process behind the maps uh map making to be fair yeah so the l.a county clerk is going to be talking on saturday i've heard a preview for anybody who's interested at three o'clock they're going to be talking about their district in plan uh their district in plan for the next election cycle is to have no district so free things sorry thank you so you'll be able to vote in l.a county anywhere you won't have a designated voting place you can vote anywhere and it'll be over the course of 11 days so they're going to be talking about the open source software that they're going to be they're releasing that software as part of this conference i mean so why don't you come to their talk on saturday now it's actually where i was going with this is how do you see that affecting your work as as their things progress yes so we're really interested to talk with them too and and see the the thought process behind that um but i i sort of see two two possibilities here one is if we stop um reporting the the data at such a granular level that means that the people who are in power also don't have access to it so um in that case it might be something that uh no longer has to be a project if all of l.a county is just reported like at the county level you're going to have to make districting decisions in other ways um but another thing that i've heard a lot of people talk about is uh figuring out a way to to make sure that our our votes the ballot is like linked to your precinct and i think that this is what's happening in places like oregon that are moving to completely um mail-in ballots you have some counties that have the ability to add the technology and vicki might know a little bit more about this no the the the precinct i don't think i mean the problem with with i mean it's like you could um uh the question is just like where do you want to anonymize the data right like if we if we have um you know i'm curious to see this talk because if the data is uh collected at a really granular level it could be reported at a really granular level uh but then you run into anonymization issues and that that's the same for if you're reporting if you were to start reporting results by census block lots of census blocks have one person in them i was curious about the technology that was used for the mapping to store the data and then display it i was wondering if you could say anything about that on the on the open precinct's website um so behind the scenes we're going to be using post gis um right now most of the data is just stored in shapefiles um this work has been done pretty incredibly by the team so far in google spreadsheets um so this is like the next generation tool um so we're getting things in the post gis and the map um will actually wrote the map is going to be using map box um pretty pretty standard open source stack for all that so you speak pretty glowingly about independent commissions or states that happen how well does that correlate with improved outcomes on the gerrymander incident uh very well in general so it's um well i shouldn't i shouldn't jump straight to that it's there hasn't been a lot of data so um arizona had an independent commission in 2000 um and uh california had one and then arizona and california had it in the 2010 cycle um and that's it so we have basically three instances of independent commissions at the congressional level anyway um and we should have some more data soon because michigan and colorado will be joining the ranks for the 2021 redistricting but in general people are pretty um people are pretty uh people who study this think that california california's commission did a really good job of changing the process which was um i mean independent commissions are a messy process but what was happening before in california i there are probably people in the room who know more than i do but there's a lot of um that i think that the the district map before the that was used in the aughts was generally considered a bipartisan gerrymander not a very at the congressional map anyway not very competitive um it was a and i think that the uh the california commission definitely people think that it made elections much more competitive even though the law i think does not say anything about competitiveness it doesn't value competitiveness at all um and they can't look at partisan data actually but they still got a pretty nice competitive map where if you were to plot the um the districts like i did on that dot plot they are kind of like more or less linearly arranged doesn't seem like there's any evidence of of any shenanigans not to be completely paranoid but do you anticipate bad actors perhaps um employees of the consulting firms who are doing this currently for uh the people in power uh creating accounts and trying to flood out your good data with garbage data um not necessarily that particularly but um you know we are consuming i mean there's a crowd sourcing element to this and so there's a there's a bit of protection that goes into it um there's kind of one of the reasons that we were switching to the model we're switching to to open this up nationally is that we need more granular permissions and so we'll be able to much more closely track what individuals are doing and if someone someone is a bad actor or people aren't always bad actors i found in crowdsourcing projects but often they're kind of um you know reckless um and don't really realize that they're making lots of errors and so we'll have a lot more of an ability to like roll back changes on the user by user basis or not grant them permission in the first place but it's um it's definitely something to worry about because um something that independent commissions face a lot when they you know they have independent commissions often um the bills that create them also create like massive opportunities for public comment and uh i think in every instance of an independent commission there's been uh people who say who they're not um people who are part of our community but it's it has a ill intent for sure and rooting those out is difficult for these commissions as a person who lives in one of those red states north carolina that's been very gerrymandered uh i just want to thank you for this work so appreciate it you can tell you're very passionate about it um and thank you thanks are there metrics that can tell you that um precinct is drawn to represent the district on the metrics that can tell you that precinct is drawn to i don't know specifically there are metrics for that but there is a concept called communities of interest um and one of the things that we're also looking at is a tool to let people kind of draw their community of interest which is that can really be a very generally defined term and i think figuring out exactly how those communities are just split or not split might be in that direction did you mean precincts or districts okay yeah seems answer applies it's a fascinating thing does people register to vote and that's the only way you know who they are and i noticed i noticed that estonia at psychon last year they were talking about this a lot that estonia has established a national id for people and it's really great because you have a you have an id and you can vote as many times as you want only the last time counts but have have you run into uh what kind of issues have you run into is identifying the actual people that are registered in actual address is that actually collected so we do we do a little bit of work um with what's called voter role or voter record which is a record for the an entire state that has every registered voter um some other voter information and then what precincts they're designated to um so we use this not at all we we don't actually want to know anything about the individual because all we care about is the individuals within a specific precinct so what we figured out to do with this data it's it's definitely not good enough to just um make a map out of it but if you if you can take every person who's registered as a voter assign them to their precinct and then put that on a map you can sort of build a decently good map and we've been using that to just verify that our our naming is correct and that uh no no boundaries have been split no precincts have been split or merged because when I was talking about precincts changing 90 percent of the time it's two precincts being added together because the lines were too short at the polling place or one precinct being split in half because the lines were too long so um I I've I've lost my train of thought now uh can you did I answer your question am I missing something I I lost my train of thought yes you did but you also brought up another question is is as you know all the news today quite with reasonable reason to be thinking about this all the number of people that live in a community or a precinct that don't vote because they don't know to vote or whatever reason doesn't matter doesn't matter how they go it's just they don't get counted and how do you approach that I'm remembering um what my train of thought was first let me finish that real quick uh yeah the only way that we use voter registration information the only way that we use individual information about an individual is just to make sure that we basically have the naming correct um yeah I mean if people who who if people don't vote then um they're that that affects the data but they're probably not going to vote in future elections either so there are like other projects um with the with the main goal of of people who are eligible to vote who don't vote because they're not on the voting roles because for whatever reason they were told they can't vote whatever reason they didn't know to vote you know but there's always like a household they have four adults in a household four people that could vote maybe only one of them vote so that one person would be on a registration somewhere a voting registration the other three might not so how is there any any effort to try to find out the people that are eligible to vote that didn't get on the voting so the thing with the the thing about that that makes it difficult is uh we have an anonymous vote so you don't actually know you know how many uh eligible voters you know how many registered voters in a precinct actually went and and cast their vote and that's called the the turnout but it's it's sort of a whole there are so many different uh ways that you can attack the problem of like uh unfair democracy and that's just like a whole another project of making sure that people are educated um and and knowing where to vote making sure that they know that they are accurately um registered and if someone tells them that they aren't they need to fight it a little bit um but i think that getting people invested in redistricting uh getting them to talk to to their community making sure that everyone they know who's eligible is registered and then maybe talking about uh the election after voting and and hearing if someone was told they they weren't eligible uh and making sure that they go and fight that a little bit i think just community involvement and public education in general which this data does promote because everyone wants to look at the map and try to find their house and see what's going on around there it's just gonna uh bolster people's uh involvement in democracy in general thank you very much let's give them another round of applause and thank you all for coming out to the open government track we've got a bunch of other open data and some government related talks going on saturday and sunday as part of the open data track so definitely take a look at that oh and then uh also if you're interested in these like civic technology type projects hack for allay is the local code for america brigade we're a volunteer group that works on these kinds of things in our spare time as well uh we have a booth in the expo hall as part of the open source dot com table so come check us out there uh and yeah thank you all for coming