the MIT Communications Forum and a professor of literature at MIT. I'm delighted to welcome you all to today's forum. Let me begin by announcing very briefly our two final events for this spring term. We've set, I think, a very high standard this term. And I'm very proud of the forums we've managed to put together. On Thursday, March 21st, our forum will focus on massive online courses and their implications. One of our speakers is Anant Agarwal, who's the president of the edX program centered at MIT in Harvard. A second speaker is the co-founder of Coursera, Professor Daphne Kohler of Stanford. And those two panelists will be joined by a professor of English who has recently been appointed the president of Lafayette College. And she has a peculiarly humanistic view, even a skeptical view of the power of these massive online courses. What I'm expecting is a wonderfully rigorous discussion of a central educational question. And then on Thursday, April 11th, we will have, I hope, a very powerful and interesting debate about the moral quality of our press campaigns. And our title is something like News or Entertainment, the Press in Modern Political Campaigns. And one of our speakers is our MIT colleague, Tanahisi Coates, who writes for The Atlantic. And I'm looking forward to that event as well. It remains for me now as briefly as I can to introduce our event for tonight. But let me begin by saying that Nate Silver's appearance in this conversation has him join what I think of as an especially distinguished subset of visitors to the MIT Communication Forum. And they include such figures as Stephen Greenblatt of Harvard and his Harvard colleague Stephen Pinker, the poet laureate Robert Pinsky. So Nate is in very distinguished company, although he knows a lot more about a good many things that many of us are interested in than any of those three speakers. But I'm very happy to call attention to that subset of forum activities because we're particularly proud of them. And I want to also add that tonight's moderator also joins a distinguished group of moderators who include, among others, my wonderful past colleague, Henry Jenkins, and the legendary MIT linguist and empresario, S.J. Kaiser. So I'm very happy that Seth is joining this distinguished group. And my hope is that Seth will continue to play a significant role in future forums. But our featured speaker tonight, Nate Silver, surely because of the size of our audience doesn't really need an introduction. It occurred to me as I was thinking about his career that it's an interesting cautionary tale for MIT students. There's something profoundly entrepreneurial and deeply intelligent about the way Nate Silver has made his career. And I'm sure many of you know about his progress from moving from a bachelor's degree in economics from the University of Chicago to working as a blogger and a pollster to creating his own blog, the 538 blog, then to have been, in a sense, hired by the New York Times to become, which have proprietary control of that remarkable blog that had such a powerful influence on the recent election and predicted it so accurately. Well, that remarkable career is a kind of model in a way that I know that many MIT students think about a young person who uses the qualities and skills that he learns in university to be a kind of entrepreneur. And I think one of the things that occurred to me is that many MIT students might consider, given the nature of Nate's career, to have taken their statistics courses and their economics courses and then to have spent the rest of their time at Fenway Park. But that may not be the most appropriate lesson to have learned. In any case, we're very happy to have Nate Silver with us. And I turn the podium over to our two speakers. First, thank you all for coming. And I hope we will keep it lively enough so that the folks standing on either side don't get too frustrated. The hashtag for the event is MIT Nate Silver. I'll be monitoring that. And we might get some questions from the Twitterverse as we go on. Some comments so far are nerds nerding out, standing room only. But I think the format is going to be Nate and I are going to talk for about 45 minutes to an hour. And then we'll open it up for questions. His new book is On Sale Outside, which means that the people who are in the overflow will get first dibs at buying that. And he'll stick around for a little while afterwards to sign books. So I wanted to start just going back to the beginning of your career. After you graduated from school, you're obviously best known for your political work and your baseball work. That isn't what you started out doing. No, I should say it's a real honor to be here. We are kind of at ground zero for nerds, I think, here in Cambridge. But it's a real pleasure to be here. But no, I started out in 2000 and the economy was good back then, kind of last year. The economy was good for a while. And I took a consulting job at KPMG where it was built as something where I do some public sector consulting and some private sector stuff. But I wound up spending most of my time doing something called transfer pricing consulting, which concerns tax regulations that govern how much income a multinational company can get out of its subsidiaries. You're like, well, is motorless factory in Singapore? It's really boring, right? So it's supposed to sound boring. Are they paying enough to the US government for the cell phone parts that are made in Singapore, et cetera, et cetera? And so that was pretty dreadful. I think too many kids probably do take a job as a consultant. But I haven't really thought very much about when I wanted to do, despite going to U of C, the place where fun goes to die. I tried to have fun in college. And my degree was in economics. That's not MIT, apparently. Not MIT, yeah. I took a year in London. So I think it's tough for someone who's 21 or 22 and necessarily know exactly what course they want to take in life. And I'm suspicious, by the way, if there are 21-year-olds who know exactly what they're going to do. So it took me a few years to realize that I was been made pretty miserable by a job that wasn't very challenging and started kind of rebelling against it in different ways, in which meant having more fun with my friends in Chicago. But also, I started to play poker more often and develop this forecasting system, which eventually became Pekota for baseball players, which I sold to baseball prospectus. And so you, well, first, why don't you just briefly explain what Pekota is and why it was seen as so revolutionary at the time? So the notion behind, well, there are a couple of things that made Pekota different, one of which is that when you had seen projections for baseball players would say, well, Dustin Padroia is going to hit 303 next year with nine home runs and 42 doubles. Only nine. Yeah. Well, I don't know, right? But some exact sequence of numbers, and it felt to me like what you should really have is a range of outcomes. And where anyone who's watched baseball knows that, yeah, it's predictable to a certain degree, but especially for a younger player, you could have a guy who goes on to be the next, a young player, basically be the next Mike Schmitter, the next Kevin Orry kind of cubs disaster, basically, even though they have the same profile, the same statistics up to a certain age. So to capture the range of forecast was one of the innovations. The other one was to do it all by means of comparable players. So in baseball, you have, it's the world's best data set, basically. You have a very, very rich database. For every player, you can find dozens of players in the past who were similar to him. And so that's the way we mapped out different possibilities for his career trajectory. And so it also made it more fun and more tangible. It kind of shows you the work a little bit more. That's one thing I try and do in the forecasting products. So I put out, those things have remained true. Where number one, things are framed in terms of probabilities and uncertainties. And number two is you do show some of the intermediate steps. And we do get criticism. We're definitely not, five, three, eight, at an academic level of disclosure. But it does show people the work a little bit, as far as how do you get there. And I think that has more value than a kind of black box approach. The journey is often more interesting than the result, I think. And so you developed PCOTI, you sold it to baseball prospectus, which is probably the premier Saber metric site online. Saber metrics for people who aren't familiar with the term just really refers to using statistics in a fairly sophisticated way to analyze baseball. And then you moved over to baseball prospectus yourself. So did your parents or your friends or anyone think that you were insane for leaving what I assume was a good pain consulting job? Well, see, most of my income came from playing poker at that time. So there was in 2003 a guy named Chris Moneymaker, one of the world series of poker. And he actually was an accountant with a job, not totally unlike the one I had, right? But he basically is an okay poker player who got really lucky, right? Of course, when you have this turn into a television broadcast and every decision he's making is made to look as though it's genius and perceptive and the bad calls he makes are shown whereas, or excuse the ones that turn out to be right, right? We make a decision where you're wrong. But if you made the right decision to be shown on TV we get it out if you made a dumb choice instead. Anything about poker on TV you actually see because of the whole cam what cards someone has. And so poker is really easy to play if you know what family the person has, right? So people will watch this on ESPN and get the sense like, oh, I can tell that guy is bluffing. Well, yes, because you know he has seven deuce offsuit because it's showing. So a lot of people thought they were better at poker than they were and so particularly online it was possible to make pretty good money playing poker for a couple years. And so that enabled me to leave my job and it was like I would stay up all night and make 500 bucks or something playing online poker to seven in the morning and catch a cab to work and kind of sleep and at my desk, right? And it's like, I don't care anymore, right? Kind of we're hacking the system a little bit. I wish I'd play more poker actually because now the games have gotten very tough and you can't really play online. But those things went together where it was less of a financial leap than it might seem because I had two different ways to make income. And so you can't play poker online anymore now because the sort of naive players who were over assuming their skills have been weeded out? Well, there are two things. Number one is the U.S. government passed a law in 2006, I think it was, that whether technically banned online poker or not is debatable but it made it very hard to get money into the sites and so what happened is that once you had the bad players, the fish we call them, bust out there was no more ready supply of them. So you kind of had the whole ecology, the whole kind of ecosystem collapsed basically where the predators became the prey.  Yeah, yeah, yeah. Although now I should say Nevada and New Jersey, this law did permit states and as well as subsequent interpretation by the Department of Justice did permit states to run their own online gambling sites. So online poker in Nevada and New Jersey is going up later this year. Other states I'm sure will follow. I'm not opposed to gambling. What you have seen in general context where gambling is legal and one jurisdiction then it becomes legal and neighboring jurisdictions because you're otherwise just giving up money basically. So you probably will see a revival of online poker and the play has become a lot more sophisticated but if you have a lot of new and naive players then hey look, if you're here at MIT you probably have a lot of good gainful ways to employ yourself but to watch out, if there is another poker boom in 2014 or 2015 then it might not be a bad hobby on the side. So you brought PCOTA to baseball prospectus and then actually went and joined the site and were there from when to when roughly? So from roughly end of 2003 to 2008 or so. And when you were at baseball prospectus were you primarily writing or doing, developing more analytical tools? So it was combination, so I was also helping to manage the business and so I was doing like three or four things at once which is I guess something that I've gotten used to doing. But it is tricky, it was a small business and in some ways it was ahead of the curve as we had a paywall very early on and was basically successful where it was self-sustaining and you allowed a half dozen people to have an income from it. But also the competition got a lot better I think in a hurry in baseball. The competition in analyzing baseball or the competition from the sense of the people who were putting teams together, the actual development? One problem that baseball prospectus had is that it became wound up being a kind of feeder system for major league teams where I think it must have had a dozen people graduate to go work for foreign offices somewhere and that became a real problem. And you see that now where John Hollinger who was ESPN's kind of the guru of basketball stats and ESPN's best kind of basketball writer got hired by the Memphis Grizzlies I think it was not long ago. So that kind of it's become surprising now where now you can kind of make a better living I guess on the inside and the outside where Supermetrics always began as kind of an outside agitation against like all these stupid decisions supposed to that teams were making and now you see it's been co-opted. Well and Bill James is working for the Red Sox. And Bill James is working for the Red Sox. Although seemingly for the last couple of years might not have that much influence at the moment. Well I do think it's, so I guess I'm one of those rare people where I don't find, I wouldn't say never but I wouldn't find it all that attractive necessarily to work for a team in part because I like to be able to analyze data where your only incentive is your own boss and you're able to share your conclusions with the broader public. I think that's more interesting than taking a conclusion where this is the same thing I've done consulting projects from time to time where you find some really interesting data and you share it with a room full of 12 people or something, right? It's much less satisfying psychologically than kind of being able to disclose it to the general public. Right, right. So as you were working at Base Power Perspectives and as you were doing all of this you developed what turned into 538 at the same time, right? So how did that start? So it began, I mean, you know, I think in a lot of things in life there are always a few threads that kind of are pulling together in different ways. So in part because of this law that Congress had passed in 2006 to ban online poker, I'd followed the ins and outs of that a lot and then what happened to, excuse me, to the representatives and senators who took the lead and passed in the bill and unfortunately a lot of them lost or retired. But so that got me more engaged in politics and then I was living in Chicago at the time I had gone to University of Chicago and so Barack Obama was a name that always like the campus progresses where I was like, oh Barack Obama, he'll be president someday, right? I'm like, give me a break, who cares? But it wasn't a name, it wasn't a name that was similar to me so it was kind of exciting to have a candidate who I felt like was in the same circle that you have see was running for president and then just that campaign was so darn interesting between Hillary and Barack and Sarah Palin and everyone else it was a great news story. But also I found that having seen, by that point I think in 2007 and 2008 you had seen a lot of the money ball clash kind of play out in favor of this kind of emerging consensus where you can use both types of information, you have the same goals ultimately especially with regard to- Both types of information meaning sort of eyes on the ground, intelligence. This gets complicated where I get finicky in my book about how to use the term subjective and objective but basically stats versus kind of subjective scouting use of players and we can break that distinction down more later on, it is a bit problematic but basically the stat geeks and the stats versus makeup. Yeah, we're getting along better and better but it seemed like politics coverage was still very stuck in the stone ages by comparison and not very data driven at all and so having seen this development in baseball it seemed like it was kind of overdue for some of the same approach. So it's interesting because in baseball it seems like one of the things that drew you to that was the fact that the people who were actually putting together the teams were not using this analysis and that created an opportunity. And in politics, if I heard you right it wasn't that the campaigns were not doing this but specifically that the press corps was doing this. Yeah, so it really was more. So the revolution so to speak in baseball happened from very much from the outside in where Bill James was doing this stuff 20 or 30 years ago. But you always have had campaigns that were reasonably sophisticated about this stuff including the Bush-Rove campaigns in 2000 and 2004 for some reason in 2008 and 2012 both John McCain and Mitt Romney did not seem to learn that lesson that you want to invest in your voting targeting efforts but Obama did kind of inherited that mantle in 2008 and 2012. You also have things like the Howard Dean campaign in 2004 that were a part of this story. And when you say this, we're sophisticated about this stuff you do you mean using whatever data points are available to craft a path forward? Well, particularly that, you know, like I think it's always dangerous when campaigns are too worried about here's what the polls. I think there's not often a lot of context about why people believe what they do.  But this particular activity of micro-targeting where you can say here, so the way the Obama campaign does it to simplify slightly is you have every voter in the country supposedly is measured by two scores one of which measures on a scale of zero to 100 where zero means you're a sure Romney voter, 100 means you're a sure Obama voter and in the other dimension is how likely you are to vote period. So you might have someone who if they're gonna vote they're almost for sure gonna vote for Obama but they might not vote and they're gonna get a different type of literature than someone who is definitely gonna vote but they're a swing voter there or someone who might be able to peel off from the Romney campaign. So coming to those profiles based on collecting all types of information about someone's name, by the way I mean you can tell an awful lot about someone just based on their name and where they live, right? If you have a name and address then you know an awful lot about what their income is, their ethnicity, their age actually because different first names go into style at different periods of time. So just making those inferences alone you can do reasonably good voter targeting. Of course there are problems like for example I had a friend who was from the Philippines because last name was Rodriguez and so you have a lot of heroes get like literature targeted toward Mexican Americans there because you have a lot of Hispanic surnames in the Philippines. But still you can get pretty far with that and that helped I think both those campaigns a lot. It might not be a coincidence that in 2000 the team with a better voter targeting which was Bush and Rove won the electoral college despite losing the popular vote whereas in 2008, 2012 of course Obama won both but if you break things down then Obama did a little bit better in the electoral college in the swing states and you expect based on the popular vote nationally alone. So it seems to be worth an extra couple of points. And the Obama campaign in this last election actually went drilled down to the extent that they were doing studies as to what email subject lines were most likely to get people to open the emails.  But you click on that email and after a while you get annoyed by it. And that literally is what the Obama campaign was doing. They found it very informal like hey look at this was much more effective than the future is at stake or something like that. See what happens if there's like some national security emergency and like Obama actually has to get in touch with someone by email, right? They'd be really, hey Nate. So you got interested in politics or you got interested in working more in politics and then how did 538 actually come about? So I started writing it at Daily Coast in part because I don't know. Under your name? Under, anonymously under the pseudonym Pablano which I don't, I always like Mexican food. So it's kind of anonymous and part of it was, part of it was I thought well, here I'm known for running about baseball and baseball and politics don't always intersect that well. But probably also in the end. And what way don't always?  Instead it's writing about baseball for. So you didn't want people to reflect on baseball perspectives or on the work you were doing there. That was part of it, right? Part of it was I was kind of playing hooky from my job at baseball perspectives to do this stuff. Seems like you have a history of that. I know. Every, I know, every four years. Future employers beware. I hope no one tweets that, but yeah. But you know, look, I do think. Too late. I do think in the long run that you're so much more productive doing something which captures your attention, right? That you really should do that 90% of the time. And you know, there are always bad days where although I somehow avoid writing about the sequester. I mean there are very slow periods in the political campaign and there are slow periods in the baseball calendar. And so you know, you can't always just say oh I'm gonna totally not do my job at all. But in general having a wider range of topics that you can focus on I think is something that kind of your career I think embodies as well. But it's super, super duper helpful. You know, one problem that a lot of political journalists have is that look, you have the same amount of column space and a print paper every day more or less. But the news doesn't flow that way at all. We're seeing these peaks of coverage where very important things are happening. Then you have months and months where nothing important happens at all. And so some days it should be like we're just gonna leave this page blank, right? Nothing important happened in Washington today. Go see a movie, right? But you can't, it's not the convention of political journalism. I think that's what's on. Newsweek decided actually. Yeah, right? So then if you were worried about alienating your audience what made you decide to go public? I think part of it is you had people like from Newsweek for example, the New Republic who would have wanted to talk to me and it felt silly to say, you know, poblano, a synonymous blogger with Daily Coast. Right. So at some point it became like if I wanted to capitalize on this in any way, meaning let it grow bigger and potentially shift careers and I had to kind of out myself, I guess. And so by the time that you did that you had some experience with coming into a field that was fairly set in its ways and certainly sports journalists are not the most forward thinking group of people in the country. What was the reaction among political journalists to your arrival on the scene in 2008? I think at first it was actually friendlier at first than it became later on. When you were a sort of cute little diverge and then someone who was threatening their jobs. Yeah, before you become someone who becomes threatening, right? So, you know, it did get a lot of attention in the kind of mainstream press and that helped quite a bit. I mean, part of that is, you know, the flip side of having cases where there's no news but you have to have a news story is cases where there's so much demand for coverage and 2008 was one of these historical years. You know, maybe they have, you know, one of the five great elections of the century. So you might have one fast election every 20 years or so and so that's a huge story about the biggest possible political story except maybe president getting impeached or something, right? And so you didn't have enough coverage relative to the demand for it and so different ways to look at it where it was a good time to be in that space. And so, if I'm hearing you right, so political journalists weren't threatened in the same way. One of the reasons was because there was no fear of someone reading Nate Silver and not reading something else because people were, their appetites were endless at that point. That's right. I think partly I, I don't know, I was gonna say the tone of the blog wasn't as combative but it's probably not true. It was actually probably more combative. It was more kind of bloggy and stuff. But for some reason, so once you start writing for the New York Times become more of a target, definitely. But also, yeah, once you realize like, you know, we do actually have a different model. I guess to say, so maybe the tone of the blog was actually more cheeky and more confrontational and less mature, I guess I'd say, back in 2008. But I did realize I went along that philosophically actually have opposition to the way that campaigns are covered by much of the mainstream press. So it's not just, oh, here's a different way. I think it's like actually you guys are doing it wrong. And this blog contains a strong implicit element of, and sometimes explicit element of media criticism for lack of a better term. So I want to come back to that. But so in 2008, you famously got 49 out of the 50 states, right? Was it Indiana? Indiana, yeah. You missed. Indiana. And all 35 Senate races, right? And so after that, was it clear to you that this was something that was gonna be more than an election year sort of sideline? So yeah, at some point, I mean already by late in the 2008 campaign, it had become my full-time job and I'd stopped writing about baseball. I'd stopped playing poker a couple years earlier. So that had become fairly clear, although if McCain had won, and we had him with a 2% chance to do so, then it would not be a very good career choice. I don't think any longer. But. Well, and that's something I also want to come back to. But so two years later in 2010, you moved over to The New York Times. How did that conversation begin? So actually, there's a little bit of a Boston connection, but I went to the MIT Sports Business Conference in 2010, I guess it was, and ran into one of The New York Times editors on the train platform coming back to New York. Who was that? It was Gerald Marzotti who runs the, ran the magazine at that time. And is that, I think, just yesterday was possibly put in charge of their digital strategic planning, yeah. Look, I think there are a lot of really great people at The Times, but when I ran into him, I had also been talking to other news organizations about bringing 538 on board. And I really did find that, so if you take The Times operation as a digital operation, I think it's the best digital journalism operation in the world. For some reason, because it also has this print operation, which you can talk about separately, it's not meant in a pejorative way, right, but people forget that this is a really successful experiment in how to get people to read content online, for how content looks online. NewYorkTimes.com is an amazing website, and I found that the people I talked to there were quite progressive about, not their politics per se, but in terms of what they thought about where journalism was headed. So they compared very well. I took a long time, several months to make that decision, compared very well against some other news organizations that were thought of as being more modern and digitally focused. And so it wasn't an easy choice by any means. You had, I mean, there were some other very attractive offers. Like, from what other news organizations? I can't, yeah. That I have to be a little bit careful about. Can try.  That's fine. But was, so at that point, had you pretty much made the decision that you were going to join some media organization and not exist on your own? I think so. I think I kind of come to that decision. I think part of it is that it's hard to have time to both be a content producer and kind of be a business person who's managing the brand. Right, so the idea is that if you, if you latch on with someone else, then you have fewer business decisions to make once you make that decision. I should say, you could maybe have done both, but it's also running a book at that time. And just so it was a way to say, like, first of all, The New York Times is a great way to distribute your product. But I think I felt like, yeah, in theory, there could have been a lot of upside to building the business out myself, but look, I think a lot of mistakes or mistakes that a lot of businesses make, right, is you take someone who's a developer or a producer of content or ideas and they become a manager instead. And sometimes you're not utilizing that person as well as you were originally, right? Where maybe they're even an above average to very good manager, but they were someone who was top of the charts as a inventor or a content producer and so you're not really maximizing their skill set enough. So. And did you find when you joined The Times that your reach or influence immediately changed? It was not like an order of magnitude shift. I think we probably got maybe 50% more traffic, which is still pretty significant. But it depends. With The Times, the slow days are actually no better than they were at 538, but when you get on the front page there and when something blows up, the peaks are a lot higher, is the big difference. So what we're at 538, right before you moved over to NYTimes.com, what kind of page views were you getting? So during the election, I think on election day itself, we got like 2008, I should say we got like three million unique visits, but the standard during a kind of regular day leading up to the campaign was, well, I guess I should say, it was always on the steady ramp up throughout 2008. So it started out where I had some following from Daily Coast. I think the first day you had like 500 views, which isn't nothing. And then there was some profile and in Newsweek we got to about 3000 and then became 10,000. Then after Sarah Palin got nominated and the conventions and the interest really peaked up to 100,000, then up to 3 million on election day. And then you have the big fall, it would be nice to have 3 million page views every day, but it's not gonna happen. Most people are sane and have lives and so once the election's over. Not sure I agree with you there, but. Well, yeah, actually. But most people have other interests apart from politics, apart from campaigns. So it's a blog that has very good and steady traffic. But not well-being traffic during the non-election part of the campaign and then goes kind of off the charts during the election cycle. So this election cycle, certainly your relationship to the mainstream media, despite having become a part of the mainstream media, seemed to be pretty radically different from what it had been in 2008. A number of people seem to be going out of their way to pick fights with you or you and Joe Scarborough got in a little dustup, which I think you won fairly handily, but also places like Politico, as the campaign was unfolding, we're writing pieces asking whether you were gonna be a one-term celebrity. So my first question about that is, why do you think that is? Was that just a case of people wanting to kick the big guy on the block down a notch? I think that's part of it. I have one Politico reporter who shall remain nameless, but tell me, not Dylan Byers, no. Hire by the time, I was told by this reporter like, oh, now that you're working for the time, somebody can't quote you anymore, right? So part of it is you have people who are deliberately... Oh, someone at Politico said now that you're working for the time, so you can't quote you. Yeah, yeah, yeah, so it's deliberate competition, right? It's not even, a lot of it is kind of, some of it's premeditated, but more, so we were talking about this earlier, right? So in 2008, it was fairly clear for the most part that Obama was gonna win, right? To the point that even the press who was trying to report on polls, there are some counter-examples with the McLaughlin group, threw their four panelists on election 2008, said, oh, it's a toss if it's too close to call. Monica Crowley from Fox News said, she thinks McCain would win by half a point, right? But... And she still has a job. Yeah, but in that election, it wasn't controversial to say Obama's gonna win kind of easily, right? Whereas this one was right on that precipice where on the one hand, it wasn't truly 50-50, and as much as it was misinterpreted, the blog, you know, we couch things inexplicitly in probabilities, but we were never saying, oh, Obama's gonna win for sure, we were saying, but he is a favorite, and he's enough of a favorite that to describe it as a toss-up is probably wrong, right? And for some reason, the fact that the set of facts we were working from were so simple, right? Which just, you know, look, the polls by the campaign are pretty accurate. If you take an average of the polls, you can do it our way, which adds a little bit of finesse in terms of how much we weight the polls, but any side, with their politics or, you know, the Princeton Election Consortium or the five or six different ways to do it all had, we're calling, you know, 49 out of 50 states, the same anyway, and all of them had Obama winning the Electoral College. You can debate about how much uncertainty there was around that, but, you know, I think that was part of what made the attacks kind of personal at the end is that, you know, we were asserting a very simple set of, you know, kind of sky is blue, two plus two is four is that a fact's like, look, you know, can you add up to 270? It's how many votes you need to win the Electoral College? Can you take an average of the polls, right? Well, if yes, you come to the conclusion that Obama's ahead right now. You can debate, again, does that mean he's 9% likely to win or 55%, but that's what I think made it so personal is there wasn't this ambiguity. You couldn't just get lost in the weeds and debate the interpretation so much. Well, there also seemed to be this confusion among some of the people who were critiquing you about process and results and having a process that essentially is a good process and is gonna more often than not get you to where you wanna be going and always getting the results that you want or that are predicted from that process. And when you, I guess you bet Joe Scarborough, is that right? He didn't accept the bet actually. Okay. We both wanted to be contributing a charity. Right. Which you got in some trouble for there. Yeah. Which sort of showed old and new media. Well, part of it is that, you know, you talk about people being two results oriented and they actually, they certainly are two results oriented and I think I said before the election in different venues that like I'll get too much credit if we have a good night on November 6 and too much blame if I don't. And we had a good night and I'm getting too much credit right now. But the point was in a literal sense, I had in terms of kind of net future earnings discounted somehow then I had a lot more than 2,000 bucks or whatever the bet was on the line. And I don't think it's a bad incentive. Look in business, you want your product to be successful and there's some risk there. You can have a good process and get a bad result or vice versa. But the idea that you're tying your incentives to your notion of correctly forecasting an outcome, why that's objectionable, I wasn't really sure. When I played poker, it's honorable to bet on things, right? So I kind of sympathize with the Mitt Romney, like $10,000 bet against Rick Perry in one of the debates where it's like, at some point this guy's yammering on, yammering on.  Well, then bet on it, right? Right, but let's say if you had not gotten all the money of the state, did you get 50 states right this time? Yeah. So you called Florida, right? No, I, yeah, I did. No, I, so I don't think, I'm not sure that the 50 out of 50 thing matters so much as game election, right? Like if we had had some election where we had gotten, you know, so for example, we had Obama winning by, by an popular vote by 2% on our election night forecast. He wound up winning by 4% instead, right? So it's almost as big an error or the same error as if Romney had won the popular vote by, by 1 tenth of a point or something. But I don't think it would have been seen symmetrically, certainly. If Romney had won by 1 tenth of a point, right? Yeah, I mean you can criticize that, oh Nate, you know, underestimate Obama. One thing that helped, I guess what's fortunate for me is that you had a lot of states that were, were just barely leaning Obama.  Whereas if Romney beat his polls by a couple of points, he might have won four or five states. So you know, you'd be far from getting 50, right? But so apart from what the perception would have been in the public at large, let's say that you had gotten it wrong, those two percentage points that had gone the other way, ended it gone the other way in enough swing states so that Romney eked out an electoral college victory. Would you then have doubted the formula and doubted the system that you put together? Not too much, because you know, the funny thing about the 538 formula is that there's more uncertainty really, I mean in a literal sense, when you're early in the election in June or July, there's more uncertainty, because there's more time for different events to intervene, but there's also more structural uncertainty in building the model, where for example we had a way to account for the economy where we created an index of economic conditions and then, but there are a lot of ways to do that or how you weigh and come and see. So, you know, but by the time you get to the end of the campaign, you know, all the model based approaches really, really converge, right? Where it's, because we weren't saying at that point it wasn't a complicated model at all, it wasn't trying to balance the economy and gets the poll saying, we're just looking at the polling at this point, anything from the economy should be priced in by now, and it's just an exercise in saying, how much uncertainty is there? And if you have a four point lead in a state or a three point lead in a state on election eve with robust polling, how often do you win that state? And that's a case where you come closer to there being like an objectively kind of right answer, whereas earlier it's a more difficult modeling problem. And so, you know, I wouldn't have felt too bad if, well I would have felt bad for a lot of reasons, right? But I don't think I would have said, oh, I gotta go back and share them all. When you say for a lot of reasons, you mean including the fact that you're an Obama supporter?  And now it'll be fun to troll Politico for a few days. But no, I mean, the fact that the, you know, I mean, and this is probably a bad sign in some ways, right, but I was working a hundred hour weeks and he's become so involved in this kind of psychodrama around what the 538 predictions mean, and also just the fact that you're making these forecasts and blogging and doing a book tour at the same time, right? So you become kind of abstracted from the real world of the election. And even in 2008, we're actually, was in the museum in Washington doing stuff for HDNet on election night, right? Where we were doing coverage until one in the morning, right? And you kind of say, oh, Obama win.  Being in the bubble is kind of a real thing, I think, and something that maybe of the other problems I have with campaign journalism, I think the lack of self-awareness of how your perceptions are skewed when you're in that bubble as a reporter. 75% of the American public has heard nothing about this sequester, right? But journalists think that, oh, the way Bob Woodward's column is interpretable radically shift public opinion on this stuff and that can be something you want to avoid. Right, right. So there are a couple more points I want to get to before we open it up. One is just your future with The Times. I just want to read what Jill Abramson, the editor, executive editor of The Times said recently. So this was from after the election, I guess, late November, she said, half the people coming to nytimes.com searched for Nate. They weren't coming for the rest of The Times, they came for him. She also said that when asked if you would get a wing of the building, I think she said that you have your own tower. So your contract, you signed a three-year contract in 2010, which comes up this summer, is that right? Yeah. So what do you see for your future? So I don't have any news to break. I'm inactive discussions with The Times, and like I said earlier, it's a really great fit in a lot of ways, I think Jill is a terrific editor. I mean, anything can happen in a negotiation, but we'll see, I'm pretty happy there. And the best things, and the worst things we're working at The Times are, for the most part, pretty self-evident, right, where the best thing is that it's still one of the most trusted brands in news that make the content look really good and kind of they let their people have a fair amount of voice, right? So you look at someone like David Carr, for instance, their media columnist, you know, he has found a way to have his own voice and somehow have it be in harmony with the mission of The Times brand. I think there are other news organizations that tend to swallow up their voices, and that's problematic now. So knowing that is definitely a reason to stay with The Times. The negative is just that you do get a level of scrutiny that I think is deserving in many cases, but also times. Are you talking about scrutiny internally or scrutiny because you're at The Times from the external world? Well, I think The Times, like any newsroom, is often internally competitive as well as competitive with its rivals. But I mean more in terms of, you know, everyone comes after The Times, like it's the New York Yankees basically, right? And a lot of people hate the Yankees, right? But everyone pays attention to what the Yankees do potentially. You know, the less obvious downside of that is that sometimes it's hard to be kind of casual at The Times, right? Where, you know, with the blog, sometimes you can kind of say, well, here's an observation. You wouldn't quite phrase it exactly this way, but you'd say, here's an observation I have, but I haven't really thought about this very deeply, right? I'm just kind of, you know, farting around, I guess, right? And it's hard to do that at times. There's no farting at The Times. It carries such authority, right, that when you say something that people don't, you know, like we got some mini controversy with Politico the day about the choice of language I use to describe the committees of super PACs, right? It's like, well, we don't actually, it's not actually important. Is this the clarification versus correction? Yeah, yeah, yeah. It's actually not that important in the context for us to be super precise about the language that we're trying to communicate this other point to the audience, right? But at The Times, you can get into more trouble for that kind of thing, because people do treat it as being so authoritative, which I'm not saying is a healthy thing necessarily, but so it's harder to find that voice. But like I said, for people like myself, or like David Carr or some of their tech columnists, or their sports columnists, they do, is a space that cultivates having many voices and not just this one disembodied kind of voice of the year in time. So that leads into another thing I wanted to get to, which is you mentioned how your methodology is not as transparent as it might be in, for instance, an academic environment, but you're fairly upfront about what you do. We talked a little bit earlier about how this is actually not rocket science.  So what is it you think that has made you so popular? Is it the voice of the blog? Is it the fact that I know you worked for your college paper, have spent a lot of time at writing?  Yeah, I mean, look, I spent a lot of time on my writing. I just don't, I don't usually put a blog post up without reading it and rereading it four or five times and the times encourages you to think about your pros. I spent a lot of time on the graphical presentations of things we do and the times interactive journalists, the graphic journalists are, and this is not hyperbole, they really are the best in the world at what they do. And so that helps a lot. The way you're communicating information, I think a lot of these things we're talking about are not really that complicated, right? And I'm a guy who says, yeah, the details do matter and sometimes I'll get back and forth with academic political scientists about the nuances of how do you measure the economy in your election model, et cetera, et cetera, right? But mainstream criminals are missing very, very basic. But that's very stuff in the margins. Yeah, at the margin, right? They're missing very basic points and I think sometimes when, I don't know, it's a balance because you can also have cases where conclusion is oversimplified. So I guess that's kind of the skill in good nonfiction prose writing about science or technology, right? Is how do you distill the gist of something without dumbing it down, right? And that does require some skill and because I wrote about baseball and because I had some background also as a kind of pseudojournalist in college where I actually did go and- Why pseudo? Well, not pseudo, I guess, right? We actually go and track down like they're gonna close down the softball field so not as much of it as some people might have but they do have more of that. It's a big story in Chicago. Yeah, yeah, right? In high school and college. So to have those skills was really helpful. So I think two more questions. One is, two more questions from me and then we'll open it up to all of you. One is, do you think that we could or are moving towards the type of predictions that you do at 538? Being able to apply the methodology to make those predictions about a broader subset of issues. Could that enter into a sort of scientific discipline the way that political forecasting has? Well, look, there's more interest in data science now and kind of big data. I tend to be suspicious of claims that suggest there's some massive inflection point in technological growth, I guess I'd say, right? So you clearly have... Meaning what? Well, so you clearly have an exponential increase in the amount of data and information available to people measured in a variety of ways but that's never in the constraint, right? The constraint's not the amount of information you have but our ability to come to useful knowledge from that data that yields tangible improvements in technology and the economy and everything else, right? And so our human constraints are gonna be a much bigger factor than this growth of information, right? And people also don't realize that sometimes you might have a data set which is large but not especially rich necessarily, and this is part of what I talk about in the book but you have problems like kind of overfitting as technical name for a problem where you're mining data from the past and claiming it's predictive but really it's just giving you a very detailed description of past happenings and it may not carry over and make good predictions as well. So the book is a survey of basically is big data producing or progress in the context of prediction? The answer is yes in some cases but as a general proposition it's slower than you might expect. And so does that mean that you've gotten some criticism most notably I think in a New Yorker blog post about your reliance on Bayesian analysis. Am I hearing you're right in saying that there's a place for both types of analysis? Well I think so, what qualifies as Bayesian is kind of a sticky subject where if you talked about people who are formal statisticians that means a specific set of methods, right? And in my book I don't usually mean it in that technical way. Instead I see it as almost a series of values. One of the most important of which is that you start from some base where you do have assumptions and biases that maybe are equal to the wealth of knowledge accumulated up to that point in time or what the consensus at that point in time. And so the alternative where you're testing a null hypothesis right is kind of arbitrary. And that if you find something which is statistically significant relative to the test you set up it doesn't make any sense in the broader context of other research you should be suspicious of that for example. You see that in a lot of science journalism for example where sometimes I think there are now something like three academic papers published every minute or some ridiculous statistic like that. Tens of thousands every year and sometimes the one that make the most outrageous conclusions are the ones that get reported on and passed along often times right. And the peer review process is by not any means perfect but you do gradually have this synthesis where someone says no this is the cure for everything and someone says no it's a cure for it.  And then eventually there's some synthesis where you move toward the truth. You have that process. And I see that as being Bayesian the idea of kind of updating your beliefs as you move forward. So I think I wasn't. So you're referring more to a Bayesian philosophy as opposed to. So I think you know I think some of the critiques took to literally the sense so you should only apply these particular methods where there's maybe a kind of frequentist technique you can apply in a Bayesian way I suppose or maybe vice versa. Right. Okay. And actually I guess I lied because there is one more thing I wanna touch on before you open it up and that's you had some strong words for the punditocracy. There are a number of columnists I think we probably both have some problems with but what I find particularly frustrating is columnists who assert things with no factual basis. And then claim that glad you agree. And then claim if you criticize the fact that they have no factual basis they say well what they're doing is exercising their right to voice their opinion. It's just an opinion man. Right. Apparently they also smoke a lot of pot. Yeah. That's the wonderful thing about prediction though is that if you're making a prediction a true out of sample prediction not a model that was fit to pass data then you're actually putting your opinion on the line. You're testing whether your perceptions have any correspondence to reality and I'm not just being cheeky I mean you see Phil Tetlock was a guy who used to work at Berkeley is now at Panda Psychologists and he studied over 20 years how well that the predictions of political experts do. And so that included people you think of as pundits also academics and journalists and people on the inside of politics and found that they weren't really any better than random for the most part. And there were some exceptions there were people who have certain personality traits what he called foxes instead of hedgehogs who did a little bit better. But the industry has a history of getting things wrong. What was kind of amazing too was that you had people who some very very smart conservatives like George Will and Michael Barone who were not just asserting that well it's close and a Romney could win Ohio. But we're predicting a Romney landslide and contradiction to all available evidence. And Peggy Noonan also very smart who's saying that the fact that she saw Romney yard signs meant that there was a groundswell in the country as a whole. Yeah, so her neighborhood or Tony neighborhood in Virginia or whatever, right? And people at their Georgetown. I mean this really is like you have a better source. The idea that she could read the average voter better than the actual random voters who were like literally called by the polls. I mean that's the pretense behind why what organizations like the Times sponsor polls actually is that. So you have reporters who report and although they're gonna represent the kind of expert or elite consensus you also wanna hear from the men or women on the street and what better way to do that than to randomly survey random men and women on the street. And lo and behold polls do fairly well about under certain assumptions, right? People are bad and polls are bad explaining why people believe what they do. And people are bad at predicting their behavior far in the future. But for the most part, if you ask someone who you're gonna vote for tomorrow then they'll give you an honest response. We know that empirically. And it's a lot better than trying to count up the number of yard signs or something. So the fact that Peggy Newton obviously continues to be employed by the Wall Street Journal is that, do you think that's indicative of the sort of power of narrative and that she is a very good writer and is that a positive writer? She's a very, she's a, it's a prilative writer. Or is it, does that have more to do with the fact that we now can create these information cocoons and we're getting more and more used to only consuming the information that is gonna validate our worldview? It's both and I'm trying to figure out which of those, you know, but I actually think that the latter concerns. So kind of the story I tell in the first book, The Signal in the Noise, is kind of the story about, well, our brains are wired to detect patterns and to build stories around essentially random data. And that's a big part of it. But I do think these kind of information feedback loops is a bigger and bigger element of the story as well, where if you look at the demographics, actually the Wall Street Journal is an exception where it has a fairly bipartisan audience. But, you know, only 5% of people who watch Sean Hannity on a given night are Democrats, right? Only 2% of people who watch Rachel Maddow on a given night are Republicans. So you really have people consuming the news in two different ways. One thing I'd like to do is go back in maybe a somewhat formal way and study which polls were cited by the media most often and when. Because this seems like a fairly tangible way to test media bias, right? If you have two polls, for example, I remember late in the campaign, there was a case where you had two polls came out on the same day in Minnesota, one of which had Obama up only three points, suggesting it was in play, one of which had him up eight, right? And the poll that had him up three points, I'm showing it competitive, was cited like five or six times more often, right? The only reason for that was because you want to create the narrative of the campaign still. That's the story bias, right? Yeah, I'm not saying, believe me, I'm not saying that the media has a Republican leaning bias, believe me. But I think it does have a. No, but that was a bias for a story. You have a bias for a story, right? You see this absolutely in the general election campaign but also at the end of the primary campaign where people were writing, you know, when Rick Santorum was hundreds of delegates behind and really didn't have a viable path unless there was some major scandal involved. And people were saying, well, you know, Romney must win this Illinois primary or what? Then he'll only be 180 delegates ahead, right? So the primaries especially, to some extent in 2008 also with Clinton versus Obama, that was a little closer than- But later in that- But yeah, later you're kind of playing out the string a little bit, unless you had some monumental event occurring, then that outcome was preordained for the most part. Right, all right, why don't we open it up? We have two microphones. We're gonna try and keep the questions pretty short to about a minute so that we can ask as many as possible. Why don't we start over here? Oh yeah, and please identify yourself so we know who to come after afterwards when you're at the mic. Hi, thanks very much. I'm Ian Condry and I'm a professor here in comparative media studies. Really enjoy your stuff, it's really interesting. My question is about political reporting and these questions of objective, subjective, because I certainly agree there's a problem with media bias, but I wonder if there's not also a danger of too much accuracy. And this has sort of been- Too much, wow. Accuracy and predictive power. I mean, one of the things that's so frustrating, I think, about the political campaigns is the oversimplification of red and blue, the emphasis on only the swing states, the undecided voters, and then of course the last minute somehow it's all about the base that needs to get out and there's a kind of common narrative. But I wonder if there's a point where the predictive power of 538 becomes kind of self-reinforcing, does it really to a decline in democratic participation because we think it's already decided. I mean, certainly in the red and blue already decided states that happens to some extent. And I think in some ways because we still didn't trust you until this election, that now it's, I wonder if it changes the dynamic now that 538's so accurate, did so well, that then do we even need voting? And I'm wondering what's your- Yeah, I hope people don't- Well, that's my question is sort of in your sort of self-reflexive and thinking about your role in this, what do you think about that problem? You know, as I've said before, if I feel like 538's forecast are actually influencing voting behavior, then that would be a very dangerous precedent, I think. Although I would say media coverage in general influences elections, especially in the primaries, for example, where in one case, for example in the GOP primary this year, for some reason the kind of media decided early on that Gary Johnson was not a serious candidate for some reason. Even though, credential-wise, he was someone who had been elected governor of a swing state twice, I think, where some other guy who ran a pizza company was more entertaining, and so I guess was determined to be more viable, right? But you see the media put its finger on the scale in a lot of different ways, and to the extent you have that, then I'd rather have, I guess I'd rather have accurate information affecting public sentiment than inaccurate information. But the other point is that, you know, I am not someone who always says that, oh, everything is super predictable, right? And people are just being boneheaded about it. If you read the book, then we say a lot of times, in general, more often the opposite error is made when people overrate how predictable things are. And so if you see, for example, when you have a jobs report come out where say we create 20,000 jobs in a month, where we're supposed to create 90,000 or something, right? Those types of errors happen all the time. When average, the miss is about 70,000 or 80,000 jobs, but it's always seen as, you know, a catastrophic mistake. People don't understand how much economic uncertainty there is, for example. So, you know, I mean, part of it is, I think journalists seeing empiricism as kind of the enemy and seeing, looking at the work that economics do and more technical analysis. I think that's more the problem than saying everything is up for grabs because sometimes the opposite mistake is made instead. So, and you did say recently that if in 2014 or 2016 you saw 538 influencing elections in some way, that you might then step away from it because that would make you feel uncomfortable. So specifically, what did you mean? How would you see 538 influencing elections? I mean, I think it's probably more of a risk in say individual campaigns for the US House or something where one case where the polls probably do matter some is in a stage where you have candidates competing with another for scarce resources. So in the primaries, for example, Tim Pawlenty's people quit the campaign and retrospect Tim Pawlenty and his people quit the campaign. In retrospect, they could maybe have one who knows, right? But they felt like they were far behind enough where in the polls where they were not gonna get donations from leading Republicans and therefore they couldn't build out the campaign so it kind of became a self-fulfilling prophecy. So you have cases like that. So if you have cases, for example, where we have influential Republican donors or Democratic donors trying to figure out which Senate campaign do I donate to when they go to 538 and they see that, well, this campaign, the GP candidate has a 30% chance of winning. They're still very viable. I'll give money to them. And not the candidate who has a 5% chance of winning instead of that kind of, that might be a way in which the outcome is influenced potentially. But why, so why would that, I'm not entirely clear on why that would make you uncomfortable because isn't that a valid data point for them to make those decisions on? I guess that's a counterpoint, right?  But remember, implicitly, models are based in the assumption that, I don't know. I mean, there are some kind of academic questions, I guess, about do you have a model which is premised on the notion that people don't know what the model says and then does the behavior change as a result? So one problem we had, for example, in 2010 is we had races for individual US house seats in 2010, which we didn't do in 2012. We will again in 2014. But one of the inputs to that model was the forecast made by independent analysts like Cook Political who have a very good track record, right? But we were basically free riding off them. But the problem is that, you know, I know the Cook Political guys, you know, organizations that do this and they would read 538 and they would say, well, you guys in this county has a 90% chance of winning so we're gonna upgrade back from safe or from, you know, leaning Republican to safe Republican or whatever, right? So you have a feedback loop effect instead. So we're gonna cut that out of the model in 2014, I think. You mean cut out the using things like Cook Political reporting? So this is a slightly esoteric point, but I think if you build a forecast model, I think the metric you should use, it's not solely about how accurate you are but it's how much value kind of do you add. So if you have a model that like basically, and there's some models like this that for example, take what intrade says, which are prediction markets, and then slightly tweak that or something, right? If you're trying to gamble on an event then that might be best, but you're probably adding very little value to the overall public knowledge.  So the extent that you can say here's an approach, it's more proprietary, right? And here's a value that it adds, I think is preferable. And so to try to avoid those feedback loops, I think would be one small step there. Yes. John Hawkinson, Electrical Engineering and Computer Science. My question for you, it kind of reflects I think what a lot of people I know would want to ask you, and apparently the person in line behind me too, go figure. What are the chief impediments to open sourcing your model or variant of it or historical version of it? Because people want to play. To open sourcing them up, well first of all, it's not that hard to reverse engineer it necessarily, but you know, I'm very sympathetic to the kind of academic viewpoint that science should be open source, right? But I don't have a 10 year track position. I make money in part for my ability to license proprietary content, and if you open source something, then that value is diminished. But I would never rule out that I wouldn't do it. One solution maybe, if I do feel like I get bored of this or 538 is influencing too much, just to publish the source code and say now it's in the public domain and go ahead and have fun with it. But I do think we can reach a better level of disclosure where things are more explosive. If you go back to 2008, things were fairly explicit about how we disclose things, but we didn't release the source code, we didn't release the actual database of polls. But you know, I don't know. I mean like I said, I'm still thinking of the point of view that should be completely open source, but you know, it is a for profit business and that influences things a lot. Even next year we could see last year's model. Well, the model shouldn't change to all that much from year to year. Yeah. Yeah. Hey Dan, I'm Ryan, political junkie, so that's why I'm here. I wanted to ask you, so basically in order to forecast you have to understand the system you're forecasting for. And I think kind of your hallmark right now is the Electoral College. And you've kind of voiced, I think pretty clearly that you're opposed to it. You think probably a direct vote would be better, a popular vote. And that's great in terms of, I mean I think it's a pretty simple explanation for a Unitary Executive. I'm wondering for congressional and parliamentary systems, if you have an opinion on what might be a better way for elections to work, like there's, because there's a lot of debate in that theory, whether like an AV and AV plus mixed PR, stuff like that, the way systems could be used for congressional and representative elections. If you have an opinion on what might be best. I mean the funny thing is that the, in some ways the Senate is otter in that you have as many senators in California as you do in Wyoming, but you're actually getting more of a skew now in the House of Representatives and either the Senate or the Electoral College because you have Democrats concentrating big cities like Boston and Chicago. And you just wrote about this in the last day or two, right? And in some ways the fact that, so states happen to work out fairly well where most states have a combination of urban areas and rural areas. But congressional districts where you really are packing Democrats in or shouldn't use the active voice unnecessarily, where Democrats are kind of packing themselves into big cities now, sometimes abetted by gerrymandering and states where Republicans control the process. But yeah, that has interesting implications where this urban-rural divide, the Constitution kind of part by X and part by design tends to give slightly more voice to rural areas relatively speaking. People talk to you about other Republicans that have disadvantage in the race for the presidency. Well maybe, but also they're doing well in rural areas. That means they are gonna always do well on the race for the House, for example. So trying, if the party's trying to optimize its overall political portion, that might be kind of a quasi-stable equilibrium where Democrats do have an edge in the presidency, but the price for that is that in midterms where you have voting groups like young voters and Hispanics, African-Americans are less likely to turn out where the way district structure tends to favor rural areas a bit more, you'll kind of have, maybe we'll have a Democratic president in a Republican House for a very long time. So get used to the current gridlock, I suppose. Yes. Hi, I'm Rachel, I'm just a student here, but I was wondering, you were talking earlier about the issue of punditry in the media, and also about the issue of feedback loops, and just like what thoughts you have on the media's ability to fix that problem since people do pay for the media that they want to receive and they want to hear the things they want to hear. So it's a difficult problem, and one thing I've been thinking about is it's fun to critique individual journalists who kind of are doing things in kind of a silly way, but their self-interest is often in providing coverage that's sensationalistic, right? And so now I'm in a position now where, look, I work for The Times and I publish the book and have enough income now where I can say, well, I can be magnanimous and kind of, so you have to have the truth, and I always have done that throughout my career, but it's easier to do that when you have the luxury of doing that instead of someone who's struggling to make a name for themselves and publishes some link bait that's trolling Nate Silver or something as a result or that's hyping up less than that, that's hyping up the result of some poll because they have to meet a quota for how much traffic they get, and I don't necessarily have a good sense for how to solve those problems. I do think, and maybe this is a romantic idea, that sometimes journalists or news organizations think too much about how many pages something get in the short term as opposed to what's it say about your brand in the longer term, right? That when you link to something that's stupid, then there are two things that happens. Number one is that you're hurting your brand in the long term. Number two is that it's easy to compete with stupid, right? Anyone can link to someone else's articles and do this kind of cheap little value aggregation, right? And so you're not gonna have an advantage in that department for a very long time. And so how can you be differentiating is the question I would ask, but for people who are kind of engaged in the career being a political reporter or something, and it's more difficult potentially. I think you have to have, look more at the organizational level in terms of what values does the organization have and how are they incentivizing their people and do they care about their brand? And do you think that there's any possibility that news consumers will start to play a more active role? And I guess this is my sense of things and not based on actually any data. But my sense is that as consumers we're still functioning as we were 20 years ago. So sort of assuming that there are these big media conglomerates that essentially are working on similar stories and have similar story lines where that's obviously not true. And it seems like we have a ways to go before the active intelligence of news consumers kind of catches up to their influence in the process.  I think not everyone's gonna want the kind of wonky nerdy coverage that might be. Meaning what? Meaning that they think that people are gonna don't see the forest for the trees basically. And I think that if they say, well, the race is rapidly shifting that people are just gonna buy that and not have their own scrutiny of it necessarily. And very often for the campaign you saw events. So for example, people may remember after the attacks in Benghazi, which is now seen as a failure on the part of the White House, right? But there was this kind of very arbitrary media consensus that Mitt Romney's reaction to Benghazi was a big story would really hurt him in the campaign. And it didn't really move the polls very much at all, right? But I think the news media tends to have an overly simplistic view of how voters think about elections where voters might not think about exactly like someone in the Beltway establishment might, but they do have their own way. People think about presidential elections pretty carefully, where for example, the heuristic that people use to measure economic growth is arguably pretty sophisticated where it's not just oh, how good is the economy right now, but actually looking at is the economy getting better or worse? And I'm gonna look at it relative to the baseline of a couple of years ago and not over the longer term because that's a way to assign more credit or blame to the incumbent, right, Roman? So you had a case like this year where the unemployment was 8% in election day or also in 1984 where it was 7.5% where you had incumbents in Ronald Reagan's case a landslide but elected easily and then not so easily but clearly enough because you had economic progress being made or some reporters like oh, economy bad, voters tossed out incumbent, it's like no, actually people are like a little bit more sophisticated than that if you read the political science on it then it describes what should have occurred. But Politico is actually from a financial standpoint, I would argue a journalistic success story over the last five years. I don't know much about their finances. Look, there's gonna be a market for certain things and one secret about being a rag, I guess targeted toward Beltway insiders is that it's very advantageous to reach that crowd with lobbying money in essence. So if you have sites that are targeted for inside audiences you can make incredible CPMs by running ads for clean coal or whatever else. And CPMs being? Being how much you get per ad pressure, right? By orders of magnitude higher than reading it to a general audience because you are literally reaching congressmen and senators and staffers on the hill and people who are peddling influence in different ways, right? So there is definitely a market for an insider audience that can sustain a certain number of reporting organizations. I happen to think that kind of the traditional ones like Roll Call and the National Journal and stuff do a somewhat better job with their reporting than Politico does, for example. But that's certainly a market niche. And I do think that this demand for data driven coverage or kind of wonky political journalism is also a market niche, right? I think it's not gonna ever displace pegging in terribly but I think it might be that it's something that 20% of consumers want and it represents 5% of the coverage. And so right now you still have things where I think people like me are in demand. You've probably seen increase until our rich state equilibrium and then maybe kind of goes over it in some sense, right? Like in baseball, for example, I feel as though there are plenty of stat heads providing very, very good analysis, right? So there's no more low hanging fruit as far as we have a more quantitatively flavored coverage. The market's pretty saturated at this point. Right, right. So I'm not in the prediction game but I'm gonna go out in a limb and say that Politico is not one of the news organizations that you're in negotiations with. No, they're not. Although maybe for an April Fool's joke, I was thinking about like, right? Yes. Hi, I'm Charles. I'm just a little bit undergrad. And I was actually worried that the guy in front of me stole my question but thankfully he went in a different direction. Basically what I was wondering is this, whatever your personal opinions about the electoral college versus national popular vote, would you say that you have something of a vested interest in our keeping it just because it makes the minutiae of state polling so much more relevant and it gives you so much more material? So I mean, the electoral college is like a super interesting modeling problem, actually, where, so the two assumptions that are clearly false are to assume that you don't want to assume that every state is independent from the next one. Clearly when you had, for example, this is very easy to perceive, when you had Obama surge after the Democratic Convention, he moved up by about the same amount in every state when you had Romney surge after the first debate, about the same amount in every state, little bit of difference around the margin, right? But how do you strike that balance between, assuming the outcomes are perfectly correlated versus perfectly independent? That's an interesting challenge. And that's kind of one of the things that it would be worth getting into technical debates about because I think we had a pretty good, although too complicated to explain in a forum but kind of methodology for that. So it's definitely more interesting in that sense. It adds a layer of intrigue to it. So for purely selfish reasons, yeah. And the name of the site, 538, it's based on the number of votes in electoral college, right? So we had a situation a couple of years ago where when Democrats controlled Congress, they almost gave DC a seat in the House. It would have been 539. She would have been screwed, I know, yeah. But fortunately for me, the NRA somehow tied this bill to a bill on gun control and so it got stymied in committee 539 is much less mellifluous than 538. I think so. It would have been, and someone else had already registered it instead, so. So I'm really against, in a selfish way, against Puerto Rico gaining state power. Have you now gone and registered 540 and 541? Oh, it's too late now. In fact, it was quite, I mean, every number spelled out and written from zero to like 6,000 or something is now. It was now taken, right? And if I'm allowed to do like a quick follow-up, I'll enverse some of the lines. If we somehow move to a national popular vote, whether by interstate compact or somehow constitutional amendment, beyond just changing the name, what would you do if 538 is still active at that time? I mean, it's still due forecast. One point we tried to make during this last campaign is that even absent the Electoral College, state polls provide a lot of information about where the national popular vote is. We can, you know, if you wanna estimate what will the national popular vote be there two ways, you can just say, well, average the national polls. Or the other approach is to, it's just, you know, roughly how many voters vote in every state, then average each state poll and add that together. It turns out that second approach actually seems to work a bit better in part because you have more data points. So that's a case where actually doing a bit more work. You know, in general, I think one theme about modeling is that you know, you get very diminishing returns to adding complexity to your model. That's a case where you have enough, the data is more robust if you're averaging from the individual states and so it could have still been useful in some ways, but it would definitely be less interesting as a modeling problem, I suppose. Yes. Hi, Nate. My name is Michael Steppner.  And my question is this. It seems like you have the presidential prediction model down pat. You're doing pretty well at it. And looking at your career, it seems like you enjoy building models. So I guess my question is, what do you see as your future in terms of technical innovations? Is it kind of improving your existing models? Is it building new political models or going in new directions? Maybe that's secret, but. Well, you're talking before about the history of kind of, once I get bored of doing things every four years and kind of veer off in an unpredictable direction, you know, one area that I find interesting so I did a project for New York Magazine in 2010 where we rated different neighborhoods. And of course, it's a magazine seat. They want some objective answer for which is the best neighborhood. But my whole point was that, well, you get to weigh the factors that you want and you find a neighborhood that's a comparative advantage relative to your price point. But, you know, urban data, for lack of a better term, data about cities is another case where you have a data set that's somewhat rich, but it's maybe not being used all that well and so you could kind of strike that middle ground potentially. But yeah, I definitely will get, you know, it's fun to kind of be more abstract and write and talk about this stuff, but at some point I'm sure I'll have the temptation to go delve into a big new data set because that modeling is a lot of fun, right? When it's a fresh problem and you're not just kind of making improvements around the margin. Turns out that 539, 540, 541 and 542 are also all taken as Twitter handles. Yeah, yeah, yeah. Yeah, yeah, yeah. But 543, however, is available. Yes. Hi, Nate. Sunny Sudu, I'm a graduate student in the Comparative Media Studies program. Just wanted to thank you and you, Seth, for this been a really great discussion. So my question actually has to do with, I think that it's really clear now that there's a lot to be gained from taking a data-driven and model informed approach to political campaigns. I'm wondering, do you see much of a room to grow in terms of applying that approach to policy and to governance rather than just the horse race coverage of political campaigns? And so, I think in the same way that it's just more productive and insightful and fun to talk about baseball when you talk about wins above replacement and not building percentage and wins and losses. I think that there's some debates that could be richer in this country if we moved away from sort of misleading and reductive approaches to data. So for example, we've got this makers versus takers debate that's going nowhere because it's informed by this really silly model that if you subtract somebody's social program benefits from their federal tax payments then you can tell whether they're a maker or a taker. So what I'm wondering is, why don't we have a better way of talking about these issues? Do we need something akin to like a bureau of advanced labor statistics? Are there people in government doing this good work and people just aren't talking about it? Well, if you talk to people in LA, I live in New York, but if you do talk to people in the Treasury Department, for example, they're actually pretty darn sophisticated about something else they're doing, but it's different when you have to communicate that to the public, I think. But you do have, for example, the work that Ezra Klein does at the Washington Post in some way is a model of that where it's a more data-driven approach to things and some of the economics bloggers. I find myself actually doing a little bit less of that in part because other people do it well, but I do think it becomes trickier when you have these problems that are more open-ended. So I guess it is kind of a, polling is a simple set of facts, like I've said.  But other things like, what will the effect of Obamacare be on the healthcare system is a more, it's very good to have data-driven approaches, but it's a more complicated set of facts. In some ways, it cop out for me to say, here's a field where it's easy to do this stuff and there's not as much ambiguity and I'll let Ezra, everyone else, do the tough stuff instead. But there's no doubt that, I mean, that would be really valuable. I think it's probably a little bit optimistic to assume that that's what people are really interested in. I think there's a market for data-driven college of elections because it seems like it's actually telling you more of the essence of something than you would have gotten before. But it's hard to, I think, to separate out your partisan desires, maybe, from your analysis. If you're in the policy arena because the set of facts are more ambiguous, potentially. But in some ways, also, you do have good debates about some of these topics in academic circles and I'm someone who, you know, I try and read academic journals as well as the Wall Street Journal, I guess, and there are blogs like The Monkey Cage, for example, that are trying to take academic political science and interpret it in ways that the journals are gonna read. I very much support those efforts as well. Yeah. Hi, my name is Ethan Wolfman and I'm a journalist for USA Today. I, you know, you said earlier, people love to build kind of stories from very scant data and fine patterns and stuff. How do we make sure that our politicians don't do this and why does this happen? And if I can rephrase the question. Or do you have any interest in politics?  Yeah, I think one of the, one thing that's maybe under remarked upon is how the professionalization of politics, which is not anything new, but has been happening for a long time. But the average tenure now of a Senator or a Congressperson is as long as a Supreme Court Justice where it's like 18 years or something on average. And you have people who have become more and more sophisticated in manipulating public opinion, I suppose. And it's kind of the downside, I guess, to big data. Where on the one hand you can say, well, it's voter targeting stuff that Obama did is really admirable from the standpoint of solving a difficult data challenge. But it's also kind of creepy. Maybe potentially too, where they know that much about you based on your name and kind of what websites you're looking at and whatever else. And so I think there is a real downside to that or maybe politics has become kind of too efficient for its own good, where they're very good at optimizing their actions relative to voter opinion in the near term. And as a result, long term problems become more and more neglected. People are better about realizing what do we have to do to get reelected every two years or four years or six years? And that might have negative externalities, I guess, for the rest of society. Is it possible that as the professionalization of politics gets codified even more, that voters then will start to realize the effects of going after those short-term payoffs and we'll start to develop a longer-term view? I mean, could that- Potentially, I mean, this is a very, this is a very kind of, you know, David Bricksy and- Yeah, right. No, we should just stop things right there then. And if some kind of gets up and, you know, and says, well, let's build a policy based on sustainability about the national debt and about the environment in America's place then. Well, I mean, that's very compelling to me, right? But I don't know. I mean, you haven't really seen it. I don't think either party has been very courageous, I suppose, right? But, you know, so you now have Chris Christie, for example, who is disinvited from this- From CPAC. For instance. I mean, that's one of the ironic things is Chris Christie is a guy who probably right now is the second most popular active politician in America after Hillary Clinton, I mean, that's pretty close, but probably can't get his party's nomination, you know? And that's interesting. And you seem to think that he, that you don't place a lot of stock in independent campaigns for president, but you seem to think that he's a possibility. If he decided early on that he wanted to do it and he has some, first of all, again, so to have someone who is that popular, but couldn't get nominated, it's a fairly rare overlap.  So you probably would need, I think, financial backing, whether you like it or not, right? You probably want a super PAC to back you.  So I know the history of this is always kind of the journalist's pipe dream of having an independent candidate or another three-way election like you had in 92, but it does seem like if he decided today that I want to run a serious third party candidate for the presidency in 2016, then he'd have a shot at winning, right? Maybe not as good as it might seem, but, you know, it's a viable path, and at some point it might become a more viable path for him than trying to win the Republican primary. Right. Well, certainly right now it seems like there's, I mean, not getting invited to CPAC is not a song. And I think it's, he really is, and this is part of the point of the column, is that he really is a moderate on quite a few issues. See, we've now written this a couple of times, versions of it, and you've had, you know, Democrats push back and say, well, he's not, you know, conservative on these and that issues, well, yeah, it works, right? He, on some issues, he's right. He's center, or center-left on some something.  And he did, but, you know, one thing about Romney is that Romney was always pretty, it was kind of an interesting experiment in what happened if you had no long-term consistency at all in your policies, right? Like, maybe it's a dual equilibrium situation where you should either be really consistent or not consistent at all, right? Because then there's no cost anymore to being able to flip flop, or when people already have priced that in, right? Right, right. Whereas Christie's brand is more, he's someone who speaks his mind and has a lot of integrity, and so ironically, there's more of a cost for him to flip flop later on and cater to the Republican base because he has a reputation for not doing so. Right. Which side were we, we're here, yeah. Hi, I'm Jack Liberson, I'm a grad student in the financial economics program. I was wondering, Nate, as more young people have cell phones and not landlines, how well you think traditional phone polling is going to be able to adjust for those changes and whether online polling or something else is going to supplant it in the future. I think polling has already built that in. Yeah, so most of the New York Times poll and the NBC poll, most of the credible mainstream media polls do include cell phones. You know, the Rasmussen's and public policy pollings of the world do not and they by and large did not do very well. And do you wait the, do you wait polls that include cell phone surveys differently? Yeah, so that's part of the secret sauce. Well, it's disclosed somewhere in some article. Right, but the relative waiting. So one thing we do is we adjust, we calculate pollster house effects, which is kind of what does the average pollster say, right? And if you have a poll who's consistently two points more Republican leaning in the average, you just kind of take those two points back out. But the question is what do you mean by average? It's not actually the average, but it's an average which is weighted based on different criteria, how well the pollsters are doing in the long term and that consensus only uses telephone polls that include cell phones, right? So you're looking at how do these polls compare relative to these kind of premium polls that are expensive and that's where you adjust everything else to. So that's where you have an implicit cell phone adjustment, but you actually did have a number of polls that were conducted online this past year that actually did better than anything else. And if you look at the graph. Better than any of the telephone polling. Better than any of the telephone polling that included cell phones was fairly good for the most part, but the online polls were almost eerily accurate in a number of cases. So if you kind of look at the graph of what's internet penetration versus what is landline penetration, right? Well, those wires crossed a couple of years ago. So I would think the equilibrium that you might have in the long term is that you'll have, instead of having these robopoles like Rasmussen and PPP, you'll have, you'll still have a role for expensive traditional telephone polling and you'll get more and more expensive to have to call more cell phones and you'll have that supplemented by kind of cheap internet online polls instead. Although one issue also is that, is that sometimes you say, well, for example, these robopoles that are using automated scripts, they had a bad 2012. They haven't done that badly over the long run, but there's some evidence that they actually calibrate themselves off the good polls. So they're freeriding a bit. So that's another issue where we have to probably introduce some assumptions about what's gonna lead to better poll quality in the long term when you're weighing polls and not just assume that you can't be clearly results oriented about that for that freeriding reason. Yes? Hi, Nate. My name's Nick. I'm a big fan of yours, going back to the BP days. So it's been great to see all the mainstream success. I remember when Pocota first- Base college and mainstream, I thought it was a new day where when Pocota first started out, it outperformed the other projection systems pretty handily. And as time went by, the gap is kind of closed and a lot of the innovations of Pocota are now being used in other systems. Zips will use percentile forecasts and comparable players. And to be fair, a lot of that happened after you left Pocota, but the gap is, they're all pretty much the same now. So I was wondering if you worry about a similar thing happening to 538 that the other guys will start to catch on. I've heard it said that's next election. Every media outlet is gonna have their own Nate Silver. And so I wonder if it'll be, where is your place in that? Oh, sure. Now you're gonna have, and this is why maybe I'll get into neighborhood prediction instead of, but there's no doubt that you're gonna have people who are catching up. And there already were like a number of models that were pretty good, right? That had 98% of the same DNA as the 538 model did. And when you say 98% of the same DNA, you mean the results tracked 98% of the time? The results were similar, but also because the process for evaluating data was. The methodology. The methodology was, even just kind of taking a, if you look at, as you add complexity to a model, how the accuracy tracks, right? You do encounter diminishing returns pretty quickly, right? Where there's some maximum on how accurate you can even be intrinsically. And I think the 538 model does a lot of things well, if I thought someone else had a better approach than I duplicated myself, right? But you're talking about fairly marginal differences and you did see that with the Coda. And so really it's like, look, if I'm competing against the Bloomberg model, the Washington Post model and everyone else's model, then it's not gonna do that much better over the long term, which is why I like this confrontation against the kind of mainstream pundits who are just terrible at what they do. It's really, it's very easy to be really bad at things, right? It's less than a poker player, right? It's really easy to be incredibly bad at poker, right? You can be really, really dumb if you're on the McLaughlin group every week, but it's hard to be that much better than other people. So no, but it is a reason why- So you have a vested interest in bad pundits keeping their jobs. Well, it is, and it's another poker player thing, right? But you'd have some poker players who'd always kind of berate the fish, right? Like, why'd you play that hand that way? You didn't have the odd straw, you're flushed. Like, well, you make money from that. In the long term, when people are making decisions badly, so it should probably be really nice to political. Right. And stuff. If they go out of business, then the competition's gonna get tougher and, yeah. Well, that relates a little bit to, it's now 30 years since Bill James introduced cyber-metric analysis to baseball. He's still extremely popular and still does a lot of writing, not because what he's doing is unique, but because of his voice and the narrative that he brings to the data that he analyzes. And I think that's, I mean, and Bill James is kind of a hero of mine. That's great. You know, the one thing I do worry about sometimes is that if I have them myself, my voice, something that'll be seen now is like too authoritative, right? Talk about this probably a little bit before where with the times, it's such a, it gets echoed so much, it's hard to kind of whisper something quietly and say, oh, here's kind of an interesting thing I found in the data, right? It gets blown up and magnified in ways that it's not intended sometimes. So with some of this buzz that surrounded what 538 did in 2012, and I worry it's kind of hard to kind of tiptoe into something more, when that's a lot of the way that- So if we see another, a new really interesting blog that's applying statistical analysis to something and written by Chili Pepper, we should wonder what you're up to. Maybe, yeah, yeah, yeah, right? But you should pay more attention to people who are innovating in different ways. You know, I mean, that really is that people talk about, well, what's the kind of the money ball approach?  And actually, I mean, the misinterpretation of money ball as being on base percentage is really important, led to on base percentage being overvalued. Oh yeah, and you saw shifts where there was one year where Billy Bean talked about, he's drafted like a ton of high school pictures, right? His equilibrium had shifted so much. And so I guess that is kind of where you, you know, I talked before about how 538 involves an element of media criticism, but to be competitive, you kind of have to have a sense for what your competition is doing, including kind of what they're doing wrong. And they're definitely cases where I found people are acting much too certain about an election outcome. People are putting too much faith in models. So, but no, I expect that space to get a lot more competitive. And like I said, you had a number of very good models already in 2012 that were doing 98% of what 538 was doing and in some cases were just as successful. Right. I think we'll take a couple of more questions and then probably wrap things up and Nate will be out in the lobby selling books. Oh, okay, Nate will not be out in the lobby. Nate will stay here and so they should buy the books out there and then bring them here. Okay. Okay, most are gone. So if you didn't get one already, too late. Yes. Hi, my name is John Trulev. I'm an MIT spouse and big data geek. So my question is, I guess in kind of the story you told about your career is you kind of had these hobbies or interests that you kind of identified as being low hanging fruit for kind of rigorous analysis. But obviously, you know, it's not just that you kind of haphazardly, you know, fell into them. You know, there are some characteristics that you identified that these had. So I was wondering if you could kind of maybe describe the way you were able to identify these areas that kind of rigorous analysis could greatly improve them and maybe if, you know, some examples. Well, you do kind of have this chicken and egg thing where you become curious about something and it's that because of the kind of data environment it presents or because you're just kind of a baseball fan and then that flows from that. But yeah, I mean, I'm looking for areas that have kind of a medium level of predictability and a medium level of data quality sort of where, you know, the problems where everything, you know, so for example, many years ago it was problematic to predict the orbits of planets, right? Where some calculations had Jupiter crashing into Saturn every 50 years or something. And now that problem is solved when we think of it as being a prediction. You see a title table, it's technically a prediction of when the tide will peak or when sunrise in the sunset is, right? But so precise we think of it as being absolute fact now, right? And so that kind of thing is not that interesting because it's kind of been solved, right? And you have problems like, you know, like trying to improve on the margin on economic forecast, I think. We're probably, people have tried it for a very long time and you probably encounter diminishing returns, I think very quickly. So cases where for whatever reason the culture has been against using metrics the reason why I say well maybe cities are an interesting case and point is because you have often times urban bureaucracies that are probably not taking a very data driven approach to things. There's a lot of low hanging fruit potentially or education might be another area where you see more attempts to be quantitative and evaluating teachers and students but I have no idea whether those are productive and smartly applied or not. So yeah, it's a combination of looking for things where you have this medium level of predictability so it's kind of fun still. There are challenges to be solved but also that the market's doing a bad job by comparison. Yes. Hi Nate, I'm Michael Chung, a freshman here. As you've moved from saving metrics into election forecasting, at its base you're still analyzing lots of data but what are some of the most surprising differences between the two fields? You know, I think you get more vitriol from political people than from sports people. A good thing about... And when you say from political people to sports people, are you talking about people involved in politics and people involved in sports or political providers? You did have political journalists who were kind of anti-stethed. So I mean, maybe you have that same problem within the industry, I suppose but I do mean that, well first of all, people in sports, unless they're Skip Bayless or something like, don't take themselves quite as seriously, I don't think, right? Or Dan Shaughnessy. Yeah, yeah, yeah. But people I think have a sense of humor about it at the end of the day whereas political people take themselves so seriously a lot of the time and have no... Well, the stakes are higher. Well, the stakes are higher but not necessarily in, you know, there's not much, again, I'll just keep polling political for fun, right? But the stakes in, you know, whether the email to Bob Woodward was correctly characterized as a threat or as a pressure or whatever, there are no stakes at all in that, right? And there's not much sense of perspective in politics about, in political news coverage about is the story important or isn't it important, right? It's too much play by play, I guess, almost. But I'm sorry, what was the original question? I got... Just talking about some of the most surprising things we've experienced in this new area of forecasting. Yeah, but when you only have an election every four years or something, then it's more difficult, certainly. Well, you can't say you have midterm elections and other ways to test yourself, but that's part is you had actually in baseball this kind of money ball revolution happened really pretty quickly where you went from in 2002, the kind of period Michael Lewis was describing where you had the stat head was kind of locked in a clause that was some Cheetos or something, right? And internet connection to where kind of teams are having stat heads run their foreign offices now, including MIT graduates and so forth. And so that happened very quickly, but in sports you have this thing where it's very competitive, you have a fairly objective way to test success and failure, of course there is definitely luck involved in sports, but that combination's actually quite rare. So the money ball story is more an exception than the rule where you have the combination of things were done badly for a long time beforehand and you have good ways to incentivize and measure success. So you had very quick progress in that field where you might not in other domains as much. So one question that's come in a couple of times from outside of this room. I think among science writers and science communicators, there's a fair amount of thought slash anxiety about the best way we can reach the public. Are there lessons from what you do and the success you've been able to have that science writers can take away in terms of how we can use statistics or statistical analysis to communicate the stories that we're trying to tell more effectively? You know, I spoke earlier about having a lot of, having a lot of respect for the reader, right? And assuming that, you know, if you're not communicating something to the reader, then it's often your fault, right? I think also, if you have trouble communicating something effectively, let's vary abstract stuff like quantum physics or whatever, I'll give a pass to that, please, right? But if you're having trouble articulating what an argument is, right, then maybe it's not a very good argument to begin with. And so going through the process of writing and paying attention to your writing is important. You know, I think one thing that helps me a lot is someone who's actually been kind of somewhere between being a practitioner where you're building models, working with data and kind of a journalist and you're some of both. I mean, that skill set- You mean you're some of both, exactly, right? Yeah, so that skill set is kind of, it's kind of because of my odd background, but that helps a lot, I think. And it also means that when I talk to, like, scientists for my book, for example, I can talk to them a little bit more as a peer, and I think that- You can communicate in my language. And of course, believe me, there are plenty of things that go over my head, right? But still, you can have more of a conversation about, okay, tell me about how you're actually kind of playing with this data and what challenges you face. And there should be, I think, a lot more science journalism, but you know, I think having the, beginning with the inherent skepticism about, here's some big splashy conclusion that goes against conventional wisdom. Well, as a default, you should be skeptical of that conclusion, right? And you know, but it's not necessarily what sells books and so forth in the short moment. Does anything sell books? Going on The Daily Show sells books. See, I've been on The Daily Show and it did not sell my book. But it's, you know, all these different formats, books and newspapers and magazines have different issues, but I know, at some level you just have to, this is where I talk about the importance of kind of branding in the long term, but if you're someone who readers, so you do have a relationship where you can develop, or excuse me, you do have the opportunity to develop credibility in a relationship with the readers over the long term, right? And if you're someone who is seen as trying to get things right, then I think that does count for something in the long term. Right, I hope so, yes. Chris Tam here, I'm an undergraduate. Earlier you were discussing digital journalism and how the New York Times' expertise in that area kind of drew you toward them. Could you elaborate on both your experience and what innovations you see or would like to see that sort of separate an individual paper or person from their competitors? Well, I mean, there are a few things here and I've probably referred to a couple of these in passing, but number one is that the visual presentation of materials at times is really outstanding where if there were a Pulitzer for graphic journalism, they would win pretty much every year. I think it's the most underrated part of their operation, actually. It's incredible. It's super important, especially when you're taking, and the thing about the digital journalists and the reason why they insist and I insist on calling them journalists is because they understand that part of what you're doing is taking complex information and figuring out a way to convey the essence of that information in a way that's accurate. You're telling a story with data. You're telling a story with data. And sometimes they'll build things in a way that there's one way to look at it, but you can interact and play around with it as a reader in different ways and uncover more complexity. In some ways, I think it's a good model even for written content, right? Where you want to make sure that the basic story in line you're telling is true, that if someone just gets, it's just reading the article, skimming it and coming with one thought that the kind of elevator pitch they're getting is true, but also provide enough detail kind of in the footnotes or the links or the discussion at the end or the references to other pieces that people who want a richer experience can also get that as well. So they do that very well. I think they do very good about taking a digital first mentality in the event of breaking news. So when the Pope announced that he was resigning, for example, the front page of the Times had the Twitter feed from the Times' Rome Bureau chief, because they realized that was the best way to communicate things in a breaking news context instead of waiting for someone to file a story. So that's something I've been doing for more. The front page of the Times' homepage, not the web. Yeah, not the New York Times.  But the web is seen as leading the print edition out and they've gotten away from doing things where we're going to hold, there are some exceptions, but we're not going to hold this story online until the print edition comes out. It's usually reversed. And the third, like I said before, is people do have their individual brands within the Times, and that seems to be very important, where one unfortunate reality, I think, that I've discovered looking at traffic at 538 and at based on perspectives is that you really do, people do follow individual writers more than they follow the kind of umbrella of the brand. You can get some synergy, it's not zero, but you have to make sure that your individual contributors are really talented and looked after when they tend to do a job of a lot of people to have voice, even though you operate with a lot of editorial constraints, relative to being a blogger on your own, to have a strong pro-style and to trust their writers to get the story right in the end. And to be fair, in the end, they, for example, allow their writers a lot more leeway on Twitter than other organizations do. And as a result, their people tend to be followed a lot more and a lot more worth following on Twitter than other news organizations. So they get all that stuff, which a lot of places don't, I think. Did you read Snowfall, the Times package that they put together, and what did you think of that? This was a package that the Times put together about an avalanche that ran, I think, as its own section in the Sunday paper, but what was online was actually much richer than... And I think it's pretty amazing. Actually, the person who created that story was at the TED conference just yesterday, and she was talking about their process for your testing a lot of things and thinking about the user experience where does this multimedia content get too much in the way of the narrative of the story? So it's not a matter of just kind of putting up a bunch of interactive graphics, there's a craft there and thinking about how is this conveying information to the reader and how these things actually complement one another instead of just having different static elements that are kind of lumped together on an overly busy page. That's difficult, right? But the fact that they are thinking about this user experience in a way is important. All right, I think we have time for one more question. Hi, my name's Tom Pounds. I'm here as an alumnus of the Sloan School. Depending on what news outlets we favor, we could all get concerned to one degree or another about either voter fraud or voter suppression. Do you think the success of your result in the last election should give us confidence in our voting infrastructure and system and how much sleep do you lose over that issue? Well, you know, I mean, we talked about where are the social benefits of polling, right? And yeah, one of them is that in theory, you're serving with the average men or women on the street things, but another one is that it can serve as something of a check against massive electoral fraud potentially. I tend to think that these problems are pretty overstated by both parties and maybe the fact that the polling does do a pretty good job is partly a testament to this. But you have, we're going to complain about, for example, illegal immigrants registering to vote in North Carolina didn't audit on this and I think you had eight people, right? I have a state where you have five million voters. So it's non-zero, for instance. They're kind of often democratic conspiracies about what voting machines are doing. I think they're kind of often, Democrats are missing the low hanging fruit there, right? Well, how about people have to stand in line for three hours to vote? In Miami, right? Yeah, Miami or even in New York City, right? There's no shadowy conspiracy there, right? Just a tangible fact, right? Or for example, why do we even require people to register in advance at all? Where in states like Wisconsin or North Dakota we can register on the same day, turnout's quite a bit higher. Why do you have to go to a physical polling place to vote now? Where I can, from the comfort of my phone now, transfer $20,000 from my checking account to my savings account if I want, or unlimited amounts of money, right? But I have to go to the polling place to actually vote. So I think people sometimes love these conspiratorial stories about how the vote's been suppressed and misways that would make it a lot easier people to vote just in very elementary ABC kind of reforms. Well, I want to thank all of you for coming. A big thanks to Nate who arrived. Who arrived in Boston at six this morning after a red-eye flight from California, so he could be here. You were shockingly eloquent considering the amount of sleep you must have had. I'm traveling so much now that get excited. Tomorrow's March 1st, I think, right? So you have new in-flight magazines. I'm March 1st. This has been a great forum. Thank you all for the questions and we hope we see you later in the semester.