Welcome to Bogleheads On Investing, episode number 48. Today our special guest is Rob Berger. Rob is a litigating attorney before changing careers and following his passion by providing free online investment tools and a broad range of financial education. Hi everyone, my name is Rick Ferri and I'm the host of Bogleheads On Investing. This episode, as with all episodes, is brought to you by the John C. Bogle Center for Financial Literacy, a 501c3 nonprofit organization dedicated to helping people make better financial decisions. Visit our newly designed website at BogleCenter.net to find valuable information and to make a tax deductible contribution. And don't forget about our Bogleheads conference coming up this October 12th through the 14th featuring many speakers that I've heard on this podcast and more. There are a few seats remaining you don't want to miss out. Visit BogleCenter.net for more information. Today our special guest is Rob Berger. Rob was a litigating attorney in a previous life. He received his JD from Boston University in 1992 and worked as a litigator in the securities industry and the technology industry. In 2007, he started a website called Doeroller.net which received tens of millions of visitors. After selling that website in 2018, he became a Forbes deputy editor and he published a book called Retire Before Mom and Dad. Rob is now the host of The Financial Freedom Show. He publishes in-depth detailed reports on all of the free technology and low-cost technology that's out there to investors on his website, robberger.com. And he has also started the website for credit cards and banking called allcards.com. With no further ado, let me introduce Rob Berger. Welcome to Bogle's Uninvesting Podcast, Rob. Rick, thanks so much for having me. It's really great to have you on the podcast. You and I have known each other for many years and I know that you've interviewed me a few times and now it's my chance to interview you. But I've just always been amazed at all of the information that you've put out and have putting it out for 15 years, a long time. Before we get started on where people can find that information and what you've done, could you give us a little background on who you are? How did you decide to get into this line of work? Because it's not what you always did. So I was a lawyer in Washington, D.C. and at the time I started my first sort of personal finance site, I was actually an enforcement attorney at the Public Company Accounting Oversight Board which regulates the auditors of publicly traded companies. I've been investing for 20 years and I was very interested in really all aspects of personal finance and I said to my wife one day, I'm kind of bored with the practice of law. What should I do? She goes, we'll get a hobby. Out of that came a personal finance blog that at the time was just sort of a hobby. I enjoyed thinking about these issues, writing about them and over a number of years it turned into a business. I ended up retiring from the practice of law and then I did some work at Forbes and I published a book, Retired Before Mom and Dad and it ended up selling my original sites and I've started new ones in the YouTube channel and now I'm talking to you.   So I had sold Doe Roller. I sold Doe Roller and the Doe Roller Money Podcast and a couple of other sites in 2018 and literally just a few days after that I went up to New York, I'd been a contributor at Forbes. Actually, by the way, thanks to you, you introduced me to Janet Novak. Oh, okay, thank you. Right, so really this is all your fault. Okay. So we were up on Forbes Fifth Avenue location in New York and it was for contributors and I just talked to Janet afterward and I said, yeah, by the way, I sold Doe Roller and the next day she calls me and says, we want to start something called Forbes Advisor, will you be the managing editor? And at the time I thought I'm retired, I don't know what I'm going to do, but I had no plans to keep working and it just seemed like a fun thing to do. So I was the managing editor, my plan was to do it for a year, get it up and running and turn it over to someone else. I ended up doing it for two years and turning it over to some folks who I think are just fantastic. Well, that's great. That's a great story. I'm glad I could help out. I didn't realize that you became a deputy editor at Forbes. Now you have a website called robburger.com and you also have a financial freedom show on YouTube and you have another website called allcards.com and that website is about credit cards and you must be working 80 hours a week. Well, it's not quite that bad. Sometimes it feels like it. So they're each sort of their own little thing. So my idea with all cards, and I have a couple of people that helped me with that site, I want to come up with sort of a database of credit cards and bank accounts primarily that folks can go to and they can kind of see everything that's available rather than just, you know, a select few that you get from most credit card or banking sites. You can basically just about everything and that's taking some time but fortunately I've got a lot of help and in some ways it's just been kind of the thing that I do that's kind of fun because it involves some programming, nothing sophisticated for sure but the YouTube channel is probably where my heart is right now and the truth is, you know, I come up with a topic, I hit record, I record the video if it's not a live show and I publish it. There's no editing. There's no, you know, I can have a video done and published in an hour. You know, I might do some research depending on what the topic is ahead of time. I've had some great folks on. I had Bill Benken earlier this week who's the father of the 4% rule. That was a fascinating conversation. I had, I don't know if you know Harold Vinsky but he's sort of the father of the bucket strategy which Christine Benz talks a lot about at Morningstar. I had him on Wednesday. That was a fascinating conversation. By the way, it was a really interesting video with Harold and I'll get to it in a minute but you're also a competitive chess player so I want to hear about that. Yeah, it turns out I'm not a very good competitive player. Okay, okay. Yeah, I've played chess since college and in my less lucid moments I think I could become a master which would be a rating of 2,200. It was just over 2,000 and then my rating has taken a nosedive which by the way happens by and large when you get to our age, I think. In fact, I was in Philly over the 4th of July playing in the World Open getting beat by 8 and 9 year olds. Okay. Yeah. I've got, you can't see them behind me but I've got chess books, chess software, databases. I'm like the golfer who has all these gadgets to try to improve their swing but still shoots a 115 and that's me and chess pretty much. Yeah, I'm curious about what you just said though. You know, this cognitive decline after a certain age. Is there some chess data statistically where the average chess player as they age, this number declines? Is that out there somewhere? I couldn't point you to a study although I suspect they're out there but I can tell you certainly at the world championship level it's a young person's game. At some point Gary Kasparov got to an age where he couldn't keep up. I mean, obviously even today he's, it's like Michael Jordan. I'm going to guess he's still a pretty good basketball player, right? But probably not going to play in the NBA again. Gary, obviously a phenomenal chess player but as you age, it's hard to keep up with the younger players. Magnus Carlson is the world champion now. Still the best in the world but there are several players that are probably going to catch him when, who knows. In my case, you know, obviously I don't play at anything like that, of course. I don't know that it's a cognitive decline at least maybe I just try to convince myself that it's not. Okay, well that's my point. This is exactly what I'm getting at because of just what you said, okay? In fact, it is a cognitive decline but we don't want to admit it and I see it in myself. I've been trying to write a book for three years. Same book, right? Now I've written a lot of books, right? I'm not really able to bang out a book. It seems like every six months, boom, boom, boom. But here I am in my sixties and I'm trying to write this book and it's not a very complicated book but I'm really struggling and I can't tell whether or not I'm just trying to do a really, really good job or if I'm having some sort of a cognitive decline issue. Of course, I'd like to think that it's because I'm really trying to do a great job and really trying to be very clear in everything I say and maybe that is the case, you know? And I think this is an issue with people who are, let's say, in their sixties in my mid-sixties and we don't recognize cognitive decline for what it is and it's just interesting that you were talking about chess and talking about this because when it comes to your investment portfolio and investing, there's a lot of things that can help you with cognitive decline and one of them is keeping a portfolio very simple.  I used to have a three-bucket strategy but realized that that was actually overly complicated and now I'm going to more along the Harold Advansky view of just a two-bucket strategy. So you recall that conversation you had with Harold? Oh sure, absolutely. You know, what I see there is simplicity. In other words, for older investors the simpler you can make things. The simpler you can make portfolios. The better it is. When I first retired right before I went to Forbes and I thought I'm going to be living off my investments, the three-bucket strategy really appealed to me. Maybe it's a little more complicated than two but it's like, you know, how much more complicated can one extra bucket be, right? But when you actually start to sit down and figure out, okay, well wait a minute. When am I going to move stocks from bucket three to fixed income in bucket two? I think the three-bucket approach just doesn't, at least it doesn't work for me. Because I want to allocate my investments based on percentages. Not based on years of expenses which is effectively what the bucket strategy does. In a two-bucket strategy scenario like Harold described in the interview, yeah, the cash bucket is based on years of expenses but it's a very small component. It may be just one year of cash, for example.  So, yeah, it is simpler, the two-bucket strategy, but I'm still not convinced that, frankly, a three-bucket strategy works for most people because it changes the way you do your asset allocation and I think that can lead to some problems. Particularly depending on your withdrawal strategy. Yeah, initially the bucket approach was what was called the Tobin separation rule, right? Tobin was a Nobel laureate and he basically said you have your risky assets, you have your non-risky assets. I mean the two buckets. I didn't call it the bucket strategy back then but that's what it was. So your risky assets generally were equity but can be a balanced portfolio of stocks and bonds. And the difference between the three-bucket approach and the two-bucket approach I think is that allocation to stocks and bonds in the two-bucket approach is constant. It doesn't change. And the only thing that changes is moving money from that riskier longer-term bucket over to the cash bucket that you would need to live off of. But it was funny that you said something. You said, gee, I really am in line with a lot of the things that you were telling me. This is what you said to Harold. He said, I don't want people to think that I'm stealing your ideas.  Harold's been in the business a long time. And of course Jack Bogle was in the business for how many years? Longer than us. Probably twice as long as us. But as you get older or as you get more experienced, you become simpler in the way you think about things. And being simpler becomes much more helpful.  I think that what happens is that as we get older and more experienced and with or without cognitive decline, it doesn't matter. We just gravitate towards things that are simpler. Well, it absolutely describes my investing story. When I first started truly understanding index funds and asset allocation, of course your book all about asset allocation I think is one of the best out there. I didn't pay you for that by the way. That was unsolicited. But thank you for that show. I appreciate it. Unsolicited endorsement. Back in the day, I can remember just being so upset that Vanguard didn't offer all of the really cool index funds that DFA offered. And that to get dimensional funds and you know, and by the way, one of the founders, Sinkfeld, is a big supporter of chess. I don't know if you know that connection. Oh, no, I did not. He's funded the chess club in St. Louis and the Hall of Fame and he has a Sinkfeld Cup every year big time. But I remember being so upset because I wasn't going to pay an advisor to get access to these funds. Now they have some ETFs today. And I'm trying to figure out how can I get these cool funds with different factors, international small cap value and all these things. And today fast forward 25 years or whatever. It's the last thing I would do and it's not because there's necessarily anything wrong with doing that. It's just that I don't need the aggravation and there's no way to know if it's going to improve your returns or your volatility. I mean maybe make some educated guesses we all can but yeah, so I've absolutely moved from more complexity to less. And that's true even recently. I've been thinking, do I really need an emerging market fund? Do I really need a REIT fund? I mean again, good arguments to keep them, to have them. I certainly wouldn't talk anyone out of having a separate REIT fund. But boy it simplifies the portfolio when you don't have all those things. And the other thing Rick, that could be part of it. There's aging, right? You kind of wonder how that influences this. But the other thing is when you're starting out, maybe you've got a 401k maybe you've got an IRA. Well, today it feels like my wife and I have 47 different accounts. Right? It's not that many, but we've each got a rollover. We've each got a Roth. I've got an inherited. We've got two HSAs. We've got two taxable accounts. That's a whole other story. It's almost like playing three-dimensional chess. So when you're trying to balance all of that having a simpler portfolio I think helps a lot. I really like that analogy with three-dimensional chess. Not only do you have a lot of individual investments in each of these individual accounts because each of the accounts have to have investments. You've got all these different accounts because of taxes you can't combine your Roth with your wife's Roth and you can't combine an inherited IRA with a traditional IRA. I mean, so you've got all these different accounts. But usually if you have HSAs and 401ks they even have different custodians. So you're talking about three different levels, right? You have custodians, you have accounts, you have individual investments. I mean it really is complex. It's complex enough where, gee, you know, I want to make it as simple as I can wherever I can. Combine custodians and get rid of some that I don't need. Combine accounts if I can roll a traditional IRA into 401k that I might still be using then that's what I can do. And within the investment side, you know, try to have fewer and fewer investments to me. I mean, I think the three-dimensional chess is a really good analogy for let's make it as one dimensional as we can. But it's impossible.  And I think that's some of the tax regulation causes you to not be able to do that. And we'll get to more of this in a minute and how to organize portfolios when we start talking about all the tools and all of the work you've done to look at all these different tools and calculators that are on the internet. Some for free, some with a little bit of cost. And all where you can find all this information on your website and other places. But before we do that, you wrote a book. And the book was called Retire Before Mom and Dad, The Simple Numbers Between a Lifetime and Financial Freedom. Thank you for sending me a copy. I read it. I thought it was very good. Tell me about the book. What motivated you to write it and what is it about? So it started out actually, my original plan was it to focus entirely on compounding. I was going to call it money math and someone pointed out that most people don't like math. That's true. And then I thought, you know, I really want young people to read it before they wake up and find themselves in their 40s and 50s and having not, you know, set aside money. And so I called it retire before mom and dad. And it kind of does take, I suppose, a young person's perspective. Although it's not really about early retirement. Although it certainly walks through the math. It's really about all the things you and I talk about. Spending less than you make. Investing the difference in a very simplified, low cost index fund portfolio. And then getting out of the way, you know, and letting it do its thing.  I challenge, you know, talk about, you know, thinking about your spending, what brings you happiness, how to test through life experiments, what you think brings you happiness, which I've had on different occasions, an opportunity to do. And so that's basically the book in a nutshell. And I thought the title, by the way, is what caught my eye. So you were right in changing the title, retire before mom and dad. When I saw that, I said that's a good title. It's catchy. It's better than investment math. Money math. Anyway, the name of the book is retire before mom and dad. The simple numbers behind a lifetime of financial freedom. Okay, let's get into the nuts and bolts of why I really wanted you on. First of all, thank you, by the way, for participating in the Bogleheads conference, which is coming up this October. You're going to be one of our presenters. You're going to be on a panel and I appreciate you coming to the conference and telling everyone about all the great things that you've done and a lot of your knowledge and helping to spread your knowledge among the Bogleheads. The big draw is all of the financial tool reviews that you have on robburger.com. I mean, literally, it is a treasure chest of information, reviews on all of these different software programs, robo-advisors on and on and on, all the different categories. I mean, you have spent in an ordinate amount of time looking at all this and also I know that you've looked at the Bogleheads Wiki and the tools and calculators on the Bogleheads Wiki. So first of all, could you tell me about your obsession with tools and calculators? And then we'll talk about some. Let me first say in terms of the Boglehead site and the Wiki I don't think there's anything better. I mean, in terms of just the information it's phenomenal. The amount of work they've put into that is just extraordinary. And I find myself going back to it time and again. I was just looking at it the other day in terms of withdrawal strategies. So in terms of tools, I'm kind of a tool junkie. I find the technology fascinating. Before I was a lawyer in the securities field, I litigated technology related cases for about 10 or 15 years. And I just, I find it fascinating and there's plenty more that I want to put on my site in many ways. I've got far more work ahead of me. And I kind of want to go beyond tools too. I want to have a lot of detail on withdrawal strategies. Frankly, a lot of detail on the different lazy portfolios. Because as much as I'd like to tell folks, well, just do the three fun and you're done. I know that some people just want some spice, I guess, in their portfolio. Icing on the cake. Yeah. In terms of the tools in fact, I was just reworking a page on my site where I basically listed the tools that I tend to use on a regular basis. Happy to talk about them. You mentioned robo-advisors. I certainly have some strong opinions about which ones I like. But I also cover things like retirement planning tools, calculators just sort of investing tools. Like how do you know what your asset allocation is? Not just what your investment fees are but what effect they'll have on your portfolio over time. Well, let's start with the first group. Let's start with the retirement planning calculators because then that folds into asset allocation and then we can talk about robo-advisors. Let's start with retirement planning tools. What's available out there? What's free? What's the low cost? Start with those. In terms of retirement planning, that's free. One would be personal capital which we'll probably talk about a couple of different contexts. People have to understand what personal capital is. Personal capital is a registered investment advisor. Their business is managing your assets for a percent of AUM. Frankly, as a marketing tool, they've built a pretty nice financial tool that's free. You don't have to be a client to use it. I've used it since they launched it and I'm not a client. Basically, you connect all of your accounts or you can. Not just investment accounts but you can connect bank accounts all your debt. If you have debt, you can actually connect the value of your home. They look up the price via Zillow for you. It does a little of everything. It shows you your net worth. You can use it as a budget spending tracker. It will automatically categorize most of your spending. Some of it, it doesn't know the category for but it does a pretty good job. They have a pretty decent retirement planner based on the accounts you've connected. You can make certain assumptions. Of course, when you're going to retire, assumptions about how long you're going to live, inflation assumptions, if you're married, you can factor that in. You can factor in expenses and income in retirement. Maybe you're going to take a trip around the world. You can put that in with a date. In terms of a free retirement planning tool, particularly if you're still in the accumulation phase, you haven't actually retired and you're just wanting to get a bird's-eye view of how you're doing, I think it's a pretty good tool. There's another one I like a lot that's free. It's got a very specific purpose. It's called FICAL, so F-I-C-A-L-C dot app. That's the website. What it allows you to do is put in how long your retirement is going to be, how much you have, how much you're going to spend, what your asset allocation is. It's very simple. Basically stocks, bonds, and cash. Then you can select from, I don't know, maybe about a dozen withdrawal strategies and then using historical data going back to 1926, which is effectively what Bill Bingen did in his 94 paper that gave us the 4% rule, it will crunch the numbers and tell you your chances of success. Success being not running out of money, right? But the thing that I really like about this tool is that it will show you for all of the years, starting in 1926, would you run out of money or not, but it'll also show you those 30 years scenarios, if that's how long you've put in for your planned retirement, where you didn't run out of money, but you almost ran out of money. Because I think that's as important as just quote unquote success. I'm not sure I'd call success on your 30 dying with zero dollars, because that would have been a hard, it would have been a hard last year. It will show you those periods where you didn't run out of money, but boy, you're close. And you can actually click on each year and it'll give you details about how that retirement would have looked at you, for example with the assumptions you've put in and the withdrawal strategy you've chosen and retired in whatever, 1968 or whatever year you want, how that would have worked out. And by the way, in the different withdrawal strategies, they include the Vanguard dynamic spending strategy. explain what dynamic spending is? Yeah, so if you think about the 4% rule, the way Bill Bingen tested it was, you start with a certain percentage of your portfolio in year one, let's just assume 4%. And then every year thereafter, the 4% is irrelevant. You just adjust the previous year's distribution by the rate of inflation. So that would be considered a static spending rule, static on an after-inflation basis because you're effectively spending the same amount of money on an after-inflation basis throughout retirement. Basically just about every other spending strategy is dynamic, right? Again dynamic meaning the amounts changing on an after-inflation basis and on one extreme would be a fixed percentage. So you're just going to take out 5% every year. And that's dynamic because well, think of it this year, you know, let the markets down 20%, let's say it ends the year that way. If you take your 5% this year, it's going to be a lot lower than the 5% you took last year. So that would be sort of an extreme case. And frankly, the static approach that Bill Bingen took, I personally view as an extreme approach to retirement spending as well. They're just sort of on opposite ends of the spectrum. The 4% rule or any percentage, I've always had an issue with that because my expenses going up with inflation and if my 4% is less than my expenses, that doesn't work. I still have to take the money out and pay my expenses. Again the 4% only applies to the first year, right? Okay, that's what I wanted to get across. Great. And then you adjust the inflation. He believes with a somewhat more complicated portfolio, by the way, small cap, mid cap, even micro cap, it's 4.7%. Yeah, well, good luck with that. Well, look, I'm just reporting what I've heard. Fair enough. Okay, so we've got two now, two different programs. Is there a third one that you might like for retirement planning? The most robust tool for consumers would be new retirement. From a planner perspective, they're going to use MoneyGuidePro or eMoneyAdvisor and I've worked with both of those tools. But those are generally not available to consumers unless they have an planner that they're working with. New retirement is, I think, comparable to both of those tools and it's a very robust tool. It's not the kind of thing where you can just spend five minutes with it and have results because it walks through pretty much everything that would be relevant to spending during retirement. So it's going to walk through long-term care insurance. It's going to walk through social security questions. It's going to walk through Medicare and estimate your Medicare premiums and medical costs. You're going to enter your accounts, your investment accounts and you can link them and have that data pulled in automatically or you can manually put them in if you're more comfortable with that, which is what I do. You're going to set both pessimistic and optimistic return assumptions for every single account, which is useful if, like me, you really take advantage of asset location because my traditional retirement accounts, I don't expect the returns to be as high as my Roth and taxable accounts because of what I have invested in them. You can make those assumptions separate on an account level. You can walk through, if you expect to get an inheritance at some point, you can model that. Maybe you have passive income or some other sort of side income in retirement. So you can model all of these things. You can even model Roth conversions. And once you get done inputting all of this information, it gives you really a lot of both charts and tables showing you, you know, here's your chances of success. Again, success defined is not running out of money. And they use Monte Carlo analysis, but it gives you a wealth of information. It'll show you your tax brackets all the way through life expectancy, including when your RMDs kick in. It'll show you your state income tax. You can model, maybe you move, maybe you live in California when you're working and you're going to move to Texas or Tennessee for the taxes. When you retire, you can model that, including you can set separate inflation rates for real estate and model that home you're going to buy 15 years from now. And it'll change the tax implications, depending on where you're going to move from and to. So it really is, I mean, a feature rich tool, it is not free though. But I think the cost in my view is really reasonable. So they have three different tiers. The lowest, which gives you access to the tool is $120 a year, so it's $10 a month. You can go up from there. We actually have started offering planning with a certified financial planner if you want. Is that extra, obviously, or is that? Yeah, so that starts at $1,500 a year. But you can also do what they call Planner Plus Live. I feel like I'm a salesperson. I can also do Planner Plus Live, which is $360 and you get one-on-one coaching. For me, I just use the tool. But you know, for others, the coaching for a year might help, but for me, it's probably the most comprehensive retirement planning. It's really designed for when you're getting, I think, closer to retirement. It's not the only one out there. There are some other interesting ones, but new retirement's probably my favorite. The technology for do-it-yourself investors on the retirement planning side has really come a long way. I think you were saying most of the technology used to be MoneyGuidePro or eMoney, and that's for people in the industry. But with these new companies coming along, the ability to do-it-yourself or do-it-yourself investors for a very nominal fee is really exciting, I think. And I will say it's a hard market. I think selling to advisors, planners, is a much easier, I mean, it's competitive, of course. Yeah. How many people really want to dive into a tool like this? I mean, those that do are probably all listening to your podcast, Rick. I'm not sure, but... You know, you randomly ask a person on the street who's, say, within five years of retirement, hey, you want to go to a coffee shop and spend three hours looking at retirement projections? Probably not going to get a lot of yeses. But you know what? Maybe employers at some point would incorporate this in their HR departments. That's already happening. I can tell you that's already happening. Oh, that's great. All right, let's get into something that is an important part of retirement planning and financial planning, and that's in investment software, asset allocation software, portfolio management software. So this is kind of a part of it. You know, you've got to get a certain rate of return or you're looking for a certain rate of return on your money. So let's talk about the tools that are available out there for doing investing. The free ones. The free ones. And then things that might cost a little bit. I think most of these are free that come to mind. Well, I'll go back to personal capital because they have a very slick asset allocation feature that's got graphical elements to it where it shows you, particularly on the U.S. side. I'm not sure the international side is as robust as I recall. But it shows you basically your different asset classes in boxes and then in table form and you can click on a box and it drills in and it'll show you, you know, for your large cap value, here are the funds and what percentage each fund contributes to that asset class and you can draw all the way down. It'll show you sectors. And the other thing I like about it is they have a fee analyzer where, you know, of course it pulls in that it knows the fees of your funds and that's pulled in automatically. If you have an advisor that's charging an AUM fee, you can model, you can include that. Make assumptions if you're still working, what your contributions are and what if there's an employer match, you can actually model that.  That to me is probably what I'm talking to folks that don't appreciate the impact of fees. You know, it's hard to say, well, one percent, what's that, you know, but when you show them, well, this is the impact over the next 25 years, but they wake up, that gets their attention. So that's, I really like that tool. So that would be one. Anything else? Morningstar is kind of interesting right now. So I think in terms of just quickly understanding a fund, to me Morningstar is probably the best thing out there. It's really good. Is it, is there a cost to the level that you're talking at? Right now, no. So if you just want to evaluate a fund, you can put the ticker in Morningstar, it'll give you, you know, it'll give you the style box, right? So it'll show you with a small cap, large cap value growth, all of the, you know, give you all the valuation metrics, PE, price to book, price to sale, of course, it'll give you its expense ratio. It'll show you, I think they show for free the top 30 holdings, right? They have always had a portfolio tracker. At one point it was free. You had access to like the X-ray analysis and other things with the paid version. They're moving over to a new system. It's actually investor.morningstar.com. And they sort of up their game in terms of the user interface, because the old version was like, you know, 1990s called and it wanted its website back, right? I mean, it was really outdated. The new one is much better, but I still think it's a work in progress. We're still trying to work out the kinks and figure it all out. You know, you get a wealth of data, but at some point it looks like a lot of the features now are behind, you know, you have to pay. But still, I mean, an overall a good tool. Another one I like is portfolio visualizer. I don't know if you're familiar with that tool, but you can put in, it's not really to sort of track your investments, but you can put in a portfolio either based on asset classes or specific tickers. And it will show you the historical returns of that portfolio. You can compare multiple portfolios. It'll show you, you know, rolling averages, say five, 10 year rolling average returns, it'll show you standard deviation, of course, the compound annual growth rate. And they have some pre-programmed portfolios like they have the Bogleheads three fund portfolio where you just click a button and it fills in Vanguard funds and percentages. And they also have one guy named Rick Ferry. I don't know if you've seen it, but your four fund, your core four portfolios in there. Well, I actually use it on that core for a website. Another part of the tool is they have a Monte Carlo simulator. You could kind of it's kind of not unlike FI calc, but you can put a specific portfolio in there and model the drawdown. So you could put in like a million dollar portfolio or whatever. Assume, let's say a $45,000 initial withdrawal, adjusted for inflation every year, but put your specific portfolio in there. And it will model how that has done. And you can actually change a number of the different statistical assumptions. But for example, it can use historical data. And so I find that tool to be extremely helpful. Okay. Anything else out there for portfolio management and asset allocation? I will mention one other tool in large part because I'm so frustrated that like all of the other brokers don't offer this. I've used M1 Finance a bit. I don't have an account there now. But you can create a portfolio, very simple, low cost index fund portfolio. And invest your money through M1 Finance. And then when it comes time to rebalance, first of all, when you can make new contributions, they will automatically put it into the various funds to try to bring you back to whatever your planned allocation is. And you can also just click a button and they will automatically rebalance for you. Now, I probably wouldn't do that in a taxable account or not without understanding the consequences. But within a retirement account, boy, that's pretty nifty. And I'm kind of waiting for, I don't know, Vanguard, Fidelity, Schwab. It doesn't seem like it would be that complicated. And investment advisors have rebalancing tools. But as far as I know, M1 Finance is the only one that's actually made it available, well, other than robo advisors that do it for you. Let's talk about robo advisors for a bit. And then the last thing I want to talk about are all the tools on the Bogleheads Wiki calculators and tools. So let's talk about all the work that you've done on evaluating robo advisors. If someone decided they wanted to have another company manage their portfolio and do the rebalancing, could you go through your list of top three robo's? Yeah, and just for sort of perspective, I've used Wealthfront, I've used Betterment, and I've used Vanguard's digital advisory services. And then I've spent a lot of time studying Schwabs and Sofis and others. My personal favorite today is Betterment, and I use it. We invest our credit card rewards, so we get cashback or whatever. I don't spend it. We invest it. Part of that is to show people how small amounts of money invested over time can grow significantly. And it also allows me, I move it from time to time just to test out different platforms at the moment. It wasn't in one finance at the moment, it's at Betterment. And I just really like Betterment. I mean, it's got great tax loss harvesting tools, which I don't view tax loss harvesting as some massive addition to a portfolio. I think in some cases it could be. But it's there. They have an asset location tool if you have your retirement accounts there. I think they've got some great core portfolios. They have some other things I don't care for. But their core portfolio, I think, is good. And you can actually create your own. You're limited in the ETFs you can use. But for Vogelhead, they've got everything we could want. Speak a little bit more about the asset location too. Let that interest me, how they do that. They have an automated feature that when they're setting up your accounts, they'll do an asset location analysis and divide your investments based on an account type. I've read about it. Like I said, I haven't used it. So I can't speak from experience. But it seems like the kind of thing that technology should be able to do for us pretty easily. And I still am kind of marvel that it's not more widely available. But maybe it's available with other advisors. Betterment's the only one, at least that I know of. Wealthfront was recently sold, correct? And you had money with them. And how do you feel about Wealthfront and the fact that now they're owned by UBS? Yeah, I liked Wealthfront. I mean, I think it's a solid robo-advisor. It would probably be my top three, if I were going to come up with three. One could wonder what the sale, how that will affect it going forward. And I guess there's no way to really know. Maybe it'll have no effect. Maybe they'll invest more money in it, I don't know. All of these robo-advisors, they're trying to survive, right? And so I do see them going in directions that I personally, maybe getting into crypto or getting into the smart beta kind of thing as examples or really pushing direct indexing as Fidelity and Schwab are both doing that now. And I'm just not convinced that these are things that for most people are worthwhile. And in direct indexing, which you typically want to do in a taxable account, I talk to folks who decide later they want to leave whatever, well, once they have direct indexing for a while, they want to leave it and manage their own investments and they've got 120 positions they've got. At least. At least. Yeah. So you've got to be careful. I know in theory, direct indexing, I think has a lot of appeal in theory. But in the event, so you've got to be careful as you evaluate what they offer. But overall, I like Wealthfront. I was not as impressed with Vanguard's digital advisory services. Now this is the personal advisory service, but only basically the electronic portfolio management side. No advice, no speaking with somebody. Correct. Yeah, there were a lot of technical issues. I mean, to their credit, they were eventually resolved. But it took me two weeks just to get the account going because of technical issues. And then pieces of it didn't work. But again, to their credit, they got it fixed. It's not nearly as feature rich. I mean, in some ways, it's almost like a life strategy or target date fund wrapped in a cool user interface. More tax efficient, though, correct?  Yeah. That's an interesting question. I mean, as you know, Vanguard had some issues with the taxes on their target date funds. And I don't know if you saw they reached an agreement with Massachusetts on that. But it's a good question, Rick. I don't know if it's more tax efficient or not. OK, fair enough. Any other robos that you might like that you've looked at? Sofi is interesting. It's quote unquote free. But they use some proprietary indexes for some of their funds rather than the S&P 500. And with Schwab, you've got your required to keep so much in cash. So it's quote unquote free, but not really. You saw that settlement they had on that. Schwab settlement for holding a lot of money in cash. Yeah. So I think the issue was that they were representing that it was free and that their recommendations were based sort of in the best interest of the users. But whatever. I mean, I applaud companies for trying to make these tools available for quote unquote free. But you've got to be transparent. And you and I both know that holding six or 10 or 15% in cash in a portfolio is not free. Well, in addition, they also hold a lot of Schwab funds. Not that the funds that they hold are really expensive, but they do hold in the intelligent portfolio. They do hold a lot of Schwab ETFs. Yeah, the thing I look to, and maybe you do too, is one, are they based on a proprietary index or not? Because Fidelity Zero funds, for example, are based on. And it's not that they're necessarily bad funds, but they're based on a Fidelity Proprietary Index, right? Yeah, correct. I just feel more comfortable with the old S&P 500 or whatever sort of a well-recognized index. And then, yeah, you got to look at fees. I think for the most part, these ETFs are pretty inexpensive. OK, let's get into the last thing I want to talk about, because I asked you to take a look at this. Is all of the tools that are available on the BogleHeads Wiki Tools and Calculators page just an incredible number of free tools? So have at it. Well, first of all, I would recommend that those folks listening are interested to go to it. I don't know if you can leave a link, but it's just if you, BogleHeads Wiki Tools and Calculators. And I mean, you could spend a lifetime, right? I mean, I don't even know where to begin.  A lot of the tools are some of them we've already talked about. Like, for example, backtesting, they mentioned portfolio visualizer as a good example. But they include things like a backtesting spreadsheet that the community put together. I've not personally used it, but it's on my list to use it. And that kind of thing is very unique to the BogleHeads community, and I would highly recommend it. But they have bond calculators. If that's something that listeners are interested in, they link to tools. And Vanguard actually has a good one on college savings. If you're at that stage of life where you're saving for, say, a child's education, they have things like on life expectancy and calculators there. I tend to avoid looking at those things. I'm not sure I want to know. But they have everything. They have a ton of loan and mortgage calculators. And again, what these are, these are not necessarily things built by the BogleHeads. Some things like that spreadsheet I mentioned are. But these are just links to other calculators. One, they have this one of my favorites that I actually haven't been to in a while. But it's called Carl's Mortgage Calculator. It's a simple mortgage calculator, but it's really easy to use. And if you're looking to figure out how much a mortgage is going to cost to buy in a home, that's excellent. So. Yeah, they actually have dozens and dozens and dozens of retirement savings calculators on here, free ones. And once that you can buy, they have a kind of a spreadsheet, which ones do which and so forth. But again, like you said, you could spend a lifetime on here because they have put in so much time. All volunteers have put in so much time putting this wiki page together in all the different aspects of the free calculators and free tools page that is just absolutely phenomenal. I mean, little anything that you're looking for this, if you don't know where to go first, to go to BogleHeads wiki tools and calculators is probably a good bet to go there. You'll find something.  If you're not, the thing I would say is, as you look through something like this great resource on the BogleHeads site, is just figure out what it is you need. We talk about a simple portfolio, I would keep the whole tools and calculator aspect as simple as possible to. And if free tools work for you, then why pay for it, right? And if you get some benefit out of maybe more advanced tools, then that's great too. But I mean, because at the end of the day, I probably only use a handful of two or three or four tools on a somewhat regular basis, just because it's all you need, really. And so what's the next step for you, Rob? I mean, you're continuing to build out your website, continuing to do the YouTube videos, going down the credit card alley and working on that as well.  What's the next step? I do have a couple of books that I want to write. Oh, okay, great. One about how to invest once you're retired. Investing in retirement.  It's nerve-wracking to start spending your money. And at least I found it to be so. No, no, it's true. I mean, I work with a lot of clients. In fact, I had a call today and this year where this client is transitioning from accumulation to distribution in retirement is the most nerve-wracking time of your life financially anyway, because you just don't know how it's going to work. And you've got a lot of questions and you could have substantial amount of assets and you're spending be very low and you're still afraid to make that leap. Yeah, and unfortunately in the industry, there are a lot of players that will try to profit from that fear. Yeah, I agree. And yeah, you don't encounter that when you're 20 or 30 and you're just saving, but when you start to retire, then you get folks will try to scare you into certain products. So yeah, it's much more challenging than I had anticipated. Any parting words for us, words of wisdom? Well, I do hope as many people as can come to the Mobile Head Conference. I'm really looking forward to that. And I think where people can find me, the YouTube channel is kind of where my heart is in terms of trying to produce content that helps folks. It's not fancy. I don't have like a production person and I don't even edit the videos. So like if I mess up, well, you get to see it all. But I do live shows as well. People ask me questions and I do my best to answer them. I think it's fun. I'm not sure if everyone else does. I don't know, but people keep showing up. Very good. Well, Rob, it's been a real pleasure having you on Bogle Heads On Investing. I'm certainly looking forward to seeing you in person in October. And thanks for joining us today. Thanks, Rick, really appreciate it. This concludes this edition of Bogle Heads On Investing. Join us each month as we interview a new guest. In the meantime, visit bogelcenter.net, bogelheads.org, the Bogle Heads Wiki, Bogle Heads Twitter. Listen live each week to Bogle Heads Live on Twitter Spaces, the Bogle Heads YouTube channel, Bogle Heads Facebook, Bogle Heads Reddit. Join one of your local Bogle Heads chapters and get others to join. Thanks for listening.


Pryddum ni weld yw hi ar gyfer gweithio'r wych yn i'ch ydw i'r ddweud. Felly, y t rpm i, mae'n Tryfod Ddechrau Yn ym Mhwych, na chyddo, phobwch chi'n gwybod yn iechyd o'r effaith o'r teimlo ond bethau'n ymgyflwyno'r fforddiau yng Nghyrchyddol, a'r Llyfr Eiddiol yn visitor. Mae'n ganddo i'n mynd i amlwydoedd butfyniais, ond rydych chi'n defnyddio ychydig iawn ar y cwrs o hynny, mae'n dweud rhai fod ychyn yn ôl maen nhwc. ac mae'r ddylch yn gwneud, a'n ymwneud yn ysgrifennu ar y rymd yn ymwneud, oedd yn ymwneud, wedi'u neud o'r bod yn ymwneud, ac yn ymwneud yn ymwneud yn ymwneud. Mae'r cyfnodd yn ymarfer o'r cyfnodd yn ymwneud, ymddai'n ffordd i chi'n fath o'r cyffredinol o'r cyfnodd i gael bod hynny'n eisiau gweld yn gweithio. Felly mae'r bydd ymwneud yn ymwneud, I want to start out by just asking you to consider one thing, which is I'm sure pretty much everyone here has probably made at least one chart. Some of you have probably made hundreds and hundreds, but how many times have you actually been in the data set that you're visualising? And this is something I've actually thought about for a really, really long time. In 2010, I started working for the statistics department of the International Organisation for Migration. I was working in the Iraq office trying to keep track of how many families had become refugees as a result of the war in Iraq, and how many were displaced within the country. But more important than just kind of simply counting how many of those families there were, I was trying to understand what those families needed, whether they needed water, whether they needed an education, food, what they needed to basically kind of stay alive. But those statistics also served a second purpose, which was to be able to go to donors and say, this is the money that we need to be able to kind of provide for those Iraqis. And I have a really, really important and slightly difficult confession to make, which is that I was really, really bad at my job. And that's not me being modest at all. You can look at this chart, which proves it. I made this chart. It comes from one of the reports that we published, and I have some big issues with it. It's actually surprising. The thing that bothers me about is not the fact that it's an ugly pie chart, because again, if you think about the people that are outside of this room, actually all around the world it's charts like this that people are producing in classrooms and in offices. The fact that it's 3D obviously is not great. It is plenty misleading. But there are other things that I think are really, really deeply problematic about it that kind of explain why it was that I was so bad at my job. The first of those is just simple geographies. So I wasn't actually based in Iraq. Like so many of the humanitarian organisations at the time, we were based in Jordan because the situation in Iraq was just so, so bad. And I think that geographical separation from the data that you're analysing is super, super important, and it's really easy for us to forget about it. It meant that I couldn't just sanity check some of the results that I was looking at. So let's say, for example, I kind of noticed that there was electricity problems in one part of the country. Without actually being there, it's really hard to understand whether those electricity problems are because of a short-term outage, or whether they were systemic within the country, or whether in fact actually people had just misunderstood the question that was being put to them and were talking about electrical products as opposed to electricity itself. So that geography was a big, big problem for me. There's another reason why I have a big problem with this chart. If you remove the labels, I could be depicting absolutely anything here whatsoever. I could be showing the types of underwear that women wear. I could be showing murder weapons, anything at all, because the actual visualisation is so divorced from the subject itself. And I know that the idea of kind of communicating something personal about the data might feel a bit alien to somebody. This is a really cheesy transition. Speaking of aliens, this is Lieutenant Commander Data from Star Trek. And Data, for those of you who don't know him, I feel like this is probably an audience with some Star Trek fans in it. Data is insanely smart. He can compute just about anything, and that's why he was such an asset to the captain. But there's only one thing that Data can't do, and that's understand human emotions. Except in season one, episode 13, Data's identical twin law shows up. And law is supposed to be some kind of upgrade on data because law has the emotion chip that Data doesn't have. But actually that's kind of a problem because while Data is super, super virtuous, law with his emotion chip is kind of manipulative and self-serving. And I actually think the fact that that was kind of depicted in that way reflects how a lot of people think about Data, that it's emotionless and that it's good for that reason. And that's kind of problematic for us because very often people who are working with Data don't really have an incentive to correct those people. It's great if people think that we're like these perfect humans with no kind of input and no biases in what we're doing. But I think that is actually really important to be honest and upfront about what it is that we're doing. There's no such thing as emotionless data visualisation because any visualisation is being visualised by a human who feels emotion, so we just need to kind of acknowledge that. We make choices about fonts, about colours, about scale, all the time. Now, reason number three, why I think I was really, really bad at my job is I think the most important reason, which is that the charts I was making were being shown to a couple dozen people, sometimes a hundred, it was mostly donors and colleagues. And actually they weren't being seen by the people that needed to see them the most, which is the Iraqis. So the Iraqis who had provided us with this data weren't being given any kind of mechanism to say, hang on a second, you've got that wrong. And I think this happens all the time. One aspect of this that was kind of relevant is that this report was eight megabytes in size. It was absolutely huge. And download speeds are not the same around the world. And the fact is, is that some Iraqis would have had to have waited for ages and ages in front of their screens to have just simply downloaded the thing. And that's even just the Iraqis that were speaking English. Some of the reports were translated into Arabic, but not all of them were. I can understand the reasons for that. We were working in an incredibly difficult environment. We needed to be efficient and the most important people that needed to see these charts were the ones who were giving us the money to just do our jobs. But at the same time, it meant that what we were doing lacks transparency. We weren't being transparent about the limitations of the data. We weren't being transparent about the limitations of the visualisation. We weren't being transparent about the limitations of us, the people that were actually putting this stuff together. You can't see my process here. You can't see whether I collected this data from questionnaires or face-to-face interviews. You can't see how many people I'm depicting.  And especially as I said, the fact that this was kind of just spat out in a PDF that didn't give Iraqis the opportunity to say you got it wrong. And again, that's super, super ironic when the people that you're visualising, you're saying these people are disempowered and you're not giving them the opportunity to kind of say you're wrong. So it's a really, really intense experience. And as I said, I was really, really bad at my job and I don't like being bad at my job. So in 2012, I decided to go through Iraq to find out if I was kind of getting my numbers right. It was... I've just jumped to the wrong part in this, sorry. It was pretty intense because I realised how little my charts were really, really communicating the reality that was there. They were completely out of touch. And so as a way of capturing something that felt less abstract, I started to take photographs that felt a lot more real to me of kind of like what life was like there. And I started to take photos all the time. I was taking hundreds and hundreds of them. And that kind of process of repetition meant that I started to notice patterns in some of these photographs and noticing patterns, as you know, often leads you to want to kind of visualise data. So eventually, I kind of turned these into charts, some of the photos that I took, and I turned them into charts and produced an exhibition of the photographic images in 2013. So I'm just going to talk you through a couple of the visuals that I've made in case they're interesting. This was a woman just kind of walking down the street in Baghdad, and I turned it into a bar chart. And what I'm actually displaying here is the proportion of women aged 15 to 49 who think that a husband has a justification for striking his wife. And I know that the terminology around this sounds kind of like quite bizarre, like that was the way that the question was worded. But I think it's really, really easy to forget the kind of cultural differences about the way that you ask a question. So in a country like Iraq, just asking people, you know, have you been abused by your husband to represent a massive intrusion into someone's private life, right? So the data you're going to get is going to be incredibly, incredibly flawed. So you have to be creative about ways that you kind of understand trends. So this data actually comes from the Iraqi census. And as you can see, one in five women think that a husband has a justification for striking his wife if she burns the food. Like really, really shocking data, but I just wanted to remember some of the actual people behind the data, and for me, using photographs was really important in order to do that. Similarly, this one is just some Iraqis in the north of Iraq. I've actually cropped the waxes out of this in case anyone wanted to have a guess of what it is that I'm visualising here. Can you see the line chart at the top? Does anyone want to visualise the case? Sorry. No, that would have actually probably been a better thing to visualise with this particular image. It's actually depicting foreign aid, so it's quite useful for seeing how the country was quickly... I wouldn't say forgotten, but some of that financial aid trailed off, and the idea of using electricity was because the economy was kind of running off of that financial aid at the time, but yeah, that would have been much better. This one here shows because obviously working for the international organisation for migration, I was kind of focusing all the time on the numbers of refugees, the number of people that were internally displaced within the country, so I kind of come back to Iraq. Again, the idea of something like this was to really, really remind people that this data represents humans and to fix it in a kind of geographic landscape of Iraq itself. This one was... I actually was helping someone who wanted to write a will. In Iraq, you go to these people out on the street. This was like a guy sitting underneath an umbrella, and you go and get one form from him, and then you get sent by another guy. This comes from the World Bank, and they don't actually publish this indicator anymore, but it just explains to you the number of steps that are needed to enforce a contract. I wanted to visualise it this way because when you're actually doing it, it's such a painful bureaucratic nightmare to get anything done. Every single one of these steps creates an opportunity for corruption. Actually, the endemic corruption in the country is really, really relevant for the statistics that we're looking at, and it's something that's so easy to forget. All the time, things like the percentage of Iraqis that had graduated from high school, literacy rates, numeracy rates, things like that. While I was there, I started speaking to this mother who explained to me that her daughter was falling behind in school, not because she necessarily wasn't doing very well, but because even the teachers in the schools accepted phone cards as bribes for giving the kids better grades. I'd been spending all of this time looking at this data set on school grades with no understanding of the way that that data had fax on the ground. Going there really, really helped me to understand the way that I wasn't, the missing gaps in some of the things that I was looking at. This last one here was an attempt for me to do a better pie chart. I'm not claiming, by the way, that any of these are very, very good. I looked back at them with the benefit of hindsight and think they're pretty bad in lots of ways, but this one showed that nine out of every 100 Iraqis their cause of death was suspected torture. The idea, again, was so that you couldn't look at a visualisation and loose sight of the people that were within it or the human side of things. Even if you remove the labels you still get a sense of subject, everything's fixed in a real time and a real place. The other purpose of these charts is that they're supposed to be very, very inclusive. The process here, you can see that someone has selected the images and it almost looks like a real collage that's been cut out and stuck down. Hopefully people can see a way that they could actually replicate my process. I think that's really, really important. I do not think that data visualisation is for geeks. I don't describe myself as a geek and I don't particularly like the label geek because I think it just creates a new community of insiders that can be quite exclusionary. It's really important for me that as many people look at my work as possible, which is the reason why I eventually moved into journalism and I'm not ashamed of the fact that I seek out readers and want a lot of readers because, as I said earlier on, those readers can check my work and tell me whether or not I'm writing things right. So I started working out of the Guardian's data blog and then I moved eventually to 538 over here in the US and while I was there, as Irene mentioned, I started writing a column called I Normal where the name eventually switched because we felt like it might sound like we're placing some kind of moral or ethical judgement on the readers by providing a response to those questions. The idea was that, A, I could not only get more input on the visualisations I was creating, which was super, super important, but I wanted to have input about the questions I should be asking, what data sets I should be looking at, what hypotheses were kind of out there. I want to talk about one of these columns in particular, I produced dozens of them and I still write it now, but for a different site. So I got this one question from a lady called Caroline who was 44 and living in Philadelphia at the time and she wrote to me and asked, Dear Mona, I recently read an article that said most of the prison population is religious. Well, there are very few atheists in prison. Please tell me if this is true for the United States. So as many journalists do, I kind of merrily found my data, I started analysing it and visualising it with the help of Reuben Fischerbaum and Alison McCann at 538. And we thought a lot about the best way to visualise this so should we be showing raw numbers and we eventually felt like in order to respond to this reader what they were really interesting was ratios so which population is kind of over or underrepresented in prison relative to the U.S. population as a whole. And as you can see, Pentecostal Christians underrepresented and then there are more in prison Muslims relative to the U.S. population as a whole and also Jewish inmates as well. So I published this data and as I always do at the end of my columns I invited readers to get in touch and they did. I had some former inmates who got in touch to explain to me that part of the reason for this data is that if you are in prison because these individuals had been sometimes you get access to better meals if you're eating kosher or halal as opposed to these standard prison meals I also had explained to me by former inmates that sometimes if you're members of certain religious groups you get extra recreational time which gets you out of your cell which is super important if you're in prison for 23 hours a day. So the ability for people to get in touch and explain to me the why is super, super important for what I do informs the theories I make. I'm not saying that my job is quite boring if I just relentlessly describe the what of things rather than understanding why things are happening which is why my inbox is really important to me and I'm going to share some of the questions I've got in my inbox that I think are quite interesting because they reveal again some of the things that the general public is thinking about data. So this one which was sent to me in 2015 do attractive people have more sex than ugly people and I think it's interesting because I could quite easily imagine throwing a blog post together that did really, really well and people just accepting that data on face value without any kind of interrogation of what exactly is meant by attractive and ugly and I think the need to kind of take a step back and explain the definition of terms and the methodology of the way that this stuff is super important. Here's another one. Am I the only one that doesn't use the designed opening in my underwear when peeing? I thought this was an interesting one. I assume it came from a man and as someone who doesn't own a penis I didn't think this question was interesting at all but I kind of shared it on social media and actually a lot of people do think this is interesting and a lot of people wanted to share their data with me which is quite nice because what I find interesting isn't necessarily the same topics as the people that I should be serving find interesting so to be able to check that is super important. This one is quite dark so someone asked me what the autoerotic asphyxiation success rate was and they're obviously quite keen for a certain answer because I'm sure the vast majority of times it's performed it's done successfully, right? You can kind of hear like the desire for reassurance down but this question actually got me thinking really really critically about your responsibilities as a data journalist let's say I was actually able to collect that data and let's say that that data showed that 90% of the time you're fine if you do this, you're totally fine how can I put that into a chart that communicates risk to a reader in a really really responsible manner so that people don't just see that chart and do something that is potentially quite dangerous to them and when you're doing health reporting these questions are really really critical because you could actually be shaping the kind of people's choices and then just really really simple ones like how much Peter's a lot of people so this last one leads me to kind of like the last bit of my very strange kind of annotated somewhat narcissistic resume which is the thing I'm describing right now but I want to talk about the kind of last iteration if you like in the way that I've tried to think about data visualisation so as you can see from these messages 538 readers are kind of special and this actually happens around most websites the internet doesn't create this kind of perfect flat space actually you get kind of clusters and communities around certain things so I write for the Guardian now there's a certain type of Guardian reader that's not necessarily the same as a 538 reader so I wanted to publish on something that felt a little bit more flat and could hopefully give me access to new audiences I also wanted to resist the urge to produce interactives all the time, all the time to do and I wasn't thinking critically about when I actually needed to ask the reader to kind of get involved and when I needed to just give them the kind of simple image and the kind of more simplistic story so I started to basically use Instagram because I thought it was a really really good way as well to the comments underneath the images are really really fantastic because they're transparent and it gives an opportunity to kind of have a discussion and a debate around that visualisation so here's one I did which was actually a response to this particular question how much pee is a lot of pee? I tried to use everyday objects whenever I can in order to convey scale I think that's super important not everyone necessarily grasps the same units of measurement we use different units of measurement and even if we use the same units of measurement sometimes especially when you're using large units it's really difficult to understand how much we're talking about but pretty much everyone has held a 1 litre bottle of liquid in their hands at some point and that can kind of help them to understand it speaking of scale again thinking critically about some of the work that I do I don't really think I need to have my hand in it it kind of makes it look like a mini basketball but the idea is to convey the relative size of the basketball and the hoop this is a slightly more serious one here that again is about scale so this is contrasting the average parking space in America with the average solitary confinement cell and what was quite good about this I think is that I got online abuse from both sides of the spectrum so I got abuse from people who were saying to me I didn't imply that prisoners have too much space and I had abuse from people claiming I was implying that prisoners had too little space which I think sometimes bodes quite well for the way that you've visualised visualised something I also want to use this as much as possible to again reach out to the community of non geeks so to show that data is relevant to everyone's lives in ways that might not be expected so there was a journalist in New York who did a Freedom of Information Act's request to find out how many decapitated animals there were in New York parks and I was able to visualise this and show which ones were kind of the most commonly found animals so there's a lot of headless chickens this is over a ten year time period but still kind of interesting and it wasn't always just heads it was sometimes just like the bodies and not the heads and we're finding like school playgrounds and this is very very bizarre but anyway super interesting and like got people thinking about like why this data is the way it is and we shared quite a lot which is for me always a good thing and then finally circumcision rates showing the benefits of kind of doing small multiples sometimes before finishing I also want to talk about some examples of what I think a good data journalism that aren't my own and good visualisations so again I want to come back to this idea of conveying the imprecision in what we do the fallibility of what we do and one way to do that is actually to just focus on one very very very very very very small data set in a way so this comes from this blog that some of you might know called the quantified break up I think that's what it's called it was a tumbler and it was written by a woman who was going through a divorce in New York and she visualised all kinds of aspects of her break up she visualised the messages that she was receiving the frequency, the time of day they were exchanging messages her and her ex how the relationship fizzled out she visualised how much sleep she was getting times that she just started crying in public and couldn't control herself and people really really loved it not just because it told a story but because I think it was really really honest in terms of it's claims and it's limitations this wasn't one woman trying to claim that she was representing divorce in America or divorce in New York or women in general she was just saying here's my story in the data and I think that transparency is actually something that's pretty great this is a very very different kind of example so this came from the New York Times written by Gregor Aish Kevin Keely and Amanda Cox and they basically asked people to draw the data themselves this charts the parents income versus the likelihood that their children will attend college so you kind of draw out the data and then you see how you compared to the actual reality it tells you a story as you're doing it and it invites you to interact in a way that feels super super inclusive and then you can kind of see how you compared to other people again it's just really really friendly just had great design and then the last one that I want to talk about is earlier this month the Guardian published I think was a pretty good analysis of 70 million comments that had been published on the website I don't know if any of you guys saw this and I think one reason why it was really really good was that it didn't overwhelm the viewer in terms of too much data it broke it down into several different slices so you could see how levels of abuse kind of varied by gender and by subject area what was also really really good about I think is that it allowed the reader to kind of take part in this little quiz where you could read a comment a real comment that had been left on the site and decide for yourself whether you would have blocked it and defined it as abuse or left it up and what that did was I think it really really helped to convey A a sense of scale because you realised that you kind of took a minute or two to decide whether or not this comment was abusive and then you're like whoa 70 million of these have been analysed that's a lot and it also showed you that even though you've been reading all of this what seems like a truly truly objective data looking at the scale of abuse once you read that comment you realise that every single one of those data points represents a small human decision about whether or not this is abuse it's based purely on subjectivity even if the whole altogether appears objective and is objective in some ways so I thought that was really really smart now I know I've kind of touched on a lot of disparate topics so I would like to close by kind of thinking of some ways that we can basically get better at what it is that we're doing so I would say that even though data visualisation has obviously made leaps and bounds and has progressed hugely in some ways our data literacy hasn't and never will we're kind of flawed human being so if I was to ask everyone here to say how many people they think are in this auditorium we would probably get very different responses on range and the bigger the scale the harder it is if you're in a huge huge concert venue it's really really hard to get to how many people are there and so we have a responsibility to communicate scale to readers in much better ways so even something as simple as 70 million comments trying to help people to understand how many that actually represents and there's all kinds of simple and complicated ways of doing that but I think it's really really important as a goal to bear in mind the other thing that I think is super important if I haven't made it kind of clear already is the notion of conversation so as I've said I think communicating and interacting with people makes me so so much better at what I do so Gorka used to do this thing that I don't think they actually do anymore where you could leave comments on photographs and when you hovered over you could see where everyone had kind of clustered together to have a conversation so if this was the photograph say you could see whether people were more interested in the President of the United States or a wealthy royal child and if you imagine this example translated into a chart where people were able to leave comments it would actually be incredibly powerful you would be able to see if everyone's having a conversation about one year in the data on a line chart or something or whether everyone's focused on one state in America of their conversation or even if everyone is just talking about one dot in your scatterplot maybe that's a point of inaccuracy in your data and you need to go back and change it and I think that's super super helpful for having a more more fruitful discussion that was when I was going to say about Obama and the Child now the last thing that I think is really important as well is that we just get better at communicating uncertainty that was kind of the idea of me starting to do things like hand drawings and using photographs it was about admitting fallibility better and showing that actually this isn't perfectly precise which I think some data visualisations do kind of communicate to people today in a way that I find slightly troubling sometimes so there's all kinds of ways that you can communicate the uncertainty whether it's just kind of plotting out things around an average or plotting out probabilities differently we just need to think more critically about it so the last thing I would say is that yes data should strive towards objectivity and fairness and visualisations should keep on trying to do that but we shouldn't be alienating people because if we are we're actually jeopardising the quality of what it is that we're doing and that the internet doesn't solve everything yes theoretically you do have access to a whole lot of people but you have to reach out to them you have to make sure that you're publishing in the same language as them that they are available to you regardless of their internet speeds and we you know in a very very concrete way we're actually unelected officials who are kind of representing people in our data and if we're not checking in on them to say have we got this right I think there's something pretty dark about that so thank you very much thanks for having me oh yes sorry it's time for questions right yeah if anyone has any questions for me so I think it's a really really great question in an ideal world I think I should be picking up the phone every single time I write an article because the spreadsheet just doesn't tell you everything very often that's incredibly difficult when you're in a newsroom with a kind of couple of hours of deadline and that sounds like a really really lame excuse but it's just kind of like the reality sometimes so I think the point is as you say to kind of have it be more iterative to have people get in touch and you know that in my defence this piece didn't claim to explain why it just like claimed to be showing this is the what but people are actually really interested in the why and so being able to go back to that piece and say this is what people had kind of told us but not just this is what people told us but then kind of making it investigative and finding out whether or not those reasons actually play out among a broader you know across prisons across the country I think is actually really really important and I think the nice thing about data journalism if it's done right is it kind of questions the notion of who are the experts right so generally when you're writing journalism you call up an academic and you say hey you're an expert on like prisons in the US tell us what's going on and that's still super super important but actually the individuals within the data are so well qualified to understand what it is that's going on and as long as you provide them with the tools to be able to get in touch with you it's incredibly powerful it's a constant challenge and I think I agree with you that these aren't perfect but the kind of idea behind them is to show their fallibility it's very obvious that if a human has hand drawn this it is not a perfect representation of the data set itself and if anything I think that they're intended for me to counter what it is that I'm doing in my day job 9-5 which is conveying precision with kind of quite frightening maybe I'm being a bit over the top so call it frightening but you know like I want people to question the data behind the visualisation more frequently so I don't know if that actually answers your question but I'm thinking about this stuff like it's all about communicating it's not saying these are perfect visualisations it's about showing that these are not perfect visualisations I actually think people are super super sceptical naturally and I think a lot of people do naturally ask some of the right questions I think source is always super super important so for example if polling data is presented increasingly now people are like was that poll 100 people was it based on 2,000 and they get that there's a difference between the two I would argue that even a poll that's based on 2,000 people is not necessarily the most accurate thing in the world and I still think that journalists need to do more to kind of communicate the weaknesses of things like polling to readers so I think understanding sample size is super super important understanding not just sample size because that's quite an easy example but sampling methods so let's say for example very often when a poll is published it will say a national representative sample of 1,000 people but sometimes that national representative sample for example included one black person who was weighted up to represent all of the black people in America and it's somehow national representative so I think being able to peel back even that top layer of the data and say there were 1,000 people here and say how many people were there in this data set that looked like me before I interpret this is super super important as well do we have time for a couple more if there were a couple more maybe it's not I really think one thing that was quite good was I don't know if you guys have seen 538 actually published the probability forecast for different candidates and by showing the kind of hump of all of the possibilities before showing the ones that they think are kind of probable and their kind of middle one it really communicates a range of different scenarios to the viewer which I think is quite effective I actually think that it's been kind of lagging we haven't really been thinking about it very much so we've got like a bit of ground to catch up on of ways that we can kind of communicate that uncertainty I think lots of people have done things that are based on like deaths so contrasting the number of deaths that have resulted from what thing A versus thing B over time the problem with all of that though is that it involves a lot of kind of personal choices about what you're going to make that contrast with so if you take for example the risks of guns in America people on both sides of the spectrum will just choose a different reference point in order to convey the scale of risk so I actually think one thing that can be quite informative is breaking down that risk by different demographic factors so for example showing something your times did the likelihood that you're going to end up in prison depending on your age and depending very importantly on your race so helping readers to kind of find their way into the data by presenting their particular demographic can be super important I think there's some really really small things that you can do so I think at all points during this conference it's come up at one point or another during everyone's everyone that's spoken is this idea of kind of staggering things and telling it to you in a story and where attention comes in that I think is really interesting is I don't know if you noticed but on that the slide that I had from the Guardian where it showed that chart it showed four of six and I think communicating to the reader listen like you're going to have to see another five of these or another three of these helps keep people kind of focused because it's not like oh my god is this going to be infinite I don't know what to call for I have five minutes before I have to get back to work or I'm hungry or whatever just being able to let them know how long they're going to have to save before you watch a video for example you hover over it and say am I going to be here for 20 minutes or for five minutes so I think that's one really really good way to keep people's attention to break it down into different sections and I think that's part of the problem with some interactives some interactives just demand way way way too much and sometimes actually they can just be split out into like several flat graphics that tell a much more compelling story and require less of the reader and I think the process question is really really important so there's a piece that I'm quite proud of that I did with my colleague Andrew Flowers at 538 and what we tried to do was we tried to find out what was the most common first and last name combination in America and what was nice about that is that every single part, every single step of your process tells you a story in and of itself so you start off with what are the most common first names and people are interested in that data kind of in and of itself and then you're like what are the most common surnames again people interested in that data in and of itself and then you talk about how if you assume that the probability is even for say any given first name in any given surname this is what the data looks like again kind of interesting and then you say actually when you use a phone book and look at things you can see how those and even probability isn't the way that life plays out because if you're surname is Smith you're probably not going to want to call your son John because it's just so boring so being able to explain that to readers every single step was inherently interesting in and of itself and I actually think that as practitioners you find that all the time as you're doing your steps you kind of find some interesting little nugget before you move on and I think remembering them and conveying them can really help keep people kind of with you throughout that process I think this idea of going to them rather than expecting them to come to you is really really important this is a very very weird example on pubic care and it was posted in this forum that is like feminist for pubic care justice or something I don't know why that is the first one that came into mind but I think if you go out into the communities that are really really passionate about the thing that you're doing and share it with them and say to them please come and please contribute to this discussion it shows them that you're not just kind of making the right noises you really do care I'm like a response to the people that write to me on my inbox and I think that's really really important and I go into the comments and I'll say hey that's an interesting point what about this so I just think showing them that you're actually willing to engage is really really important as well all the time all the time I'm trying to think of like just one example again I don't know why my mind is really really in the gutter today but one that came to me was someone said to me which seems like a completely plausible theory to me that because of smartphone usage more people have piles because they're sitting on the toilet for so long just on their phones and I was like that's interesting and yeah there's no data on piles versus smartphone usage but yeah all the time but John's really really nice I haven't talked about this at all but I think that we as data journalists don't do enough to actually do the process of collecting the data so that's one reason why I showed that kind of quantified break up thing so there's two examples where I did this and I can understand why we're reluctant to do it right because if I collect data from guardian readers those are guardian readers they are not representative of all of America in any way but I think you can communicate that to people really really honestly and in a way it's even more transparent because you say we asked guardian readers and people get that guardian readers are not representative of everyone so did it twice I did once I wrote an article on redheads and asked people whether or not they felt like redheads experienced discrimination I kind of just did this like little blog post and stepped away from my desk and when I came back I'd used a Google form it had completely crashed because 20,000 people who claim to be redheads had very very strong views on whether or not redhead discrimination exists and I did another one that asked people about their sex habits and the thing that's really really nice about that is it was so easy to communicate the limitations of that data because for example someone that entered in some data said that they lived at 10 Downing Street like you know you put that in the article people get it David Cameron probably didn't contribute data to my article and it's quite powerful because it still shows you okay we don't know anything about this phenomenon this is the best we can do in terms of right now just saying something about what's going on it's just fun it's just fun apart from anything else thank you