Welcome everyone to the 12th episode of Bogleheads On Investing. Today we have a special guest, Larry Svedro. Larry is the chief research officer of Buckingham Strategic Wealth and the author of 17 books on investing. Hi everyone, my name is Rick Ferri and I'm the host of Bogleheads On Investing. This episode is brought to you by the John C. Bogle Center for Financial Literacy, a 501C3 corporation. Today we have a special guest, my friend Larry Svedro. Larry and I have known each other for many years and we've had a lot of healthy debates. So with no further ado, let's bring Larry in. We are pleased today to have Larry Svedro as our guest. Welcome Larry. Great to be with you Rick. I'm really excited to have you on the show Larry. You and I have known each other for many years and we've gone back and forth on the forum on many different topics and everybody's got a different opinion and different view and you and I have had our days. But it's always been very cordial and I'm really excited to have you with me today. I'd like to start with you telling us how did you get involved in this whole investing business? Give us some of your bio. My father did something very small. He was a real stock junkie. He would follow stocks daily and go to the Merrill Lynch office in the neighborhood and watch the ticket tape. And when I was born, he took like, I think it was $500 in gifts and he bought like seven different individual stocks for me. And then when I turned 13, we had my bar mitzvah and we got a little bit more cash and then we sat down together and we bought one stock. At that time while other kids were reading the sports section, I was already following the Wall Street Journal and all these stock prices. So then you decided to make it a career or at least go to school? I then went to school. My undergraduate degree is from City College, the Business School, Bernard Baruch, and Finance and Investing. It really was just about the first finance degrees handed out anywhere because there was no finance theory, as you know, until Bill Shopp and others created that CAPM. Before that, finance was hidden inside some accounting program or maybe an economics class. So Larry, while you were at college, I understand you were a pretty good basketball player. Is that true? Well, that depends on your definition of a pretty good basketball player. I was pretty good for my neighborhood, but I was so short and I got to high school I was five foot one and 110 pounds so I couldn't even make my high school basketball team but I grew in my next two years when I graduated high school. I was 5'10 and all of about 140 and I was still only 16 and I went to Baruch College and I made their basketball team. You know, it's a D3 school, so it all depends upon your definition. Relatively speaking, I was good. So basically you decided not to have a career in basketball and so you went on and got your master's degree? I then went to NYU for my master's and when I got out of school Wall Street had collapsed. 73-4, the crisis in addition to which the end of the fixed commission era was over and many firms were going bankrupt. Even Ross Perot lost a fortune trying to buy a brokerage firm and so I ended up taking a job at CBS in their international finance department and then I was going for my PhD in international finance and economics. Ultimately, unfortunately, the New York City subway system had gotten so bad and dangerous I didn't quite finish my PhD because I had to have my mother even literally drive down from the Bronx to pick me up the last semester and it had gotten so dangerous. So Larry, I understand from knowing you that you actually got not one MBA but you got two MBAs in a way. Is that what you're talking about with the PhD? Yeah, in effect. So I don't really technically have two MBAs but I have an MBA in finance and investing and then going for my PhD which would have been in international trade theory and economics I have all of the required courses and I have a master's in international finance and economics so in effect you could say I have two MBAs. So let's move on and start talking about your career on Wall Street. I really got lucky after that it's one of those strange things I feel like the Zellig character in Woody Allen's movie where he's inserted in all these major events in history I was involved in I think four of the biggest revolutions in finance just happened to be at the right place in the right time. The first was when Brent Woods broke down all of a sudden everyone had to deal with floating exchange rates and interest rates going crazy and I was hired by CBS to be manager of international finance and helping them to manage those risks. So CBS you're talking about the entertainment company. Yeah exactly right and there's actually an interesting story behind there Rick we could divert for a moment so the only reason I interviewed with CBS I was hoping to get a job with a Wall Street firm and I did have one offer from IBM but Wall Street as I mentioned earlier was in a sad shape and they could hire somebody with ten years of experience for about the same or not much more than some young MBA so I went to I said I gotta get a job I'm getting married and CBS happened to own the Yankees at that time and so I said I don't see if I don't care if I drive a bus I'd see if I can get a job with the Yankee organization so I go to the interview and on my way there I pick up the newspaper and the headline reads CBS sells the Yankees so I decide I'm dressed up in my suit anyway I'm on the subway I might as well go I went and ultimately I did get a job there and had a good two years there but then two years later at the grand old age of 23 and with all of two years of experience City Corp and other banks were getting in the consulting business to help these big multinationals deal with foreign exchange risk and they hired me to be a consultant so I went and joined them this was our new exciting area and then two years later City Corp asked me to go set up a whole West Coast investment bank at the grand old age of 25 I had never run a foreign exchange trading room I had never funded an offshore bank I had never managed more than one person to pay out to San Francisco to build an entire operation and the second revolution was occurring around that time City Corp and Solomon Brothers were leading the way in creating what Buffett ultimately called these weapons of mass financial destruction these derivatives and they were more basic at that time interest rate floors and ceilings and collars and foreign exchange floors and ceilings and collars to manage these risks and so I was right there with that and even helped design and create some of these derivatives and then 10 years later my boss at City Corp left to join Prudential Home Mortgage and that was a whole new revolution in mortgage banking the first private mortgage backed securities so I got involved there and with that big revolution did that for eight years and then we ultimately sold the company and then I was going to look for something to do I thought I'd go teach something I'd always wanted to do that's why I was originally going for my PhD and friends of mine had started this RIA a registered investment advisory firm in St. Louis and they were good planners doing a state and tax type of work but they didn't really know anything about investing so I thought this would be a great fit I told them I didn't want to have to do anything to do with running the company but I would help set the investment strategy teach them about managing all types of risk so we joined forces and right at that time as you know the early 90s Fama and French right there groundbreaking paper now in this three factor world it changed the way we thought about investing and so that was a fourth revolution that now just happened to be at the right point in time so I do myself as very lucky and had a lot of vast different experiences which has helped me I think be a much better advisor having lived through lots of financial crises So the firm you're talking about Buckingham asset management out of St. Louis well now it's called we changed the name about I think it's now maybe two years ago it's Buckingham strategic wealth we're just about a $17 billion RIA and now 35 cities and then we have TAMP which runs both BAM advisor services helping other advisors and a firm called Loring Ward together we're about $52 billion I believe How many advisors are under the TAMP program? Larry do you have a basic number? Yeah I can give you some general guidelines at Buckingham which is BAM advisor services about 135 firms around the country with about I think 19 billion of assets and Loring Ward has over a thousand advisors many of them small broker dealers who really are nothing more than investment advisors and then they have also about I think a hundred plus advisory firms who are really more wealth management as well as investment advice and they have collectively about 16 or 17 billion So what helped this whole thing grow was your book your very first book correct called when that book came out just before I went to a DFA conference more right after that Yeah what happened was I got to Buckingham and they had no marketing material at all the firm had just been formed and they had decided like you did to at least use some of DFA's funds this was a DFA shop if you will and that had used DFA products exclusively in the early years today we use many multiple advisors but DFA clearly was the leader there and I went to a DFA seminar like you did to learn everything about them and they had like three slides at the time as to explain what they did so I said we need to have a brochure to at least tell people what we're all about so I took what I learned in those three days at the University if you will and put together a 40 page brochure and then I said what we really need is a book and I had no intention of writing one so I went and looked in the libraries and bookstores when there actually were bookstores No there are still bookstores there are still bookstores out there Yeah there are a few not many I think we have one left in St. Louis one major one but at any rate the good book I thought out there at that time was Random Walk Down Wall Street and all it did really although it was a great book it told you the markets were pretty efficient and you should act as if they were but it didn't really tell you what to do with it in the way like your book all about index funds did or my book or others like John Bogle's book so I said alright I've got a 40 page outline let's see if I could put a book together so I spent the next 2 years writing a book and when I completed it we got a publisher who liked it and decided to publish it but the interesting story there was he hated my title which was what Wall Street doesn't want you to know so he said Larry go back and give me some other options and I went to a bookstore like any of them and he pulled some things together and he came up with the title the only guy you'll ever need to a winning investment strategy now the funny thing about that is 2 years later I wrote my second book and I really liked that first title what Wall Street doesn't want you to know and I I was obligated to show it to him he had to write a first refusal and he read it and said Larry I love that title same guy who had rejected it 2 years earlier editors are strange people I remember when I first saw that book I can't remember where it was one of those bookstores that don't exist anymore and I looked at the title and I said what is this whatever it was the only investment guide you'll ever need I looked at that and I go what the heck is this nonsense I started reading it and I realized what it was and I said actually this is pretty good you've done really well in fact that book if I'm not mistaken has you've done a couple of additions now correct I did one addition to that 7 years later in 2005 updated that and now I've written a total of 17 books which 4 are originals and 3 of them are updates I wrote in 2001 I think it was rational investing in irrational times which covered 52 investment mistakes I thought people made several years later I updated it and had 77 mistakes that book was investment mistakes even smart people make and then in 2000 I think 15 I wrote reducing the risk of black swans and in 2018 we updated that one of the books that I like in fact I think it's a great title is the incredible shrinking alpha and I've always been kind of fascinated with could you talk about that and what's going on out there the book itself is a nice little short book I got that idea from Bill Bernstein to write these little short books that are really narrowly focused so even a slow reader might read it in maybe hours so the idea is there are 4 big themes that are happening that are making it harder and harder for active managers to win the first is that academics simply really by studying the great active managers and seeing how they beat the market could they identify was it a unique skill that you can't replicate were they buying certain types of stocks with certain traits or characteristics and of course as you well know farm in French summarized a lot of other people's research they didn't discover any of these factors themselves at all but size and value so prior to the publication of the course section of expected stock returns in 92 you could simply say I'm beating the market I'm generating alpha and therefore get away with charging high fees because alpha is a very scarce resource but after night you could do it simply by tilting your portfolio to owning more small and value stocks in the market you could claim out performance and you'd be right but after the publication of that paper and then index funds became available in these asset classes or giving you exposure to these factors you can't claim alpha anymore either regression analysis or attribution analysis will show the alpha was nothing more than due to exposure to these factors and then along comes Mark Carhart 97 he adds a fourth factor momentum and so once again alpha gets converted into beta and then really in 2012 Robert Novie Marks adds profitability which gets expanded into quality and what's important is these conversion of alpha into beta doesn't take anything away from all those great investors like the people from Graham and Doddsville who are buying value stocks that were highly profitable or quality because they've discovered this 50 or 60 years before the academics but if now we can replicate that and invest in exactly the same types of stocks and there are two good studies showing that Warren Buffett almost all but not all of his stop picking advantages so not counting the benefits from the leverage he gets from his insurance company or not counting the benefit he gets from being able to give Goldman Sachs $10 billion at a much lower price than Goldman could have raised it at if they had 30 days to go get the capital but just looking at the public securities that he owned most of it goes away if you simply bought an index of stocks that look like meaning they had the same traits and characteristics so that's the first thing today every investor can invest in very similar so alpha got converted to beta which is nothing more than exposure to factors so that's the first thing we're still on the first one you have three others we don't have like a day and a half maybe you could summer up a little okay so turn to the second one which is in order to beat the market you have to have victims to exploit 70 years ago 90% of all individual stocks were held directly by individuals in their brokerage accounts today that maybe closer to 10% 90% plus of trading is done by big institutions you just don't have those dumb retail investors who buy stocks that go on to underperform and sell stocks that go on to outperform as the research shows they're exploited by the more sophisticated institutional investors but that pool of victims is shrinking dramatically so today when Goldman Sachs is trading it's 90% chance it's SAC capital or renaissance technology in the other side of the trade the third thing is the quality of the competition has gone way up you know when you and I got out of school most of the people who are managing money very few of them were world-class scientists with PHD's in finance and where of all this academic research today everybody managing money is like a rocket scientist DFA's CIO is orinautical engineer with a PHD and Andy Berkin my co-author of that incredible shrinking alpha PHD in physics so the competition is much tougher and that makes it much harder to outperform those people and the last thing is 30 years ago or so there was only about 300 billion and hedge funds out there chasing this limited amount of alpha which is now a smaller pool and less victims to exploit today we have 10 times that amount of money so just not a lot of alpha to go around and the pool is much bigger so 20 years ago when Charles Ellis wrote his famous book winning the losers game about 20% of active managers were outperforming on a statistically significant basis today several studies show that numbers less than 2% and that's even before taxes that's pretty lousy odds which means passive investing is the winners game I guess I know who that 1% is because I hear it all the time how they've outperformed so I think the fifth one if there's going to be a fifth one is the speed of which information is disseminated now it's just so much faster than it used to be I have no question that that's true as well you could add that one absolutely let's go ahead and then move on to the current book the 17th book which is your complete guide to a successful and secure retirement and the forward is by Wade Vow good guide to write the forward what motivated you to write this book well I think the key here is that so many people at least today are at least beginning to focus on the need to have an investment plan there are lots of good books on investment planning you've yourself written several that really go a long way to helping people build an investment plan but the key is that you could have the perfect investment plan and it could blow up for reasons that have nothing to do with investing you're a young man you've got three kids and you don't have enough life insurance you die early I don't care Rick you've got the perfect investment plan but you don't live long enough to save and invest and watch it grow you're a doctor perform surgery and you don't have a disability policy and you're getting an accident and you can't perform surgery anymore you don't have an umbrella policy you get in a car accident you don't have long term care and you have dementia there's all kinds of risks related to elder abuse we even have a chapter on that and there are so many ways that investors can make mistakes that lower their odds of success like location holding the right assets but in the wrong location taking Social Security wait too early not using annuities where they are appropriate and buying the right kind so what I wanted to do was to create a book that looked at all of those things and the first chapter is maybe the most important one which is why we put it there which has nothing to do with investing or other financial issues but having a plan to have a meaningful and successful life in retirement and that's so important one the highest suicide cohort in the United States I would have guessed might be teenage girls unfortunately I lost a sister at a very young age to that but it's retired men because they've lost the meaning and their purpose and sense of fulfillment and social connections which they typically get at work and the highest divorce cohort is what's called silver divorce and my line about that is well I married you for better or worse but not for lunch and you have no plan to have a meaningful life did you come up with that I married you for better or worse but not for lunch I don't know if I actually can take credit for that but I steal it and use it all the time great I'd like to add one more thing to your threat of elder financial abuse chapter I wrote an article for forging about it I call that article you may never see your grandchildren again because I was invited to a local dinner seminar put on by a local advisor I live in an over 55 community out here near Austin and we just get hit with all of these dinner seminar things anyway decided to go to this one there's no word of a lie the advisor must have said three times if you want to see your grandchildren again talking to the woman in the audience you want to come and see me because I have had so many widows who didn't do it the right way when their husbands died and by the time they came to see me they didn't have any money they never got to see their grandchildren again after that so if you ever want to see your grandchildren again you need to come and talk with me and the guy was a snake because he was selling all kinds of high commission annuities and all this other stuff I mean he had a management fee that was something like 2% he was a real snake but this is what he was pitching to the people in this community you know we see that unfortunately all the time literally you and I agree on lots of things even though we've had you know some good debates about most of stuff that's relatively minor but this is a perfect one where I know we agree I literally don't know how these people look themselves in the mirror when they get up in the morning and live with themselves and unfortunately it's not going to change tell me a few other highlights of the book and then we're going to move on to some questions by the Bogleheads well yeah a couple of things I think are important one I think we really cover an incredibly broad spectrum I don't believe there is any book that is as comprehensive as this there are other good books on estate planning or investing but none is comprehensive. Second thing I want to highlight is the book is filled with lists and checklists action items for people to do so I believe literally everyone who reads the book will walk away with some things that they can focus on and I'll just touch on in the introduction we talk about what I call the Four Horsemen of the Retirement Apocalypse and I can take credit for that term and the Four Horsemen here that are creating real threats to successful retirements are first equity valuations at least in the US are much higher than they have been historically and bond yields are much lower and unfortunately so many people look at the past and automatically project them in the future. I meant to that I tell you the and they're related to I mean the high equity valuations that we have are is related to the low bond yield. Yeah exactly the Fed unfortunately pushed a lot of people to take more risk because they can't live on that 2% bond yield but here's an amazing stat from 1982 through now as typical 6040 S&P 500 intermediate treasury portfolio has earned 10% amazing most financial economists today would put that number maybe in the area of five. I put that like four and a half. Well I won't disagree and that's a real problem if you're counting on 10 or anything near there your plan is likely to blow up so those are the first two horsemen the third is we're now living much longer when I was growing up even as a teenager I almost knew nobody who was 75 years old and today 65 year old couple second to die is almost 25 years and that means half the time someone is going to live longer so you really need to plan for at least 30 years not 10 or 15 so you got higher valuations lower bond yields lower expected returns and the money has to last a lot longer fourth thing is the good news of course we're living longer the bad news is the longer you live the increased risk of dementia and long term care needs which could be very expensive and that's where the risk to many people also that's got to be dealt with we do have a chapter on long term care as well as annuities to help deal with that and there is even a fifth horseman which I mentioned which is the risk of social security if congress doesn't act and this looks like another do nothing but if congress the estimate is in about 13 years they'll only be able to pay out about 70% it's not going bankrupt so my advice is being conservative starting in 2032 or so you should only plan on getting 70% of social security benefits but I remember that you and I when we were growing up when we were 30 or 40 years old I think we were saying we weren't going to get anything either but something the question is you may not get as much as you thought you were going to get exactly you know and there are simple solutions here you know raise the amount of income that's tax you can raise the rate slightly those are two simple solutions that they could do and then of course you can raise the age that you're retired you can't expect to get 30 years of payments because design may be when people live 5 to 10 years and so you're living longer and people are working most of my friends are working till 68, 70, 72 now they don't want to retire to quote nothing well I agree in fact you know this is my encore career doing what I'm doing now as you know I sold my company a couple years ago and now I'm just doing consulting hourly advising and it's an encore career but I admit I like the idea of continuing to get paid enough money so that I can pay all of my bills and travel and such and not have to touch my savings the encore career is a good idea and a lot of people are probably going to do that if they can yeah and I would add one other thing on top of the points you made which are critical is it gives you that social connection you stay engaged and I'm sure you're just like me you get great psychological joy out of helping people knowing your increase in their odds of being them able to achieve their life and financial goals by putting them on the right path well it's interesting from where we sit because we get to pair into people's lives like most people can't there's an old joke that if you really want to get to know someone you either marry them or manage their money and it's true I'm talking with three or four people a day and really getting to know all about them and all about their finances and all about how they're planning things and one thing you brought up and I want to reiterate it is this idea of disability insurance I see a real, real lack of disability insurance out there particularly among physicians and among people who are high wage earners for some reason it sort of passed them by and I ask them do you have disability and they're making $500,000 a year and they say well I think I've got a policy I think I've got a policy and I ask them well what would you make and it turns out that somebody who's making $500,000 a year and has $300,000 a year in bills because they bought a new house and they want to send their kids to college and this and that and the other if they became disabled and they couldn't work they would be making about $100,000 a year that's a problem yeah and it's a Pascal's wager bet it's one of those things it doesn't matter what you think the odds of you being disabled are you're likely not willing to live with the consequences of being disabled without disability insurance and I would add the other thing I would imagine you also look at it's one of the first questions I ask somebody because it's the cheapest insurance there is and everybody should have it is an umbrella policy you can get a million bucks for something I think in the three to four hundred dollar range and you should have at least as much I think as your financial assets and maybe even most people up to about ten million because the lawsuits today are paying out such large amounts on claims I personally carry a ten million dollar umbrella policy mine isn't quite that large Larry but I haven't written seventeen books you and I know you don't get rich writing books I've got to move into the next area of our interview you're very popular with the Boglehead community I've always been for twenty years hopefully we're going to get you out to a conference again one of these days when your name comes up and you're active on the Boglehead site and a lot of people have read your books and so when I went on the Bogleheads and I announced that I was going to be interviewing you I got literally dozens and dozens and dozens of questions that people wanted to ask so I have a print out here of several pages long of questions and now we're not going to be able to get to every question but I want to get to as many as we can so here we go with the first question a Boglehead asked what your thoughts are on the new Vanguard factor funds and particularly the Vanguard multifactor funds these came out a little later on and they don't seem to be publicized that much by Vanguard but do you think that the way they're doing it is adequate well I think a short answer would be this number one virtually anything Vanguard does is likely to be done very well one highly ethical people and number two they do have good researchers there their funds we did take a look at them so I can comment they're well designed and they're cheap so those are all Vanguard traits if you will so somebody looking to add a little bit of further diversification in their portfolios away from a market like portfolio this is certainly worth looking into the only negative if you want to call it that is they tend to have very low exposures to each of the factors they're not a deep exposure to them so one of the things you have to be careful about people often especially Bogleheads focus on expense ratios I think the right way to focus on it is the expense ratio relative to the amount of exposure to these factors you're getting so for arguments sake just to make it simple if you thought all of the factors would get you 2% a year and you're getting 10% exposure to it well you're getting 20 basis points and if you're paying 8 for it and their implementation cost you're probably not getting very much of any benefit and the end if you paid 20 for 30% you're way ahead I call what you just talked about the cost per unit of risk right you could pay a lot of money for a small cap value fund that's basically has mostly beta and doesn't have much small cap and not much value exactly you have to look at that I'll give one very brief example we used to use DFA we switched in 2011 to Bridgeway because it was much smaller and deeper value it was 8 basis points more but the market cap was half the size of DFA and the price the book and earnings and cash flow were much lower we estimated if the factors delivered what we expected it would outperform by about 60 basis points we were willing to pay 8 to get 60 I have a couple of people who asked about risk parity so I'll let you go ahead and explain quickly what risk parity is and you can give your opinion so here's the basic concept of risk parity or why people should at least be thinking about it so a typical 60-40 portfolio when I ask people if you have a million dollars 600,000 in stocks and 400,000 in bonds how much of your risk not money but risk is in stocks they'll say 60% but that's not true because stocks are so much more riskier than say a 5 year treasury they're about 4 times as volatile so about 85% of your risk in that portfolio is in your equities if you believe that as you and I think can agree they're pretty highly efficient then you should also believe it has to follow that all risky assets have very similar risk adjusted returns so small caps are riskier than large caps so they should have higher expected returns it doesn't make them a better investment it means they're riskier and once you adjust for the risk their risk adjusted return should be similar risk adjusted returns are not just the sharp ratio you have to think about liquidity and trading costs etc but that's the idea so risk parity the idea is to diversify of course as many unique sources of risk as you can identify that provide premiums that have evidence using my terms from my factor book that they're persistent all over over very long periods of time they're pervasive all over the globe they're robust to various definitions so you don't have likely data mining so for value, PE, cash flow EBITDA all work momentum you can have different formation and holding periods they work has to survive transactions costs meaning it's implementable you don't want a 5% premium of micro caps and of course 6% to capture it and lastly it has to have intuitive reasons for you to believe it will likely persist the idea of risk parity then is to put equal amounts of money in every one of the assets that you could identify so you end up having an equal amount of risk so that's the concept directionally that makes perfect sense and logic I would say however I don't have the same degree of confidence in all of these factors or asset classes so I want to put more of my weight in my portfolio on things I have the most confidence in those are things that are risk based solutions things like small and value where momentum is purely behavioral so I don't want to ignore it because it has a lot of evidence that meet that criteria so I just try to eliminate negative momentum as an example and then I invest in other assets like reinsurance quality profitability etc putting more weight on the ones I have the most confidence in and less weight in the others so each person should decide how much confidence they have in each one and then diversify logically bottom line is risk parity is a good general idea but I think it's not the right answer you should not be looking to have exactly the same amount of risk in each of your assets you should put more weight on the ones you personally have the most confidence in that will deliver above market returns or have unique risks that are rewarding Talking about risk one of the Bogleheads asks isn't a simple portfolio of few index funds still the best way for the general public to grab their fair share of market return without breaking the bank on expenses and if not what can the general public do to try to take advantage of some of your ideas First of all I think you and I can agree that a simple two fund portfolio of a total US and a total international to the global market cap today is actually roughly 50-50 you want to tilt a little bit or as Cliff Asnes would say sin a little bit so you want to have a little more US or a little more international that's fine and that's a very low cost tax efficient way to do it the problem is you have all of your risk then in market beta where correlations go very high especially in crises but even not in crises you can go through very long periods we happen to be one now for the last 10 years US small value is performed relatively poorly from 66 to 82 on the other hand a total US market fund would have underperformed totally riskless one month treasury bills for 17 years and underperformed small value by over a thousand percent to me the right answer is you want to be low cost tax efficient to yourself the chance to stay disciplined and the right portfolio is the one that you are most likely to stay disciplined so no matter what you invested whether it's total stock market funds there are three periods where US total market is underperformed totally riskless T-bills for at least 13 years that's cumulative 45 of the last 90 years in the other 45 years it dramatically outperformed we don't know which period you are going to get maybe the next 15 years when you retire tomorrow is one of those where US market beta does poorly so to me if you are able to deal with this tracking variance issue that once you diversify you know you are not going to look like the market and you have to live with that if you can't do that well then you shouldn't tilt into smaller value or own reinsurance or anything else because every risk asset guaranteed will go through long periods of underperformance there is no such asset that doesn't and that has to be the case or there would be no risk for the long term investor on the other hand if you do diversify and diversification is difficult because you are always going to own some things that are doing poorly which is why so few people are good at it then you increase your odds of getting a return that's expected you are narrowing the dispersion of your outcomes because you won't own all of the best and you won't own all of the worst if you will and that actually goes a long way to giving you a better chance of getting the mean return expected out of a potential dispersion when you run money call analysis diversify across asset classes or factors, unique sources of risk will come up with much higher odds of success but it doesn't do you any good if you can't stay disciplined so don't do it if you are not going to be able to stay the course and that means rebalancing along the way I call it a lifelong investment strategy because you really need to be in the thing lifelong to believe in it so strongly that you are willing to stick with it for the rest of your life if you don't, odds are the time you jump out is going to be the worst time and then looking back what you should have never done it to begin with I completely agree let's get on to something that I think is really important right now it's called recency bias and somebody asks do you think that investors are suffering from recency bias in the stock market going up over the last 10 years well we know recency bias is prevalent through all investors not just individuals we spend more of our time managing people than we do managing money whatever is done well recently they want to buy I remember well today everyone why we own international emerging markets or small value if you just go back to 2007 at the end of that year period that prior 5 years the S&P had a great period it was up 82% EFI was up 162 called a double and the DFA emerging market fund was up 565% and money was flowing in like crazy as I'm sure you remember into emerging markets I do it was all the rage yep and then of course the reverse happens and then floods out so the problem is people only buy after the good returns and they panic and sell so they hurt themselves the best thing as you said and I completely agree you got to have this kind of lifelong approach and you got to stick with it now people are fleeing emerging markets they're fleeing small value and by the way I'll point out one of the techniques I use when I talk to people I point out value went through exactly the same period even much worse performance just not as long in the late 90s and Buffett in many cases was being ridiculed in the media as the time has passed him by in the new dot com era you got to own Intel and all of these kinds of companies but it's the same thing that happened in parts of the 80s digital equipment and those stocks were there and we're seeing now the same thing Buffett didn't give up on value in the 90s and then next eight years with the biggest value premium ever by far and right now value relative to growth valuations are pretty much almost exactly where they were in 99 now I'm not making any prediction I have my crystal ball is cloudy but valuations look really stretched and value looks really cheap and I ask people what do you know that Warren Buffett doesn't know he's not giving up on cheap value stocks why should you it doesn't matter some people you can't protect from themselves Larry question from a Boglehead having to do with the 4% rule and I'm referring to the idea that it's safe to withdraw 4% of your portfolio is there now a 3% rule instead of the 4% rule here's the way I would answer that first of all I don't believe people should use that is anything more than a guideline number one but the 4% rule was based on the historical evidence when valuations of course were much lower and expected returns much higher and bond yields were much higher which meant yields were much higher today if you ran Amani Carlos simulation using the same type of life expectancy and withdrawal rates but now just projecting returns based on today's valuations you get a number closer to about 3.3% to be safe I think withdrawal rates like 4% or 3% or whatever you're going to use are fine when you're you know in your 40s and you're still working and you're looking toward retiring in your 60s or mid 60s and when you're projecting out how much rate of return you might get on your portfolio and how much money you need to accumulate to get to a certain point and then from there if you took 4% or 3% how much would that give you I think that's where the 3 or 4% with safe withdrawal rate works but in retirement when you're actually getting ready to retire the withdrawal rate is the amount of cash flow that the portfolio creates to me so we're looking at dividends and interest income and what amount of dividend and interest income and what amount of social security are you getting and how much cash flow are you getting in this is your sustainable amount this is the sustainable amount that you can get out of the portfolio if you go more than that then you start cutting into principal then it becomes not as sustainable so I think that this 3 or 4% withdrawal rate is good for planning purposes long term long range planning purposes but you actually get into retirement as you said it's sort of year by year and your budgeting and what are you going to do and I know you use Monte Carlo simulation I don't particularly use that but I think we all get to the same spot when you're actually in it and you're actually taking money out of the portfolio you've got cash flow coming in and then you've got things that are occurring on a year over year basis that might need to be looked at as they occur I'll just add quickly Rick one I certainly agree with the first half completely what you're saying 3 or 4% reasonable for long term planning in retirement the only thing I would add to what you said is not drawing any principal I think becomes way too conservative because we know we are not going to live forever so you can certainly dip into some amount of principal unless you have that I'm going to want a bequeathed desire there so we don't want people not to be able to enjoy their life and spend money so you want to at least consider you can dip into that portfolio a little bit every year and just depending upon how far you're along certainly at age 90 someone maybe got a 10 more year horizon of most they should be able to take out a lot more than 4% of their portfolio right? absolutely just like exactly what the required minimum distributions are on a retirement account for sure now if you could only spend it at age 90 that would be better right? alright somebody is asking what about if you're in a situation where you have nobody who can take care of your affairs in retirement you're unable to handle your own affairs you're unable to manage your own portfolio at some point what do you suggest for a person or a couple who is in that situation where there's nobody who can actually come in and help them and take over for them what suggestions do you have for those folks? yeah well first I'm going to broaden the question because I think it's going to be helpful for every bogal head just in my own particular case we're all living longer the risk of dementia go up is the most likely to prey on the elderly and they know women are more susceptible so they're targeted I have a whole chapter on the book there but one of the things that I did personally in creating my estate documents my wife has a right which I have a reciprocal right if she believes I'm in cognitive decline and are unable to make sound financial decisions she can request that I have or require that I have a cognitive test and if I can't pass it then I am to be removed from power of attorney over bank accounts financial matters brokerage statements everything to protect myself and the family and of course I have the right to the same thing and that's right in my estate documents and obviously if you're alone you want to have that situation or some trusted family member or friend or whoever it might be and I would prefer it not be some corporation like a bank trust that creates all kinds of problems with family members having difficulty accessing funds at times but if you don't have that then some trusted trusted advisor could play that role we do that for a select few of our clients but you need absolutely to have somebody who you trust the most trusted person in your life I'll just add one other comment sadly the person most likely to rip off the elderly is some family member who might be in trouble either drug abuse their business went south they got a divorce and got in trouble and often will sadly rip off the parents I've seen people who were financially in trouble just ruin their parents fortune by taking over and by convincing the parents to give them control over the assets and very soon thereafter there were no assets left I've seen that happen it's really unfortunate because these are unpleasant things to think about and contemplate and therefore people delay and don't do it so I'm going to urge all the Bogleheads who don't have a disability policy don't have an umbrella policy and don't have this request for have an exam where you could pass a cognitive test if you don't have those in your documents go out and get them tomorrow here's a couple of questions about different asset classes you could just answer these very quickly because they have here there and everywhere so one of them is what role does cash play in an investor portfolio almost none because today you can stick with short term bonds just as highly liquid so other than money in a checking account you typically don't need much cash because as long as in a short term bond fund your volatility is going to be one or two percent depending upon how short it is so you want to keep maybe a couple of months expenses in cash and beyond that there's no need to have a lot of cash around that's one of the most common mistakes I see is people sit with lots of cash and what about peer to peer lending something new that I haven't heard you talk a lot about but I know it's in your portfolios right so this is a natural the banks have been disintermediated by fintech providers partly Dodd Frank opened the doors for them the banks were short of capital and today you can get a replacement for a 22% credit card if you're a good credit 13 to 14% kind of range and your credit isn't quite as good the rates go up from there after expenses we think a well run fund should be able to generate 4 to 5% again above T-bills and you're getting compensated for credit risk so last question Larry what is the most recent version of the most up to date version of the Larry portfolio so the Larry portfolio phrase was coined by the New York Times and it only referred to basically the equity portion of the portfolio so it was US small value international small value emerging market value so it's all in the highest expected return assets now you could use that Larry portfolio in one of two ways I have investors who are young willing to live with the tracking variance and they're 100% equities and 100% the Larry portfolio trying to earn that extra premium on the other hand you have people like me who have won the game already and I'm trying to protect my wealth I'm trying to create more that risk parity portfolio I don't want a lot of market beta exposure but I'm trying to keep my expected returns up by the equities I'm holding have higher expected returns that allows me to own less market beta and then more safe bonds so my portfolio was roughly at the time was 30% public equities 70% safe bonds it's a little more diversified today adding things like reinsurance and alternative lending I also happen to own something called the variance risk premium fund or all asset variance risk premium it's also one of the most studied research well documented premiums and all of finance which is the variance risk premium so you're selling volatility insurance I think that again has a 4-5% above T-bill expected return and I also own AQR's alternative style premium fund which is a long short portfolio of four different factors across four different asset classes so moving more towards risk parity but each one of them is a few percent of my portfolio because I want most of my portfolio in that safe for bonds well Larry it's been a very great having you on here and as always every time we talk we are 99% correlated and only 1% uncorrelated yeah and people love to focus on the 1% that creates the tension right well thank you so much for being a guest on the Bogleheads on investing show great to have you and hopefully we'll have your back again soon yeah my pleasure and you can let the Bogleheads know if you didn't get to any of the questions they can always shoot me an email at elswedrow.bamadvisorwithn0.com I'm always happy to answer questions alright thank you Larry take care this concludes the 12th episode of Bogleheads on investing I'm your host Rick Ferry join us each month to hear a new special guest in the meantime visit Bogleheads.org participate in the forum and help others find the forum thanks for listening


Rheumatoid arthritis is a chronic systemic autoimmune disease affecting millions characterized by persistent pain and stiffness and progressive joint destruction, particularly in the hands and feet leading to crippling deformities. What can we do to prevent it and to treat it? In a famous 13-month long randomized controlled trial of plant-based diets for rheumatoid arthritis, patients were put on a vegan diet for three and a half months and then switched to an egg-free lacto-vegetarian diet for the remainder of the study. Compared to the control group that didn't change their diet at all, the plant-based group had a significant improvement in morning stiffness within the first month, cutting the number of hours they suffered from joint stiffness in half. Pain dropped from 5 out of 10 down to less than 3 out of 10. A drop in disability reported subjectively feeling better. There were significant improvements in their grip strength in their hands. Fewer tender joints, less tenderness per joint, and less swelling with the added benefit of losing about 13 pounds and keeping most of that weight off throughout the year. They also had a drop in inflammatory markers in their blood, sederates, erectile protein, and white count. The question is why? What does diet have to do with inflammatory joint disease? Well, rheumatoid arthritis is an autoimmune disease in which your body attacks the lining of your own joints. Why would it do that? Well, there's a different autoimmune disease called rheumatic fever, in which your body attacks your own heart. Again, why would it do that? Well, it appears to be a matter of friendly fire. Rheumatic fever is caused by strep throat, which is caused by a bacteria that has a protein that looks an awful lot like a protein in our heart. So when our immune system attacks the strep bacteria, it also attacks our heart valves, triggering an autoimmune attack by molecular mimicry. The protein on the strep bacteria is mimicking a protein in our heart, so our body gets confused and attacks both. That's why it's critical to treat strep throat early to prevent our heart from getting caught in the crossfire. So researchers thought maybe rheumatoid arthritis might be triggered by an infection as well. A clue to where to start looking was the fact that women seem to get rheumatoid arthritis three times more frequently than men. What type of infection do women get more than men? Urinary tract infection. So researchers started testing the urine of rheumatoid arthritis sufferers, and lo and behold, found this bacteria called Proteus Mirabilis, not enough to cause symptoms of a UTI, but enough to trigger an immune response. And indeed, there's a molecule in that bacteria that looks an awful lot like one of our own molecules in our joints. So antiproteus antibodies against the bacteria may inadvertently damage our own joint tissues, leading eventually to the joint destruction. Therefore, therapeutic interventions aimed at the removal of this bacteria from the bodies of patients with consequent reduction of antibodies against the organism should lead to a decrease in inflammation. Well, as we saw before, urinary tract infections originate from the fecal floor. The bugs crawl up from the rectum into the bladder. And so how might one change the bugs in one's colon by changing our diet? Some of the first studies over 20 years ago on trying to fundamentally shift people's gut flora were done using raw, vegan diets, figuring that's about as fundamental a shift from the standard Western diet as one can get. And indeed, within days one could significantly shift someone's gut flora. And you put rheumatoid arthritis sufferers on that kind of diet, and they experience relief, and the greater improvements were linked to greater changes in their gut flora. But the diet was considered so intolerable that half the patients couldn't take it and dropped out, perhaps because they were trying to feed people things like buckwheat beetroot cutlets buttered with a spread made out of almonds and fermented coconut juice. Excuse me, not coconut, that might have been good. Cucumber juice. Thankfully, regular vegetarian and vegan diets work too, changing the intestinal flora and improving rheumatoid arthritis. But we didn't specifically have confirmation that plant-based diets brought down anti-proteus antibodies until now. Those that responded to the plant-based diet showed a significant drop in anti-proteus-marabolous antibodies compared to the control group. Maybe just dropped immune responses across the board? No, antibody levels against other bugs remain the same. So the assumption is that the veg diet reduced urinary or gut levels of the bug. A shift from an omnivorous to a vegetarian diet has profound influence on the composition of our urine. For example, higher levels of lignans in the urine of those eating vegetarian. Up until now it's just thought that they protected people eating more plant-based from getting cancer. But now we know lignans can also have anti-microbial properties as well. So maybe helping to clear proteus from our system. Either way, this suggests a new type of therapy for the management of rheumatoid arthritis. This new treatment includes anti-proteus measures such as dietary manipulations in the forms of vegetarian diets.


Hi, so my name is Radhavi Kintalbis and I run a small design studio based in Bangalore called Pixalore. I'm Ashash, and I'm a sleepholder engineering, that's a Ruby on the Edge services company, two years old based on a Bangalore. We are all the company behind Ruby on the Edge platform for learning Ruby on the Edge, and that's what we are going to talk about. Before actually getting into the design media, the talk is actually about the collaboration between developers and designers and how we kind of came to some consensus that this is the best way to work. We are still figuring out certain aspects, but whatever our learning has been in the past four weeks, we are going to share that. Let me just give you a quick introduction, so this is the application that we are going to talk about. The challenge is that we faced while designing it was primarily because of the interactivity. So this is the application that I have some content for learning Ruby, or we have some programming languages. I have some code, people can type code over here and actually execute it, it will get executed real time and show the results to you. That's pretty much the whole application. Leaving developers around is a bit dangerous, they don't really know how to control their emotions. So going back to sleepholder engineering, before we started RubyMunc, we have been an Agai company for almost two years. Agai by definition or by the philosophy is an iterative way of doing development. What I mean by iteration is I try to release a really small amount of functionality but a finished functionality to my audience, get the feedback, incorporate the feedback into the product, again get into another iteration and do it as frequently as possible. Now in these two years we have dealt with almost, like I have possibly worked with around four or five applications with my clients. When we started we didn't really have any design capability in-house. So we were forced to work with external designers and most of the time our clients will actually choose the designers that they want to work with. So we ended up working with designers in the US, UK or Australia which pretty much meant that there was no collaboration whatsoever happening, at least not face to face. One practical problem that external designers would face is if we want to work in iteration that means I just care about one big worth of work. But if I'm dealing with an external designer they certainly care about at least one month or two months of work because a client is going to give them the whole idea and you want to really just design one component on a page. You actually need to have an idea about the whole holistic application and then only you can decide what the theme is going to be, how layout is going to be or what kind of feel a user should get when designing that application. So what it meant is whenever we get the final design we have almost complete product with us. Now there is a fundamental problem with having a complete design on table and getting into a creative way of development because even if I gather feedback of the first release I can't really use that feedback to change the product unless I am asking the designer to go and rework on it which almost always didn't happen because of financial constraints on my client side. With RubyMonk we had just one month to come up with a prototype, really easy to beat our customers, get the feedback, again incorporate that feedback into the system and actually do a release. So the reason we had this one month deadline was when we started with RubyMonk we had RubyMonk in the US and it was happening exactly one month after that and we wanted to release the product on that day. Now with this design restraint we had to really break this designer and developer disconnect that was happening because otherwise it would have been impossible to release a good product in one month. When we started we had a lot of ideas, we had a lot of principles like we wanted to have a clean UI, we didn't really want any page scrolls happening, we had some very weird constraints like don't ask me reasons why we had those. Then we had a lot of functionalities as well where we wanted to create gamification, we really wanted social integration to the application, we wanted to cover everything under the sun when it comes to Ruby. We wanted an experience where people can actually type in code and actually run tests around those codes and tell them this is the aspect of your code which is wrong and this aspect you are going to get right. Now the more we thought about all this the more confused we get because we really didn't have any way of ideating. I have 10 concepts in mind how do I communicate with other developers let alone with my clients to get feedback. We started reading a lot about designing UX and we realized that there is something called wireframing. So we started doing wireframings which is this was the first homepage or really shitty one that I came up with. This was the other one that a friend of mine came up with. What we decided to do was we had two parallel tracks, two wireframes happening and we will keep on looking at each of those wireframes and based on that collaborate and take the best of those words. Sorry the batteries are off. Hello. So they wrote to me on the... So this Aakash's colleague wrote to me on the 3rd of September. Then we said we have basically 20 working days to release a product. You want to help us design it? And of course being a slight master-kist I said sure I'd love to do that with you. I came in and they were really at the idea stage. They said we want to be able to teach people how to learn this new... They want to teach people this new language Ruby. And they had some ideas. They said we want to clean and minimal UI. Who doesn't? And then of course despite not knowing anything about design they of course have very strong opinions. And they said oh we love World of Warcraft. Let's make it like World of Warcraft. And... Yeah. So eventually over the course of some drunken evenings we came up with a basic theme for the application. We said let's make it like an RPG set in a Buddhist temple. It was at this point in time that we actually called in and that we called the name of the place. We decided we... I thought that wasn't really ready for the market. The only reason I had that name I could create a video repository to check in my book. Sure. So we came up with the name Ruby Monk. And then we went through a couple of stages of actually picking out what would the visual sort of influences be. So we looked at... We looked at Avatar which is a series that a lot of people like. We looked at the genderless headaches that are involved with Monk's monastery. We were looking really at oriental monks not occidental. And eventually at this particular design I created. We started here. This is exactly where we started. Akash and his team had put together wireframes which really just expressed functionality. And they really liked that. But this was a really good way to communicate. I found this extremely helpful as I'm sure any of you who are designers also do. Because it was a complete feature listing or what they thought should be a complete feature listing. These are some more... These are other wireframes. This is the core of the application. This particular page for example showed that there was a problem. A space to enter code which would then be evaluated and the results outputted. Similar to the way in which it would work in a terminal or an IDE. This was one of the... This was the design at the end of the design process. Similarly, this was a problem page which talks about... Once you've learned a bunch of concepts, how do you put them together to solve a design problem? Similarly, we went from here to here. How did we actually go from A to B? That's really the interesting thing. Not because it was a terribly hard design challenge. The design challenge itself was minimal. But the biggest problem was making sure that we could do this in a short amount of time. Because there were financial constraints. There were constraints in terms of wanting to launch it at a particular event. So we came up with ProtoDrawing. So we started off here. We started off with a really basic homepage. And this is something that I can't stress. It has been a useful learning for me. How do you build something which as designers, we have a very strong impulse to not push your half-finished work to our clients? How do you actually get over that? How do you convey a concept which is half a completely half-baked to your clients and allow them to see your vision rather than what you have at a specific instance? So this was the end of the first day of our workshop. We put this very, very quickly together. We did some basic sketching. And you can already see that things are changing around. We moved particular blocks of content into different places with color schemes which we thought might reflect the original theme of monestries, monks, etc. And then we got an illustration in. And then put bits of it together. So at all these stages, we didn't quite have exactly what we wanted. But we were quickly able to iterate over less than a week. I think this whole process took the first seven days only. At which point we were here. And this is pretty close to what we released at the end of the initial configuration. The application has since been revived. It looks slightly better. But this is really where we were at at the end of the previous. I then changed it around a bit. I don't know if you can see the differences but toned down the colors, cleaned up certain things, changed around the fonts. And this is an important aspect as well because I was here and I wasn't terribly happy with this. But what I think is extremely important as designers is to go with what your visual instincts are. This definitely was according to the theme. But it didn't feel right. And as developers, they were more than happy to take whatever. But it's important to listen to your visual instincts or your design instincts and go with them. So we cleaned it up. So you can see the initial processes of another part of the application. This was the lessons page. We started off really simply. And this is another thing that's important. If there are existing design patterns which are good ones. If there are things which you can easily reuse, go for it. Essentially at this stage, all that I did was add in a basic grid to the application and perhaps rationalize the fonts to some degree. After that, we added in certain features. And the interesting way in which we came up with features or decided to act particular features that they planned was the fact that we were co-located. We were working out of the same office, which was extremely useful because there was constant conversation about where the application should actually be going. And this is extremely easy then for me to have a sense of the changing requirements because I was involved in the conversations which were happening. I could hear debates about whether features should be included or not right next to it. So that's another important thing to take away. So we moved from here, went along with the other visual theme that I showed you and then finally came out with the page that's online right now along with particular features that we thought of. So we moved from a wireframe to then a completed design. And this process as well took, this was the first two weeks essentially. So we were within, at this point in time we were on target more or less. Of course, the developers were holding me back. That's how it was, completely. So the collaboration was literally like this. There was the developer, we were working together, there was Akash right next door and we could see each other's screens, we could see the processes that we were taking. If there were features which were too expensive, either from a design perspective or from an engineering perspective, we asked them right there and then. There was no further reviews, there were no changes which occurred on everybody who is part of the project knowing about it.  There are people going to say, oh that logo, can we make it bigger? You know that font, do you really think it belongs there? Yes. You just have to say no. That's an important thing, when you're working with a product company or when you're working very closely with a collaborative, there has to be a point at which somebody says, okay the buck stops here, we're not changing anything further. It was nice to have this debate, let's all go home. So similarly, there were features which we designed initially because we thought that you know gamification, everybody is talking about it, let's do it. It seems to be the mentality of a lot of startups these days. So we put in the ability to track progress, to have badges, we actually did it over a couple of times and then we just decided this was going to be far too expensive from an engineering perspective so we cut it, we completely cut that right there. The scope management aspects of it are extremely important for an agile project. So I was kind of wearing two hats while developing this product. I was a developer as well as product owner to say so. Now as a product owner, one thing that I would like to say, what I would like to be is someone who knows exactly what I'm going to develop have perfect estimates as developers and actually meet the deadline. Obviously it was not possible. Every day morning, and that actually holds to till the last 30th day as well, we would meet in the morning, figure out what our things have been developed, what exactly is the stage design in, estimate the existing stories based on whatever work we have done so far and then based on that take a call assuming five days of buffer, how much work we can get done and remove the features, change certain features or even add more features. We are going to a stage where a particular feature we actually removed and added twice. And all this while, obviously he was taking the back. Now this was actually a very important point in the whole product life cycle, but for the MVP is one week before the actual release. We have to take a call that we have one week. We have certain really important features that are important to be developed. Either we can complete those features or we can just stop the development where it is and just make sure that the product that we are releasing is finished and it wasn't an easy decision to make. We had so the social aspect, social integration that he showed earlier. That is one thing that we removed on this day. The challenges or the badges that we saw is something that we used on this day. We actually wanted to integrate identification in terms of getting score and then sharing that score with your friends. We removed all that on this particular day and it was a really good decision. When we started, we wanted to pretty much something like this. We thought it's possible to do it. Somewhere in the middle, we thought this is what we are going to do. And in the end, we shut this. But trust me, it is still more than what we should have shut. We should have shut a lot more late. But all these changes that we saw made a huge impact on the design. At least it flows. So how exactly did we make those decisions? How did we actually decide whether we want certain features or not? Like this is, I'm not sure, this is not really a part of a developer design or collaboration, but you can't really take any calls unless and until you are becoming a product owner. You are completely involved into the process. And the best way to get involved into the process and understand the product is not to look at it from your perspective, but look at it from your user's perspective. And it gets really, really tricky to do that because once you start looking at an interface for seven days, eight days, ten days, you just become blind towards negative aspects.  And that's where showing it to people who are not used to the interface becomes extremely important. Just to give you some example. If you can see that we have this really perfect error message over there. As a developer, my target audience is developer. I really like this particular message because it's told me what exactly is happening within a particular spec. But when we actually started showcasing this product to other people and actually we showcased just this screenshot. We didn't even actually showcase the actual product. A lot of people told us that they didn't really care about this. It's just too confusing. And that's when we removed this. The feedback cycle itself doesn't have to be after you have released the product. You can actually or you must do it even before you start writing the first form. We started getting feedback even on this. So when Rahul came in, he already had my wife or anyone else, even people who are coming in to my company for an interview will just sit over there and both do bar bar steps. And that actually worked very well. When Rahul joined in, we had these half-baked sketches. There is some sketch over there that we are not able to see there. We even showed these sketches to other people and tried to get feedback. When you just have 30 days to release, it's not really important to complete everything and get the feedback. Once we were done with the development, and that's the reason we actually stopped 7 days before the release, we started showcasing our product to certain private customers. Basically people who understand where we are coming from, what our limitation is and we actually explain to them what the idea is going to be and get the feedback. For me personally, this has been the best feedback I have ever received. The way we generally do it is I actually sit next to that person, ask him to go through it. You don't talk to him. If he can think about that, it's great. And you just jot down everything that he's doing. When he's clicking on this, you feel that he's confused. You actually jot that down as well. Once he's done with the product review, you actually talk to him and figure out what was his perspective when he was looking at certain features, what exactly he was thinking about. So it's not a private beta. It's a personal beta. Yeah. It's a new term. You heard it here first. Yeah, sir. We are a giant company, right, sir? They grow ice. One thing that you made a huge difference for every customer was a concept that they called a new feature that they introduced for example. So when we started writing content, we realized that rather than just explaining a concept through the text over here, a best way to show a programming concept is to actually show them an executing code. So over here, basically, these are exercise blocks and you will have incomplete code examples there. The user is supposed to fill in, complete the code example, and then submit the code. What we came up with was something called an example code where in the same box, I'll give you a completed code form. You just execute it and see the outcome. A very simple concept, right? Like, there's nothing rocket science to it. And we were very, very aware of people are going to get confused because of this, so we added this text example code over here. I showed it to three people, or three of them, my lovely clients, actually. And every one of them, right there, clicked something, executed a draft, and is looking at it, like, what do I do with this?  He's like, what do I have to do with this? This is already solved. Nobody read the example code. Not a single one of them. That's where we came up with this idea of doing a distinction through color. This is really similar. So this is supposed to be of a yellow, sort of off-whiteish background. We came up with a really simple idea that the background for example code and the exercises are going to be different. And that was enough to tell people that case is something different. I need to have a different expectation. And these kinds of inputs we could get only after having these feedback sessions. This was another part which came out of this personal feedback. As we showed you, we came up with a design where a best way we thought to showcase our product on home page is to push the core functionality on the home page. That is, if I click on run, the code executes over there. And to our surprise, this was the most misunderstood part of the website. Nobody got exactly what we do after this. Even people who are experienced Ruby developers, they click on this. They look around the page for a while. Read, code run, learn, which were unfortunately links. What to do after that? So we finally removed everything. As I showed you earlier, we just changed that to a single link saying start learning Ruby now. That's all. It worked like a charm. I unfortunately don't have statistics for that, but I can show you something else. Another way to get direct feedback that actually started happening after 30 days, but otherwise that's a very important part. We were using this tool called the linkage and then people are pushing in their feedback. We were skeptical how many people are going to do that, but we actually received more than 100 feedback in a single link. This is something I wanted to show. Whenever you change design, there is a way to get direct feedback. When I'm moving from homepage to the get started button that you can see over here, we wanted to know how many people are actually clicking on that. We went through three iterations to reach a level where the conversion rate was earlier 21%, after a certain result, it was 33%. Unless you have access to insights like this, you can't really make a decision. Don't make decisions based on just feedback. So that is pretty much the crux of all our decisions making nowadays, where rather than getting into a subjective conversation for half an hour, we just decided to do something which is cheap to implement and go ahead. Show it to as many people as possible. Nothing beats getting user feedback. At the end of this whole process, we did have this methodology as a theory that we studied before and then implemented. It sort of fell together because of the circumstances. Because it was a short time that we had, we had to release a product and it had to be a working product on day 30. We sort of found these ways of getting around that by working together. It wasn't something that we'd done before, but it did fit into some theory or some assumptions that we'd had. So the most important thing for me was that we need to move from being a visually designer. Of course, all of us know that. We need to become an interface designer. That's the new buzzword. Of course, an experienced designer. But more importantly than interface, we need to become a product designer. That's an extremely important step because you need to move from saying what does a product look like to say how does it fit in with all the other constraints that exist. How will this product actually perform on the market? And the best thing is if you can then pick up a product on your own or yourself. We look at what's happening in terms of not just the interface, not just the product but the constraints of the company. We look at how much engineering resources do we have today? Are they free to want to work on something? You look at what are the long-term goals of this product? And I feel like this step is extremely important to move from becoming mere visually designers to becoming product owners ourselves. It also works backwards. So for a product owner and we started, we were very skeptical about getting into the design. At the end of this whole 30-day period, or even after that as well, what we realized is there is a massive distinction between a visual designer and a product designer. As a product owner, maybe you are not great at visual designing but that doesn't really stop you. But nothing stops you from actually becoming a product designer because you actually understand the product best. You are going to assume that you are talking to customers. If you are not talking to customers, you can't be a product designer. But if you are doing the right things, you can certainly come up with a pretty good design in collaboration with people who understand duets, who have done studies around user behavior or obviously, as a visual designer or a professional designer, you have a lot of insights that you can give to me. But as a developer or as a product owner, don't shy away from designing. You can add a lot more value to the designer himself. This is something that we used to say, design is too important for a YouTube designer. The crux is for developers as well as product owners to say that I understand something about it. I can take ownership out of my opinions and obviously a lot of responsibilities come with it. So we spent a lot of time reading about case studies as well as how people approach this in the first place. You can't just go and say that as he was saying, we can't just go and say that I have an opinion about this logo. You need to justify it. This is more for developers that don't be afraid to get your time dirty either using pyrogorps or doing HTML CSS. Just get into it. It's not really that difficult to do it. Even with the first 30 days of development, one mistake that we did is we outsourced our HTML CSS. Even though the company that did it was great, thank you. Yeah. What we have started realizing now is what I was saying earlier about the iterative development, anytime you have any difference, which is not deeply coupled with the universe cycle, it's going to be a hit. So it's very, very important for people to get into this. This is a very important part. I can't say that this is going to work for every team, for every designer and every developer. Because from a developer's perspective, you really need to appreciate and respect designer's knowledge. So from a designer's perspective, you need to appreciate where the product comes from. One more thing to do now is we can actually check this interface at staging.beamong.com. We are kind of going for new features as well as designs. And more important than that, there's a side to experiencing. So now that we are doing experiments with continuous design and other things, maybe next year we'll share that experience. It would have changed, but not for the good. Sorry. The question is, if we had more than a month to develop this product, would we choose the same methodology? If we say yes, if I know how this process worked out, I will choose or rather we are doing the same thing now as well with Beamong. In that, at that time, probably we wouldn't have. That is the way to go about it for even a longer period. So the process is a slightly scary process. We are talking about essentially an agile philosophy. And this is a philosophy I have been introduced to by our passion. I don't think that it can work even for designers. We need to move away from thinking of design as a process with fixed deliverables and outcomes and move into a phase of continuous design. Not just having that continuous development, but you also iterate on your future design. You see what needs to happen to you. You're not afraid of cutting the stuff that you've done. And you can actually work it into cycles and release something. So even if it's a six month process, iterate on features, re-actuate, keep going over them, redo certain things, see how it's actually working. I think it's an extremely important thing for any product. So the question is, what is the team composition? So we started with one developer. I kind of joined into the process. So we had two developers. Around four days down the line we found Rahul as well. So these are the only three people who are involved from this company. We were working with Swabic for HTML, CSS. He was directly working on the board-based, which is one of the reasons why we could actually deliver it on time. But yeah, that's it for me. We have the quickest question of all. How much data is required for decision making? So there are certain decisions which we made based on feedbacks. Feedback from just three users. Things like the home page resigning, all that happened with more than 20,000 users. I don't have a good answer to your question. I would just say go on a good feeling where I have this data, do you feel it's enough for a given product statement? I really don't have a good answer.  So we generally think nowadays when we are taking decision, we have about for any decision making, at least 10,000 users going through the board. That's what the AP testing. Yeah, so we did conduct AP testing as well. But I personally found AP testing working for a certain set of problems, not for everything. As I said, my best, my most favourite way of getting feedback is sitting next to the person. Obviously you can't do that with a person. Anyone wants to criticise the design? Yeah, so the question is what was the budget or actual cost of doing this one month of development? It's pretty tricky because being a services company, we were always looking at opportunity cost. That means if these two people are not working on the product, they are working on some background. Yeah, on some clients. And the earning is certainly a lot more than the salary that I'm actually paying for this. But I guess in this one month, we spend probably two and a half to three lakh rupees in terms of salaries, in terms of infrastructure. Because when we release this within, after three hours, we had to increase, basically we had to push our infrastructure from one server to around five servers. So considering all told, yeah, it was around three lakh rupees. So we have two questions. One is what's our view on eye tracking software? Switch, basically you can get statistics based on either cursor movement or scroll movement. You can include some JavaScript and it will tell you that these many people were focusing their attention on this part of the application. That's eye tracking software. And the second question was, if we had any visual disagreements in normal process, I'm sure we have a lot of questions. Answering the first question, we haven't talked about this before. Yeah, we didn't use eye tracking software. We didn't do very gorilla usability testing throughout the process. We did not do formal studies on what users are looking at, which parts of the pages are getting board attention, apart from the basic data that you get by our embedded analytics. So we didn't use any eye tracking software. So we defended heavily on mid-spanel to give me the funnel, that is basically the interaction studies, right? Like where exactly users are clicking. In hindsight, yeah, maybe that would have been useful, but personally I don't really trust that particular data very much. So where did we disagree? We disagreed several on various occasions, most often on the home page. More or less, at all times I said, you know, sure, I love this agile process, etc.  So that's a call that I made and that's a call that an enlightened product company gives a designer the freedom to actually make that call. So that's an important thing. You work with the right clients, don't work with everyone. So one of the very important part, aspects of having a collaborative design is you have stake owners, or product owners basically, right? Like from a product perspective, if there is a future discussion as to what exactly we need to put in, my call would have been the final, like, okay, this is the future we are getting in, this is the future we are not getting in. In terms of UI, we would discuss whatever till the cows come home. Like, his call would be the final. So we had given these positions or, you know, authorities and we didn't challenge them. Because otherwise it's going to get picked. So one area I remember disagreeing quite strongly was the gamification. I thought it shouldn't go in. I thought the social aspects shouldn't go in. I still designed it. And though I think it's important to also, if I look back at my sort of mental state, I don't think I did my best work on those bits. So it's extremely important. This goes back to the point about being a product owner. Once you move from just being a visual designer to a product owner, you are able then to do your best work. Because you believe in really what's going to come out of it. And it's not just, you're not just working on features for someone else. And eventually we did leave the game to you as well. That's because of the estimations.