I'd like to call the order the Tuesday, September 19th, 2023 meeting of the Sheboygan County Board of Supervisors Are we certified in compliance with the open meeting law? Yes? It was posted at 11 a.m. on the 15th of September Please join me in the Pledge of Allegiance Next is roll call Supervisor Schobert, are you in remotely? Supervisor Schobert Yes, I am here. Thank you As long as we're checking with Supervisor Schobert, Supervisor Coulson or Ellis or Obler anyone else chiming in remotely All right. Thank you 22 supervisors present Next is the approval of the August 15th 2023 journal Supervisor Brower Mr. Chair, I would make a motion for approval of the minutes. Thank you Supervisor Brower Supervisor Wagner, thank you, Mr. Chairman. I'll move. I'll second that Thank you, Supervisor Wagner. Is there any discussion seeing no discussion, please vote Supervisor Schobert I Prove unanimously Next is consideration of appointments by County Administrator affirmative action Commission David Loomis Airport Advisory Committee Carol Lena Brian Olson and reappointments Mindy Smith David Hilper Schauser Lee Kunze Deidre Martinez Glenn Valenstein and Matt Grenoble Supervisor Wagner Thank you, Mr. Chairman. I'll move for approval of the appointments of all of them Thank you, Supervisor Wagner Supervisor Brower. Thank you, Mr. Chair. I will second that motion. Thank you Supervisor Brower under discussion Seeing no discussion, please vote Supervisor Schobert I unanimously Presentations Brian Grunewald of Clifton Larson Allen with the 22 annual Comprehensive Financial Report. I'm Brian Grunewald with CLA. Thanks for having me here this evening In terms of the audit discussion and the audit presentation I just want to remind you that there's two different audit documents the annual Comprehensive Financial Report As well as we also issue what I'm referred to as a management communication letter I believe you've seen those documents and for purposes of tonight's discussion Just pulling information from both those documents together Just wanted to give you a real high-level overview in terms of the content of those documents But if anyone does have any questions, please feel free to ask Okay, and in terms of the agenda real briefly just I'm gonna walk through and give you an overview of the audit results Similar to what you've seen in the past There's a few a few pieces of key financial information that I'd like to walk through and then last but not least just share some information With regards to new standards as we look forward and some additional information that will be dealing with the future and future audits In terms of the audit and our responsibilities Just want to give you a real brief overview and reminder in terms of management's responsibilities Management is responsible for the preparation of the financial statements You're also responsible for the design and implementation of your internal control structure Essentially, that means how you do things how you handle cash how you pay your bills for example Our responsibilities as auditors is to express opinions on your financial statements We do that by evaluating accounting policies and procedures We perform tests of audit procedures to gain audit evidence and we also consider the organization's internal control structure We do not issue an opinion on your internal control structure Again that being how things are done In terms of the results first starting out with the audit opinion And this is included in your annual comprehensive financial report on pages one through three The audit opinion is considered to be a clean or unmodified opinion Essentially means that I believe that your financial statements and your annual comprehensive financial report are complete and accurate and in accordance with our professional standards And just as a reminder that is the highest level of assurance that you can receive As part of the audit process And then in addition we also do issue a report on internal control again, that does not include an opinion But as part of the audit process we do consider your internal control structure We use that for planning purposes how we design tests for audit purposes When we do become aware of conditions that we refer to as findings we are required to communicate those I'm happy to report that we have not identified any issues or concerns with regards to internal control So no findings no reportable conditions related to internal control And that report is actually included in the management communication document on pages five and six Then just in addition to our review and our audit the county also Voluntarily goes through the process of submitting your financial statements to the government finance officers association as part of their review They've issued their certificate of achievement for excellence in financial reporting So just wanted to acknowledge that that is included in your annual comprehensive financial report on page six and that's certainly a Proceduous award. It's actually the ninth year that you've received that in a role. Congratulations on that and based on our review I fully expect that you will continue to receive that report going forward In this side, I also wanted to give you a real brief overview in terms of the management communication document There is a letter that's included in there I'm referred to as it as a governance letter it includes information that we are required to communicate as part of the audit process a Language in the terminology that's included in there is fairly consistent with what you've seen in the past It's a lot of very standardized accounting terminology as you can see on some of the references on the slide in front of you I just wanted to share that because it is information we're required to communicate from my perspective There's nothing in there that I would consider to be unusual certainly nothing that I would consider to be a red flag The one thing that I do want to emphasize or point out is you will see several references to the governmental accounting standards board and statement number 87 that's a new standard that was implemented You implemented that because you were required to so all other organizations that are following government auditing standards are also Implementing that what you will see that referenced multiple places throughout the government's the government's communication as well as in your audited financial statements So no other issues or concerns with regards to any of those topics And then real briefly just wanted to give you an overview of the financial statements and some of the key financial information Again, I know your financial statements are very large. This is just a summary of some of the key Items that I tend to focus on for example when I think of your overall financial condition the number one thing that I look at first Is typically your fund balance and your fund balance levels? What do you have for general fund and your fund balance reserves? So we'll talk more about that Another thing that I always like to look at when someone says hey, what's our financial condition is your general obligation debt? So you can see we have some additional information, and I have some additional slides relative to general obligation debt and Then in addition to that I just included some information with regards to Rocky Mullen Highway Really because of the size and significance of those funds and operations So again, this is a very short list in a real high level if you have any questions Any topics that you don't see here that you'd like us to talk further about Okay, then just digging in a couple slides here related to your general fund and your fund balance levels again So that's your reserves that you've built up over time The first thing that I always like to do is see what's happening from a trend perspective What is your overall fund balance reserves? What's happening? How are they fluctuating? You can see on the slide here if you just look across the top of that bar graph Your general fund fund balance reserves have been increasing steadily the last few years So certainly positive financially back in 2018 the total of all those categories listed there You're an approximately twenty one point five million if you roll that forward to 2021 you're approximately thirty two million And again if you end at the end of two thousand twenty two Approximately thirty four thirty four point five million for the year ended December 31st two thousand twenty two Your general fund fund balance did increase by two point four million And again if you do want to get into more details of why that happened a couple spots that I would recommend to refer you to There's certainly a detailed budget to actual report that's included within your financial statements on page that starts on page 32 And then also if you haven't read it the management discussion and analysis section of your annual financial report Does include in a real nice narrative overview that the county prepares and does touch on some of the budgetary highlights I just wanted to emphasize that or mention that as well So again if you step back certainly a positive trend in terms of what you see in the slide in front of you In looking at those different categories The one that I tend to pay the most attention to is that unassigned category That being available for contingency and working capital reserves And if you can see the trend there back in 2018 Approximately eighteen point five million and unassigned you can see how that's fluctuated over time a slight dip in 2019 and then increasing steadily for twenty twenty one and twenty two ending two thousand twenty two with twenty seven point seven million In your unassigned fund balance There are other categories there particularly assigned and committed and restricted There is additional information in your annual financial report on pages seventy four and seventy five if you're interested It gets into a little bit more detail about what's included within those categories. I Will say that at a high level the assigned and committed I typically refer to those as the categories where you have some planning opportunities the County can determine what they want to set aside funds for The restricted is really driven by outside third parties when you receive money for example They might be telling you what you can and can't spend that on So they're restricted by that outside third party and the not spendable is really set aside based on some accounting definitions that we work through So again high-level overview certainly positive trend in your overall and positive trend particularly in that Questions or thoughts on that before I move on Information there then from a benchmarking perspective and certainly the county has a policy in relation to this too When we look at that general fund and your unassigned fund balance Looking at how much do we have how much should we have? You know certainly your policy helps guide those decisions and what you're doing with that money And here again just giving you a snapshot of the recent trends of your overall unassigned fund balance You can see the policy minimum and maximum that being 30% and 15% and then that red line being your actual Experience for the last few years in terms of your unassigned fund balance in comparison to your budget or expenditures When you look at that trend that red line is moving really consistent with the dollars that we saw in that previous slide You start out 22.9 percent in 2018 a slight decrease in 2019 consistent with that decrease in your overall fund balance Excuse me the unassigned fund balance and then from 20 21 22 Again consistent with the growth in your overall unassigned unassigned fund balance dollars You'll also see the growth and increases in your unassigned fund balance as a percentage of your expenditures Ending 2022 at 29.6 percent. So certainly when I see that again very positive in terms of your overall trend Some of the things that I look for do we have a lot of fluctuation? Or is there a decreasing trend? Obviously those are things things that we look for and monitor Then overall in terms of that balance being 29.6 percent certainly very strong financially. I Was like to say that creates planning opportunities Yeah, yeah, so maybe just to step back and give you perspective. I mean CLA does audit approximately half I will say that certainly while they're we're in a very challenging time when I think Like inflation and whatnot. I do think there has been some counties that have been able to take advantage of things like ARPA funding To help to help with their overall fund balance position Overall every county that makes local decisions on where they want to be I would say there's a lot of healthy counties out there right now And I think you're in that in that mode as well So if there's a county that has a lower fund balance There could be a financial reason that's driving that or there could be a board decision to I think From my perspective for Sheboygan County again as we go through this you're going to see a lot of positive news financially I think you put a lot of emphasis on planning and budgeting and you can see that in your financial results There's a lot of positive news financially And I would just encourage you to keep doing that going forward again You know plan ahead budget be conscious with your budget and overall you're in great financial Thoughts or questions on that? Now moving on to Rocky Knoll is a good time for that question moving into Rocky Knoll Certainly when I think of Rocky Knoll an area that has been a little bit more challenging when I think of The impact of COVID and then coming out of that dealing with inflation Rocky Knoll and nursing home being a little bit more challenged and not just you that's throughout the state in that service This slide just gives you a snapshot of your overall and position again that being your equity reserves So you can see what's happened over time Now you do see steady increases in your net position your fund balance Approximately again some of all those categories Approximately 10.6 million back in 2018 you can see how that's grown Approximately 12.2 million in total at the end of 2021 you did increase in 2021 to 22 There was approximately like $800,000 increase and you ended 22 with approximately 13 million overall net position So certainly positive that you had that increase When you get into the individual categories You can see a little bit more detail there for example that net investment in your capital assets from an accounting standpoint That's how we account for and we depreciate your capital assets over time From your perspective when I look at this, it's how much do you have? What was your cost that's left in terms of those assets? And you can see the trend you've gone from 7 million up to 8.3 If you think of some of the remodeling and stuff that you've done out there You've been investing continuing to invest dollars in Rocky Knoll. So that's increased your net position It's increased that green category Again, you can see that in the trend If you jump back and look at that unrestricted category that blue category Again work across the graph. You can see you were at 3.9 million and how that's fluctuated and come down a little bit again consistent with the challenging times of COVID ending 2022 at about 1.8 million in terms of your again positive trend financially Overall, but just have to understand that Any thoughts or questions on that Then I also just wanted to take a step back and look at the net position for highway. So here to highway you can see Very steady increases overall if you get into individual categories your unrestricted category that blue category You can see this. I'll say steady increases there as well For example, 2019 you had approximately 4.2 million if you work across you can see 2021 at just over 5 million and then how that's increased from 21 to 22 up to approximately 5.8 million. So Overall increase in your net position And also the increases in that unrestricted category If you relate it to that net investment just to kind of correlate back to what I said with Rocky Knoll You know here to I see 37.2 million at the end of 2021 36 37 million at the end of 2022 very consistent in terms of your overall in that position there What that tells me is is you're continuing to reinvest for example buying new trucks So those get me those those are added to your net position. What happens from an accounting standpoint? We appreciate those so as we depreciate those placing them and overall you're fairly consistent And I know you do pay a very detailed attention to what you do with your equipment So there I think again the takeaway is probably the overall increase in your unrestricted So certainly very strong Lots of questions on that And then I also just wanted to step back and take a picture of your general obligation debt Really two parts to this slide the top part I just wanted to give you a visual of your overall outstanding general obligation debt and what's happened over time You can see back in 2018 Approximately 36 million of outstanding debt you can see again. What's happened as you as you've repaid principal at the end of 2021 You're approximately 23 million of outstanding debt And ending 2022 with approximately 24.7 million again. I know that's very detailed and laid out within your budget in your budget model So just wanted to give you a snapshot of that The bottom portion of that slide and why I included this in here There's also a statutory debt limit So you can see that label on the slide the statutory debt limit is based on the equalized value of the county You can see how that's gone up over time Really certainly a positive indicator in terms of the overall increases in the last value of the property within the county increasing that debt limit that statutory debt limit and From a benchmarking perspective one of the things that I always like to look at is how much general obligation debt Do you have as a percentage of that limit and here too on the side you can see from 2018 2020 how that's fluctuated so 2022 you actually In terms of percentages those are I would say relatively low whenever I talk about that I think it's also important remember that You know, it's my job to come in here and report on your financial statements Your financial statements include a lot of disclosures about the debt balances that you have Obviously from the board's perspective you got to continue to Assess the needs of the county and how you're going to pay for those So I'm not trying to tell you what you should or shouldn't do with your debt All I'd say is if you do issue new debt make sure it's built into your budget Which you've done in the past then also from a benchmarking perspective one of the things that I like to look at is How much debt service meaning your principal and interest payments? What are our debt service payments in comparison to the size of our budget? I'm just focusing here on your non capital expenditures and then expressing that as a percentage and this information is coming right from your annual financial report on page 118 There's a common industry benchmark by rating agencies of 20 percent That's the red line if you look at the blue line. You can see your actual experience And and you can see the decline there and what's happened as your debt payments have come down That percentage has come down. So back in 2018 approximately 9.8 percent 2022 you get about 6.9 percent and again what that's telling you is how much of your budget is used Fact that the gap between the red line and the blue line is widening is certainly Okay, and then last but not least, you know again, I did reference new accounting standards There was a standard that we implemented for this year. So Want to acknowledge your team your finance department for the assistance of working through that process There are additional new standards that will be required to be implemented for the next year's audit to 2023 audit There's another standard you can see listed on your number 96 with the subscription based IT agreements I know your county has started the process of gathering information on that There's also another standard there label as auditing standards 143 to 145 Just wanted to acknowledge that Understand that we're sharing the information with regards to the requirements of that Certainly, there's more information on our website if anybody wants to take a look at those topics But we will be working with your finance department And I went through that pretty quickly any thoughts questions concerns. Thank you very much. Thank you Are there any public addresses there are none letters communications and announcements County administrators report Good evening everyone I want to start this evening by saying thank you to Brian Not only for being here this evening to share the results of our Financial status, but also for sharing all of the good news Certainly good to hear that we're in a very positive place financially. So thank you Brian and quite fitting not only the report on our 2022 audit, but the themes of being conscientious and planning Fits well with one of the items on the agenda tonight, which is the five-year capital plan And so I wanted to just quickly touch on that. So that's resolution number seven And these are for projects that are over a hundred thousand dollars the proposal before you tonight is within our bonding limit of 11 million and I just wanted to touch on a couple of the significant projects that are included in the five-year capital plan One of those being the Marsh Dam replacement, which is already underway, and I'm pleased to report that that's going well I don't want to jinx it, but thus far it is on schedule. So pleased with the progress of that Project in addition one of the other significant projects included in the capital plan is Courthouse tuck pointing and this is a once in a century project So this will be a significant investment for the county in this very building and that will include tuck pointing The exterior of the building as well as replacing all of the windows in this building and of course Maintaining the historical Appearance of the building so that will be a significant project for us in addition, there's other Regular items that we often bond for such as elevator upgrades roof replacements other routine maintenance pieces In addition one of the newer projects added to the capital plan is radio upgrades for the sheriff's department There's a couple projects at the airport with some taxiway and runway projects And then also equipment and some upgrades at Rocky Noel And so I'm pleased to share that all of the projects that are included in the capital plan are Responsible requests from our department and they are needed for Maintaining our facilities and being able to provide the quality services to our residents and Next I wanted to just quickly touch on and recap the Wisconsin counties Association conference that we just Returned from myself and 10 of your fellow supervisors were in the Dells for the last two days And actually we just got back a couple hours ago. So Hot off the presses. I wanted to just touch on a couple of the Highlights from the conference for those who were not able to attend it was held in Wisconsin Dells this year and These conferences are a great opportunity to not only hear from speakers and subject matter experts But also to network with colleagues So not only do you get to network with other county board supervisors from across the state but you get to see other county staff and hear what they're doing and Learn from them and discuss ideas So it's a great opportunity to connect with people that you may only see once or twice a year This year they had a number of interesting speakers as part of either breakout sessions or the general conference Sessions that everyone attended that included governor Thompson Governor Evers the DOT secretary Thompson in the DNR secretary pain I don't think any of us have ever heard him speak before so that was new No And in addition today our final general assembly speaker was a gentleman He was the former CIA chief of counterintelligence and I thought he had a really great message and Hopefully everyone that attended the conference was able to stay. I know it's the last portion of the day but he talked about his time as a spy and Taking significant risks for the greater good and just a calling to a career in public service And I thought he had a really a nice message. It was very interesting. He interjected humor It's not often that you have a room of hundreds of people and everyone's on the edge of their seat paying attention Closely to a speaker, especially is the last session of the day. So I thought he did a really great job In addition we got legislative updates We had a number of breakout sessions covering everything from a to z Related to county government including cyber security mental health parliamentary procedure child care shortages PFAS next gen 9-1-1 broadband Transportation housing. I mean you name it. They had a really wide variety of topics that you could hear from and Throughout the conference one of the themes that kept Cropping up during the different sessions was really about collaboration Partnerships working together, etc And so I'm pleased to share that during a number of different instances by a number of different people Sheboygan County was brought up as a leader in a number of different areas across the state So it I think I share this sentiment with the supervisors It brings a lot of pride to be at a state convention like that in here Sheboygan County Being talked about as a leader in some of these areas So overall it was a great conference and for supervisors who haven't attended in the past I encourage you to consider participating in the future Next year it will be held in La Crosse and we again are requesting a budget for a Number of you to attend so I would encourage you to take that opportunity And that is the conclusion from my remarks this evening All right next is consideration of committee reports executive committee resolution number seven regarding 2024 five-year capital plan recommendation to adopt Supervisor Gehring. Thank you, Mr. Chairman. I move for adoption. Thank you supervisor Gehring Supervisor Brower you mr. Chair. I will second that motion. Thank you supervisor Brower. Is there any discussion? Seeing no discussion, please vote Supervisor Schobert, how do you vote? Hi Thank you Next is resolution number eight Authorizing County aid for covered bridge replacements in the towns of Holland Lima Linden Mozo Wilson and Sheboygan Falls Recommendation to adopt supervisor Wagner Thank You mr. Chairman. I'll move for passage of resolution number eight. Thank you supervisor Wagner supervisor Tastrodi Thank You mr. Chairman. I'll second the motion Thank You supervisor Tastrodi. Is there any discussion? Seeing no discussion, please vote Supervisor Schobert, how do you vote? Hi Number eight's privy unanimously Coordinates number five regarding modifying fee schedule of the medical examiner in chapter 96 recommendation to enact Supervisor Gehring. Thank you, mr. Chairman. I move to enact Thank You supervisor Gehring, Supervisor Jorgensen I second that. Thank You supervisor Jorgensen. Is there any discussion? Seeing no discussion, please vote Supervisor Schobert, I vote aye. It's also privy unanimously Alright under resolutions introduced resolution number nine Finance committee regarding authorizing the issuance and sale of 10.9 million of general obligation promissory notes Resolution number nine will go to executive committee resolution number 10 guarding authorizing application for DNR surface water grant Resolution number 10 will go to finance and resolution number 11 Regarding accepting donation of property Added to the Sheboygan County Broad and Marsh Resolution number 11 will go to executive ordinance is introduced ordinance number six Finance regarding amending chapter for the investment policy and procedures Ordinance number six will go to executive Next supervisor to Strowdy Move to adjourn. Thank You supervisor to Strowdy Supervisor Brower. Thank You mr. Chairman. I will second the motion. Thank You supervisor Brower Please vote Supervisor Schobert. Hi. Thank You and good night. We're German past unanimously


This is Rob Johnson, president for New Economic Thinking. I'm here today with Dr. Glenn Hubbard from the Columbia Business School, who's formally worked in George H.W. Bush's administration and in the U.S. Treasury. And secondly, he was the head of the Council of Economic Advisers under George W. Bush, H.W. Sun. He, from my perspective, has a tremendous awareness of both the nature of markets and economics, the history of economic thought, as well as a deep comprehension of that intersection and inseparability between politics and economics. He has a new book that's coming out that is absolutely extraordinary. Yale University Press has talked about it and John Lennon used to sing about it when he wrote his Walls and Bridges album in 1974. Glenn, I was inspired to learn of what you were writing, but as I read it, I became more inspired and I want to thank you for joining me today. I know my young scholars will be very deeply nourished by our conversation. Well, thanks, Rob, and thanks so much for having me. What, you know, I really like to start with, you're looking at the world. You have a lot of awareness and intellect and experience and scholarship. Like all these receptors, I'll call them. What inspired you to write this book and this message right now? And then we'll talk about what is within the message. But what tickled your fancy? Made you reach into yourself and go after this book? Well, thanks. I think it's a great question. You know, one of the reasons I became an economist was I actually believed what classical economists and enlightenment thinkers like Adam Smith believed in mass flourishing in a society. And it requires no deep insight on my part to observe that we're not seeing mass flourishing right now in society. I saw this up front close and personal as an economist in George W. Bush's administration. You know, I led a battle against steel tariffs. The president made the opposite decision. He told me perhaps just to be nice that he wasn't disagreeing with me on economics, but he felt I had the wrong idea. I had all of these ideas about maximizing national income. But what was I going to say to people in Wheeling, West Virginia or in other places? That put me on the road to thinking about things in a different way. I've taken MBA students over many years to parts of a heartland that have been left behind by change. And economics really does have an answer here. It goes back to Smith and it's not capitalism versus socialism. It's walls and bridges. There's the seduction of the wall. If you don't like change, whether that changes from technology or globalization, we'll build a wall. We'll lock it out of your life. We'll make it 1955 again if you'd like. That doesn't work. It never worked. Smith wrote The Wealth of Nations to say that idea was crazy and doesn't work. Yet doing nothing isn't the answer either. The modern Smith would build bridges, helping people compete, helping people to adapt. There are tangible things government can do, business can do, and individuals can do. That's the debate we need to have. The antidote to the wall is the bridge. And I want to add a little more nuance or texture. What's a wall and what's a bridge? How do they differ? Well, I think of a wall as sand in the gears for change. So it could be literal. Go back to Roman times, the idea of building a wall to guard against the other. It could be metaphorical, but we'll simply block out or high tariffs and trade against other parties. Smith, when he wrote The Wealth of Nations, was reacting to mercantilism, which was the standard economic view of the day. Trade surpluses are good. The purpose of the trading system is to bolster the golden silver that the sovereign would have to fight wars and personal luxury. Smith stood all about on its head. He said, no, no, no. It's consumers. It's average people that are the center. And the wealth of the nation is their ability to consume. And that ability to consume needs efficiency. It needs openness to change, whether that's to trade or the adaptation and adoption of new methods. That's a very important method. But you know, economists have talked about that for generations. And I've written a freshman textbook. So a lot of people hear my own views on this. We talk about the gains from openness. And we do also, at least mumble, that there are people left behind as a result of that openness. To winning on average doesn't mean that everybody wins. And there's always the expression that gainers can compensate losers. But the question is, do they? And how do they? And that's why we're in a thicket of populist wall discussions. And discussions of trying to protect people as opposed to empower people to compete. So in one sense, the wall is just trying to stop the transition. The bridge might be to use public resources to facilitate the transformation and bring them along with the higher level of prosperity. Exactly. Because if we stop the dynamism to use my colleague Ned Phelps, Nobel War, the economic term, to use Ned's term about dynamism, if we stop that, we are literally killing the golden goose of our future prosperity.  We don't want to do that. At the same time, we do have to be prepared to help people compete. Now you mentioned government. Again, I don't think of this as capitalism and socialism or welfare versus no welfare. It's what do you want government to do? So two big examples I use in the book were the genius of the land grant colleges in the moral lab. Lincoln, in the middle of a civil war, believed that we needed to jump start opportunity. The nation was making a transition from agrarian to manufacturing and commercial. It needed help across regions to do that. It needed to be able to prepare people. Franklin Roosevelt had a similar insight with a GI bill. With all of these soldiers returning from force to the United States, how would they be prepared to compete in the new economy? So government doesn't have to provide extra welfare. It doesn't have to own things in a socialist model, but it does have to be a battering ram for opportunity. Smith himself saw that in the wealth of agents when he talked about the importance of infrastructure and public goods and education. His economy, of course, was much simpler than ours. The Smith today would be more focused on a bolder use of government, but in the way Lincoln or Roosevelt might have done. So in the spirit of transformation, the disruptions that have been with us, our globalization and technology, and so I'm going to ask you kind of two questions because the one on the horizon is climate change. What should we have done? Let me let me back over a second. I've made a lot of podcasts with people about what's going on in West Virginia, Bob Pollan and others, and they say in West Virginia, if you're watching social media, people will say, perhaps sponsored by the fossil fuel industry, look at what they did to Cleveland and Detroit. They went through that change, globalization, automation, whatever, and you got nothing. That's what they're getting ready to do to you because they're trying to mobilize, which you might call more support, against the transformation. And my intuition as I was listening to these arguments is you're right that we didn't succeed in Cleveland, Detroit, or what you might call the automotive transformation, but what do we do now credibly for the people of West Virginia? So that's the forward-looking, the slightly, I wouldn't even call it backward-looking, the contemporaneous look is how with globalization and how I grew up in Detroit. I watch people talking about you can't raise taxes, you got to reschedule in the Detroit bankruptcy what's happening to women who've worked 45 years in municipal government, we're going to cut their healthcare and cut their pension from 19,000 to 12,000. Bankruptcy in the private sector when you don't have revenue, bankruptcy in this context, which was authorized by the courts, is when people are afraid to ask people to pay because they think they're going to leave the region. What do we do right now? Looking at what happened with globalization and technology, what do we do vis-a-vis climate change is the one making everybody most anxious now? What does the bridge look like that we should have done? What does the bridge look like we should do next? It's a great question, Rob. If you go back, what we should have done is remember the admonition that Nicholas Caldor, a great economist, had said that when you're making a change, it makes us, on average, better off. And by the way, globalization and technological change do that. They make us better off. We should not ever forget that. You need to compensate people left behind. And what Caldor meant was not write people a check or pinch them off, he meant in Smith's sense, you know, giving them that ability to compete. So we had programs going back in the day from the late 60s on called trade adjustment assistance. They're pennies on the dollar. They were nothing. In fact, politicians rarely even spoke of them unless they wanted to expand trade agreement. We could have had bolder initiatives. And in my book, I mentioned the idea of applied research centers to borrow from the land grant moral act colleges, a block grant for community colleges, which nationally are the foot soldiers of training people for a different economy. Both of those ideas, like the moral act, build on the need to make decisions at the local level. There isn't a one size fits all problem and solution. Detroit is not the same as Youngstown, which isn't the same as Wheeling, West Virginia. We need to let local business people do it. Now, we know it can happen. So Pittsburgh is an example of a city and area that did in fact reinvent itself. The steel industry's demise was met with a big investment in education and healthcare. In the case of Pittsburgh, that was led by local business people who were trying to make that pivot. I don't think we can count on that everywhere. I do think government has to provide regions that means so economists used to be very suspicious of what you might call play space eight. We used to say, well, we should just move people to wherever the jobs are going to be. Well, for a variety of reasons, people aren't moving. And we do need to think about communities that are distressed and giving them the chance to adapt to new industries. So that takes me to West Virginia. So you could be forgiven in West Virginia for not worrying, why am I not going to meet the fate of Detroit or Youngstown? And I think the idea is not to promise people that we're going to continue to use a lot of coal forever because, you know, frankly, we're not. It doesn't matter whether you're coming at it from solutions from the left to the right. I'll come to those in a moment. That's not going to happen. But what you can say is, what are ways we can prepare you for a different economy? And I think there are abundant ways to do that. Again, it goes back to the kinds of training programs in play space, aid and luring, luring businesses. You know, climate change is a great example. What government should could do is use a price on carbon as a way to get business to innovate. Business is more efficient at deciding how we're going to decarbonize than government. But businesses aren't necessarily going to come to that on their own without the incentive of a price on carbon. So you need both. And it would be entirely appropriate to use some of the revenue from a carbon tax to find transitions, you know, so that individuals and areas are there. And they use the word transition guardedly because, you know, one of the things I learned in researching the book, to my chagrin, talking with a lot of people affected by change is, you know, they're just alarmed when economists refer to things like transition costs. Would you like to be called a transition cost? I don't think I would. I think I'd rather believe that I'm a human being with hopes and aspirations. And maybe I'm not going to be a great entrepreneur, but maybe I have my own aspirations. And I think that that kind of dignity goes very much back to Smith, the Smith of the theory of moral sentiments and economists lose it at their peril. So I think we have answers going back and we have answers going forward. And walls I know will fail. Bridges I believe can succeed. Well, I think there was a key word you said at the outset. When you said technology and globalization can make us better off. I agree with that as long as us is defined correctly. Who is us? Because a lot of people experience being outside of us, like you said, called the transition cost. And we've got to we've got to establish in the private sector and we're watching a lot of evolution towards multi stakeholder awareness, ESG kind of work and so forth. We got to understand that the dynamisms, which am I called vitality and possibility, is at risk at this juncture of social rebellion. That is exactly the case, Rob. I think too often business leaders, and frankly, economists as well, assume that social support for the system or capitalism is given. And then we use policies to tweak it around the edges. I think that's up for grabs. Even teaching in a business school, I have many students in political economy who would question whether capitalism is really working for the economy. And they would raise it in the same way you did of who's us. So if your definition of us is all of us on average, it's definitely working. But of course, none of us is exactly average. Some of them, hopefully including all my students, will be decidedly above average. Same is true of you and of me. But it's not true for everybody who's influenced by change. Yet we all are part of the same community. You know, another thinker that I mentioned in the book is Carl Polanyi. You talked a lot about markets is needing to realize that there's a tissue, if that's the right word, of connection from communities and values. And you can't operate in the neoliberal way that the market system totally away from those communities. So I think that the debate is much more subtle than this capitalism, socialism debate in Washington, and much more about what kind of bridges can we defy to beat the wall flavor of the month and the left has its walls, the right has its walls, but they're all harmful. Yeah, I'm sure you'll understand what I'm saying. Years ago, when Occupy Wall Street was unfolding, a friend of mine, the late James Cohn, that uni theological seminary, challenged me to come teach a thing I called the title of course was economic symbology to the graduate students. And my code word for that, my code phrase was means and ends. And what I was trying to show them was why people at that distressful time were so adhering to the market logic. But they'd almost made an error in deifying the market as opposed to seeing it as a tool to reach moral and philosophical ends, of which it is a very valuable tool. And it was very interesting to watch them unfold and also see the promises. I remember teaching them a book by a famous author Christopher Lash called The True and Only Heaven. And it was about how when you tell people to delay gratification, meaning wage demands, it creates capital formation and productivity for the future. But Lash, who wrote the book in the late 80s, came with a very interesting perspective, which was that was working until there was globalization. Because now in that 70s, 80s period, which you refer to in an early in the book about, you know, when you began your awareness and education, people started asking you to delay gratification for foreign direct investment that came back to compete with you. And which you might call that salvation in the future, that secular salvation was postponed or eradicated. And there's also a lot of discussion in that course about Martin Luther King, a Philip Randolph and others, who didn't go out and create a civil rights budget. They created something called the freedom budget for all Americans.  And in particular, they were advocating, rightly or wrongly, that militarism was eating a big hole out of things that all Americans would consider essential to their well-being. And so I watched the kind of mindset and I invoked chapters from the theory of moral sentiments to go to the place where you are right now, which is at the Union Theological Seminary. The market wasn't a deity that you had to get out of the way of. It was a vital and important part of a process toward achieving social ends, which the early Adam Smith articulated, of which you're bringing back to the surface. Well, I very much agree with that, Rob. I also think it's something perhaps more subtle that economists and sometimes policymakers treat this market worship as a very technocratic set of problems. So you mentioned Occupy Wall Street. So the view of the bank bailouts at the time were that they were very important for the plumbing of finance and the economy, and I share that view. I shared it and share it today. At the same time, our failure to do other things, a colleague of mine, Chris Mayer, and I wrote a proposal in 2008 for a mass refinancing of home mortgages once all the credit risk had been taken on to the government's balance sheet. Secretary Paulson did so for Fannie Mae and Freddie Mac. We didn't do that. And so you could forgive a lot of average people from saying, you know, huh, how come Wall Street got bailed out, but I got stuck paying mortgage payments that were very high, and it really undermines this trust in the system. You know, I thought the most thoughtful comment during the financial crisis actually came from the Queen of England. When she said, why did nobody see it coming? She said it to a group of faculty at the London School of Economics. I expect with a little bit of regal condescension that you guys are so smart, how come you didn't see it coming? I'm glad she didn't come to Columbia because I would have been just as embarrassed probably as the LLC colleagues. And I think part of the answer is we don't get out much. You know, Smith did. He talked about a pen factory for God's sake, not abstract intuition. We need to get out and see the concerns of average people and of average business people rather than retreating to worship of the equations of the market. Well, I know because of my proximity here in New York to Columbia University that you and people like Ned Phelps who you mentioned and Joe Stiglitz and others nourish one another. There's a dynamism to the debate there. I just went to Ned's conference recently and it was a very open-minded conference about the challenges and it wasn't a defensive, you know, circle of my wagons conference. These people really high calorie, Eric Maskin, everybody was going for it. But I remember when Joe and I were working on a UN Council on Commission on Global Economics and Finance 2010, 29, 2010 and Joe said we're going to have a real problem. It became a famous quote because the polluters are getting paid. He was drawing an analogy to climate change, which was we bailed out the people, but we didn't do the transformational things. And by the way, that part, I know you worked with Paul Sen, you worked in the Bush administration. That part was largely the responsible of the Obama administration. Right, of course. And they didn't realize that, which I might call fairness or distributional vision. And I know, I don't know the man, but I've heard Steve Bannon give speeches where he said what brought you Donald Trump was the failure of the structure of the bailouts of the 2008, 2009 financial crisis. That's got to be at least partly true because again, what President Trump brought, and you know, frankly, he's not alone. He has competitors on the left doing it too. We're walls, you know, very seductive stories that I can make the change go away. That of course isn't true. And it isn't going to serve people's interests. But you know, if the alternative is an economist who simply says, trust me, the market will work. Well, that's not making me say I'll leave it. Yeah. But in Detroit, right after, and my guests on this podcast have heard this before, but right after the nomination of Donald Trump, I believe it was in Cleveland in a convention, he went to the Detroit economic club and he wailed on the big three management for losing jobs in Michigan. And he said, this system is rigged. Now the state of Michigan has obviously been through a lot, but all of my friends with graduate degrees, JDs, MBAs, CPAs, MDs, the whole thing all voted for Trump because he's somebody's finally calling out this. Right. And they were disappointed in me because I didn't. I was at a dinner with them on election night in Detroit as it turned out. And they said, Rob, you know, you brought up your family in New York and so if you don't understand how stagnant this is, et cetera. And they were critical, obviously of Obama's restructuring of the auto industry, albeit allowing plants to be built in China and Mexico with the money, protecting the white flight, as they called it, upper management service class, but not the manufacturing class. And there was a lot of discord and they felt like Trump was the solution. I told them, well, at this dinner, I lost seven to one in terms of who I voted for. But I don't think you're going to be happy with the outcome. Because like I said, he'll build walls and he'll seduce and abandon, but he's not going to do the dynamism. We know, of course, it's an old story. And again, this was the very story that got Smith angry enough to write The Wealth of Nations, a canard of mercantilism. But you know, we just need to block out the other and trade and it'll help the domestic economy. Of course, nothing could be further from the truth. But if the only counterpart to that argument is what a rip, which would be sort of the neoliberal version of the story, I think that's a problem. We need to put the liberal back in neoliberal, i.e. Smith and the classical thinkers who did care a lot about mass flourishing and the dignity of every person, not just the most successful. Yeah.  And as you described, and I really enjoyed that part of your book, because when I'm asking myself what are walls, you're what you might call passage through mercantilism and how Smith saw beyond it was very important. It was very illuminating as to what you meant and what we shouldn't be doing. Well, ironically, it was falling walls that got us into a lot of the modern mess. It was the collapse of the Berlin Wall, the so-called death of distance, the lower cost transportation, communication, all these are good things. But suddenly they made the economy hyper competitive at a time when we really had not prepared people for that competition.  None of that plays to the dignity. Nobody wants that. They want to be part of a society. And unless and until we do that, we're not going to have broad social support for the system. And we shouldn't be surprised if we don't. I've spent a lot of time, I went into the private sector in 1989 and was involved in the hedge fund industry. And for a number of years, I ran the Don Japan Asia portfolio for George Soros. In that process, just after Tiananmen Square, I got very involved in visiting and dialogue that's always going on. And what's really interesting to me is when Donald Trump came into power, a number of the Chinese leaders, and I'm talking, not I don't have a magician pain, but all the next layer would say to me confidentially, I'm watching this guy demonize Mexico and China. But it's wasn't we, they said they were very coherent about this. We had one 40th of the capita income of Americans at the starting gate, say 1979, 1980. Deng Xiaoping comes in. We're transforming. We're four times the size of the American population. It is going to be the globalization, the integration of our two sites is going to be changing relative prices sectors. It's over the profoundly, but it wasn't within our power, meaning the Chinese to create that transitional sexual movement assistance support education. And the anger that is now directed at us is something we could we understand where it is. But it was the responsibility of American elite governments to handle that transformation. And I think that's actually very insightful and true. And that fall was dropped. And it's not a partisan comment. It was dropped by Democrats. It was dropped by Republicans. You know, people think of Republicans as being the champions here, but you know, no one was more neoliberal than Bill Clinton, who talked about globalization as being like the rain. You know, you can't stop that. I think Tony Blair said something to the effect of it's like autumn following summer. You know, it's just, it is what it is, which is a kind of hands off, we're not going to help you every person for himself or herself. And that doesn't work. You know, China and its transition was heavily influenced by Janos Kornai in trying to gently wean away the state sector. So when Deng Xiaoping opened up the economy, there was a more gradual incremental change rather than what today we might call shock therapy to use the Russian or Eastern European example. And I think it benefited China and in the US, you could have imagined a system that used maybe very large wage credits for a creative time or things that would give firms a transition as people moved from one industry or type of work to another. Of course, we didn't do any of that. Yeah, I'm really inspired by what you just said, because there are people like Julian Gourwitz who wrote the book about Kornai and the transformation and, you know, people from Milton Friedman to James Tobin and everybody were involved. And then people like the, she was born in Germany, the scholar, Elizabeth Webber, they are talking about a transformation process in China that's neither state alone or markets alone, but it did have a social awareness. And the reason I'm underscoring that right now is I don't have a, I have a lot of fear of some of the things that Chinese leadership thinks are legitimate to do. But in that realm of economic adjustment, there are lessons to learn in how they transform themselves in a way where, I mean, it wasn't like you said, shock treatment, like Kornai won't just let it go, let everything collapse and then rebuild the wreckage that created the transformation, a transition that's not unlike the challenge we probably have to make right now related to fossil fuels.  One is on artificial intelligence, which is probably the latest general purpose technology, perhaps along with another robotics that could completely change the nature of work for many more Americans, including some Americans who would consider themselves very skilled, maybe even somewhat professional people. The other is climate change, which is probably one of the big social problems of our age. Both of those call for innovation, both of them call for perhaps at least looking at where the boundary is between the market and the state, and both would benefit from the Lincoln's and Roosevelt's of the world saying, what's a way that I could use what I have in power as kind of a battering ram to open up opportunity for people in this transition, and then let the private sector work out that transition as efficiently as it can. Let's talk a little bit. I think I've mentioned to you at times that in our preparation for this, that a lot of times people treat the market in the state like they're separable domains. There's a lot of concern now about the role of money in politics, which you might call, I've used this phrase in previous part, the commodification of social design and enforcement. And I'm very nervous that at times there is no sense of what you might call the incentives of governments, that allow it to represent the we rather than donors. So while I see exactly the pathway that you're, how we say, putting into the mix with the markets, how do we allow that governance to shape things in a way that is for the moral purpose of the collective? Well, you know, this is one of those where I could be very naive, not being a politician, where the right thing looks like a political winner. If you do think more collectively about everybody in the same boat, then it becomes easier to think about a policy of a different kind of approach to community colleges or to federal aid that helps communities, all of that should be possible. So I think this is actually a winner, not a loser in the political process, particularly up against the liberal vision of what it read, but also against the vision of what's built walls around the country. I just don't think it works. So, Glenn, when we're talking about how to build bridges, and there's been such little trust in government, there's concern about capture and so forth. What historical, which you might call experiences, would you point to, to give people a guiding light as to how to do it again? I know you talked about Lincoln and you talked about Franklin Roosevelt. How did they jumpstart confidence in them? And what can we learn from that now? Well, Roosevelt's a great place to start, Rob. You know, people normally think about the Roosevelt of the early New Deal, but I think apropos of the bridge, more the Roosevelt of the GI Bill, and the huge desire to reintegrate returning service people into a dynamic economy, the same was true of the big Lincoln era reforms. You know, doing the right thing of using government as a battering ram, really to make sure everybody has the ability to compete, to use Smith's terms, is the right answer. And I think it's not that people quote, trust government or don't quote, trust government, it's a matter is government using its energies to empower people or to frustrate them. If it's empower, that trust will come. So at this juncture, let's look, let's look on the horizon, that climate change. I know the technology issues, but how do you see coming down the path? Let's just say President Biden or his successor brought you back into the White House. What are the pieces? I know, you know, from reading your book, there's private sector engagement, understanding, there's government sending incentives, there's transformational support for the population so they go along with the plan. But give us the vision of how you would be if you were the czar of achieving climate change and perhaps have a little international collaboration because this isn't just an American problem. It's a global problem. It's a great question. You know, climate change like artificial intelligence will be one of the forces that shapes and transitions our workforce, our economy, our public policy, and our well-being, frankly, for decades to come. I think what government can do is first, in the case of climate change, put a price on carbon. If we want business to do what it does best, which is innovate different solutions, it needs a price on carbon to do that. But government can't simply do that and then assume that everything else will take care of itself. That would be repeating the same mistakes of globalization and technological change and forgetting people and transition. So there's still a large role in place-based aid and individual assistance to help people get ready for that new economy. We're certainly up to that. We've done it at other transitions and turning points in our economy and we could certainly do it again. Climate change, in a sense, offers a tailor-made example. Because it involves a tax putting a price on carbon, it also offers some of the funds that might be needed for exactly that purpose. I know people like the economist James Boyce who talked about a carbon dividend where the angst of the body politic can be offset a little bit by using some of the proceeds to assure them that they won't be devastated or left behind. But I would also say we really need that engineering expertise related to the new technologies that are necessary. And then the final piece that lots of people are knocking on my door about that I'm curious of your perspective is they're saying let's just use Africa as an example. Africa is an equatorial region can benefit a great deal from solar power. But the risk premium people pay for private investment on that continent is very high. Should we be creating governmental guarantees to put those, in other words, any default risk premiums or fear of the integrity of their capital markets? Should we all bear that together because we're all going to receive benefits if they do realize that climate change technology? Well, you know, that's a that's actually a difficult question. I would think before doing that one wants to make sure you have the kind of governance of projects and economies that make sure that you're getting the right investment. But yes, you know, one of the things that's got to be true, not just of climate change, but frankly the pandemic, rich countries have to realize that we are all in this together. Where you're talking about the pandemic or climate change, there's no America only pandemic or America only problem with climate change. So some kind of transfers have to happen. I like to think of them as mainly being in the technology area, or in the case of the pandemic vaccines, that this is an area where we have a responsibility. And frankly, it helps us.


Good afternoon stage B My name is tear father Gill and I will be your herald for a little while My main purpose at the moment is to introduce you to the absolutely brilliant Rob Miles who will be talking about AI safety Thanks Hello everyone. I want to clear up a couple of things first off Or you know start start off with some things I want to first off say that I am not an actual expert on this subject I have a YouTube channel where I share this research Because I think it's important and I think it's really interesting There aren't very many experts in the world. I'm not one of them. I'm also not Rob Miles the lecturer from Hull University. He's here somewhere. I think I'm not him if you're looking for him He's around anyway So and we don't have quite as much time as I was hoping so we're gonna move fast This is the question that I am trying to get people to ask themselves with this talk What is the most important problem in your field? Take a moment to think about it And why are you not working on that if you're not working on that? So I'm gonna argue that I think the most important problem in the field of AI and Possibly in the field of computer science possibly in the world is AI safety So why what's the risk here from AI? Pretty much you can divide it up into four categories along two axes, right? You have your short-term stuff your narrow stuff This is the kind of problem you can get with AI systems as they currently exist or in the very near future and this varies by by Misuse or accident right it is this somebody using this technology With the wrong aims or is this somebody just making a mistake and the technology is doing something that's unintended So I've got some examples there. I don't have time to go through What I'm mostly interested in and what this talk is mostly going to be about is the long-term risks or the general AI risks and in this case I feel as though General AI systems are so difficult to Understand and control that It almost doesn't matter who's using them So the risks from misuse are quite similar to the risks from accident because the chances are it's very unlikely that Somebody's gonna make general AI and use it to do something bad It's more likely that it's going to do something bad entirely of its own accord So that is what I think is the most important problem. Thank you. Wow. That's a lot of water. Okay So This is this is like the statement of the problem I think that we will sooner or later build an artificial agent with general intelligence And I'm gonna just go into each of these things and explain what I mean by them. So first off sooner or later What do I mean by that? This is a survey from a bunch of AI experts These are people who published in major AI conferences ICML and nips and they were asked to survey about when they thought we would achieve Human level machine intelligence is what HLMI there So that's a that's a system that's able to perform any cognitive task as well as humans or better And so this is gonna work. Yeah, this is the outcome point with this So if you look at the point at which they think that we get a 50% chance of having achieved that this red line is their sort of aggregated Probabilities that comes down about that didn't work about 45 years from 2016 But then if you're looking at let's say a 10% chance that comes in about nine years, so We're talking about that general kind of timescale worth taking with a huge pinch of salt because if you ask the question very slightly differently You get numbers more like 120 years, but in general we're talking about that that that range and ultimately This is a serious problem if it happens in 10 years I mean it's an impossible problem if it happens in 10 years. It's a serious problem if it happens in 45 It's still a serious problem if it takes 120 or 200 Because and I think that we will get there sooner or later, right? We know that general intelligence is possible It's implemented by the brains of humans Brains aren't magic. We will figure it out. This is going to happen eventually So when I when I say an artificial agent, what am I talking about? agents are a An abstraction from economics and like game theory decision theory that kind of thing And they basically they just things that have goals and choose actions to further those goals So like the simplest thing that you could think of as an agent is something like a thermostat It's kind of slightly absurd to think of something so simple as an agent, but it's sort of the simplest possible agent So it has a goal which is for the room to be at a reasonable temperature And it has actions it can take in the form of like turning on the air conditioning or turning on the heating or you know Turning them off adjusting the settings on them or whatever it has some actions It can take and it chooses which action it's going to take in order to achieve its goal It's like extremely simple a more complex agent might be something like a chess AI Where it has a goal of winning the chess game You know if you're playing white then your goal is the black king to be in checkmate And it has actions in the form of moving the chess pieces and again chooses its actions to achieve its goals You can think of human beings in the same way and this is how it's usually done in economics. You think about a Goal of something like maximizing your income or a company maximizing its profits And then making its decisions to achieve its goals Quite a simple concept, but you can see how it's a useful way of thinking about intelligent systems So what does it mean to be? intelligent within this framework an intelligent agent Pretty much intelligence is the thing that lets you choose effective actions Which means it lets you choose actions which actually achieve your goals so if you have two different agents in an environment that have Conflicting goals like let's say your environment is a chessboard and one of the agents wants white to win and one of the agents wants black to win then The agents will choose actions to try and achieve their goals and generally speaking the more intelligent agent is the one That gets what it wants. It's better at choosing actions and better at directing the world towards the state that it wants the world to be in even in the face of opposition So finally the last term here is general intelligence. What does it mean to be a general intelligence? So you notice we can compare one chess AI to another chess AI But and and you might think that it would be sensible to say that The chess AI is like more intelligent than the thermostat because it's much more complicated. It's much more like sophisticated But ultimately a chess AI can't do a thermostat's job Like it's too narrow. It doesn't there's no like position on a chessboard that represents the room being a good temperature There's no move on a chessboard that represents turning on air conditioning It can't think about that problem and that's because it's a narrow Intelligence it has intelligence but only within one domain and pretty much everything that we build So far all the AI we have so far is narrow AI it does one task and Potentially does it very well, but it's not able to generalize So it's a continuous spectrum of course like if you write an AI system that can play an Atari game Then that's pretty narrow if you write a machine learning system, which like DeepMinded in like 2013 That can play a wide range of different Atari games. That's more general. It's not a general intelligence, but it's more general it's a continuous spectrum and the most general intelligence that we're aware of right now is human beings obviously human beings are very general we're able to Operate behave intelligently in a very wide range of domains Including domains that we've never seen before Domains that we didn't evolve in domains that we aren't prepared for like human beings can play chess and we invented chess Right. It's it's a brand new thing human beings can you know drive a car and we invented cars Human beings can operate in alien environments human beings Can can drive a car on the moon right you can build a car take it to the moon and drive it on the moon general intelligence lets you do this kind of crazy Unexpected thing. We're the only thing that can do that so far But you know sooner or later right So oh yeah, right so so I said I think this is the most like the most important problem But on the surface of it kind of looks like a solution right you build this agent with general intelligence You give it your goal like cure cancer or like maximize human happiness maximize the profits of my company something like that And it can choose the actions it needs to take in the real world to achieve that goal but I think it's a problem because Choosing good goals is really hard So this is this is an AI. This is so annoying that mouse cursor isn't mine. That's like that's in the video anyway This is this is an AI system built by open AI it's playing a game called coastrunners Which is a racing game? You may notice it's not really racing they trained it with the score you can see the score there in the bottom left and It's discovered that every time it picks up one of these turbos it gets some points and If it just like drives around in a circle and smashes into everything and catches fire The the turbos respawn at a rate that it can continually pick them up And this turns out to be a much more effective way of getting points than actually racing So that's what it's doing it is winning By the metric it was given and my point here is that This is not something unusually stupid by open AI like open AI is not making it Necessarily an obvious mistake here. This is just like the standard way that these systems behave and I'm gonna pause it because it's distracting This is the standard way that these systems behave and you can find Dozens and dozens of example pretty much any time somebody tries to build an AI system the first few times Their reward function or the utility function or their metric or whatever they're using their objective Doesn't do what they think it did like There was one thing they were trying to create Something that would design creatures that would run quickly So the first thing you do is you say you have like a center point an anchor point on the creature And you say I want to measure how far that's moved Right and that encourages you to like detach the smallest possible thing. That's your anchor point and fling it or Sort of unfurl like a spring and not go anywhere. So yeah, okay right now You do have to actually like move yourself. You can't just be a catapult Okay, we'll measure the center of mass and what they found when they did that was you end up with these extremely tall and thin Creatures with a big giant mass on the top that then when they're started to be simulated fall over And that moves their center of mass a long distance Like this is this is this happens all the time or you may have heard about that tetra say I that Would like play reasonably well And then just when it was about to lose would pause the game because it lost points for losing but not for staying on the Poor screen indefinitely This is the default right? This is the normal thing even in very simple situations simple games simple evolutionary algorithms What happens when we start talking about general intelligence? We start talking about the real world This is a quote from professor Stuart Russell. He sort of wrote the book on AI if anybody's studying AI and hasn't read his textbook you should So I'm gonna read this out for the purposes that recording a system that is optimizing a function of n variables where the objective depends on a Subset of size K, which is less than n will often set the remaining unconstrained variables to extreme values If one of those unconstrained variables is something we care about the solution may be highly undesirable And so when we're talking about the real world, we're talking about tremendously large values of n right the chance of getting K anywhere near to n is Zero so this is something that becomes more and more of a problem the more complex the system Your AI is operating in and if you're operating in the real world. That's the most complex system. We've got So I Forgot how I was gonna move on from here hang on a second Oh, yeah, right. So what do I mean? What is what is meant here by like setting unconstrained variables to extreme values? Like let's say you You create your AI system, it's in a robot you want it to get you a cup of tea, right? You're giving it a simple goal. You manage to specify. This is what a cup of tea is I want there to be one, you know on the table in front of me so far so good But oh no, there's like there's a priceless ming vase on a narrow Stand right next to the door to the kitchen and of course the robot immediately just smashes the vase and goes in and makes you tea You didn't tell it that you care about the vase it doesn't know it doesn't value the vase at all So obviously it's gonna smash it right.  And if you do that, what does it do? I? Don't know But there will be another thing right there will always be another thing That you forgot to include something you care about so I mean like here's something completely stupid It might reason something along the lines of okay now I care about the tea and I care about the vase The vase has to remain unbroken. There's a human being they're kind of hard to predict They move around they might knock over the vase so I have to like definitely make sure the human being doesn't move at all ever And you know kill you or something like this is ridiculous obviously this is absurd But at the same time if the objective function doesn't contain the fact that you should be alive at the end of all of this The system doesn't care about it right. It's not in your set K So it's not it's not interested and Even if you have a really fantastic Utility function you manage to program in the 20 most important things the hundred most important things There's always going to be a hundred and first Right because human values are complicated and not very well understood and the problem is slightly worse than that because There's this thing about setting things to extreme values. So when you're being when you're when you're making decisions What you're always doing is making trade-offs You have multiple things that you value and you're deciding how much of one thing you're prepared to trade off for how much of another thing and You know like oh, I could do this I could do this faster But it increases the probability that I would make a mistake or I could do this better But it would be more expensive for you know these kind of trade-offs the more intelligent you are of course the more creatively you Can find ways to make these trade-offs well and The problem is that if you only care about a subset of the variables in the environment You will be willing to trade arbitrarily huge amounts of any of the things you don't care about For arbitrarily tiny amounts of any of the things you do care about right the this system would be perfectly willing to Destroy an entire city or something for a point zero zero zero zero zero one percent increase in its ability to get you tea And that the more intelligent the system is the more able it will be to find new and creative ways To destroy things that aren't in its objective function or you know sacrifice them to get tiny increases in its objective function um So yeah, this is a problem Now you may have noticed that there are a few ways in which the scenario that I am talking about here is unrealistic This kind of audience. I feel like you might one of the important things in One of the important ways in which it's unrealistic though is that when the system went wrong You just like turn it off and try again and reprogram. That's how we do programming, right?  It will behave as though you've programmed it correctly so that you don't mess with it until it's in a position Where it's confident that you can't turn it off and then it will go after its real goal And you'll notice like this is a convergent instrumental goal, which I'm not going to have time to go into But what it means is that this pops up for pretty much any goal, right? I've chosen making a cup of tea. It's pretty arbitrary. The point is it doesn't really matter what your goal is You probably can't achieve it if you're dead so Pretty much whatever goal you give a system. It's going to display this behavior By default of trying to avoid being turned off There are a few more convergent instrumental goals These are things that you just would expect to show up Thanks to show up as instrumental goals across a wide range of terminal goals So self preservation is an obvious one. You can't achieve what you want if you destroyed or turned off Goal preservation is another one like whatever your current goal is Allowing somebody to change that goal to something else is Probably a really bad way of achieving that first goal so Systems will be incentivized to preserve their own goals and try and avoid being modified specifically in their utility function or their goal function And this is true for almost all goals Resource acquisition again pretty much doesn't matter what you're trying to do You probably can do it better if you have more resources in the form of money or energy or matter or Computational resources or whatever so we would expect these systems if they have some goal that we gave them by accident Which isn't what we wanted The vast majority of those accidents will involve Will be will be helped by acquiring resources so we expect that to happen self-improvement again. This is Like sci-fi stuff, but it seems perfectly feasible that AI systems could improve themselves perhaps by just acquiring more hardware to run on so that they you know Parallelizable algorithm if you can acquire more computing power you can become smarter. Maybe they could rewrite themselves to become better as well Very difficult to predict this kind of thing But if it's possible you would expect these systems to go for it because whatever you're trying to achieve You can probably achieve it better if you're smarter and Yeah, this is the explanation of what that we don't have time for that Ask I want to leave time for questions if I can so So yes, this is my point right Artificial general intelligence is dangerous by default. There are all of these behaviors Which we would expect to arise by default unless we specifically design ways around them Unless we can explicitly design systems that do not have these shortcomings if we just extend what we're doing now and Manage to come up with something general That could be a tremendous problem And it's kind of it's possible that we only get one shot It's possible that the first general intelligence we build will manage to succeed at whatever its stupid goal is that we gave it by accident And that really makes the problem much more difficult because it's so much easier to build this kind of Effectively malicious agent which will deceive you which will fight to prevent itself being turned off and go on to do some crazy Thing that you don't want it to So much easier to do something like that than to build something which is safe So we kind of have to beat this challenge on hard mode before anyone beats it on easy mode Yeah, so so so this is why I think this is an important problem You can see how there may well be more than 45 years of work to do here There may be more than 120 years work to do. It's actually a hard problem I've just picked out a couple of so the point is there isn't there is any field of scientific research right called AI safety, which is People trying to solve this problem. It's very difficult work because we're trying to Design safety into systems which do not yet exist, but it's not impossible work One avenue of work I'm not gonna have time to do these properly one avenue of work is Courageability trying to make systems that don't have this convergent instrumental goal of preventing being shut down and modified There are some interesting designs there, but nothing which we're confident would definitely provide Courageability it's like a property that we're aiming for in AI systems Value learning is a way of getting around this idea of having to program in the values of the system like where you have a hundred things And you forget the hundred and first So you have the system try to learn the values of humans by observing them This sounds simple, but actually there's no proposal right now that would work. They'll have weird Edge cases where if the system is sufficiently powerful it ends up doing things like You know taking the nearest human apart to find out Rather than just observing them like what counts as an observation and what counts as interference is a fuzzy line You know with all these weird philosophical problems Side effect reduction is a way of getting around this like level the whole city to make a cup of tea thing You try and build into the utility function a general preference for having a small impact on the world Supartial you know it would help, but it's not a solution to the problem An interpretability is something that people are working on a lot right now because the current AI systems we have are really Black boxes which makes this whole thing so much more difficult So if we can find systems that we can actually understand their thought processes and their decision processes We're in with a better chance of being able to make those systems safe This research is happening currently a few companies and a few academic institutes. It's a very small field. It's extremely exciting such interesting problems and Like currently there are about 50 people working on this and that and everyone is hiring very aggressively right now because People high up at starting to realize that we do actually have a problem here So if you are interested in AI if you're in a place where you're thinking about what you want to do I would urge you to consider this as a possibility I think it's by far the most in problem most important problem facing humanity right now. Thank you What questions do you think that The solution to the AI problem will be solved with better technology Which will circumvent the issues that we currently have or more by developing thought to the extent that we solve the AI problem and But with the constraints of the technology we have at the moment I think that the The problem So there are some there are some solutions or some approaches which Probably perform better the better the technology is like if your system involves Modeling human beings to try and understand what the human beings would want then the better your technology is the better You can model the human beings and so the better that system performs on the other hand The more powerful the system is the more like anything that involves like containment or restriction or Safeguards stuff like that that stuff becomes much harder the more powerful the system is and keeping a keeping a weak system Safe, it's not actually safe obviously, but it's like they're dangerous, but contained is easier if the technology is weaker so I think most of the work right now is Not there's some philosophical work and there's a lot of mathematics and there's a lot of just pure computer science so I Don't think the technology will solve the problem. I think it's got to be for work Thank you for the top I had a question that maybe Do you think that the solution that you proposed or evoked at the end of the talk might in the end hinder the goal of making actual artificial general intelligence as in like make the development slower or even prevent the like prevent the AI to find solution that couldn't be thought by a human by by having like mimicking the human Pafior or Like valuing the side effects too much. Yeah, this is definitely a trade-off where Often you have to trade off like effectiveness or capability for safety Like obviously there's a sense in which a less capable AI system is automatically a bit safer just because it's less able to completely out with you and So, yeah, I think I think being careful about safety is definitely going to Reduce the overall like speed and effectiveness with which we can develop this stuff And that's just necessary ultimately. It's like it's not optional. I feel as though you could probably The example I use of this well Related example is the the work that they did at Los Alamos during the Manhattan project to sort of double-check that The first the Trinity test would not ignite the atmosphere. This was a serious I mean not not a very serious concern but it was something they thought about and they put some very good scientists on the task of really nailing down that This could be done safely before they did it and that slowed them down at a time when there was a war and there was There was like an argument to be made for speed In the in the order of however many thousands of lives every day And yeah, it's a trade-off But ultimately I'm glad that they checked that they weren't going to literally destroy the whole planet before they did it And I think it's the same here Thanks That's good. Oh, that's it. Yeah. Okay. Thank you very much. That's all the time we have We are a bit behind today everybody. Thank you very much for your understanding. Thank you very much Rob