Okay, we're back. We're live. We're here on a Saturday. Huh, Saturday. No, Wednesday. Well, every day you want. This is Sharon Moriwaki. She's the co-chair of the Hawaii Energy Policy Forum. And guess what? She's the co-host of Hawaii, the State of Clean Energy, which does broadcast at 4 p.m. on Wednesdays. We're making a special show today because we have a special guest. A special guest is Gavin Bade, and he is the editor. That means the senior editor, the most important editor, the guy at Utility Dive, which is an app and a newsletter about utilities and energy and environmental mixed up together out of what? Washington, D.C. Washington, D.C. Wow. Now I've told the essence about Gavin, but Sharon is going to actually introduce him. Well, I'm so pleased to have Gavin here. I'm pleased to be. We, you know, at Hawaii Energy Policy Forum are just coming out of our annual legislative briefing, and we needed to know for Hawaii what is happening in Washington. It's so unpredictable, and we're so pleased that we have Gavin. Gavin is, as Jay said, the editor of Utility Dive. It's now an app that keeps the thumb of the nation, all the states, and we thought that it was really important for us to hear that as we go forward with this legislative session. And that's why the name of this episode is the same as the name of Gavin's speech, his presentation as keynote on Thursday, and it is what? Climate action in the age of Trump. Nothing political about that, okay? And Hawaii's role in the clean energy revolution. There's a lot of assumptions and publications in there. How is that? Yeah, well, what does it do for our 100 percent renewables goal in this age of Trump? We need to stay in touch, especially now. Yeah, yeah. Absolutely.  I don't know about that. Can you give us a praise, see what you said on Thursday we need to know about? And yes, you can refer to your PowerPoint. Oh, yes, fantastic. Well, thank you. You know how I love the slides. Well, I just want to thank you, Jay, and you, Sharon, for having me out here on Think Tech. You know, I do watch it from D.C. from time to time, and really love the production coming out of here. You guys do great work. I'm so happy to be here on the show. Basically, I, you know, kind of thinking about how to frame this question of climate and clean energy in the Trump era. I've done a very journalistic thing and resorted to a shorthand trope. And that is that it's kind of for climate and clean energy, it's the best of times, worst of times, right? Yes. When it comes to, you know, the beginning of 2017, we have this big energy paradox, right? It's, we have a climate denier going in the White House who said he's going to cancel the clean power plant, potentially pull us out of Paris, although Rex Tillerson is saying some other things about that now. But for a planet that's already kind of behind the pace in limiting carbon emissions to kind of limit our temperature increase to two degrees this century, the loss of that diplomatic will from the federal government would seem to be catastrophic for the planet, right? For the planet, yes. Well, I think that's the way a lot of people looked at it the day after the election, and certainly from an environmental standpoint, that was my initial reaction, right? You've changed your mind? Well, not exactly. But I think that there's, it's not all gloom and doom. There is reason for hope. And I think that's because there are states like Hawaii that are doing really ambitious things, setting ambitious targets and figuring out how to make a clean energy economy and a clean energy power generation system that can power the rest of the clean energy economy. So basically my thesis for everyone here in Hawaii is that you're going to show us how to do it. You know, I don't think climate denial will be the policy of the U.S. federal government forever. So if and when the pendulum does swing back the other way... I sure hope it's soon. Sorry. Well, I will, you know, leave those value judgments up for anyone, but for, I think for, from the standpoint of climate action, if and when it does swing back the other way, we do want to do something federally on climate, states like Hawaii are going to show us how to do it, and they're going to dictate whether we can do it cost-effectively for consumers. So I know it's a slog here going through and trying to get through, get to the 100 percent renewable energy mandate, but it's critically important that, you know, you guys do well at it, succeed and teach us all else, all throughout the nation how to do it. That's a great message for Hawaii. And that's the Port Parole message of our whole program here, and it's Gavin's message too. I love it. Yeah, I guess so. So I've got a couple slides to just kind of illustrate just kind of the market, the market situation from clean energy and decarbonization in, at the beginning of 2017. Go for it. So the first slide here, if we can bring that up. It's coming soon. There it is. Okay. So going back to the Dickens things, Pastor Times, Worst of Times, do we have a tale of two nations when it comes to energy and climate policy? And at first blush, you would think, yes, right? So this is the percent change in CO2 emissions from 2014. There is Hawaii over on the right side of the screen. Yeah, I had to split it up because it was a long interest for me. This is from a Brookings Institution study that came out last month. And basically what I wanted to do with this is just show that, you know, there are a lot of states, especially in the Northeast, the West Coast, and down south, that have had some good success in decoupling their economic growth from their carbon emissions. You know, states like Georgia and Tennessee, which you wouldn't think are big clean energy states, because they've switched from cold to natural gas, and because they've sourced a lot of power from nuclear, they've been able to support, you know, strong economic growth period without raising emissions. A similar story for the states in the Northeast and out West. So, but when I looked at this at first, I thought, oh, well, now that we're entering the kind of the Trump era, we're going to maybe lose some of that impetus, that momentum, maybe we will see this nation's keep pulling apart a little bit more. But the Brookings people that I talked to and other analysts, you know, they said, well, maybe yes, but maybe not, because renewable energy has been making inroads in many of these, you know, orange, some of the more conservative states in the land that we see. And everyone expects that to continue. So, if we can go to the next side, I think we can demonstrate why. So, this is the, this is a slide from the investment firm, Lazard, and it's just comparing the market, the market price for renewable energy and all of the conventional generation. So, if you see at the bottom there, I have highlighted, that says gas combined cycle. And that's your kind of run-of-the-mill modern gas generator on the electric system ranges from $48 to $78 a megawatt hour in the average levelized cost in the survey. And the interesting thing about this was that it showed for the first time, if you look at the wind energy, which is at the bottom of the alternative energy block up there, the range is actually quite competitive with natural gas and the range is actually lower. It's from $32 to $62 a megawatt hour. This is on an unsubsidized basis. It's without federal tax incentives for wind and solar. And if you look at the solar PV, it's on a utility scale, a thin-film utility scale. Very, very competitive with natural gas in many parts of the nation and cheaper than coal. So this is the market situation without subsidies today. What is this telling us this chart? Well, it's telling us that renewables today are at grid parity with fossil fuels. Now, they have problems. They're not dispatchable, right? You can't have wind and solar all around the clock without energy storage. And we can talk about that a little later. So this is without the problem of storage. But if you add the cost of storage and these numbers will change. These numbers will surely change. But we see really a lot of really encouraging developments coming out of Hawaii, actually, with bringing the cost of storage and solar plus storage facilities down. We're determined to do that. You know that. Absolutely, absolutely. And it's going to be critically important for a state like Hawaii that wants to take fossil fuels off the system. I think if we can go to the next slide, we can get a little... Is storage, going back to that storage question, is storage commercialized enough that you would get some kind of rate like that to say how much more you would add on the storage to the utility... That's a tough question. I think it's... Utilities and developers are still really figuring out... Like some of these... A lot of the projects are one-offs right now. Storage is just maturing into a grid-scale resource. And I think that that's one of the big stories of 2016 was that energy storage did kind of prove that it could be a viable replacement for natural gas or other things on the system. But we're on Sharon's point. That chart, you just showed us. Yes. Right now it doesn't include storage, but there will come a time, maybe the next year or two, when you could rebuild that chart and include the cost of storage and see if it still works as against fossil fuels. Sure. And I think it's difficult. There just simply aren't enough solar plus storage facilities on the system today to really model that out, right? You can't do an average thing. But the ones that we do see, the contracts that are being signed are really quite encouraging. You know, KIUC, the Kawhi utility cooperative over on the island of Kawhi, in 2015 signed a power purchase agreement with Solar City and Tesla. 100% dispatchable solar. It's a big 52-megawatt battery. Solar and storage together. Solar and storage. Same deal. So they charge these big batteries during the day. And then when the sun goes down and everyone goes home starts to use electricity. This is what's going to happen. This is what's going to happen. That's the idea, right? It's that combination package thing that KIUC did. We're going to see that on the chart that you just described. Well, that's the hope at least. And we're going to see that hopefully continue to come down. The KIUC actually just signed another PPA last week with an energy storage company, AES. And it's actually 30% cheaper than the storage contract that they just signed not even two years ago. So that's, you know, obviously Hawaii is a special test case. I haven't looked at the contract itself. A lot of these things are proprietary. No, but you are very familiar with Hawaii things. Yeah. It's not fooling around. That is not fooling around. Well, you know, if we try to be as, you know... And we should see it, don't we? Yeah. But it's, you know, that was really encouraging and exciting for a lot of people in the system today. That storage was, that solar plus storage was signed for 11 cents a kilowatt hour. And the exciting thing about that is the prevailing electricity rate on the mainland. If you take all the residential rates and average them, it's 12 and a half cents a kilowatt hour. So if I don't know if AES can make that big, that big facility into something that goes, you know, into your garage and on your roof yet, probably not because the economies of scale probably help them keep the cost down. But... But you know, you can do it again. Just 28 megabytes here and 28 megabytes there. Well, we will see. That's the hope. Look at how much Hawaii has accomplished because their rates are, like, 40... Yeah, this is remarkable. Exactly. And they will save a lot of money doing that. Yeah, they're showing us all of you things or two. Yeah, yeah. Let's go to the next slide. Let's go to the next slide because it's a little bit more detailed look at... Oh, this is... Is there the one with the map? Well, this is... Well, we've established, you know, renewables have their grid parity throughout the nation. And, you know, there's especially very competitive for wind in the heartland, you know, places like Nebraska, Kansas, you know, where there are windy, windy plane states, and then solar you can actually see in many parts of the Southwest is actually the cheapest resource on the system. So that would make you feel very good about the U.S. climate situation, right? You would think, well, if the... if we've been switching from natural gas to renewables that's driven decarbonization for the past 10 years, then we continue doing that. Everyone expects that to continue despite Trump. Well, then what's the problem, right? And this chart kind of shows us what the problem is. It's that those trends alone aren't enough. We're simply not moving fast enough on decarbonization to hit the federal targets. So this is a... So all of these bars show what's called the carbon... the annual decrease in carbon intensity. So it's basically your economic growth, the difference between your economic growth and your carbon emissions growth. So as we can see, the highlighted bar in the middle there is the United States as an average. And annually we've been, you know, taking down the carbon intensity economy by about 2.1 percent a year from 2014. The problem is we need to be all the way up where that orange line is up there if we're going to meet the goals for the United States under the Paris Protocol. That's the progress line. That's the target line, right? And you can see places like North Dakota or D.C. that have had really explosive economic growth. It's not that North Dakota has actually cut their emissions. They've just... So what is that line at? What is the metric of that line? If I could read the thing, but I think it's... Can you move that graph a little bit? I think we need to be a little bit... I think it's the way I cropped it, but I think we need to be up a little... We need to be in abstraction, but we need to be doing two to three times better, faster than we are now, than we are today, on a nationwide basis, right? Okay, well, why don't we go... You want to see a next slide then? Yes, please. Okay, next slide. So here's why I'm saying...  We're not going to see much more action on the federal level, but there's reasons to be optimistic about the energy why. This slide is from the most recent Department of Energy Quadrennial Energy Review. What it shows us is the expected life for all of the generation capacity on the system today. So the different bars you can see on the bottom relate to different resources. Coal is the gray on the bottom there. Natural gas is the green. And the point of this chart is showing, you know, taken today, and this report came out last month, you know, how long can we expect every plant to stay on the system? The point of this is that you can see the gigantic glut of natural gas capacity that was put on the system about 15 years ago, all those tall green bars there. A lot of analysts I talked to don't expect them to get to that expected life of their natural gas. Natural gas plants are cheaper than coal plants, the nuclear plants, so they depreciate faster. Utilities basically get their bang for their buck much faster out of these plants. Cobals and other, you know, dispatchable, clean generation or even, you know, more efficient natural gas plants come out of the system. A lot of people are telling me they expect those big, you know, big green bars of natural gas that 150 gigawatts of combined cycle, 70 gigawatts or more of combustion turbines that they might go offline and be replaced by new capacity in the mid to late 2020s or early 2030s. That is, it opens up a gigantic opportunity for dispatchable renewable resources, not just to tinker around the edge of the system, but to really make inroads and replace carbon emitting resources with clean technology. So what does this tell us about what we should be focusing on? Yeah, what it tells us is that we have a big opportunity around the corner if we can get the price of dispatchable renewables and other clean energy down to where when all of those natural gas plants come off the system, the energy in instead of putting other natural gas in. The whole natural gas has been a positive for the climate over the past decade and a half. A lot of utilities have switched from burning coal in their plants to burning their natural gas plants more just because the gas price has been so cheap. And fully that kind of accounts for about two-thirds of the decarbonization we've seen, but going forward it presents a problem. If a utility makes an investment in a natural gas plant or a pipeline today it's a system for decades in the future. 20 years or more in the case of a natural gas plant typically speaking. So we need to be, you know, when that natural gas that glut of natural gas gets ready to retire, if we replace it with clean energy we'll have a much better chance of hitting our targets under the Paris Accord. If we don't replace it with clean energy well we're just going to put more carbon we're going to just lock ourselves in to more carbon emitting resources on the system for decades to come. Well that teaches me two things. Transformation is change. Change is a matter of looking forward having new ideas and it's also retiring old ones. We have to move from old to new. We always have to move from old to new and that's the only way you can handle a transformation in energy or anything else in the mainland or Hawaii. That's one thing it teaches me. The other thing it teaches me is that Gavin has an incredible ability to give us an enormous bit of information without taking a breath. Yeah, I know that was a... I'm only going to let him take a breath now. We're going to take a break so Gavin can breathe. Ready? Go! Hi, I'm Donna Blanchard. I'm the host of Center Stage which is on Wednesdays at 2 o'clock here on ThinkTech. On Center Stage I talk with artists about not only what they do and how they do it but the meat of the conversation for me is why they do it, why we go through this. A lot of us are not making our livings doing this and a lot of us would do this if we didn't take a breath if we had that choice. And that's what I love to talk to people about. I hope you enjoy watching it and I hope you get inspired because there's an artist inside G2. Join us on Center Stage at 2 o'clock on Wednesdays. Bye. Aloha, my name is Josh Green. I serve as Senator from the Big Island on the Kona side and I'm also an emergency room physician. My program here on ThinkTech is called Health Care in Hawaii. I'll have guests that should be interesting to you twice a month. We'll talk about issues that range from health care to drug addiction to our health care system and any challenges that we face here in Hawaii. We hope you'll join us. Again, thanks for supporting ThinkTech. Okay, we're back. We're live with Kevin Bade. He's the editor, the editor of Utility Dive. Wonderful. Sorry, one of the editors. We'll talk more about that. We'll talk about climate action in the age of Trump and Hawaii's role in that action. This is in the Clean Energy Revolution into exactly the Energy Revolution in Hawaii's role. I want to ask you some more about Utility Dive. What is it? How big is it? Who's involved? Why? So Utility Dive is an electric utility trade publication. We're based in Washington, D.C. and we focus on the entire U.S. nation in the power sector here. Some other international coverage but mostly the U.S. power sector. And I guess our mission is just to provide policymakers and industry officials with actionable information so that they can deliver a cleaner power system at lower cost. Did they listen to you? I hope so. It seems like we've been... Over the past two years since I've been here it seems like we've caught on a little bit. We've been growing. We appeal to people, renewable energy executives, so people who are putting wind and solar on the system. Our target audience includes them, utility executives, so people who work at HECO or KIUC on the mainland. Absolutely. People who work at the PUC, in the governor's energy office. Any policymaker or industry official, anyone who basically has a vested interest and a say in the energy transformation, we want them to read Utility Dive. Where do you get your data from? Who makes the analysis at Utility Dive? The five of us who work there, it's me and my co-editor, Christie, are the two editors in the office overseen by senior editor, Davide, and then we have two full-time writers who are fantastic, Robert Walton and Herman Trebish and our time writer who's equally as fantastic, Peter Maloney. Shout out to all of them. Basically, it all comes from us. We do some semi-aggregation stuff, so we'll take headlines from throughout the nation, kind of distill them and then add our insight. But just kind of trying to talk to all of the industry experts out in the field. I try to talk to consultants and lobbyists and analysts and people who work in policy offices. You're on this phone a lot. I'm trying to kind of piece through the energy transformation. How can I get the benefit of Utility Dive? Well, we have a free daily newsletter that goes out to more than 30,000 utility executives and policy makers throughout the nation. You can sign up at UtilityDive.com. All of our content is 100% free. You won't pay for anything on Utility Dive. Our reports, our newsletters or anything. So there's a daily newsletter and then there's three weeklies, one on energy storage, one on efficiency and demand-side management and one on solar. So we're always looking to expand, but those are our options today. Can you just say the app? What is the app? Yes, we have the app as well. Get it on your phone. It's free and you can just have basically all of the utility headlines right in your hand every day. We've got the app, we've got the mobile web and we'll shoot the email to your inbox every day. So I think... I want that. If you want to be inundated, we can do that. Happy to be inundated. So one more question before we go back to the substance here and that is, Dive? What is Dive? Diving in Utility Dive. I didn't mean to know that. So Utility Dive is one of 12 publications that's a part of a fantastic parent company called Industry Dive. And these three guys who founded it just kind of saw that there was a demand for really succinct to the point analysis across a bunch of different industries. So they kind of started off a few years ago. It's actually we're just moving out of the startup phase. Started working in a Korean grocery store and they put these newsletters together and we've since... Well, it all makes total sense. So it's Utility... so it's Industry Dive, so we are Utility Dive. But then there's... Diving in. But then we have Retail Dive, Healthcare Dive, Biopharm, Food, everything you can imagine. What a great brand. I think so, yeah. The name, I mean, Ival was a little... It threw me for a loop when I first... But it's memorable. I'm saying, it's a drive. It's a drive. After you get past that... So let's dive into what's going on in Hawaii. Connect it up for us, Kevin. Yes, so I guess before the break we just got done saying that just around the corner there's going to be... depending on what happens in generation markets there are all these caveats, but a lot of analysts think that there's going to be a big opportunity to replace a big amount of coal and natural gas on the system in the near future.  Well, in part, that depends on what you say about us, doesn't it? I don't know about that. We're going to get to publish what we're doing everywhere, yeah? I would say that you guys are pushing the narrative and that we're just reporting on it. Okay, okay, okay. True journalism, yeah. In any case, I think that there are a lot of really encouraging things happening in Hawaii. We touched on the 2KIUC projects that, you know, are going slated to be dispatchable solar plus storage, so, you know, storing up all of the solar energy that you generate during the day in big batteries. So you're looking for leadership points. You're looking for things at the frontier that we may have. Absolutely. And then you're going to sort of publish that. You are publishing that out so everybody can see. And that creates the leadership process you're talking about. Absolutely, and that's a big, I think that's a big, you know, motivation behind what we do every day is taking, you know, local or state level stories of clean energy success, you know, or difficulties to be perfectly trained.  So, you know, people who work in Ohio at AEP or, you know, up in the Northeast at National Grid, they can learn from what you guys are doing here in Hawaii. So make a prediction for me, Gavin. How is Hawaii going to do? How is Hawaii going to do? From all that you know. And in meeting the 100% renewable energy mandate? Whatever. Whatever we do, what do you think we're going to do? Okay, I'm waving into, you know, quite hard. This is deep water. Deep water. I think out of turn here, but I think, you know, based on what we've seen is that, you know, being the leader in the clean energy, being one of the leaders in the clean energy revolution is never easy. And it's always going to be fits and starts. So, you know, going through a couple iterations of the Hawaii Electric Power Supply Plan is probably how, you know, things have to go at first so that regulators and the utility can come up with a good plan to get going. It's going to be fits and starts, but I think you're going to do it because you're going to do it hook or by crook. There's political will behind it. You know, I don't think I, you know, unless things go terribly, terribly wrong, I don't see the Hawaii State Legislature bringing back the 100% renewable energy mandate. At least not any time soon.  PSIP talking about, you know, 2040, not 2045. So there's a certain amount of interest in moving that deadline closer. Yeah, indeed. I think the point is there's political will behind it and people are really pushing the utilities and I think that we will. The question is just kind of, you know, I like that Hawaii, you know, California, other states have kind of defined the what, but now we have to work on the how, right? So I think the first thing that it seems that we need to do is, you know, we're talking to utility people is that the modernized grid is a necessity, right? Getting the grid ready for two-way power flows so more people can hook up rooftop solar, can hook up batteries, and so that the utility can solve problems in the system with distributed resources instead of building a new power line or, you know, upgrading a power plant or something like that. So upgrading the grid is going to be really critical.  After you do that, then it's just an issue of, okay, well, which technologies do we choose?  You know, do you focus on biomass? Do you focus on institutional issues there? Maybe not going to happen. There are political considerations on that that will pose impossible obstacles. Yeah. In any case, I think it's, you know, it's going to be really tough for, you know, there's going to be a lot of work ahead between the utilities and the regulators to kind of plot out what's going to be the most cost-effective transition process. What I get from you is that things are basically in place. It's a maturation process. We've been doing this actively, politically, since 2008, right? If not before the Clean Energy Initiative here. And, you know, we, you know, we have had our fits and starts and we may have fits and starts going forward. So the question is, are we past fits and starts? Probably not. But we are at a point where we can see the outlines of the available options right now. We can build a pathway, recognizing there will be changes in technology and all that, and to be political issues they always are. But at least we can see the outlines and we can make a plan. Maybe, my question to you, maybe we are at a level of maturity that allows us to go forward with a minimum of disruption, a minimum of back and fill, if you will, so that the world can watch us, really watch us, and really learn. Are we there, do you think? I've certainly been encouraged, you know, to see people from the utility, the solar industry, policymakers, you know, in our discussions this week, the minds of communications seem open. And in the conversations that we have, they seem honest and frank. And that's the first thing you need if you're going to make any progress on these contentious issues of rooftop solar compensation. Which resources do you source? Where do you source them? How do you cite them? And it was a big issue. So kind of having a constructive dialogue between the commission and all of the stakeholders, the first step, and I've been encouraged to see that, I think it's always difficult when you, there's been a lot of stuff going on at the Public Utilities Commission, basically, right? There's the next era merger, there's the PSIP, you know. How about impatience? Is there room for impatience in this? I think everyone's impatient. I think Sharon's impatient, too. Because the planetary situation should make us impatient. We have to go there now. Exactly, but she wants it done right. And there are some real technical and market issues that need to be solved before we actually can get there. Will you come back and talk to us again about this? Absolutely. She wants to ask you some cross-examination now before she summarizes what you've said. Go for it, Sharon. No, I think Gavin has brought a good discussion to us about what's happening in other states, but also that we can be a model. And part of what we're talking about is the trends going forward and the market being such that we can actually keep moving forward. I think we're a little in disarray thinking, we're going to bump us all and we're going to go back to what we were before. And I know long ago, the Bush administration started this. We had this push toward renewables and then the oil prices went down and we left it. And so I think the late Senator Inouye said, look, we can't afford to blink. And that's what this is all about, is how do we keep going forward. And I think Gavin, looking at what's going forward and moving around us, kind of keeping our eye on the prize and then really moving in that direction. So my question to Gavin is, how can we do that? I mean, you mentioned some of the areas where we should really be focusing on, but what can states do? And we talked about this a little bit in terms of the public and how we can get the public engaged and what we should do. Good news. You got the question? Public engagement, no pressure. Well, I think the public, so the first thing is that public engagement and energy is higher in Hawaii than anywhere else because there's a big packet book issue here. But I think it's, you know, it shows like this. It's the utility and the stakeholders coming together to present a united option and just being kind of frank with the stakeholders and with the press about what's going on. I think open lines of communication and transparency is the biggest thing because people are watching what's happening in Hawaii. I think that elected officials, especially, you know, the federal elected officials who go to Washington but also people on the state level, I think a big thing that's going to be important during state and federal level during the Trump administration is building political power for clean energy, getting the solar people and the wind people together with the nuclear people, with the people who support biomass and even the natural gas industry. And that can and must and will and should happen in the states and in Hawaii. You've got it from Gavin Bade, one of the many editors at Utility Dive in Washington, D.C. talking with us today about climate action in the age of Trump and Hawaii's role in the clean energy revolution. And he will continue to keep taking that. Thank you Gavin. Aloha. See you soon. Thank you Sharon.


We have won a significant majority of the votes of young people and in fact By young you know when you get older being young gets older, too. You know vigorous young man of 65 Wow And We have won in most of the Elections of people 50 years of age or younger and we have those just the study that came out I think yesterday that said we have received more votes from people under 30 That Hilary Clinton and Donald Trump combined When I was young My 20s we used to say don't trust anybody over 30. I guess that remains true today. I don't know But I think the reason that we are doing so well with younger people and by the way younger people are the future of this country and What the support that we are receiving tells me that the ideas that we are bringing forth The notion that it is too late for establishment politics and establishment Economics that we need some bold ideas. This is what the future of America is about


Combining similar patterns is easy. The only thing you need to do, make sure they're different sizes or scales. So let's say I want to pair a stripe tie with this bangle stripe shirt. What I don't want to have is a tie like this. The scale of the stripes of the tie are very similar to the stripes of the shirt. So what we end up seeing is there's not much definition between the two. But if we have a tie where the stripes are larger, a bigger scale, now we've got a lot more contrast. And as a result, it's much more pleasing to the eye. Now here's an example where we've got pretty much the same scale, but the stripes in the tie are much thinner than the stripes of the shirt. So there still is quite a bit of contrast there, which is why this also works. It's all about creating some distinction between the two items. So if you follow the rule of making sure your similar patterns are different sizes, it's always going to look good.


I come from Rwanda and the reason why I chose TSE is because I wanted to go further in my learning of the classes that I'd appreciated during my first two years at the University of Toulouse Capitone. Given that numerous professors taught me in my second year of bachelor's degree, I knew that I would be able to receive quality teachings, to learn new theoretical and applied economic concepts, and also to improve daily before entering the job market. With its excellent national and international reputation, TSE therefore filled in all the boxes of the best places where to complete my last cycle of studies. I feel extremely flourished at TSE. It's been three years now that I attend classes in this school, and I've always been impressed by how students were kind and supportive between them. To be honest, I take advantage of a tremendous environment where I know that if I have any questions concerning my studies, I will find my answers quickly thanks to great proximity between the members of the TSE community. My adaptation went extremely well and was quick, since I had already lived in Toulouse for the first ten years of my life. Concerning the welcome organization, TSE definitely offers a fantastic welcome to students every year by directly interviewing them in their associations and by providing all information from the first year of school to succeed and to feel like home at TSE. In general, Toulouse is one of the best students' cities in France, as shown by numerous rankings. In my opinion, this is completely deserved, given the great sports and entertainments that a city offers to its students to relax and to enjoy, such as the capital or the city de l'espace. There are also many libraries and cafes where students can meet up to study by group if they want. I think that all these elements make Toulouse a fantastic living environment where everyone can find their place and flourish. In my opinion, Command Grid is anything that benefits to all members of a given community. Nowadays, positions such as data scientists or data analysts are extremely essential because of the exponential availability of data and I feel that we need to have more skilled professionals who will be able to analyze, exploit and create value from this data. This is my goal and I think that TSE gave me both the technical and the self-skills needed to reach my objective.


understanding millions of gates, introduction to IC reverse engineering for non-chip reverse engineers. Our speaker Kitty will provide a summary of methods and counter methods for integrated circuit reverse engineering and why you should care and what you can do at home. Our speaker Kitty is a researcher at the university and likes reverse engineering and cats. Please welcome her with your round of applause. So, hi. Thanks for the intro. I hope you like my pink cats. I tried to match them with my hair. It didn't quite work, but I'm getting there. I like to talk about understanding millions of gates, also known as how I learned to love looking at gates all day. Quickly, just for you, we had had a talk just before mine, which has some basics to reverse engineering, so I will try to build on that. I may repeat myself, I may repeat the other talk a little bit, so don't mind that. What I won't talk about, any kind of PCB reverse engineering or any kind of process teardown, I don't do that, unfortunately. I also don't do any kind of probing, so that's out of my expertise. I can duck-duck-go, but I don't do it for any kind of data sheet. I also don't do any kind of process analysis, so I don't actually use any SEMs to figure out how chips are made, and I don't care very much about side channel-based reverse engineering. There is a whole heap of research out there, which is really interesting. I don't do that, unfortunately. What do I do? So what I'm interested in is what the hell is this chip, and some of you may recognize it's an open titan chip, so it's actually even going to be open source. Does that mean we actually know exactly what's on there? Well, we have a bit of a gap right here, so that's a bit of a problem. We do know the RTL description, and apparently, from the packaging, it's all going to be open source as well, but any kind of fabrication data we just don't get told about. So that's where I kind of come in and go, look, I really would like to know everything about this chip. That'd be pretty cool, so let's try to reverse engineer it. So we've had this motivation before, right? So if you start googling software reverse engineering, you get told these are the top 12 tools, these are your best 10 tools. If you do the same thing for hardware reverse engineering, you get the Wikipedia entry for reverse engineering, which is not so helpful, and maybe kind of a couple of maybe talks or some kind of academic research, but you don't really get given any tools, you don't really get given any methods. So that's a pretty big problem for me, and that's what we're trying to change. We've had this introduction of the life of a chip, of how a chip is made. Usually you have some kind of design that's made, you know, you have like design house or IP vendor, you write your RTL description. That gets verified, synthesized, placed in route, you eventually have some kind of layout right here, and it goes off into the foundry, wherever that may be in the whole entire world.  You know, do I recycle it and all that. Why do we care about reverse engineering? Well, what is a company worried about when it comes to chips? They don't want the foundry stealing that design, so the foundry goes cool. Thanks for your design. I'll produce the 2 million you wanted. I'll also produce 20 million that I can sell, so that's pretty cool. They don't want that happening. They also don't want out of spec or badly tested chips to be going anywhere, so basically the foundry says, look, we tested it, we tested it, and 90% are okay, and we'll keep the 10%, and we'll definitely get rid of them. Yeah, yeah, sure. They don't want any kind of out of spec chips to get back in the market, and they don't want any kind of recycle chips to get back in the market because they're losing money. So that's why a company generally cares about reverse engineering, about chip security. Why do you care? It's a little bit different, so you don't want any IP to be bad. What does bad mean? Well, it might have a Trojan in there, so some kind of malicious hardware. What could that be? Well, we have chips pretty much anywhere, so maybe in your car, and all of a sudden your car starts stopping and everyone else is two, or in your nuclear power station, I won't go on. It could also be that maybe the in-house design team made a mistake or decided to hide something in there which is not in the specification. We all know how that goes when we start finding random things that never got documented anywhere, when we bought a chip and are using it. We're also worried about the foundry putting into something in there that not even the company knows about, so we stand up buying chips and the foundry in wherever this chip was produced starts doing bad things. So we very much care about the use, right? We want chips that are working and not evil. That's kind of why we care. Obviously, that also means we don't want failed tested chips in our product. So let's have a look what the life of a test chip is like. We've also seen this, so usually we come out of the foundry, we have our lovely little chip, and we start de-packaging the poor thing. We delay it, we image, we have to stitch all that, so that's very much kind of an image processing kind of topic and before that a physical processing kind of topic, and eventually we have some kind of interconnect identification so that might look something like this, but you start finding pictures and then you have to do image processing to figure out where exactly the connections are, or some kind of standard cell identification, so you go through and you have some poor guy reverse engineer or 300 cells, and then you find them again with pattern matching, the rest 2 million minus 300. So that's pretty much not trivial, it's complex, but we know how to do it. We know how to image processing, that's kind of easy. We kind of have the, what the hell do we do now? Topic, we have this net list, what does it actually mean? And I certainly hope you have these at home to do kind of reverse engineering, so maybe they're standing out in your garage, if they are, call me. And I'm obviously just kidding, you don't actually need any of these as we've heard before, companies do do these kinds of things for you, and for the interesting part, the actual what comes after I have a net list part, you don't need any of this, so that's quite nice, usually this kind of laptop I have here is enough. Let's talk about the problems we're actually looking into. So what is the really interesting part of net list reverse engineering, we also call it abstraction, trying to get these maybe millions of gates into something that you can understand.  And the first thing we do, and we've had this in previous talks, is to figure out what the hell is the hierarchy, right? So what modules do we even have on there? What do they look like? Where are the borders to these modules? And this would be kind of the topic partitioning. So in synthesis, we partition to do good layouting and reverse engineering, we partition to figure out where the modules are. That's kind of our first step. And we actually already failed pretty badly there as a spoiler. The second thing we do is then to actually identify what those modules are, right? So now we know, hey, we have ABC. What is that? So maybe we have something like, that's a crypto core. And this is actually coming back to the point of the previous talk that graphical analysis of net list is not so great. That's 250,000 gates of a risk five. And you can see quite easily if you do present it as a graphical thing, at least we can kind of figure out maybe where the modules are. So I'll do some more of that in a little while. So we have these two main problems. We first want to find the modules, then we want to identify them. How do we do that? I'll give you a quick overview of what we do in academia now. By the way, in big companies, we usually just have like 200 EDA specialists who look at the design and go, yeah, look, I wrote this design in 1980. So maybe it's probably that thing again. So the HR costs are actually quite big for this. And as we've heard in previous talk, there is no automation here as of the time being. Okay. So let's talk about partitioning methods. The first idea that we have is generally in a chip. We do any kind of work usually on a data path. So maybe you have a 32-bit multiplier and what actually happens is that you have two 32-bit inputs. And every input has the same thing happen to them 32 times. So every bit basically has the same thing happen to it. And that's duplicated 32 times. And we classify something like that as a word. So a data word. And we can try to figure out where these words lie and then propagate them through the whole entire design. So we hopefully come from input to output. And then say, look, these are probably at the boundaries of modules. So in particular, when new words are starting to be combined in different ways or maybe words are being split up, that's probably a new module. So if you go through a bus to your multiplier, all of a sudden you're now working on 32-bit architectures, you can say, look, beginning word and word, everything in between is probably my module. The other thing you can again do is graphic analysis. So this is actually a fully paralyzed AES implementation, which you can tell by the fact that it has 200 S boxes. Please don't write your IS like this unless you really have to. Because it's very easy for us to find it looks exactly like this. The 200 spots you can see are exactly these S boxes. And if you do graphical analysis, you will find that they're more densely interconnected and you can get these kind of images, which are actually, yeah, pretty easy for you to see. So if you ever start clustering a design and it looks like this, that's probably an IS. So those are kind of two different ways of trying to petition things. It tends to be quite difficult. We would like to be able to petition each cell specifically into one design. That's not usually possible. So we fail at that quite valid time being. We can usually get like 80, 90%, which would be pretty cool and a normal sort of thing. But here it's not really enough, unfortunately. The second part, so we have now got our module. We would like to know what is it and what we usually do is comparison. So hopefully we have somewhere a database with all known designs. So this would be our database. And we compare our unknown design to a known design in this database. We've had the topic of trying to figure out or trying to get lots of samples. That might be, for example, a design library. You could also use something like OpenCourse or LibreCourse. And maybe as a company, you also probably have access to some kind of IP that you can use here. And the idea is, look, I've got an unknown net list. It's the same as a known net list. Let's get some inputs. Let's get all the inputs and let's connect them together and let's get our outputs and see are they the same. So if for every single input that we can put into both designs, the output is always exactly the same, it's probably functionally exactly the same thing. Which is quite nice. We have a couple of problems here. So we actually need a perfect functional match which requires a perfect net list extraction. So while we're de-packaging, while we're de-layering and imaging, you better not have any dust on your pictures because all of a sudden you might be missing a connection or a gate and this doesn't work anymore. When you have two million gates and that's a pretty high number of polygons in your image, the chances of there being no problems with your image are pretty minuscule. So this is going to fail. Let's say you do have your perfect net list extraction, maybe you're working on FPGA, so that's easier to get there, we don't get errors. You also need a perfect partition. So this step I just described, you also need to perfectly in order for this to work. And you also need a known net list. Obviously, if you don't have that, that's going to be a problem. You can tell why this stayed in academia and never went out into the real world kind of thing. The second thing we can do is kind of more graph-based. So we can use some kind of fuzzy methods. I have nine designs here. You might be able to tell that three are kind of similar and two others are kind of similar. So we can do some kind of graph analysis and have a look at this kind of fingerprint of what it looks like. And spoiler, it's these three. These are all IS rounds or IS implementations. And these two, these are both catch-back implementations. So SHA3. And you can try to do some kind of fuzzy matching based on this. So if you're interested in graph theory, this is now your part right here. And figure out what could be a structural similarity. And the first question I always get asked here, well, you don't know what kind of optimization tool or synthesis tool the other people used. You don't know what kind of cell library they use. You know, how are you going to be able to compare that kind of stuff? So what we actually did is we had a look at three different cell libraries and two different synthesis tools. And we had a look at the same design. So this is actually an IS round. And how to look how they actually turn out at the end. And I would say that's kind of pretty similar. At least with your eyes, you should be able to match it. Teaching an AI to do it does require a lot of data. Spoiler, we have done it. So that's regarding the topic machine learning. That's actually possible if you have enough samples. Cool. So we have some kind of output, hopefully. We have some kind of hierarchy. We figured out our modules are these bits. And we've also figured out what they do. And that's pretty cool. So yeah, we can reverse engineer all the things. Great.  What's our problem? Attackers can do it too.  We don't want attackers to reverse engineer our stuff. They might be able to put hardware trojans in there. They might steal IP. We don't care so much. We care probably more about the hardware trojans. We don't really want that happening, though. So there are a couple of countermeasures that people have thought about. The first one has kind of been mentioned a little bit, logic locking. There's also split manufacturing, camouflaging. I'll go into those three. There's a couple of more. Spoiler, they're all pretty broken. So the first thing we did, not me. This is research I'm presenting now from other people. Please do note that. It's to say, hey, let's do it like in software. Let's encrypt the functionality, right? So we add some kind of key. And we do that by adding key gates. So if you have, for example, this lovely little net list, we might add two more gates with some key inputs. And if you don't apply the right key, the functionality is not there. So your chip doesn't really do what it's supposed to do. So that's quite nice. We've actually had quite a different few different ways of doing this. So for example, the first idea was we just add key gates everywhere randomly. You know, it'll be good enough for the functionality and eventually that got broken. So we kind of had a smarter method used on fault analysis. That got broken. So we started doing something called strong logic locking, which got broken. So we started doing something called sat resistant logic locking, which also got broken and so on. So it's a bit of a cat and mouse game that we have going on there. I'll get back to that in just a minute. Let's have a look why it does fail. So the first thing is that these key gates are pretty easy to find. There's actually a nice paper which says, look, these are the key gates that we put up in here. First thing, first, they're usually XNOR gates or XOR gates. Yeah, let's look at all the XOR gates. These are pretty easy to find. And even if we do do some kind of optimization afterwards, to try to sort of hide those XOR XNOR gates, it's still able to reverse engineer it because it's very local and very deterministic of how our synthesis tool might do this. So we can actually find all those locations pretty easily and maybe even cut out the key gates if we're trying to redesign it or get the right functionality if we're trying to add in the Trojan. Furthermore, the structure doesn't really change. So if we're doing any kind of fuzzy analysis like structural stuff, we have, again, an IS round here. And if we do, like, 20% key gates in there, well, it doesn't change that much. So if I ask you, is it still an IS round? And you said, no, I'd probably, yeah, wonder if your AI is broken, maybe, or something. I don't know. So we can still do functional or structural analysis quite easily with these key gates.  We also have a lot of attacks on specific schemes. So this is the current landscape of logic locking schemes. In blue, we have our schemes. In red, we have everything that's breaking them. And I think there is only a few which are currently not broken. Those are generally the ones with a really high overhead. So that kind of sucks, which leads me to the next problem. We have practical difficulties, right? We have to have some kind of key in there. We always hate having to securely store keys. We need some kind of secure storage, or maybe some other way to do it. We also have overhead that we have to kind of look into.  I've actually talked to someone who said for the NASA, they don't do any kind of logic locking because it just sucks too much to verify. So this kind of fails quite badly. Okay. We actually kind of get to this kind of situation where if someone says let's do chip security, everyone's like, yeah, let's do logic locking instead of actually putting some thought into it. We have had some information on a FSM or sequential type of logic locking scheme in the previous talk. I would just quickly like to introduce that also other schemes is broken. This is a scheme based on black box. FSMs. So the idea is that you have in the middle your original FSM, and from each state you can get into these black box states.  And those happen if you apply the wrong key. To get into this original, I'm going too fast, into this original thing here, you actually have to apply the correct key. As soon as you take it away, you have a wrong FSM. And that's actually also broken because as we've seen in previous talks, if you actually look into that, this is the reverse engineered FSM. It's visually possible to identify all our black box states here. And you can even figure out what the key is supposed to be.  There you go. Good. Let's get to something a little bit different. So logic locking is pretty broken encryption on hardware. It doesn't work. We don't have one-way functions.  Let's do something different. Let's say in Germany we can maybe fabricate 45 nanometers. That's pretty cool. We want to do 8, though, right? And 8 doesn't work in Germany. Okay. So we're scared that the foundry is going to do something with our design that we're doing the 8 nanometers at. So we only give them the most bottom layer, the one where it's actually important, the one that needs the 8 nanometer technology. And we give that to them. They fabricate that and we get it back. And in our own foundry we do all the rest. We do all the wiring and the upper metal layers, and all the connections. And the idea is like here, without the connections we won't know the functionality because we can't actually reverse engineer the whole entire net list. That also fails pretty badly. Why? Well, first things first, we usually have some kind of physical proximity. If we design stuff, gate A at the top is usually not going to be connected to gate B at the bottom of this trip. So if you consider this gate, it's probably connected to this one or this one. We also don't usually have that many loops. So the chances of stuff going back on itself are pretty low except in FSMs or maybe for flip flops. And these gates can only drive a specific amount of other cells. So we have some load capacitance constraints that we can also basically throw into our attacker model or attack a model, attack a type or attack a scheme. And we can figure out pretty quickly what the connections are actually supposed to be by brute forcing. The next thing is that we also have sometimes bad designers who don't do this very well. So they just design usually and like normally and then take away the top metal layers and you get something like this. So you have your source gate and you have a unconnected connection. And then you sit there and go, I wonder if it's connected to gate A or gate B. And I think spoiler should probably be gate A, right? So that is also part of why it's really broken. Yeah. Good. The final kind of counter measure I'd like to talk about is something called cell camouflaging. If we can't hide the connections, let's at least hide the functionality of the cells. The idea is that if you consider these two cells here, it's a nanonore cell, they look visually different. So the metal actually looks different or maybe the dopant looks different. We can design cells where the metal doesn't look different. So for example, these are two obfuscated nanonore cells. Visually they look the same. So if you reverse engineer it on the picture, they should look exactly the same, but they are different functionalities. With the idea being, if you don't know what the actual functionality is, you can't reverse engineer the chip. We have quite a lot of problems here. The first thing is those cells are huge, right? So the area overhead is going to be pretty big, means your chips are going to cost more, you're going to have some kind of heating problems eventually. You also need to find a manufacturer willing to actually produce these because you do need special tech to actually do this. You need a whole entire new cell library and we all know how tech companies are with the cell libraries. In the past, they've now also been published some decamouflaging attacks, so they're also based on what would be normal to have in a kind of chip and to try to reverse engineer the function. They're even to brute force it because you know this can only be a nanonore. Well, that's two options. Let's brute force that. Finally, we have the problem that our SCM, so our scanning electron microscopes can actually distinguish between some dopant changes. So if it's only, if the only difference is, maybe the metal looks the same, but the dopants are different, we can actually see that. So if you have a look here at these dots, the optical microscope can't distinguish between those two, but the SCM, we have the lighter and darker dots and the same thing with the fib as well. So we can actually see the changes depending on our tech anyway. For very small dopants, this won't work, but for very big dopant changes or dopant differences, you will actually be able to see the camouflage gates again as well. Okay, reverse engineering is pretty cool, right? So we have some kind of IP protection for the companies. We can try to figure out is there something in our design we didn't want there, some kind of malicious logic. We don't want nuclear power stations blowing up because we designed our chip here and then sent it off to wherever. However, foundries use this kind of stuff to figure out where to place hardware trojans, at least that's the fear. If they know the functionality, they can figure out, look, let's place it right in the crypto core. That's what we want this thing to be. If they don't know the functionality, they won't actually know where to place the hardware trojan. And any kind of countermeasures we have are broken. So we don't really have anything very good working for us here to actually prevent some reverse engineer chips. The ugly, we don't really have any tools. I know we've previously had some discussion or some talk on the tool. We've actually found that we haven't really found anything that can be used commercially for an actual chip. So it's all very nice if you have your 6000 gates and then you get your actual chip and has 2 million gates and nothing works. So there's that. We also don't have any formal methods. So that kind of sucks because we can't actually prove anything or it's not provably secure. And sometimes it feels a little bit like no one cares, right?  And yeah, it is. But if your chips are insecure from the hardware point of view, you're going to have a bit of a bigger problem. And it's going to be a bit more difficult to replace those. So that's kind of sad sometimes. What can you do? So those two main problems I mentioned right now, the petitioning and the identification. You don't need any kind of specific hardware for that. You don't need any kind of specific tools for that. Any kind of designs you can use here are open source. So I mentioned open cores and Libre cores. Feel free to download everything there. Feel free to synthesize everything there with wonderful open source tools you can get. So you also have Qflow, all that kind of stuff. So that's something you can do. And you can find new methods to petition, to identify, maybe to counteract everything here with a normal kind of laptop, for example, this laptop which I work on. And you don't actually really need anything else. So that's quite nice. In particular, it would have been good if you had some interest in graph theory, if you had some interest in functional analysis. So I think the previous talk mentioned that they're looking for software engineers. We would like your sort of research expertise here. Maybe have some ideas of how graphs could maybe have some other functionalities to figure out how to petition better or maybe how to fix some errors there. So that's quite nice. And for that, please do feel free to contact me. I mean, if you have the SCM in your garage, also contact me. I'd love to see it. But if you have any kind of ideas of how to do this better, do let us know. And thanks so much. And I hope you have maybe some questions. Thank you very much for your talk. If you do have questions, please line up, add the microphones in the room. If you have questions from the internet, just keep asking. Signal Angel, do you have a question? What awesome software do you use to visualize such graphs? Okay. So we use a couple of different ones. We're big fans of GIFI graph tool. And we sometimes use graph viz. So those are all you can look into. With the extremely big graphs, we've had really good, good stuff going with GIFI. So even so we've had this problem with HAL, where you can't visualize stuff on the go. GIFI is able to actually run different kind of graph algorithms on your graph in real time. You may need to up the memory a little bit sometimes, but it does, I think we've done up to one million gates in there, which is quite cool. Graph tool is awesome as well, though. Microphone number four, your question. Hello. I have two questions. The first one was with the scanning electron microscope. To my knowledge, you can actually see the chip working when you are using an SEM because you can see the loads of the electrons and the transistors themselves. Why do you even need all this camouflage? Because you can see it working. Why do you need all this camouflage? This is regarding the camouflage in chips. Yeah. Well, this is the question I asked myself as well. Sometimes it becomes so small that you can't see it working anymore. So if you're camouflaging in a really small technology, we actually start getting troubles to be able to properly visualize that. So it's actually difficult enough to get pictures to reverse engineer, to actually have it functioning and to see that what's happening inside is becomes difficult. 429 minutes, yeah, feel free to just see what it does. My next question is, isn't there like, of course, this wouldn't work with the foundry, but are there like physical methods, for example, special coating, which you cannot remove without destroying the silicon? Yeah. So there have been ideas in that direction also to try to, so one of them is a coating. The other one is to try to put into the metal layer something that will, if you start physically removing them to sort of, which will scratch up the surface, there is ways to that. That is very much a physical problem though. So that's what the material guys do. And as of now, as far as I'm aware, they haven't found anything where that's really been a problem. Right. Signal Angel, do you have more questions from that? You're good. And I don't see anyone else being lined up. All right. We have microphone one. Sorry. Hello. Just a small question. Do you know a tool to do, let's see, to recover clock groups out of an at least because you mentioned the partitioning problem. I would say if they are clock gating and all the stuff, you get the function as one clock group. Yeah. So actually clock grouping is probably the first thing you would do when you start partitioning any kind of design. So your first step would always to be to have a look at the clock tree and see which parts are clocked by the same design.  As far as I'm aware, HAL does do that. I'm not sure if there's any other tools out there which specifically do that. It would be nice just to have a tool with weed in the net list showing the placement and say this flip flops belongs to this clock group just by coloring or something like that. So I think this is something that HAL does do. So that is possible with HAL. Yeah. My phone number four, your question. Yes, thanks. Can you tell me how much overhead do you add if you use this obfuscating and how often is actually used because it seems that it's quite easily broken? So is this really a thing that's widely used? Okay. So what happens with chip design is that people start designing now for chips which we've done in like three years time. And when this was the hot shit, people decided let's the logic obfuscation right before our chip in three years time. So there's chips on the market which do have some kind of logic locking on there. The overhead depends on how you implement it. You're not going to put in 100% key gates. You usually only do it for the parts that are important to you. So for example, for crypto modules or for your CPU. And so the overhead depends. Are we talking about the logic encryption or the obfuscation or the camouflaging? Because that's a little bit different. So the one depends very much on the cell library you end up using for this camouflaging stuff. There is a lot of different ideas of how to do it well. The more difficult it comes to for us engineer, the bigger the cells are going to be and you have overhead to two, so 1.5 to five times as big in your chip for the part that you camouflage. Again, that's not going to be your whole entire chip. That's going to be the part that's of interest for you. For the logic locking, you can usually choose a little bit more, you know, maybe you have some space left in your chip. You go, let's do 20% logic locking on this one part and that's just enough to fill it in perfectly. So you have a little bit more possibility to choose there. Thank you. On microphone number one, your question. Hello, thanks a lot for the talk. I was just wondering with the potential evolution of packaging technologies, like 3D stacking, how do you perceive the future of your field with that? Okay, so we've actually worked in the past with some people who are trying to do 3D printing and so try to integrate more in there. Again, that is kind of a physical process that has to be done. My research very much does this high level. I would assume that as long as we have to test chips, we're always going to have the tools to be able to analyze them. And even with 3D stacking, we're going to have to be able to have tools to be able to actually get in the chip and do take the images. So for fault analysis we need it, we can use reverse engineering. And the same thing goes for low-anameter sizes. Thank you. Microphone number two, your question, please. FPGA. Can you do this also with FPGAs? Okay, so I'm not an FPGA person. That would have been the guys from Bochum, they do that. As far as I'm aware, this whole clustering stuff, so this first kind of stuff, that's exactly the same for FPGAs. I am not sure how camouflaging works for FPGAs. I don't think it does. Logic locking you can do. So that's fine. Thanks. Microphone number four, your question. Yes, hi. Regarding the problem of sending your designs to the foundry and getting it back modified. So I'm asking, what's the status of this in the real world? So is it a real problem? Can companies actually verify quite well that they received what they sent? Or is it you have to trust them blindly? How much is it actually possible? Okay, so there's two parts of this. And this is a typical do hardware choices really exist problem. As far as I'm aware, it hasn't been seen in the industry. But again, they probably wouldn't tell me if it had been. The overhead to actually get something in there in the foundry is quite large. So this is not going to be something that one single person does. This would need a probably some kind of state actor to do this. The problem is that our foundries do lie in countries where we have maybe that kind of problem. At the moment, I know companies are working on checking their own product. So I am aware of big chip companies that do do this kind of thing where they get their chips back and reverse engineered. It feels like it's very much in its baby steps. I do hope eventually we're going to get some kind of certification. So we have certification now for chips. And I hope eventually they'll have some kind of we reverse engineered it when it came back and it was fine kind of certification. That's where I hope it would go in the future. But I think it's a long way off. If you had to guess, so if let's say if a state entity sends out a design to a foundry and then a state operator there makes a modification. So who wins? Kind of depends on the state I would say. So in America, all the big research and this is founded by DARPA and they have a lot of money. I'm going to leave it at that. All right. Thank you very much for your talkity and for answering all the questions.