this question that comes up and even today it came up, oh so your parents are doctors and why didn't you get into medical? And so this question I've been answering for many years but I think what has carried forward is their ability to be able to understand people and also to be able to empathize.  So much sensitive that sometimes my parents would be like, oh but if you see blood you have to be strong because you can't be like oh that person is crying because you have to be stronger to see their pain also. But I think that sensitivity has helped me become who I am today and also helped me become a designer. And of course you know seeing my parents I was also keen on to pursuing medical but I'm glad to say that I'm in the design profession which requires you to be more sensitive, more empathetic and also more understanding towards others' needs. So here I am today presenting to you some of my experiences and although these experiences are related to the work that I'm doing at SAP but I think all of us could relate to it because we are in product teams, we are in big IT companies, some of us also have our own startups and some of you are entrepreneurs. So I think we also face similar challenges and I hope that my talk helps you connect with it and also helps you understand how we could kind of deal with those challenges. I just, how many of you, I'm just deviating a little bit from the topic but how many of you have followed the Olympics for 2016? Can you please raise your hands? Okay, great. So this is the excitement that just got over around all the events and what we really saw was a testimony of great effort and passion by Indian sports women and of course there were a lot of prejudices and a lot of things but they stood tall against it and these are some of the images of their grit and glory I would say and of course this is a great inspiration for women, not only in design but in all walks of life. So here I am just sharing some of these images but what we take from this is how it got translated into the design journey. So the preparations and hard work that have gone over the years and motivation to break the record was even higher at the Olympics and of course what we see here is a lot of effort that have gone into it but where I work in custom development it's really tough because it's not like a product scenario where we have huge time cycles and of course there is advantages to that where we have enough time to do user research and all of follow the exact process but in custom development I see it as a marathon. Running a marathon is like a 100 meter race and you have to get prepared for it and all eyes are on that. So what happens really is like it has that enthusiastic start and also dramatic finish in a very short span of time. So taking this analogy forward at custom development we learned that in shorter design sprints with the same excitement and rigor we could open doors to collaborate with customers on further requirements. So what exactly do I mean by the design sprint or with an SAP we also call it sprint zero. So the design sprint is a mini project in itself which delivers tangible results within a short period of time. So what we have is like the execution plan that has to be quick and collaborative in nature and which helps our customers envision the solution that much before it gets developed and also kind of gives them confidence that their expectations would be met and their vision would be converted into reality. So basically this is where the design sprint kind of fits in. So we have the UX concept and design which we call the design sprint and then we have the following sprints sprint one, two, three carried forward. I'm sure most of you who are in the agile mode understand the sprint cycle. And so what really happens is that a small engagement that has encapsulated in such a manner and helps build customer trust. Also assist customers to take conscious decisions while spending their money and also reduces risks. And what benefits is that we have tangible results. Also we get a better understanding of requirements and build deliverables which can further be consumed by the following up custom development project. So our focus really is individual interactions over processes and tools. A working software or prototype over comprehensive documentation. Customer collaboration over contract negotiations and responding to change over following a plan. So this is the shift that we have made and of course this is how it typically looks like but it's generally following the same cycle of design but in a shorter cycle I would say where we can speed up the innovation, we can issue discovery interview and based on the workshop mode and also build a quick prototype which leads to finally a specification and backlog which gets used up in the sprint, upcoming sprint. So where this actually helps us deliver quick tangible results in a short time, in a close collaboration and also it helps us create a joint understanding and of course budget is always an issue so it's an attractive pricing because we are doing a smaller sprint and we could use those design sprint deliverables into this upcoming sprints. So this is from the methodology point of view in CD where we think of the objective and what would be our commitment and of course what would be the customer's commitment and the deliverables. So I'll quickly skip through this and walk you through the other areas which is so what it really helps is that we could actually identify the initial ideas and a rough project size gets discovered. We are in the customer, we get the customer support and readiness early on which leads to close collaboration during the design sprint and we also get the required customer resources which means the user representatives or end users in IT. So how does it work in the plan? So basically the business process workshops that we conduct are generally over a span of three weeks to two months and these business process workshops lead to product vision. They also convert user research which further leads to understanding the end user requirements, the technical assessment which leads to architecture understanding and the mock-up or prototype creation which leads to a tangible result and then finally validation which leads to customer acceptance. So this design sprint actually helps us build a very good relationship with the customer early on and we actually try, the customer kind of gets to understand that this is SAP's way of working and there's a comfort level and the customer opens up to further talks and iterations. As we already saw in the earlier talks, design is a iterative process and until and unless we get our customer support we will not be able to reduce good results. So what is the design sprint we already talked about and why the design sprint? So SAP, as some of you would be knowing, is a world leader in enterprise applications and we are world's third largest independent software manufacturer but still we have our challenges and what are those? Basically, the top use cases are identified in the traditional approach but they're never budgetized. The value of the app is not tangible before the final design is there and of course we have concurrent business requirements yet their unclear effort estimates are based out of uncertainty and there are unrealistic customer expectations. Also, the backend integration is one of the main effort drivers which gets ignored. So the typical software development approach as we all know it is that as soon as we get the contract that gets signed, we get into the execution phase where we quickly start gathering requirements, documenting specifications, sprint, scrum planning, resources. That's the original approach which most companies follow and of course the shorter timelines but there is a shift with the design sprint is that it acts as a hook to build trust early on. So it helps. Basically, the short timeline helps us build a relationship with the customer. The iterative approach helps us get quick feedback on from the customer and end user and once again it's a door opener for bigger deals and projects and we build our solution which is focusing on user friendliness so it's the end user that we get directly involved with and stay in focus. The visualization part helps us to deliver reusable mobility components and also reuse the design and specification backlog which was created in the design sprint in the following CDP and also the design to budget. So if the budget is constrained basically the customer gets to understand what is the design process and the value of design gets up front, you know. So basically the customer support and helping building the requirements base and the design solution gets evangelized early on. So customer point from the customer point of view it's valuable because it increases trust, decreases the entry barrier and we give tangible results so it's a wow factor and of course we've mitigated a lot of risk of taking on a huge piece to be able to do it all in one go and we establish a working model. From a custom development point of view it's higher win rate and we could support sales cycle. We have given accurate estimate because we've already worked with the customer once so we understand what is there and there is IP protection for the early sprint so it doesn't really go to the competitor and there is software and software related there is a lot of revenue generated from this software and software related services. So the design sprint thus fits very well from a technology and business value point of view and it is this impact which is created by the design sprint by focusing on smaller milestones which leads to bigger milestones in future. So the impact that is made through the design sprint and that gives this end users a glimpse of what could be achieved and in a short span of time it further helps gain trust and give assurance that SAP has the right skillset and know-how and could be a trusted partner in the journey. To gain insights into the requirements and also tie up with them in their vision. So these are some of the deliverables which come out of the design sprint which is our vision nerves and scope document and initial product backlog as well as a user experience of a prototype and then a final financial proposal for the next offer. So this is our new approach towards focusing on design and helps building trust, excellence and success. This impact which is getting created is at the low entry barrier risk mitigation and tangible results which we already talked about. Again, an established working model and so many times I've seen customers, project managers saying that what is the value of UX? And of course to that I would just say it's experience by doing. Until now, this is the experience they work with us. Just recently I was on a customer site and my project manager was like, but you're a UX designer, what will you do? I don't know why am I taking you on the customer side and by the end of the workshops, by the end of the design sprint he actually came and thanked me that thank you. I mean today I finally realized what a great impact it has made and it saved us so much cost of building it straight away. So I think it is essentially experience by doing and we all have to fight this battle as UX designers and we're continuously striving but it's the same grit and determination that we saw in Olympics that women in design have to also take up. So with that I would like to, since we're running out of time also, I would like to thank you and for questions, are there any questions? So are there things that women's print cannot solve? Cannot solve. So see design sprint is focusing on smaller tasks so we're not taking the whole chunk altogether. So yes, it's not solving all the problems in one go and that's the whole idea that we are not going to solve all the problems in one go. We're just going to give the customer a flavor of what it is to get one problem solved at a time and once the customer is happy, those deliverables get anyways converted into the project and we obviously have continuous workshops later on. So what that answers? That the research work is included in this grid of we do a full research and then start sprint design. So the research phase is included in the design sprint. There was one slide which I showed, I'll just go back to it maybe. That was the how part. So we have in the design sprint, we are focusing, that's a good question because we definitely are not covering all the research part in one go because as I said, we're limited on the time and it's a very short design sprint. So but yes, it helps in a way because we are focusing on, so we understand and prioritize. So it's called prioritizing backlogs and when we're trying to understand user requirements. So we definitely get an overview and from that we prioritize that in the design sprint we're taking these and in the upcoming sprints, in the upcoming engagements we'll be taking onto those. So at least the customer gets to understand the process and the value of it which is otherwise missing. Otherwise everything's taken, you should only take it back much that you can chew. So it's like that. But sometimes we have to go back and get, if I get further research, I get points where I need to go back and change my design to further older ones. How do we deal with the situation? Yeah, so if you do change, see in any case, you will be, at least for the design sprint, in the workshop mode it's like a war room situation where you're actually there understanding, creating and coming back with validated stuff. So it's not that you close everything there and there. And what comes up is the next new thing. So sorry, just one minute. So yeah, so basically you're closing all the things that you promised for that design sprint within that period of time, whether it's three weeks or four months or two months, basically it's shorter time duration. It's not like a one year or two year cycle. And what comes up next is the next phase of workshops. So basically this is like sprint zero. You could say it's like preparing for a race. Thank you.


So thank you for that introduction and to the organizers for this opportunity to speak to you guys So my name is Peter Sewell. I'm an academic computer scientist and as such I Bear a heavy burden of guilt for what we have to work with shared guilt to be fair But guilt I'm the same In many many years of higher education. I know two things about computers Maybe you've figured them out slightly faster First thing there's an awful lot of them about shocking And the second is that they go wrong they go wrong a lot really a lot a lot a lot a lot and they go wrong in many interestingly different ways So sometimes they go wrong in spectacular ways like with fireworks And here we see Here we see half a billion dollars worth of sparkle You don't get that every day and so what was going on? So as I understand it What was going on was that the this was the first flight of the Ariane 5 launcher And it reused some of the guidance software from Ariane 4 That's fine. Good tested code should reuse it But the launch profile for Ariane 5 went sideways a bit more On purpose than the Ariane 5 launch And as a result one of the sideways values in the guidance control software overflowed bad and Hence the guidance software decided that something bad was going wrong and in order to not land rockets on people's heads It kindly blew itself up. So that's a spectacular Sometimes they go wrong in insidious ways and new people probably know much more about that than me Very insidious ways exploited by by bad people to our detriment and they go wrong in insidious ways exploited by our kindly governments for our protection and these There are many many many different causes of these things, but many of these causes are completely trivial You know, you have one line of code in the wrong place something like that So that's very very bad and you're all conscious of how bad that is But from my point of view, this is as nothing With the real disaster that we have been facing and continue to face the real disaster is that Enormous waste of human life and energy and effort and emotion dealing With these crack machines that we build It's hard to quantify that. I did a little experiment a couple of months ago. I googled up Android problem and Windows problem and Got about 300 million hits for each Somehow indicative of the amount of wrongness So this should seem for kind of familiar to you if you think back think back to when you were young Maybe back in the 19th century When you were young and what were you doing then as a community of hackers and makers and builders? Well, this was the the full-on industrial revolution. You were building stuff You were building beautiful bridges of cast iron and stone with numerous arches and pillots and every so often They would fall down Believe someone miscalculated the wind loading and we would build Magical steam engines to whisk us away and every so often They would blow up And that kind of thing as you may have noticed in the last hundred years Sorry blow up that kind of thing doesn't happen very often anymore. We have pretty good Civil and mechanical engineering. What does it mean to have good civil and mechanical engineering? It means that we know enough Thermodynamics and material science and quality control management and all that kind of thing to model our designs Before we build them and predict how they're going to behave pretty well and with pretty high confidence. I See an optimization in this talk For computer science for computing systems, we can predict with 100 percent certainty that they will not work Time for lunch, but it's not a very satisfying answer really. So why is it so hard? Let me discuss a couple of the reasons why computing is tricky in some sense trickier than steam engines The first reason is that there is just too much code All right, so this is one of the more scary graphs. I found on the internet recently This is a measure of code in hundreds of millions of lines each of these blocks I think is 10 million lines of code and in the little letters You might not be able to see we have Android and the large Hadron Convert Collider and Windows Vista and Facebook US Army Future Combat System Mac OS and Software in a high-end car I'm flying home So that's a bit of a problem really. We're never going to get that right Then there's too much legacy complexity there really too much So as right-minded pure-minded people you might think that software and hardware should be architected beautifully with Regular structure a bit like a mead Vandero skyscraper or something clean You know, that's not realistic. All right, you might expect a bit of Baroque nests. You might expect Maybe something more like this, you know, it's a bit twiddley, but it hangs together. It's all got structure still works But then you stop and you think and you realize but software and Hardware it's built by very smart people and usually in big groups and subject to a whole assortment of rather intense commercial pressure and Related open-source pressure and using the best components and tools that they know and that they like and The best management that human beings can arrange So we end up with something spectacularly ingenious And you can see here if you look closely you can see all the parts.  Let's say that's our hardware and over here. I see a C compiler with all of its phases and These bottles I that's the file system. I reckon and There's a TCP stack in here somewhere and There's a piece that belongs to the NSA, obviously and Up here. I think up here. Yeah. Yeah, there's a JavaScript engine and TLS stack and web browser and Then perched at the top is the software that most people use to talk to their banks It's just It's just insane And then we have to look at how we build these pieces So we build them fundamentally by a process of trial and error, right? Just occasionally we write some specifications in text And then we write some code and maybe we write a few ad hoc tests And then we test and fix until the thing is marked all and then we test and fix and extend Until the thing is no longer marked full And then we keep on using until we can't anymore So the fundamental process that we're using to develop these systems is trial and error by exploring particular execution paths of the code on particular inputs But you know this because I want to highlight there are too many The number of execution paths scales typically at least Exponentially with the size of the code and the number of states scales Exponentially the size of the data that you're dealing with and the number of possible inputs is also shockingly large. There is no way That we can do this and we should perhaps stop deluding ourselves that there is and Then the most fundamental reason But computers are hard is that they are too discreet If you take a bridge or I know a chair or something Where's something slightly bendable You take a chair and you exert a bit more force on it than it's used to I'm not very strong and it bends a bit. It deforms continuously If you take a bridge and you tighten up one of the bolts a bit too much or you undispecify one of the girders by 10% Occasionally you will reach a point of catastrophic failure But usually it will just be a little bit weaker or a little bit stronger or a little bit wibbly But computer systems you change a bit Sometimes it doesn't make any difference, but disturbingly often what you change a line of code as in some of those bugs that we were talking about The whole thing becomes broken So it's quite different from that traditional engineering. It's really different Okay, so a New dawn this thing is titled. What can we do? What might we do? So I'm going to talk about several different possibilities One thing We can do more of the stuff which is traditionally called software engineering It's not really engineering in my book But some of it's good. I mean, it's useful You know, we can do more testing and have more assertions in our code and better management Maybe and all that kind of stuff we can do all that we should do all that but it's not going to make a very big change It's not addressing any of those root causes of the problem or else could be too So as an academic computer scientist, I've devoted several years of my life to working in the area of programming languages And I look at the programming languages used out there in the world today.  It's shocking the reasons that languages become popular and Built into a shared infrastructure the reasons for that have almost nothing to do with how well those languages are designed and in fact Whether they are designed at all. I didn't really want to get too much hate mail after this talk So I'm going to try and avoid naming particular languages as much as I can But I encourage you to think away as I speak and imagine the examples No, that was I didn't want to get much hate mail. Sorry So at the very least we could use that programming languages based on ideas from the 1970s instead of the 1960s I mean really come on I You know back in 67 or 69 or whatever we had B CPL and C and only a couple of years later We had languages that guaranteed type and memory safety and Enforcement of abstraction boundaries and gave us rich Mechanisms for programming so that we could say what we mean and not fuss about with 23 million pointers About one percent of which were completely wrong So for any particular in the context of any particular project or any particular existing legacy infrastructure There may be very good reasons why one just can't make a switch But for us collectively there is no excuse It's just wrecked it completely bonkers and then the other thing that you see as a programming language researcher is that Well to make another analogy What it looks as if anyone who was capable of driving a car Considered themselves an expert on car design There is in fact a rather well-established subject of You know programming language design and we know how to specify Completely precisely mathematically precisely not just the syntax like we had in these B&F grammars from the 1960s And still have but also the behavior of these things We know how to specify the operational semantics and the type systems and we know how to do mathematical proofs that Our language designs are self consistent, but they do guarantee good properties of Arbitrary well type programs that you're right So we can do this now We can do it if we have to post talk and people are doing this for JavaScript and see and God help them PHP and other such languages But even better we can do it at design time and if you do this at design time you see the structure You have to the act of writing that specification Forces you to consider all the cases and all the interactions between features you can still get it wrong But you get it wrong in more interesting ways So people are starting to do this now So I think there was a talk on the first day of this by Hannes and David who are building system software in Maybe not the bestest language you could possibly imagine but something which compared with C is as manner from heaven An annual matter of petty and his colleagues in Cambridge have been working hard to build system software in at least Moderately modern languages and it works Sometimes you have to you see but surprisingly often you really don't Yeah, so I Teach as my introducer said in University of Cambridge and in fact I teach semantics One of my fears is that I will accidentally let someone out into the world who will accidentally over a weekend find themselves Inventing a little scripting language, which will accidentally take over the world and become popular and I won't have explained to them what they have to do or even that there is a subject They have to understand and if you want to understand it. There's lots of stuff you can look at Okay Let's go to the the uppermost extreme if we want software and hardware that actually works We should prove it correct So this is something which academics especially in the domain of labeled formal methods in the 70s and 80s They've been pushing this Promoting this as an idea and I promise and an exhortation for decades Why don't we just prove our programs correct and for some of those decades it wasn't really plausible I remember back when I was considerably younger It was a big thing to prove that if you took a link list and reversed it twice. You've got back the same thing That was hard then But now Well, we can't do this for the Linux kernel. Let's say but we can surprisingly often do this and I just give you a couple of the examples of modern-day state-of-the-art academic verification projects people have verified compilates all the way from see like languages down to assembly or from ML like languages, you know higher-order functional imperative languages all the way down to binary models of how x86 behaves and People have verified LVM optimization passes. There's verified software fault isolation Which I believe goes fast as in the original non verified form win for them There's a verified hypervisor from the next group in Australia Verified secure hypervisor with proofs of security properties. There's any amount of work verifying crypto protocols Sometimes with respect to assumptions, but I think reasonable assumptions on what the underlying mathematics is giving you So we can do this I maybe have to explain a bit what I mean by prove to a some computery people I proved it works means I Tested it a little bit In extremis it compiled to a program analysis person you might run some sophisticated tool But written in untrusted code and possibly doing approximate and maybe untowned analysis And you might find an absence of certain kinds of bugs. That's tremendously useful It's not what we would really call proof. I mean nor were they to be fair Scientists generally don't use the word because they know they're not doing it All right, they're doing controlled experiments which gives them, you know information for or against some hypothesis mathematicians write proofs And you probably did that when you were young and those proofs are careful mathematical arguments usually written on paper with pencils And their aim is to convince a human being that if that mathematician was really nailed up against the wall and pushed They could expand that into a completely detailed proof but what we have In computing we don't have the the rich mathematical structure But these people are working with we have a tremendous amount of ad hoc and stupid detail Mixed in with a smidgen of rich mathematical structure So and we have Hundreds of millions of lines of code So this is just not going to cut it and if we want to say the word proof ever Then the only thing which is legitimate really is to do honest program proof And that means you need to have some system that machine checks that your proof is actually a proof and Sometimes we also have machines that will sort of help and sort of hinder that process And there's a variety of these systems cock and hole for and is well for is well hole and what have you that one can go and look up So using these systems we can prove Non-trivial facts about these not necessarily that they do what you want them to do But that they meet the precise specification that we've written down We can do that But it's tricky So these projects these are by academic standards all quite big You know, this is something like no ten people for a few years or something by industry Scale maybe not that big by your scale Maybe not that big but still a lot of work for a research group to do and quite high-end work It can take you know, literally a few years to become really fluent in using these tools And there isn't as yet sort of a open-source collaborative Movement doing this kind of stuff. Maybe it's the time to start Maybe in five years ten years. I don't know If you want a challenge There's a challenge But it's really quite hard It's not something that one can quite put forward credibly as a solution for all software and hardware development So that leads me to like an intermediate point. That should have been a four there What can we do which improves the quality of our systems? Which is inject some mathematical rigor, but which is less work, which is kind of easy and what we can do is take some of these interfaces and be precise about what they are and Initially we have to do that after the fact we have to reverse engineer good specs of existing stuff but then we can use the same techniques to Make much cleaner and better tested and things which are easier to test In a future. That's the idea so So my colleagues and I've been doing this kind of stuff for the last quite a few years And I just want to give you just the two two little vignettes Just to sort of show you how it goes. So I can't give you much detail And this is obviously joint work with a whole bunch of Very smart and good people Some of which I name here. I think so This is not me. This is a whole community effort So I'm going to talk for a little bit about the behavior of multi processors So at that hardware interface And I'm going to talk a tiny little bit about the behavior of the TCP and sockets API network protocols So and there'll be some similarities and some differences between these two things So multi processors You probably want your computers to go fast most people do and It's an obvious idea to Glom together because processors don't go that fast Let's glom together a bunch of processors and make them all talk to the same shared memory This is not a new idea It goes back at least to 1962 When the burrows d825 and I think two processors Talking to a single shared memory it had outstanding features including truly modular hardware with parallel processing throughout and Some things do not change very much The complement of compiling languages was to be expanded 1962 so that'll be 52 years ago, dear me Okay, now how do these things behave so let me let me show some code right? It's a hacker conference They should be code. Let me show you two example programs and these will both be programmers with two threads And they'll be acting on two shared variables X and Y and in the initial state X and Y are both zero so first program On this thread we write X and then we read Y and over here we write Y and then we read X And he's operating you know in an interleaving concurrency you might think so There's no synchronization between those threads so we might interleave them We might run all of thread naught before all of thread one or all of thread one before all of thread naught Or they might be like mixed in like that or Like that or like that or like that as six possible ways of interleaving two lists of two things and in all of those ways You never see in the same execution that this reads from the initial state and this reads from the initial state You never see that So you might expect and you might assume in your code that these are the possible outcomes So what happens if you were to actually run that code on my laptop in a particular test harness? Well Okay, you see occasionally they're quite well synchronized and they both read the other guy's right Sorry big call they both read the other guy's right and Quite often one thread comes completely before the other but sometimes Sometimes they both see zero Huh rats Well, that's interesting. Let me show you another program So now Thread nor writes some stuff.  Let's say that's a flag Announcing to some other thread that that data is ready to go now and the other threads busy waits Reading that value until it gets one till it sees the flag and then it reads the data So you might want the programmer you might want to guarantee that this read will always see the data that you have initialized Okay, that'd be like a message passing kind of an idiom. So what happens if you run that? well On the x86 processors that we've tested you always see that value good This is a decent coding idiom on x86 apparently on Arm and IBM power processors. However, you see Sometimes you just ignore the value written a new read initial state Okay, so this is not a good message passing idiom. All right, so what's going on? well This behavior it might be surprising And when you see surprising behavior in hardware You have two choices either it's a bug in the hardware and we have found a number of bugs in hardware It's always is Very rewarding or It's a bug in your expectation Or it's a bug in your test harness So what and what you do is you walk along and you walk up to your friendly processor architect in IBM or arm or x86 land and we have worked with all of these people And you ask them is this a bug? and a processor architect is a person who is Has the authority to decide whether some behavior is intended or is a bug That's what they do essentially And for these they say oh we meant it to be like that No question. We meant it to be like that. That's perfectly proper We have because you and Everybody else wanted their computers to go fast. We as processor designers have introduced all manner of sophisticated Optimizations, which if you were running sequential code you would never notice But if you start running fancy concurrent code like this fancy concurrent code that isn't just trivially locked You can notice so this is intentional behavior And if you want to write that code like they are mutually exclusion algorithm or a message passing algorithm or something Then you need to insert special instructions memory barrier instructions and so you go away and you say What do they do and you look up in the vendor documentation? You get stuff like this I'll read it out For any applicable pair a B the memory barrier ensures that a will be formed with respect for any first-general mechanism to the extent required by Dissociated memory coherence required attributes before B is performed with respect to that first-general mechanism A includes all applicable storage access by any such first-general mechanism that have been performed with respect to be one before the Pro memory barrier is created B includes all applicable storage access by any such first-general mechanism that are performed after a load instruction executed by That first-general mechanism has returned the value stored by solve it is indeed Hands up if you understand what that means Hands up if you think that if you thought about it and read the rest of the book you would understand or you could understand what that means Hands up the people who think that we doomed forever So I'm sorry to the first few groups, but the last ones are right So this looks Like it's really intricate and really carefully thought out stuff And we thought that for a while and we spent like literally years trying to make precise mathematical models that give precise meaning to all of these words But actually it can't be done so So what do you have to do in that circumstance you have to go away and You have to invent some kind of a model And there's a really fundamental problem that we on the one hand we develop our software by this trial and error process But on the other hand At points like this we have these loose Specifications supposedly that have to cover all manner of behavior of many generations of processes that might all behave the same way Written just in text and we tell people they should write to the spec But the only way they have of developing their code or their hardware usually is to run it and run particular Executions on the particular hardware implementations that they have whose relationship to the spec is very elusive We can't use those thick books that you get all those Quite weighty PDFs that you get these days from the process of vendors to test programs You can't feed a program into that thick book and get output out And you can't test process of implementations and you can't prove anything and you can't even Communicate between human beings So these things really they don't exist So what can we do? Well, the best we can do at this point in time is a bit of empirical science So we can invent some model off the top of our head Trying to describe just the programmer visible behavior not the internal structure and we can Make a tool because that's not froze now now We can now it's stuff It's real mathematics and we can turn that into code and execute it We can make tools that take programs and calculate small programs the set of all the behaviors allowed by that model and then we can compare those sets of behaviors against the experimental observations and You might have to fix the model and you might find hard bugs and then Well, the model also has to make sense to the architect So you can just can discuss with the architects whether it matches what they intend and what their intuition is and then you can Also prove some other facts about it to give a bit more assurance and then because you probably got this wrong in the first couple of times You can go back And this results or has resulted in models which are not guaranteed to be correct, but they are Effectively the de facto standard for understanding the concurrency behavior of these things Yeah, and various people use them And just to give you a tiny snippet I have to speed up a bit a tiny snippet of what the model looks like It's basically just an abstract machine It's got some stuff for the threads that handles the program invisible speculative execution and some stuff for a storage subsystem That handles the fact that in these architectures memory rights can be Propagated to different threads at different times And There's a that model has a state which is just a collection of some sets and lists and partial orders and a few other things Which I won't talk about and it has some transitions in any state There might be several possible transitions. So this is just a sample. I'm not going to explain all of this But when you want to propagate a right to another thread, they have to there are some preconditions that you have to satisfy And then there's an action. It's not amazingly complicated really But this is so this is text and but it's very precisely written text And it has a great advantage that you can go through these bullet points with your friendly architect and say is this what you mean And they can think about it and say yes or no But for the actual definition that's in mathematics But it's mathematics which is quite close to code. I mean it's terribly close to a Pure functional code with outside effects and just to give you an idea This is some of it and those are three of those conditions in like the real honest true version Then we can take this Mathematics and because it's in a particular style we can compile this into actually OCaml and then OCaml bytecode Well, and then we can run that for batch jobs, but then you can compile that OCaml bytecode to JavaScript and glue on a user interface and Stick it into a browser and then you have a web interface that lets people explore This rather intricate model and that's also a necessary part of validating that it makes sense Okay, this is not rocket science. You've missed the talk for rocket science. I'm sorry All we're doing here is specifying the intended behavior of a system Okay, that's it's not very technical, but it's unusual We happen to be doing it in this mathematical language, but you could use in fact almost any language So long as you understand what you're doing what you understand What you have to understand is that you're writing something which is not an implementation It is something which given some observed behavior some arbitrary observed behavior will be able to decide if it's Allowed if you want it to be allowed or not, right? It has to be executable as a test or a call the key question for getting this to work is to Understand how much non-determinism or loose specification there is in the system that you're working with So if everything is completely deterministic you can write a reference implementation in the cleanest language of your choice and Just compare the output of that and the output of your real thing This is more non-determinism then you can't do that I'm gonna have to abbreviate this a little tiny bit So so for TCP and network protocols there is more non-determinism. You know what TCP is yes Protocol gives you sort of reliable connections assuming that the world is made of good people You look at the standards for TCP and they're the same kind of ribbily text from the 1980s And you look at the implementations and they're a ghastly mess And you try and understand the relationship between the two of them and you can't because the standard again There's not he's not a definition. It doesn't define in all circumstances. What behavior is allowed and what isn't So again, we had to make up a specification in the first instance just of this host a single endpoint and Observing its sockets API calls and returns and its behavior on the wire And when you've decided and a few internal debug events when you've decided what to observe then an observation is just a trick It's just a list of those events and you have to ask be able to ask your spec. Is that list? admissible or not? but There's a lot of non-determinism Deration between implementations variation within an implementation and that's internal a lot of it It's not announced in the API or on the wire Maybe until much later when the implementation picks some new I don't know a window size or something You can't tell until quite a lot later in the trace what it's picked And that makes the job of checking whether a trace is admissible by the spec much harder than it has to be And if you had designed the TCP protocol and those implementations For to be testable in this very discriminating way back in the 1980s when you were designing TCP protocol It would be much easier And for new protocols one should make sure that you're doing this in this particular way I answer come on to talk about that slide That's just one of the rules of that specification But the key fact about that spec is that well we handle the real protocols for arbitrary inputs But it's structured not just for this execute of executability as a test oracle, but it's structured for clarity not performance Which is scarcely ever what anyone ever does so it's a whole new kind of thing and The testing is very discriminating so we found all manner of amusing and bizarre bugs Which I think I don't have time to talk about Okay, so So I've described three kinds of things that we might do First thing oh for goodness sake. I mean it's just a no-brainer Just do it already. Everybody. All of you. All of you this middle zone is Very interesting zone as far as I'm concerned It's none of what a place where we can get a very good benefit for not that much effort We can do it if necessary post-talk. We can also do it and even better at design time We have to do this in a way that makes our specifications Executable as test oracles and another thing that enables is completely random testing when you've got a test oracle You can just feed in random goop. You know, this is fuzzing, but better fuzzing feed in random goop and Check at every point whether what the behavior that you're getting is allowed or not and then they're written for clarity not for Not for performance and that means you can see what you're doing You can see the complexity if you go ahead and you add some feature to your protocol or your programming language or whatever And you're working just with text specifications Then you can't see the interactions.  I'm being silly. I'll make it simpler That's the usual for proof So this I think also is pretty much a no-brainer and it doesn't require great technical you know mathematical skill either lots of people can do this and Then there's an option three best option full-on formal verification of the key components And that is now also within reach you can imagine secure systems made using actual verified compilers and Verified hypervisors with possibly verified TLS to acts and you can imagine making things out of those Or making those and then making things out of those and maybe one should be thinking about that so Maybe not appropriate for everything if you are writing word you would not wish to do these things probably you're not But for this key infrastructure that we really depend on that we trust even though it's not trustworthy We have to do this Is this a new dawn? I wonder if there's anyone standing next for light switch that can dim them for me I didn't ask them beforehand so If you think back the last 70 odd years we started building computers in around 1940 It's been pretty dark From the point of view of rigorous solid reliable engineering stuff that is actually trustworthy pretty dark I would say um Maybe just maybe there's a tiny smidgen of light coming through the doors And we start to have a little bit of a clue and we start to get reusable models of Processor architectures and programming languages and network protocols and what have you it's just the beginnings of the analog of that thermodynamics and material science and quality control management and what have you that we have for mechanical engineering and and We've got no choice If we don't the next 75 years is going to be a lot worse You can just imagine all right So I went to this as I'm sure many of you do this great talk on some Apple bootloader Bootloader exploit yesterday, which was relying on some feature that had been introduced in the early 80s and was still present And you can imagine in a hundred years from now you can imagine as long as human civilization endures The x86 architecture and the sockets API and all of this stuff It will be embedded in some monumental ghastly stack of virtualization layers forever So I like to thank Especially all of my colleagues that have been working with me or not with me in these directions I'd like to thank our generous funders who support this stuff I'd like to thank you for your attention and I exhort you Sort it out. Thank you very much Peter for those interesting insights to our programming We have now time for a Q&A so those people who have to leave the room, please do so quietly and As fast as possible so we can go on and hear what the questions are So we start with microphone for please Hello, thanks for the talk and my question is if you got an oracle of the software behavior Which can translate every possible input to a correct output? How can this be less complex and error-prone than a reference implementation? So the good question. So the first point is that this oracle Doesn't have to produce the outputs. It only has to check that the inputs and the outputs match up And then the second point is that in general it might have to have Much of the same complexity, but by writing it to be clear as opposed to be fast You may adopt a completely different structure And so the structure of our TCP spec is organized by the behavior that you see in for particular transitions the structure of a real implementation has a fast path code and lots of complicated intertwined branching in all manner of of Excess complexity of implementation complexity if you will Thanks So microphone free, please. I wanted to ask you what you thought about Programming by manipulating the abstract sentence syntax treat directly so as to not allow Incompilable programs because of some like you're missing a symmetrical or something like that What's your thoughts and something's that so that's in the grand scheme of things? I think that's not a very big deal right so there's people have worked on Structured editors for Lordy knows how many years It's not a big deal because it's very easy for a compiler to detect that kind of thing And even more it's very easy for a compiler of a sensibly design language to detect type errors Even for a very sophisticated type system so I Don't think that that's not I mean some people might find it helpful, but I think it's getting to the heart of the matter Thanks So we got some questions from our signal angels from the IRC Yes, there's one question. It's about the repository for Isabel and whole proofs It's this should be on source forge and you said there are no open source repositories What about them? Not quite sure what the question is really so there's a repository of Isabel formal proofs Which is The archive of form and proofs it's called I didn't really mean to say that there are no open source Repositories and in fact, I don't know under what conditions most of those proofs are licensed They probably are open, but there isn't an open source community building these things It's still pretty much an academic pursuit Microphone to please yeah, hello, thanks again for your talk I just want to add something that you didn't address And That's that we actually need more good studies in software engineering We often hear a lot of experts and also very well known computer scientists say things like Well, we just need to do functional programming and OOP is bad and stuff like that Which there I think there's a lot of truth to it, but Um, we actually need studies where we test these kinds of claims that we make Because when you look at other fields, which you also did compared to like medicine If somebody comes around and is well known and says like things like homeopathy works. We just We don't believe them. We we we do trials. We do good trials and there's a lot of myths out there like or or not well tested things like Hire good hire the best programmers. They are a hundred times more productive to use static type systems and stuff like that And we actually need to verify that this is actually true that this Helps and not just okay, so so in the grand scheme of things Arguably, you're right. I mean in the grandest scheme of things computer science is actually a branch of psychology or Really of sociology We are trying to understand how large groups of people can combine to make things which are insanely complicated now for Would it be good if we had you know a Solid evidence that programming in this language was better than programming in that language well, yes, but It's staggeringly difficult and expensive to do honest experiments of That nature and they're scarcely ever done right you might do you know a little experimental little groups of students But it's really difficult and then some of these things which I'm saying When one is familiar with the different possible options are just blindingly obvious When there are reasons why these people are using OCaml for their system programming It's not you know, it's not homeopathy is right, but homeopathy is wrong Which is the kind of argument being put forward So question from microphone 5 please So where are you using ECC memory for your testing up here? When you say up here, there's a bit of ambiguity Thank you So where are you using ECC memory for the testing you showed about the Results of the multi-threaded Results due to memory barriers and memory reorderings Well Okay, but even if you were or you were not the point I want to make is that you can expect to have about one bit flip each minute in a modern computer system that Might completely change what your software is doing So perhaps we also have in to look in ways how we can Work if something goes wrong at the very low level so that we kind of have a check against our Specification on a more higher level of our software So it is valuable to do good engineering on the low levels But still I think we will not solve the problems of Computation and computer engineering just by proving things in the mathematical domain because computers are physical entities They have errors and so on and we really have to deal with them as well So it's it's certainly true that there are such random errors In fact, I would put the point that I would argue that you have to be able to model the physics well enough And you have to be able to model the statistics of those errors well enough So that's a different kind of mathematics and it's certainly valuable and necessary But if you look at the even the statistics that you just quoted is that the main cause Why systems go wrong? Except possibly for space hardware no So important. Yes, the most important thing that we should be paying our attention to I don't really think so Microphone 6 please Hi I really think that what you're proposing to specify and Verify more key components is it would be a valuable addition But how do you make sure that your specification actually matches the behavior that you want to do want to have? So so as I described we do as partial validation of those specifications We do a lot of testing against the implementations Against a range of existing implementations. That's one source of validation Another source of validation is that you talk to the architects and the designers You want the internal structure to match their intent you want it to be comprehensible to them Another kind of validation that we do is prove properties about them So we prove for example that you can correctly compile from C11 concurrency from a mathematical model of that to IBM power concurrency and that proof that kind of proof gives you a bit more assurance in both So none of this is giving you a total guarantee. You certainly don't claim a total guarantee, but it gives you Pretty good levels of assurance by normal standards Mike for yes, thanks again, you proposed Random and tests and with my expertise. It's very annoying if you have Tests where the outcome is sometimes Failure and you want to have reproducible tests Yeah, to fix a bug. So how do we bring these aspects to test more? More variety and data and to have that reproducible together Well if as I was mentioning for the TCP thing one of the Problem with reproducibility is exactly this internal non-determinism Right the system makes some scheduling choice or what have you and you can't see what it is Or the the checker can't see what it is So one way to do that is to really design the protocols in a different way to expose that non-determinism the other fact about this kind of general purpose specification as test oracle idea is that in some sense, it doesn't have to be reproducible the specification is Giving you a yes. No answer for an arbitrary test and that means you can use arbitrary tests Figuring out and the root causes of the differences afterwards may be tricky, but you can use them for assurance So we now got time for two more questions microphone one please Hi, thanks for a great talk so what you've described seems to be a middle ground between a Like full-on for more verification and more practical testing like in between So where do you see in the future that where for more applications will go with Converge to the middle or whether it will still be there just to verify something that's more well specified but so the the The progress of sort of the whole subject to form of verification in general if you look at that over the last I don't know 10 years or so It's been very it's been really amazing compared with what we could do In the 80s and 90s and early 2000s right so the scope of things the scale of things which are For which it is becoming feasible to do verification is that he's getting bigger and I think that will continue Yeah, I don't know when you might see a completely verified tall client let's say But that's not inconceivable now and 20 years ago that would have been completely inconceivable So that is expanding and at the same time. I hope that we'll see more of this interface Test oracle-based specification and these When you when you're doing full verification of a piece of the system you have to have these interfaces already defined So these two fit very well together So the last question for microphone to please Hi, you mentioned that you often find bugs in hardware Is there any effort to like verify the transistors on chips themselves like the whole design? So there's a whole world of hardware verification. That wasn't really my topic today and the Most of the big process of vendors have teams working on this Unsurprisingly if you remember your history many of them have teams focusing on the floating-point behavior of their processes and They're doing they're also doing By the standards of ten years ago pretty spectacularly well All right, so there are companies that have you know the whole the whole of some execution unit formally verified There's been a lot of work over the years on verification of cash protocols and such like There's a lot of work on Not verifying the hardware as a whole but verifying that you know the RTL level description of the hardware matches some Lower-level description. So there is a lot of that going on Hey, thank you very much again for this great talk. Give him another applause. Thank you Peter


Yeah, welcome back to Think Tech. I'm Jay Fidel and this series is History is Here to Help. We need to know history. We need to learn history. We need to see how history affects the present and the future. So how history explains Europe's move to the right and what historical factors are now emerging or re-emerging in Europe. And for this discussion, Tim Affichella, my co-host and Manfred Hennigson are esteemed guests who is an emeritus political science professor who served in Manoa for a long time. I'm not even going to say how many years, Manfred. Fifty years. Oh my god. All right. Thank you for that. Tim, let's start with you. The first point is, is Europe moving to the right? And what are your reasons for saying that? Yes, I believe Europe is moving to the right. They have been doing so for quite some time. And I think a big part of it is immigration and what is the underbelly of immigration, racism. You can think Brexit partially because they didn't want to accept the EU's quota on immigration numbers. And it's problematic in France because there's a large influx of Muslim population. And socially, culturally, that's clashing with the existing French culture. Germany has its issues with the same. So when you have populations intermixing in cultures that are, I won't say segregated, but have not had a high degree of immigration, there's conflicts that are bound to take place. And I think all populism, nationalism, begins with immigration issues. And that is the ultimate social wedge issue. You know, I was so happy to see Angela Merkel accept Middle Eastern migrants when they were having so much trouble in the Middle East. But you know what, what you just said and what we know today in retrospect was predictable, wasn't it? I mean, she was being very high moral bound in leading the effort in Germany to accept immigrants. And other countries were following her lead, too. It was a very noble thing. But wasn't it predictable that this would have a corrosive effect ultimately? I think so. When President Biden came into office, there was outcry about him lifting the numbers of immigrants above and beyond where Donald Trump's numbers were. And I think it goes into a fear of economic insecurities. The old adage that they're going to take your jobs away. Well, those aren't true. Those jobs aren't going to be taken by Americans in the first place. But there's a economic fear factor and it's alive and well. And that fear factor, like all fear, is a great rallying point for populism. Yeah. I don't know why, but this reminds me of Trump's comment, Manfred. Well, the comments of some of his followers about how this group of that group would not replace us. We are the white supremacists and this group will not replace us. And maybe that's what was going on in various countries in Europe when the migrants were heading north and resettling themselves all the way up to Papua Scandinavia and to Britain. And so I ask you, what is the process historically? Is this something that was predictable and historical basis? Look, I'm a little bit reticent to follow Tim's suggestion that immigration is the major issue. You have to remember that you have large non-ethnic groups in almost all Western European countries, especially the people who have colonies like France, Holland, Belgium, England. That was not the issue in Germany. In Germany, you had the guest workers coming in in the 60s from Italy, Spain, Turkey, especially Turkey. And you have now more than 3 million Turkish Muslims living in Germany. Half of them are integrated at German citizenship. The other half don't want to apply for it because they lose property rights in Turkey. So what you have is a somewhat more complicated story. It's not only the recent immigrants. I mean, Angela Merkel asked to keep German borders open in 2015. The reasons for that are not completely clear. She is personally very reticent to talk about motivations for political decisions that will come out maybe in a book in two years. But people, I mean, she often talked about her experience as living in the GDR, having no freedoms of movement, and these restrictions played a major role. And there was another one that I think played a role also. And that was this failure of the conference in 1938 that FDR had organized in order to deal with the issue of the Jewish refugees from Germany and Nazi-occupied Austria. And as you will know of the 32 countries that came to this conference, only one was willing to accept refugees, namely the Dominican Republic. All the others refused. And Angela Merkel is a person who reads a lot of history books and invited when she was chancellor, historians to give talks in the treasury. And I think these two backgrounds, motivations played a major role in her decision. Now, you have also, I think, to understand that in East Germany, you didn't have much experience of non-ethnic Germans. You had workers from Mozambique, from Angola, from Cuba, from North Korea, but they were living in camps. They were not roaming around in West Germany, where you had not only the Black GIs, but you had three and a half million of them from 1945 to 1990. But you had a lot of students from Africa, from China, and other places. So you have, in a way, an ethnic mobility in Germany long before this immigration problem, this recent immigration problem, became an issue. What's the question of degree? There were millions of Middle East migrants, millions of them, they were everywhere. Well, there was a million coming in 2015, but you have a lot of, I mean, look, you had three and a half million Turks in Germany, and you had close to 60,000 German Chinese students. You had a lot of African students. Let me ask this question. Was her decision and the increase in migrants a part of Germany, or that group in Germany that has moved right, did it make people move right to see these migrants? Would you agree that Germany is moving right? No, it's not moving right. These Reichsberger people who want to restore the Reich, it's not the Third Reich, it is the Second Reich that was founded in 1871, and there is a very funny German song you can get on YouTube, it's called Wir wollen uns ein alten Kaiser Wilhelm wiederhaben. We want to get our old Emperor Wilhelm again, but the one with a long beard, meaning it's not Kaiser Wilhelm II, the nut who came to power in 1888, but his father. It's all about nostalgia. I mean, it's a part of it, but look, the Reichsberger are also people who did not want to accept the legitimacy of the Federal Republic, and they did something that you had in the United States too in the Northwest with refusing paying taxes, and you have these anti-tax movements in a way you could say in some countries as a beginning of right-wing populism. You have it in Scandinavia and Denmark. Glystrup was a very famous anti-tax activist, and he graded then strangely enough a party that was called Progress, even though it was just the opposite of Progress, what he graded. Sounds like that's sedition, if you reject the federal government. No, absolutely. It's a seditious movement.  But I mean, look, in the United States, you have the background of the Civil War. And as I mentioned the last time in our discussion, what is sad about the history of the United States is that people do not remember that the people who were in charge of this sedition in 1861 were not indicted. They were not put before a court. They were not- It's too late for that now because Congress might get around to that. No, but I think people don't know that, you know, that Lincoln's successor, I mean, Lincoln was assassinated in April of 1965, and his successor Andrew Johnson was a Southerner, a former slave owner, and he exculpated all of these treasonous people. And Jefferson Davis, the president, was under house arrest for a few years, but he was never indicted. He should have been hanged.  And so what you have in the United States is this institutionalization of the lost cause, and we are still, in a way, suffering from this- Oh, sure. I mean, there are people around that say the Civil War is still going on. Let me shift gears for a minute, Manfred. So we saw an attempt to turn the government upside down in Germany, and what does that represent? Who is doing what and why, and what can we learn from their effort? And what can we learn from the reaction response of the public and the German government? Look, it is a deranged group led by a deranged leader, finally the 13th, Prince from Royce, whatever, all of the other family members call him a nut. And he is- I mean, that group is not a neo-Nazi group. It is a resentment group that for some strange reason feels that the West German constitutional order that came into being in 1949 is an illegitimate imposition by the Allies, the victors of World War II. And for that reason, they want to return to the Reich. Now, the Reich is not the Third Reich, as I said, but it's the Reich that collapsed in 1918, you know, when all of- when the Kaiser and all of the other monarchs in Germany- They don't have the Nazi ideology then. No, but I mean, they're attracting some of the people from the right wing also, so they have not merged.  Okay, well, let's go to others in Europe. I mean, Hungary moving right. Marine Le Pen, you know, made an interesting showing against Macron, and she'll continue to- So, Tim, tell us what it looks like in terms of these various countries, which all have indications. You know, as Manfred says, maybe it's just spurious, but it's there, and it keeps on repeating itself across borders. What's the connection, and what causes this, and what effect does it have, and where is it? That's multiple compound, but I really- I'm sorry to say this, I enjoy asking you multiple compound question. Yeah, I know that. You know, what's the common denominator? I'll kind of stick with what I said initially. I think it's someone's attempt, and I'll call them a wingnut, be it Donald Trump or this prince the 13th. I mean, the common denominator is preservation of a culture, and the audacity of a world leader or my country's leader to somehow diminish or blemish the culture that we're accustomed to. By the way, for the record, the best thing that ever happened to England was either through colonialism or just immigration over the last decades. Their food has never been better. That is true. I mean, that was- Thank you, Manfred. You measure society by its food, eh? Yeah, absolutely. Food, art, you know, what they drink. The empire. Yeah. The bad food.  I'll call it cultural preservation. Let's call it populism. And I think that is the common denominator. That's the entryway. That's the camel's nose under the tent, and it goes from there. Then it just becomes an under-adulterated quest for power, which ultimately leads, I think, to fascism. Yeah, but populism is dangerous. Is populism good for Europe? No. It's not my opinion. No. But you see, the Italian case, there is a different dimension also. They have never really processed their history. I mean, in 1943, Mussolini becomes removed from office. It was just the reaction of other military leaders to the successful landing of the Allies in Sicily. And then the German hijack, the hijack Mussolini, and put him up in office in northern Italy, where he was then, in the end, captured by the partisans with his mistress, sentenced to death and hung by their feet in Milan. It was really an extraordinary picture. Never escaped in a red, sports car. But that also, no. But you see, the interesting thing about Italy is that you do not have really the same kind of active confrontation of the record as you have had in Germany in the last, since the late 70s, or the mid 70s. So for that reason, I always surprised, you know, encounter fascist statues in Italy. And now, you know, the Prime Minister, she is self-declared neo-fascist, you know. So, when you go to Scandinavia, it's different. You know, in Scandinavia, you have this nostalgic notion of a past, you know, that they do not want to let go. They want not to have the Euro. Only Finland has it, but Sweden and Denmark, even though they are part of the European Union, they don't want to have it. Nor we didn't join. They want to keep their monarchies. You know, even though you have these tensions between these three countries, because Sweden was too close to Nazi Germany, the king was a close friend of Hitler. Whereas, you know, the Norwegian king left. And the Danish king, you know, was obviously not supportive of the Germans when they occupied it in April of 1940. But you have within, you know, Scandinavia, these tensions between the Norwegians and the Danes on one side and the Swedes on the others. You know, they have not forgiven the Swedes for their collaborationist mentality during World War II. The Finns are different, you know, they were fighting on the German side. So there are all kinds of complicated historical issues, you know, in Scandinavia when it comes to that. But what combines them now is this unwillingness to replace their ethnic identity through letting millions of people come in. In Germany, you have now 25, 20 percent of people having a migrant background. Now, it's not always migrants from Africa or some of the Middle Eastern countries. They also come from within the European Union. So you have a lot of, you know, Bulgarians, Romanians, Poles, Hungarians, Czechs, and, you know, over a million Russians. Yeah, I wanted to talk about that. You know, so Ukraine has a fair number of migrants, millions coming, a lot of them from Poland, but some of them to other countries, you know, in Eastern Europe and moving to the West. And so we have a mass migration again. At the same time, we haven't stopped the migration from the Middle East. That continues. But, you know, what I'm getting is now, you got it coming from both sides. If you weren't irritated with the migrants coming from the south, you can be irritated. Some people, I'm sure, will be irritated by the migrants coming from the east. So the whole thing seems to be in flux. And my question to Tim, to both of you, is how does that affect the solidarity of the EU? So one thing we know that people, you know, react to migrants by trying to return to their ethnic history. But the other thing is we need the coalition now in order to, you know, in order to support Ukraine in a war that disturbs the global liberal order, in a war that, you know, undermines the notion of sovereignty. So what effect does all that have on their ability to stay with the coalition? Well, the effect, obviously, I think maybe I'm wrong on this opinion, but it was an underlying reason why Brexit vote went the way it did. And that is, again, the concern of taking on immigration quotas from the EU that Britons felt and they were stoked. But Britons felt that their society and their cultural existence was being diminished.  You'll hear the term white replacement. These are very, very emotional issues that people don't take for granted. They're embedded in the human psyche. And when you start threatening people and their cultural existence, be it true or not, and most of it's not true, but the right leader, the right demagogue, can persuade them that that is the issue of the day. You agree with that, Manfred? No. Well, yes, but no. Thank you. We've had that before. One of the most fascinating export articles of Denmark told the United Kingdom and the United States, my brother who's a Scandinavian specialist in Germany always tells me, and it's not a joke, it's male semen. They all want to follow up on Trump's comment, why can't we get these Norwegians? And so the semen export is really a booming industry in Denmark. That's not a joke. I don't think it was the Norwegian aspect. It was the white aspect of that. Yes, but that's no. Yes, the white aspect, but was especially pronounced by Trump's remembrance of how Norwegians look. But that's not the issue. I think Brexit, for example, you always emphasize the quota issue. But you see, England had more than two million workers from within the European Union. Because of the Schengen Agreement, they could move there. Now they are lost, and England suddenly realizes, and I say England because it's not a Scottish or Welsh and a Northern Irish problem because they did not vote for the Brexit. Well, at least Northern Ireland and Scotland didn't. They wanted to stay within the European Union. Now what you are facing in Great Britain today as a result of the Brexit is an economic collapse. And they do not know how to manage that. Now that was predictable. But when you are looking, you also have to remember, and Jay is always talking about contemporary immigration. You have to remember that all of these colonial powers, after they abandoned or their colonies became independent, a lot of collaborating people moved to the metropolitan societies.  You had it in suddenly in Great Britain. You never had it in Germany because Germany was not long enough a colonial power. It ended in World War I. So what you had in Germany, were these guest workers who came in the 60s from European countries initially and then Turkey, and now Germany needs every year 500,000 immigrants in order to keep their level of productivity. Now, ordinary Germans are not aware of that, or I don't know whether they understand that, but people in power know that. The German economy will simply collapse if you do not have this input of foreign... Let me just ask this though. If you have a downturn in the economy, that kind of validates the concerns of the people who think that the migrants are a negative feature. It validates their fear that they will lose their jobs, they will lose their quality of life, their middle class status, whatever it is. And if we have a downturn, for example, in Britain, am I right? Britain is going to move right. So when you have instability in the economy, you have a turn to the right. Isn't that predictable? Yes and no. But remember, the English Prime Minister now is not a white man. He is a guy with Indian background. I'm talking about the economy, Manfred. If the economy goes down, people tend to track that back to the migrant issue. Margaret Thatcher got in that way. Yes. But the interesting thing in England is that despite this downturn, these problems as a result of Brexit, you have now, you could say, a migrant prime minister. So the contradictions are quite amazing. And they are also fascinating when you're watching the Harry and Meghan story. I mean, the unwillingness of the Royal House and the Boulevard press. And I would say a lot of people in the society as a whole, they didn't want to accept Meghan, you know, this mixed race American woman because of racism. I mean, you have not only classism in England, but you have racism as well.  And it's not simply a kitschy performance. No, it's a very powerful political story and the failure of, you know, the Royal House, you know, after the disaster with Diana, to have the second controversy hit them, you know, this is not good. No, it's not good. It's not good. Let me shift to one other area that we have talked about so often. And that is the connection that that right wing movements use autocrats or would be autocrats use in order to advance their interests. And that is technology. It is social media. And let me add that, you know, Vladimir Putin is using that actively. There was an article a couple of days ago about how Vladimir Putin is trying to divide the German group, the German people on the issue of whether they should support Ukraine. And his argument is a subterfuge argument that he uses social media to advance is that the Ukrainian refugees in Germany are having this negative effect on Germany, on, you know, its social and economic status, and therefore the German people should oppose supporting the war. And the article was in The Times. But look, he had done that before. He does that all the time. That's what he does. But, you know, the thing is that when we find it would appear to be a coincidence about movements around Europe that go to the right, we find also that those movements are supported, if not fomented, by social media, which come from somewhere else or which blanket all of Europe. And so if you say as we suspect that Europe is moving to the right, then social media and the internet have to be part of that. Tim, your thoughts. I couldn't agree more. I mean, social media is the vehicle. But what's behind the vehicle? What's in the vehicle? And that is propaganda. Propaganda is the dark art of rhetoric. Sorry, but Donald Trump, if you didn't watch him from day one, he's using multiple propaganda techniques that are quite effective. And the problem with the United States is, and the media specifically, they weren't well armed to understand and recognize propaganda as it was occurring in front of their very eyes. And so moving forward, I would appreciate the media to point out propaganda techniques that are being employed right then and there in front of the population. Well, they did that in Germany. It came out in the newspaper and, you know, and Vladimir Putin was blamed for it. Manfred, what are your thoughts about this? You have a specific problem in Germany because you have a lot of Russian, Russians living there.  So you have the Jewish population in Germany being around 250,000 at this point, many of them coming from Russia, and they are not joining the traditional Jewish communities in Germany. But you have then in addition to that, 1.5 million German Russians, I mean, Germans who were invited by Catherine the Great in the 18th century to settle in Russia, they have come to stay in Germany, and they are very often a target of Putin's propaganda. And sometimes, you know, it works. There was a story of, you know, that they used in order to incite all kinds of unrest that a young 15 or 13 year old girl was raped by a group of Middle Eastern immigrants. And that story really made headlines in Russia, a lover of the Russian foreign minister, you know, used it. And then it turned out that was not happening. What was happening is that this young woman, you know, had Middle Eastern friends and talked with them. But there was no rape involved in the story. But it had become an independent story and worked, you know, emotionally, very well for Putin for some time until they realized he was lying. So I... Well, that's what he does. He does it in Europe, he does it everywhere, does it in the United States for elections. Right. Well, remember propaganda is the creation of a whisper of the truth that's been concocted into a big lie. So there's always a whisper of something that was truthful at the moment. Maybe the characters were the truth, but nothing else other than that. You asked about the European Union before. And I think one very interesting, I mean, one bad thing that happened was this corruption scandal with the vice president of the European Parliament. You know, her father was seen with a suitcase leaving a hotel with, I don't know, 500,000 euros. But the other thing that I... The positive thing about the European Union at this point is that they managed to stop giving Orban money because, you know, and they had the majority for that, which is quite extraordinary. And on the other hand, Orban did not stop these plans for the reconstruction of the Ukrainian infrastructure. You know, the European Parliament passed a resolution that they will do that. And Hungary did not veto that, and Poland didn't veto that either. So those are positive signs that indicate to me that whatever the scandal, you know, of the Greek second deputy leader of the Parliament may do to the stability of the European Union, it's the Oslo von der Leyen, you know, the president is still, I think, doing a good job. I wanted to ask you about that, Manfred, because, you know, the two are, they affect each other, the stability of Europe, which we've been talking about, and the stability of the United States, which we haven't talked about. And so the question I put to you, and it's the last question because we're running out of time, the question I put to you is, what kind of policy should the United States have in terms of affecting this of, you know, somehow encouraging, somehow, somehow preventing, preserving, preventing the move to the right in Europe, and preserving democracy such as it exists. What can we do? You said that Biden doing a good job. What else can he do? What can his successor do? Well, Biden is, I think, doing a good job by supporting the European Union. It's political goals. The European Union as such, Trump was against it. You know, he didn't want the European Union to succeed. He wanted to promote nationalism. And I mean, he went to... He was successful in the sense that he tended to destabilize the EU, didn't he? Yeah, that would work. And now it's a matter of recovering from that. It didn't work.  I'm sorry, that's the acronym for the ruling party in Poland. He supported both of them because he felt, you know, these are the two countries that almost troubles him for the stability and unity of the European Union. And so he exploited that. But I do not think he has succeeded. I think on the contrary, whatever Trump does when it comes to Europe will unite the European Union. And for that reason, you know, I do not think that the European Union is in danger. I mean, there are troubles, but it's not in danger of falling apart. So what can we do to preserve it? What can we do to, may I say, reverse the move to the right? By getting rid of the right being threatened in this country. Well, what a perfect answer. Okay, we're almost out of time. Tim, why don't you take the anchor on this? I agree with Manfred wholeheartedly on his comment here. What can we do? Try to educate all citizens that immigration isn't a threat to their sovereignty, if you will, to their cultural identity, that a lot of positive things do take place when we get different society and cultures enmeshed into our society, our current society. So the greatest thing you could do is start educating on the values of different peoples and population and helping them out in really dire situations. Yeah, it's about the United States as the city on the hill, the beacon. And if we could be more pure about that, then we could be a global leader on these issues. Well, thank you so much, Manfred. Great to have you on the show. Manfred Heddington and Tim Apichella, thank you both. Aloha. Thank you. Aloha. Facebook, Instagram, and LinkedIn, and donate to us at thinktecawaii.com. Mahalo.