Alright so once again thank you all for coming out um let's give Anish a big round of applause and welcome him to the stage. Thanks for the introduction Jurist. Uh today I'm going to talk to you about how we can build more secure systems by applying the age old wisdom of turning it off and on again and how we apply this to the design of a hardware cryptocurrency wallet. Quick show of hands uh how many of you own one of these security devices, these security dongles, things like UB keys or hardware cryptocurrency wallets?  Uh so we believe in the value of these uh external devices to augment our PC in order to increase security and it's not like all these devices are created equal, they're actually getting better over time. Some are maybe not the best idea like SMS based two factor auth is kind of a mess and uh I don't know about you but at least for me it doesn't really work at DEF CON because my phone's been on airplane mode since Thursday. Uh but uh other devices do a pretty good job of preventing certain kinds of attacks. For example U2F tokens have more or less solved phishing and that's pretty cool. Um but here this is not just devices becoming kind of better and better decreasing the size of the TCB in a straightforward way, it's not just a continuous improvement of two factor authentication devices, there's something a little bit more interesting going on here. I think there's been a paradigm shift. On one hand we have two factor auth devices which are all about doing more secure login on your PC. This does things like protect from stolen passwords or against phishing attacks. But on the other hand we have this new type of device uh a hardware cryptocurrency wallet or at a higher level uh or more general sense a external transaction approval device. And this device removes the PC from the trusted competing base entirely. There's scenarios where two factor auth won't save you but transaction authorization will, namely if your PC is compromised. And uh these may seem like they're solving different problems because two factor auth is traditionally associated with websites whereas hardware wallets are obviously associated with cryptocurrencies. But this idea of factoring out the most sensitive approval decision onto a separate secure device, well this idea is universal and it could be used by websites too. And so this is a way to improve the security of a large class of applications. Uh now you might ask can we just make the PC secure instead? Well uh the PC is kind of a disaster, everything's terrible. It's it's just this tower of complex technologies where every one of them has major security flaws. And while we have mitigations and workarounds for a lot of these uh it's just this cat and mouse game. New attacks come out then developers patch those exploits and then this cycle repeats. And we have no strong security in any principled sense. We have applications that are millions of lines of javascript code uh using who knows what dependencies from NPM. Probably running inside an electron container. And this is running on top of a complex operating system that's at least tens of millions of lines of code probably filled with bugs. Uh and this operating system is running on a CPU that's a leaky abstraction for the abstract machine it's implementing. And so our CPU's are susceptible to all sorts of things including these micro architectural bugs that have been getting a lot of attention recently. Uh things like Spectre and Meltdown and Four Shadow and Zombie Load. And then on top of that we have hardware that's also a leaky abstraction because physics is kind of weird. And so we have attacks like Rohammer and Rambleed where you can do things like repeatedly right to a row in DRAM and cause bit flips in adjacent regions and this breaks isolation. And so everything's a disaster it's too hard to build secure applications on the modern PC. But if we can't make the PC secure maybe that's okay. Maybe we can use this transaction approval paradigm instead where critical approval decisions and applications are factored out and go through a separate secure device. And we can use this as a way to bootstrap security. And these hardware devices as opposed to our PCs just might be simple enough that we can get them right and have strong confidence that we've gotten them right. Maybe with these devices it's possible to move beyond the just it hasn't been hacked yet and towards having a more principled reason for believing that such a device is secure. And so for those of you who are not super familiar with these well how do these devices remove the PC from the trusted computing base? Well it's really natural for applications like cryptocurrencies because they're already structured to require private key signature in order to do the sensitive operation. And so when using a hardware cryptocurrency wallet the private key never leaves the hardware device. And so the way it works is you can pair it with an untrusted PC and if you want to do anything like send a transaction what you can do is craft that transaction on the PC. You can say like oh I want to send 0.1 Bitcoin to address XYZ but you can't actually sign and broadcast the transaction because the PC does not have the private key. Instead all you can do is send the transaction from the computer to this hardware device which can then parse the transaction, display it to the human in a human readable way like 0.1 Bitcoin to address XYZ and then only if the human confirms the transaction on the device by pushing the button does this device sign it with the private key and send it back to the computer so it can be broadcast. And so if something goes wrong here like if the PC is compromised it might do something evil right? It might do something like swap out the recipient address without the attacker or it might change the amount that's being sent. And if there's malware running on the PC it could do all sorts of wacky stuff like it could even fabricate the contents of the screen so it could display on the screen like oh you're sending a Bitcoin to your friend when in fact what the computer is actually doing is sending all your money to the attacker. But even if the PC is compromised the hardware wallet user is safe because even if the PC's screen lies to the user the device will show the actual recipient address. And so we can think of these devices providing a secure IO path to the user where the PC was previously failing us. And it's good to remove the PC from the trusted competing base because things indeed go wrong. Like there are tons of attacks I could talk about where people still cryptocurrency. Here's this one that I think is kind of clever. In December 2018 there was a phishing attack on users of the Electrum Bitcoin wallet. And so the way the Electrum Bitcoin wallet works is that it's an SPV wallet. And that's like most other Bitcoin wallets people use today. These wallets don't download the full blockchain onto your computer because that would be kind of impractical like having to download hundreds of gigabytes. Instead of what these wallets do is that they rely on untrusted servers in order to do payment verification. So like you rely on this untrusted server and it can prove to you that a particular payment is on the Bitcoin blockchain without you having to download the blockchain and also without you having to trust the server. And this is pretty cool. In Electrum's implementation these untrusted servers could return error messages to clients. And that's also fine. But in Electrum versions before 3.3.3 these server error messages could be arbitrary text. And not just arbitrary text actually but HTML. And so the screenshot here shows what happens if you take an old version of Electrum and connect to one of these compromised servers which anybody could set up. So it's like hey, require security update, version 4.0.0. Go download the software here. Nothing bad will happen I promise. And by getting people to run bad software on their PC attackers made off with hundreds of Bitcoin. And this is just one example. There's tons of examples of this kind of thing out there. If you have bad software running on your PC it can steal your private key. Even if your private key is encrypted on your device the malware can just wait until you decrypt it when you're about to send a legitimate transaction and then once it has a private key it can do whatever it wants. And in this scenario and most of the others a hardware wallet could have saved you. So why shouldn't every application behave like cryptocurrencies in the sense that these sensitive approval decisions are factored out and go through a separate secure device? Well right now for sensitive websites I use two-factor auth. And if my computers hack I'm in trouble. Two-factor auth doesn't guarantee anything if your computer is compromised. And now these websites already have confirmation prompts for doing sensitive operations. Like if I look at my open stack if I try to delete a VM it's like oh confirm terminate instance are you sure you want to do this? Or if I'm using my I'm doing something with my domain name and trying to delete a DNS record it's like oh are you sure you actually want to remove this DNS record? And these are not for security. These screenshots you see on the left those are just like confirmation prompts to make sure you don't click on the wrong thing and accidentally do something you didn't mean to do. But in theory this approval decision could be easily factored out. At signup time a user gives a website their public key and then for sensitive operations the user must sign a human readable description of the operation with their private key. And the signing could happen on an external device on a hardware cryptocurrency wallet just like you do with cryptocurrencies. And this could be really useful for things like bank transfers and stock market trades and manipulating VMs and infrastructure and manipulating DNS records and deleting backups. Like basically any sort of sensitive thing could go through such a device. And there's actually some work on standardizing this kind of behavior. So web authentication is this web standard that's kind of like a successor to U2F. And as it says on the box it's an API for accessing public key credentials. And a large part of the standard focuses on doing better login like basically doing better U2F. But a small part talks about doing transaction authorization on an authentication device. So I think that's pretty cool. But this is pretty recent stuff and even though some people have started supporting web authentication as a second factor for login. As far as I know nobody's really using this transaction authorization stuff. But I hope it takes off in the future. So now that we're convinced that transaction approval is a great idea. Like this transaction approval of sensitive transactions happening on a separate secure device. Well of course we'd want to use it everywhere. With all our cryptocurrencies like Bitcoin and Ethereum and Monero and all the sketchy altcoins I didn't bother to list here. And even websites where we do sensitive stuff like where we manage our virtual machines and our DNS.  But if we do that we lease some of the simplicity of other security tokens. Things like the UB key or U2F keys. Where those were basically fixed function. Here web authentication support maybe could be built into the device. But supporting arbitrary cryptocurrencies would for sure require running different applications. Because these different cryptocurrencies all use different transaction formats and different crypto systems and so on. And so now this device that we're imagining needs to run arbitrary third party code. While sand boxing all the applications from each other. Like I really want to be able to install a sketchy altcoin wallet without having my Bitcoin stolen. Is that too much to ask for? And so this device must provide strong isolation between these mutually untrusting applications. And now our device is starting to look complicated. Earlier I was saying oh the PC is this complex beast and it's going to be impossible to get it right. And the hardware wallet. Well that can be simple and therefore maybe secure. But now the hardware wallet is starting to sound kind of complicated too. Now it needs an operating system. It needs to multiplex shared resources like the CPU and RAM and USB and storage and display and buttons between applications. And sharing leads to complexity. And complexity leads to bugs and vulnerabilities. And hardware wallets have indeed had bugs in the past related to their operating system. For example the ledger wallet. One of the most popular cryptocurrency wallets had a series of bugs related to bad argument validation and system calls. So one example there is that there was a SHA-256 system call. And you could pass a pointer to it and it should do some amount of validation on that pointer so you don't read memory you're not supposed to. But turns out you could pass the null pointer and it would gladly compute a SHA over that. So you could call SHA-256 with the null pointer and varying length arguments and get hashes of prefixes of memory so you could dump contents of flash. Like you could call SHA-256 n times. And then compare that with independently computing 256 times n hashes and you could use this to dump the first n bytes of memory. And in practice this could be used to dump the first 8K of flash. Isolation bug. And there are other bugs in this style in the ledger wallet. There was some interesting security research done by Riskyr and presented at Black Cat in 2018 that talks more about this. And both ledger and trezor two of the most popular cryptocurrency wallets have had bugs where the arm memory protection unit in the device was misconfigured by the operating system. So in ledger's case it allowed read accesses to memory regions which applications are not supposed to read. And in trezor's case it allowed write accesses. And so at a high level uh these bugs are because of sharing. Cause if you think about it, if you had no sharing you wouldn't need a sophisticated operating system with a complicated memory protection policy. You wouldn't need the memory protection unit and so on. Okay now maybe I'm sending you mixed messages. Uh I seem to be hinting at security through physical separation. Like this is an easy solution in some sense. Uh one device per app. In this picture mutually untrusting applications are just air gap from each other. And physical separation gets you a lot of nice properties. So why don't we do this? Well it's impractical. Especially to support a lot of applications. And practicality is super important for adoption. And so there are these competing tensions here. It's like sharing versus security. And the question is can we have the best of both worlds? Well here's an idea. Uh what happens if we try to simulate physical separation? And I'm going to give you a very high level overview of it and then we'll talk about it in more detail. But what if we could have this device where we run an application and whenever we want to switch applications we just blow away all the state on the device. And in a sense logically have like a fresh factory new device in order to run the next application on. Well for whatever vague description of fresh device uh you might have in mind. Like if this is done correctly maybe every application that runs could be isolated from all the others. Cause if you kind of clear everything on the device away before running the next application. Well whatever rend before can't accidentally leak secrets to whatever ran afterward. And so maybe this sounds reasonable at a high level. Like oh let's simulate physical separation through reset. But uh obviously there's some details that need to be worked out here. It's like what what would the hardware architecture look like for something that's built around this idea. And then also what does it mean to reset hardware? Well here's a proposed design. What if we run third party code kind of on its own computer? A stateless computer. And we reset it between task switching. So like on the left or uh yeah on the left we have this thing we call the application core. It's basically a computer and it's stateless. We kind of load an application on it just as we want to run it. We run it to completion. And then before we run the next application we somehow magically clear all the state from this application core. And in this picture we have uh another computer basically for managing the computer on the left. Uh and the computer on the left is stateless but runs third party code. And the computer on the right is stateful. It can write stuff to flash memory but it never runs third party code. And so this has some other nice properties. Uh you actually don't need an operating system with a structure like this basically. Or maybe you can think of it as splitting a kernel and a user up in this way. Like kernel and user uh uh space rather than being like hardware protection on a single CPU is just two separate computers talking to each other. But in this picture whatever code runs on this like application computer we have on the left. Well it doesn't need an operating system. We can just give it access to bare hardware. Cause operating systems are all about kind of mediating access to shared hardware. But in our picture we have this idea of resetting all the state before letting the next guy run. And so we don't need to really worry about what kinds of wacky things some malicious third party code might do. As long as we can get reset correct. And so we need no operating system or hardware protection mechanisms. And so that gives us a lot of simplicity. And so it's easier to reason about the correctness of such a system. And then in this picture we also don't really need things like system calls. Like basically we need no communication back from potentially sketchy third party application code back to whatever management code is managing this device. This communication can happen over a really simple interface like a single wire UART. And maybe like the only system call you need is for the application to be able to terminate itself and maybe save some state for the next time it runs. And maybe this seems like a really wacky design. And this wouldn't work for things like your desktop. Where you want to run multiple applications simultaneously. Or you want to run sophisticated applications. But for the class of applications we're targeting this restricted model does work. And so this is what the architecture looks like. Now what do I mean by reset? We still need to talk about reset in more detail. So what state is there in this application core? Well there's state in many places. Obviously the CPU and RAM are stateful devices. But also even things like the USB peripheral or the display can maintain state. And in order to have strong isolation we really need to clear all the state that may be present in such a device. Like basically everything is stateful except for things like the buttons which are the switches. And to have strong isolation these need to be reset in some meaningful way. Clearing away whatever is there before. So that any application that runs is not influenced by whatever was running on the device before. And so again what do we mean by reset? Well we'll actually formalize this later. But for now you can just think of it as like factory new or untouched or untainted. Like get everything back into an untainted state. Any to pre-reset states should be indistinguishable post-reset. Well so how do we implement this? For this talk we'll focus on the CPU because it's the most interesting and challenging component and all the other things are kind of analogous. So first attempt. Turn it off and on again. What happens if we just take the CPU and cut the power to it and then plug it back in? Is this sufficient for clearing all internal state? It might seem like a good idea. Well it turns out that removing power from these chips doesn't actually clear all the internal state. Chip states such as SRAM can actually persist for tens of minutes in these devices. And so power cycling isn't really going to achieve anything. At least not on a reasonable time horizon. And in a sense we can think of power cycling as roughly equivalent to just asserting the reset line. Speaking of which, these processors have a reset line. Like there's a pin on the CPU labeled reset. Does this do the right thing? Well to answer that question, to understand what this reset line really means, well the first place we might look is the ISA manual. And so here I have a screenshot from the RISC 5 instruction set manual. Other instruction sets say things like SRAM are kind of similar. RISC 5 had a really clean description of this so that's why I chose this one. And looking at this we can see that the reset specification of RISC 5 is really weak. All that the manual really tells us is that after reset the program counter is set to an implementation defined reset vector. And all other state is undefined. And this is kind of like undefined behavior in C or something like that where something specific does happen. It's just that the standard doesn't guarantee that any particular thing will happen. So what happens in practice? Well all state isn't reset. Like for example registers may retain their pre-reset value. And so just asserting the reset line of a CPU isn't really enough to achieve the kind of stronger notion of reset. Maybe we'll call that deterministic reset that we were thinking about. So okay here's another attempt. What happens if we run code post reset? We might be inspired by the part of the manual that said the program counter is set to an implementation defined reset value. That means that once we reset a CPU we can return control to a known location. And maybe there we can have a read-only region of memory where we have some code that cleans up the rest of the state in the CPU. Like reset guarantees that the program counter will go there and we can run the special code that does things like clearing the registers of the CPU. So what needs to be cleared? Well maybe we can read carefully through the ISA manual and identify all the registers and stuff internal to the CPU. Things like the flags registers and all that. And if we think about it really carefully maybe we can write code that clears it. Well it turns out that that still isn't enough. There's actually more state inside a CPU beyond what is even named in the ISA manual. Because at the abstract machine level there's no notion of a lot of the details of what's inside a CPU. Like in the ISA there's no notion of a branch predictor. There's no notion of caches and things like that. These are all implementation details designed to make a processor go faster. But oftentimes these details end up causing problems. As we've seen in micro architectural side channel attacks in things like Spectre and Meltdown and so on. Because this micro architectural state which can't be named at the architectural level. You can't write code that directly talks about it. Well this micro architectural state can end up leaking into architectural state which is exactly what's going on in attacks like Spectre and Meltdown. For example through timing. And this is problematic. So are we stuck? Because code can only talk about architectural state.  Those just don't exist at the architectural level. And also every CPU's micro architectural details are going to be different. So seems like it'll be hard to support many CPUs. Well I think one step we could take is just minimize complexity. Like in our relentless pursuit of simplicity we should just use a processor that doesn't have all these fancy features. We're not trying to do anything super complex. Like we're not running Fortnite on this thing. We're just signing like small blobs of data and displaying some small text on the screen. And so we don't need a fancy processor. We can just use something that doesn't speculate and has no branch predictor and stuff like that to minimize micro architectural state. Like if there's no branch predictor we don't need to worry about clearing it, right? But even the simplest of CPUs is going to have some micro architectural state. Some implementation details that are not named at the ISA level. But there's still state inside the CPU. And it's really hard to reason about whether or not this could be problematic. So we better figure out how to clear it. And so here's our final idea. It actually turns out that it's possible to write code that resets a CPU. Like using this idea of asserting the reset line to return control to a particular point we control and then letting the CPU run for several cycles to run some code that we wrote. Now I might say, hey wait a minute. I thought you said that code can't talk about micro architectural state. And yeah that's true. But running code on a CPU affects micro architectural state. It's not described in the ISA manual how it does so because it's specific to a CPU. But for any given CPU, any particular CPU, the implementation is deterministic. It like does some particular thing even if it's not defined by the ISA manual. And so what if we wrote code to very carefully have the desired effect on a specific CPU. Like for example you might imagine if a CPU is a pipeline CPU and you just execute a bunch of instructions they'll end up kind of clearing away a lot of the state that was in the pipeline. And some CPUs are open source. So we can actually consult the gate level implementation of the CPU and understand the effects of code at the micro architectural level. Now what if we got it wrong well then we just have no more isolation. Like this is our kind of tool we're using to achieve isolation. And so it's really important that we get it right. But this process seems very challenging and error prone. Like this code itself seems hard to write. Like how do you write code that has the right effect at the micro architectural level. And also reasoning about the correctness of such code also seems complicated. Like how do you know that this code does the right thing no matter what state the CPU is in. Like the CPU can be in so many different states. How do you know that the code always does the right thing. Well so how do we know that reset is correct. Well what is reset trying to do. Reset is a single operation of this like asserting the reset line and then letting the CPU run code for a little bit that's applied to an arbitrary state and and it should turn the CPU from this arbitrary state into a purge state. Like basically no matter what state you started and before you should end up in the same state after this purge operation. And maybe another way of looking at it is like we have these two worlds one where a secret bit is zero and another where a secret bit is one. Like whatever software is running before is operating on some secret data. If these worlds are indistinguishable post reset then code that was running beforehand on that secret data can't accidentally leak the secret to whatever ran on the CPU after this reset process. And so this is a slightly more formal definition of our reset property. But how do we apply it. It seems really hard to just like sit down and think really hard about whether this property holds for a particular CPU and particular sequence of code. Like just we can't sit down and like think about all of this. So what do we do. Well we can use tools to do this reasoning for us. So a little bit of background. There's this really neat tool called an SMT solver. SMT stands for Satisfiability Modular Theories and it's this very powerful tool that has many uses including security. Like these tools are used for things like crack mese and bug finding and static analysis and things like that. I highly recommend you look into these after the talk. Just like a super neat tool. And SMT solvers are based on this more primitive thing called a SAT solver. So what is a SAT solver. It's a it's a tool that can solve Boolean formulas like for example in the top left you might have the formula X and Y or not Z. And it turns out that this formula is satisfiable. So if you give this to the tool it will say oh this formula is SAT. And the satisfying assignment is X equals false Y equals false Z equals false. So if you plug those things in on the left and evaluate it you'll see that that expression evaluates to true. And similarly you might have a different expression which is not satisfiable in which case the tool will return unsat. So then what are SMT solvers? SMT is SAT on steroids. Basically it supports more data types than just Booleans and more operations than just Boolean operations. So you might have support for things like integers or addition or inequalities or bit vectors or things like that. And different tools have support for different sets of things like you might have a theory of strings or a theory of arrays or things like that. But basically it's SAT on steroids. It's like a fancy equation solver. But it's not just an equation solver. Because when SAT solvers find a solution, well that's like an equation solver. But when they return unsat it actually it means something. It's not just like the tool is failing to find a solution. The tool has determined that there is no solution. You can think of it like it's doing a smart exhaustive search and proving that whatever formula was input into it has no solution. And so it turns out that we can use this fancy equation solver to automatically prove theorems. So consider this abstract way of stating a theorem, for all x p of x. Well we can mechanically translate this to an SMT formula. And the way we do that is we just strip the for all from the left hand side and negate the proposition. So for all x p of x turns into not p of x. And then we can consider what happens when we feed this to the SAT solver. If this theorem is satisfiable, that means that the theorem is false. Because if not p of x has a solution, well then that means that there is an x where not p of x is true. And so it can't be the case that for all x p of x. Like that's a counter example. On the other hand the SAT solver might say unsat. And that means that our theorem is proven. It's a proof that our theorem holds because if not p of x is not satisfiable, there's no x that makes not p of x true. And so it must be the case that p of x holds for all x. So in more concrete terms like how do we actually apply this to a real theorem. Well here's a simple theorem. This theorem says that the average of two real numbers is between their min and their max. And so the way we translate this to SMT remember is we just strip the for alls and negate the proposition. And even the like actual code for checking this theorem is actually pretty straightforward. So this is Python code using the Z3 theorem prover. And basically it's like oh x is a real number, y is a real number, let me define min and max because those are not built-ins. And then put in the theorem statement, assert that not of the theorem statement holds and check if that's satisfiable. And then of course like we know this theorem is true and indeed the tool returns unsat here, proving the theorem. Okay so going back to our reset property like the whole thing we were talking about earlier, here's the picture I had on the slide before but phrased this text. What does reset mean? It means that if you're in any two different CPU states, if you apply this operation to those states they must converge to the same state and here purges like this process of asserting the reset line of the CPU and then just letting it execute for some number of cycles so it executes whatever instructions we put in the right place. Okay so as long as we can express this in the language of SMT then we're good. We can just throw this at our SMT solver and hopefully it can prove it correct. So so far I've only talked about Boolean formulas and like numerical equations and stuff. How does that translate to CPUs? Well let's start with simple circuits. Let's start with combinatorial circuits. So these are stateless circuits that compute Boolean functions and here is a full adder circuit. So this takes two inputs in a carry and produces an output in a carry out. Pretty simple component, something like this might make up a very small part of a CPU. Like this might be chained together with other full adders to make a wide adder like a 32 bit adder and that might be a small part of an ALU and that might be a small part of a CPU. But we can see how this might be representative of a small part of a complex circuit. So the circuit could be represented in verilog code like this. It's like a pretty straightforward translation of exactly what you see in the picture. And it turns out that the Python Z3 code for this is actually pretty straightforward. Like this tool supports doing Boolean operations so we can just translate this code into Python and use that library. So this is all you need in order to represent that circuit. So let's try to prove some property about the simple circuit. We all know that addition is commutative, A plus B equals B plus A. And since this is a full adder and takes a carry in, let's just say that the carry in has to be the same. Well, what does this look like when we try to use the tool to formally prove this property? Well, this is all the code you need. Basically you put in the preconditions. We have two possible inputs. We have the A plus B input to the adder and the B plus A input to the adder. And we say that, okay, like the A and B are swapped in the two cases, but the carry in stays the same.  Like remember from earlier we stripped the forals and negated the proposition. So here we say to the solver, like I am asserting that the result of the full adder applied to the two possible inputs, the one where the inputs are normal and swapped. Well, the result is different. And if you try to check that property with the solver, it'll say that this equation's unsatisfiable, which is a proof that our theorem is true. That addition is indeed commutative as we all knew. And then I think the last piece of the puzzle here is how do we deal with state? Because CPUs are stateful things. Uh, well, let's, let's not go all the way to the level of CPU yet because that's kind of complicated. Let's start with this eight bit counter. So here's verilog code implementing an eight bit counter. It's pretty straightforward. It has an eight bit register inside. And then what it does is at every clock edge, if the reset line is asserted, it'll set its internal state to zero. Otherwise if the enabled line is asserted, it'll increment the count by one. Otherwise it'll do nothing. This is like a simple example of some stateful circuit. Uh, verilog code on the top left, graphical representation on the bottom right. Well, here's a translation of that circuit to, uh, SMT. It's a little bit different than before because I'm using slightly more fancy features of, uh, Z3 here. I'm using data types to represent kind of the different parts of the counter. Um, but it's still pretty straightforward code. And, uh, the thing of most interest here is the thing on the bottom left, what I've labeled the transition relation. And this describes the behavior of the counter. And what this is saying, saying is that if we have two states, S1 and S2, if we assert that counter T of S1 and S2 holds, that means that state S1 steps to state S2 after one cycle of execution. And this is a pretty straightforward translation of the verilog code. Like we could have done this mechanically. Just in this example, I did it by hand so it'd be cleaner to look at. So let's try to prove something about this. Here's another proof. Uh, we might want a property like, oh, the counter's count doesn't decrease over its execution. Um, and we might have a precondition to that theorem, like as long as it's not reset, the counter doesn't decrease. Well, here's a little bit of code. Let's instantiate two states, S1 and S2, and then say, okay, here's our precondition. Like asserting this counter T thing says that S1 represents the state of the counter at cycle one and S2 represents the state of the counter at cycle two. And then we have this other precondition that the counter is not reset. And then here's our negated theorem statement. So we're trying to see if there's a case where the count decreases. So if the count at the second cycle can be less than the count at the first cycle. And as you might be able to figure out just by thinking about the implementation of the counter, it turns out that this theorem we were trying to prove is false. The counter returns SAT, or the SAT solver returns satisfiable, which means that there's a, there's a counter example to our theorem. And here, looking at the counter example, we can actually get some more information beyond just like the theorem we were trying to prove as false. Uh, looking at this a little bit more closely, we can see that here, uh, at S1, the, at cycle one, the counter had its reset line low, it's enabled line high, and the count was at 127. And then thinking about this, we can realize like, oh, the count's wrapping around. And that's why our theorem isn't true. Okay, so how do we apply this to CPUs? Well, the CPU is basically like our example of the counter. It's just a little bit bigger. Like it's a big circuit that has some state, has some inputs, has some outputs, and we can think about it kind of in the same way. Like, just like we had this counter s to represent the state of the counter, and the counter t to represent the behavior of the counter, we can just have a CPU s and a CPU t. And this translation can be done entirely mechanically. So on the left hand side, I have Verilog implementation for the gate level implementation of a CPU. And on the right is the extracted z3 code that describes one cycle of executing the CPU.  So you can be like, the CPU is in some unknown state, let me reason about what happens after some number of cycles of execution. You can even do reasoning over, like, partially symbolic state. Like, you might say, if the program counters zero, but all other state is unknown, let me reason about what happens over several cycles of execution. And so, now again, visually, this is what our reset theorem looks like. So we're considering two worlds where they can be in any possible initial states. And what we do is we apply the reset operation to, like, in one cycle of execution, like, we assert the reset line and simulate the CPU for one cycle, and then after that, we let go of the reset line and just let the CPU execute a whole bunch of cycles and over symbolic states. And if they converge to the same final state, no matter what the initial possible states were, then our reset operation, our purge operation is correct. And note that this is not like a test. This is not doing, like, test cases and checking for a bunch of different things. It's a proof. This must hold for any two possible starting states. And if the SAT solver can prove this correct, well, it will always work. And with these tools, we get a really nice workflow for developing this reset code. So remember, I was talking about how it might be complicated to write code that has exactly the right effect at the micro architectural level. Well, what's kind of cool is that with these tools, we can write some code and just throw it at the verifier. And if the verifier says unsat, then we're done. Like, that means that the CPU is always reset correctly. But if the verifier says SAT, not only do we know that our reset code doesn't quite do the right thing, we find out that there's a concrete counter example. And the tool can actually tell us, oh, this particular piece of micro architectural state was not properly cleared or not provably cleared. And then as the human in the loop, we can think about this, consult the implementation of the CPU and tweak our code in order to have the desired effect on that piece of state. And just keep repeating this process until we've developed code that always does the right thing. So what does this actually look like in practice? Well, here's a demo of using our tool to verify a CPU. And so in this demo, we're going to be developing reset code for the PicoRV32 processor, which is a simple risk five processor. And here I've already converted SMT fully automatically. And we're just going to develop the reset code and also see the output from the tool as it tells us like either we've done it correctly or we haven't done it correctly and what pieces of state we haven't cleared. OK, so we go ahead and run it. To start with, let's just run it for one cycle of execution. So let's see what happens when we just assert the reset line of the processor and let it go for one cycle. Well, as we might have expected, it's not correctly reset. There's a bunch of internal state that's not reset. And this tool tells us one particular piece of state that it complains about. And it also gives us a concrete counter example where it gives us an entire CPU state where if you take these two CPU states that are different from each other and then try to apply the reset procedure, they still haven't converged. And then the tool also hopefully tries to point at a bunch of other differences between the states. So this is like pointing out a whole bunch of internal micro architectural state, which is not cleared. And so as a programmer, we can look at this, like think about what's going on. And in this particular example, the one piece of state that the tool complained about way at the top was part of the instruction decoder. And so thinking about this, we can be like, oh, maybe we can just write some code. Like if we let the CPU execute for more cycles, maybe whatever's going on in the instruction decoder, well, it might be like cleared up by this. And so what happens if we just put a bunch of knobs at the start of the region which the CPU executes from when it's reset? Okay, the tool still failed. Oh, I only ran it for one cycle. To actually execute the knobs, I should probably simulate the CPU for more cycles than zero cycles. So okay, here I'm running the tool again. It's complaining about something in the instruction decoder. And we'll see that after a while, it starts complaining about something else. So now it's complaining about the sub part of the state called CPU regs. That's not always reset correctly. And so now again, as the programmer, like we can think about this and consult the CPU implementation. And if we did that, we'd see that CPU regs is something that's corresponding to the architectural general purpose registers inside the CPU. And so now we can clear, we can figure out how to clear that state. So we could just write some code that goes and clears all the 31 general purpose registers. Okay, so let's do that. And try running the tool again. So here we're running it again. Let's symbolically simulate from kind of unknown initial states and see what the tool can figure out. Okay, if a couple cycles in, it's complaining about whatever was in the instruction decoder. Now it's complaining about CPU registers, but we just wrote code to clear them. And oh, the tool failed. Okay, so scrolling up, we can see that, well, in this case, it just said failed to prove reached max depth 10. Well, with this tool, you need to tell it how many cycles to simulate for. You should give it some upper bound. Otherwise it might go on forever. And so I just didn't run it for long enough. So let me go and rerun it.  Okay, so the tool is simulating for many cycles now and still complaining about the CPU regs thing. So whatever is corresponding to the architectural general purpose registers. And we'll see that eventually it stops complaining about this because it's actually gone through enough of the reset code that all that stuff is reset. But now it's complaining about something else. Now there's this thing called memw data that's not provably reset. And so again, we can go back to CPU implementation, figure out what this corresponds to and this is something related to the memory write machinery. So now we can go and write a little bit of code that deals with that. In this situation we can just issue like a dummy write to read only memory region and that'll end up having the desired mark architectural effect. And finally, when we do this, we'll see that the tool, well once it gets through enough cycles of simulation, it finishes and it's like, oh, your CPU, your reset sequence is actually proven to be correct. And again, this is not like a test. We didn't try this for a couple different initial states. The tool's actually proven that this works no matter what state the CPU is in before. That no matter what state you start off in after applying this procedure, you always end up in the same resultant state. So that's the tool. And then of course we also wanted to make sure that this idea of like building a hardware in this wacky way where we have third party code that runs on its own CPU and like management code that runs on another CPU and stuff like that kind of makes sense. And so we built a hardware prototype.  But as we can see like to the end user, this behaves more or less like any other cryptocurrency hardware wallet. You do things like install applications on it and use it to sign Bitcoin transactions and things like that. And really the only difference the user will notice is that since it's built around this idea of like only one piece of third party code runs at a time and between applications which is you reset the processor like every once in a while the device will ask you like there's nothing more to do now. Like you need to reset the device before you can continue. Like here for example, you've installed a program but in order to launch the next program you have to clear all the state. Which the device will do for you by while asking you to press the reset button. And then the rest of the functionality like here we're just running a Bitcoin application. It works like any other Bitcoin hardware wallet so I can get the public key from the device and then go on my computer and set up a watch on the wallet. Like go here, open up Electrum, put in a public key, type in my very secure password that's I think DEFCON all lowercase D-F-C-O-N. It's a weak password. And it basically does the thing. We first magically receive Bitcoin so we can actually send a transaction. And you can see like of course we can't actually send the transaction from the PC because the PC doesn't have the private key. The whole point is to have these transactions go through the separate hardware device which is the only thing that knows the private key. We can use the PC to construct this transaction, send it over to the device which will parse it, display it on the screen in a human readable way and only if it's confirmed does the device sign the transaction and send it back to the PC at which point it can be broadcast on the internet. And so this is a reasonable design for a hardware wallet. Like we could make it run the same kinds of applications that other hardware wallets could but we get this really strong isolation of mutually interesting applications by applying this idea of deterministic reset and reset based task switching. And we've also made it work with websites and stuff like that.  You try to do something on a website, any sensitive operations are required to go through this external device. It'll display a human readable summary and only if you approve the transaction on this external device will the website let you proceed with the transaction because it requires a signature from a private key that's on the device that's not accessible to the PC. So if someone tries to do something malicious like delete your domain name, your PC's compromised and it's trying to do weird stuff on the website, well you could just cancel it on your device and the website wouldn't let you complete that operation. And so in conclusion, we've talked about how we can apply the idea of turning it off and on again as a building block for better isolation but how there's also a little bit more detail there. Like there's a lot of details to getting the deterministic reset right and there's this neat idea of using formal verification as a technique for both developing this reset code and also reasoning about its correctness in order to gain confidence that this primitive that we're using for isolation actually does the right thing, the thing that we expect it to do. All the code that was shown in the demos is available online at this link if you wanna check it out and play around with some formal methods tools. And I hope that the impact of this presentation is that we as users more of us start using and demanding these transaction authorization devices and demanding that websites have support for this sort of thing because I think it can really increase our security. And I also hope that we as developers while we start supporting, factoring out approval decisions and also I think it'd be really cool to see more people use formal verification as a tool to improve security. Like maybe in the long term, maybe eventually we can move beyond the just it hasn't been broken yet and towards having a principled reason for believing that a system is secure. And I think I don't have any time for questions but I'll be outside if anybody wants to talk to me.


Good morning Internet and welcome to another installment of the Visual Studio Remote Office House. My name is Matt Christensen, and thank you so much for joining me here today where we're going to learn about Git and working with Git Source Control in Visual Studio, and nowadays with all of us being at home, team collaboration is super important and it's changed its character. So having a great Git experience and source control experience is paramount. So I'm super excited that we today have a couple of program managers from the Git Source Control team within the Visual Studio organization to talk to us about what they've been working on. They have some really cool stuff. So let's just see what they have to say and let's move that into Tayser. Good morning, Tayser.  That would be fantastic. Thank you. Hello. Good morning. Thanks for the introduction, Matt. So my name is Tayser. I have been working on the Git experience or the source control experience inside the Visual Studio for about a year now. As Matt said, we are very excited to start to talk about some of the new features and functionalities that we have been working hard on. So this is a very fun and exciting time for us. Awesome. And Prateek. Hey, hey. Hi, everyone. My name is Prateek and I also work on the Visual Studio team along with Mads and Tayser. I joined the Git team, the version control team specifically with Tayser, a little bit later than he did a few months ago. I've been working on the Git features and super excited to revamp the entire Git experience in Visual Studio and add more GitHub integration features as well into Visual Studio to make your GitHub and Git experience super easy and convenient. Awesome. So that's really cool. I can't not notice, Prateek, that your home office, I presume your living room or something like that, it looks very much like one of those backgrounds from teams that you can inject is like that clean, white, perfect. Yeah. That was totally unintentional, but I do have a nice plant up here and it is a white background. I have to keep my blinds closed because otherwise there's too much glare.  But hopefully it looks good. Yeah. I like their setup too. It looks more organic, more real. It's actually staged. I put up paper in the windows here to block some of the light, but I just got a new camera. So I used to have just the camera in my laptop. But what you were saying, the white balance and all this sort of stuff was just completely off, it couldn't handle it. So I got cheap no-name cameras because all the web cameras are sold out, like Amazon, Best Buy, you name it, anywhere. But there are still some of those no-name brands. So I got one and it's much, much better than what I had in just my laptop. So it's also wider. So you can see more of my messy garage. But yeah, that's how it is. So just a reminder to the people online right now who are live, we do have a Q&A panel. So if you look in your browser or in Microsoft Teams, you can see a Q&A is available. And so please ask any questions you've got to Tayshir and Pateek or myself about working with Git or GitHub within Visual Studio and or any other Visual Studio questions for that matter.  So with that out of the way, I'm curious to know, let's start with you, Tayshir. What is some of the new stuff? What is it that you've been working on? Yeah, so we were, you know, at towards the end of last year, we, you know, we started to discover that actually the, you know, the previous experience that we had in or we had in Visual Studio was first of all, not very discoverable to lots of our Visual Studio users. So we would talk to folks and say, well, what do you think about the Git functionality inside of Visual Studio? And they'll say, you know, what good functionality, right? So that was a big aha moment for all of us here. So discoverability was a big thing. And then we got also a significant amount of feedback about user experience and the amount of clicks that is needed to execute specific functions, also having to switch between different pages to be able to follow a specific Git flow.  So like those were the two things that actually, you know, made us pause and take a step back and, you know, think about, you know, the new experience. And that is, you know, for example, one of the main new things that we have introduced is the top level Git menu. And that is really great because that enhances the discoverability of the Git features. And that is a great add-on for keyboard users now because using Alt-G, you can very quickly get to that menu and then access, you know, the different Git tooling or the different Git functionalities that we have, for example, right? Yeah, so that is one of the things that we added specifically to address that feedback. And then we added other features into the product also to address the user experience feedback that we got as well. For example, the Git tool window is an example there. It's a lightweight experience designed for the inner loop where you can very quickly, you know, code and then stage, commit, push your changes without introducing any context switching in that case. So let me just ask a question here. So I've heard people talking about the inner loop. I've heard that before. Like it's something that we talk about in Microsoft, but what is the inner loop and why? Like why does it matter and why do we care? What is it that we're talking about when we say inner loop? Yeah, so the inner loop is really important at the piece for, you know, because it is the flow that you go through on a daily basis as a developer, right?  And it is a goal for all of us here in Visual Studio to make sure that we focus on the inner loop to enhance the day-to-day experience for our developers, right? So when we talk about inner loop and Git specifically, we are talking about, you know, the steps that you, that a developer would need to do while they're coding, right? If you are fixing a bug or if you are working on a new feature. So in that case, you would need to, you know, write your code, start staging and then committing and pushing and doing all of those functions, right? So we consider those to be a part of your inner loop experience. Okay, well, that makes good sense. Yeah, but yeah, to add to that as well, navigation was a big pain point that we observed in users and that was affecting their inner loop as Tayser mentioned, you would have to navigate between different windows in Team Explorer and the other Git pages and version control pages that we had in Visual Studio and users complained about that, right? There was a big learning curve in understanding how to use Git functionality and it was taking away time from actually coding and developing, which is what we want developers to be focused on when they're using Visual Studio. So to simplify all of those experiences was one of our main goals and to streamline that into a single window that developers could use on the side and not get distracted from when they're actually coding and doing their day-to-day work. And this is supposed to be a very intuitive sort of add-on to that. All right, so this makes a lot of sense. So I assume that, you know, all you said that there was a lot of feedback from users about not being able to find the Git tooling, they didn't even know that we had Git tooling in Visual Studio, but that the tooling we also had was inefficient, was not good enough. And so I assume they just kind of didn't use it and instead used either the command line and I know that, you know, for instance, GitHub has like their own client 50 work on GitHub, you could use the desktop client or there are other products out there. Is that what people did? And then how did you then find out? How did you approach them and ask them about this? How do you know all this information, I guess, my question is? Yeah, so, yeah, so I guess the answer there, the short answer is yes, a good number of Visual Studio users relied on other Git tooling to do their Git work, including the command line and other visual tools like, you know, Git Kraken and GitHub desktop, as you mentioned, but we also had some Team Explorer users, right, that we kept talking to and get feedback from, right? So it has always been a combination of users using different tools for us, right? And sometimes when it comes to Git, our users tend to use, you know, a number of tools together. What was also interesting for us to see is that a number of our users were also relying on Visual Studio Code to do some of their Git work, which is something that we have noticed because, you know, they had Visual Studio Code had a lighter experience, which is a little bit more streamlined than what we had before. So is it fair to say that the approach you've taken is to maybe take the best bits of what was successful from Visual Code and the best bits of what was in GitHub desktop client or and take, you know, merge all those best parts into Visual Studio, how did you come up with what the right approach is? Well, not only that, Mad, it's not only the best bits, but because other tools also may have issues and problems that they don't know about and we need to discover and we need to kind of create a better tool. And this comes to like a part that I love doing as a program manager in Microsoft is we talk to customers, right? We talk to a lot of customers. We have a user experience lab back in Redmond when we all used to still work out of the office in Redmond, but in building 16, we have a great user experience level where we bring customers in developers of Visual Studio, developers who use other tools, very experienced developers, completely new developers who are just starting out in their careers. And we ask them questions. We sit them in front of us and we ask them questions about how they work, what they do, what makes sense to them, what doesn't make sense to them. We show them prototypes of our features. We show them working bits of stuff that we're just starting to work with and we ask them to like, you know, take steps and do tasks and we watch them observe them and we learn a lot from seeing them and not just the questions that they answer, but just, you know, the expressions that they make and kind of when they try to use a tool and are stuck and confused, you can really see that sense of frustration on their face and that's what we want to avoid and that's what we wanna help eliminate. And we learn a lot from these micro interactions that the customers have with these tools. Today in the climate that we have today we can't really go to the lab and bring users to the lab. So we do remote studies, right? We join teams calls with customers all throughout the day. We have some scheduled for today itself actually, right after this, we have customers coming and joining us on the call and we're gonna show them what we have in Visual Studio, the tooling in Visual Studio, as well as prototypes and mockups of new things that we're kind of working on and we get their feedback and we use this to kind of direct our vision and our designs, not just what other tools have and the best of other tools, but like new things that we can come up with and show to customers and users. Yeah, so to elaborate here a little bit, like what Pritik was trying to say here is that we are trying to build a first class get experience inside of Visual Studio by talking to customers and doing all of those activities that Pritik have been talking about. So that has been our goal to say here. Okay, so that's really cool. So you wanna build the best possible get experience inside of Visual Studio. So I guess that also assumes that, well, I will then assume that you are saying in a perfect world with no knowledge of what the constraints might have been architecturally and whatnot in Visual Studio, what is the pinnacle, the best possible get experience that we can provide to our users in their Visual Studio interloop, in their workflow and start from there. Is that how you started it or did you start more constraint and if you had those constraints, what would they be? Yeah, so that is actually a good point, right? So in our first user studies that we did, we discovered that there were, when it comes to get in general, there are two types of activities for users, right? Those interloop activities that users would need to do while they're coding. And there are those other activities that users don't, like they need to focus on, right? And when they are not actually coding, but when they are browsing, managing the repos, right? So the approach that we took here is to start by focusing on the interloop and then slowly expand into the other, richer, more focused get experiences that would require more real estate and require more interaction with users. Okay. So I've used Git for many years. I have like, I think 250 open source GitHub repositories. And, you know, one of the things that where I'm a little weird is I've always kind of been afraid of Git. I've always kind of been not wanting to do feature branches.  And that's been how I've been working all along. And so when I take a pull request, I take it and merge it straight into master. I don't rebase, I don't do any of that sort of stuff. If I get a pull request that needs to rebase because there's merge conflicts, I ask the pull request D to do that on their side before I can take it. That has worked okay for me, but I also know that it's a rather limiting thing because I don't do these feature branches. I don't have release branches.  I'm fully aware of that. But my fear has been just, you know, another merge conflict and another headache. Now I need to read up on like Git command line because I knew the Visual Studio tooling. But I guess I didn't know about it, but I felt like the Visual Studio tooling back then was not gonna help me in these advanced cases. So that was sort of where I came from. And I've been playing around with the newest Git tooling for the last, like a lot in the last couple of weeks, especially, but also a little bit before that. And I'm resolving merge conflicts now that, you know, I would never do that. I was afraid of that before. And now I'm just, I'm merging them. I'm doing feature branches. I'm doing all this sort of stuff. And I can't believe like if that has been possible the whole time that I've never done it before, but now you've wrapped it up in a, such an easy thing to do, right? It's such a pleasure to do. It feels natural. I don't have to guess on how to do things. I don't have to go read documentation on how to use Git command line or the Visual Studio tooling that you've built for me now. And so it's like a whole world opening up. And so first of all, thank you so much. But second of all, if anyone listening out there, you're in the same boat as me where you're not super confident using Git, try out the new tooling. So the Git top level menu that they're talking about, but also the tool window and all the features in there, that is just an absolute pleasure. So thank you so much. Is this available to everyone? Because I'm on internal builds. I get sort of the latest and greatest, you know, from our daily builds. We have like five daily builds and I get them all almost. So how do people get this? Can they do this today? Like if you're listening to us right now and you open Visual Studio, do you have it? Pratik? Yeah, sure. So the, well, it depends. The answer to your question is it depends. You may have it, you may not have it. We are right now in preview for the Git functionality. All this new, the new Git experience is in the preview channel of Visual Studio. So if you are, have downloaded and installed Visual Studio 2019 preview and are on the latest version of that. So 16.6 onwards, you will get this Git functionality. It is not yet in the release channel and we want to get there. We want to get there soon to be able to provide a complete Git experience to everyone using Visual Studio, the public version. And we want to get there soon, but we are, we're working hard. The entire team is working super hard to reach a point where we can say, yes, we're confident that all our customers, all our users, whatever they want to do with Git, they can do it in Visual Studio. They can do their inner loop. They can commit stage, stash, push, pull, make branches, solve merge conflicts. We want them to be able to do all of those core features within Visual Studio and we're slowly building that up. And while we build it up, we don't want to GA and say, hey, everyone use it in the public channel. We're slowly testing it out and making sure we're building the right experience with our preview channel customers. So if you are on Visual Studio preview, you can get this experience under a feature flag. So all you have to do is go to tools options and in environment, there is a preview features pane. And there you'll see all of the new, the latest features of Visual Studio that aren't available to the public directly yet. And all you have to do, there's a checkbox for new Git experience. And if you check that box, you'll switch from your old previous team explorer view of version control to this new Git experience view. The Git tool window will light up, the Git top level menu will light up and you'll be able to use these features. All right, that's awesome. I will say that before you rolled out this, I would say rather elaborate change. It's a big change. It feels big. It feels significant. There was one thing that I for many years asked for that never came, which was, I wanna have a keyboard shortcut to do a pull from, let's say, from Git up, let's say. So the typical scenario would be I would come into, I would open a repo that I haven't had open for quite a while. I knew that I've had taken pull requests for it or maybe I would not be sure if there were new pull requests that had come in in the meantime. And so I didn't know if I had the latest. So I would like to be able to, without opening Team Explorer at the time, this is before the Git tool window, the Team Explorer tool window, I had to like manually go in and say pull to get the latest or sync. And I would like that to be just a keyboard shortcut. And so what happened in visual through to 2019, I think the first version of 2019, so not even in an update, but correct me if I'm wrong, Taysher, I don't know if you were the one that did it, but I'm super happy that you made that into a command that I could find to a keyboard shortcut. And I could also, so I wrote an extension as well that automatically did it. So whenever you open a solution, it would automatically update all your branches. Like I think it did a fetch or something like that on your various branches that you've got. So you were always kind of at the tip when you started your day opening Visual Studio. And so that was like just a game changer for me. That meant that I didn't come to these situations where I had merge conflict, right? Because, oh, I forgot to do a pull and I just made a change and now I can't merge that thing in and all that sort of stuff. So this seemed this whole notion of like improving the Git tooling seems to have started, like actually quite a while back, this is not something new. Is that a correct assumption? Yeah, that is right. So some of the works has started some time ago, including what you mentioned. And I also know that there were some work that has been done on the stash functionality. Stash wasn't there before and that was added based on customer requests and feedback. So that's totally accurate. Yep. All right. So we got a questionnaire from Jerome. He says, I've been trying to use the new Git tooling but since the Team Explorer, that's the tool window Team Explorer is disabled and there's no show solutions and show folder view, I've disabled it. I'm frequently closing solutions and finding the active solution through the MRU is easy to work with. What is that? Is that something that you're looking into? What is that problem he's talking about here? So yeah, we have been getting a lot of feedback about the repos list and the solution list. Those were features in Team Explorer that we initially took out and that's why we started from kind of a methodology or a viewpoint of let's start simple. Let's not add on a whole bunch of features in the beginning but as we build up this Git experience let's start with the very simple basic features and as required we'll add on features and add on functionalities and capabilities where they make the most sense and if we get feedback from users that, oh, we were like the team in Visual Studio was not completely correct in the first assumption that this is not a necessary feature, right? And so their list of solutions and the list of repositories is something that we've been hearing feedback on from Team Explorer users to say that this is something that they're used to seeing and we do have some workarounds for that. So you can see your list of solutions once you've selected a repository and you can see your list of solutions in the MRU that you have in the start window in Visual Studio 2019. You can see your recent solutions that you've opened in the file, recent projects and solutions view. You can also, once you've opened up a folder in solution explorer, you can switch views to see your recent solutions in the folder view itself in solution explorer. So there are a few different ways where you can see your solution list within Visual Studio once you've selected a repository. So we do have those workarounds and we're trying to see if those are sufficient workarounds because I understand that it's completely a muscle memory and you kind of expect a design and once you're used to that design you kind of expect to see it there and switching to something new is always hard, right? Whenever an app that I use changes its UI I get like wait, where is everything? So it's definitely something that we want to help make easier and change management when we change UI on developers and users is always something that we're concerned about and looking for. So we don't want to rationally bring in a solution list anywhere, somewhere that may not make sense to the overall goal of the design and so we're trying to be very deliberate about how we bring features in. It's definitely, we definitely have an eye on it and these features that we're missing, please give feedback. You can go to our developer community where we have a lot of suggestion tickets and different features that users have requested. So if you go to aka.ms slash vsfeedback that's where you'll see all our suggestions and you can create a suggestion or even vote on the suggestions there and that helps us prioritize one of the most important things for you. Thanks for asking that. Yeah, that was a great question and a great answer. Thank you, Pratik. So it kind of illustrates a funny kind of thing about Visual Studio is that I've never myself, I've been using Visual Studio for like several decades and I've never used Team Explorer to open solutions. I've always used the file recent solutions in through there. I used to use the start page when we had that. Start window is also good. But it kind of illustrates like people just have different ways that are natural for them. And the funny thing is that oftentimes when we get suggestions coming in through the mechanisms that Pratik was just mentioning, people think that it's for them it might be the most important feature in the whole world and very few other people feel the same because they have a different workflow. And so it's actually hard for us to prioritize without a very large sample size of people to kind of figure out the priority of these different things. So I guess that's just sort of an anecdote to throw in there. But I guess that illustrates also why we don't have that solution view. The people that you talked to didn't have a strong opinion about having those out there but some people will just like Jerome that asked the question. So that's, this is fantastic. And this is why we do the preview features by the way such that we get to test it in the wild to get a much bigger sample size and broader set of feedback. So here's another question. This is from Larry. When will the Git global settings be integrated into the standard VS options? Teijer? Yeah, so actually that is working progress. That's a good question. So actually now we have the general settings already integrated into VS, VS's options. And we are working progress to finish the work on the repo settings, right? So that's definitely coming in. And an entry point to that would be, you know the top level Git menu and then you will have settings there. So if you click on settings that will take you automatically to that page. All right. Well, so that's good news Larry and everyone who wants that, it's coming. So Teijer, in the beginning you were talking about like discoverability problems that people didn't know that we had Git toolings. So for the people that did know that we had and used the Git tooling that was there did they ask for features that we already had but was kind of hard to find or they just for some reason didn't know we had like I think the office team had like some issues with that back in the day and that led to the ribbon to expose more of the features they already had. And was that the same case here for us? Oh yeah, absolutely. A good example that you mentioned today is the merge experience, right? We always had a great merge experience inside of Visual Studio. That is rich in functionality but few folks knew about that because an entry point to that was a little bit hidden wasn't very discoverable. So even though we had like we have started to enhance that experience actually but even before we started to do that like we started to get feedback from folks saying that wow, thank you guys this is a great merge this is a great new merge experience and we already had that before. So the work that we have been doing to enhance the discoverability helped and make it easier for folks to find the existing merge tooling that we already had into the product. And that is something that we started to focus more on and just to make sure that we keep enhancing it. So more and more enhancements are going to be coming and very soon for the merge resolution experience. Very cool. So I'm doing a lot of work with GitHub and I expect a lot of the viewers also work with GitHub repos on a regular basis. And so is there anything in the new tooling that's specific to GitHub? Because so far I think we just talked about Git in general and so any remote that supports Git will work including GitHub and others. But is there anything that's specific to GitHub? I think that's interesting now that GitHub is a Microsoft thing now. And so what do you got for us there? Yeah, so one thing we have there is the new init and push functionality where now very quickly with a single click you can not only initialize your local code but we do a bunch of steps for the user.  So once that's a significant shortcut that we introduced in that new feature and the feedback we have been getting so far is very positive there because like the previous workflow was if you already have code locally and you need to push that code to GitHub you would need to go manually to GitHub create your new repository there and then get the URL go back to your local repo and then add the remote and push your code. So lots of steps there to do for the user manually and we have automated that flow into end. So that is one example that we have been working on here. Yeah, and another example as well and I don't know how many people know of this feature or if it has been very discoverable is the ability to clone GitHub repository straight from Visual Studio. So you can not only clone a repository using a URL or an SSH format but you can also browse your GitHub repositories. If you've signed in to Visual Studio with a GitHub account on your key chain then when you go to clone a repository you can also browse through your GitHub repositories and it understands our authentication and just shows you a list of all the repositories that you have as well as your organization as under that one single list. You can also search different repositories, click on them and with a single click then that repository will be cloned down. So when you want to get started with code if it's either way you have existing code on GitHub and you want to bring that down or if you're starting with a new project and you want to push it to GitHub those are the flows that we started off with making super easy to get started and just to get started working with the code and we're going to start making more inner loop as we mentioned earlier actions in GitHub a lot more convenient and intuitive also. So that's on the roadmap. Sweet, so a lot more GitHub specific integration that sounds great.  So it's an authentic thank you, not a corporate thank you and so that's great. I've been using that to publish some GitHub repos and even though I could do that before with the GitHub extension for Visual Studio now it's all sort of within that same workflow of setting up your Git repository you immediately just go to GitHub and you don't even think twice about it. It just works fantastic. So has any of this been in collaboration with GitHub? Is there any like since we're now part of the same family is there a bigger push on like improving sort of the GitHub integration? How does all that work Tisha? Yeah, so we have, you know we started to work more closely with the GitHub team to enhance, you know our Git tooling instead of the product itself and there are, you know based on what Pritik has just said we are, you know there are still a lot to do there. So we are just, you know since we discovered that we had these issues discover abilities or experience and into our Git tooling and functionality we had to take this step back to rebuild or revamp the Git experience the basics in Visual Studio and then from there the plan is to continuously to add new features into it, right? A very popular example is the is the PolarQuest integration into Visual Studio. So that is something that we have started to work on last year and we had to close our work there for so that we can revamp the basics and then the plan is to get back to that, right? That is just an example that we have been working closely with GitHub for or on. Yeah, and as Tayser mentioned the GitHub extension for Visual Studio we've had some questions about like what's going to happen with that extension, right? If we're building functionality into Visual Studio and there are a lot of good features in that extension if you haven't tried it out you definitely should. There's the PolarQuest functionality there's forking and a lot of it we want to get into and inbuilt in Visual Studio but we're not there yet. So you can continue to use the GitHub extension and as we build more features in Visual Studio we'll try to create a fully functional GitHub experience inside of the product itself. Some of the other features are also like opening code from github.com. So if you have and we've been speaking with the GitHub team about this is if you're browsing github.com and you're on the browser and you want to open and you see a repo that you want to maybe clone down and start working on then you can with a click when you try to clone it instead of just cloning it and then having it flown down and then doing that maybe through the terminal or the command line and then you have to open the IDE or editor of your choice and then open find that repo and then open it in the IDE you can in a single click just open in Visual Studio so that will launch Visual Studio start the cloning the repository and it'll open it for you in the product itself. So that's also one of the features that's a great feature of the extension that is on our roadmap to get into Visual Studio as well. Yeah, I was just about to say we already did that like some years ago, right? There's if you have the extension and you've used it with GitHub then that menu item shows up on GitHub that you can clone with Visual Studio or open in Visual Studio. Open in Visual Studio, yeah. That's a feature of the extension and we worked extensively with the GitHub team in order to create that functionality it was a lot of collaboration between us and them and that was before the acquisition so that was pre-acquisition times itself. Okay, so are there any features that you already know now that you're not gonna do going forward like either features that the Visual Studio extension for GitHub already had or other features that maybe other clients have that you feel like they don't really belong in Visual Studio or something to that effect? Tejer? That is a very good question. We always think about the features that we miss. Yeah, and then we are actively working on this so we continuously focus on the user feedback so going back to what Pritik said earlier please, please, please continue to use the new functionality and provide your feedback, suggestions, we continuously monitor those. Actually, we have lots of good requests so our suggestions and feedback has always been full of things that it's not us not wanting to implement those, it's just prioritizing and figuring out where we should start. For example, very popular requests that we still haven't had the chance to get into is multi-label support but in order for us to be able to provide a great experience there we need to have a solid base first. So some of those suggestions and feedback we have been pausing the work on just to make sure that we are starting or have the right foot in to be able to build that functionality. And so we will continue to do our best to prioritize and figure out what makes sense implementation wise and from our users point of view to start to work on. Yeah, yeah, I'd say that nothing is thrown out or everything is game still. We haven't said that we're not gonna do anything yet. It's all a question of and it's a classic engineering problem you have a limited amount of time scope and effort that you can put into something. And so it's all about prioritization and that's why against suggestions and developer community and interacting with the entire developer community is super important for us to understand because we're gonna keep building and it's gonna be a long journey and we're gonna keep doing it as much as we have the time to. So the more you tell us as an audience and as our customers, the better we'll get it. Oh, that's fantastic to hear. So nothing's off the table, anything can happen. And basically the users guide the show, right? They make the suggestions, prioritize it, comment on it, vote on it. And that helps you prioritize and maybe get to things faster than you would have otherwise because you're realizing there's a priority over here that is now greater than this other thing you thought you were gonna work on. That's absolutely fantastic to hear. So are there any of those big ticket items that you are very excited to start working on that you don't have it yet but you're almost ready to start it and what would those features be?  What do you got coming for us? Yeah, so that's an interesting question. So for example, one of the main feedback points that we have been getting into this new good functionality that we are introducing was on branch management. So on our tool window, we introduced a very lightweight branch management which is not up to the functionality that we used to have on Team Explorer before. So I'm happy to say that we are taking all of the feedback into consideration and that is one of the main things that we are starting to look into. And based on our studies before, actually branch management belongs to the focused experience, right? So right now we are starting to focus on like building that focused experience which is more of the, you can think of that as your repo management or you repo explorer. It's where you'll be able to do a rich branch management navigation. You'll be able to see a graph of your repo. So that is something that we are starting to look into and work on. So we are very, very excited to start to share bits and pieces there very soon. All right, that's cool. So branch management. So what does that mean? I can create branches, I can switch to branches or check out branches, I guess it's the right terminology. Delete, like is that sort of what you're talking about or what does it entail? Actually, we already have those lightweight functions that you mentioned as a part of our inner loop experience. So today you can very quickly create a new branch. We have different entry points. Their top level get menu is one. You can do that from the get tool window as well. So we already have that functionality but I'm talking about the richer experiences that we have been getting lots of requests on including the ability to delete multiple branches at the same time or the ability to visualize and look at your repo in a rich perspective or a rich way. And those things, we discovered that we need to build like a focused experience for. So that is going to be the base for all of these rich experiences, right? And as the more we build, the more functionalities we will be integrating there, richer experiences that users have been asking us for. And poor request is one of those experiences, right? For example, interactively base is another experience that we will be able to benefit by having that larger real estate to build and make sure that it is functional, it is usable, yeah. Sweet. So that's very interesting. The pull request thing you also mentioned here at the end, one of the problems I have with pull requests is I always, whenever I get a pull request, I always go to GitHub and look at it in the browser and it's, so that works okay. But it's not a very rich experience. And what I really like is to be able to actually just run the program with those changes from the pull request, maybe run the unit test, just make sure everything works, see it in action before I, before I merge that PR. And I don't really get that when I do it in the browser. And you know, so I will be the first to admit and I don't know if I'm alone in this, but I have merged pull request before that I kind of didn't know if they would fully work. I wasn't sure about the quality of them. This doesn't happen that rarely, to be quite honest. So is that something that you're gonna be able to help me out with so I can do like a more holistic view of that pull request in the full context of like it's running, the app is running, there's a database underneath, all that sort of stuff. Where are we with that? Yeah, so that is some of the stuff that we have started to work on last year and we had to pause, but then the plan is to make that a part of this focused experience, right? So we found out that, you know, it's really, when it comes to pull requests, you have two experiences. You have the author experience and you have the reviewer experience, right? So like we will look into those two experiences as soon as we are done with our basics. So for example, the author experience, Mads here, imagine that you have created a new pull request, right? And then you are waiting for feedback from your colleagues, right? So, you know, today we know that there is lots of context switching happening for the author, trying to switch between the browser and the editor or the IDE to basically bring in the feedback as they go through, you know, the feedback. So that's one thing. And then you also mentioned the reviewer experience is something that we looked at and we will continue to look at, which is basically about like making it easier to, you know, run the code pieces that you are reviewing into your IDE, which is something that you cannot do on a browser today. So those two experiences, you know, are things that we'll look at for sure. All right, so it sounds like they've already been on the drawing board and they will get back on and development will continue in the future at some point. So that's great to hear. So we are getting closer to the end here. So for those of you online, please ask any question you might have to Tasia and Pertig here. We also have a question. So the very first question we actually got in here was not about, it's not about Git or GitHub, but just the general revision studio question. So those are fine too, but that's why I haven't brought it up, but I will bring it up now. And then maybe get Tasia and Pertig's take on this question as well. So this is from, this is Martin. That's my old colleague from back in the day before I joined Microsoft, hello, Martin. He asks, when do you think developers will stop installing Resharper? Visual Studio has gotten a lot better and Resharper replaces or hides default functionality. So long time Resharper still think VIAs lacks a lot of features. That is a good question. What do you guys think? That's, I mean, it's a good question. Like discoverability is always an issue. And Resharper is one of our partners and it does have a, they do have a lot of great features and we do have a team, our .NET team, our C-Sharp team has been continuously working on adding more features that help a user in their development state when they're editing code and working on their applications. But yeah, we don't have a timeline we can't say if a developer is gonna actually stop when a developer wants to stop using Resharper, right? It's up to your preference and up to what you're used to and what you want to use. If you want to really see what Visual Studio is capable of, I'd suggest try turning it off for a bit. Like you can disable an extension pretty easily and see what Visual Studio has and try using it without Resharper for a little while and see what's missing still. And if that's really necessary for your development experience and the trade-off that you get of using Resharper and not using Resharper. So I'd just say give it a try and let us know what you think. Yeah, yeah, that's a good one. We hear from people all the time, you know, that they fall into camps, they tried to disabling it and some people like, whoa, I didn't know Visual Studio could do that. They haven't used Visual Studio without Resharper for like a decade. So they were not aware of what features are actually now in the product. And then you have the other hand, people that are like, hey, I don't want to use Visual Studio without Resharper. Those features are just super important. I tried to disable it and just didn't work for me.  Like that's whatever makes you happen. And that goes for any extension. Like there's, I think there's over 4,000 extensions now for Visual Studio 2019 alone, over 10 or 11,000 for all of Visual Studio in the history of Visual Studio. So like they're there because they're personal preferences. And so we want to make sure that you're happy, you feel most productive. One thing we do know is that the more you personalize and customize your Visual Studio experience, the happier you are with Visual Studio.  This theme, maybe you customize your theme. Keyboard shortcuts, like you can customize those as well. So the more you tailor that to your workflow and to your personal preferences, the happier you are. And we know this because we've asked people a lot. So Resharper, no Resharper. As long as you're happy, we're happy. Here's another question. Let's see here is from Larry. Any plans on making it easier to see what changes that are to a function without multiple steps to open up and commit and then find the file compared to changes and then locate the feature in the compare? Like something similar to the peak help or peak definition? Yeah, so that's a good question. And we actually got some feedback on that. And we have been thinking about enhancing that experience. So the feedback we got there was, oh, I actually want to very quickly focus on a specific piece of code, a function or something and then be able to see what went with that function. Like go back and forth in history, but focused on that function, right? To see what happened, what is the history of that function and what might have introduced a bug or something. So that is something we are looking into. We still don't have a great answer for it, but we keep that into our backlog and we'll get to that as well. Oh man, that just sparks an idea. Like how cool would it be that if you want a method or even a line of code or on a method or on a class or on a whole file, you could just scroll back and forth in time. There's a slider.  You can see who did what, what it used to look like in time. Is that sort of, is that sort of, I guess that's sort of the question here, right? Yeah, I'm just looking at, you know, we all seen movies where they build these, they program in 3D on multiple monitors and they already reported whatnot. It seems a little bit like that. Yeah, and that is the exact feedback we got, right? Like I want to get that rich experience that allows me to very quickly use that slider to go back and forth and see what happened there, right? And yeah, I mean, that is a part of the richer experience is that we are really looking forward into getting into as soon as we are done with our base, right? So we want to make sure that we have a strong solid base for functionality inside of the product. And then from there, we will start to look into integrating richer experiences. And that is just a good example there. All right, so we're kind of coming to an end here. I wonder if, before we hang up, do you have any closing remarks? Is there anything you want to tell the audience? Like what should they go do? What should they test? What is the one thing you want them to leave with here? They critique? Yeah, sure. And I kind of touched upon this before as well. For Visual Studio and the get experiences and for GitHub, our team is working to build up functionality and build up features. And we want to GA this, right? We want to GA this as soon as we can so that the entire Visual Studio developer community can use it. And right now, we're in a state where we think we're close and we think we're getting there, but the feature is still in preview and we want to get feedback from the developers from all the different kinds of developers, Madz, as you said, like every single developer has a unique way of doing something, of interacting with a feature. And we want to understand that we know that the way that we do stuff is not necessarily how other developers use Visual Studio. And we want to know if we're missing anything, if there is any critical features that are preventing us or that we should build out and prioritize before we actually GA this functionality to make sure that every developer and every kind of developer who uses Visual Studio can benefit from it. And so that's, I guess, my ask to the entire audience is to try it out, let us really know what's working and not working, and every single release we're adding new functionality to it. So if it's not working for you today, give us that feedback and switch it off if it's not working for you, but in the next release, try it again, turn it on and try it again because we've added more features and let us know then if what we've done has benefited and improved your experience in any way or not. So that's just my request to everybody, thanks. Awesome. Tejo, do you have anything? Yeah, I would like to add to what Pritik said.  So please, please, please try the NuGet functionality and share your feedback. So we are trying to build something that is usable by all of our users. So we would like to make sure that we get that broad perspective. So please try the new feature, we are actively working on it, share your feedback and stay posted because we are continuously adding fixes, adding features, as Pritik mentioned. All right, well, thank you so much guys for coming on. This was super awesome. I'm a big fan of what you're doing. You're doing amazing work. So thank you. And I can't wait to see what is coming out in the next preview. So yeah. And so that concludes our fourth episode of our Visual Studio Remote Office Hours. We'll be back same time next week. So Thursday morning at 9 a.m. Pacific time. And as usual, we will tweet out links to the event so you can join live and ask Q&A. Thank you so much. And have a great day.