He's a cross-program speaker. So Scott Aronson is, in fact, an old friend of PCMI. He's been here before. And I think he's somebody who almost needs no introduction. He's a great luminary in the world of quantum computation, AI, and a number of other things. And we're very pleased to have him give a talk about verifiable quantum supremacy. Can you hear now? All right, can you hear? All right. So thank you for inviting me. Let's see. So I am currently on leave from quantum computing for a couple of years working at OpenAI on how to stop the robot apocalypse, which I haven't figured out how to do that. I did ask Dali for his insights about verifiable quantum supremacy. And it came up with this image here. I'm not sure if there's any insight there or not. But basically, what I want to do in this talk is tell you about quantum supremacy or advantage experiments, what's been accomplished over the past four or five years, and in particular, with demonstrating the possibility of a real quantum speedup on a NISC, noisy intermediate scale quantum device, one that is not a full error-corrected quantum computer like we heard about in the last talk. And then I want to tell you, what are the shortcomings of these experiments? Why are we not satisfied with them? And what remains to be done? What I hope will happen next. And if I'm too busy trying to stop the robot apocalypse, then I hope that the truly important problems, like these, of designing better sampling-based quantum supremacy experiments, can be taken up by one of you. And so I'm offering a research program free of charge for anyone who would like it, basically. So I don't actually have many slides, so feel free to interrupt me if anything is not clear. I mean, I know that this is a diverse audience in terms of people's backgrounds, so feel free to interject. OK, so you've all heard something presumably about quantum supremacy experiments, which a lot of people have switched to calling them quantum advantage experiments for various reasons. I might use the two terms interchangeably. But so what is the idea here? So since the 1990s, we have known that of certain quantum algorithms, such as Schor's factoring algorithm, that seem to get exponential speed-ups, at least compared to any currently known classical algorithm, for certain very specific problems, in the case of Schor's algorithm factoring integers and taking discrete logs. But these algorithms generally seem to require a fault-tolerant quantum computer, so one that is constantly doing error correction to cope with the unwanted interaction between the quantum computer and its environment. And to do error correction by any known method seems to require hundreds or thousands of physical qubits for every logical qubit that you are trying to simulate. So if you wanted to run Schor's algorithm, let's say to factor RSA 2048 beyond what classical supercomputers could do in a reasonable time with any currently known algorithm, then it seems like you would need thousands of logical qubits, which then translates into millions or hundreds of millions of physical qubits. And maybe someday that will happen. Maybe there will be a Manhattan Project to build that. I understand that there was a PCMI show of Oppenheimer last week. It'll be a lot harder to keep secret this time, I think. I think that if you try to do the Manhattan Project today within about 15 minutes, what is happening in Los Alamos would be trending on Twitter. Excuse me, on X, yes. But OK, with enough effort, maybe you could build that. But that is not where we are now. And so maybe 12 or 13 years ago, some of us started asking the question, well, suppose you want to use a noisy device, a non-corrected, non-fault tolerant, maybe not even universal device, but such as might be available within the next decade or so. And your goal was to do something, a some well-defined task, much faster than any known classical method could do that task. Now, notice that I did not say a useful task. That's not part of what we're aiming for here. And I feel like part of the problem when there is all this press about quantum computers speeding up finance and vehicle routing and industrial optimization, I mean, it really gives people the impression that quantum speed-up is a done deal, and now we just have to get all the commercial value out of it. When actually the core scientific question, just can we beat a classical computer in a fair comparison for anything at all? Even that is still, I think, still not experimentally demonstrated or not clearly enough. And that is still true even today. But at least on that scientific goal, we've managed to make some progress. So how was that done? So if your only goal is to do something faster with a quantum computer and you don't care about whether it's useful, it turns out that a key thing that you typically want to do is to switch attention from problems with a single right answer, like factoring, to sampling problems. A sampling problem is one where the desired output is a sample from a certain probability distribution. So even if your quantum computer is running perfectly, it might never produce the same output twice in its whole lifetime. It'll generate a different output every time and now verifying whether the quantum computer is working correctly, well, that'll be a key question. But it'll be something statistical that we'll need to do to say, are these a reasonable set of outputs that have been generated? So actually, there was a very, I think, prescient early work in that direction by Tarhal and Divincenzo in 2004 on constant-depth quantum circuits. Barbara Tarhal is here, I think, somewhere here. But so around 2010 or so, I guess independently, Bremener, Joseph, and Shepard, and me and Alex Arkhipov started thinking about different ways that you might achieve this with near future quantum computers. And what we both found was that there were at least two advantages to switching your attention to these sampling problems. The first advantage is that the problem that you're solving can be much, much closer to the native hardware. The analogy that we used in our paper was if you wanted to prove that a dolphin is smart, you might be able to laboriously teach it arithmetic, teach it to simulate solving some puzzle. But actually, it would be much easier to just watch it in its habitat doing whatever dolphins do. And so likewise, rather than teaching a quantum computer to do modular exponentiation, maybe if we just want to see that it is hard to simulate with a classical computer, it will be easier to just apply a more or less random sequence of gates and just sample from the resulting distribution and then argue that a classical computer would have taken a long time to do the same thing. And that brings me to the second point, which is that we were able to give evidence that at least the exact versions of these sampling tasks really are hard for classical computers. We could show that if there were a fast classical algorithm that could sample from exactly the same distribution that the ideal quantum device would sample from, then the polynomial hierarchy would collapse to the third level. Now, if you don't know what that means, you could take my word for it that it's bad. But if you know that the P versus NP problem, it's sort of like P equaling NP, but just sort of at a higher level up, where we wouldn't quite notice it here on Earth. But in the heavens, they would notice it. So in the present state of complexity theory, that's close to the strongest kind of evidence that we hope for, that anything is hard. So the proposal by me and Arkhipov was called boson sampling. And all we said to do is generate a bunch of single photons, send them through a network of beam splitters and phase shifters, what are called linear optical elements, and then measure where they end up. So have an array of photo detectors that measure how many photons there are in each output port. And since this is quantum mechanics, the answer will be a sample from a probability distribution. The photons could end up in a different place each time. But the task is just precisely to simulate what the device does. And now the key fact that we relied on is actually something that I learned from Avi Wigderson back in 2000, when I was just starting as a grad student. And he said, it turns out that there are these two really central functions in computer science of matrices, the determinant and the permanent. And it turns out that if you have n, and there are two very basic types of particles in the universe, fermions and bosons. And it turns out that if you have n identical fermions, and you want to know what is the amplitude for them to go from some input state to some output state, and they're not interacting, then you have to calculate the determinant of an n by n matrix. And if they're bosons, then you have to take the permanent of an n by n matrix. And now in computer science, we know that these two functions, I mean they're almost the same in their definition. They differ just because the determinant has minus signs, and the permanent does not. But they differ dramatically in computational complexity. The determinant is computable in polynomial time, for example, using Gaussian elimination. The permanent was famously proved by Valiant in 1979 to be sharp P complete, which means at least as hard as any combinatorial counting problem, like computing partition functions or things like that. OK, so now, you know, I'm avi. I remember remarked on how unfair it was that the permanent, sorry, that the bosons have to work so much harder than the fermions just to calculate their own time evolution. And that joke kind of stuck with me.  But you have to be careful. And the subtlety is that, well, if you measure a bunch of photons or other bosons, you just see them in one particular place. You never get to actually see the amplitudes directly in quantum mechanics, which means that you never actually get to see the permanent of some matrix of your choice. And so nature is more subtle than just giving you the answer to your sharp P complete problem. What it lets you do, in some sense, is to sample a random matrix, say, a random N by N complex matrix. But in a way that is biased toward those matrices that have larger permanence. And so then what we did was to argue that even that task seems to be hard for a classical computer, not sharp P hard, but one can use the sharp P hardness of the permanent to say that even if you had a fast classical sampling algorithm, that would still have unlikely consequences. Now, if your classical simulation is only approximate, then the discussion gets complicated. You have to make some stronger conjectures than just that the polynomial hierarchy or whatever does not collapse. And the status of those stronger conjectures is still open, I would say, like more than a decade after we posed them. But it seems like you have pretty robust evidence for the hardness of these sampling problems. And I should mention that a Boson sampling machine would not be a universal quantum computer, we don't think, or even, for that matter, a universal classical computer. We don't even know how to build an AND gate or a CNOT gate using just these passive linear optics. If you have adaptive measurements, then it does become universal for quantum computation. That is the famous Canilla-Flam Milburn, or KLM theorem. But without that, you have a model that in some ways seems very, very weak. And yet it can sample these distributions that it seems that there's no classical polynomial time algorithm to sample. So we proposed this, and we were led to this not by any God knows, by any knowledge of experimental physics, but mostly just because we knew that something about the permanent and its sharp peak completeness, and we had heard that Bosons give rise to permanence and their amplitudes. So maybe one can do something interesting with Bosons because of that. But then after we started talking about it, we found that the quantum optics experimentalists had been sort of looking for something to do, and they just kind of ate this up. And so initially, in 2013, they did experiments with three photons verifying that, yes, their amplitudes look like permanence of 3 by 3 matrices. Obviously, that's not in any way challenging a classical computer yet. But then they scaled up to, I think, five or six, and then 14 photons. And then in 2020, so during the pandemic, actually, the group of Chaoyang Liu at USTC in China reported that they had done boson sampling with about 50 and then 100 average photons detected. Actually, they did a variant of it called Gaussian boson sampling. And this seemed very hard to simulate with a classical computer. We're going to come back to that question because a recurring theme is that often because of all of the shortcuts that have to be made or the corners that have to be cut in order to do these experiments at all, often they turn out to be much, much easier to simulate on a classical computer than you would hope. And in this case, a big issue with current boson sampling experiments is that something like 70% of the photons get lost on their way through. And the more photons are lost, the better classical simulations can work. Now, even before that, actually, Google had beaten them to the punch with a different kind of quantum supremacy demonstration. So basically, in 2014 or so, Google hired John Martinez, who is one of the top superconducting qubits people in the world to build a group that was going to build like a 50 or 60 or 70 qubit superconducting chip, which would go into a dilution refrigerator. That's that upside down wedding cake over there. And be cooled to about 10 millikelvin so that the qubits in it actually behave as qubits. And then you have 50 or 60 qubits arranged, let's say, in a rectangular lattice with controllable nearest neighbor couplings. So you can do two qubit gates between the neighboring pairs. And now they face the question, well, once we build this, what should we do with it? And they looked around, and the only thing that seemed like you could be pretty confident that it would be a classical computer was, well, something like boson sampling. But their system was not designed for bosonic excitations. It was based on superconducting qubits. So they asked us, can you just adapt the theory to that setting? To, let's say, applying a random quantum circuit to a collection of qubits, that will be easier than spending 100 million or whatever to redesign the experiment. So we said, OK, we'll do that. We are indeed cheaper. And so then Liji Chen and I had some work, evidence for the hardness of simulating that in 2017. And then in 2019, they reported that they had actually done this experiment using a chip called sycamore with 53 qubits. Actually, 54, but one of them didn't work. And they sampled repeatedly from some probability distribution over 53-bit strings. And at the time, they estimated that the best classical algorithm that they knew running on a supercomputer would take 10,000 years to produce the same samples. And then that number got endlessly repeated in the press. But the fundamental problem, you could say, with quantum supremacy experiments is that you can't take any of those numbers too seriously. Because then people will think about it more. And they'll say, oh, actually there's a better classical algorithm. And the time will be whittled down, let's say, to a few weeks. And then if you can do something classically in a few weeks and quantumly in three minutes, OK, well, then maybe that's still a quantum advantage. But that's less than you thought. And then you say, oh, but actually if I just spend more money to use more and more cloud resources, then I can just parallelize my classical effort. And if I spend enough money, I can do that in two minutes. I can do that faster than the quantum computer. And now we have to say, well, OK, I guess time is not the only resource we care about. Maybe we also care about electricity costs. Or we care about CO2 emissions or whatever. And by those metrics, I think, compared to the best current classical algorithms, we still have some kind of quantum advantage. But maybe it's a factor of 100 or so. It is hanging by a thin thread. So that already kind of indicates the need for doing better experiments. And this is before we're even talking about anything useful. This is just establishing the reality of any quantum speed ups at all. But I think the issue is even, and the hardware has improved. And USTC has also done superconducting experiments with 60 qubits and the two-qubit gate fidelities, which is maybe the most important number. Those have improved since 2019. And you could do a better quantum supremacy experiment today. Most of the major labs seem to have moved on from this. They seem like they feel like this is a done deal. Now it's just fault tolerance or bust. I want to make the case to you that this is not a done deal. Yeah. Yeah, well, this was Richard Borchards who had that metaphor. And I had a blog post to sort of, I mean, he was treating that as a reductio ad absurdum, basically, of this entire approach. So the point of my blog post was to explain why his analogy doesn't really work. Basically, what I said was that what you want to know is sort of that the difficulty has to do with some sort of asymptotic scaling. You want to know that there is something that is sort of increasing exponentially with the size of your problem. In the case of the T-cups, the issue seems to be merely that, well, OK, number one, there's an issue of Avogadro's number, that you have this large but constant number of particles. And maybe a simulation would have to keep track of all of them. And the best computer to simulate the dynamics of your T-cup really is just the T-cup itself, which you think of as an analog computer. You drop it. And actually, I tried it at home with my kids. We dropped a bunch of T-cups, and we plotted a distribution of how many pieces. I mean, it was very ambiguous what counts as a piece, because they get really, really tiny. But we tried to count how many pieces and see if we could learn something about the distribution. But the other difficulty there is that a large part of the problem is just that you don't have sufficient knowledge of the initial state. It's just that you don't know the initial conditions. Whereas here, what we're trying to put our finger on is that you can know with, for example, the Google Quantum Supremacy Experiment, you can know the initial conditions perfectly. You can know the circuit perfectly. And yet it would still be exponentially hard to generate the samples for a completely different reason than in the T-cup case, right? Not because of chaos and the initial conditions, but because of the exponentiality of the Hilbert space, right? So that's kind of the difference between the cases. And it is not the same as, but it's related to the fact that these are programmable devices, right? You can return your qubits to the old zero state and then you can return to applying the same circuit that you did before, right? Up to small inaccuracies and calibration, but those small inaccuracies are not the reason why it's hard to simulate, right? The reason why it's hard to simulate is just the enormous size of the quantum states. All right, so to say it in a little bit more detail, the random circuit proposal is basically that you, the classical skeptic, challenge a quantum computer by sending it a randomly generated quantum circuit, call it C, on some number of qubits N, and then you demand that the quantum computer send back to you quickly a list of samples, call them S1 up to SK, which in the ideal case, would be drawn independently from probability distribution that I'll call D sub C, okay? Which is just whatever distribution you get by starting with all qubits in the zero state, applying C and then measuring each of the qubits in the zero one basis, okay? So there's this sort of tricky conceptual distinction here, which is that the problem is not simulate this device or simulate the physics of this particular device. If that was the problem, then by definition, the device could never get it wrong, right? Just like when you drop the T cup, right? It never fails to simulate that T cup, right? But instead, we are giving a mathematical specification of what we want, okay? Such that you could easily imagine that the quantum device would fail to solve the problem, right? The fact that the quantum device correctly solves this is itself something that has to be experimentally checked. Okay? So you get K samples. So imagine that N is 53 here. Imagine that K is a few million, let's say, okay? The nice thing about superconducting qubits is that they're very fast and you can get each sample in something like, you know, 40 microseconds, okay? So in a few minutes, you can get millions of samples. Now, what does the random circuit look like? Well, basically, you know, it has these two qubit gates which are staggered in such a way that every qubit can influence every other one, right? So we want, you know, scrambling to happen, which means, you know, we need horizontally and vertically adjacent qubits. You know, we need them in a pattern where information is getting across. And in Google's experiment, the depth of the circuit was 20, okay?  And now we face the really crucial question, which is once you've measured and you've gotten these outputs, how do you verify that the quantum computer did anything interesting? So what Google did was, you know, one of the simplest things that you could imagine, which they called the Linear Cross Entropy Benchmark, or LXEB, okay? Even though it's not really about entropy at all. I don't know why they called it that. Okay, so, but here's what it is, okay? So you, using your knowledge of the circuit C and using a classical computer, where, you know, which is assumed to, you know, be able to do two to the n classical computation. And, you know, we'll come back to that point. Okay, you calculate what would have been the probability for an ideal quantum computer to have outputted each of the samples that you saw, okay? That's these squared matrix elements there, okay? And you do that for each of the S sub i's that was observed, okay? And then, you know, that gives you the sort of predicted probability for each of the outputs that you observed, and then you add all those numbers up, okay? And, and then you, you consider the quantum computer to have passed the test, if and only if that sum exceeds a certain threshold, okay? So what's going on here? Well, if we imagine that you just picked the S sub i's uniformly at random, just, you know, with, you know, not needing a quantum computer at all, then we would expect that, you know, since these probabilities have to add up to one and there's two to the n of them, you know, on average they're gonna be two to the minus n, right? And if so, if I add up k of them, then I would get k over two to the n. That would be my expected value for this sum, okay? But if the quantum computer is really working properly, then the sum ought to be larger than that, okay? Why? Because, well, you know, the outcomes with the higher probabilities should be occurring more frequently, right? That's what it means to be, you know, doing sampling in the correct way. So how much larger should it be? Well, if you do the calculation, you find so, so the amplitudes can be approximated as just independent complex calcians with means of zero, which means that the probabilities can be approximated as exponentially distributed random variables. So like, you know, on average they're two to the minus n, but they have little wiggles in them. Some of them are two over two to the n, or three over two to the n, or a half over two to the n. They're, you know, exponentially distributed. And if the quantum computer is working right, you should be preferentially seeing the heavier ones.   And the answer you get is two over two to the n. So precisely twice the classical value, okay? And so now you have a situation that's very closely analogous to the Bell inequality, okay, for those who have seen that. It's like you have some number where, you know, you believe that any classical theory could achieve at most a certain number, and an ideal quantum device would achieve some larger number, right? And the goal with your experiment, well, your, you know, your device is not ideal, so you're not gonna achieve that larger number. In this case, two, you know, a b equals two in that expression there. Your goal is just to get any value of b that is larger than one, right? That beats what you expect is the classical bound. And what Liji Chen and I did was precisely to give some complexity theoretic reduction, you know, admittedly based on a non-standard assumption that said that, you know, it is unlikely that a classical algorithm, that an efficient classical algorithm could generate samples with a b value grade that is anything bounded above one, okay? So what did Google report in its 2019 experiment? Well, they reported, you know, drumroll, that they achieved a value of b that is 1.002, okay?  So, you know, while it was only a tiny bit more than one, it definitely was more than one, okay? Yeah. I mean, it is the simplest test that I could think of also. Well, we had this in the paper with Liji. We used something else called a heavy output generation, hog, but, you know, that one, like it involved this like two thresholds, whereas here you only need one threshold. And so, you know, actually once I saw this test, I was like, yeah, this seems simpler. And actually I had an analysis of, you know, like, you know, how to generate certified random bits from quantum supremacy experiments that was just in stock a month ago.  It's, you know, it leads to the, by far the simplest analysis. So that's one thing that made me warm on this test.   Like that's not, you know, you just want kind of the simplest thing that's going to differentiate the quantum sampling from classical sampling. And that just calculate all the probabilities and add them all up. So, yeah, I mean, you know, we're gonna see, like, what are the drawbacks of this whole paradigm? But I think that, to be honest, that the drawbacks would apply equally to this linear cross entropy, or to just about any other similar test that you could come up with. If there's a counter example, then I'd love to hear about that offline. Okay, so that brings me to, well, what is wrong with the current experiments?  They are not error corrected at all. And because they're not error corrected, you know, we are extremely limited in what can be done. In Google's experiment, each two qubit gate had a fidelity of about 99.5%, okay? And that's, you know, that's amazing compared to what was achievable 10 or 20 years ago. But it still means that if you apply 1,000 two qubit gates in sequence, well, your total circuit fidelity is gonna go like 0.995 to the 1,000 power, or something like that, right?  Now, you know, and this is the reason why they only got an LXCB score of 1.002, right? It's just because, you know, now, maybe scientifically, the most important thing that we learned from the Google experiment was that the circuit fidelity seems to only decrease in that way and not in any worse way, okay?  That prediction was tested and in these experiments, it was wrong, okay?  So, but, you know, now USTC has also done this kind of experiment. So, you know, I think, you know, that, you know, that position might become increasingly difficult for him.   But in the meantime, you know, before we can get to fault tolerance, it means that there's sort of no hope of scaling these experiments up to hundreds of qubits or, you know, thousands of gates because the signal will just be too tiny to detect. Like, the distribution that we're sampling from basically will get, you know, exponentially closer and closer to the uniform distribution, okay? And then, you know, the deviations from uniformity will get tinier and tinier and, you know, we'll need exponentially many samples in order to even notice those deviations. You know, and then there goes any hope of a quantum speedup. Okay, but, you know, the even more immediate problem is that, you know, there are taking advantage of the fact that the gates are spatially local, you know, in superconducting qubits or in the case of Boson sampling that many of the photons are lost. Okay, so taking advantage of the imperfections of the existing experiments turns out to allow classical simulations of these experiments, spoofing attacks that are much more efficient than we would like. Okay, so some of the main attacks, Pan-Chen and Zhang have shown how to do tensor network contractions where, you know, as I said, it just, you know, if you wanna do it on a classical computer, faster than the Google experiment, it just comes down to a question of money and electricity at the end of the day. And there's another classical algorithm by a Gal et al, which is actually polynomial time, so it just runs on a laptop, and it achieves a linear cross entropy score with Google's parameters that's like 1.0002. Okay, so it gets about one-tenth of the excess that Google got in its experiment. So, you know, you can see the experiment is still winning, but only by one order of magnitude, right? If they improve their algorithm, then, you know, the quantum advantage could just go away entirely. Okay, but none of this even gets to what I see as the most fundamental issue, which is that, you know, in order to verify the results of these experiments at all, we had to calculate this linear cross entropy number with our classical computer, and that took two to the end time, right? Classic, or, you know, you could say the time that is needed for verification is directly tied to the time that a classical person would need to spoof the results, right? They both involve sort of calculating, you know, the same basic numbers, right? And so what that means is that there is no hope of scaling these experiments who just completely beyond the range where any classical computer could hope to compete, like where it really would take, you know, billions of years or whatever, because if we went to that range, then we couldn't even verify that the experiments were doing what they were supposed to do, right? So, you know, now Shor's algorithm doesn't have this problem, right? You know, because the factoring problem is in the complexity class NP, right? If you, you know, it might be very hard to find the factors, but once you've found them, they're easy to multiply, and actually also easy to check that they're prime. But, you know, with these experiments, the difficulty of classical spoofing seems directly tied to the difficulty of classical verification. And that is what I want to get around while remaining, you know, in the realm of things that we can implement on current quantum computers, okay? So, this one slide here kind of summarizes how I think about the present situation in quantum algorithms, okay? There are basically three goals, three things that we want our quantum speed up to achieve. The first is that it should be niskey, okay? That's not a technical term, but it should be implementable on current devices, or current or near future devices, okay? And roughly speaking, anything that involves doing, you know, arithmetic on superpositions over integers that are encoded in binary as ensures algorithm or as in all sorts of recent cryptographic protocols is not niskey, okay? That, you know, we have no hope of doing any of that with current devices, okay? Something that looks more like a random circuit or maybe like some physics simulations, that has a chance of being niskey. The second thing we want is of course, at least in principle, there should be a quantum advantage compared to the best known classical algorithm, okay? Both, you know, concretely, you know, in terms of like actual numbers, you know, with our device compared to the best classical computers, but also asymptotically, right? We want to see that as we scaled to more and more qubits that, you know, ideally we're beating the best classical algorithm by an exponentially scaling amount, right? We don't want to be in the T-cup situation. And then the third requirement, as I said, is that we ought to be able to efficiently verify the outputs of the experiment using our classical computer, okay, without having to take classical exponential time as with random circuit sampling, okay? And I would say that the current situation is that we know how to achieve any two of these three goals. Okay, so if you want something that's niskey and that's efficiently verifiable, I mean the whole field of quantum machine learning, right? And like these optimization methods, like QAOA and VQE, right? These are all giving you candidates for that. Okay, the trouble is despite thousands of papers that have been written now about these kinds of heuristic optimization algorithms, we still have no clear case that any of them are beating a classical computer in a fair comparison, okay? For any optimization or machine learning problem, right? And, you know, until that question is addressed head on, it doesn't matter how many more papers are written about these algorithms, right? Okay, if you want in principle quantum advantage and efficiently verifiable, I mean that goes back to the beginning of the field, Schwarz's algorithm is a candidate for that.    It would be enough to do x squared mod n, you know, on a f of x equals x squared mod n on a superposition of x's. And then you can use that in a cryptographic protocol to prove quantum advantage, which has, you know, they have this beautiful argument that I loved at first sight, where here's what they say. They say, look, either, you know, you can't break, you know, this cryptosystem of like squaring modular composite number, either, you know, that cryptosystem is secure and you can't break it. And in that case, their protocol is sound and, you know, you prove quantum advantage to the classical skeptic, or else the cryptosystem is broken. And in that case, you've proven quantum advantage in a completely different way.  Okay, so, okay, and then if you want something that's both Niskey and that gets an in-principle quantum advantage, but that's not efficiently verifiable, then that's exactly the current generation of sampling-based quantum supremacy experiments, such as random circuit sampling and boson sampling. Okay, so the challenge, obviously, is to get to the center of this Venn diagram, either by starting with these heuristic optimization methods and getting, you know, a clear quantum advantage out of them, or by starting with these, you know, cryptographic protocols or things based on lattices and number theory and figuring out how to implement one of them on a NISC device. You know, good luck with that. Or, you know, and this is the direction that I've personally thought about the most, starting with the kinds of quantum supremacy experiments that we know how to do, and then figuring out a way to make their outputs efficiently checkable with a classical computer. Okay, so how might we achieve that third arrow? So I wanna give you, so I don't have an answer. If I had the answer, then that's what the talk would be about. Okay, but I'm at least gonna give you a pretty clear open question that I think directly bears on this. Okay, so here's my open question. So given a quantum circuit C, let me define C to be peaked. If it has the property that when you run C on the all zero initial state, and then you measure the output in the standard basis, there is some particular basis state, call it X, that occurs with a really large probability. By really large, let's say we mean at least a tenth. Okay, now we could generalize this definition. To the output state of C having any properties at all that we can feasibly detect by measuring that state, like better than just by running C inverse, and seeing that we get back to the all zero state, more efficiently than that. I wanted to have some property, it could be in the Hadamard basis, it could be biased in some way that is easy for us to detect. Okay, that's what I want. Okay, and now I'm gonna imagine the following thought experiment. Okay, we're going to pick a random quantum circuit, just like in the quantum supremacy experiments, with let's say N qubits and some number M of gates. Let's say M is N squared, or at some other polynomial in N. Okay, but now we're going to throw out the circuit unless it is peaked. Okay, we're going to condition on the event that our random circuit is peaked. Okay, so now we have a random peaked circuit. And now my question is what can we say about these random peaked circuits? What do they look like? Okay, and let me give you two extreme possibilities, neither of which I know how to prove as impossible. Okay, the first extreme possibility would be that a random peaked circuit just looks like a whole bunch of random gates followed by the inverses of those gates. Right, so if I have U and then U inverse, well of course that is peaked because it just computes the identity. Which means that the all zero string will have a probability one of being output. So, or more generally, the random peaked circuit could be such that when I do a whole bunch of cancellations with my classical computer, I reduce the circuit to something trivial, right? And like I easily see with my classical computer that this is a sort of trivial circuit, not hard to simulate. Now the other extreme possibility would be that a random peaked circuit is indistinguishable by any efficient classical algorithm from just a fully random quantum circuit, right? They look identical to any classical test that I can apply. And nevertheless, I can take the peaked circuit, put it on my quantum computer, and then I'll see the difference, right? I'll see that there's some output that it's producing at least a 10th of the time, okay? So in this second case, I think that what we would have established would be that the kinds of quantum circuits that we want for verifiable NISC quantum supremacy at least exists, okay? We still would not have answered the question of how to find those circuits efficiently, right? We would still have that problem, but at least the circuits would platonically be out there, okay? You know, circuits that sort of look random, look like the things that we can run on current devices, but secretly that are obfuscatedly, they are concentrating the amplitude on some particular output that we can recognize at the end. Okay, and one can ask a whole bunch of related questions. So here's one of my favorites. Given an N qubit, M gate, a random quantum circuit, let's say that you get to add more gates onto the end of that circuit, okay? In order to get an overall circuit that is piqued. Okay, then how many more gates must you add, okay? So I claim a clear upper bound here is M, right? You know, if you can add M more gates, then you just add, you know, the inverse of the random circuit, and then of course you have something piqued. Hey, but can you do it with substantially fewer than M additional gates? Even let's say what, with M over two additional gates. Now using recent results about random quantum circuits, you know, giving you T designs, it is possible to prove that the number of gates you have to add is at least like M to the one fifth power or something like that, okay? But that's, you know, that leaves an enormous room between the upper bound and the lower bound. One could also ask, you know, what fraction of quantum circuits are piqued? Again, I have upper and lower bounds, which are pretty easy, but which are far from matching each other. And then of course how hard is it to sample a random piqued circuit? Or what about a piqued circuit that is hard to distinguish from random, okay? That looks random. So I did work with a recently graduated student at UT named Yuxuan Zhang, and we have at least some preliminary empirical data, okay, albeit not a theorem. So what we did is we generated random quantum circuits with different numbers of qubits, you know, eight, 10, 12, and so forth. That's the x-axis. And then we looked at, we fixed some number of gates, you know, let's say, you know, the random circuit has maybe 100 gates. And then we try to add either like 50 or 33 or 25 gates onto the end of that circuit to get an overall circuit that is as piqued as possible, okay, and we just use local search, you know, we use like a heuristic optimization to try to search for the circuit that is piqued. And so what we found is that, you know, actually much more piqued circuits than I would have guessed exist, okay? So we find, for example, when there are 12 qubits, you know, you can take 100 random gates and you can add 50 more gates onto the end of it.  How is that happening? I don't know. You know, if the, if our 100 gate circuit, you know, if it had instead generated a hard random state, then it is easy to check that this would not have been possible, okay? Which means that whatever the last 50 gates are doing, they are exploiting some sort of structure in the output of our 100 gate circuit that is not shared by a hard random state, okay? But what is that structure? We have no idea. We've examined the circuits, you know, they look random to us. We don't see any pattern. And yet clearly there is something, right? They, you know, they end up, okay, that's the good news for verifiable NISC supremacy. The bad news is that the amount of piquedness that we're able to find seems to be decreasing exponentially with the number of qubits, okay? So it might be that there's a surprising thing here. You know, maybe even usable at the level of 50 qubits or 60 qubits, but at least from the numerical data, it's not looking like it would scale to hundreds of qubits. But, you know, again, we don't know or maybe our solver is just not working well enough. So what if the suitable piqued circuits just don't exist or they do exist but they can't be found efficiently? You know, I heard just a month ago, I heard a really cute idea for another type of quantum supremacy experiment that we can do, which is that we could just take the standard random circuit sampling experiments and run over and over, you know, until we start seeing collisions, right? Which that should not take two to the end time because of the birthday paradox.  Likewise, we should only need to order two to the end over two samples until we start seeing collisions. And with those collisions, we could start doing statistics that can differentiate or the distribution that we're sampling from the uniform distribution. So, you know, with Andrea Mari, we ran some numbers and it looks like you might be able to do this by taking the current superconducting devices and just running them for like a few months. That, you know, maybe that's worth doing, I don't know. So, let me conclude. You know, like, you know, in some sense, the real question I'm trying to ask here goes all the way back to the beginning of quantum computing theory. Okay, we're asking sort of how generic are quantum speedups?  You know that a random circuit can give you a speedup in some sense, but it's, you know, it might take you exponential time just to verify that you achieved the speedup, right? So, can you get the best of both worlds? Can you have a circuit that looks basically random and it nevertheless, you know, choreographs a pattern of interference, you know, in a way to achieve a huge and verifiable speedup over any classical algorithm? Or is there some law of conservation of weirdness that says that like any quantum speedup, if you want it to be exponential, you know, it has to be for some really weird task with some really weird circuit and there's no free lunch. So, you know, we now urgently need to know just how non-weird can we make it. All right, so thanks. Thanks very much, Scott. So we have time for a few questions, comments. Yeah, please. Oh, yes. Why? Well, because I can't rule out the possibility that there might be interesting physics simulations, which in this case might be verifiable not because the problems are in NP, which is, you know, the computer science reason, but rather for the physics reason that we could take the output of our quantum computer and compare it to experimental reality. Like, you know, for estimating the rate of a chemical reaction, we can compare that to the, you know, data from the lab. What is the rate of that chemical reaction, right? And we can validate the results of our physical simulation in that sort of way. You know, that's the hope, anyway. You know, the big question is whether you can do that with a NISC device and in a way that actually beats the best that you can do with a classical computer.  Good, anything else? Yeah. Thank you. I want to follow up on the random circuit sampling. And I wonder when you and your student were doing these experiments, do you do the Clifford Plus T random circuit sampling or some very special class of quantum circuits? Let's see, I'm trying to remember which basis of gates we used. Yeah, I mean, my guess would be that it doesn't matter very much, but, you know, it's possible that it would matter. I think that, I think it was like C knots in random one qubit gates, but I could check that and get back to you. Right, the reason I'm wondering this is, so in some cases, a restricted set of Clifford Plus T circuits is also very interesting. For example, the Toffley-Hagmar circuits, because they correspond to some very nice group structure that we could leverage. So I wonder, is it possible in order for us to understand about this problem of peak circuit, is it possible to look at it from a number of theoretic perspective and say, well, I want to get this group element like from, let's say, the orthogonal matrix over diatic fractions. Then how far am I from this current matrix to this one specific matrix? So I cannot rule out the possibility that the gate set might matter for that random peak circuit's problem.  My intuition would be that probably like the gate set matters if you care about which unitaries can be exactly generated or so forth. And if you just care about these approximate questions, like do you have some basis state that's output with at least the 10th probability, then my guess is that the answer should just be universal. It shouldn't really depend on the set of gates. But you know, I don't have a proof of that. I could be wrong. Thank you. Anything else? Yeah. Well, it wouldn't mean that you have to calculate the probabilities, not necessarily, right? Like, you know, someone might equally argue, well, with Schor's algorithm in order to generate a circuit that outputs the factors of this number, don't you have to already know the factors? Okay, well, no you don't because Schor explained why not, right? You know, it might be that there is some way of generating a circuit that looks random, knowing that the circuit is piqued, you know, maybe even knowing what is the special output, okay, but with some additional secret information that would come out of the generation process, but it would not imply some general ability to calculate any of the output probabilities. I don't know if that's possible or not. That's precisely my question. It's just a question of sampling what, okay? For Boson sampling, you're given the beam splitter network and now you have to calculate the output probabilities, okay? In this case, we get to pick the circuit. That's the key difference. I should mention that on Thursday, I'll have a technical talk apparently with yet another approach to near-term quantum supremacy which uses the time-honored trick of completely changing the question. Okay, one last question maybe, yeah.  This is a question that might be answerable, you know, using tools from information theory or analysis of entropy or things like that. I don't know, that's for any of you to try to solve while I go and deal with the robot apocalypse. Okay, and on that note, I think why don't we, thanks, Scott, and have a nice talk.


My name is Matt Shannon. I work for KPMG as part of the Information Risk Management Group. We do penetration testing, vulnerability assessments, general security work. What I'm going to talk about today is Microsoft Front Page. And I'm going to cover a couple of different topic areas. It's a pretty heavily deployed tool that's not very well known. I'm going to cover the protocol, the back end systems. I'm going to cover a simple brute force authentication scanning program I put together. And I'm going to talk about what to do when you gain front page access. I'm going to talk about ASP. I'm going to talk about some of the powerful ASP components and tools you can use to gain further access. And then I'm going to wrap it up with how to secure front page. All of you have friends, relatives, girlfriends, clients, others that insist on running front page. This is how you take the extra steps to prevent the people who have attended this conference from doing anything to them. And just to get us all on the same page, before we begin, we're going to talk about a lot of tools and techniques in here. We all know you've got to use these on machines that you have permission to use them on. So get your written permission ahead of time. Let's get started. Lay in the ground front page. Microsoft front page, what is it? God, you have to ask that question. You've been under a rock for the last ten years. It's Microsoft's integrated website development tool. It's a system for adding basic to advanced functionality with little or no web page experience. In addition, it's an integrated Microsoft Office package, and in my personal opinion, the security nightmare. It was built by Vermeer Technologies in early 1995. Vermeer was one of those dot com explosive firms that created front page to capture the web boom. And following an enormous success that were bought out by Microsoft in 1996, the original programming staff still works for Microsoft, although the management's moved on to new endeavors. As a side, all DLLs, all functions within front page start with the under store VTI under store Vermeer Technologies. Decoding the system. We're going to talk a little bit about the HTTP protocol and what's going on behind the scenes when you access it and administer a front page site. It's client server communication, pretty basic. Front page client talks to the front page server extensions over HTTP put requests. The front page client typically makes requests against author, admin, shtml.exe, as well as the VTIRPC commands. We're going to talk about the different DLLs. The admin DLL is typically used for authoring commands, uploading content, downloading content, reviewing properties, setting permissions. The author DLL is used, like I said, for uploading and adding content. The VTIRPC and shtml are typically used for initial, starting your initial session or beginning access. Alright, there's a lot put on those slides. I'm going to try to break it down for you, but you can go back and read it at your leisure. The authentication system is kind of interesting. It authenticates you every single time you do anything. When you first make a connection, it authenticates. When you scan the directory, it authenticates. When you pull up a file, when you upload or download a file, it authenticates. What it essentially does, the first half of the authentication is let's determine if this is actually is a front page-enabled server. So the first request is a client request for underscore VTI, underscore INF, dot html. Basically, this gives this information on server version. In other words, is this client compatible with this version? It gives me locations for the admin, the author, exe's or DLLs, whatever case that may be. Following this, the client takes this information and makes a post to VTI RPC. It's kind of an initial session start post. It's trying to initiate communication. The server will typically respond with a 200 okay. The client then posts to VTI author or admin, depending on the action. In our example here, it's author. In the case that the administration has elected to allow everyone access, at this point, front page will receive a 200 okay, and you will have authenticated.  At this point, the client knows it needs to communicate to the server. It needs to authenticate so it takes its NTLM. In the case of Windows, it takes the NTLM username and password, builds the NTLM challenge, and sends that back to the server. The server will, once again, respond with a 401.3 authorization required. Except this time, it passes back the server side challenge. This point client now knowing its own username and password and having a specific server challenge pushes back the challenge response. It closes with a 200 okay. It's a very simple system, but it happens again and again and again throughout the process of accessing front page. We'll talk a little bit about VTI INF. This does provide all the configuration information. It's embedded in the comments of the VTI INF.html. If you were to open that page, it would just say front page configuration information. What resides inside that, on the other hand, is actually the comments and they show the location of the file. They show the location of the server, version information, things like that. Alright, like I said before, VTI INF actually gives us a lot of extra information about the server itself. It tells us, believe it or not, tells us the operating system in most cases. The .dls and .exe's can help us determine that. Anytime the VTI INF contains author or admin .exe, nine out of ten times you're dealing with an Apache or a Unix box. Dls, of course, come on the Windows sign. More importantly, the version information can help us narrow down the OS version. This is important for other attacks. I put together a version information table. I was not able to obtain a Windows ME personal server version information. I apologize if the formatting got munged. Windows NT, typically it is a 4.x version. Pretty easy to remember. Windows 2000 and Windows XP both fall under a 5x version, although it's most often than not a much lower subversion number for 2K. It's a 5 for Windows 5.2 versus 5.6 or 5.8 on XP. I didn't create the name. That's Vermeer. But Vermeer created, they call their packets that they cast back and forth, Vermeer RPC packets. Vermeer RPC packets resemble web services. They resemble encoded HTML with variables. In my opinion, it might have been an early precursor to XML. The following is a sample Vermeer RPC packet. It contains a large amount of information. It's a little clearer, I'm sure, on your CD than it is up here. What you'll see embedded down in there is a physical drive path to a file you attempt to download. It's pretty much a big no-no. You really don't want to keep a complete physical drive path to the file. It's possible to use that kind of information for future Unicode exploits. In addition, if you climb in and start reviewing the different information, you'll find a lot of different functions or features, revision level, additional authorizations and permissions, all embedded in those strings. It would be interesting, and it would make multiple attempts to modify different values. I think there are still additional exploits to be had there. Let's talk about the back-end for a minute. One of the more interesting things we were able to do was download a free copy of Microsoft's Windows Debugger, the WinDebug tool. It's a free tool. Microsoft gives it away. Attach that to a back-end running finite info process. Listen as you connect to and make a front-page session to watch what DLLs are being used and what functions the DLLs are calling. You can download WinDebug from Microsoft's debugging site, which I've included here. When you download this, you will also want to enable it to use Microsoft's symbol server. It'll help you translate what functions they're calling. Both the access to the symbol server as well as Microsoft's debugging tool are completely free and a really easy way to get an understanding of what's going on under the hood. Front-page server extensions use, as a partial list of the front-page server extension DLLs, the off-L and AWEL DLLs you'll find are used more often than any of the others. The author and admin DLLs are almost contained stub-like functions that are called and that are used to call functions in the earlier DLLs. I don't know the level of debugging skills, but debugging is an art as well as a science and it takes time to understand it. There are multiple guides and tutorials available on the Internet. A couple of quick Google searches and points you to dozens. WinDebug's got its own set of help files. But I included here the simple commands I used when I was getting started. You can use the VM to set break points and it's kind of interesting. You can give it the DLL and a global variable like star. In other words, in the example I did, fp4awel bang star, it'll essentially set a break point for every single function in that DLL. Now, that's going to hurt because you're going to be stepping through that again and again and again. However, what you're going to get is a list of three, four hundred functions. You can immediately pinpoint and narrow down the functions that seem most interesting to you, such as set permission, get permission, check authorization, things like that. bc to clear those break points.  She is, of course, go. Once you hit a break, you can restart it with go. He pauses the activity. All right. What did I discover when I went through it? This is just a very brief list, but one of the things that I hit on was the fpawel service actually forms a listing of all documents in the defined web prior to accepting a request. It was my hope that it was coded a little more poorly, that it would write for your request, then go and pull the file and retrieve it for you. However, the server does pre-query the site and then create an array, a stack array listing of these values, and then it will reply with what you, it will search this list when you make a request. However, is there still an opportunity to break out this list? Are there opportunities for the buffer to be overrun? Sure there are. DSP context get file is another one of the get file and get document. They're called when downloading from front page. Set your break points to these and you can step through and figure out exactly where that array is getting called and whether or not there are opportunities for buffer overruns. NTDLL is used exclusively in the authorization process, and we all know that there were attack vectors on that. Microsoft came out and said they will not patch for NT4. If you do the same attack vectors applying via front page, entirely possible. Okay. Taking some of this background information, I learned about front page through my job. I went ahead and put together automated tools to grind through accounts, to grind through passwords, to grind through client front page enabled servers. FP exploiter was what came out of it. It's a really, it's a simple tool. It's a front page password grinding and vulnerability scanner built as both command line and Perl GTK. In other words, now that you have a better understanding of how front page works, let's find out about finding vulnerable targets. This is what brought about the tool. The tool locates front page accessible web servers, uses default options, in other words, tests for the everyone account, then follows up using user defined accounts and passwords. It comes with a default password list. If I recall, it's one of the ones that the, one of the NetBios forums uses. It's a typical simple password list. It's a Perl GTK and a command line tool. When you open it under Linux, you will see a window pretty similar to this. The scanner works by you provided a class C range. You set partners. Yeah, I could definitely have made it a lot larger than that, but most clients only have a class C network and we didn't want to help the script piece out any more than I had to. You can also set the password list. Like I said before, the password list is created initially with a roughly 25 or 30 standard administrator passwords. Very simple things. Password 1, 2, 3, add root. The password list can be modified. Now, keep in mind that using an enormous password dictionary, something on the tune of tens of thousands of passwords, will chew up your memory very quickly. The default account is set to administrator. This was done on purpose. The administrator account does not lock out. If the client has set, has a, has a better account for you to use. If you know that their web, their web administrator uses the account webmin or webadmin or mic, then that would be, of course, the one you would use. Keep in mind that those accounts can be set to be locked out. Again, you can, the tool allows you to save the log. It wouldn't be very useful to me if all I could do was look at it on the screen. So you save the log to text, print it up and add it to your scanning reports. Screen capture during operation. There are more interesting ones on the website. Now, what I didn't cover is that the tool is built for and only runs against IIS-based web servers. I didn't make any modifications to support front-page Apache. The authentication system and the processes are very different. Front-page Apache uses DES or a crypt function on the Unix sign. However, they do store their passwords in individual files. Many times these password files are accessible. So I wouldn't be too surprised if in the next three to six weeks you happen to see a version that supports Apache up on the website. There's also an opportunity to redesign it for generic front-page access. Perhaps the open office guys, if they're listening, are interested in adding front-page-based access to the open office suite. And I'm all for rebuilding it something a little faster than Pearl GTK. Support for uploading content is very close. There's a few Unicode-based characters that Microsoft puts in, but once all of that's been cleared up, I don't see it taking very long. All the code is available at the website. It's on your DEF CON CD, but I recommend you go to the website tomorrow. The code will be unlocked once I've got access to a console. I recommend you go and download that if you plan on using it. That's where the latest code will be. So let's say you gain access to front-page and you're looking at the next steps. In other words, where do you go from here? In a little session we're going to call ASP for hackers. Active Server pages have been around for quite a while. And if you ever take the time to develop any web-based content, ASP is much like PHP. It gives you a lot of options. It gives you a lot of areas you can exploit. You can build some rather impressive administration and control systems straight from a web page. One of the first tools I'm going to talk about is a tool I discovered out on ASP Alliance. Michael Brinkley developed it. Real nice guy. His tool set is rather extensive. It's a SQL Server database tool. If you've ever administered or accessed SQL Server, you've gone and been using Query Analyzer. Most of us folks are command line-driven types. And Query Analyzer allows you to go in and execute all of the administrative queries you would want to do at an account, removing users, analyzing the data. What SQL Ultimate allows you to do is it allows you to run a Query Analyzer session directly out of your web browser. Once you've used your front-page exploiter to find a vulnerable server, you've logged in and uploaded SQL Ultimate. You'll find you're able to access, administer, and control internal SQL Server databases. As most administrators do place the ASA account embedded in the ASP code already on their site. SQL Analyzer, to give you some suggestions, some interesting things to try, adding SQL users is very simple. Most of the time you are part of the ASA role. Accessing corporate data, accessing the back-end to web applications can also be very powerful. You can execute extended store procedures as well as locate application accounts and passwords. Here's a screen capture of SQL Ultimate. You'll see the query window on the left and the resulting window on the right. Go ahead, go over to ASPAlliance.com and check out the stuff that was put together by Michael. He's really done a lot of great work. He was very amenable to me putting it up here. Like I said, there's the link for the ASP Alliance site. Let's talk about command line ASP. This is an old one. It's been out for quite a while, but there are still some folks that don't know about it. I'm going to butcher this poor guy's name or girl's name. Maseo is what I'm going to try. This tool allows for the execution of simple commands on the command line. It's just that basic. Some of the more interesting things you can do with it is connect to netstat minus a, create your own makeshift reverse DNS lookup. Do a netstat minus a, get the names, do a netstat minus n, get the numbers, match the ports, and you've got kind of a makeshift reverse DNS. You can do an IP config, you can do a ver, you can do a set users, you can net users, net local group. Remember that the account permissions that you have typically on a site are when you, when you crack the site or something like local system. However, if you built your own front page site and you set that site to require authentication, the account you log in with is the account you execute these as. Front page explorer helps us find vulnerable web servers. Command ASP.asp contains custom code to execute simple console commands. Now like I said before, you can use a netstat minus a or a netstat minus a n to make a makeshift reverse DNS. You can use net local group and net view to understand drive mappings in groups. It's a very, very powerful option. In fact, many clients in the past have had their web server mapped to multiple internal file servers to make retrieving documents on the website much easier. As well as using ping and defined additional servers. There's a screen capture of the command ASP tool in action. It's a very simple punch in the command and hit go and you'll have the results passed back to you in HTML. The code is not available at my site, of course, it's available at secureteam.com. ASP code can be used in conjunction with Winsot controls to provide scanning from the web server. What I'm talking about here is there are a lot of other options. I've just mentioned too, there's a lot of other things you can use ASP code for. You can upload controls, Microsoft, as well as a lot of other companies have developed web-based controls to do any manner of things and one of which would be a Winsot control perhaps programmatically scan the internal network. There's some really fascinating work being done with ASP.net for Samba shares and remote administration. You can use that tool to reach out and do the same sorts of right-click and administer sort of activities you would do on the internal network. In addition, you can use the ASP code and an XML HTTP control to proxy-surf the intranet. This was a previous client. This was beneficial as the client had mounted web-based cameras in their intranet to provide security monitoring for their employees. So it was very interesting to double-watch their employees go about their business. We're going to close a little early today, I think. I don't want to get the attorneys mad at me. We're going to talk about holding down the cord. How do you secure a front page? You can use simple best practices. As in most systems, your first, last, and primary line of defense is your password. The front page really is no different. You want to choose strong passwords consisting of upper and lower case characters, numbers, special characters. Let's make it something a little more difficult than dog, cat, password, girlfriend's name. I prefer using passwords over eight characters in length. We can go into a lengthy discussion about, well, it only uses the first seven. If you have a 14-character password, it's really like two. That being said, passwords, eight characters in length, typically they're easy enough to remember. You have to remember your own phone number and characters in length, so what's one more? Change the admin account. Contrary to popular belief, Microsoft, actually I encourage you to change the administrator. The name of the administrator account, it will not break all functions. It won't cause the machine to explode. Choose a meaningful admin account, but avoid typical choices like root, admin, supervisor, boss. Concept of least privilege. Only provide the access necessary to get the job done. Use front page roles to assign users author rights versus admin rights. In fact, if there is no reason to have the administrator account have front page access, then pull it. Within front page, there is a, the front page has its own set of permissions. Use the front page permissions tool to add or remove accounts as needed. Pull the administrator account or reserve it for, reserve the front page administration rights for specific accounts. Your best option, and this is my personal favorite, is remind your employees, spending another girlfriend, family members, that you don't use front page from just anywhere.  By that, I recommend using the IP based restrictions that are native to IIS. Go and take the admin, author and VTIRPC files and set those to only be allowed access from internal IP addresses only. Consider segmenting your web developers onto a separate segment of your network and allowing that group access to those DLLs only. All right. There's the site. I don't have a lot of artistic talent unlike some of the other folks here. So it's pretty plain, but it contains the code and the commentary and all of the tools packaged both in typical tar GZIPs or RPMs. I want to thank some of the guys who came before me.  Some of them are more pertinent than others at this point, but I've included a link to one of them there. The guy's over at the Pearl GTK tutorial. It's been a long time. It was good to get up to speed. And Microsoft put together a pretty big site devoted entirely to front page. It tends to rotate and move around, so you got to pay attention to where it is. But they've got a lot of the back end processes mapped. They talk a lot about some of the VTI information that gets passed back and forth. We'll close with the thanks. Of course, my lovely wife, my boss. It's a shameless attempt to get a race. The graphics designer who did the image, Mike D'Andrea. He's a President Blunderhead. He has a lot of this kind of stuff. I could never have come up with anything like that. Steve Fickle, who did the technical review and the guys over at the GTK tutorial put up with my emails. All right, if you've got any questions. Otherwise, I do want to cede the pool, but the best question does happen to get one of these cellular PC cards. They won't let me keep it, so I have to give it out to someone with an informative and entertaining question. You're close. The difference is the way the information is passed. Front page has a VTI-specific header. It doesn't use a typical NTL and header, so when you go to a website to log in, say you've got a Microsoft and IS site, and the site has authentication required, the authorization string is passed back and forth. It's just an NTLM authorization on the front page side that got their own VTI headers embedded in that. From the standpoint of technology, it's very similar. Or from the standpoint of the headers and the exact style of the information is passed back and forth. It's different. I wish I could answer that. I haven't had a lot of exposure to the .NET platform. I know that the front page is used on both XP and the 2003 server. I know that the DLS have not changed. They're all the same. The tool really hasn't changed a lot in the last five or six years, and with 32,000 of them deployed on last Google search, it's remarkable that it hasn't been more followed than it has. Then most likely, whatever the same rules would apply. The biggest vulnerability, the question was what's the real problem besides passwords and access with the front page? I think we've got that right. The real problem is it's typically not configured. It's typically deployed right out of the box, and the first person to request access for it someone goes in and sets access to everyone. Immediately, within a few weeks, that's forgotten and the site exists. What's interesting is it's not more heavily scanned or it's not more heavily detected. The IIS scanners, I think, will tell you if there's a front page-enabled site, but they won't go any further than that. They won't tell you if it's got a weak password or missing an account or the everyone account is configured. Yeah. Honestly, the guys I wouldn't worry as much about in a professional opinion, the guys I wouldn't worry as much about are the guys that go in and modify their site. If someone goes and modifies my main page, I'm probably going to figure that out pretty quickly with my wife, a significant other, a family member calls and asks me what the heck this is. The guys I'm worried more about are the ones that slip copies of some of these ASP tools deep down in my site, rename them to make them sound like there's something that should be on my site, and it takes me weeks or a well-configured tripwire-like configuration to even find them. That's where the real power is. That's where the real risk is. Yeah. Can we speak to some of the... Yeah, there are some. In the past, you could go to the VTI underscore CNF file for each directory, and of course it would give you a directory listing that would allow you to execute different content from there. I found, I went through, and one of the links that was in here spoke about a lot of the different, a lot of those older exploits. However, from 2K forward, a lot of those rules do not apply any longer. The majority of the clients I was running into were 2K or beta testing O3. I didn't spend the additional time to go in and review those. If you have a client that's on an NT4 or even for all its instant purposes, an ME or Windows 98, it might be worthwhile to go back and take a look at that. However, a lot of those rules don't apply any longer. From what I saw, most of the older vulnerabilities were information leakage or directory scanning, and remember that what I really want to hit on is, the tool gets configured, installed, and enabled more often than not. 32,000 sites on a Google search is enough to kind of make your head spin. When it's there, more often than not, it's completely misconfigured. What you'll find, I'm sure, in this kind of work, is that it's not so much vulnerabilities in systems as it's vulnerabilities in people's ability to manage systems that cause problems. Alright, that's a good question. Here you go.