I'm going to talk to you about distinguisher-dependent simulation. This is joint work with Avishek Jain, Yael Kalai, and Ron Rothblum. Isn't working? So let me start by telling you about interactive proofs for NP. An interactive proof is a protocol between a prover and a verifier, where the prover has access to an instance and a witness for the membership of this instance in an NP language. And the job of the prover is to convince the verifier that the instance is indeed in the language. The completeness requirement from this protocol, or the correctness requirement, is that if both the prover and the verifier behave correctly, and if the instance is indeed in the language, then the verifier should output except at the end of this protocol with overwhelming probability. The soundness requirement formalizes the notion of security against malicious provers. And we need that for any prover, no matter how it behaves, such a prover should not be able to convince the verifier to accept if the instance is not in the language, meaning that in such a situation, the verifier should reject with overwhelming probability. What about security against malicious verifiers? Or in fact, privacy against such verifiers tries to formalize the requirement that such a verifier should not learn too much information about the witness that the prover used. This is formalized in a variety of ways with different definitions. And the goal of this work is to minimize interaction in such proofs while achieving strong privacy guarantees, which will include many of these, but I'm going to touch upon them later in the talk. For now, I want to focus on the strongest privacy guarantee out of these, which is that of zero knowledge. Roughly, zero knowledge requires that the view of any malicious verifier should be simulatable by a simulator that does not have access to the witness, only has access to the instance and yet is able to generate a view that remains indistinguishable from the view of the verifier when he was interacting with an honest prover. And a canonical application of zero knowledge, for example, is to implement the commit and prove functionality.  This is used in several crypto applications. This or variants of this are used in several crypto applications. But for simplicity, let's consider the situation here, where the proof is just required, where the proof is just required to be a proof that there exists some message such that C1 is a correctly generated commitment to that message. And normally, in order to get some meaningful notion of hiding while giving such a proof, the guarantee that is required from such proof systems is typically zero knowledge. Why is that? Because the whole point of having a commitment there is to hide the message M. And if the proof revealed this message, then that is potentially disastrous. So you'd want to have a proof which hides the message. And the way this is implemented with zero knowledge is that there is a simulator that is able to generate a simulated proof without knowing the message or the randomness used to generate the commitment, such that the view of the verifier remains indistinguishable. And once this is done, since the simulator doesn't need to know the message or randomness that was used, we could rely on the hiding of the commitment to replace it, let's say, with a commitment to zero. And this means that the message M really remains hidden in this situation. And so the goal again, like I already told you, is to minimize interaction while achieving privacy and enable applications such as these. And maybe, potentially, maybe this requires two message zero knowledge. So now let's review what we know about whether zero knowledge is possible or not in two rounds. If we had four rounds or more, there have been constructions for many years now that achieve zero knowledge in four rounds or more based on standard assumptions. On the other hand, if we limited ourselves only to three rounds, then we know how to achieve notions such as witness indistinguishability, which is a weaker notion, or we know how to achieve zero knowledge, but that's based on new and relatively unstudied assumptions and requires non-black box simulation. On the other hand, there have been a bunch of impossibilities that show barriers to achieving zero knowledge or other strong notions of privacy in just two rounds. In particular, we know that the standard notion of zero knowledge is impossible to achieve in just two messages. So what do we know? What can we achieve in just two rounds? Well, so the notion of witness indistinguishability, which is a weakening, is known to be possible to achieve not just in two rounds, but in one round. And the question is, can we do better than witness indistinguishability? Can we get stronger guarantees? And in order to attempt to answer this question, we relax the standard notion of zero knowledge, introduce a bunch of relaxations that allow us to bypass impossibilities while simultaneously sufficing for applications like the one I just demonstrated. So the first relaxation that we are going to make to the notion of zero knowledge is that we will consider the distributional setting where the instance and the witness are sampled by the prover from some public distribution. Moreover, it is easy to jointly sample the instance and witness together. However, given just the instance, it's hard to find a witness. And we require that the indistinguishability of the real proof and the simulated proof hold over the randomness of sampling the instance. In particular, the simulator is going to be able to sample other instances and witnesses from the same distribution. However, the simulated proof that he generates should be for some external instance for which he does not know the witness. And this is important because otherwise simulation would become trivial. The second relaxation that we will add is that we will restrict ourselves to a delayed input setting. Meaning that the input about which we are going to, about which the prover is going to provide a proof will only be decided in the second round, in the second, along with the second message of the prover. And the third relaxation that we will impose will be that we will allow the simulator to depend on the distinguisher. So quantifiers will, so the quantifiers are changed in the following way. We only require that for every distinguisher, there exists a simulator that can potentially depend on the distinguisher and that generates a simulated transcript that is indistinguishable to this distinguisher from a real proof. So these are the three restrictions that we impose. And so finally, to summarize, we're going to be in a setting where the prover samples the instance from a distribution and sends it in the second round and the verifier is restricted so that his first message must be sent before he sees the instance that the prover is about to send. And note that zero knowledge is usually used to prove correctness of cryptographic computation. And in two round protocols, when we use this in applications, we would normally want to use this in a protocol that is itself two rounds. And over there, if the prover is the one that is choosing the instance and proving something about it, in many situations, it makes sense for the prover to just send this in the second round itself. So this will have applications, for example, in the commit and prove scenario that I just talked about where the prover can sample the commitment and send it in the second round. And so now, if some parts of that definition were hard to remember, don't worry about it because I'm going to repeat them when I tell you about the technique. So now let me move on to the technique where we give a way to prove, to give a distinguisher dependent simulator for two message arguments. And to begin with, so we want two message arguments with privacy. And let me start by telling you the three round proof system for NP that has a weak privacy property. This is just the blunt protocol for graph Hamiltonicity. And note that since graph Hamiltonicity is NP complete, a proof for graph Hamiltonicity can be used to generate a proof for all of NP. So the protocol, the details of the protocol don't matter, but all you need to remember is that this is a three message protocol that is honest verifier zero knowledge, meaning that a simulator that knows in advance the challenge of the verifier can actually generate a simulated transcript easily. This particular protocol with a one bit challenge has soundness one half. However, it can be repeated in parallel to amplify soundness and this parallel repetition preserves honest verifier zero knowledge. So this is all you need to know about this protocol. We will use another tool, which is a transformation that was developed by Kali and Raz, that helps transform any interactive public coin proof into a two message argument. And the transformation just asks the verifier. So the verifier has public coin queries in the interactive proof. The transformation just requires the verifier to send these queries as the queries of a private information retrieval scheme. And the prover sends his answers, his first message in the clear, his second message as a response to the private information retrieval query that is under the hood of the PIR scheme, and so forth. And Kali and Raz showed that assuming quasi-pollinomially secure private information retrieval, the second protocol is sound against adaptive cheating provers. So soundness of this protocol is already guaranteed. So if we were to perform this transformation, soundness is already guaranteed. Our strategy is going to be to apply this transformation to the three round proof system that I just told you about. So let's take the three round proof of blum on the left and squish it into two rounds using this transform. And what results is a two message protocol where the verifier first sends his challenge encrypted inside a PIR, and the prover then responds with his first message for the blum protocol and the third message for the blum again comes inside the hood of the PIR. Now this protocol, it's easy to see that this already remains honest verifier zero knowledge. That means if the verifier was honest but curious and the simulator knew the coins of the verifier, then it would be easy to simulate because he would just learn E and use this E along with the honest verifier ZK property of the underlying protocol. But what if a malicious verifier sends a malformed query that does not encode any valid bit at all? So any arbitrary verifier could just send a weird message that doesn't correspond to encryption of anything. What happens in that situation? It's not clear and this could ruin privacy potentially but we're going to prevent this by using a special kind of private information retrieval scheme or a special kind of oblivious transfer which I'm going to tell you about next. So oblivious transfer and since we only care about two message protocols right now, we're going to restrict ourselves to two message oblivious transfers. This is a protocol between a sender who has input two messages and a receiver who has input a choice bit. The receiver sends his first message of the protocol as some function or an encryption of his choice bit and the sender sends a response to this by sending an encryption of both his messages with respect to the first message sent by the receiver. At the end of this protocol, the receiver learns the B-th message of the sender and the standard privacy guarantees that we require from oblivious transfer are that a cheating sender should not be able to guess what the receiver's choice bit was and a cheating receiver should not be able to guess what the other message of the sender was, the one that he did not ask for was. We will add another, we will require another, a stronger privacy property here, which will be that every acceptable string that was sent by a cheating receiver should correspond to an encryption of some bit, meaning that for every string C sent by the receiver, it should correspond to an encryption of either B equals zero or B equals one and if it is an encryption of B equals zero, then the second sender message corresponding to his actual inputs M0, M1 should remain indistinguishable from a sender message that has both messages set to M0 and similarly if the choice bit is one. So, such OT is already known based on the decisional Diffie-Hellman assumption or quadratic reciduosity or entresiduosity. We will make use of this OT and we'll apply the Kalaira transform that I just told you about on the three-message lump protocol, we'll squish it down to two using this particular oblivious transfer. And Kalaira's already showed and I already told you because this OT is a special kind of private information retrieval, the second protocol remains sound against probabilistic polynomial time provers, even if they choose the instance adaptively. But what about privacy? We show that the argument that we get as a result of this transformation satisfies meaningful privacy guarantees when the PIR is instantiated with this particular oblivious transfer. In particular, it satisfies delayed input, distributional, distinguisher-dependent zero knowledge which is what I told you about initially but don't worry if you don't remember all of this. It also satisfies a bunch of other privacy notions which are not the focus of this talk. So, I'm going to tell you about them just a little bit about them later. So, let me show you the privacy analysis that gives us our main result. If we take our protocol, we know because of the security of oblivious transfer that every message sent by the verifier corresponds to an encryption of some challenge. This is what the oblivious transfer gave us. And therefore, in order to show that there exists a simulator that can start generating simulated proofs, if this simulator somehow knew what this challenge was, then we can invoke the honest verifier zero knowledge property of the underlying BLUM protocol and use that to simulate. However, the simulator does not know what the challenge is. Maybe it can break the encryption and figure it out. So, privacy of this protocol is indeed possible to prove via super polynomial simulation. This is explored in a concurrent work along with various other applications of this. However, in this talk, we really want polynomial simulation. We don't want the simulator to run in super polynomial time. So, now how should the simulator find out what the verifier challenge E is? Remember that one of the relaxations that we added was to allow the simulator to depend on the distinguisher. So, the simulator is allowed to observe what the output of the distinguisher is, and we're going to crucially use this to help the simulator find out the challenge E. Another simplification that we're going to make is that so far we were working with a parallel repetition of BLUM protocols, so as to amplify soundness. But let's simplify and look at a single parallel repetition right now and see if we can prove something about this. Note that unlike the case of BLUM, here it's unclear how to simulate even a single parallel execution because even if the simulator were to try to guess the challenge, it cannot really check it trivially. So, let's consider the setting of a single parallel execution and as a first attempt, let's have the simulator just send a second message that consists of A, which is the first message of the BLUM protocol, sampled honestly and a completely junk value inside the private information retrieval answer. And the question is, can the distinguisher tell the difference? Suppose the distinguisher cannot tell the difference. Now this could correspond to a situation where the verifier generated the encryption of E using some randomness, but did not convey this randomness to the distinguisher in the view. So the distinguisher has no idea what randomness was used for the OT encryption and maybe as a result, it cannot decrypt and recover the answer from within the PIR scheme. Also, the BLUM protocol already has this property that the first message A computationally hides the witness. So the simulator can easily sample some other A that looks similar to the honest A and send a junk message with it. And if the distinguisher really did not know the randomness that was used to generate the encryption of E, then maybe it's not going to be able to tell the difference and the simulation would be successful. But the more interesting case is when the distinguisher can tell the difference. For example, in this case, the distinguisher does know the randomness used to generate the encryption E and when the simulator tries playing this trick of sending a completely junk message, the distinguisher can tell and just says abort. So the simulator cannot sample this. This will be distinguishable. So what do we do? Well, in this case, we are going to use the fact that the distinguisher knows what E is to find out E from him. So the simulator will use the distinguisher to extract E in this situation and this is really the technical heart of our paper. Recall that we wanted a simulator that could simulate a proof for some x sampled from the distribution without access to a witness for this x. However, the simulator could sample other instances and witnesses from the same distribution by the distributional relaxation that we imposed. So our main idea is that the simulator is going to sample these other instances and witnesses from the same distribution. This is going to be allowed because of our distributional delayed input setting. And the simulator, since it has these other instances and witnesses, it will also be able to sample honest proofs for these other instances using their correct witnesses. So now the simulator that can sample another instance will keep the first message of the verifier fixed and is now going to sample a different instance and test what the distinguisher says on this instance. So let's say it samples some x prime and then A and Z0, Z1 correspond to an honest proof for x prime. The simulator tries to observe what the distinguisher does in the actual game. It also runs in parallel a game zero where within the OT, it guesses that maybe is an encryption of bit zero. And as a result, it sends its third message, so it's message for the OT to correspond to an encryption of Z0 twice. So the zero game basically is the simulator guessing that E was zero, sorry. And the one game is the game where the simulator guesses that the encrypted bit E was one. And therefore, the simulator sends under the OT hood and encryption of messages Z1 and Z1. And note that because of security of oblivious transfer, either the actual game is indistinguishable from game zero or the actual game is indistinguishable from game one based on what E is. They cannot both be distinguishable and therefore the simulator can check if the distinguisher's output when the simulator played with the actual game is close to the distinguisher's output when the simulator guessed that E was zero or if the actual game is close to the game where the simulator guessed that E was one. And one of these has to be close. Maybe they're both close, but in that case, that's like the distinguisher not knowing what the challenge is and that's okay anyway. So it's going to use this to extract E. And now let me summarize the simulation technique. So the simulator wants to simulate a proof for an externally sampled X without knowing its witness. In order to do this, it's going to sample a different X prime together with the correct proof and see what the distinguisher says. Next, it's going to sample a fake proof with both answers for the Sigma protocol corresponding to challenge equals zero. And it's going to see what the distinguisher says this time. Next, it's going to sample the second message corresponding to both answers for the underlying BLUM protocol when the verifier's challenge bit is one. And it's going to again see what the distinguisher said. And in this way, the simulator is going to rewind the distinguisher and use this to learn the OT challenge E. And this technique is going to be successful whenever the distinguisher D knows what the challenge E is because one of the two games will be distinguishable and the other will not. And if the distinguisher doesn't know what E is, then it suffices to just send a junk message because the distinguisher cannot tell the difference between both games anyway. And we note that unlike the case of standard, of the three round BLUM protocol that I just described, this technique extends to extracting the verifier challenge from a parallel repetition of these protocols. And therefore, if we repeat this in parallel to amplify soundness, the entire system still lets us extract the verifier challenge E bit by bit. So as a result, we get distributional distinguisher dependent zero knowledge. In the delayed input setting where for all distributions, for all verifiers and distinguishers, there exists a simulator that generates a simulated proof and is allowed to depend on the distinguisher. This implies indistinguishability or hiding based notions such as witness hiding in the delayed input setting where we require that for all efficiently sampled distributions, a verifier that interacts with a prover should not be able to output the witness at the end of the protocol. Another notion that this implies is that of strong witness indistinguishability, where we require that when two instances come from indistinguishable distributions, then the instance along with a proof should appear indistinguishable in case one versus two. In order to get this implication, we need an additional property of the zero knowledge that I don't have time to talk about right now. But to summarize, assuming Poise polynomial DDH or quadratic residuality or nth residuality, we obtain two round arguments in the delayed input setting that satisfy distributional zero knowledge where the simulator depends on the distinguisher. They also satisfy witness hiding, strong witness indistinguishability, and they also satisfy the standard notion of witness indistinguishability without any of the delayed input restrictions. Previously, witness indistinguishability was known based on different assumptions, and so we get it based on other assumptions than were previously known. We also get three round versions of our arguments relying on polynomial hardness of the same assumptions. And we give multiple applications of these proof systems and argument systems, and to get some perspective, like this introduces a new extraction technique for cryptography. It's a black box polynomial simulation strategy that only requires two messages, whereas previously revinding required more rounds. Here, we are able to rewind the distinguisher and use that to cut short one message. This helps us go towards resolving open problems on the round complexity of witness hiding and strong WI and has applications to multiple two round and three round protocols beyond argument systems. So to conclude, in the round complexity timeline, here in two rounds, we obtain additional privacy guarantees such as distributional week zero knowledge, witness hiding, strong WI, et cetera. We also obtain two round witness indistinguishability from different assumptions than before. And in the paper, we demonstrate applications to a variety of round optimal protocols, which I don't have time to go into right now. So to conclude, sorry, let me just conclude with some open questions. First, can we get two round versions of these protocols based on polynomial hardness? Second, can we make our protocols a strongly private coin? Can we make them public coin? Can we get other interesting applications of distinguisher based simulation? And can we get other black box or non-black box techniques for two round protocols? Thank you.


Thank you. So my name is Simon and I'm here to present a Python toolbox that we have recently developed at the University of Oslo for performing and certain modification and sensitivity analysis on computational neuroscience models. So to give a brief overview of what I will be talking about this presentation, first I will try to motivate why we need to perform and certain quantifications in the first place. Then I will talk a bit about what we get from an uncertainty quantification as well as what a sensitivity analysis is. And then I will end with a specific problem that crops up when we perform uncertainty quantification in neuroscience and how uncertainty is tailored towards neuroscience and solves this problem. So in this presentation I will use the Hodgkin-Axley model as an example model and here you see a typical evaluation of this model. When it receives a step current you get some action potentials. And here you have the equation that describes this model as with all computational models. You have a various set of parameters that describes the system that you want to model. Here you see a few of them marked with color. So I will focus on three of the parameters of the Hodgkin-Axley model and namely the ion channel conductances for the potassium, sodium and leak ion channels of the model. The Hodgkin-Axley model has other parameters as well that can be set, but I will not focus on those. So normally what you do, you have created your model or found the equations that describe the system and then you perform some measurements that either directly measures the parameters you have or you have to fit your model to the measurements. So you change the parameters out and try different sets of parameters until your model reproduces your measurements. The problem is that the parameters that you have are generally not fixed. They do not have one exact value. And there are many reasons for this, but to mention a few one is that we have measurement and certainty. So whenever we perform a measurement or an experiment, we always have some kind of uncertainty associated with our measurements. And we generally do not take this into account when we use or create our model. Another reason which is really important in biology is that we have some type of biological variability. For example, the parameters can change with time. They can be regulated by the system itself. So if we measured a specific parameter, now we would get a slightly different result than if we measured it say two hours ago. Another cause of biological variability is that we have in the case of ion channels, many ion channels of the same type within a cell and each of these ion channels can have slightly different ion channel conductances. In a similar way, when we have a network, we can have different parameters for each neuron of the same type in the network, but this is generally not taken into account. So you can of course also have all of these effects at the same time and all the various causes of uncertain parameters. So all of this leads us to the fact that often the parameters of our models are best described by using a distribution for the parameter instead of a fixed value. And this is where an uncertainty quantification enters the stage. When we perform an uncertainty quantification of our model, we are able to take the effect of uncertain parameters, the parameters that have distributions into account. Traditionally, what is done is that if you perform the measurement, we choose the mean and then we use that in our model and get a single deterministic result out. When we on the other hand perform an uncertainty quantification, we are able to take the entire parameter distribution into account and instead of getting a single deterministic result out, we get a range of possible values for our model. It is therefore much easier to see or find out how much can we test our model result. So this is where uncertainty comes in. As I mentioned, this is a Python toolbox for performing these uncertainty quantifications. It uses the very efficient polynomial chaos expansions or the more traditional, slower, but more robust Quasi Monte Carlo methods. I'm not going to go into any details on how this is done. Instead, I will just skip ahead to see what type of results can we get when we perform an uncertainty quantification. Here I have performed an uncertainty quantification of the Hodgkin-Huxley model that receives a step current. One of the statistical metrics that the uncertainty quantification gives us is the mean and the standard deviation or the variance of our model. Here you see the mean in gray and the standard deviation in red. So what can we observe from this? Well, one thing is that during the rise and fall of the action potential in the Hodgkin-Huxley model, the standard deviation is quite low, but it is greater at the peak of each action potential. Another type of metric that we get out is the 90% prediction interval, which means that 90% of the model evaluations occur somewhere in this grayed-out area. And we're able to observe the same thing as from the mean and the standard deviation. During the peak, we have the most uncertainty. So UncertainPy also performs a sensitivity analysis in addition to the uncertainty quantification. And a sensitivity analysis tries to quantify how much of the uncertainty or the variance of our model is caused by the uncertainty in each parameter. So basically what we are trying to do is to assign blame. The sensitivity generally sums to one, which means that we can say that for each point in time, the third parameter here is responsible for 60% of the model variance. To show you a result for the Hodgkin-Huxley model, here you see the sensitivity of the potassium conductance and if we add the confidence interval just to easier see what when things happen, we can see that potassium conductance causes the most variance during the rise and the fall of our model, while during the peak of our model, there are the action potential, there are almost no variance that is caused by the potassium conductance. If we add the sensitivity for the sodium conductance, we see that as we perhaps would expect. Sodium conductance is responsible for variance during the peak of the action potential. So by performing sensitivity analysis, we can gain additional insight into our model and it can help us determine which biological mechanism are affecting our model at each point in time. So it helps us understand or potentially can help us understand what's going on. For completeness, let me add the leak ion channel or the sensitivity to the leak ion channel conductance and as you can see that is not very interesting. It's at the bottom there in purple and it's the leak ion channel is not responsible for any variance at all. This result here is quite typical. Most often there's only one or two parameters that are responsible for most of the variance of our model, while the rest is responsible for almost nothing or absolutely nothing. Just note that I said the sensitivity to normal is 7 to 1. This Hodgkin-Huxley model here had 11 and certain parameters and I only showed you three of them. So these three do not sum to one. But the fact that the model most often is sensitive to a couple of parameters is really useful both in guiding the experimental focus as well as guiding the computational focus if you want to develop our model further. For example, if you want to reduce the uncertainty of our model, we can go to the lab and we know that we need to focus on measuring the potassium and sodium conductances very accurately while we can ignore the other parameters. Similarly, if you want to reduce the complexity of our model or if you want to or need to fix some parameters, we know that leak channel conductance, it's not very important at all. We can set that at fixed value and it will not change the variance of our model much or in some cases it might even indicate that we can remove mechanisms from the model. So, since things never are as straightforward as we would hope, one problem that occurs when we perform a certain quantification in neuroscience is that we get a huge variance due to small shifts in the spike timing. Here you see another neuron model that exemplifies this problem with three evaluations with slightly different parameters and when we want to find say the variance of our model, we compare what is the membrane potential to each point in time and as you can see the difference here is really large since this, which for biological intents and purposes is the same spike, occurs at different times. So by performing an uncertainty quantification of this model, we can see that 90% prediction interval gets really big and it's hard to draw a conclusion on what happens in this at the point of the second spike. The solution for this is quite simple. We can do as when we perform a parameter estimation. We can look at features of our model in addition to the model result itself. For example, if we count the number of spikes, this does not change due to small shifts in spike timing and such features are often much more robust. So, this is what we do in UncertainPy. We have a set of features that the user can choose from and then calculate both the uncertainty and sensitivity of the model and or the model result itself and all the selected features of the model. Here you see just a small set of the features that are already implemented. So, this is one way that UncertainPy is tailored toward neuroscience. Another is that it has built-in support for a couple of neuroscience models such as multi-compartmental models in the Neuron simulator and network models in the Nest simulator and implementing custom models is quite easy and implementing custom features are also quite easy and a set of features for both network models and multi-compartmental models are already available. So, to quickly summarize what I've been talking about, most parameters or many parameters in neuroscience are best described by distributions instead of fixed values and performing uncertainty quantification enables us to take this into account. So, UncertainPy is toolbox that perform all the calculations for you. So, you do not need to know how to perform an uncertainty quantification and it is tailored toward neuroscience for example through the calculation of specific features of your model. So, if you want to check it out, it is open source and I find it on GitHub. It's quite easy to install through PIP and it has an extensive documentation with several examples. So, it should hopefully be easy to use. Additionally, we have just gotten an article accepted in Frontiers in Neuron informatics which goes into details on how the uncertainty quantification is done. So, if you want to know more, come talk to me at my poster tomorrow. And thank you for your attention. I'm open for questions. It's nice to see the quantification of the different parameters and the impact that they had, but don't you need to know when you're, so when you introduce it, you kind of lump together heterogeneity in parameters and experimental uncertainty. Don't you really need to separate those out before you then apply this method? Well, you both need to and you don't need to. It depends on if you want to know the uncertainty just due to measurement uncertainty or if you also want or you only want to know the uncertainty due to, say, biological variability. But you are able to do this in uncertainty. You only, what uncertain by takes is the model, of course, and the parameters that are uncertain and then you can just define or give it the first parameters that are uncertain due to biological variability and next you can do the parameters that are uncertain due to measurement uncertainty. So we're able to split it up if you want to. Okay, but so in a related question really, does it, can you deal with correlations between uncertainties? Yes, you can define multivariate dependent probability distributions for your parameters. Okay, that's great. Very nice. I thought it was really interesting. So I was reminded a bit of a recent talk by Eve Marder and it's kind of a theme that she goes to sometimes. Don't model the mean. I don't know if you're, you know, familiar with some of her work where she's looking at large changes in parameters, you know, big parameter sets and showing that you can get different kinds of behaviors at the, you know, at different, I'm sorry, the same behavior at different points in parameter space. So I just want to make sure I understand what uncertain by does. You're starting from like an already optimized model. You're thinking about a set of parameters that give you the behavior that you want and then that's what you consider to be the mean and then you're kind of going from there with the uncertainty analysis. Is that right? Yes, what uncertain by requires is that from beforehand have the uncertainty of your parameters either through measuring it or to a parameter fitting but that you have the uncertainty of each parameter and then it considers the model as a black box. This is really, I think it's really critical what you're doing this, you know, understanding the things. I think it's fantastic. One question as a biologist, I like to understand. So many of the parameters in real neuronal networks are log normally distributed, right? And how do you capture that kind of variability where you have some, you know, very rare, let's say, synaptic sizes of firing rates and things like that, you know, that would not be really captured by let's say standard deviation. So that is I use the standard deviation, yes, as an example of a distribution, but you have an infinite possibility of different distributions to give to your parameters. The package that I use to calculate or perform many of these calculations has support for 64 different distributions built in and also support for creating your own and easily create your own multivariate distributions. So you should be able to give them or use whatever type of distributions that you want. When your simulations themselves have stochastic elements, the output inherently has a fairly wide or can have a very wide distribution. So have you got a way of handling situations like that? What do you mean the range that the range that you get out is really large? That means that every time you run the simulation, you're going to get a different result. Yes, and this might follow a very, very non Gaussian, non normal distribution. That should not be a problem. As long as you get an output back that and certain pi is able to handle currently, if you have a say multi-dimensional output, it has not support for that. But if you get back C1d to D dimensional output, it does not matter how much each model evaluation changes from one run to the next. If that answered your question. I mean does your system then, I mean do you have to explicitly tell it then that each run of this is going to do something different and then you have to find some way of analyzing the statistics of the of the model itself. Is that what you're saying? So all that you need to define or make sure for the mold run is that you give it some way of ability to set your set the model parameters that are uncertain. So as long as those can be set as specific numbers, that is chosen by and certain by when it evaluates a model, it does not need to know or it does not care what else your model actually does. So then it can has as many stochastic elements as it wants as long as you return a result. That should not change the output much. But wouldn't, okay, maybe we can talk about it offline, but you might get I think I know what you mean. But since you have some random element of your model that can affect the output even though we have the same parameters that goes in. Yes, that's correct. That can be or that will be a problem. So then it's best to be able to set the seed to get precise results even though that really influences your model. Well, it's not I mean, okay, we can perhaps discuss it offline because there is no precise result for a model of this kind. You can only get a distribution. Yeah, and that will be any given instantiation of the run will give you something different and that's it. These all equally legitimate outcomes of the model. Then that will probably be a problem. I haven't tested it, but I think that will be problematic, yes. Okay, if there are any more questions, then thank you very much Simon.